Hello all,

I am running a OpenBSD 3.3 firewall.

The box freezes after several hours (between 1 to 30 hours) with no
warning. No debug prompt or error messages appear on the console or
syslogs... it just stops and no traffic passes. I have disabled all
apm/acpi, unused com ports and usb in both the kernel and bios. The box
uses sis nsphyter 83816 interfaces (which show up as "NS DP83815 10/100"
in dmesg). *I have tried the generic kernel and it also freezes.*

Oddity #1: is that my ISP has put two overlapping subnets on our DSL
connection. Some of our machines are on one subnet, and some on
another... all coming through the same DSL. Our ISP might be nuts. So
we get messages like this:
Nov 29 09:44:16 192.168.3.1 /bsd: arplookup: unable to enter address for
XXX.XXX.XXX.XXX

And the XXX.XXX.XXX.XXX is an address from the other subnet (but one of
our boxes) that is NOT assigned to the firewall. But has a situation
like this been known to freeze up a machine? I doubt it.

Oddity #2: As I stated, the generic kernel freezes too. Regardless, I
noticed the following with my custom kernel and I thought you should see it:
# netstat -m
514 mbufs in use:
512 mbufs allocated to data
1 mbuf allocated to packet headers
1 mbuf allocated to socket names and addresses
384/204/8192 mbuf cluster pages in use (current/peak/max)
1044 Kbytes allocated to network (85% in use)
0 requests for memory denied
0 requests for memory delayed
0 calls to protocol drain routines

The values in question are current/peak/max for mbuf cluster pages. This
doesn't make sense to me... why would the "peak" be less than the
"current" value? Could that indicate a potential problem? On a
different box the current is less than (or equal to) peak, as I would
expect.

I've been checking the mbuf values from time to time, and they never get
beyond 1000, even when the firewall hangs. I doubt I'm running out of
mbufs, or that they are leaking.

Regarding the netstat -m Did I make a mistake in the kernel config?
Here it is (it was altered from a soekris kernel):

machine i386 # architecture, used by config; REQUIRED

#option NTP # hooks supporting the Network Time Protocol

option DDB # in-kernel debugger
#option DDB_SAFE_CONSOLE # allow break into ddb during boot
#makeoptions DEBUG="-g" # compile full symbol table
#makeoptions PROF="-pg" # build profiled kernel
#option GPROF # kernel profiling, kgmon(8)
option DIAGNOSTIC # internal consistency checks
option KTRACE # system call tracing, a la ktrace(1)
#option KMEMSTATS # collect malloc(9) statistics

option CRYPTO # Cryptographic framework

option FFS # UFS
option MFS # Memory FS
#option FFS_SOFTUPDATES # Soft updates

option TCP_SACK # Selective Acknowledgements for TCP
#option TCP_FACK # Forward Acknowledgements for TCP
#option TCP_SIGNATURE # TCP MD5 Signatures, for BGP routing
sessions

option FDESC # /dev/fd
option FIFO # FIFOs; RECOMMENDED
option KERNFS # /kern
#option NULLFS # loopback file system
option PROCFS # /proc
#option UMAPFS # NULLFS + uid and gid remapping

option IPFORWARDING
option NMBCLUSTERS=16384
#option GATEWAY # packet forwarding
option INET # IP + ICMP + TCP + UDP
option ALTQ # ALTQ base
#option ALTQ_NOPCC # We don't have Pentium features on 486....
#option INET6 # IPv6 (needs INET)
#option PULLDOWN_TEST # use m_pulldown for IPv6 packet parsing
option IPSEC # IPsec
#option PPP_BSDCOMP # PPP BSD compression
#option PPP_DEFLATE

option BOOT_CONFIG # add support for boot -c

#option I486_CPU
#option I586_CPU
option I686_CPU

option USER_PCICONF # user-space PCI configuration

#option KGDB # Remote debugger support; exclusive of DDB
#option "KGDB_DEVNAME=\"pccom\"",KGDBADDR=0x2f8,KGDBRATE=9 600

#option DUMMY_NOPS # speed hack; recommended

maxusers 32 # estimated number of users

config bsd root on wd0a

mainbus0 at root

bios0 at mainbus0
pcibios0 at bios0 flags 0x0000 # use 0x30 for a total verbose

isa0 at mainbus0
isa0 at pcib?
pci* at mainbus0 bus ?

# PIT clock runs at 1.189161 MHz instead of the standard 1.19318 MHz
# on the AMD Elan SC520
#
#option TIMER_FREQ=1189161

option PCCOMCONSOLE
option CONSPEED=9600

option PCIVERBOSE

pchb* at pci? dev ? function ? # PCI-Host bridges
elansc* at pci? dev ? function ? # AMD Elan SC520 System Controller

ppb* at pci? dev ? function ? # PCI-PCI bridges
pci* at ppb? bus ?
pci* at pchb? bus ?
pcib* at pci? dev ? function ? # PCI-ISA bridge

pciide* at pci ? dev ? function ? flags 0x0000
wd* at pciide? channel ? drive ? flags 0x0000

npx0 at isa? port 0xf0 irq 13 # math coprocessor
isadma0 at isa?

#pcppi0 at isa?
#sysbeep0 at pcppi?

pccom0 at isa? port 0x3f8 irq 4 # standard PC serial ports
pccom1 at isa? port 0x2f8 irq 3

wdc0 at isa? port 0x1f0 irq 14 flags 0x00

wd* at wdc? channel ? drive ? flags 0x0000

#fxp* at pci? dev ? function ? # EtherExpress 10/100B
ethernet
#vr* at pci? dev ? function ? # VIA Rhine ethernet

lpt0 at isa? port 0x378 irq 7 # standard PC parallel ports

sis* at pci? dev ? function ? # SiS 900/7016 ethernet

nsphyter* at mii? phy ? # NS and compatible PHYs

# crypto support
hifn* at pci? dev ? function ? # Hi/fn 7751 crypto card

# Wireless network cards
#wi* at pci? dev ? function ? # MiniPCI wireless card

#pseudo-device pctr 1

pseudo-device crypto 1

pseudo-device pf 1 # packet filter
pseudo-device pflog 1 # pf log if
pseudo-device loop 2 # network loopback
pseudo-device bpfilter 16 # packet filter
#pseudo-device sl 2 # CSLIP
#pseudo-device ppp 2 # PPP
pseudo-device tun 2 # network tunneling over tty
pseudo-device enc 1 # IPSEC encapsulating interface
pseudo-device bridge 2 # network bridging support
pseudo-device vlan 32 # IEEE 802.1Q VLAN
pseudo-device gre 4 # GRE encapsulation interface
#pseudo-device strip 1 # Starmode Radio IP interface

pseudo-device pty 32 # pseudo-terminals

# for IPv6
pseudo-device gif 4 # IPv[46] over IPv[46] tunnel (RFC1933)
#pseudo-device faith 1 # IPv[46] tcp relay translation i/f

The dmesg follows:

OpenBSD 3.3-stable (CFKERNEL_V1) #1: Mon Nov 24 00:36:58 MST 2003
root@blah.localdomain:/usr/src/sys/arch/i386/compile/CFKERNEL_V1
cpu0: VIA C3 Ezra ("CentaurHauls" 686-class) 665 MHz
cpu0: FPU,DE,TSC,MSR,MTRR,PGE,MMX,3DNOW
real mem = 125415424 (122476K)
avail mem = 114380800 (111700K)
using 1556 buffers containing 6373376 bytes (6224K) of memory
mainbus0 (root)
bios0 at mainbus0: AT/286+(00) BIOS, date 04/23/03, BIOS32 rev. 0 @ 0xfdb30
pcibios0 at bios0: rev. 2.1 @ 0xf0000/0x10000
pcibios0: PCI IRQ Routing Table rev. 1.0 @ 0xf8920/192 (10 entries)
pcibios0: PCI Interrupt Router at 000:07:0 ("VIA VT82C686 PCI-ISA" rev 0x00)
pcibios0: PCI bus #1 is the last bus
bios0: ROM list: 0xc0000/0xc000 0xcc000/0x1000 0xcd000/0x1000
0xce000/0x1000 0xcf000/0x1000
pci0 at mainbus0 bus 0: configuration mode 1 (no bios)
pchb0 at pci0 dev 0 function 0 "VIA VT8601 Host-PCI" rev 0x05
ppb0 at pci0 dev 1 function 0 "VIA VT82C601 PCI-AGP" rev 0x00
pci1 at ppb0 bus 1
"Trident CyberBlade i1" rev 0x6a at pci1 dev 0 function 0 not configured
pcib0 at pci0 dev 7 function 0 "VIA VT82C686 PCI-ISA" rev 0x40
pciide0 at pci0 dev 7 function 1 "VIA VT82C571 IDE" rev 0x06: ATA100,
channel 0 configured to compatibility, channel 1 configured to compatibility
pciide0: channel 0 ignored (disabled)
wd0 at pciide0 channel 1 drive 0: <SanDisk SDCFB-32>
wd0: 1-sector PIO, LBA, 30MB, 489 cyl, 4 head, 32 sec, 62592 sectors
wd0(pciide0:1:0): using PIO mode 0
"VIA VT82C686 SMBus" rev 0x40 at pci0 dev 7 function 4 not configured
sis0 at pci0 dev 8 function 0 "NS DP83815 10/100" rev 0x00: irq 11
address 00:02:b6:15:7b:f9
nsphyter0 at sis0 phy 0: DP83815 10/100 integrated, rev. 1
sis1 at pci0 dev 9 function 0 "NS DP83815 10/100" rev 0x00: irq 10
address 00:02:b6:15:7b:fa
nsphyter1 at sis1 phy 0: DP83815 10/100 integrated, rev. 1
sis2 at pci0 dev 10 function 0 "NS DP83815 10/100" rev 0x00: irq 12
address 00:02:b6:15:7b:fb
nsphyter2 at sis2 phy 0: DP83815 10/100 integrated, rev. 1
sis3 at pci0 dev 11 function 0 "NS DP83815 10/100" rev 0x00: irq 9
address 00:02:b6:15:7b:fc
nsphyter3 at sis3 phy 0: DP83815 10/100 integrated, rev. 1
isa0 at pcib0
isadma0 at isa0
npx0 at isa0 port 0xf0/16: using exception 16
pccom0 at isa0 port 0x3f8/8 irq 4: ns16550a, 16 byte fifo
pccom0: console
biomask 8000 netmask 9e00 ttymask 9e00
dkcsum: wd0 matched BIOS disk 80
root on wd0a
rootdev=0x0 rrootdev=0x300 rawdev=0x302

The system runs off a flash card with a root partition that's mounted
read-only, and mfs for tmp. But this setup has worked on other hardware
perfectly, and in fact, the same system. But those were NOT on this
weird subnet. Could our subnet truly be the issue?

We've been trying to hammer out these freezing issues for over a week
with no success. I appreciate the help.

Thank you,
Jim