I'm sure it's not due to high load. The load average hovers around 0.08, I
have over 90 megs of free physical memory and another 256 of virtual that's
not used at all. Network traffic is very nearly nonexistant, as my firewall
only forwards port 22 to the OBSD box, and the only traffic there is me and
the odd person looking for vulnerable SSH servers a few times a day.

I set up a ping in the background to ping the firewall machine every 60
seconds, and I haven't had the problem since. This is clearly a workaround,
but to me it makes the APM issue seem more likely.

Anyway... That's all the stuff I can think of for now.