Hello,

I observed a little odd behavior of the NFS client and server
connectivity between AIX boxes (5.3 TL4 client and AIX 5.2 ML9 as a server).

There is a firewall between them so I opened these ports:
port 111 UDP (sun rpc)
port 2049 TCP (nfs)
port 32700-32900 TCP(ephemeral ports)
http://www.ncftp.com/ncftpd/doc/misc...ports.html#AIX
# /usr/sbin/no -a | fgrep ephemeral
tcp_ephemeral_high = 65535
tcp_ephemeral_low = 32768
udp_ephemeral_high = 65535
udp_ephemeral_low = 32768


Now when I check from the client resources on the server:
showmount -e server
I get the filesystems and able to mount them. This is a part of the
tcpdump on the server during showmount -e from the client:

07:34:38.798820823 client.798 > server.32783: tcp 0 (DF)
07:34:38.798926395 server.32783 > client.798: tcp 0 (DF)
07:34:38.800673029 client.798 > server.32783: tcp 0 (DF)
07:34:38.800745118 client.798 > server.32783: tcp 44 (DF)
07:34:38.807053003 server.32783 > client.798: tcp 488 (DF)
07:34:38.810154446 client.798 > server.32783: tcp 0 (DF)
07:34:38.810213400 server.32783 > client.798: tcp 0 (DF)
07:34:38.810392712 server.32783 > client.798: tcp 0 (DF)

32783 is pretty logical, it was served by portmapper:
server # rpcinfo -p |grep mount
100005 1 tcp 32783 mountd
100005 2 tcp 32783 mountd
100005 3 tcp 32783 mountd
100005 1 udp 32812 mountd
100005 2 udp 32812 mountd
100005 3 udp 32812 mountd


I mount the filesystems (NFS3, bg,soft,intr). So far so good.

Few hours later, sometimes just few minutes later when I am trying to
again check the resources on the server I do a showmount -e server and
the command hangs for a minute and finally I get RPC timeout:

# showmount -e server
server: RPC: 1832-018 Port mapper failure - RPC: 1832-008 Timed out


this is the tcpdump of this communication. Started on the server.

07:42:21.333809913 client.836 > server.sunrpc: udp 56
07:42:26.331438846 client.836 > server.sunrpc: udp 56
07:42:36.331665184 client.836 > server.sunrpc: udp 56
07:42:56.330956968 client.836 > server.sunrpc: udp 56


Still the mounts created earlier are accessible.
I do a ls on one of the mountpoins.

tcpdump on the server says:
07:59:42.120892807 client.32785 > server.shilp: tcp 0 (DF)
07:59:42.126034715 client.32785 > server.shilp: tcp 0 (DF)
07:59:42.126534273 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.127702191 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.128826386 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.129876331 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.130878705 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.131904956 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.132952712 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.133935908 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.134966085 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.135989063 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.137110763 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.139064942 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.140050363 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.141256566 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.142258576 client.32785 > server.shilp: tcp 136 (DF)
07:59:42.158518620 client.32785 > server.shilp: tcp 0 (DF)
07:59:45.356356575 client.32785 > server.shilp: tcp 140 (DF)
07:59:45.358219786 client.32785 > server.shilp: tcp 0 (DF)
07:59:46.342325451 client.32785 > server.shilp: tcp 140 (DF)
07:59:46.343288988 client.32785 > server.shilp: tcp 160 (DF)
07:59:46.358420845 client.32785 > server.shilp: tcp 0 (DF)

# grep shilp /etc/services
shilp 2049/tcp
shilp 2049/udp


showmount still hangs. I repeat the command and still no luck:
08:00:32.945658156 client.802 > server.sunrpc: udp 56
08:00:37.943803847 client.802 > server.sunrpc: udp 56

What is interesting now local port is changed to 802, previously it was
836.

Again, after some time showmount -e server starts working.

One more tcpdump. When I am unmounting the filesystem (success):
08:11:59.549821319 client.659 > server.sunrpc: udp 56
08:11:59.552718693 server.sunrpc > client.659: udp 28
08:11:59.612331087 client.659 > server.32812: udp 124
08:11:59.675376805 server.32812 > client.659: udp 24




and mounting it again (success):
08:12:52.656996398 client.659 > server.sunrpc: udp 56
08:12:52.657528865 server.sunrpc > client.659: udp 28
08:12:52.662170488 client.659 > server.32783: tcp 0 (DF)
08:12:52.662279072 server.32783 > client.659: tcp 0 (DF)
08:12:52.665904430 client.659 > server.32783: tcp 0 (DF)
08:12:52.665963042 client.659 > server.32783: tcp 44 (DF)
08:12:52.692099534 server.32783 > client.659: tcp 28 (DF)
08:12:52.692938414 client.659 > server.sunrpc: udp 56
08:12:52.693185757 server.sunrpc > client.659: udp 28
08:12:52.696760037 client.702 > server.shilp: tcp 0 (DF)
08:12:52.696826773 server.shilp > client.702: tcp 0 (DF)
08:12:52.698429550 client.702 > server.shilp: tcp 0 (DF)
08:12:52.698489020 client.702 > server.shilp: tcp 44 (DF)
08:12:52.698608790 server.shilp > client.702: tcp 28 (DF)
08:12:52.699680197 client.659 > server.32783: tcp 128 (DF)
08:12:52.699794730 client.702 > server.shilp: tcp 0 (DF)
08:12:52.699845968 server.shilp > client.702: tcp 0 (DF)
08:12:52.699867684 server.shilp > client.702: tcp 0 (DF)
08:12:52.701097633 client.702 > server.shilp: tcp 0 (DF)
08:12:52.839179105 server.32783 > client.659: tcp 0 (DF)
08:12:52.855485048 server.32783 > client.659: tcp 76 (DF)
08:12:52.856605351 client.659 > server.32783: tcp 0 (DF)
08:12:52.856649832 server.32783 > client.659: tcp 0 (DF)
08:12:52.856800205 client.32785 > server.shilp: tcp 136 (DF)
08:12:52.856964885 server.shilp > client.32785: tcp 116
08:12:52.858082591 client.32785 > server.shilp: tcp 136 (DF)
08:12:52.858169327 server.shilp > client.32785: tcp 116 (DF)
08:12:52.859090871 client.32785 > server.shilp: tcp 136 (DF)
08:12:52.859228051 server.shilp > client.32785: tcp 116 (DF)
08:12:52.860135333 client.32785 > server.shilp: tcp 140 (DF)
08:12:52.860223923 server.shilp > client.32785: tcp 124 (DF)
08:12:52.861099931 client.32785 > server.shilp: tcp 136 (DF)
08:12:52.861180653 server.shilp > client.32785: tcp 168 (DF)
08:12:52.866172958 server.32783 > client.659: tcp 0 (DF)
08:12:52.867225813 client.659 > server.32783: tcp 0 (DF)
08:12:52.935294368 client.32785 > server.shilp: tcp 0 (DF)


Any thoughts of this random behavior?

Regards,

rafal