SEO

Hi,

I have a problem with configuring pf and would very much appreciate
some help.

Some background is needed: I have to migrate some web applications from
one machine to another, and I decided to install an OpenBSD box so I
can give it the name of the old web apps machine over night (in DNS),
and then redirect ports gradually from the old web apps machine towards
the new machine as I migrate the individual web apps living on
individual ports over the course of some days without any impact on the
users.

I won't need any firewall functionality because the entire excercise
happens inside a firewalled environment, and I don't think I'd need any
NAT since I have no one-to-many IP relations involved. It should be a
very straight forward setup with a few lines of pf.conf for all I can
see, but it doesn't work.

The old and new web app machine are say 10.4.11.26 and 10.4.10.235
respectively. Those addresses are usually a 255.255.0.0 network (so,
both in the same network). My OpenBSD box is 10.4.12.142, which usually
also has a 16 bit netmask (meaning, all three would be in the same
network).

I must admit I am not an expert when it comes to routing, so I have put
three network cards into the OpenBSD box and hooked all of them up to
the LAN, and I have configured them as 10.4.11.28/24, 10.4.10.237/24
and 10.4.12.142/24. So I tell the box to treat each part of what
usually is a 16 bit network as a class C (24 bit) network.

I don't know if this is a smart thing to do, but I figured the netmask
only tells the box which network portion is attached to what interface?
Essentially it seems that I achieve exactly what I want, I can
communicate (say, ping) my old web app machine (10.4.11.26, through
network card #1) from the OpenBSD box, and my new web app machine
(10.4.10.235, through card #2) as well, and it can reach all other
clients which are all in the 10.4.12.xxx range.

Vice versa, both web app boxes can ping my OpenBSD box, and my openbsd
box can be reached by clients, say 10.4.12.1 for example.

For short, every machine can talk to the OpenBSD box, and be talked to
from the OpenBSD box, and OpenBSD knows how to route.

Then I made a very small pf.conf:

--- snip ---

ext_if="xl0" # interface for inbound traffic
old_if="xl1" # interface for old web apps machine
new_if="xl2" # interface for new web apps machine
old_ip="10.4.11.26" # old web apps IP for rdr rules target
new_ip="10.4.10.235" # new web apps IP for rdr rules target

set block-policy return

rdr on $ext_if proto tcp to port 80 -> $old_ip port 80

pass in keep state
pass out keep state

--- snip ---

This should, me thought, allow a client (say 10.4.12.1) to connect to
the OpenBSD box itself (10.4.12.142) on port 80 and allow me to
redirect this to the web server on the old web apps machine, port 80
(and later, with each software migrated, to redirect to the new app
servers IP instead).

But it doesn't work - I can't see the web page from a client at
10.4.12.1. In tcpdump -i pflog0 -l when adding "log" to the pass out
and pass in rules, I see that my workstation tries to connect to the
old web server (successfully redirected it seems), like so:

17:11:53.730275 myclient.mydomain.com.1059 >
oldwebsrv.mydomain.com.www: [|tcp] (DF)

but no reply ever seems to come from the old web server, through the
OpenBSD box and back to my client.

I believe that the webserver tries to reply directly to the client at
that point and the packets get dropped by the client because they're
out of sequence or so. But I have no idea how to make this work.

Could anyone with more routing skills kindly offer some advice for the
above scenario and how to best build a machine that allows me to
migrate from one web apps machine to another by bending ports one after
another?

Many thanks,

Carsten
The Netherlands

<snip>

> but no reply ever seems to come from the old web server, through the
> OpenBSD box and back to my client.
>
> I believe that the webserver tries to reply directly to the client at
> that point and the packets get dropped by the client because they're
> out of sequence or so. But I have no idea how to make this work.

it sounds like you need to set a route from your webserver back to the obsd
box.
stefan.

Hi Stefan, and others,

First, many thanks for your reply.

But sorry, I don't quite understand this (being not an expert with
routing).

I believed that I set a route to allow one machine to reach another.
The point at hand is, my webserver can already reach the OpenBSD box.

Unlike the OpenBSD box, the webserver operates with a 16 bit netmask
for the 10.4. network that itself is in, so it can reach (ping etc) the
OpenBSD box at 10.4.12.142 with no problems. So, if the packets coming
port-redirected from the OpenBSD box would look right, the web server
should already send a reply back to the OpenBSD box as far as I can
see?

Maybe you or someone on the list could kindly explain this proposal and
maybe even help me out with a sample route command for this scenario -
it would be greatly appreciated as I have to alter a production
environment and wouldn't want to bring that down by "trying things
out".

Thanks
Carsten

Some more info: It appears that the port-redirected packages from the
OpenBSD box never reach the webserver in the first place, although the
OpenBSD box can generally reach the web server! Puzzles me.

Client IP: 10.4.10.234
OpenBSD receiving if (to client): 10.4.10.142
OpenBSD if for redirection to webserver: 10.4.12.237 (xl2)
Webserver if: 10.4.12.235

Connecting from the client to port 80 of the openbsd box gets
redirected to port 80 on webserver fine:

tcpdump: WARNING: pflog0: no IPv4 address assigned
tcpdump: listening on pflog0, link-type PFLOG
10:45:30.690277 10.4.12.234.4399 > 10.4.10.235.80: [|tcp] (DF)

and on the interface itself (different tcpdump run, thats why different
high port)

tcpdump: listening on xl2, link-type EN10MB
10:38:16.914070 10.4.12.234.4388 > 10.4.10.235.80: S
2959675908:2959675908(0) wi

but on the web server the command

tcpdump -i eth0 -l -n | grep 10.4.10.23 | grep -v ssh

which should show an incoming connection from 10.4.10.234 that isn't
ssh, I see nothing incoming at the same time - and that despite being
able to reach

10.4.12.235:80

from the OpenBSD box with lynx or telnet 80.

I am totally puzzled by this.

I am wondering now: I could simplify my setup even more and get rid of
the different subnet masks by using just one network card on the
OpenBSD box, maybe that solves the problem? But is it possible to use
just one network card for port redirect? Like saying in pf.conf
"everything coming in on xl0 on port 80, redirect this to another
machine in the same subnets port 80 via the same interface"?

In that case, I could have one card only as 10.4.xxx.xxx/16 that can
reach all machines at once.

Or do I need two network cards and/or two different target subnets for
port redirection to work?

Many thanks

Sorry, have an error in my second-to last port. The correct IP
addresses are:

Client IP: 10.4.12.234 OpenBSD receiving if (to client): 10.4.12.142
OpenBSD if for redirection to webserver: 10.4.10.237 (xl2) Webserver
if: 10.4.10.235

Apologies.

google@humanprototype.com wrote:
> Hi Stefan, and others,
>
> First, many thanks for your reply.
>
> But sorry, I don't quite understand this (being not an expert with
> routing).
I'm not an expert either but some simple reasoning:
If the webserver returns a packet, it has to know a route to the client.
You say that the webserver knows the OBSD box, but does it know how to reach
the client ?
If not so, maybe the webserver needs the OBSD box to be its router (set as
default gateway or having a routing rule that tells the webserver to send
packets destined for the client network to the OBSD box). I think that is
what Stefan meant to say, no ?

HTH, Jaap

<google@humanprototype.com> wrote in message
news:1132822831.900534.164920@g47g2000cwa.googlegr oups.com...
> Sorry, have an error in my second-to last port. The correct IP
> addresses are:
>
> Client IP: 10.4.12.234 OpenBSD receiving if (to client): 10.4.12.142
> OpenBSD if for redirection to webserver: 10.4.10.237 (xl2) Webserver
> if: 10.4.10.235

try this on your webserver:
route add default 10.4.10.237

stefan

Hi,

Thanks for the replies.

Jaap:

> You say that the webserver knows the
> OBSD box, but does it know how to reach
> the client ?

Yes. The webservers at 10.4.12.235 and 10.4.11.26 know 10.4.x.x as a 16
bit network, hence it can reach the clients, which are all 10.4.12.x.x.

The clients are accessing the web server (at least the old one) as part
of daily business so that routing is fine.

Stefan:

Thanks, I understand now what you are suggesting. Still I can't make
the OpenBSD box the default route for the web server - it needs the
default route to be set to the company gateway or some of the web apps
running there will break. I might try this as a last resort thing, but
I can't really risk breaking the apps for the users.

I have throught this through in the meantime and I discover further
flaws in my original plan. E.g. the packets from the client, through
the OpenBSD box to the web server will leave the OpenBSD box on one
interface and come back on another (due to my rather "excotic"
subnetting), this will probably break the statefullness anyway. I don't
think that statefullness spawns over multiple interfaces.

I will now try to use one network card in the OpenBSD box, and try to
use 10.4.x.x/16 on that one so I can address both the client and the
two web servers with one network card and have all machines in the same
subnet. I will then try if port forwarding from the OpenBSD box to the
web server works this way. If that works, its the easier and better
way. If not, I am lost

Will keep you all posted. If anyone knows already that the approach
with one common subnet and port forwarding wouldn't work however, I'd
be happy to hear about it.

Cheers,
Carsten

Got it! Has to do with routing, but default gateway didn't help.

I'll explain:

I added the default gw as you suggested, but the web server still had a
way of reaching the client directly (by being in the same 10.4.x.x/16
network) so it never used the default gateway to reach the clients.

However, then I added this:

route add -net 10.4.12.0 netmask 255.255.255.0 gw 10.4.12.142

to force all packages coming from clients, going through the OpenBSD
box, and back to the clients on the web server to use the OpenBSD box
as gateway.

This works I think because the route above is more stringent (for a 24
bit subnet) than the 16 bit network direct route it has, and hence ...
it works - I can go, on my client, to http://10.4.12.142 and see the
web page the same way I can do by going to http://10.4.12.235.

Many thanks for all replies.

Happy too soon. It works now, I can bend ports each way, and I can
reach the web servers from clients directly (and the return traffic
through the OpenBSD box) or through the OpenBSD box entirely. Totally
my desired setup.

But I end up getting only half web pages now. It connects, but pages
are returned half, or miss elements towards the bottom of the page.
Seems that now I run into to some weird "timing" or sequence number
problems with the tcp stack

Lost again ...