Discussion of technical issues relating to NetBSD's networking subsystem

der Mouse | 6 Apr 2011 04:13

4.0.1 NAT checksum failure?

I'm seeing something which looks like failure to recompute the IP
header checksum when NATting packets with 4.0.1.

I can't believe this wouldn't've been noticed long ago if it were a
generic problem (I'm even on i386), so there's obviously some respect
in which I'm pushing an envelope here.

For example, here's the post-NAT header of a NATted ping:

45                       ip_hl=5 ip_v=4
00                       ip_tos [Routine]
00 54                    ip_len [84] (dropping 2 trailing bytes)
1a d8                    ip_id
00 00                    ip_off [0]
fe                       ip_ttl [254]
01                       ip_p [ICMP]
18 82                    ip_sum
45 c4 b5 1d              ip_src [69.196.181.29]
d8 2e 05 0d              ip_dst [216.46.5.13]

ip_sum is definitely wrong.  But the pre-nat source address was
172.16.0.3, and if I compute the checksum with 45 c4 b5 1d replaced
with ac 10 00 03, I find that 18 82 is correct.

There's another machine (also i386 4.0.1) which is set up to do NAT for
two others, and it works for one of them and doesn't work for the
other.  The only thing that I can see that could be related is that in
each of the failure cases, the failing address is an alias address on
the interface in question rather than being the principal address.

(Continue reading)

headers

Matthew Mondor | 6 Apr 2011 05:33

Re: 4.0.1 NAT checksum failure?

On Tue, 5 Apr 2011 22:13:43 -0400 (EDT)
der Mouse <mouse <at> Rodents-Montreal.ORG> wrote:

> Can anyone confirm or refute the theory that 4.0.1's NAT simply doesn't
> get checksums right for addresses on alias networks in this sense?
> I'll be digging through the code, but I don't know that code, and
> strengthening or refuting my guess would help me focus my search.

It's not exactly clear to me which is inbound and outbound in the
example (it's also late in my tz, admitedly), but is it possible
that because hardware TCP4 checksum is enabled the dumps don't report
it properly yet output packets get the sum adjusted at actual delivery?

Thanks,
--

-- 
Matt

headers

der Mouse | 6 Apr 2011 05:41

Re: 4.0.1 NAT checksum failure?

>> Can anyone confirm or refute the theory that 4.0.1's NAT simply
>> doesn't get checksums right for addresses on alias networks [...]
> It's not exactly clear to me which is inbound and outbound in the
> example (it's also late in my tz, admitedly), but is it possible that
> because hardware TCP4 checksum is enabled the dumps don't report it
> properly yet output packets get the sum adjusted at actual delivery?

Well, not TCP4, because this isn't TCP; it happens even on pings.  But
presumably IP-layer checksums have similar bits.

It definitely is not as simple as I thought/feared it might be; I did
some more tests and found a test case where a ping from a non-alias
network does not get NATted correctly.  Now I need to figure out what
the _actually_ relevant difference between working and broken is. :-/

Also, it's not just checksum offload; all network interfaces on this
machine are configured with no checksum offload, even the ones that are
capable of doing it.  I definitely need to focus on the checksum code,
though since it's a checksum issue that's been obvious from the start.

/~\ The ASCII				  Mouse
\ / Ribbon Campaign
 X  Against HTML		mouse <at> rodents-montreal.org
/ \ Email!	     7D C8 61 52 5D E7 2D 39  4E F1 31 3E E8 B3 27 4B

headers

Matthew Mondor | 6 Apr 2011 06:22

Re: 4.0.1 NAT checksum failure?

On Tue, 5 Apr 2011 23:41:53 -0400 (EDT)
der Mouse <mouse <at> Rodents-Montreal.ORG> wrote:

> Well, not TCP4, because this isn't TCP; it happens even on pings.  But
> presumably IP-layer checksums have similar bits.

Indeed

> Also, it's not just checksum offload; all network interfaces on this
> machine are configured with no checksum offload, even the ones that are
> capable of doing it.  I definitely need to focus on the checksum code,
> though since it's a checksum issue that's been obvious from the start.

Oh, I didn't even notice that those were listed as capabilities but
indeed disabled, or they'd also show on the first line.  Sorry about
that.

In case this can help any (this could have changed since netbsd-4), but
in netbsd-5 where I'm more familiar with code location, I seem to see
calls to in_delayed_cksum() in sys/netinet/ip_output.c which calls
in4_cksum.c's in4_cksum(), in turn using cpu_in_cksum.c's
cpu_in_cksum() (I've not checked but it'd be possible for each arch to
supply its own perhaps).  It seems somewhat tricky as the offset/length
are parameters that could be wrongly passed in any caller.
Then in sys/dist/ipf/netinet/ there are a number of matching lines for
cksum as well, including in ip_nat.c ...

Good night and happy debugging,
--

-- 
Matt

(Continue reading)

headers

der Mouse | 6 Apr 2011 06:43

Re: 4.0.1 NAT checksum failure?

>> Also, it's not just checksum offload; [...]

Okay, I believe I know what was going on here.

In my tree, if_srt.c has code to call pfil_run_hooks, so that you can
configure srt to point out an interface and then configure NAT for
outgoing packets on that interface and have it work.  (Without this,
the NAT code will run for the srt interface, not the forwarded-to
interface - but, because the return traffic comes in on the
forwarded-to interface rather than the srt interface, return traffic
won't be handled right even if you configure the NAT on the srt.)

Turned out the actual relevant difference between my "works" and
"fails" cases were that the "works" cases were hitting host routes that
bypassed the relevant srt - nothing to do with primary versus alias
addresses anywhere.

I've added code to recompute checksums after running pfil hooks

--- a/sys/net/if_srt.c
+++ b/sys/net/if_srt.c
 <at>  <at>  -298,6 +298,13  <at>  <at>  static int srt_if_output(
   { simple_unlock(&sc->lock);
     return(rv);
   }
+ if (dst->sa_family == AF_INET)
+  { struct ip *ip;
+    ip = mtod(m,struct ip *);
+    ip->ip_sum = 0;
+    ip->ip_sum = in_cksum(m,ip->ip_hl<<2);

(Continue reading)