[Jool-list] SNMP packets disappear through NAT64

Mon Apr 22 17:28:37 CDT 2019

Hi there,
I have following configuration:
* 2 linux namespaces with GRE tunnels connected and some basic routing
* 2 GRE tunnels towards each namespace from external routers
My intention is to use NAT64 to access same IPv4 spaces from within single
applicaton unaware of existing overlapping spaces.

The problem: my following setup works OK (ping, ssh, tcp, et all), EXCEPT
SNMP. Some SNMP gets work correctly, while snmpwalk or snmtable rather do
not.

My complete setup is as follows (easy to copy on an machine,

ip addr add 192.168.1.51/24 dev enp0s3 && ip link set enp0s3 up
ip route add 0.0.0.0/0 via 192.168.1.1
# Adding namespaces and interfaces
for i in 10 20; do
ip netns add dm$i
ip link add veth$i type veth peer name br-veth$i
ip link set veth$i netns dm$i
ip -n dm$i address add 192.168.$i.100/24 dev veth$i
ip -n dm$i link set veth$i up
ip link set br-veth$i up
ip link add name br$i type bridge
ip link set br$i up
ip link set br-veth$i master br$i
ip address add 192.168.$i.1/24 dev br$i
ip -n dm$i route add 0.0.0.0/0 via 192.168.$i.1
# Adding same ips to dm10 and dm20
ip -n dm$i address add 10.0.1.1/24 dev lo
done
# Adding different tunnels
ip -n dm10 tunnel add gre10 mode gre remote 192.168.1.61 local
192.168.10.100
ip -n dm20 tunnel add gre20 mode gre remote 192.168.1.62 local
192.168.20.100
ip -n dm10 tunnel add gre101 mode gre remote 192.168.1.71 local
192.168.10.100
ip -n dm20 tunnel add gre201 mode gre remote 192.168.1.72 local
192.168.20.100
for i in lo gre10 gre101; do ip -n dm10 link set $i up; done ; for i in lo
gre20 gre201; do ip -n dm20 link set $i up; done
# IPs of tunnels are accessed externally, for lab we need something to
route to & from
for i in 10 20; do ip -n dm$i route add 10.0.0.0/24 dev gre$i; ip -n dm$i
route add 10.159.146.0/24 dev gre${i}1; done
ip addr add 1.1.1.2/30 dev br10 && ip addr add 1.1.1.6/30 dev br20
ip -n dm10 addr add 1.1.1.1/30 dev veth10 && ip -n dm20  addr add 1.1.1.5/30
dev veth20
# NAT towards services
ip netns exec dm10 iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o veth10
-j NETMAP --to 10.7.7.0/24
ip netns exec dm20 iptables -t nat -A POSTROUTING -s 10.0.0.0/24 -o veth20
-j NETMAP --to 10.8.8.0/24
ip netns exec dm10 iptables -t nat -A PREROUTING -d 10.7.7.0/24 -j NETMAP
--to 10.0.0.0/24
ip netns exec dm20 iptables -t nat -A PREROUTING -d 10.8.8.0/24 -j NETMAP
--to 10.0.0.0/24
ip route add 10.7.7.0/24 via 1.1.1.1 && ip route add 10.8.8.0/24 via 1.1.1.5
# Statefull NAT64
for x in 10 20; do
y=$(printf '%x\n' $x)
ip addr add 2a01:2e0:$y::2/96 dev br$x
ip -n dm$x addr add 2a01:2e0:$y::1/96 dev veth$x
#ip -n dm$x link set veth$x mtu 1280 ???????? doesn't help either
ip netns exec dm$x jool instance add --netfilter dm$x --pool6
2a01:2e0:d:3:$y::/96
ip -6 route add 2a01:2e0:d:3:$y::/96 via 2a01:2e0:$y::1
printf "Network 2a01:2e0:d:3:$y::/96 added\n"
ip -n dm$x link set veth$x mtu 1476
done

Symptom of a failure (executed from host):
snmpwalk  -v2c -r0 -On -c public 2a01:2e0:d:3:a::10.0.0.1   1.3.6.1.2.1.4.24
.1.3.6.1.2.1.4.24.3.0 = Gauge32: 8
.1.3.6.1.2.1.4.24.4.1.1.10.0.0.0.255.255.255.0.0.10.0.0.1 = IpAddress:
10.0.0.0
Timeout: No Response from 2a01:2e0:d:3:a::10.0.0.1

Dump form the host:

23:06:13.327447 Out 9e:e1:20:11:58:31 ethertype IPv6 (0x86dd), length 125:
2001:db8:a::2.48485 > 2a01:2e0:d:3:a:0:a00:1.161:  GetNextRequest(46)
.1.3.6.1.2.1.4.24.4.1.1.10.0.0.0.255.255.255.0.0.10.0.0.1
23:06:13.327506 Out 9e:e1:20:11:58:31 ethertype IPv6 (0x86dd), length 125:
2001:db8:a::2.48485 > 2a01:2e0:d:3:a:0:a00:1.161:  GetNextRequest(46)
.1.3.6.1.2.1.4.24.4.1.1.10.0.0.0.255.255.255.0.0.10.0.0.1
23:06:13.328715  In 96:4e:14:f9:c7:60 ethertype IPv6 (0x86dd), length 134:
2a01:2e0:d:3:a:0:a00:1.161 > 2001:db8:a::2.48485:  GetResponse(55)
.1.3.6.1.2.1.4.24.4.1.1.192.168.1.0.255.255.255.128.0.192.168.1.61=192.168.1.0
23:06:13.328724  In 96:4e:14:f9:c7:60 ethertype IPv6 (0x86dd), length 134:
2a01:2e0:d:3:a:0:a00:1.161 > 2001:db8:a::2.48485:  GetResponse(55)
.1.3.6.1.2.1.4.24.4.1.1.192.168.1.0.255.255.255.128.0.192.168.1.61=192.168.1.0

Dump from the namespace:

23:06:13.327551  In 9e:e1:20:11:58:31 ethertype IPv6 (0x86dd), length 125:
2001:db8:a::2.48485 > 2a01:2e0:d:3:a:0:a00:1.161:  GetNextRequest(46)
.1.3.6.1.2.1.4.24.4.1.1.10.0.0.0.255.255.255.0.0.10.0.0.1
23:06:13.327608 Out ethertype IPv4 (0x0800), length 105: 10.0.1.1.61639 >
10.0.0.1.161:  GetNextRequest(46)
.1.3.6.1.2.1.4.24.4.1.1.10.0.0.0.255.255.255.0.0.10.0.0.1
23:06:13.328655  In ethertype IPv4 (0x0800), length 114: 10.0.0.1.161 >
10.0.1.1.61639:  GetResponse(55)
.1.3.6.1.2.1.4.24.4.1.1.192.168.1.0.255.255.255.128.0.192.168.1.61=192.168.1.0
23:06:13.328708 Out 96:4e:14:f9:c7:60 ethertype IPv6 (0x86dd), length 134:
2a01:2e0:d:3:a:0:a00:1.161 > 2001:db8:a::2.48485:  GetResponse(55)
.1.3.6.1.2.1.4.24.4.1.1.192.168.1.0.255.255.255.128.0.192.168.1.61=192.168.1.0

(tu my understanding, this means packets are replied and delivered to the
requesting IP, but in reality they never reach the snmpwalk back)

Stats from jool (number of unknown_l4_proto grows)

jool -i dm10 stats display
JSTAT_SUCCESS: 912
JSTAT_BIB_ENTRIES: 3
JSTAT_SESSIONS: 3
JSTAT_UNKNOWN_L4_PROTO: 4708
JSTAT_UNKNOWN_ICMP6_TYPE: 73
JSTAT_POOL6_MISMATCH: 6
JSTAT_POOL4_MISMATCH: 46957
JSTAT_BIB4_NOT_FOUND: 18

I tried to understand MTU concerns. I tuned MTU of a namespace interface
(veth10) to lowest amount expected in the network (1476 as observed in the
GRE) but that didn't help. I tried replacing snmpwalk with pysnmp (exactly
the same results) and direct IPv6 from namespace to the router on pure IPv6
(the last worked, and it proves the snmpwalk / pysnmp can into IPv6).

So, all in all, I think I have all alright but SNMP packets are getting
lost.

Any more hints? MTU considerations? other setups? some netfilter rules to
log a failing rule?

Thanks.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail-lists.nic.mx/pipermail/jool-list/attachments/20190423/64cdaedf/attachment.html>