[Jool-list] Jool performance help
Alberto Leiva
ydahhrk at gmail.com
Wed Jul 18 19:34:52 CDT 2018
Hello again:
Just in case you're not listening to the issue tracker
(https://github.com/NICMx/Jool/issues/267), I found a likely formal
fix to this problem, and uploaded the code to the issue267 branch.
(https://github.com/NICMx/Jool/tree/issue267)
Performance should increase dramatically regardless of interface MTU.
Though the code is scheduled to undergo formal testing by NIC Mexico
next week, I would be truly thankful for some additional user
feedback. As soon as the code is tested, Jool version 3.5.8 will be
released.
Greetings,
Alberto
On Tue, Jul 17, 2018 at 4:48 PM Alberto Leiva <ydahhrk at gmail.com> wrote:
>
> > That is an interesting find about MTU size. I'm curious how MTU size can impact the performance?
>
> Whenever active, offloads tweak and make a mess out of packet sizes.
> Then Jool sees incorrect data and the resulting translated packets
> exceed the MTU. By increasing the MTU, we prevent these packets from
> being dropped.
>
> Because offloading is kind of random, some packets manage to reach the
> destination. Because TCP retries a lot, the result is extreme slowness
> instead of utter DoS.
>
> > I also tried to use packet size less than 512B without any improvements.
>
> It's not effective because these small packets are being merged into
> very big packets somewhere.
>
> > Looking forward to any work around to this problem.
>
> Well, if you're only dealing with virtual interfaces, the solution for
> now is to just increase the MTU. If you try the experiment on actual
> hardware, then the problem is unlikely to exist in the first place.
>
> --------------------
>
> By the way:
>
> This is not really that important, but your network looks a little
> strange to me. It's not really an "idiomatic" SIIT setup.
>
> Is this the intended behavior that you want?
>
> 1. IPv4 iperf writes packets [172.17.1.2 -> 198.10.10.2]
> 2. According to the EAMT, Jool translates those packets into
> [2001:db8::c60a:a03 -> 2001:db8::c60a:a02]
>
> Right now the situation is a little strange because the packet appears
> to be directed towards a node on the same network (2001:db8::/64).
> This is more akin to the nature of NAT rather than SIIT's.
>
> It's not something that can't be made to work with a few strange
> routing rules, but it does mean that your translator will only be able
> to mask packets coming from ONE IPv4 node. Unless I'm missing some
> really important detail.
>
> The idea of SIIT is simply to "rename" networks, not have someone else
> impersonate them. See the image attached to this mail for a visual
> depiction of what I understand to be "idiomatic" SIIT.
> On Tue, Jul 17, 2018 at 1:54 PM <ali at acreto.io> wrote:
> >
> > Hi Alberto,
> >
> > Yes, these are similar performance numbers that I was having in my environment. i.e. 3.15 Mbits/sec
> >
> > That is an interesting find about MTU size. I'm curious how MTU size can impact the performance?
> >
> > I also tried to use packet size less than 512B without any improvements.
> >
> > Appreciate your help with reproducing the issue. Looking forward to any work around to this problem.
> >
> > Thanks
> > Muhammad Ali
> >
> > -----Original Message-----
> > From: Alberto Leiva <ydahhrk at gmail.com>
> > Sent: Tuesday, July 17, 2018 11:22 PM
> > To: ali at acreto.io
> > Cc: tore at fud.no; jool-list at nic.mx
> > Subject: Re: [Jool-list] Jool performance help
> >
> > It does seem to be some sort of offload/MTU issue. If I increase the interfaces' MTU, I get significantly better results:
> >
> > $ ip netns exec IPv4 ip link set ipv4_to_jool mtu 65535 $ ip netns exec nsJool ip link set to_ipv4 mtu 65535 $ ip netns exec nsJool ip link set to_ipv6 mtu 65535 $ ip netns exec IPv6 ip link set ipv6_to_jool mtu 65535 $ iperf -B 172.17.1.2 -c 198.10.10.2 -i 1 -t 2
> > ------------------------------------------------------------
> > Client connecting to 198.10.10.2, TCP port 5001 Binding to local address 172.17.1.2 TCP window size: 2.50 MByte (default)
> > ------------------------------------------------------------
> > [ 3] local 172.17.1.2 port 59082 connected with 198.10.10.2 port 5001
> > [ ID] Interval Transfer Bandwidth
> > [ 3] 0.0- 1.0 sec 679 MBytes 5.69 Gbits/sec
> > [ 3] 1.0- 2.0 sec 662 MBytes 5.55 Gbits/sec
> > [ 3] 0.0- 2.0 sec 1.31 GBytes 5.61 Gbits/sec
> >
> > I'm still investigating.
> > On Tue, Jul 17, 2018 at 12:36 PM Alberto Leiva <ydahhrk at gmail.com> wrote:
> > >
> > > Muhammad:
> > >
> > > I might have successfully reproduced the problem. Can you please
> > > confirm if these are the kinds of numbers that you're seeing?
> > >
> > > ------------------------------------------------------------
> > > Client connecting to 198.10.10.2, TCP port 5001 Binding to local
> > > address 172.17.1.2 TCP window size: 85.0 KByte (default)
> > > ------------------------------------------------------------
> > > [ 3] local 172.17.1.2 port 5001 connected with 198.10.10.2 port 5001
> > > [ ID] Interval Transfer Bandwidth
> > > [ 3] 0.0- 1.0 sec 384 KBytes 3.15 Mbits/sec
> > > [ 3] 1.0- 2.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 4.0- 5.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 5.0- 6.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 6.0- 7.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 8.0- 9.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 10.0-11.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 11.0-12.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 12.0-13.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 13.0-14.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 14.0-15.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 15.0-16.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 16.0-17.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 17.0-18.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 18.0-19.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 19.0-20.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 20.0-21.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 21.0-22.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 22.0-23.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 23.0-24.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 24.0-25.0 sec 128 KBytes 1.05 Mbits/sec
> > > [ 3] 25.0-26.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 26.0-27.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 27.0-28.0 sec 384 KBytes 3.15 Mbits/sec
> > > [ 3] 28.0-29.0 sec 384 KBytes 3.15 Mbits/sec
> > > [ 3] 29.0-30.0 sec 256 KBytes 2.10 Mbits/sec
> > > [ 3] 0.0-30.3 sec 6.38 MBytes 1.76 Mbits/sec On Mon, Jul 16, 2018
> > > at 3:00 PM <ali at acreto.io> wrote:
> > > >
> > > > Hi Alberto and Tore,
> > > >
> > > > Thanks for your feedback.
> > > >
> > > > I've done removed the modules using following
> > > >
> > > > #rmmod jool
> > > > #rmmod jool_siit
> > > >
> > > > Re inserted only jool_siit module using following (In global
> > > > namespace)
> > > >
> > > > #modprobe jool_siit no_instance
> > > >
> > > > In Jool name space I did the following
> > > >
> > > > ip netns exec nsJool jool_siit --instance --add
> > > >
> > > > ip netns exec nsJool jool_siit --eamt --add 172.17.1.2
> > > > 2001:db8::c60a:a03 ip netns exec nsJool jool_siit --eamt --add
> > > > 198.10.10.2 2001:db8::c60a:a02
> > > >
> > > > Note: Ping and connectivity bw IPv4 and IPv6 host works fine.
> > > >
> > > > Please find attached the "ethtool -k " output for interface. All the veth pairs have similar ethtool configurations.
> > > >
> > > > Also find attached traffic capture on IPv6 interface of nsJool. I've observed a lot of TCP out of order and retransmissions.
> > > >
> > > > Can you take a look at the configurations and provide your feedback on why could be there TCP out of order and retransmissions.
> > > >
> > > > Appreciate your help
> > > >
> > > > Thanks
> > > > Muhammad Ali
> > > >
> > > > -----Original Message-----
> > > > From: Alberto Leiva <ydahhrk at gmail.com>
> > > > Sent: Friday, July 13, 2018 12:09 AM
> > > > To: tore at fud.no
> > > > Cc: ali at acreto.io; jool-list at nic.mx
> > > > Subject: Re: [Jool-list] Jool performance help
> > > >
> > > > Thanks, Tore!
> > > >
> > > > I would like to add the following:
> > > >
> > > > > $ modprobe jool
> > > > > $ modprobe jool_siit
> > > >
> > > > Are you sure that this is what you want?
> > > >
> > > > I'm not sure why you would want to insert both modules in the same namespace. One is a SIIT and the other one is a NAT64. Particularly if you're performance-testing, I'd normally expect you to test one *or* the other.
> > > > On Thu, Jul 12, 2018 at 2:09 AM Tore Anderson <tore at fud.no> wrote:
> > > > >
> > > > > * ali at acreto.io
> > > > >
> > > > > > I’ve installed Jool kernel modules and userspace application on Ubuntu. jool_siit is running in a network namespace and I’m using veth pairs for network I/O. Please find attached details of my test environment.
> > > > > >
> > > > > > However, while running the TCP throughput test, I was able to achieve only 6Mbps of throughput. I’ve tested it by doing both *GRO on and off* on all the relevant interfaces with no performance improvements.
> > > > > >
> > > > > > We are evaluating Jool for carrier grade NAT64 in our network infrastructure.
> > > > > >
> > > > > > I was wondering if you can help me to improve performance results. Is there any tweaks or workarounds to overcome the performance limitation by Jool.
> > > > >
> > > > > Hi Muhammad,
> > > > >
> > > > > First off, you're definitively not hitting the performance limit
> > > > > of Jool - it easily scales to multiple Gb/s of throughput. There
> > > > > must be something else that is causing your issues.
> > > > >
> > > > > Even though you said you turned GRO off, my suspicion would be
> > > > > something with packet sizes. Are there other offload settings you
> > > > > can turn off? MTU settings on all the interfaces are all okay?
> > > > >
> > > > > Also, check with tcpdump on all the relevant interfaces to see if
> > > > > the test traffic is causing lots of ICMP Frag Needed/Packet Too Big errors.
> > > > >
> > > > > Tore
> > > > > _______________________________________________
> > > > > Jool-list mailing list
> > > > > Jool-list at nic.mx
> > > > > https://mail-lists.nic.mx/listas/listinfo/jool-list
> >
> >
More information about the Jool-list
mailing list