[Jool-list] Jool performance help
ali at acreto.io
ali at acreto.io
Tue Jul 17 13:52:02 CDT 2018
Hi Alberto,
Yes, these are similar performance numbers that I was having in my environment. i.e. 3.15 Mbits/sec
That is an interesting find about MTU size. I'm curious how MTU size can impact the performance?
I also tried to use packet size less than 512B without any improvements.
Appreciate your help with reproducing the issue. Looking forward to any work around to this problem.
Thanks
Muhammad Ali
-----Original Message-----
From: Alberto Leiva <ydahhrk at gmail.com>
Sent: Tuesday, July 17, 2018 11:22 PM
To: ali at acreto.io
Cc: tore at fud.no; jool-list at nic.mx
Subject: Re: [Jool-list] Jool performance help
It does seem to be some sort of offload/MTU issue. If I increase the interfaces' MTU, I get significantly better results:
$ ip netns exec IPv4 ip link set ipv4_to_jool mtu 65535 $ ip netns exec nsJool ip link set to_ipv4 mtu 65535 $ ip netns exec nsJool ip link set to_ipv6 mtu 65535 $ ip netns exec IPv6 ip link set ipv6_to_jool mtu 65535 $ iperf -B 172.17.1.2 -c 198.10.10.2 -i 1 -t 2
------------------------------------------------------------
Client connecting to 198.10.10.2, TCP port 5001 Binding to local address 172.17.1.2 TCP window size: 2.50 MByte (default)
------------------------------------------------------------
[ 3] local 172.17.1.2 port 59082 connected with 198.10.10.2 port 5001
[ ID] Interval Transfer Bandwidth
[ 3] 0.0- 1.0 sec 679 MBytes 5.69 Gbits/sec
[ 3] 1.0- 2.0 sec 662 MBytes 5.55 Gbits/sec
[ 3] 0.0- 2.0 sec 1.31 GBytes 5.61 Gbits/sec
I'm still investigating.
On Tue, Jul 17, 2018 at 12:36 PM Alberto Leiva <ydahhrk at gmail.com> wrote:
>
> Muhammad:
>
> I might have successfully reproduced the problem. Can you please
> confirm if these are the kinds of numbers that you're seeing?
>
> ------------------------------------------------------------
> Client connecting to 198.10.10.2, TCP port 5001 Binding to local
> address 172.17.1.2 TCP window size: 85.0 KByte (default)
> ------------------------------------------------------------
> [ 3] local 172.17.1.2 port 5001 connected with 198.10.10.2 port 5001
> [ ID] Interval Transfer Bandwidth
> [ 3] 0.0- 1.0 sec 384 KBytes 3.15 Mbits/sec
> [ 3] 1.0- 2.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 2.0- 3.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 3.0- 4.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 4.0- 5.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 5.0- 6.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 6.0- 7.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 7.0- 8.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 8.0- 9.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 9.0-10.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 10.0-11.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 11.0-12.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 12.0-13.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 13.0-14.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 14.0-15.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 15.0-16.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 16.0-17.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 17.0-18.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 18.0-19.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 19.0-20.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 20.0-21.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 21.0-22.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 22.0-23.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 23.0-24.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 24.0-25.0 sec 128 KBytes 1.05 Mbits/sec
> [ 3] 25.0-26.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 26.0-27.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 27.0-28.0 sec 384 KBytes 3.15 Mbits/sec
> [ 3] 28.0-29.0 sec 384 KBytes 3.15 Mbits/sec
> [ 3] 29.0-30.0 sec 256 KBytes 2.10 Mbits/sec
> [ 3] 0.0-30.3 sec 6.38 MBytes 1.76 Mbits/sec On Mon, Jul 16, 2018
> at 3:00 PM <ali at acreto.io> wrote:
> >
> > Hi Alberto and Tore,
> >
> > Thanks for your feedback.
> >
> > I've done removed the modules using following
> >
> > #rmmod jool
> > #rmmod jool_siit
> >
> > Re inserted only jool_siit module using following (In global
> > namespace)
> >
> > #modprobe jool_siit no_instance
> >
> > In Jool name space I did the following
> >
> > ip netns exec nsJool jool_siit --instance --add
> >
> > ip netns exec nsJool jool_siit --eamt --add 172.17.1.2
> > 2001:db8::c60a:a03 ip netns exec nsJool jool_siit --eamt --add
> > 198.10.10.2 2001:db8::c60a:a02
> >
> > Note: Ping and connectivity bw IPv4 and IPv6 host works fine.
> >
> > Please find attached the "ethtool -k " output for interface. All the veth pairs have similar ethtool configurations.
> >
> > Also find attached traffic capture on IPv6 interface of nsJool. I've observed a lot of TCP out of order and retransmissions.
> >
> > Can you take a look at the configurations and provide your feedback on why could be there TCP out of order and retransmissions.
> >
> > Appreciate your help
> >
> > Thanks
> > Muhammad Ali
> >
> > -----Original Message-----
> > From: Alberto Leiva <ydahhrk at gmail.com>
> > Sent: Friday, July 13, 2018 12:09 AM
> > To: tore at fud.no
> > Cc: ali at acreto.io; jool-list at nic.mx
> > Subject: Re: [Jool-list] Jool performance help
> >
> > Thanks, Tore!
> >
> > I would like to add the following:
> >
> > > $ modprobe jool
> > > $ modprobe jool_siit
> >
> > Are you sure that this is what you want?
> >
> > I'm not sure why you would want to insert both modules in the same namespace. One is a SIIT and the other one is a NAT64. Particularly if you're performance-testing, I'd normally expect you to test one *or* the other.
> > On Thu, Jul 12, 2018 at 2:09 AM Tore Anderson <tore at fud.no> wrote:
> > >
> > > * ali at acreto.io
> > >
> > > > I’ve installed Jool kernel modules and userspace application on Ubuntu. jool_siit is running in a network namespace and I’m using veth pairs for network I/O. Please find attached details of my test environment.
> > > >
> > > > However, while running the TCP throughput test, I was able to achieve only 6Mbps of throughput. I’ve tested it by doing both *GRO on and off* on all the relevant interfaces with no performance improvements.
> > > >
> > > > We are evaluating Jool for carrier grade NAT64 in our network infrastructure.
> > > >
> > > > I was wondering if you can help me to improve performance results. Is there any tweaks or workarounds to overcome the performance limitation by Jool.
> > >
> > > Hi Muhammad,
> > >
> > > First off, you're definitively not hitting the performance limit
> > > of Jool - it easily scales to multiple Gb/s of throughput. There
> > > must be something else that is causing your issues.
> > >
> > > Even though you said you turned GRO off, my suspicion would be
> > > something with packet sizes. Are there other offload settings you
> > > can turn off? MTU settings on all the interfaces are all okay?
> > >
> > > Also, check with tcpdump on all the relevant interfaces to see if
> > > the test traffic is causing lots of ICMP Frag Needed/Packet Too Big errors.
> > >
> > > Tore
> > > _______________________________________________
> > > Jool-list mailing list
> > > Jool-list at nic.mx
> > > https://mail-lists.nic.mx/listas/listinfo/jool-list
More information about the Jool-list
mailing list