[Jool-list] Moving to jool

[Nico Schottelius]

Good morning Alberto,

Alberto Leiva <ydahhrk at gmail.com> writes:

> Hi, Nico
>
> Thanks for the heads up. I gave a quick read to a few points in your
> thesis and I'm finding some of your results intriguing.
>
> In particular, this:
>
>> Both software solutions showed significant loss of packets in the UDP based benchmarks (Tayga: up to 91%, Jool up to 71%)
>
> is quite worrying and suggests a substantial bug in Jool's
> implementation that needs attention. I will try to replicate your
> tests and hopefully find the root cause in the following days.

While I did some preparations & tests with and without hardware
offloading, the main script I used in the end can be found on [0]. The
git repository also contains the iperf logfiles [1].

In particular if you look at
iperf-jool-from-v6-c2001:db8:23::2a-p2345-O10-t190-6-P50-u-b0-run1
(in the run10 subdirectory) you see 70% packet loss.

This is however NOT the average nor the default that I have seen with
jool, but more the exception.

> I also noticed this:
>
>> Neither Tayga as a userspace program nor Jool as a kernel module implement multi threading.
>
> Unless something has fundamentally changed in the kernel without me
> noticing, I believe this is incorrect, or at least inaccurate. While
> it's possible that your machine could be dedicating one core to all
> packets, Jool's code is certainly prepared to handle multiple packets
> simultaneously.

Very interesting. I have to admit that I did not read the full code of
jool. However what I observed is that in particular in the UDP based
iperf runs I saw exactly 1 core utilised 100%.

This was very similar to tayga, however tayga reaching this usage much
before jool.

> I have managed to surpass 100% CPU utilization in the past and even
> perceive jumbled logging due to simultaneous packet processing. It's
> been a while since the last time I tortured-tested Jool so this could
> be outdated. Maybe I should look into this as well.

Let me put it that way: if jool was distributed (how?) to different
cores, it looks like having the potential of surpassing 10gbit/s
translations easily.

> (The kernel handles packets by way of interrupts, and each interrupt
> can be assigned to a different CPU. If your kernel is assigning the
> same CPU to all your packets, I believe this can be addressed by
> configuration. Did you see https://serverfault.com/a/335251?)

I actually did not see this, however it might be related, because the
test machines had to be Ubuntu 16.04, as the P4-NetFPGA development suite
is very fragile and it is hard-to-impossible to get it running on
untested Linux versions.

That said, Ubuntu 16.04 obviously did not include a very recent kernel.

> I'm also finding other quirks that might merit some discussion.
> However, it seems that your thesis was finished in August. Would you
> benefit from a review?

While the thesis has indeed been handed in and the project finished (the
focus was mainly on NetFPGA), I am still very much into NAT64 related
topics and interested in also seeing how far we can go with jool.

I should probably have brought this earlier to attention here :-)

I am currently progressing in testing/deploying jool+joold on Alpine and
Devuan and I am likely to retest jool performance the next weeks and
have just added a public ticket for tracking the progress [2].

Thanks again for your feedback, it is much appreciated.

Best,

Nico


[0] https://gitlab.ethz.ch/nicosc/master-thesis/blob/master/bin/benchmark-run.sh
[1] https://gitlab.ethz.ch/nicosc/master-thesis/tree/master/iperf
[2] https://redmine.ungleich.ch/issues/7304?issue_count=148&issue_position=1&next_issue_id=7295

--
Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch