[Jool-list] Moving to jool

Thu Nov 7 06:02:32 CST 2019

Hi Nico,

I have read your complete document when you sent it to the list, and I want to thank you for it.

I'm a frequent user of Jool, and teach about it to the community and customers.

I was also surprised about your UDP failures, I've never seen that before, so as you just said, it may be due to your specific configuration. I recall having tested Jool the first time in Ubuntu 16.x, but I often try to upgrade the kernel to the latest available release, etc.

In fact, I usually check and adjust myself CPU affinity (even I do that in my OpenWRT routers!).

One suggestion, in case you can invest a bit of extra time on this, so to make your work more comprehensive, will be to test also using VPP:

https://docs.fd.io/vpp/17.07/nat64_doc.html

I will actually say, if you allow me, "forget Tayga", it doesn't scale, isn't longer mantained, and Jool and VPP are much better targets to focus on!

Thanks!

Regards,
Jordi
@jordipalet

El 7/11/19 12:30, "Jool-list en nombre de Nico Schottelius" <jool-list-bounces at nic.mx en nombre de nico.schottelius at ungleich.ch> escribió:

    Good morning Alberto,

    Alberto Leiva <ydahhrk at gmail.com> writes:

    > Hi, Nico
    >
    > Thanks for the heads up. I gave a quick read to a few points in your
    > thesis and I'm finding some of your results intriguing.
    >
    > In particular, this:
    >
    >> Both software solutions showed significant loss of packets in the UDP based benchmarks (Tayga: up to 91%, Jool up to 71%)
    >
    > is quite worrying and suggests a substantial bug in Jool's
    > implementation that needs attention. I will try to replicate your
    > tests and hopefully find the root cause in the following days.

    While I did some preparations & tests with and without hardware
    offloading, the main script I used in the end can be found on [0]. The
    git repository also contains the iperf logfiles [1].

    In particular if you look at
    iperf-jool-from-v6-c2001:db8:23::2a-p2345-O10-t190-6-P50-u-b0-run1
    (in the run10 subdirectory) you see 70% packet loss.

    This is however NOT the average nor the default that I have seen with
    jool, but more the exception.

    > I also noticed this:
    >
    >> Neither Tayga as a userspace program nor Jool as a kernel module implement multi threading.
    >
    > Unless something has fundamentally changed in the kernel without me
    > noticing, I believe this is incorrect, or at least inaccurate. While
    > it's possible that your machine could be dedicating one core to all
    > packets, Jool's code is certainly prepared to handle multiple packets
    > simultaneously.

    Very interesting. I have to admit that I did not read the full code of
    jool. However what I observed is that in particular in the UDP based
    iperf runs I saw exactly 1 core utilised 100%.

    This was very similar to tayga, however tayga reaching this usage much
    before jool.

    > I have managed to surpass 100% CPU utilization in the past and even
    > perceive jumbled logging due to simultaneous packet processing. It's
    > been a while since the last time I tortured-tested Jool so this could
    > be outdated. Maybe I should look into this as well.

    Let me put it that way: if jool was distributed (how?) to different
    cores, it looks like having the potential of surpassing 10gbit/s
    translations easily.

    > (The kernel handles packets by way of interrupts, and each interrupt
    > can be assigned to a different CPU. If your kernel is assigning the
    > same CPU to all your packets, I believe this can be addressed by
    > configuration. Did you see https://serverfault.com/a/335251?)

    I actually did not see this, however it might be related, because the
    test machines had to be Ubuntu 16.04, as the P4-NetFPGA development suite
    is very fragile and it is hard-to-impossible to get it running on
    untested Linux versions.

    That said, Ubuntu 16.04 obviously did not include a very recent kernel.

    > I'm also finding other quirks that might merit some discussion.
    > However, it seems that your thesis was finished in August. Would you
    > benefit from a review?

    While the thesis has indeed been handed in and the project finished (the
    focus was mainly on NetFPGA), I am still very much into NAT64 related
    topics and interested in also seeing how far we can go with jool.

    I should probably have brought this earlier to attention here :-)

    I am currently progressing in testing/deploying jool+joold on Alpine and
    Devuan and I am likely to retest jool performance the next weeks and
    have just added a public ticket for tracking the progress [2].

    Thanks again for your feedback, it is much appreciated.

    Best,

    Nico

    [0] https://gitlab.ethz.ch/nicosc/master-thesis/blob/master/bin/benchmark-run.sh
    [1] https://gitlab.ethz.ch/nicosc/master-thesis/tree/master/iperf
    [2] https://redmine.ungleich.ch/issues/7304?issue_count=148&issue_position=1&next_issue_id=7295

    --
    Modern, affordable, Swiss Virtual Machines. Visit www.datacenterlight.ch
    _______________________________________________
    Jool-list mailing list
    Jool-list at nic.mx
    https://mail-lists.nic.mx/listas/listinfo/jool-list

**********************************************
IPv4 is over
Are you ready for the new Internet ?
http://www.theipv6company.com
The IPv6 Company

This electronic message contains information which may be privileged or confidential. The information is intended to be for the exclusive use of the individual(s) named above and further non-explicilty authorized disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited and will be considered a criminal offense. If you are not the intended recipient be aware that any disclosure, copying, distribution or use of the contents of this information, even if partially, including attached files, is strictly prohibited, will be considered a criminal offense, so you must reply to the original sender to inform about this communication and delete it.