[Info-vax] Linux 40 GbE and 100 GbE NIC Performance

[Stephen Hoffman]

On 2015-01-24 03:59:57 +0000, David Froble said:

> Stephen Hoffman wrote:
>> 
>> and whether there might be a need to dedicate one or more cores for the 
>> NIC processing?
> 
> This is an interesting statement.
> 
> It's been a while since there was "The VAX".  Different world.

Yes, many things are different now.  Some are not.  Dedicating 
processors to tasks is not particularly new, though.  There were VAX 
processors embedded on some DEC Ethernet boards, and in a few printers. 
 There were dedicated CPUs on some Q-bus boards.   There were boards 
that depended entirely on the host VAX, too.  Intel is now aiming Xeon 
at embedded systems, with some changes mentioned below, too.  
Dedicating a processor to a task or to a device withi  a VMS SMP 
configuration has been possible for a while, too.

> This might have been questionable when it was one core per socket.  
> With multiple cores per socket, and with some of the on chip 
> communications, for some purposes planning on having a "network 
> processor" might be a very good idea.

Recent Intel Xeon integrated the PCIe onto the processor, and has 
provided a way for PCIe devices to initiate and perform DMA into and 
out of processor cache directly; into and out of level 3 shared cache.  
What Intel refers to as DDIO.  This helps for some tasks as it is 
faster than DMA into memory (which then has to get loaded into 
processor cache), but then the code operating on the core has to figure 
out what to do with the data and either evict the data or transfer it 
into memory.  At least for now, DDIO is local, and is tied to the PCIe 
buses associated with a particular socket and its cores, but it 
wouldn't surprise me to see Intel extend this and allow DDIO operations 
into remote sockets and cores across QPI.  ~250 Gbps in lab tests, 
reportedly.

Alternatively, there's been some discussion of TCP offload engine (TOE) 
support for OpenVMS over the years, but that hardware configuration 
hasn't ever become available and supported on OpenVMS AFAIK.

There are also changes proposed to and wholesale replacements for TCP 
and IP, but that's fodder for another time.

> Now, there might be a very good idea for VMS on x86.  Bring network 
> speeds up to date.

Alpha and Itanium already have some processing that can be dedicated to 
a core, either for device-specific I/O processing — where fast path can 
be used to distribute I/O processing, and fast I/O is intended to be 
faster than $qio — or for the dedicated lock manager, for instance.

OpenVMS network I/O is presently quite a bit slower than RHEL network 
I/O — that's using the VCI path on VMS, too — and that's before any 
consideration of the performance necessary for wire-speed operations of 
40 GbE and 100 GbE NICs.


-- 
Pure Personal Opinion | HoffmanLabs LLC