This page includes clusters that are potentially suitable for evaluating Homa Transport. To be more specific, we are particularly interested in low-latency Ethernet NIC and switch with (1) at least 8 data queues and (2) strict-priority (SP) queuing support for 802.1p tagged traffic.
Candidate 1: d430 nodes at Emulab Utah
Summary
- 160 nodes (4 racks x 40 nodes/rack) in total
- 1 dual-port or 1 quad-port Intel X710 10GbE PCI-Express NICs
- 1 Dell Z9500 132-port 40Gb switch (latency 600ns - 2us, 8 QoS data queues) connects all 10Gb interfaces
QoS Configuration
NIC
Switch
Useful Links
Utah Emulab hardware in general: https://wiki.emulab.net/wiki/Emulab/wiki/UtahHardware
Dell R430 (aka "d430") nodes: https://wiki.emulab.net/Emulab/wiki/d430
Emulab data plane layout: http://www.emulab.net/doc/docwrapper.php3?docname=topo.html
Candidate 2: m510 nodes at CloudLab Utah
Summary
- 270 nodes (6 chassis x 45 nodes/chassis) in total
- Dual-port Mellanox ConnectX-3 10 Gb NIC (PCIe v3.0, 8 lanes)
- meaning? (btw, I couldn't find enough info about the number of queues or if it support SP queuing)
- 2 HP Moonshot 45XGc switches (45x10Gb ports, 4x40Gb ports for uplink to the core, latency < 1us, cut-through, 8 data queues) per chassis
- 1 large HP FlexFabric 12910 switch (96x40Gb ports, full bisection bandwidth internally, latency 6 - 16us) connecting all chassis
- They "have plans to enable some users to allocate entire chassis ... to have complete administrator control over the switches"
QoS Configuration
NIC
Switch
CloudLab Utah cluster hardware specs:
http://docs.cloudlab.us/hardware.html#%28part._cloudlab-utah%29
https://www.cloudlab.us/hardware.php#utah
ConnectX-3 Dual-Port 10 GbE Adapters w/ PCI Express 3.0: http://www.mellanox.com/page/products_dyn?product_family=127
HP Moonshot 45XGc switch QuickSpecs: https://www.hpe.com/h20195/v2/getpdf.aspx/c04384058.pdf
R242x-HPE Moonshot 45Gc/45XGc/180XGc Switch Module ACL and QoS Command Reference: http://h20565.www2.hpe.com/hpsc/doc/public/display?sp4ts.oid=7398915&docLocale=en_US&docId=emr_na-c05039731
Candidate 3: ?
TODO
Miscellaneous
Relationship between CloudLab and Emulab/AptLab/etc.: https://www.cloudlab.us/cluster-graphs.php