This page describes issues we have had with the cluster (most recent ones first) and how we have dealt with them.
- Strange bandwidth issues: ib_send_bw tests on second rack (rc41-rc80 with ConnectX-3 nics and 56Gbps SwitchX switches) only show ~1650MB/s between nodes.
- Bandwidths are apparently fine (~3200MB/s) between rc01-rc40, which are on th old switches (though they route through the new switches)
- Strangely, bandwidths between nodes in first rack and second rack are fine (~3200MB/s)
- So it appears to be a combination of new nics talking to new nics!?
...