Service Locators

Owned by John Ousterhout
Last updated: Jun 22, 2016Version comment
3 min read

A service locator is a string that describes how to contact a particular server, such as a RAMCloud storage server or the cluster coordinator. Each service locator specifies a transport along with parameters to that transport. A transport is a particular mechanism for communication, such as "kernel-level TCP" or "Infiniband reliable connected queue-pairs". RAMCloud supports several different transports, and it is relatively straightforward  to build new transports to experiment with new kinds of networking. The parameters in a service locator provide additional information that identify a particular server. For example, in the case of TCP, the parameters specify a host name and port number. Here is an example of a service locator:

tcp:host=rc01,port=11000

The transport name is tcp; it must come first, and is followed by a colon. The remainder of the service locator specifies names and values for parameters. In this locator, two parameters are specified: host has the value rc01, and port has the value 11000. Parameters are separated by commas.

When you start a RAMCloud server, you provide it with its service locator; this tells the server which transport to use, along with other information such as which port it should listen on.

RAMCloud currently supports the following transports:

basic

Example: basic+udp:host=rc01,port=11000

Basic is RAMCloud's workhorse transport; it provides the best combination of speed and robustness, so we recommend that you use this transport by default. It uses an unreliable datagram mechanism to deliver packets, and provides reliable, flow-controlled, in-order delivery on top of the underlying datagram mechanism. The basic transport is distinct from the datagram mechanism, and it can work with any datagram mechanism that implements a simple API. As of 2016, RAMCloud contains drivers for the following datagram mechanisms:

udp: Sends UDP packets through the kernel. This driver is not very fast, since data must flow through the kernel, but it will run almost anywhere.

infud: Uses Infiniband RDMA to send datagrams. The RDMA mechanism allows direct communication between applications and the Infiniband NIC, which provides low latency and high throughput.

dpdk: Uses the Intel DPDK library to send and receive UDP packets. DPDK also uses kernel bypass, so this driver is quite fast.

solarflare: Uses the SolarFlare OpenOnload package to send and receive UDP packets. This driver requires SolarFlare NICs, but it uses kernel bypass, so it is also fast.

Service locators for the basic transport must specify both the basic transport and the particular driver, e.g. basic+udp, plus additional parameters as required by the driver. For details and examples, check the file scripts/cluster.py in the RAMCloud distribution. In addition, you may need to check the in-code documentation for particular drivers in order to see what parameters they support.

tcp

Example: tcp:host=rc01,port=11000

The tcp transport uses TCP accessed via standard kernel sockets. This is the most portable of the RAMCloud transports, but unfortunately it is considerably slower than other transports because all data must flow through the kernel.  The tcp transport takes two parameters: host is the name or IP address for the node where the server executes, and port is the port number on which the server listens for connections.

infrc

Example: infrc:host=rc01,port=11000

The infrc transport uses Infiniband reliable connected queue pairs. It is built on the Mellanox RDMA library package, which allows the Infiniband NIC to be accessed directly from user-space applications (kernel bypass). In order to use the infrc transport, you will need to install Infiniband NICs and switches, such as those from Mellanox. The infrc transport uses UDP to exchange information during connection setup; the host and port parameters in the service locator identify the server's node and the port on which it is listening for UDP packets to open new connections.Once a connection is open, UDP is no longer used; all communication happens directly using Infiniband. Note: the infrc transport was our workhorse transport for many years, but we now recommand the basic transport instead. Basic is about as fast as infrc, but it is more robust and stable.