Versions Compared

Old Version 51

changes.mady.by.user Ryan Stutsman

Saved on

compared with

New Version Current

changes.mady.by.user Stephen Yang

Saved on

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

...

...

...

The code base for this project is under heavy and recent development. Please expect nothing from it for the time being; there is not yet a version that we consider "stable".

Bugs

Bugs should be filed in our bug tracker. Questions and comments can be sent to the developer mailing list: ramcloud-dev. (As of 2019/10/7 support for RAMCloud has ended)

Necessary Tools

  • GNU Make (Anything reasonably recent)
  • GNU g++ (>= 4.49.6x)
  • git (>= 1.6.0)
  • Perl (Anything reasonably recent)
    • For mergedeps.pl, which automatically inserts included headers in source files into the make dependencies.
  • Python 2.6, epydoc 
  • Boost
    • If you're having issues with Boost on Ubuntu, check boost ticket #3844.
  • pcre
  • Doxygen 1.7.21
  • protocol buffers
    • If you're getting lots of undefined reference errors during linking, it's likely that your libprotobuf is compiled with a different library ABI than RAMCloud. Check GCC's Dual ABI page and the "GLIBCXX_USE_CXX11_ABI" flag in GNUMakefile.
  • ZooKeeper
  • java and javac (>= 1.7.0_25)

To get a working toolchain on Debian (and probably Ubuntu), this used to be sufficient:

aptitude install build-essential git-core doxygen libboost1.42-all-dev libpcre3-dev protobuf-compiler libprotobuf-dev libcrypto++-dev libevent-dev

To get a working toolchain on SUSE Linux Enterprise or openSUSE perform the following steps:

zypper rm libevent-1_4-2 *boost*
zypper install mpfr-devel glibc-devel-32bit doxygen scons libzip-devel libbz2-devel pcre-devel *boost*1_44*
Install from sources: libevent-2.0.20-stable, protobuf-2.4.1, cryptopp560, gcc-4.4.6

Here are some toolchain installation commands that have worked for other members of the community on different systems:

OSCommand
Ubuntu 15.04 /
Debian 8.2 
apt-get install build-essential git-core doxygen=1.7.1 libpcre3-dev protobuf-compiler libprotobuf-dev libcrypto++-dev libevent-dev libboost-all-dev libgtest-dev libzookeeper-mt-dev zookeeper libssl-dev

Source Code

For

...

Core Developers

For those with r+w access to the repository on GitHub, which currently hosts our git repository, clone from here for write access:

No Format
git clone gitgit@github.com://fiz.stanford.edu/git/ramcloudPlatformLab/RAMCloud.git
cd ramcloudRAMCloud
git submodule update --init --recursive
make
logcabin
ln -s ../../hooks/pre-commit .git/hooks/pre-commit

(Notes: the git submodule update retrieves additional repos needed to build RAMCloud, such as gtest and logcabin; the ln -s command arranges for various consistency checks to run during commits, such as ensuring the absence of carriage returns)

Don't worry about having read-only access; it really shouldn't impede you. We'd prefer to pull changes from you or have you send patches (see git-format-patch). This lets us ensure that the fiz repo is generally in a working state and is making forward progress.

For Core Developers

For those with fiz accounts clone from here for write access:

...

For Read-Only Access

Use the same instructions as above, except replace "git@github.com:PlatformLab/RAMCloud.git" with "https://github.com/PlatformLab/RAMCloud.git" in the git clone command. With this approach you don't need to have r+w access to the repository on GitHub, and you can't push commits to the repository. With read-only access, you can send us patches, or we can pull them from your repository.

Compiling

No Format
cd ramcloudRAMCloud   
make

LogCabin

LogCabin is a distributed system that will provide a small amount of highly replicated, consistent storage. It will be used by the RAMCloud Coordinator mainly for fault tolerance. 

More information about LogCabin can be found at: https://ramcloud.stanford.edu/wiki/display/logcabin/LogCabin.

For current users of RAMCloud who have just pulled the latest commits that link RAMCloud to LogCabin, or if the LogCabin submodule is updated at any point, run:

No Format
git submodule update --init --recursive
make logcabin -j12

Object files and executables will appear in the subdirectory obj.xxx, where xxx is your current branch (typically master). By default, RAMCloud compiles with debugging symbols, which slows the system down considerably.  If you want to make performance measurements, recompile without debugging symbols:

No Format
make clean
make -j12 DEBUG=no

CentOS Issues

This shouldn't be necessary any more on the Stanford cluster, as /etc/skel/.bashrc now includes the line. -Diego 2012-04-15

...

  • No commit should include changes for multiple issues. (Each commit in the push should complete or work towards a single issue.)
  • Pushes should complete any issues they start, but individual commits within a push do not need to complete an issue.
  • If at all possible, each commit should represent a version of the system that actually works (e.g., passes all unit tests and runs simple apps and system tests). This is important so that people can use git bisect to hunt down the commit that caused a bug: if commits are broken, git bisect won't work.
  • All commit messages should be meaningful without looking at the diff. (By reading a commit message, you should get some idea of what was modified and what the intent was. If you can't fit this in one line, skip a line and then write more.)

Speed Up Build

Put the following lines in the file `private/MakefragPrivateTop`:

No Format
# Use ccache to speed up compilation by caching object
# files across compilations.
CCACHE := yes
# Use gold linker for faster linking.
LINKER := gold

Running Under Valgrind

...

Compiling unit test:

No Format
make VALGRIND=yes test 

Running unit test:  The 'master' part in 'obj.master' could be replaced your git branch name.

No Format
 valgrind --leak-check=yes obj.master/test

Running system test with valgrind: Use 'scripts/cluster.py', which posts RAMCloud tasks into the server cluster, with '–valgrind'. Use '–valgrindArgs' to pass the options to valgrind:

No Format
  scripts/cluster.py --verbose --timeout=600 --valgrind --valgrindArgs='--leak-check=yes --leak-resolution=med --show-reachable=no --undef-value-errors=no' --client=RAM-462/ram462test

Running With Google's Sanitizers

There are currently three Sanitizers available in the build option: AddressSanitizer (ASan), ThreadSanitizer (TSan) and UndefinedBehaviorSanitizer (UBSan). They can be enabled with the SANITIZER symbol. Different Sanitizers are not consistent with each other, so there can be only one Sanitizer enabled at a time.

Building RAMCloud and unit tests with ASan: Similarly, replace 'address' with 'thread' or 'undefined' to enable TSan or UBSan respectively.

No Format
make SANITIZER=address tests

Running unit tests and system tests are the same as usual once RAMCloud and the respective test files are compiled with the Sanitizer.

For more information (e.g., the kinds of bugs each Sanitizer can find, bug report format, additional runtime options, etc.), see: https://github.com/google/sanitizers/wiki

Additional Sanity Checks

Before pushing to the master branch on fiz it is generally a good idea to make sure your changes haven't caused regressions in the recovery system.

...

  • recovery.py runs a small but complete recovery. See Running Recoveries with recovery.py for details on testing more complicated recoveries with this tool.

  • run.py runs a variety of messed up failure scenarios that try to exercise the system. As the system is able to tolerate more scenarios they will be added to this test suite.