Skip to content

Home

Jump to bottom
Jude Nelson edited this page Aug 18, 2014 · 12 revisions

Welcome to the Syndicate wiki. These pages are meant to provide an overview of how to use Syndicate, and how it is put together.

Overview

Syndicate organizes your data into one or more Volumes. A Volume is a logical collection of data organized into a filesystem. Syndicate's components work together to implement your Volumes on top of your cloud storage and caches.

There are four major components of Syndicate: the Metadata Service (MS), the User Gateway (UG), the Replica Gateway (RG), and the Acquisition Gateway (AG):

  • The MS is an always-on service that Gateways use to coordinate--in many ways, it's similar to a private BitTorrent tracker.
  • The UGs are end-points on your clients that cache and exchange Volume data and metadata on your behalf, and give you read/write interfaces to your data. For example, there is a FUSE UG, and a Web store UG.
  • RGs are proxies that take data from UGs and back it up to your storage, so other UGs can get to it.
  • AGs are proxies that expose external, existing datasets as read-only directory hierarchies in your Volumes.

At a minimum, you need to deploy the MS, one UG, and one RG. That will get you Dropbox-like functionality.

Getting Started

  1. Set up or find an MS to use. If you have an account at PlanetLab or VICCI, you can use the one at https://syndicate-metadata.appspot.com. To get an account on this MS, send an email to syndicate [AT] lists [dot] NOSPAM [dot] cs [dot] princeton [dot] edu.
  2. Set up and learn how to use syntool.
  3. Create a Volume.
  4. Create a User Gateway to access your Volume.
  5. Create a Replica Gateway to keep your Volume's data backed in the cloud.
  6. Create an Acquisition Gateway to expose existing data as a Volume.

Package Repositories

We currently have Fedora 16/17 packages and Ubuntu 12.04 packages available. They should be forwards-compatible with future releases, for the time being (let us know if they aren't!).

Ubuntu 12.04

You can get our Ubuntu GPG key here. Add it easily with the following command:

$ wget -qO - http://www.cs.princeton.edu/~jcnelson/ubuntu-12.04/syndicate-nightly/public.gpg | sudo apt-key add -

Then, add the following to your /etc/apt/sources.list file:

deb http://www.cs.princeton.edu/~jcnelson/syndicate-nightly/ubuntu-12.04/x86_64 ./

Finally, run apt-get update, and install Syndicate with:

sudo apt-get install syndicate-ug syndicate-rg syndicate-ag syndicate-ms-clients

Fedora 16/17

You can get our Fedora GPG key here. You should put it at /etc/pki/rpm-gpg/RPM-GPG-KEY-Syndicate, and add the following as a repository in /etc/yum.repos.d/Syndicate.repo:

[Syndicate]
name=Syndicate nightly builds
baseurl=http://www.cs.princeton.edu/~jcnelson/syndicate-nightly/fedora-16/RPMS/
gpgcheck=1
gpgkey=file:///etc/pki/rpm-gpg/RPM-GPG-KEY-Syndicate
enabled=1

Then to install Syndicate components, run:

$ sudo yum install syndicate-ug syndicate-rg syndicate-ag syndicate-ms-clients

How-To

Clone this wiki locally