Installation

Let's get Eskapade up and running! In order to make this as easy as possible, we provide both a Docker image and a virtual machine where everything you need is installed and working properly. Alternatively, you can download the repository and run it on your own machine.

This manual is written for Linux systems, but Eskapade also runs fine on MacOS systems.

Eskapade with Docker

There is a Docker image available to try out Eskapade without concerns about the installation process of all necessary packages. The instructions below show how one can use a locally checked-out version of Eskapade in combination with this Docker image.

This combination is a very convenient way of actively working with and/or developing code in Eskapade.

Installing Eskapade

Make sure you have installed the latest version of the public Eskapade repository, with the command:

git clone git@github.com:KaveIO/Eskapade.git eskapade

Installing Docker

Docker is a great tool to develop and automate the deployment of applications inside software containers. To install Docker, go docker.com and follow the installation instructions.

Pulling in the Eskapade environment

To pull in the environment for running Eskapade (but excluding the Eskapade repository itself), type in a shell:

docker pull kave/eskapade-env:0.5

Downloading this docker image can take a minute or two.

Running Eskapade with Docker

To start up the Eskapade docker environment, with access to the Eskapade repository, do:

docker run -it --name es-ktb -p 8888:8888 -v /path/to/your/local/eskapade:/opt/eskapade kave/eskapade-env:0.5 bash

This command will start up a bash shell in the docker kave/eskapade-env:0.5 image, and opens port 8888. The Eskapade setup file will be automatically sourced.

The option -v /path/to/your/local/eskapade:/opt/eskapade mounts the local directory /path/to/your/local/eskapade (containing your local Eskapade repository) under /opt/eskapade in the docker container. You can now edit the files in this directory, either locally or in the (docker) bash shell, and any updates to these files will be kept after exiting docker.

This combination is a great way of using and developing Eskapade code.

E.g. one can now do:

cd /opt/eskapade

and run any Eskapade code. See Tutorial section for examples.

Exit the (docker) bash shell with:

exit

See section After you exit Docker (right below) for cleaning up obsolete docker processes.

Consider adding a permanent alias to your local ~/.bashrc or ~/.bash_profile file:

alias eskapade_docker='docker run -it --name es-ktb -p 8888:8888 -v /path/to/local/eskapade:/opt/eskapade kave/eskapade-env:0.5 bash'

So the next time, in a fresh shell, you can simply run the command eskapade_docker.

Starting Jupyter notebook

To run the Jupyter notebook on port 8888 from the docker environment:

cd /opt/eskapade
jupy &

And press enter twice to return to the shell prompt.

The command jupy & starts up Jupyter notebook in the background on port 8888 and pipes the output to the log file nohup.out.

In your local browser then go to address:

localhost:8888/

And you will see the familiar Jupyter environment.

E.g. you can now do import eskapade (shift-enter) to get access to the Eskapade library.

Be sure to run jupy & from a directory that is mounted in the docker container, such as /opt/eskapade. In this way any notebook(s) you create are kept after you exit the docker run.

After you exit Docker

Everytime you want to have a clean Docker environment, run the following commands:

# --- 1. remove all exited docker processes
docker ps -a | grep Exited | awk '{print "docker stop "$1 "; docker rm "$1}' | sh

# --- 2. remove all failed docker image builts
docker images | grep "<none>" | awk '{print "docker rmi "$3}' | sh

# --- 3. remove dangling volume mounts
docker volume ls -qf dangling=true | awk '{print "docker volume rm "$1}' | sh

To automate this, we advise you put these commands in an executable docker_cleanup.sh script.

Eskapade on a virtual machine

Consistent environments for Eskapade development and use are created with virtual machines. Machines are created and managed by Vagrant, with VirtualBox as a provider.

Getting started

Required software

To build and run Eskapade boxes, Vagrant and VirtualBox are required. Vagrant can be downloaded at https://www.vagrantup.com/downloads.html. For Ubuntu (and other Debian-based systems), download the Vagrant Debian package, move to where you want to install this and run this command to install:

dpkg -i vagrant_<VERSION>_<ARCHITECTURE>.deb

where the version and architecture are in the file name.

VirtualBox can be downloaded and installed by following the instructions on https://www.virtualbox.org/wiki/Downloads. Make sure you install the latest version from the VirtualBox repository instead of an older version from the repositories of your system.

Repository

The repository, which contains the code to build your virtual environment, is hosted on github. Clone it to your machine with:

$ git clone git@github.com:KaveIO/Eskapade.git

All code below is executed in the root of this repository, unless otherwise stated.

Virtual environment

The environment box contains Ubuntu and the KAVE Toolbox, including Anaconda, Spark, and ROOT.

You can either build this box yourself, or download the image. In order to build it yourself, run the following commands. It might take a while to build (up to two hours!)

cd vagrant/dev
vagrant up

Alternatively, in order to download and import its image, do:

cd vagrant/dev_image
vagrant box add eskapade-dev.json
vagrant up

Eskapade users and developers log in as the user esdev on localhost, by default on port 2222:

ssh -p 2222 esdev@localhost

This user can log in with the password esdev.

You are now ready to use Eskapade!

The next time...

Simply do:

cd vagrant/dev
vagrant up

or:

cd vagrant/dev_image
vagrant up

depending on whether you built vagrant yourself or downloaded the image.

Then you can access it via ssh (password esdev):

ssh -p 2222 esdev@localhost

Easy log-in

To make logging in easier, the key pair vagrant/dev/ssh/esdev_id_rsa.pub, vagrant/dev/ssh/esdev_id_rsa can be used, and an example SSH configuration is provided in vagrant/dev/ssh/config. Put these files in your ~/.ssh/:

cp vagrant/dev/ssh/* ~/.ssh/

You can then log in using the command:

ssh esdevbox

Vagrant boxes

Boxes are built and started with the command vagrant up in the directory of the Vagrantfile describing the box. A box can be restarted by executing vagrant reload. The virtual machines are administered by the vagrant user, which logs in by running vagrant ssh in the directory of the Vagrantfile. The vagrant user has root access to the system by password-less sudo.

Eskapade on your own machine

The repository is hosted on github, clone it to your machine with:

$ git clone git@github.com:KaveIO/Eskapade.git

Requirements

Eskapade requires Anaconda, which can be found here. Eskapade was tested with version 4.3.

It also uses some non-standard libraries, which can be found in requirements.txt in the repository. These can be installed by doing:

$ pip install -r requirements.txt

You are now ready to use Eskapade!

After installation

To get started, source Eskapade in the root of the repository:

$ source setup.sh

You can now call the path of Eskapade with:

$ echo $ESKAPADE

or in python with

import os
os.environ['ESKAPADE']

See the readme files in other parts of the repository for specific usage.

Installing Eskapade on MacOS

To install eskapade on MacOS, see our MacOS appendix.