This document shows you how to configure a complete cluster using a virtual machine. This setup replicates a real cluster in your development machine as closely as possible. After you complete the steps outlined here, you will be ready to create and run your first Aurora job.
The following sections describe these steps in detail:
The Aurora distribution includes a set of scripts that enable you to create a local cluster in your development machine. These scripts use Vagrant and VirtualBox to run and configure a virtual machine. Once the virtual machine is running, the scripts install and initialize Aurora and any required components to create the local cluster.
First, download and install VirtualBox on your development machine.
Then download and install Vagrant. To verify that the installation
was successful, open a terminal window and type the vagrant command. You should see a list of
common commands for this tool.
To obtain the Aurora source distribution, clone its Git repository using the following command:
git clone git://git.apache.org/aurora.git
Now change into the aurora/ directory, which contains the Aurora source code and
other scripts and tools:
cd aurora/
To start the local cluster, type the following command:
vagrant up
This command uses the configuration scripts in the Aurora distribution to:
This process takes several minutes to complete.
You may notice a warning that guest additions in the VM don’t match your version of VirtualBox. This should generally be harmless, but you may wish to install a vagrant plugin to take care of mismatches like this for you:
vagrant plugin install vagrant-vbguest
With this plugin installed, whenever you vagrant up the plugin will upgrade the guest additions
for you when a version mis-match is detected. You can read more about the plugin
here.
To verify that Aurora is running on the cluster, visit the following URLs:
To SSH into the VM, run the following command in your development machine:
vagrant ssh
To verify that Aurora is installed in the VM, type the aurora command. You should see a list
of arguments and possible commands.
The /vagrant directory on the VM is mapped to the aurora/ local directory
from which you started the cluster. You can edit files inside this directory in your development
machine and access them from the VM under /vagrant.
A pre-installed clusters.json file refers to your local cluster as devcluster, which you
will use in client commands.
Now that your cluster is up and running, you are ready to define and run your first job in Aurora. For more information, see the Aurora Tutorial.
If you are changing Aurora code and would like to rebuild a component, you can use the aurorabuild
command on the VM to build and restart a component. This is considerably faster than destroying
and rebuilding your VM.
aurorabuild accepts a list of components to build and update. To get a list of supported
components, invoke the aurorabuild command with no arguments:
vagrant ssh -c 'aurorabuild client'
To shut down your local cluster, run the vagrant halt command in your development machine. To
start it again, run the vagrant up command.
Once you are finished with your local cluster, or if you would otherwise like to start from scratch,
you can use the command vagrant destroy to turn off and delete the virtual file system.
Most of the Vagrant related problems can be fixed by the following steps:
vagrant destroyvirtualbox UI or VBoxManage command line toolgit clean -fdxvagrant upIf that still doesn’t solve your problem, make sure to inspect the log files:
/var/log/upstart/aurora-scheduler.log/var/log/upstart/aurora-thermos-observer.log/var/log/mesos/mesos-master.INFO (also see .WARNING and .ERROR)/var/log/mesos/mesos-slave.INFO (also see .WARNING and .ERROR)