Installation: Linux

Owned by Anne Wright
Last updated: Jun 05, 2015 by Chris BartleyVersion comment
6 min read

These instructions cover installing Fluxtream, datastore, and Tomcat as of July 2, 2013 on an Ubuntu 12.04 instance.  This should be sufficient to run a local instance, but does not cover running under apache.  Hopefully this record will at least be helpful in getting started.  These instructions have not yet been fully tested.  If you hit rough spots, please feel free to contact us at info@fluxtream.org.

These instructions are relevant to Linux, specifically Debian and variants thereof such as Ubuntu, and contain certain steps which may be specific to Ubuntu 12.04 and require some tuning for other distributions.  For other platforms, see the corresponding installation instructions:

I arbitrarily chose to use the following paths and the notes below reflect these choices.  Feel free to make other selections and adjust the execution lines for your setup accordingly:

  • fluxtream-app and datastore are checked out into ~/projects
  • the key-value store for the datastore in ~/projects/db
  • tomcat installed in ~/fluxtream-server
  • downloaded packages put in ~/install

Before starting, make sure you have a working installation of mysql and that you have credentials for it.

Setup global environment

Add to /etc/bash.bashrc

export TARGET_ENV=local
export JPDA_TRANSPORT=dt_socket
export JPDA_ADDRESS=9000

Start new bash shell to get the environment variables Tomcat will need.  Double check that the variables are set:

typing echo $JPDA_ADDRESS should show 9000

Make sure you have add-apt-repository

The improved setup instructions below, which were generously provided by QuAnTiMoDo, require an executable called add-apt-repository which is not installed by default on Ubuntu 12.04.  I found instructions here which suggested the following install commands:

For Ubuntu <= 12.04

sudo apt-get install python-software-properties

For Ubuntu >= 12.10

sudo apt-get install software-properties-common

I can confirm that the former command worked on a fresh Ubuntu 12.04 install, but don't have personal experience with newer distributions.  YMMV.

You can go on to the next section once 'which add-apt-repository' succeeds.

Install required packages

  sudo add-apt-repository -y ppa:webupd8team/java
  sudo apt-get update
  sudo apt-get install -y make build-essential git-core maven oracle-java7-installer tomcat7 tomcat7-user
  sudo update-java-alternatives -s java-7-oracle

At the end of this step, the commands should succeed:

 git --version
 mvn --version
 java -version
 javac -version
 g++ --version
 make --version 

At the time of this writing, the versions returned were:

  git = 1.7.9.5
  mvn = 3.0.4 (using java 1.7.0_25)
  java/javac =  1.7.0_25
  g++ = 4.6.3
  make = 3.81 

If you use a system other than Ubuntu, you will need to figure out how to install git, maven, and java on your own.  Please submit these back to us at info@fluxtream.orgo.

Install Tomcat and set up permissions

The following command, also provided by QuAnTiMoDo, creates an instance of the Tomcat server which is local to the currently logged in user.  Later sections of the instructions assume that the name of this user is 'fluxtream'.  If you want to set things up differently just keep in mind which user you were when you got to here and modify later usernames accordingly:

  tomcat7-instance-create ~/fluxtream-server

Checkout, set up, and build the datastore

 mkdir ~/projects
 cd ~/projects
 git clone https://github.com/BodyTrack/datastore.git
 cd datastore
 make test

Create a directory for the key-value store the datastore will use.  This directory will get big, so the default location here may not be appropriate.  If you use an alternate location, make sure to modify the btdatastore.db.location property in  local.properties accordingly.

 mkdir ~/projects/db
 mkdir ~/projects/db/dev.kvs

Make sure that the permissions on that directory is such that tomcat will be able to write to it.

Checkout and set up fluxtream-app

 cd ~/projects/
 git clone git://github.com/fluxtream/fluxtream-app.git fluxtream-app 

Copy MaxMind's GeoLiteCity.dat database to ~/projects/fluxtream-app:

 cd ~/projects/fluxtream-app
 wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
 gunzip GeoLiteCity.dat.gz

Checkout and set up scribe-java

 cd ~/projects
 git clone https://github.com/fluxtream/scribe-java.git
 cd scribe-java/
 mvn install

Setup properties files

Copy the sample properties files to their real location:

 cd ~/projects/fluxtream-app/fluxtream-web/src/main/resources/
 cp samples/*.properties .

Edit the values in local.properties to customize your Fluxtream install to match your situation.  If you leave the default values alone you will end up with:

  • A mysql port of 3306, username of flx with mysql password fluxtream: edit db.url, db.username, and db.password to change.
  • The datastore and GeoLiteCity.dat being assumed to be under /home/fluxtream/projects: edit btdatastore.exec.location, btdatastore.db.location, and geoIpDb.location to change.
  • The URL assumed for running your local Tomcat dev instance of http://localhost:8080/: edit homeBaseUrl to change.
  • A white-label welcome page: edit homepage.name to change.  If you set the value of homepage.name to foo, the Fluxtream server will look for the file fluxtream-web/src/main/webapp/WEB-INF/jsp/foo.jsp. 
  • A default location of Pittsburgh (only affects running on a local dev instance): edit defaultLocation.latitude and defaultLocation.longitude to change.

Edit the crypto key in common.properties to some long and hard to guess phrase.  If left at the default value you will end up with a pitiful encryption key for encrypting the values in the ApiKeyAttribute table.  IMPORTANT: if you ever change or misplace the value of this key used when populating the attributes in a given ApiKeyAttribute table you will lose the ability to update any of the user's connectors in that instance.  If you copy a Fluxtream DB from one server to another, be sure to set up the crypto key with the same value, or else clear out all the connector-related tables.

Work through the other keys in common.properties and oauth.properties to replace instances of xxx with appropriate values.  See Setup and Maintenance of Fluxtream Server Instances for more details.

Build the fluxtream-app project

 cd ~/projects/fluxtream-app
 mvn install

Create database and import cities1000.sql

Versions of Fluxtream more recent than 0.9.0002 manage database versioning manually.  Prior to that we allowed the JPA auto update mechanism to manage the schema, but this became untenable once multiple developers were working on branches which differed in the way they interacted with the DB.  The setting which controls this behavior is the hibernate.hbm2ddl.auto key in fluxtream-app/fluxtream-web/src/main/webapp/WEB-INF/applicationContext.xml.  In recent versions, this is set to 'none', which means that JPA will not try to make modifications to the schema.  As a result, it is necessary to manually apply .sql files to manage DB state ourselves.

The sql files which are used in this management live in subdirs of the fluxtream-app/fluxtream-web/db directory.  The name of the subdir corresponds to the release number.  There are generally two types of files, distinguishable by name.

  • schema-after-<version>.sql files are a full dump of the schema once a given version was finalized for deployment.  
  • Other .sql files are incremental changes for moving from the previous version

The first time you set up, what you want to do is create a database called flx and initialize it with the latest version of the schema-after files available.  The following command will show which is the latest file to use for the import:

 cd ~/projects/fluxtream-app/fluxtream-web/db
 find . -name 'schema-after*' | sort | tail -n 1

Once you have found the appropriate schema-after filename, these commands will create the db.  Be sure to modify with the appropriate mysql username after the -u flags.  The following lines assume a mysql user named 'flx'.

 echo 'create database flx;' | mysql -u flx -p 
 mysql -u flx p flx < ./<version>/schema-after-<version>.sql

Now you're ready to import cities1000.  This is needed for converting lat/lon locations into city names and timezones.  You will get errors if this step is skipped.

 mysql -u root flx < ~/projects/fluxtream-app/cities1000.sql

Setup Tomcat

Setup an environment variable TOMCAT_HOME  to point to your Tomcat installation.  If you're using these directions as user fluxtream with homedir /home/fluxtream this would be:

 export TOMCAT_HOME=/home/fluxtream/fluxtream-server

Setup Tomcat to point to fluxtream-app for webapps

 cd ${TOMCAT_HOME}/webapps/
 ln -s ~/projects/fluxtream-app/fluxtream-web/target/ROOT

Make sure the permissions for everything Tomcat will try to get to are appropriate.   If using Apache, this would be the www-data group.  Modify below for other user configurations.

 cd /opt/tomcat/home/webapps/ROOT
 sudo chgrp -R www-data *

Start Tomcat, check logs

Add the following lines to the top of ${TOMCAT_HOME}/bin/startup.sh:

export TARGET_ENV='local'
export JPDA_ADDRESS=9000
export JPDA_TRANSPORT=dt_socket
export JAVA_OPTS="-XX:MaxPermSize=256m -Xms256m -Xmx2048m -Djavax.servlet.request.encoding=UTF-8 -Dfile.encoding=UTF-8 $JAVA_OPTS"

Make sure to set JAVA_OPTS, as the server typically won't start up with the default values (if I remember right it will not allocate enough memory).  The two JPDA settings are to support remote debugging on port 9000.  

Start the server:

 cd ${TOMCAT_HOME}
 ./bin/startup.sh

NOTE: Follow the above directions precisely.  It won't work if you cd into the bin directory and run ./startup.sh start from there.  Furthermore, make sure you cd to the TOMCAT_HOME directory before starting the server, rather than starting it with a command using an absolute path.  The Log4J configuration file references its output file (all.log) using a relative path, and it's relative to the directory you're in when you start the server.  So, normally, if you cd to TOMCAT_HOME before starting the server, the Log4J log file will be at ${TOMCAT_HOME}/logs/all.log.  However, if you instead start the server using the absolute path command shown above while in, say, your user home directory, then the log file will be at ~/logs/all.log.

Watch the logs for startup messages:

 tail -f ${TOMCAT_HOME}/logs/catalina.out

and

 tail -f ${TOMCAT_HOME}/logs/all.out

The catalina.out log is the most useful to watch during startup.  When server initialization is done the log will say something like:

 INFO: Server startup in xxx ms

If it has issues during startup, you'll see them in that log.  Contact us at info@fluxtream.org if you need help getting past such issues.

Once startup completes, it's most useful to watch all.out.  That is where the Log4J messages go.  However, anything printed using System.out will only go to catalina.out.

Test that Tomcat is working

Browse to http://localhost:8080

This should bring up the fluxtream login screen, as you would see at https://sandbox.fluxtream.com

Register for an account (http://localhost:8080/register), try things out, it should all be good.

Next steps

Instructions on setting up an HTTP server such as Apache to talk to tomcat is outside the scope of this page.  There is an example of one way to do it in the Amazon EC2 setup example in case it helps.

We have open source licenses for JRebel and IntelliJ IDEA which have the potential to greatly improve the developer experience.  Docs for that part of the setup are not done yet, though there is a start on that effort at Development and debugging with JRebel and IntelliJ IDEA.  

Please contact us at info@fluxtream.com if you would like further assistance on those fronts.