Installation: Mac OS X

Owned by Evan Savage
Last updated: Jun 05, 2015 by Chris BartleyVersion comment
7 min read

This guide walks through the process of setting up a development environment for Fluxtream on Mac OS X.

Prerequisites

Java

You've already got it:

$ java -version
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-11M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)

Homebrew

We'll start with the excellent package manager Homebrew:

$ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
$ brew doctor
$ brew update

Git

$ brew install git

Maven

$ brew install maven

MySQL

This is a bit more complicated than a simple homebrew incantation, but not by much. We'll follow the directions here.  

Install MySQL

$ brew install mysql
$ which mysql
mysql is hashed (/usr/local/bin/mysql)
$ mysql --version
5.6.13 (or similar.  Copy this to use for <version> below)

Set up MySQL as a service to be run automatically on startup

$ mkdir -p ~/Library/LaunchAgents
$ cp /usr/local/Cellar/mysql/<version>/homebrew.mxcl.mysql.plist ~/Library/LaunchAgents/
$ launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist

Set up the metadata tables required by MySQL

$ unset TMPDIR
$ mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp
$ /usr/local/Cellar/mysql/<version>/bin/mysqladmin -u root password <password>

Start MySQL

$ mysql.server start

It should also work to use other mysqld packages such as MAMP.  Either way, just make sure later on that local.properties is set up appropriately for the host, port, user, and password settings used by your mysql installation.

Node.js and less

The Fluxtream build process uses node.js to run a less compiler to build .css files from .less sources.  The following is from the MacOS homebrew instructions at https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager:

  • sudo brew install node 
  • sudo npm install -g less

Repos

Fork

Log into Github and fork both the datastore and Fluxtream repos:

Be sure that the github ssh keys are set up properly on the machine you are using.  A guide for how to set this up is here.  

Clone

$ mkdir -p ~/projects
$ cd ~/projects
$ git clone git@github.com:<username>/datastore.git
$ git clone git@github.com:<username>/fluxtream-app.git

datastore

Build

$ cd ~/projects/datastore
$ make test

If it won't build, you might need to install Xcode, then within Xcode go to Preferences -> Downloads and install "Command Line Tools", then restart Terminal and try the make step again.

Data Directory

This directory will be used by datastore to store channel data for the BodyTrack app:

$ mkdir -p ~/projects/db/dev.kvs

Create database and import cities1000.sql

Versions of Fluxtream more recent than 0.9.0002 manage database versioning manually.  Prior to that we allowed the JPA auto update mechanism to manage the schema, but this became untenable once multiple developers were working on branches which differed in the way they interacted with the DB.  The setting which controls this behavior is the hibernate.hbm2ddl.auto key in fluxtream-app/fluxtream-web/src/main/webapp/WEB-INF/applicationContext.xml.  In recent versions, this is set to 'none', which means that JPA will not try to make modifications to the schema.  As a result, it is necessary to manually apply .sql files to manage DB state ourselves.

The sql files which are used in this management live in subdirs of the fluxtream-app/fluxtream-web/db directory.  The name of the subdir corresponds to the release number.  There are generally two types of files, distinguishable by name.

  • schema-after-<version>.sql files are a full dump of the schema once a given version was finalized for deployment.  
  • Other .sql files are incremental changes for moving from the previous version

The first time you set up, what you want to do is create a database called flx and initialize it with the latest version of the schema-after files available.  The following command will show which is the latest file to use for the import:

 cd ~/projects/fluxtream-app/fluxtream-web/db
 find . -name 'schema-after*' | sort | tail -n 1

Once you have found the appropriate schema-after filename, these commands will create the db.  Be sure to modify with the appropriate mysql username after the -u flags.  The following lines assume a mysql user named 'root'.

 echo 'create database flx;' | mysql -u root -p 
 mysql -u root -p flx < ./<version>/schema-after-<version>.sql

Now you're ready to import cities1000.  This is needed for converting lat/lon locations into city names and timezones.  You will get errors if this step is skipped.

 mysql -u root -p flx < ~/projects/fluxtream-app/cities1000.sql

GeoLiteCity DB

$ cd ~/projects/fluxtream-app
$ curl -O http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
$ gunzip GeoLiteCity.dat.gz

Setup properties files

Copy the sample properties files to their real location:

 cd ~/projects/fluxtream-app/fluxtream-web/src/main/resources/
 cp samples/*.properties .

Edit the values in local.properties to customize your Fluxtream install to match your situation.  If you leave the default values alone you will end up with:

  • A mysql port of 3306, username of flx with mysql password fluxtream:edit db.url, db.username, and db.password to change.
  • The datastore and GeoLiteCity.dat being assumed to be under/home/fluxtream/projects: edit btdatastore.exec.location, btdatastore.db.location, and geoIpDb.location to change.
  • The URL assumed for running your local Tomcat dev instance of http://localhost:8080/: edit homeBaseUrl to change.
  • A white-label welcome page: edit homepage.name to change.  If you set the value ofhomepage.name to foo, the Fluxtream server will look for the file fluxtream-web/src/main/webapp/WEB-INF/jsp/foo.jsp. 
  • A default location of Pittsburgh (only affects running on a local dev instance): edit defaultLocation.latitude and defaultLocation.longitude to change.

Edit the crypto key in common.properties to some long and hard to guess phrase.  If left at the default value you will end up with a pitiful encryption key for encrypting the values in the ApiKeyAttribute table.  IMPORTANT: if you ever change or misplace the value of this key used when populating the attributes in a given ApiKeyAttribute table you will lose the ability to update any of the user's connectors in that instance.  If you copy a Fluxtream DB from one server to another, be sure to set up the crypto key with the same value, or else clear out all the connector-related tables.

Work through the other keys in common.properties and oauth.properties to replace instances of xxx with appropriate values.  See Setup and Maintenance of Fluxtream Server Instances for more details.

Checkout, and set up scribe-java

 cd ~/projects
 git clone https://github.com/fluxtream/scribe-java.git
 cd scribe-java/
 mvn install

Build

You should be able to build Fluxtream now:

$ cd ~/projects/fluxtream-app
$ mvn install

If it works, congratulations! You're super-close to being development-ready. If not, check for any missed steps or ping info@fluxtream.org.

Tomcat

To actually test Fluxtream out, you'll need a webserver to run it on. This part will explain how to set up Tomcat.

Install

$ brew install tomcat

Setup environment

Add the following lines to your ~/.profile (or to ~/.bashrc if that is sourced from your ~/.profile):

export TARGET_ENV=local
export JPDA_TRANSPORT=dt_socket
export JPDA_ADDRESS=9000
export TOMCAT_HOME=/usr/local/Cellar/tomcat/<version>/libexec

Start a new shell and make sure that the settings took effect

Link To Fluxtream

Tomcat ships with a default webapp. To make the server a bit more useful, let's set this default webapp aside and link to fluxtream-app:

$ cd ${TOMCAT_HOME}/webapps/
$ sudo mkdir ../hold
$ sudo mv ROOT examples docs ../hold
$ sudo ln -s ~/projects/fluxtream-app/fluxtream-web/target/ROOT

Modify Tomcat JAVA_OPTS

We need to change the JAVA_OPTS used by Tomcat or else it will give out of memory errors:

cd ${TOMCAT_HOME}/bin

Add the following just underneath the long comment at the top of catalina.sh:

JAVA_OPTS="-server -Xms2048m -Xmx2048m -XX:PermSize=256m -XX:MaxPermSize=256m"

Start

cd ${TOMCAT_HOME}
./bin/catalina.sh start

Test

Load http://localhost:8080 in your browser to test out Fluxtream. If that doesn't work, check the logs:

cd ${TOMCAT_HOME}
tail -F logs/catalina.out
tail -F logs/all.log

Development Environment

While you can develop for Fluxtream using vim and raw Maven incantations, that way lies madness. This part will explain how to setup several tools for easier, faster development.

IntelliJ IDEA

We recommend using IntelliJ IDEA for writing code. You can download IntelliJ IDEA at: http://www.jetbrains.com/idea/download/index.html

Install the disk image and enter your IntelliJ IDEA license key to continue. Once that's entered, you can use File > Open Project... to import ~/projects/fluxtream-app as an IntelliJ project. It should automatically detect that Spring and Maven are being used.

When you import Fluxtream, you'll probably see red squiggly lines everywhere. That's because you haven't set up a JDK version yet. Go to File > Project Structure... > SDKs and enter the path to your JDK in the "JSDK home path" field. It should be something like

/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home

JRebel

JRebel allows you to instantly view your code changes in the browser–even Java code!  Follow the directions to install the IntelliJ IDEA JRebel plugin: http://zeroturnaround.com/software/jrebel/intellij-idea-jrebel-tutorial-formerly-javarebel/

ZeroTurnaround allows open source developers to apply for an open source license.  If your uses of Fluxtream are eligible for this, you can sign up for an account at https://my.jrebel.com and apply for an OSS license.  

Once that process is complete, go to Preferences > JRebel > Activation and upload the JRebel license file in the "Register license file" field.

JRebel Tomcat Integration

First, download a standalone version of the JRebel JAR from http://zeroturnaround.com/software/jrebel/download/prev-releases/ and put it somewhere reasonable:

$ mkdir -p ~/lib/java
$ cp ~/Downloads/jrebel ~/lib/java

Next, go to Preferences > JRebel, uncheck "Use embedded JRebel version", and enter the path to jrebel.jar in the "JRebel location" field.

Now, go to ${TOMCAT_HOME}/bin and create a JRebel wrapper script called catalina-jrebel.sh with the following contents.  Be sure to change the javaagent flag below to point to the location of jrebel.jar on your system:

export CATALINA_OPTS="-javaagent:<YOUR PATH TO JREBEL>/jrebel.jar $CATALINA_OPTS"
`dirname $0`/catalina.sh $@

Make ${TOMCAT_HOME}/bin/catalina-jrebel.sh be executable

$ cd $TOMCAT_HOME/bin
$ chmod a+x catalina-jrebel.sh

Setup rebel.xml

For JRebel to pick up on changes, you will need to create a file in  ~/projects/fluxtream-app/fluxtream-web/src/main/resources/ called rebel.xml.  Copy the following lines and modify the <dir name=""> fields to have the right home directory (such as /Users/<YOUR_USERNAME>/projects...):

<?xml version="1.0" encoding="UTF-8"?>
<application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.zeroturnaround.com" xsi:schemaLocation="http://www.zeroturnaround.com http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd">
<classpath>
 <dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/src/main/resources"/>
 <dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/target/ROOT/WEB-INF/classes"/>
 <dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-connectors/target/classes"/>
 <dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-core/target/classes"/>
 </classpath>
<web>
 <link target="/">
 <dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/src/main/webapp">
 </dir>
 </link>
 </web>
</application>

Start the Tomcat server with JRebel enabled

At this point, you should be able to start Tomcat with support for JRebel over JPDA.  If you're using JRebel, use the following command to start Tomcat instead of running catalina.sh directly as described above.  You'll also need to make sure that there isn't already a Tomcat server instance running, as there can be only one.  See the Server Shutdown section below for how to stop an already running server.

$ cd ${TOMCAT_HOME}
$ ./bin/catalina-jrebel.sh jpda start

Server Shutdown

To shut down a running Tomcat server:

cd ${TOMCAT_HOME}
 ./bin/shutdown.sh

The shutdown does not always work, particularly if you are connected over the debugger.  The following command should come up empty if the server died cleanly, and will print a line from the ps command if it is still running:

ps auwwx | grep java

The tomcat process you're looking will probably include the substrings from the path TOMCAT_HOME is set to and "catalina.out".  

If the server is still running, you will need to kill the process manually using the pid returned by ps (pid should be the second column if you use the above invocation).  If killing it nicely doesn't work, it should die by using kill -9 <pid>.