Installation: Mac OS X
This guide walks through the process of setting up a development environment for Fluxtream on Mac OS X.
Prerequisites
Java
You've already got it:
$ java -version
java version "1.6.0_37"
Java(TM) SE Runtime Environment (build 1.6.0_37-b06-434-11M3909)
Java HotSpot(TM) 64-Bit Server VM (build 20.12-b01-434, mixed mode)
Homebrew
We'll start with the excellent package manager Homebrew:
$ ruby -e "$(curl -fsSkL raw.github.com/mxcl/homebrew/go)"
$ brew doctor
$ brew update
Git
$ brew install git
Maven
$ brew install maven
This is a bit more complicated than a simple homebrew incantation, but not by much. We'll follow the directions here.
Install MySQL
$ brew install mysql
$ which mysql
mysql is hashed (/usr/local/bin/mysql)
$ mysql --version
5.6.13 (or similar. Copy this to use for <version> below)
Set up MySQL as a service to be run automatically on startup
$ mkdir -p ~/Library/LaunchAgents
$ cp /usr/local/Cellar/mysql/<version>/homebrew.mxcl.mysql.plist ~/Library/LaunchAgents/
$ launchctl load -w ~/Library/LaunchAgents/homebrew.mxcl.mysql.plist
Set up the metadata tables required by MySQL
$ unset TMPDIR
$ mysql_install_db --verbose --user=`whoami` --basedir="$(brew --prefix mysql)" --datadir=/usr/local/var/mysql --tmpdir=/tmp
$ /usr/local/Cellar/mysql/<version>/bin/mysqladmin -u root password <password>
Start MySQL
$ mysql.server start
It should also work to use other mysqld packages such as MAMP. Either way, just make sure later on that local.properties is set up appropriately for the host, port, user, and password settings used by your mysql installation.
Node.js and less
The Fluxtream build process uses node.js to run a less compiler to build .css files from .less sources. The following is from the MacOS homebrew instructions at https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager:
- sudo brew install node
- sudo npm install -g less
Repos
Fork
Log into Github and fork both the datastore and Fluxtream repos:
Be sure that the github ssh keys are set up properly on the machine you are using. A guide for how to set this up is here.
Clone
$ mkdir -p ~/projects
$ cd ~/projects
$ git clone git@github.com:<username>/datastore.git
$ git clone git@github.com:<username>/fluxtream-app.git
datastore
Build
$ cd ~/projects/datastore
$ make test
If it won't build, you might need to install Xcode, then within Xcode go to Preferences -> Downloads and install "Command Line Tools", then restart Terminal and try the make step again.
Data Directory
This directory will be used by datastore to store channel data for the BodyTrack app:
$ mkdir -p ~/projects/db/dev.kvs
Create database and import cities1000.sql
Versions of Fluxtream more recent than 0.9.0002 manage database versioning manually. Prior to that we allowed the JPA auto update mechanism to manage the schema, but this became untenable once multiple developers were working on branches which differed in the way they interacted with the DB. The setting which controls this behavior is the hibernate.hbm2ddl.auto key in fluxtream-app/fluxtream-web/src/main/webapp/WEB-INF/applicationContext.xml. In recent versions, this is set to 'none', which means that JPA will not try to make modifications to the schema. As a result, it is necessary to manually apply .sql files to manage DB state ourselves.
The sql files which are used in this management live in subdirs of the fluxtream-app/fluxtream-web/db directory. The name of the subdir corresponds to the release number. There are generally two types of files, distinguishable by name.
- schema-after-<version>.sql files are a full dump of the schema once a given version was finalized for deployment.
- Other .sql files are incremental changes for moving from the previous version
The first time you set up, what you want to do is create a database called flx and initialize it with the latest version of the schema-after files available. The following command will show which is the latest file to use for the import:
cd ~/projects/fluxtream-app/fluxtream-web/db
find . -name 'schema-after*' | sort | tail -n 1
Once you have found the appropriate schema-after filename, these commands will create the db. Be sure to modify with the appropriate mysql username after the -u flags. The following lines assume a mysql user named 'root'.
echo 'create database flx;' | mysql -u root -p
mysql -u root -p flx < ./<version>/schema-after-<version>.sql
Now you're ready to import cities1000. This is needed for converting lat/lon locations into city names and timezones. You will get errors if this step is skipped.
mysql -u root -p flx < ~/projects/fluxtream-app/cities1000.sql
GeoLiteCity DB
$ cd ~/projects/fluxtream-app
$ curl -O http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
$ gunzip GeoLiteCity.dat.gz
Setup properties files
Copy the sample properties files to their real location:
cd ~/projects/fluxtream-app/fluxtream-web/src/main/resources/
cp samples/*.properties .
Edit the values in local.properties to customize your Fluxtream install to match your situation. If you leave the default values alone you will end up with:
- A mysql port of 3306, username of flx with mysql password fluxtream:edit db.url, db.username, and db.password to change.
- The datastore and GeoLiteCity.dat being assumed to be under/home/fluxtream/projects: edit btdatastore.exec.location, btdatastore.db.location, and geoIpDb.location to change.
- The URL assumed for running your local Tomcat dev instance of http://localhost:8080/: edit homeBaseUrl to change.
- A white-label welcome page: edit homepage.name to change. If you set the value ofhomepage.name to foo, the Fluxtream server will look for the file fluxtream-web/src/main/webapp/WEB-INF/jsp/foo.jsp.
- A default location of Pittsburgh (only affects running on a local dev instance): edit defaultLocation.latitude and defaultLocation.longitude to change.
Edit the crypto key in common.properties to some long and hard to guess phrase. If left at the default value you will end up with a pitiful encryption key for encrypting the values in the ApiKeyAttribute table. IMPORTANT: if you ever change or misplace the value of this key used when populating the attributes in a given ApiKeyAttribute table you will lose the ability to update any of the user's connectors in that instance. If you copy a Fluxtream DB from one server to another, be sure to set up the crypto key with the same value, or else clear out all the connector-related tables.
Work through the other keys in common.properties and oauth.properties to replace instances of xxx with appropriate values. See Setup and Maintenance of Fluxtream Server Instances for more details.
Checkout, and set up scribe-java
cd ~/projects
git clone https://github.com/fluxtream/scribe-java.git
cd scribe-java/
mvn install
Build
You should be able to build Fluxtream now:
$ cd ~/projects/fluxtream-app
$ mvn install
If it works, congratulations! You're super-close to being development-ready. If not, check for any missed steps or ping info@fluxtream.org.
Tomcat
To actually test Fluxtream out, you'll need a webserver to run it on. This part will explain how to set up Tomcat.
Install
$ brew install tomcat
Setup environment
Add the following lines to your ~/.profile (or to ~/.bashrc if that is sourced from your ~/.profile):
export TARGET_ENV=local
export JPDA_TRANSPORT=dt_socket
export JPDA_ADDRESS=9000
export TOMCAT_HOME=/usr/local/Cellar/tomcat/<version>/libexec
Start a new shell and make sure that the settings took effect
Link To Fluxtream
Tomcat ships with a default webapp. To make the server a bit more useful, let's set this default webapp aside and link to fluxtream-app:
$ cd ${TOMCAT_HOME}/webapps/
$ sudo mkdir ../hold
$ sudo mv ROOT examples docs ../hold
$ sudo ln -s ~/projects/fluxtream-app/fluxtream-web/target/ROOT
Modify Tomcat JAVA_OPTS
We need to change the JAVA_OPTS used by Tomcat or else it will give out of memory errors:
cd ${TOMCAT_HOME}/bin
Add the following just underneath the long comment at the top of catalina.sh:
JAVA_OPTS="-server -Xms2048m -Xmx2048m -XX:PermSize=256m -XX:MaxPermSize=256m"
Start
cd ${TOMCAT_HOME}
./bin/catalina.sh start
Test
Load http://localhost:8080 in your browser to test out Fluxtream. If that doesn't work, check the logs:
cd ${TOMCAT_HOME}
tail -F logs/catalina.out
tail -F logs/all.log
Development Environment
While you can develop for Fluxtream using vim and raw Maven incantations, that way lies madness. This part will explain how to setup several tools for easier, faster development.
IntelliJ IDEA
We recommend using IntelliJ IDEA for writing code. You can download IntelliJ IDEA at: http://www.jetbrains.com/idea/download/index.html
Install the disk image and enter your IntelliJ IDEA license key to continue. Once that's entered, you can use File > Open Project... to import ~/projects/fluxtream-app as an IntelliJ project. It should automatically detect that Spring and Maven are being used.
When you import Fluxtream, you'll probably see red squiggly lines everywhere. That's because you haven't set up a JDK version yet. Go to File > Project Structure... > SDKs and enter the path to your JDK in the "JSDK home path" field. It should be something like
/System/Library/Java/JavaVirtualMachines/1.6.0.jdk/Contents/Home
JRebel
JRebel allows you to instantly view your code changes in the browser–even Java code! Follow the directions to install the IntelliJ IDEA JRebel plugin: http://zeroturnaround.com/software/jrebel/intellij-idea-jrebel-tutorial-formerly-javarebel/
ZeroTurnaround allows open source developers to apply for an open source license. If your uses of Fluxtream are eligible for this, you can sign up for an account at https://my.jrebel.com and apply for an OSS license.
Once that process is complete, go to Preferences > JRebel > Activation and upload the JRebel license file in the "Register license file" field.
JRebel Tomcat Integration
First, download a standalone version of the JRebel JAR from http://zeroturnaround.com/software/jrebel/download/prev-releases/ and put it somewhere reasonable:
$ mkdir -p ~/lib/java
$ cp ~/Downloads/jrebel ~/lib/java
Next, go to Preferences > JRebel, uncheck "Use embedded JRebel version", and enter the path to jrebel.jar in the "JRebel location" field.
Now, go to ${TOMCAT_HOME}/bin and create a JRebel wrapper script called catalina-jrebel.sh with the following contents. Be sure to change the javaagent flag below to point to the location of jrebel.jar on your system:
export CATALINA_OPTS="-javaagent:<YOUR PATH TO JREBEL>/jrebel.jar $CATALINA_OPTS"
`dirname $0`/catalina.sh $@
Make ${TOMCAT_HOME}/bin/catalina-jrebel.sh be executable
$ cd $TOMCAT_HOME/bin
$ chmod a+x catalina-jrebel.sh
Setup rebel.xml
For JRebel to pick up on changes, you will need to create a file in ~/projects/fluxtream-app/fluxtream-web/src/main/resources/ called rebel.xml. Copy the following lines and modify the <dir name=""> fields to have the right home directory (such as /Users/<YOUR_USERNAME>/projects...):
<?xml version="1.0" encoding="UTF-8"?>
<application xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xmlns="http://www.zeroturnaround.com" xsi:schemaLocation="http://www.zeroturnaround.com http://www.zeroturnaround.com/alderaan/rebel-2_0.xsd">
<classpath>
<dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/src/main/resources"/>
<dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/target/ROOT/WEB-INF/classes"/>
<dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-connectors/target/classes"/>
<dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-core/target/classes"/>
</classpath>
<web>
<link target="/">
<dir name="<HOMEDIR>/projects/fluxtream-app/fluxtream-web/src/main/webapp">
</dir>
</link>
</web>
</application>
Start the Tomcat server with JRebel enabled
At this point, you should be able to start Tomcat with support for JRebel over JPDA. If you're using JRebel, use the following command to start Tomcat instead of running catalina.sh directly as described above. You'll also need to make sure that there isn't already a Tomcat server instance running, as there can be only one. See the Server Shutdown section below for how to stop an already running server.
$ cd ${TOMCAT_HOME}
$ ./bin/catalina-jrebel.sh jpda start
Server Shutdown
To shut down a running Tomcat server:
cd ${TOMCAT_HOME}
./bin/shutdown.sh
The shutdown does not always work, particularly if you are connected over the debugger. The following command should come up empty if the server died cleanly, and will print a line from the ps command if it is still running:
ps auwwx | grep java
The tomcat process you're looking will probably include the substrings from the path TOMCAT_HOME is set to and "catalina.out".
If the server is still running, you will need to kill the process manually using the pid returned by ps (pid should be the second column if you use the above invocation). If killing it nicely doesn't work, it should die by using kill -9 <pid>.