Build Instructions (windows specific)

These instructions last updated Jully 17, 2013.  Fluxtream developers have successfully installed the pieces needed for Fluxtream development on Windows 7 (4 or 8 Gb of RAM), and on Windows XP.

I arbitrarily chose to use cygwin for commandline execution and use the following paths and the notes below reflect these choices.  

The paths listed within the sections to be executed within cygwin are the cygwin-style paths.  When navigating to these directories within IntelliJ IDEA and when setting up the local.properties keys you will need to figure out the corresponding DOS paths.  This will typically by default be prepended with c:\<your cygwin install directory>\home\<your username>.

Feel free to make other selections and adjust the execution lines for your setup accordingly:

  • fluxtream-app and datastore are checked out into ~/projects
  • the key-value store for the datastore in ~/projects/db

Install cygwin, create ~/projects

The following instructions assume that you have cygwin installed.  If you do not yet have cygwin installed or if you do not have modules that you need, download Setup.exe from http://cygwin.com/ and run it.

Inside the cygwin setup, at a minimum select the following packages:

  • Under Devel:
    • gcc-g++
    • make
    • git
  • Under database:
    • mysql (just the client; will use the native Windows server)
  • Under Web:
    • wget
  • Under Base:
    • gzip

Within a cygwin shell, execute the following commands:

 mkdir ~/projects
cd ~/projects

Make sure that that folder and its parents are accessible and writable from your Windows user.  



Install JavaJDK

Install Maven

http://maven.apache.org/download.html
-download and upack archive
-add the bin folder to the system's PATH variable
-add your jre installation directory to the system's JAVA_HOME variable
-add your maven install directory to the system's M2_HOME variable (this is the directory which contains the bin subdir with the mvn executable in it)

Info on how to set Windows environment variables is available here if you need help with that.

 

Install Node.js

The following is from the Windows instructions at https://github.com/joyent/node/wiki/Installing-Node.js-via-package-manager:

   Download Windows Installer from http://nodejs.org/#download and follow the install instructions

The Windows Node.js installer by default will install the package manager and add its executable to the PATH variable.  This is exactly what we want.

Once node.js is installed, use the Node package manager to install less.  Double check that the command shell you are using has a %PATH% variable which includes the path to npm (by default c:\Program Files\nodejs\) before executing:

  npm install -g less

Checkout, set up, and build the datastore

In cygwin:

 cd ~/projects
 git clone https://github.com/BodyTrack/datastore.git
cd datastore
make

 

Create a directory for the key-value store the datastore will use.  This directory will get big, so the default location here may not be appropriate.  If you use an alternate location, make sure to modify the btdatastore.db.location property in  local.properties accordingly.

 mkdir ~/projects/db
mkdir ~/projects/db/dev.kvs

Make sure that the permissions on that directory is such that tomcat will be able to write to it.

You will also need to add the cygwin bin directory to your Windows system PATH.  This will be something like c:\<your cygwin install directory>\bin

Info on how to set Windows environment variables is available here if you need help with that.

Checkout scribe-java

In cygwin:

 cd ~/projects
git clone https://github.com/fluxtream/scribe-java.git

Checkout fluxtream-app and setup GeoLiteCity.dat

In cygwin:

 cd ~/projects/
git clone git://github.com/fluxtream/fluxtream-app.git fluxtream-app 

Copy MaxMind's GeoLiteCity.dat database to ~/projects/fluxtream-app:

 cd ~/projects/fluxtream-app
wget http://geolite.maxmind.com/download/geoip/database/GeoLiteCity.dat.gz
gunzip GeoLiteCity.dat.gz

Download IntelliJ IDEA Ultimate

http://www.jetbrains.com/idea/download/download_thanks.jsp

Import scribe-java project into IntelliJ IDEA and install

In IntelliJ IDEA:

-Import the scribe-java project:
  -At the IntelliJ IDEA welcome screen, select Import Project, browse to and select your cygwin ~/projects/scribe-java directory.  This directory will typically by default be <your cygwin install directory>/home/<your username>/projects/scribe-java
  -Check the auto-import for maven projects box
  -Otherwise accept defauts 

-Do maven install:
 -Along the right edge of the IntelliJ IDEA project window, select Maven Projects
 -Open the scribe-java project in the window within the Maven Projects section
 -Open Lifecycle
 -Right click install
 -Select Run 'scribe-java [install]' 

-Close the scribe-java project (File/Close Project) 

Import fluxtream-app project into IntelliJ IDEA

In IntelliJ IDEA:

-Import the fluxtream-app project:
  -At the IntelliJ IDEA welcome screen, select Import Project, browse to and select your cygwin ~/projects/fluxtream-app directory.  This directory will typically by default be <your cygwin install directory>/home/<your username>/projects/fluxtream-app
  -Check the auto-import for maven projects box
  -Otherwise accept defauts 

-Pull in maven dependencies:
 -Along the right edge of the IntelliJ IDEA project window, select Maven Projects
 -Open the fluxtream-app project in the window within the Maven Projects section
 -Open Lifecycle
 -Right click install
 -Select Run 'fluxtream-app [install]'

Don't build yet since we need to do more steps before we're ready for that.

MySQL setup

-download mysql from http://www.mysql.com/downloads/mysql/
-use all default settings for installation
-be sure to configure root user
-Create and start a server instance on localhost:3306.  A good tutorial for this process is here.  Notice that if you have a non-empty root password you will need to click on the "Store in Vault" button during the "Database Connection" step to set the root password for the server instance or else you will get cryptic failures during the "Test DB Connection" step.

 

Create the flx database and import cities1000.sql

Versions of Fluxtream more recent than 0.9.0002 manage database versioning manually.  Prior to that we allowed the JPA auto update mechanism to manage the schema, but this became untenable once multiple developers were working on branches which differed in the way they interacted with the DB.  The setting which controls this behavior is the hibernate.hbm2ddl.auto key in fluxtream-app/fluxtream-web/src/main/webapp/WEB-INF/applicationContext.xml.  In recent versions, this is set to 'none', which means that JPA will not try to make modifications to the schema.  As a result, it is necessary to manually apply .sql files to manage DB state ourselves.

The sql files which are used in this management live in subdirs of the fluxtream-app/fluxtream-web/db directory.  The name of the subdir corresponds to the release number.  There are generally two types of files, distinguishable by name.

  • schema-after-<version>.sql files are a full dump of the schema once a given version was finalized for deployment.  
  • Other .sql files are incremental changes for moving from the previous version

The first time yet set up, what you want to do is create a database called flx and initialize it with the latest version of the schema-after files available.  

-Create database named flx: Within the MySQL Command Line Client, you would do this with the command :

    create database flx;

-Find the schema directory:  
  - Go to the fluxtream-app project you earlier checked out using git  
  - Under that, go to 
fluxtream-web/db
  - Starting from the directory within db with the largest version number and working backwards through the earlier versions, look infor the latest schema-after-<version>.sql file.  For example, as of the writing of these instructions, the latest directory is  0.9.0015, and the latest schema file is 0.9.0014/schema-after-0.9.0014.sql.  Copy the path to that latest schema file.

-Load the schema: Within the MySQL Command Line Client, do this with the following commands :

      use flx;
      source  <path to the sql schema file>;

-Create the cities1000 table:
  - The file cities1000.sql will be at the top level of your fluxtream-app project checkout.  Copy the path to that file.
  - Within the MySQL Command Line Client, do this with the following commands :

      use flx;
      source <path to cities1000.sql>;

Install tomcat

http://tomcat.apache.org/download-70.cgi
-download and install windows service version of tomcat
-disable tomcat's service from running at startup

Configure Tomcat Server in IntelliJ IDEA

Add a new run configuration to set up a local tomcat server:
  -In IntelliJ IDEA select Run/Edit Configurations from the main menu
  -Click the + button in the upper left corner and select Tomcat Server/Local (Local is in a sub-dialog that comes up when hover over Tomcat Server in the list)
  -In the Server tab of the dialog it brings up, click Configure button to the right of the Application Server line at the top 
  -Set both Tomcat Home and Tomcat base directory to your tomcat installation path

Add fluxtream-web:war exploded to deployment
  -In the Deployment tab of the first dialog for adding the Tomcat Server, click the + sign along the right edge and select Artifact
  -Select fluxtream-web:war exploded

Add enviroment variables to each startup option setting TARGET_ENV=local
  -In the Startup/Connection tab of the first dialog for adding the Tomcat Server, do the following with each of the Run, Debug, and Coverage startup options in the top section selected:
    -Find the middle section under Pass environment variables
    -Click the + along the right edge 
    -Add TARGET_ENV as Name and local as value
    -Add JAVA_OPTS as -XX:MaxPermSize=256m -Xms256m -Xmx2048m -Djavax.servlet.request.encoding=UTF-8 -Dfile.encoding=UTF-8

Setup properties files

Copy the sample properties files to their real location:
 -Within your fluxtream-app directory, find the fluxtream-web/src/main/resources/ subdirectory.  Copy the .properties files from the samples subdirectory up one level into the resources directory.

Edit the values in local.properties to customize your Fluxtream install to match your situation.  These are just normal text files.  You can either edit them in a text editor like notepad or within IntelliJ IDEA.

You will definitely need to set the location for the datastore and GeoLiteCity.dat since the defaults assume Linux paths and you need to set them to be the DOS formatted paths (such as c:\) and use double backslashes to separate the path elements.  These will typically be something like:

  • btdatastore.exec.location=c:\\<your cygwin install directory>\\home\\<your username>\\projects\\datastore
  • btdatastore.db.location=c:\\<your cygwin install directory>\\home\\<your username>\\projects\\db\\dev.kvs
  • geoIpDb.location=c:\\<your cygwin install directory>\\home\\<your username>\\projects\\fluxtream-app\\GeoLiteCity.dat

If you leave the other default values alone you will end up with:

  • A mysql port of 3306, username of flx with mysql password fluxtream: edit db.urldb.username, and db.password to change.
  • The URL assumed for running your local Tomcat dev instance of http://localhost:8080/: edit homeBaseUrl to change.
  • A white-label welcome page: edit homepage.name to change.  If you set the value of homepage.name to foo, the Fluxtream server will look for the file fluxtream-web/src/main/webapp/WEB-INF/jsp/foo.jsp. 
  • A default location of Pittsburgh (only affects running on a local dev instance): edit defaultLocation.latitude and defaultLocation.longitude to change.

Edit the crypto key in common.properties to some long and hard to guess phrase.  If left at the default value you will end up with a pitiful encryption key for encrypting the values in the ApiKeyAttribute table.  IMPORTANT: if you ever change or misplace the value of this key used when populating the attributes in a given ApiKeyAttribute table you will lose the ability to update any of the user's connectors in that instance.  If you copy a Fluxtream DB from one server to another, be sure to set up the crypto key with the same value, or else clear out all the connector-related tables.

Work through the other keys in common.properties and oauth.properties to replace instances of xxx with appropriate values.  See Setup and Maintenance of Fluxtream Server Instances for more details.

 

Setup log4j configuration

The log4j logger is configured by a file called log4j.xml.  The default value for the location of the log file needs to be modified for Windows.

-Open the log4j.xml file in IntelliJ IDEA (Ctrl-Shift-n)
-Edit line 5 to point to a place to put your log file.  On windows we use  c:\\<your cygwin install directory>\\home\\<your username>\\projects\\logs\\all.log
-Make sure that directory exists and is writable by your Windows user

-When running the server,  you will want to monitor the contents of this file to watch for system messages.  After you have run the server for the first time, you can do this in cygwin by cd'ing to that directory and doing 'tail -f all.log'

Build and run the fluxtream-app project

In IntelliJ IDEA:

-build the fluxtream-app project (Build/Make project)
-click the run button
-if you get errors about sdks, configure java sdk for the project
-you should now get the login page with a fully functional app

Monitor the logs

Some of the logs will go to your log4j file, see the Setup log4j configuration section above for how to monitor these.

Other logs will go to System.out.  If you are running Tomcat from within IntelliJ IDEA as described above, this will show up in the bottom section of IntelliJ IDEA.  The bottom of the IntelliJ IDEA window will have a row of buttons used to show and hide panels for various modes.  

If you are running normally, click the Run button and you will see the output in the Output window on the left.  

If you are running in debug mode, click the Debug button and you will see a window with multiple tabs.  The output will appear in the output section of the Server tab to the right of the Debugger tab.