anchor.diff:  :toc:
   :toclevels: 2
   :toc-title: Contents
  +
  +== About
  +
  +This document explains how to install the data lake accelerator framework as well as Elasticsearch, NiFi, and ActiveMQ. There are a few different ways you can
  +install it depending on whether or not you are installing all components on one edge node vs. multiple nodes.
   
   == System Requirements
   
  
positive.code: |MySql
  |===
  
  == Installation
  
  === Procedure
  
  Follow the steps below to install the Pipeline Controller Service
  
  
  1. Find and download the RPM file from artifactory and place on the host linux machine
  
             http://54.152.98.43:8080/artifactory/webapp/search/artifact?q=data-lake   (requires VPN)
  
  
  2. Login to the the host using root or sudo access
  3. Run RPM install
  
             $ rpm -ivh pipeline-controller-<version>.noarch.rpm
  
  
  Note: If needed you can remove or un-install using the command below
  
             $ rpm -e pipeline-controller-<version>
  
  
  <<<
  
  == Configuration
  
  === Configuration File
  
  Configuration for the Pipeline Controller Service is located under the following files:
  
      /opt/tba/pipeline-application/conf/application.properties
  
  <<<
  
  === Configuration Properties
  
  
negative.code: |===
  |Configuration Property|Required|Example
  
  |server.port||8400
  |||
  |===
  
  === Database Setup
  
  Data lake services can be configured to work with Postgres or MySQL. Database and permission setup scripts are provided to assist in the initial configuration process.   The script names relevant to setup are below:
  
  ==== My SQL
  |===
  |Script Name|Description
  |/opt/thinkbig/setup/sql/mysql/setup-mysql.sh|Create tables used by data lake accelerator services
  |/opt/thinkbig/setup/sql/mysql/drop-mysql.sh DROP|Used to remove the data lake accelerator schema(s)
  |===
  
  
  ==== Postgres
  TBD - Not yet supported
  
  
  === Optimizing Performance
  
  You can adjust the memory setting of the Pipeline Controller Service using the PIPELINE_APPLICATION_OPTS environment variable.  
  
      export THINKBIG_UI_OPTS=Xmx4g
      export THINKBIG_SERVICES_OPTS=Xmx4g
      
  The setting above would set the Java maximum heap size to 4 GB.    
  
  
  == Starting the Services
  Note: These below are installed as services and should start and stop automatically when the machine is rebooted
  
  For starting and stopping the 3 data lake accelerator services there you can run the following scripts
  
     /opt/thinkbig/start-thinkbig-apps.sh
     /opt/thinkbig/stop-thinkbig-apps.sh
  
  1. To Start individual services
  
      $ service activemq start
      $ service elasticsearch start
      $ service nifi start
      $ service thinkbig-spark-shell start
      $ service thinkbig-services start
      $ service thinkbig-ui start
  
  2. To Stop individual services
  
      $ service activemq stop
      $ service elasticsearch stop
      $ service nifi stop
      $ service thinkbig-spark-shell stop
      $ service thinkbig-services stop
      $ service thinkbig-ui stop
  
  3.  To get the status of individual services
  
      $ service activemq status
      $ service elasticsearch status
      $ service nifi status
      $ service thinkbig-spark-shell status
      $ service thinkbig-services status
      $ service thinkbig-ui status
  
  == Viewing Service Output
  
  === Configuring Log Output
  
  Log output for the services mentioned above are configured at:
  
  			/opt/thinkbig/thinkbig-ui/conf/log4j.properties
  			/opt/thinkbig/thinkbig-services/conf/log4j.properties
  
  You may place logs where desired according to the 'log4j.appender.file.File' property.  Note the configuration line:
  
  			log4j.appender.file.File=/var/log/<app>/<app>.log
  
  === Viewing Log Output
  
  The default log locations for the various applications are located at:
  
  /var/log/<service_name>
  
  == Web and REST Access
  
  Below are the default URL's and ports for the services
  
      Feed Manager and Operations UI
      http://127.0.0.1:8400
      username: dladmin
  	password: thinkbig
  
      NiFi UI
      http://127.0.0.1:8079
  
      Elasticsearch REST API
      http://127.0.0.1:9200
  
      ActiveMQ Admin
      http://127.0.0.1:8161/admin
  
  
  == Appendix: Postgres Integration
  
  TBD
  	
  