anchor.diff:  == Collected Metrics
   
   == Description
  -Postgresql metrics are collected by the crunchy-collect container.  To
  -start collection of metrics on a Postgres database, you add the
  -crunchy-collect container into the pod that holds the crunchy-pg container.
  -
  -=== Requirements
  -Metrics are stored in the crunchy-prometheus container.  crunchy-prometheus runs
  -the prometheus time series database. The prometheus pushgateway is
  -found within the crunchy-promgateway container, and
  -the grafana web application is foud within the crunchy-grafana container.
  -An example of starting the metrics containers
  -is in examples/openshift/scope.json.
  -
  -To start the crunchy-scope container, run the following:
  +PostgreSQL metrics are collected by the crunchy-collect container, which collects 32 different
  +PostgreSQL metrics from a database container and pushes them to a Prometheus time series data
  +store. A web-based graphing dashboard, Grafana, runs from crunchy-grafana, which displays the
  +collected PostgreSQL metrics from the crunchy-prometheus container as a data source and uses
  +the metrics to build dashboards.
  +
  +To start collection of metrics on a PostgreSQL database, you add the crunchy-collect container
  +into the pod that holds the crunchy-postgres container.
  +
  +=== Overview
  +Metrics are stored in the crunchy-prometheus container, which runs the Prometheus time series
  +database. The Prometheus pushgateway is found within the crunchy-promgateway container, and
  +the Grafana web application is found within the crunchy-grafana container.
  +
  +The crunchy-prometheus data in this example is stored in emptyDir volume types. To persist the
  +data and Grafana templates long term, you will want to use NFS volume types as specified in
  +the script examples/openshift/metrics/run-pvc.sh.
  +
  +When running crunchy-metrics, the following ports are available:
  +
  + * crunchy-prometheus:9090 - the Prometheus web user interface
  + * crunchy-promgateway:9091 - the Prometheus pushgateway REST API
  + * crunchy-grafana:3000 - the Grafana web user interface
  +
  +=== Examples
  +
  +==== Docker
  +
  +You can collect various PostgreSQL metrics from your database container by running a
  +crunchy-collect container that points to your database container.
  +
  +Metrics collection requires you run the crunchy-metrics set of containers that includes:
  +
  + * Prometheus
  + * Prometheus push gateway
  + * Grafana
  +
  +To start this set of containers, run the following:
   ....
   cd $CCPROOT/examples/docker/metrics
   ./run.sh
   ....
   
  -The crunchy-prometheus data in this example is stored in emptyDir volume types.
  -To persist the data and grafana templates long term, you will want to
  -use NFS volume types as specified in examples/openshift/metrics/run-nfs.sh
  -
  -When running crunchy-scope, the following ports are available:
  -
  - * crunchy-prometheus:9090 - the prometheus web user interface
  - * crunchy-promgateway:9091 - the prometheus pushgateway REST API
  - * crunchy-grafana:3000 - the grafana web user interface
  -
  -=== crunchy-collect Environment Variables
  +An example has been provided that runs a database container
  +and also the associated metrics collection container, run the
  +example as follows:
  +
  +....
  +cd $CCPROOT/examples/docker/collect
  +./run.sh
  +....
  +
  +Every 3 minutes the collection container will collect PostgreSQL
  +metrics and push them to the Crunchy Prometheus database.  You
  +can graph them using the Crunchy Grafana container.
  +
  +==== Kubernetes
  +
  +This example starts up Prometheus, Grafana, and Prometheus gateway.
  +
  +It is required to view or capture metrics collected by crunchy-collect.
  +
  +Running the example:
  +....
  +examples/kube/metrics/run.sh
  +....
  +
  +This will start up 3 containers and services:
  +
  + * Prometheus (http://crunchy-prometheus:9090)
  + * Prometheus gateway (http://crunchy-promgateway:9091)
  + * Grafana (http://crunchy-grafana:3000)
  +
  +If you want your metrics and dashboards to persist to NFS, run
  +this script:
  +....
  +examples/kube/metrics/run-pvc.sh
  +....
  +
  +In the docs folder of the github repo, check out the metrics.adoc
  +for details on the exact metrics being collected.
  +
  +This example runs a pod that includes a database container and
  +a metrics collection container. A service is also created for the pod.
  +
  +Running the example:
  +....
  +examples/kube/collect/run.sh
  +....
  +
  +You can view the collect container logs using this command:
  +....
  +kubectl logs -c collect master-collect
  +....
  +
  +You can access the database or drive load against it using
  +this command:
  +....
  +psql -h master-collect -U postgres postgres
  +....
  +
  +==== OpenShift
  +
  +First, create the crunchy-metrics pod which contains
  +the Prometheus data store and the Grafana graphing web application:
  +
  +....
  +cd $CCPROOT/examples/openshift/metrics
  +./run.sh
  +....
  +
  +At this point, you can view the Prometheus web console at
  +crunchy-metrics:9090, the Prometheus push gateway at crunchy-metrics:9091,
  +and the Grafana web app at crunchy-metrics:3000.
  +
  +When accessing the Grafana web application, the default user credentials will be
  +the username **admin** and the password **admin**.
  +
  +Next, start a PostgreSQL pod that has the crunchy-collect container
  +as follows:
  +....
  +cd $CCPROOT/examples/openshift/collect
  +./run.sh
  +....
  +
  +At this point, metrics will be collected every 3 minutes and pushed
  +to Prometheus.  You can build graphs off the metrics using Grafana.
  +
  +== crunchy-collect Environment Variables
   
    * POLL_INT - number of minutes to sleep until metrics are collected.  defaults to 15 minutes
  - * PROM_GATEWAY - the http URL of the prometheus pushgateway into which the metrics will be pushed.  defaults to http://crunchy-promgateway:9091
  -
  -=== Example
  -An example of using crunchy-collect is stored in $CCPROOT/examples/openshift/collect/master-collect.json.
  -
  -This example collects metrics for the pg-master database.  Run the
  -example as follows:
  -....
  -oc process -f master-collect.json | oc create -f -
  -....
  + * PROM_GATEWAY - the http URL of the Prometheus pushgateway into which the metrics will be pushed.  defaults to http://crunchy-promgateway:9091
   
   == Collected Metrics
   
  
positive.code: 
  image::grafana-dashboard-sample.png[]
  
  <<<
  == Grafana Data Source
  
  You create a grafana data source that represents the prometheus
  database running within crunchy-prometheus:
  
  image::grafana-data-source.png[]
  
negative.code: <<<
  == Prometheus Console
  
  You can issue raw queries to Prometheus using its web
  console:
  
  image::prometheus-console.png[]
  