fix type ref to time_unit in docs

fix type ref to time_unit in docs

defmodule OpenTelemetry do
  @moduledoc """
  An [OpenTelemetry](https://opentelemetry.io) Trace consists of 1 or more Spans that either have a
  parent/child relationship or are linked together through a Link. Each Span has a TraceId (`t:trace_id/0`),
  SpanId (`t:span_id/0`), and a start and end time in nanoseconds.

  This module provides declaration of the types used throughout the library, as well as functions for
  building the additional pieces of a span that are optional. Each item can be attached to individual
  Span using the functions in `OpenTelemetry.Span` module.

  ## Example

      require OpenTelemetry.Tracer, as: Tracer

      Tracer.with_span "some-span" do
        event = OpenTelemetry.event("ecto.query", query: query, total_time: total_time)
        Tracer.add_events([event])
      end
  """

  @typedoc """
  A SpanContext represents the portion of a Span needed to do operations on a
  Span. Within a process it acts as a key for looking up and modifying the
  actual Span. It is also what is serialized and propagated across process
  boundaries.
  """
  @type span_ctx() :: :opentelemetry.span_ctx()

  @typedoc """
  Span represents a single operation within a trace. Spans can be
  nested to form a trace tree. Spans may also be linked to other spans
  from the same or different trace and form graphs. Often, a trace
  contains a root span that describes the end-to-end latency, and one
  or more subspans for its sub-operations. A trace can also contain
  multiple root spans, or none at all. Spans do not need to be
  contiguous - there may be gaps or overlaps between spans in a trace.
  """
  @type span() :: :opentelemetry.span()
  @type span_kind() :: :opentelemetry.span_kind()
  @type span_name() :: :opentelemetry.span_name()

  @typedoc """
  TraceId is a unique identifier for a trace. All spans from the same trace share
  the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes
  is considered invalid.
  """
  @type trace_id() :: non_neg_integer()

  @typedoc """
  SpanId is a unique identifier for a span within a trace, assigned when the span
  is created. The ID is an 8-byte array. An ID with all zeroes is considered
  invalid.
  """
  @type span_id() :: non_neg_integer()

  @type attribute_key() :: :opentelemetry.attribute_key()
  @type attribute_value() :: :opentelemetry.attribute_value()

  @typedoc """
  Attributes are a collection of key/value pairs. The value can be a string,
  an integer, a double or the boolean values `true` or `false`. Note, global attributes
  like server name can be set using the resource API.

  Examples of attributes:

      [{"/http/user_agent" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"}
       {"/http/server_latency", 300}
       {"abc.com/myattribute", True}
       {"abc.com/score", 10.239}]
  """
  @type attributes_map() :: :opentelemetry.attributes_map()

  @typedoc """
  Tracestate represents tracing-system specific context in a list of key-value pairs.
  Tracestate allows different vendors propagate additional information and
  inter-operate with their legacy Id formats.

  It is a tracestate in the [w3c-trace-context format](https://www.w3.org/TR/trace-context/#tracestate-header).
  See also [https://github.com/w3c/distributed-tracing](https://github.com/w3c/distributed-tracing)
  for more details about this field.
  """
  @type tracestate() :: [{String.t(), String.t()}]

  @typedoc """
  A Link is a pointer from the current span to another span in the same trace or in a
  different trace. For example, this can be used in batching operations,
  where a single batch handler processes multiple requests from different
  traces or when the handler receives a request from a different project.
  """
  @type link() :: :opentelemetry.link()

  @typedoc """
  An Event is a time-stamped annotation of the span, consisting of user-supplied
  text description and key-value pairs.
  """
  @type event() :: :opentelemetry.event()
  @type event_name() :: :opentelemetry.event_name()

  @typedoc """
  An optional final status for this span. Semantically when Status
  wasn't set it means span ended without errors and assume `unset`.

  Application developers may set the status as `ok` when the operation
  has been validated to have completed successfully.
  """
  @type status() :: :opentelemetry.status()

  defdelegate get_tracer(name), to: :opentelemetry
  defdelegate get_tracer(name, vsn, schema_url), to: :opentelemetry
  defdelegate set_default_tracer(t), to: :opentelemetry

  # Helpers to build OpenTelemetry structured types

  @doc """
  A monotonically increasing time provided by the Erlang runtime system in the native time unit.
  This value is the most accurate and precise timestamp available from the Erlang runtime and
  should be used for finding durations or any timestamp that can be converted to a system
  time before being sent to another system.

  Use `convert_timestamp/2` or `timestamp_to_nano/1` to convert a native monotonic time to a
  system time of either nanoseconds or another unit.

  Using these functions allows timestamps to be accurate, used for duration and be exportable
  as POSIX time when needed.
  """
  @spec timestamp() :: integer()
  defdelegate timestamp(), to: :opentelemetry

  @doc """
  Convert a native monotonic timestamp to nanosecond POSIX time. Meaning the time since Epoch.
  Epoch is defined to be 00:00:00 UTC, 1970-01-01.
  """
  @spec timestamp_to_nano(integer()) :: integer()
  defdelegate timestamp_to_nano(timestamp), to: :opentelemetry

  @doc """
  Convert a native monotonic timestamp to POSIX time of any `t::erlang.time_unit/0`.
  Meaning the time since Epoch. Epoch is defined to be 00:00:00 UTC, 1970-01-01.
  """
  @spec convert_timestamp(integer(), :erlang.time_unit()) :: integer()
  defdelegate convert_timestamp(timestamp, unit), to: :opentelemetry

  # span item functions

  @doc """
  Creates a `t:link/0`.
  """
  @spec link(trace_id(), span_id(), attributes_map(), tracestate()) :: link()
  defdelegate link(trace_id, span_id, attributes, tracestate), to: :opentelemetry

  @doc """
  Creates a `t:link/0` from a `t:span_ctx/0`.
  """
  @spec link(span_ctx() | :undefined) :: link()
  defdelegate link(span_ctx), to: :opentelemetry

  @doc """
  Creates a `t:link/0` from a `t:span_ctx/0` and list of `t:attributes_map/0`.
  """
  @spec link(span_ctx() | :undefined, attributes_map()) :: link()
  defdelegate link(span_ctx, attributes), to: :opentelemetry

  @doc """
  Creates a list of `t:link/0` from a list of 4-tuples.
  """
  @spec links([
          {integer(), integer(), attributes_map(), tracestate()}
          | span_ctx()
          | {span_ctx(), attributes_map()}
        ]) :: [link()]
  defdelegate links(link_list), to: :opentelemetry

  @doc """
  Creates a `t:event/0`.
  """
  @spec event(event_name(), attributes_map()) :: event()
  defdelegate event(name, attributes), to: :opentelemetry

  @doc """
  Creates a `t:event/0`.
  """
  @spec event(integer(), event_name(), attributes_map()) :: event()
  defdelegate event(timestamp, name, attributes), to: :opentelemetry

  @doc """
  Creates a list of `t:event/0` items.
  """
  @spec events(list()) :: [event()]
  defdelegate events(event_list), to: :opentelemetry

  @doc """
  Creates a Status.
  """
  @spec status(:opentelemetry.status_code(), String.t()) :: status()
  defdelegate status(code, message), to: :opentelemetry
end

defmodule OpenTelemetry do
  @moduledoc """
  An [OpenTelemetry](https://opentelemetry.io) Trace consists of 1 or more Spans that either have a
  parent/child relationship or are linked together through a Link. Each Span has a TraceId (`t:trace_id/0`),
  SpanId (`t:span_id/0`), and a start and end time in nanoseconds.

  This module provides declaration of the types used throughout the library, as well as functions for
  building the additional pieces of a span that are optional. Each item can be attached to individual
  Span using the functions in `OpenTelemetry.Span` module.

  ## Example

      require OpenTelemetry.Tracer, as: Tracer

      Tracer.with_span "some-span" do
        event = OpenTelemetry.event("ecto.query", query: query, total_time: total_time)
        Tracer.add_events([event])
      end
  """

  @typedoc """
  A SpanContext represents the portion of a Span needed to do operations on a
  Span. Within a process it acts as a key for looking up and modifying the
  actual Span. It is also what is serialized and propagated across process
  boundaries.
  """
  @type span_ctx() :: :opentelemetry.span_ctx()

  @typedoc """
  Span represents a single operation within a trace. Spans can be
  nested to form a trace tree. Spans may also be linked to other spans
  from the same or different trace and form graphs. Often, a trace
  contains a root span that describes the end-to-end latency, and one
  or more subspans for its sub-operations. A trace can also contain
  multiple root spans, or none at all. Spans do not need to be
  contiguous - there may be gaps or overlaps between spans in a trace.
  """
  @type span() :: :opentelemetry.span()
  @type span_kind() :: :opentelemetry.span_kind()
  @type span_name() :: :opentelemetry.span_name()

  @typedoc """
  TraceId is a unique identifier for a trace. All spans from the same trace share
  the same `trace_id`. The ID is a 16-byte array. An ID with all zeroes
  is considered invalid.
  """
  @type trace_id() :: non_neg_integer()

  @typedoc """
  SpanId is a unique identifier for a span within a trace, assigned when the span
  is created. The ID is an 8-byte array. An ID with all zeroes is considered
  invalid.
  """
  @type span_id() :: non_neg_integer()

  @type attribute_key() :: :opentelemetry.attribute_key()
  @type attribute_value() :: :opentelemetry.attribute_value()

  @typedoc """
  Attributes are a collection of key/value pairs. The value can be a string,
  an integer, a double or the boolean values `true` or `false`. Note, global attributes
  like server name can be set using the resource API.

  Examples of attributes:

      [{"/http/user_agent" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_14_2) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/71.0.3578.98 Safari/537.36"}
       {"/http/server_latency", 300}
       {"abc.com/myattribute", True}
       {"abc.com/score", 10.239}]
  """
  @type attributes_map() :: :opentelemetry.attributes_map()

  @typedoc """
  Tracestate represents tracing-system specific context in a list of key-value pairs.
  Tracestate allows different vendors propagate additional information and
  inter-operate with their legacy Id formats.

  It is a tracestate in the [w3c-trace-context format](https://www.w3.org/TR/trace-context/#tracestate-header).
  See also [https://github.com/w3c/distributed-tracing](https://github.com/w3c/distributed-tracing)
  for more details about this field.
  """
  @type tracestate() :: [{String.t(), String.t()}]

  @typedoc """
  A Link is a pointer from the current span to another span in the same trace or in a
  different trace. For example, this can be used in batching operations,
  where a single batch handler processes multiple requests from different
  traces or when the handler receives a request from a different project.
  """
  @type link() :: :opentelemetry.link()

  @typedoc """
  An Event is a time-stamped annotation of the span, consisting of user-supplied
  text description and key-value pairs.
  """
  @type event() :: :opentelemetry.event()
  @type event_name() :: :opentelemetry.event_name()

  @typedoc """
  An optional final status for this span. Semantically when Status
  wasn't set it means span ended without errors and assume `unset`.

  Application developers may set the status as `ok` when the operation
  has been validated to have completed successfully.
  """
  @type status() :: :opentelemetry.status()

  defdelegate get_tracer(name), to: :opentelemetry
  defdelegate get_tracer(name, vsn, schema_url), to: :opentelemetry
  defdelegate set_default_tracer(t), to: :opentelemetry

  # Helpers to build OpenTelemetry structured types

  @doc """
  A monotonically increasing time provided by the Erlang runtime system in the native time unit.
  This value is the most accurate and precise timestamp available from the Erlang runtime and
  should be used for finding durations or any timestamp that can be converted to a system
  time before being sent to another system.

  Use `convert_timestamp/2` or `timestamp_to_nano/1` to convert a native monotonic time to a
  system time of either nanoseconds or another unit.

  Using these functions allows timestamps to be accurate, used for duration and be exportable
  as POSIX time when needed.
  """
  @spec timestamp() :: integer()
  defdelegate timestamp(), to: :opentelemetry

  @doc """
  Convert a native monotonic timestamp to nanosecond POSIX time. Meaning the time since Epoch.
  Epoch is defined to be 00:00:00 UTC, 1970-01-01.
  """
  @spec timestamp_to_nano(integer()) :: integer()
  defdelegate timestamp_to_nano(timestamp), to: :opentelemetry

  @doc """
  Convert a native monotonic timestamp to POSIX time of any `:erlang.time_unit/0`.
  Meaning the time since Epoch. Epoch is defined to be 00:00:00 UTC, 1970-01-01.
  """
  @spec convert_timestamp(integer(), :erlang.time_unit()) :: integer()
  defdelegate convert_timestamp(timestamp, unit), to: :opentelemetry

  # span item functions

  @doc """
  Creates a `t:link/0`.
  """
  @spec link(trace_id(), span_id(), attributes_map(), tracestate()) :: link()
  defdelegate link(trace_id, span_id, attributes, tracestate), to: :opentelemetry

  @doc """
  Creates a `t:link/0` from a `t:span_ctx/0`.
  """
  @spec link(span_ctx() | :undefined) :: link()
  defdelegate link(span_ctx), to: :opentelemetry

  @doc """
  Creates a `t:link/0` from a `t:span_ctx/0` and list of `t:attributes_map/0`.
  """
  @spec link(span_ctx() | :undefined, attributes_map()) :: link()
  defdelegate link(span_ctx, attributes), to: :opentelemetry

  @doc """
  Creates a list of `t:link/0` from a list of 4-tuples.
  """
  @spec links([
          {integer(), integer(), attributes_map(), tracestate()}
          | span_ctx()
          | {span_ctx(), attributes_map()}
        ]) :: [link()]
  defdelegate links(link_list), to: :opentelemetry

  @doc """
  Creates a `t:event/0`.
  """
  @spec event(event_name(), attributes_map()) :: event()
  defdelegate event(name, attributes), to: :opentelemetry

  @doc """
  Creates a `t:event/0`.
  """
  @spec event(integer(), event_name(), attributes_map()) :: event()
  defdelegate event(timestamp, name, attributes), to: :opentelemetry

  @doc """
  Creates a list of `t:event/0` items.
  """
  @spec events(list()) :: [event()]
  defdelegate events(event_list), to: :opentelemetry

  @doc """
  Creates a Status.
  """
  @spec status(:opentelemetry.status_code(), String.t()) :: status()
  defdelegate status(code, message), to: :opentelemetry
end


autologin/nsm_login.ex - new menu in the new version

#!/usr/bin/expect -f

# $Id$

#### TO DO:
#
# - auto installation of NSM possible from inside this client,
# - auto log monitoring and doing an action once a pattern is found,

set logdir "/home/case/store/work/_archives_worklogs/"

set send_slow {10 .01}
set timeout 60
set app nsm
set os "Linux"

if { $argc == 0 } {
  puts "
Usage: $argv0 <username@>host <password>

If not specified username defaults to \"root\" and password to \"netscreen\".

The connection to target host is done with normal ssh command without any arguments.
My suggestion is to use ~/.ssh/config for any non standard options.
"
  exit
} else {
  set host [lindex $argv 0]
  set user "root"
  set pass "netscreen"
  if { [regexp @ $host] } {
    regexp {(\S+?)@} $host match user
    regexp {@(\S+)} $host match host
  }
  if { $argc > 1 } {
    set pass [lindex $argv 1]
  }
}

spawn ssh $user@$host

set stime     [ timestamp -format "%Y/%m/%d %H:%M:%S"]
set filetime [ timestamp -format "%Y%m%d_%H%M%S"]

puts "\033]0;$host $filetime\007"

expect timeout {
  send_user "connection timeout\n"
  exit
} eof {
  send_user "got eof\n"
  exit
} "Permission denied, please try again." {
  send_user "wrong credentials\n"
  exit
} "Are you sure you want to continue connecting (yes/no)?" {
  send -s "yes\r"
  exp_continue
} "assword:" {
  send -s "$pass\r"
  exp_continue
} "Run NSMXPress system setup" {
  set app nsm
  send -s "n"
  exp_continue
} "1-6,QR" {
  set app space
  send -s "6"
  exp_continue
} "1-7,QR" {
  set app space
  send -s "7"
  exp_continue
} "*$ " {
  send -s "sudo su -\r"
  exp_continue
} "*# " {
  send "echo \${SHELL}\r"
}

### make sure we are running bash
expect "*#" {
  if { ! [regexp "bash" $expect_out(buffer)] } {
    send -s "bash\r"
  } else {
    send "\r"
  }
}

expect "*#" {
#  send -s "set -o vi;export HISTCONTROL=ignoredups;export PS1='\\a\[\\D{%Y-%m-%d %H:%M:%S}\]\\n\\u@\\h:\\w \\\$ ';export PROMPT_COMMAND='echo -ne \"\\a\\033_\${USER}@\${HOSTNAME%%.*}:\${PWD}\\033\\\\\"';uname\r"
  send -s "set -o vi;export HISTCONTROL=ignoredups;export PS1='\\a\[\\D{%Y-%m-%d %H:%M:%S}\]\\n\\u@\\h:\\w \\\$ ';uname\r"
}

expect "*#" {
  if { [regexp "Linux" $expect_out(buffer)] } {
    set os "Linux"
  } elseif { [regexp "SunOS" $expect_out(buffer)] } {
    set os "SunOS"
  } else {
    return
  }
  send "\r"
}

expect "*#" {
  if { $os == "Linux" } {
    if { $app == "nsm" } {
      send -s "unset TMOUT;ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime;ntpdate -u 172.30.73.133;hwclock --systohc\r"
    }
  } elseif { $os eq "SunOS" } {
    send "ntpdate -u 172.30.73.133\r"
  } else {
    send "\r"
  }
}

if { $app == "nsm" } {
  expect "*#" {
    if { $os == "Linux" } {
      send -s "ifconfig | perl -nle'/dr:(172.30.\\S+)/&&print\$1'\r"
    } elseif { $os == "SunOS" } {
      send -s "ifconfig -a | perl -nle'/net (172.30.\\S+)/&&print\$1'\r"
    }
  }

  expect "*#" {
    set ourip [regexp -inline -- {172\.30\.\d+\.\d+} $expect_out(buffer)]
    send "\r"
  }
} else {
  set ourip 127.0.0.1
  send "\r"
}

if { $app == "nsm" } {
  expect "*#" {
    send "DBXML_DIR=`ls -1 -t /usr/netscreen/GuiSvr/utils|grep dbxml| head -n 1`;LD_LIBRARY_PATH=/usr/netscreen/GuiSvr/utils/\$DBXML_DIR/lib:\$LD_LIBRARY_PATH;export LD_LIBRARY_PATH\r"
  }
}

expect "*# " {
  log_file "$logdir/$host-$filetime.log"
  send_log "\n---------- log start at $stime ----------\n"

  send_user "
--- Help key ---
Ctrl+a h - all shortcuts\n"

  send "\r"

  interact {

    \003  { send "\003" }
    \004  { send "exit\r"
      set timeout 10
      if { $app == "nsm" } {
        expect timeout {
          set timeout 60
        } eof {
          return
        } "*# " {
          send "exit\r"
          exp_continue
        } "*$ " {
          send "exit\r"
          sleep 1
          return
        }
      }
      if { $app == "space" } {
        expect timeout {
          set timeout 60
        } ",QR" {
          send "q"
          sleep 1
          return
        }
      }
    }

    \001h {
      send_user "
--- Help ---

IP that will be used during the installation is: $ourip

Ctrl+a h - this message,
Ctrl+a c - menu choice, including removal, cleanup, db changes and corrections,
#Ctrl+a d - download and do a clean install of a given NSM version,
Ctrl+a i - can be entered during the nsm installation will answer all the questions (clean install Gui+Dev if installing from scratch or just refresh otherwise),
Ctrl+a x - stop/status/start/restart/version on all three services,
Ctrl+a t - prints current timestamp in the format %Y%m%d%H%M%S sutaible for naming backups/copies of files,
#Ctrl+a f - trigger an action based on a pattern match, if pattern is found, a script on the remote machine will be run /root/data.sh
"
      send "\r"
    }

    \001x {
      send_user "\nType the action (stop/status/start/restart):"
      stty cooked echo
      expect_user -re "(.*)\n"
      stty raw -echo
      set action $expect_out(1,string)
      send -s "/etc/init.d/guiSvr $action; /etc/init.d/devSvr $action; /etc/init.d/haSvr $action\n"
    }

    \001t {
      set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
      send -s "$backuptime"
    }

    \001c {
      send_user "\nType the action number:
 1 - correct /usr/netscreen/DevSvr/var/devSvr.cfg by removing unneeded white characters (make a copy) and restart DevSvr,
 2 - truncate the schema,
 3 - correct the customer db (super password, IPs),
 4 - prepare a command that will uninstall all NSM packages and remove all the NSM data from the server, the command is printed without \\r at the end,

 input: "
      stty cooked echo
      expect_user -re "(.*)\n"
      stty raw -echo
      set action $expect_out(1,string)

      # devsvr.cfg corrections of whitespaces
      if { $action == 1 } {

        set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
        send "perl -pi\".$backuptime\" -e 's/\s\s+/ /g' /usr/netscreen/DevSvr/var/devSvr.cfg && /etc/init.d/devSvr restart\r"

      # truncate schema
      } elseif { $action == 2 } {
        send "/etc/init.d/haSvr stop\r"
        expect "*# " { send "/etc/init.d/guiSvr stop\r"}
        expect "*# " { send "/etc/init.d/devSvr stop\r"}
        #expect "*# " { send "mv -f /usr/netscreen/GuiSvr/var/dmi-schema-stage /usr/netscreen/GuiSvr/var/dmi-schema-stage.old\r"}
        expect "*# " { send "rm -rf /usr/netscreen/GuiSvr/var/dmi-schema-stage\r"}
        expect "*# " { send "cp --reply=yes -fpr /usr/netscreen/GuiSvr/lib/initVar/dmi-schema-stage /usr/netscreen/GuiSvr/var/dmi-schema-stage\r"}
        expect "*# " { send "rm -f /usr/netscreen/GuiSvr/var/xdb/init/*\r"}
        expect "*# " { send "rm -rf /tmp/Schemas*\r"}
        expect "*# " { send "rm -rf /usr/netscreen/GuiSvr/var/Schemas-GDH/*\r"}
        expect "*# " { send "sh /usr/netscreen/GuiSvr/utils/.truncateSchemaTables.sh /usr/netscreen/GuiSvr/var/xdb\r"}
        expect "*# " { send "cp --reply=yes -fpr /usr/netscreen/GuiSvr/lib/initVar/xdb/init/* /usr/netscreen/GuiSvr/var/xdb/init/\r"}
        expect "*# " { send "/etc/init.d/haSvr start\r"}
        expect "*# " { send "/etc/init.d/guiSvr start\r"}
        expect "*# " { send "/etc/init.d/devSvr start\r"}

      # correct the customer db (super password, IPs)
      } elseif { $action == 3 } {
        send "/etc/init.d/haSvr stop\r"
        expect "*# " { send "/etc/init.d/guiSvr stop\r"}
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/setperms.sh GuiSvr > /dev/null\r"}
        expect "*# " { send "/bin/chmod +s /usr/netscreen/GuiSvr/utils/.installIdTool\r"}
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb admin 1 0 /__/password \"glee/aW9bOYEewkD/6Ri8sHh2mU=\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb server 0 0 /__/ip \"$ourip\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb server 0 1 /__/ip \"$ourip\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb shadow_server 0 1 /__/clientOneTimePassword \"dk2003ns\" > /dev/null\r"}
        sleep 1
        set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
        expect "*# " { send "perl -npi\".old_$backuptime\" -e 's/ +/ /g;s/(clientOneTimePassword\\s+)\\S*/\$1 dk2003ns/i;s/(default.printLevel\\s+)\\S*/\$1 debug/i;s/(ourRsaPrivateKey|theirRsaPublicKey)/#\$1/i;s/(guiSvrDirectiveHandler.max.heap|devSvrDirectiveHandler.max.heap\\s+)\\d+/\$1 1536000000/i' /var/netscreen/*Svr/*Svr.cfg > /dev/null\r"}
        expect "*# " { send "/etc/init.d/haSvr restart\r"}
        expect "*# " { send "/etc/init.d/guiSvr restart\r"}
        expect "*# " { send "/etc/init.d/devSvr restart\r"}

      # NSM removal
      } elseif { $action == 4 } {
        if { $os == "Linux" } {
          send -s "rpm -qa | grep netscreen | xargs -r rpm -e ; rm -rf /var/netscreen/*/* /usr/netscreen/*"
        } elseif { $os == "SunOS" } {
          send -s "pkgrm -n `pkginfo -c application | grep -i netscreen | grep -v NSCNpostgres | awk '{print \$2}' | xargs` &&  rm -rf /var/netscreen/ /usr/netscreen/"
        }

      # default - unknown choice
      } else {
        send_user "\nUnknown choice\n"
      }
    }

#    \001d {
#      send_user "\nPlease enter the NSM version in LGB format: "
#      stty cooked echo
#      expect_user -re "(.*)\n"
#      stty raw -echo
#      set nextnsmver $expect_out(1,string)
#      system "date;uptime"
#      send_user "|localy run: nsm_auto_install.pl $nextnsmver $ourip|\n"
#    }

#    \001f {
#
#    }

    \001i {
      send  "\r"
      expect timeout {
        send_user " "
      } "Do you want to do NSM installation with base license? (y/n) *>" {
        send "y\r"
        exp_continue
      } "Enter selection (1-2)*>" {
        send "2\r"
        exp_continue
      } "Will server(s) need to be reconfigured during the refresh? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter selection (1-3)*>" {
        send "3\r"
        exp_continue
      } "Enter base directory location for management servers " {
        send "\r"
        exp_continue
      } "Enable FIPS Support? (y/n)*>" {
        send "n\r"
        exp_continue
      } "Will this machine participate in an HA cluster? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter database log directory location *>" {
        send "\r"
        exp_continue
      } "Enter the management IP address of this server *>" {
        send "$ourip\r"
        exp_continue
      } "Enter the https port for NBI service *>" {
        send "\r"
        exp_continue
      } "Enter the https port for web server *>" {
        send "\r"
        exp_continue
      } "Enter password (password will not display as you type)>" {
        send "$pass\r"
        exp_continue
      } "Will a Statistical Report Server be used with this GUI Server? (y/n) *>" {
        send "n\r"
        exp_continue
      } "UNIX password: " {
        send "$pass\r"
        exp_continue
      } "Will server processes need to be restarted automatically in case of a failure? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Will this machine require local database backups? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter Postgres DevSvr Db port *> " {
        send "\r"
        exp_continue
      } "Enter Postgres DevSvr Db super user *> " {
        send "\r"
        exp_continue
      } "Start server(s) when finished? (y/n) *> " {
        send "y\r"
        exp_continue
      } "Are the above actions correct? (y/n)> " {
        send "y\r"
        exp_continue
      } "Specify location of PostgreSQL * bin *> " {
        send "\r"
        exp_continue
      }
    }
  }
}

set time     [ timestamp -format "%Y/%m/%d %H:%M:%S"]
send_log "\n---------- log close at $time ----------\n"
log_file
exec /bin/bzip2 $logdir/$host-$filetime.log
send_user "\n\[+\] session lasted from $stime to $time\n\[+\] logfile: $logdir/$host-$filetime.log.bz2\n"
exit

#!/usr/bin/expect -f

# $Id$

#### TO DO:
#
# - auto installation of NSM possible from inside this client,
# - auto log monitoring and doing an action once a pattern is found,

set logdir "/home/case/store/work/_archives_worklogs/"

set send_slow {10 .01}
set timeout 60
set app nsm
set os "Linux"

if { $argc == 0 } {
  puts "
Usage: $argv0 <username@>host <password>

If not specified username defaults to \"root\" and password to \"netscreen\".

The connection to target host is done with normal ssh command without any arguments.
My suggestion is to use ~/.ssh/config for any non standard options.
"
  exit
} else {
  set host [lindex $argv 0]
  set user "root"
  set pass "netscreen"
  if { [regexp @ $host] } {
    regexp {(\S+?)@} $host match user
    regexp {@(\S+)} $host match host
  }
  if { $argc > 1 } {
    set pass [lindex $argv 1]
  }
}

spawn ssh $user@$host

set stime     [ timestamp -format "%Y/%m/%d %H:%M:%S"]
set filetime [ timestamp -format "%Y%m%d_%H%M%S"]

puts "\033]0;$host $filetime\007"

expect timeout {
  send_user "connection timeout\n"
  exit
} eof {
  send_user "got eof\n"
  exit
} "Permission denied, please try again." {
  send_user "wrong credentials\n"
  exit
} "Are you sure you want to continue connecting (yes/no)?" {
  send -s "yes\r"
  exp_continue
} "assword:" {
  send -s "$pass\r"
  exp_continue
} "Run NSMXPress system setup" {
  set app nsm
  send -s "n"
  exp_continue
} "1-6,QR" {
  set app space
  send -s "6"
  exp_continue
} "*$ " {
  send -s "sudo su -\r"
  exp_continue
} "*# " {
  send "echo \${SHELL}\r"
}

### make sure we are running bash
expect "*#" {
  if { ! [regexp "bash" $expect_out(buffer)] } {
    send -s "bash\r"
  } else {
    send "\r"
  }
}

expect "*#" {
#  send -s "set -o vi;export HISTCONTROL=ignoredups;export PS1='\\a\[\\D{%Y-%m-%d %H:%M:%S}\]\\n\\u@\\h:\\w \\\$ ';export PROMPT_COMMAND='echo -ne \"\\a\\033_\${USER}@\${HOSTNAME%%.*}:\${PWD}\\033\\\\\"';uname\r"
  send -s "set -o vi;export HISTCONTROL=ignoredups;export PS1='\\a\[\\D{%Y-%m-%d %H:%M:%S}\]\\n\\u@\\h:\\w \\\$ ';uname\r"
}

expect "*#" {
  if { [regexp "Linux" $expect_out(buffer)] } {
    set os "Linux"
  } elseif { [regexp "SunOS" $expect_out(buffer)] } {
    set os "SunOS"
  } else {
    return
  }
  send "\r"
}

expect "*#" {
  if { $os == "Linux" } {
    if { $app == "nsm" } {
      send -s "unset TMOUT;ln -sf /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime;ntpdate -u 172.30.73.133;hwclock --systohc\r"
    }
  } elseif { $os eq "SunOS" } {
    send "ntpdate -u 172.30.73.133\r"
  } else {
    send "\r"
  }
}

if { $app == "nsm" } {
  expect "*#" {
    if { $os == "Linux" } {
      send -s "ifconfig | perl -nle'/dr:(172.30.\\S+)/&&print\$1'\r"
    } elseif { $os == "SunOS" } {
      send -s "ifconfig -a | perl -nle'/net (172.30.\\S+)/&&print\$1'\r"
    }
  }

  expect "*#" {
    set ourip [regexp -inline -- {172\.30\.\d+\.\d+} $expect_out(buffer)]
    send "\r"
  }
} else {
  set ourip 127.0.0.1
}

if { $app == "nsm" } {
  expect "*#" {
    send "DBXML_DIR=`ls -1 -t /usr/netscreen/GuiSvr/utils|grep dbxml| head -n 1`;LD_LIBRARY_PATH=/usr/netscreen/GuiSvr/utils/\$DBXML_DIR/lib:\$LD_LIBRARY_PATH;export LD_LIBRARY_PATH\r"
  }
}

expect "*# " {
  log_file "$logdir/$host-$filetime.log"
  send_log "\n---------- log start at $stime ----------\n"

  send_user "
--- Help key ---
Ctrl+a h - all shortcuts\n"

  send "\r"

  interact {

    \003  { send "\003" }
    \004  { send "exit\r"
      set timeout 10
      if { $app == "nsm" } {
        expect timeout {
          set timeout 60
        } eof {
          return
        } "*# " {
          send "exit\r"
          exp_continue
        } "*$ " {
          send "exit\r"
          sleep 1
          return
        }
      }
      if { $app == "space" } {
        expect timeout {
          set timeout 60
        } "1-6,QR" {
          send "q"
          sleep 1
          return
        }
      }
    }

    \001h {
      send_user "
--- Help ---

IP that will be used during the installation is: $ourip

Ctrl+a h - this message,
Ctrl+a c - menu choice, including removal, cleanup, db changes and corrections,
#Ctrl+a d - download and do a clean install of a given NSM version,
Ctrl+a i - can be entered during the nsm installation will answer all the questions (clean install Gui+Dev if installing from scratch or just refresh otherwise),
Ctrl+a x - stop/status/start/restart/version on all three services,
Ctrl+a t - prints current timestamp in the format %Y%m%d%H%M%S sutaible for naming backups/copies of files,
#Ctrl+a f - trigger an action based on a pattern match, if pattern is found, a script on the remote machine will be run /root/data.sh
"
      send "\r"
    }

    \001x {
      send_user "\nType the action (stop/status/start/restart):"
      stty cooked echo
      expect_user -re "(.*)\n"
      stty raw -echo
      set action $expect_out(1,string)
      send -s "/etc/init.d/guiSvr $action; /etc/init.d/devSvr $action; /etc/init.d/haSvr $action\n"
    }

    \001t {
      set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
      send -s "$backuptime"
    }

    \001c {
      send_user "\nType the action number:
 1 - correct /usr/netscreen/DevSvr/var/devSvr.cfg by removing unneeded white characters (make a copy) and restart DevSvr,
 2 - truncate the schema,
 3 - correct the customer db (super password, IPs),
 4 - prepare a command that will uninstall all NSM packages and remove all the NSM data from the server, the command is printed without \\r at the end,

 input: "
      stty cooked echo
      expect_user -re "(.*)\n"
      stty raw -echo
      set action $expect_out(1,string)

      # devsvr.cfg corrections of whitespaces
      if { $action == 1 } {

        set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
        send "perl -pi\".$backuptime\" -e 's/\s\s+/ /g' /usr/netscreen/DevSvr/var/devSvr.cfg && /etc/init.d/devSvr restart\r"

      # truncate schema
      } elseif { $action == 2 } {
        send "/etc/init.d/haSvr stop\r"
        expect "*# " { send "/etc/init.d/guiSvr stop\r"}
        expect "*# " { send "/etc/init.d/devSvr stop\r"}
        #expect "*# " { send "mv -f /usr/netscreen/GuiSvr/var/dmi-schema-stage /usr/netscreen/GuiSvr/var/dmi-schema-stage.old\r"}
        expect "*# " { send "rm -rf /usr/netscreen/GuiSvr/var/dmi-schema-stage\r"}
        expect "*# " { send "cp --reply=yes -fpr /usr/netscreen/GuiSvr/lib/initVar/dmi-schema-stage /usr/netscreen/GuiSvr/var/dmi-schema-stage\r"}
        expect "*# " { send "rm -f /usr/netscreen/GuiSvr/var/xdb/init/*\r"}
        expect "*# " { send "rm -rf /tmp/Schemas*\r"}
        expect "*# " { send "rm -rf /usr/netscreen/GuiSvr/var/Schemas-GDH/*\r"}
        expect "*# " { send "sh /usr/netscreen/GuiSvr/utils/.truncateSchemaTables.sh /usr/netscreen/GuiSvr/var/xdb\r"}
        expect "*# " { send "cp --reply=yes -fpr /usr/netscreen/GuiSvr/lib/initVar/xdb/init/* /usr/netscreen/GuiSvr/var/xdb/init/\r"}
        expect "*# " { send "/etc/init.d/haSvr start\r"}
        expect "*# " { send "/etc/init.d/guiSvr start\r"}
        expect "*# " { send "/etc/init.d/devSvr start\r"}

      # correct the customer db (super password, IPs)
      } elseif { $action == 3 } {
        send "/etc/init.d/haSvr stop\r"
        expect "*# " { send "/etc/init.d/guiSvr stop\r"}
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/setperms.sh GuiSvr > /dev/null\r"}
        expect "*# " { send "/bin/chmod +s /usr/netscreen/GuiSvr/utils/.installIdTool\r"}
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb admin 1 0 /__/password \"glee/aW9bOYEewkD/6Ri8sHh2mU=\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb server 0 0 /__/ip \"$ourip\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb server 0 1 /__/ip \"$ourip\" > /dev/null\r"}
        sleep 1
        expect "*# " { send "/usr/netscreen/GuiSvr/utils/.xdbUpdate.sh /usr/netscreen/GuiSvr/var/xdb shadow_server 0 1 /__/clientOneTimePassword \"dk2003ns\" > /dev/null\r"}
        sleep 1
        set backuptime [ timestamp -format "%Y%m%d_%H%M%S"]
        expect "*# " { send "perl -npi\".old_$backuptime\" -e 's/ +/ /g;s/(clientOneTimePassword\\s+)\\S*/\$1 dk2003ns/i;s/(default.printLevel\\s+)\\S*/\$1 debug/i;s/(ourRsaPrivateKey|theirRsaPublicKey)/#\$1/i;s/(guiSvrDirectiveHandler.max.heap|devSvrDirectiveHandler.max.heap\\s+)\\d+/\$1 1536000000/i' /var/netscreen/*Svr/*Svr.cfg > /dev/null\r"}
        expect "*# " { send "/etc/init.d/haSvr restart\r"}
        expect "*# " { send "/etc/init.d/guiSvr restart\r"}
        expect "*# " { send "/etc/init.d/devSvr restart\r"}

      # NSM removal
      } elseif { $action == 4 } {
        if { $os == "Linux" } {
          send -s "rpm -qa | grep netscreen | xargs -r rpm -e ; rm -rf /var/netscreen/*/* /usr/netscreen/*"
        } elseif { $os == "SunOS" } {
          send -s "pkgrm -n `pkginfo -c application | grep -i netscreen | grep -v NSCNpostgres | awk '{print \$2}' | xargs` &&  rm -rf /var/netscreen/ /usr/netscreen/"
        }

      # default - unknown choice
      } else {
        send_user "\nUnknown choice\n"
      }
    }

#    \001d {
#      send_user "\nPlease enter the NSM version in LGB format: "
#      stty cooked echo
#      expect_user -re "(.*)\n"
#      stty raw -echo
#      set nextnsmver $expect_out(1,string)
#      system "date;uptime"
#      send_user "|localy run: nsm_auto_install.pl $nextnsmver $ourip|\n"
#    }

#    \001f {
#
#    }

    \001i {
      send  "\r"
      expect timeout {
        send_user " "
      } "Do you want to do NSM installation with base license? (y/n) *>" {
        send "y\r"
        exp_continue
      } "Enter selection (1-2)*>" {
        send "2\r"
        exp_continue
      } "Will server(s) need to be reconfigured during the refresh? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter selection (1-3)*>" {
        send "3\r"
        exp_continue
      } "Enter base directory location for management servers " {
        send "\r"
        exp_continue
      } "Enable FIPS Support? (y/n)*>" {
        send "n\r"
        exp_continue
      } "Will this machine participate in an HA cluster? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter database log directory location *>" {
        send "\r"
        exp_continue
      } "Enter the management IP address of this server *>" {
        send "$ourip\r"
        exp_continue
      } "Enter the https port for NBI service *>" {
        send "\r"
        exp_continue
      } "Enter the https port for web server *>" {
        send "\r"
        exp_continue
      } "Enter password (password will not display as you type)>" {
        send "$pass\r"
        exp_continue
      } "Will a Statistical Report Server be used with this GUI Server? (y/n) *>" {
        send "n\r"
        exp_continue
      } "UNIX password: " {
        send "$pass\r"
        exp_continue
      } "Will server processes need to be restarted automatically in case of a failure? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Will this machine require local database backups? (y/n) *>" {
        send "n\r"
        exp_continue
      } "Enter Postgres DevSvr Db port *> " {
        send "\r"
        exp_continue
      } "Enter Postgres DevSvr Db super user *> " {
        send "\r"
        exp_continue
      } "Start server(s) when finished? (y/n) *> " {
        send "y\r"
        exp_continue
      } "Are the above actions correct? (y/n)> " {
        send "y\r"
        exp_continue
      } "Specify location of PostgreSQL * bin *> " {
        send "\r"
        exp_continue
      }
    }
  }
}

set time     [ timestamp -format "%Y/%m/%d %H:%M:%S"]
send_log "\n---------- log close at $time ----------\n"
log_file
exec /bin/bzip2 $logdir/$host-$filetime.log
send_user "\n\[+\] session lasted from $stime to $time\n\[+\] logfile: $logdir/$host-$filetime.log.bz2\n"
exit


more template tests

more template tests

defmodule FunWithFlags.UI.TemplatesTest do
  use ExUnit.Case, async: true

  alias FunWithFlags.UI.Templates
  alias FunWithFlags.{Flag, Gate}

  import FunWithFlags.UI.TestUtils

  setup_all do
    on_exit(__MODULE__, fn() -> clear_redis_test_db() end)
    :ok
  end

  setup do
    conn = Plug.Conn.assign(%Plug.Conn{}, :namespace, "/pear")
    {:ok, conn: conn}
  end

  describe "_head()" do
    test "it renders", %{conn: conn} do
      out = Templates._head(conn: conn, title: "Coconut")
      assert is_binary(out)
    end

    test "it includes the right content", %{conn: conn} do
      out = Templates._head(conn: conn, title: "Coconut")
      assert String.contains?(out, "<title>FunWithFlags - Coconut</title>")
      assert String.contains?(out, ~s{href="/pear/assets/style.css"})
    end
  end

  describe "index()" do
    setup do
      flags = [
        %Flag{name: :pineapple, gates: [Gate.new(:boolean, true)]},
        %Flag{name: :papaya, gates: [Gate.new(:boolean, false)]},
      ]
      {:ok, flags: flags}
    end

    test "it renders", %{conn: conn, flags: flags} do
      out = Templates.index(conn: conn, flags: flags)
      assert is_binary(out)
    end

    test "it includes the right content", %{conn: conn, flags: flags} do
      out = Templates.index(conn: conn, flags: flags)
      assert String.contains?(out, "<title>FunWithFlags - List</title>")
      assert String.contains?(out, ~s{New Flag})
      assert String.contains?(out, ~s{pineapple})
      assert String.contains?(out, ~s{papaya})
    end
  end
end

defmodule FunWithFlags.UI.TemplatesTest do
  use ExUnit.Case, async: true

  alias FunWithFlags.UI.Templates
  # alias FunWithFlags.{Flag, Gate}

  import FunWithFlags.UI.TestUtils

  setup_all do
    on_exit(__MODULE__, fn() -> clear_redis_test_db() end)
    :ok
  end

  setup do
    conn = Plug.Conn.assign(%Plug.Conn{}, :namespace, "/pear")
    {:ok, conn: conn}
  end

  describe "_head()" do
    test "it renders", %{conn: conn} do
      out = Templates._head(conn: conn, title: "Coconut")
      assert is_binary(out)
    end

    test "it includes the right content", %{conn: conn} do
      out = Templates._head(conn: conn, title: "Coconut")
      assert String.contains?(out, "<title>FunWithFlags - Coconut</title>")
      assert String.contains?(out, ~s{href="/pear/assets/style.css"})
    end
  end
end


Fix warning

Fix warning

Signed-off-by: chatgris <5c682c2d1ec4073e277f9ba9f4bdf07e5794dabe@praditus.com>

defmodule Funnel.PercolatorTest do
  use Funnel.TestCase, async: true
  import Funnel.Es.Asserts

  test "percolator is alive" do
    {:ok, percolator} = Funnel.Percolator.start_link(nil)
    assert Process.alive?(percolator)
  end

  test "notify transistor" do
    within_index do
      {:ok, token} = Funnel.register(self)
      uuid = assert_query_creation('{"query" : {"match" : {"message" : "elasticsearch"}}}', index_id, token)["query_id"]
      message = "{\"message\":\"this new elasticsearch percolator feature is nice, borat style\"}"

      Funnel.Es.refresh
      Funnel.percolate(index_id, message)
      assert_receive({:chunk, %{id: _, item: item}})
      assert item[:query_ids] == [uuid]

      Funnel.Es.unregister(index_id, token, uuid)
    end
  end

  test "notify transistor with a message matching several queries" do
    {:ok, token} = Funnel.register(self)

    within_index do
      uuid1 = assert_query_creation('{"query" : {"match" : {"message" : "thanks"}}}', index_id, token)["query_id"]
      uuid2 = assert_query_creation('{"query" : {"match" : {"message" : "fish"}}}', index_id, token)["query_id"]
      message = "{\"message\":\"So long, and thanks for all the fish\"}"

      Funnel.percolate(index_id, message)

      assert_receive({:chunk, %{id: _, item: item}})
      refute_receive({:chunk, %{id: _, item: _}})

      assert item[:query_ids] |> Enum.sort == [uuid1, uuid2] |> Enum.sort

      Funnel.Es.unregister(index_id, token, uuid1)
      Funnel.Es.unregister(index_id, token, uuid2)
      Funnel.Es.unregister(index_id)
    end
  end

  test "notify transistor with a message matching several queries, bulk edition" do
    {:ok, token} = Funnel.register(self)

    within_index do
      uuid1 = assert_query_creation('{"query" : {"match" : {"message" : "thanks"}}}', index_id, token)["query_id"]
      uuid2 = assert_query_creation('{"query" : {"match" : {"message" : "fish"}}}', index_id, token)["query_id"]
      messages = "[{\"message\":\"So long, and thanks for all the fish\"},{\"message\":\"Say thanks to the fish\"}]"

      Funnel.percolate(index_id, messages)

      assert_receive({:chunk, %{id: _, item: _}})
      assert_receive({:chunk, %{id: _, item: item}})
      refute_receive({:chunk, %{id: _, item: _}})

      assert item[:query_ids] |> Enum.sort == [uuid1, uuid2] |> Enum.sort

      Funnel.Es.unregister(index_id, token, uuid1)
      Funnel.Es.unregister(index_id, token, uuid2)
      Funnel.Es.unregister(index_id)
    end
  end
end

defmodule Funnel.PercolatorTest do
  use Funnel.TestCase, async: true
  import Funnel.Es.Asserts

  test "percolator is alive" do
    {:ok, percolator} = Funnel.Percolator.start_link(nil)
    assert Process.alive?(percolator)
  end

  test "notify transistor" do
    within_index do
      {:ok, token} = Funnel.register(self)
      uuid = assert_query_creation('{"query" : {"match" : {"message" : "elasticsearch"}}}', index_id, token)["query_id"]
      message = "{\"message\":\"this new elasticsearch percolator feature is nice, borat style\"}"

      Funnel.Es.refresh
      Funnel.percolate(index_id, message)
      assert_receive({:chunk, %{id: _, item: item}})
      assert item[:query_ids] == [uuid]

      Funnel.Es.unregister(index_id, token, uuid)
    end
  end

  test "notify transistor with a message matching several queries" do
    {:ok, token} = Funnel.register(self)

    within_index do
      uuid1 = assert_query_creation('{"query" : {"match" : {"message" : "thanks"}}}', index_id, token)["query_id"]
      uuid2 = assert_query_creation('{"query" : {"match" : {"message" : "fish"}}}', index_id, token)["query_id"]
      message = "{\"message\":\"So long, and thanks for all the fish\"}"

      Funnel.percolate(index_id, message)

      assert_receive({:chunk, %{id: _, item: item}})
      refute_receive({:chunk, %{id: _, item: _}})

      assert item[:query_ids] |> Enum.sort == [uuid1, uuid2] |> Enum.sort

      Funnel.Es.unregister(index_id, token, uuid1)
      Funnel.Es.unregister(index_id, token, uuid2)
      Funnel.Es.unregister(index_id)
    end
  end

  test "notify transistor with a message matching several queries, bulk edition" do
    {:ok, token} = Funnel.register(self)

    within_index do
      uuid1 = assert_query_creation('{"query" : {"match" : {"message" : "thanks"}}}', index_id, token)["query_id"]
      uuid2 = assert_query_creation('{"query" : {"match" : {"message" : "fish"}}}', index_id, token)["query_id"]
      messages = "[{\"message\":\"So long, and thanks for all the fish\"},{\"message\":\"Say thanks to the fish\"}]"

      Funnel.percolate(index_id, messages)

      assert_receive({:chunk, %{id: _, item: item}})
      assert_receive({:chunk, %{id: _, item: item}})
      refute_receive({:chunk, %{id: _, item: _}})

      assert item[:query_ids] |> Enum.sort == [uuid1, uuid2] |> Enum.sort

      Funnel.Es.unregister(index_id, token, uuid1)
      Funnel.Es.unregister(index_id, token, uuid2)
      Funnel.Es.unregister(index_id)
    end
  end
end


removed duplicate

removed duplicate

defmodule OpenAperture.Manager.DB.Models.BaseModel do

  defmacro __using__(_opts) do
    quote do
      use Ecto.Model
      use Behaviour
      alias OpenAperture.Manager.Repo

      @type params :: %{binary => any} | %{atom => any} | nil

      @spec new(params) :: Ecto.Changeset.t
      def new(params \\ %{}) do
        validate_changes(struct(__MODULE__), params)
      end

      @spec update(Ecto.Model.t, params) :: Ecto.Changeset.t
      def update(model, params) do
        validate_changes(model, params)
      end

      @spec destroy_for_association(Ecto.Model.t, atom) :: any | nil
      def destroy_for_association(model, assoc_atom) do
        assoc(model, assoc_atom)
        |> Repo.all
        |> Enum.map &destroy(&1)
        :ok
      end

      def transaction_return(tr) do
        case tr do
          {:ok, _} -> :ok
          {:error, reason} -> {:error, reason}
        end
      end

      defcallback destroy(Ecto.Model.t)

      @doc "validates changes for insert or update"
      defcallback validate_changes(Ecto.Model.t | Ecto.Changeset.t, params) :: Ecto.Changeset.t

    end
  end
end

defmodule OpenAperture.Manager.DB.Models.BaseModel do

  defmacro __using__(_opts) do
    quote do
      use Ecto.Model
      use Behaviour
      alias OpenAperture.Manager.Repo

      @type params :: %{binary => any} | %{atom => any} | nil

      @spec new(params) :: Ecto.Changeset.t
      def new(params \\ %{}) do
        validate_changes(struct(__MODULE__), params)
      end

      @spec update(Ecto.Model.t, params) :: Ecto.Changeset.t
      def update(model, params) do
        validate_changes(model, params)
      end

      @spec destroy_for_association(Ecto.Model.t, atom) :: any | nil
      def destroy_for_association(model, assoc_atom) do
        assoc(model, assoc_atom)
        |> Repo.all
        |> Enum.map &destroy(&1)
        :ok
      end

      def transaction_return(tr) do
        case tr do
          {:ok, _} -> :ok
          {:error, reason} -> {:error, reason}
        end
      end

      defcallback destroy(Ecto.Model.t)

      @doc "validates changes for insert or update"
      defcallback validate_changes(Ecto.Model.t | Ecto.Changeset.t, params) :: Ecto.Changeset.t
      defcallback validate_changes(Ecto.Model.t | Ecto.Changeset.t, params) :: Ecto.Changeset.t

    end
  end
end


remove debug messages from mix admin.gen.resource task

defmodule Mix.Tasks.Admin.Gen.Resource do
  @moduledoc """
  Generate an ExAdmin Resource file

  Creates a resource file used to define the administration pages
  for the auto administration feature

      mix admin.gen.resource Survey

  Creates a web/admin/survey.ex file.

  """

  @shortdoc "Generate a Resource file"

  use Mix.Task
  import Mix.ExAdmin.Utils

  defmodule Config do
    defstruct module: nil, package_path: nil
  end

  def run(args) do
    parse_args(args)
    |> copy_file
  end

  defp copy_file(%Config{module: module, package_path: package_path} = config) do
    filename = String.downcase(module) <> ".ex"
    dest_path = Path.join ~w(web admin)
    dest_file_path = Path.join dest_path, filename
    source_file = Path.join([package_path | ~w(priv templates admin.gen.resource resource.exs)] )
    source = source_file  |> EEx.eval_file(base: get_module, resource: module)
    status_msg "creating", dest_file_path
    File.write! dest_file_path, source
    display_instructions config
  end

  defp display_instructions(config) do
    base = get_module
    IO.puts ""
    IO.puts "Remember to update your config file with the resource module"
    IO.puts ""
    IO.puts """
        config :ex_admin, :modules, [
          #{base}.ExAdmin.Dashboard,
          ...
          #{base}.ExAdmin.#{config.module}
        ]

    """
  end

  defp parse_args([module]) do
    %Config{module: module, package_path: get_package_path}
  end

end

defmodule Mix.Tasks.Admin.Gen.Resource do
  @moduledoc """
  Generate an ExAdmin Resource file

  Creates a resource file used to define the administration pages
  for the auto administration feature

      mix admin.gen.resource Survey

  Creates a web/admin/survey.ex file.

  """

  @shortdoc "Generate a Resource file"

  use Mix.Task
  import Mix.ExAdmin.Utils

  defmodule Config do
    defstruct module: nil, package_path: nil
  end

  def run(args) do
    parse_args(args)
    |> copy_file
  end

  defp copy_file(%Config{module: module, package_path: package_path} = config) do
    IO.puts "config: #{inspect config}"
    filename = String.downcase(module) <> ".ex"
    # dest_path = Path.join [File.cwd! | ~w(web admin)]
    dest_path = Path.join ~w(web admin)
    dest_file_path = Path.join dest_path, filename
    IO.puts "dest: #{inspect dest_file_path}"
    source_file = Path.join([package_path | ~w(priv templates admin.gen.resource resource.exs)] )
    IO.puts "sourcefile: #{source_file}"
    source = source_file  |> EEx.eval_file(base: get_module, resource: module)
    status_msg "creating", dest_file_path
    File.write! dest_file_path, source
    display_instructions config
  end

  defp display_instructions(config) do
    base = get_module
    IO.puts ""
    IO.puts "Remember to update your config file with the resource module"
    IO.puts ""
    IO.puts """
        config :ex_admin, :modules, [
          #{base}.ExAdmin.Dashboard,
          ...
          #{base}.ExAdmin.#{config.module}
        ]

    """
  end

  defp parse_args([module]) do
    %Config{module: module, package_path: get_package_path}
  end

end


typo fix

typo fix

defmodule Noizu.SimplePool.ServerBehaviour do

  # Must be implemented
  @callback lazy_load(Noizu.SimplePool.Server.State.t) :: {any, Noizu.SimplePool.Server.State.t}

  # Provided
  @callback load() :: any

  @callback status() :: any

  @callback generate(:nmid) :: any

  @callback add!(any) :: any
  @callback add!(any, :asynch) :: any

  @callback remove!(any) :: any
  @callback remove!(any, :asynch) :: any
  @callback remove(any, :worker, any) :: any

  @callback fetch!(any, any)  :: any
  @callback fetch!(any, any, :asynch, any) :: any

  @callback pid_or_spawn(any) :: {:ok, any} | {:error, any}

  @callback start_link(any, any) :: any
  @callback init(any) :: any
  @callback terminate(any, any, any) :: any
  @callback alive?(any, :worker) :: any
  @callback add(any, :worker, any) :: any
  @callback start(any, :worker, any) :: any
  @callback start(any, any, :worker, any) :: any
  @callback worker_pid!(any) :: any

  @callback handle_call({:load_complete, {any, any}}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:status}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:generate, :nmid}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:add_worker, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:remove_worker, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:fetch, any, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_cast({:load}, Noizu.SimplePool.Server.State.t) :: any

  @provided_methods [
    :start_link,
    :init,
    :terminate,
    :load,
    :status,
    :generate,
    :add!,
    :remove!,
    :fetch!,
    :pid_or_spawn!,
    :alive?,
    :add,
    :start,
    :remove,
    :worker_pid!,
    :call_load_complete,
    :call_status,
    :call_generate,
    :call_add_worker,
    :call_remove_worker,
    :call_fetch,
    :cast_load
  ]

  defmacro __using__(options) do
    global_verbose = Dict.get(options, :verbose, false)
    module_verbose = Dict.get(options, :server_verbose, false)
    only = Noizu.SimplePool.Behaviour.map_intersect(@provided_methods, Dict.get(options, :only, @provided_methods))
    override = Noizu.SimplePool.Behaviour.map_intersect(@provided_methods, Dict.get(options, :override, []))
    asynch_load = Dict.get(options, :asynch_load, false)

    quote do
      import unquote(__MODULE__)
      require Amnesia
      require Amnesia.Fragment

      @behaviour Noizu.SimplePool.ServerBehaviour
      @base Module.split(__MODULE__) |> Enum.slice(0..-2) |> Module.concat
      @worker_supervisor Module.concat([@base, "WorkerSupervisor"])
      #=========================================================================
      #=========================================================================
      # Genserver Lifecycle
      #=========================================================================
      #=========================================================================

    # @start_link
    if (unquote(only.start_link) && !unquote(override.start_link)) do
      def start_link(sup, nmid_generator) do
        if (unquote(global_verbose) || unquote(module_verbose)) do
          "************************************************\n" <>
          "* START_LINK #{__MODULE__} (#{inspect {sup, nmid_generator}})\n" <>
          "************************************************\n" |> IO.puts()
        end
        GenServer.start_link(__MODULE__, {sup, nmid_generator}, name: __MODULE__)
      end
    end # end start_link

    # @init
    if (unquote(only.init) && !unquote(override.init)) do
      def init({sup, {node, process}} = p) do
        if (unquote(global_verbose) || unquote(module_verbose)) do
          "************************************************\n" <>
          "* INIT #{__MODULE__} (#{inspect {sup, {node, process}}})\n" <>
          "************************************************\n" |> IO.puts()
        end
        {{node, process}, sequence} = @base.book_keeping_init()

        state = %Noizu.SimplePool.Server.State{
          pool: sup,
          nmid_generator: {{node, process}, sequence},
          status_details: nil,
          status: :uninitialized
        }

        {:ok, state}
      end
    end # end init

    # @terminate
    if (unquote(only.terminate) && !unquote(override.terminate)) do
      def terminate(reason, request, state) do
        :ok
      end
    end # end terminate

      #=========================================================================
      #=========================================================================
      #  Common Convienence Methods
      #=========================================================================
      #=========================================================================

      # @load
      if (unquote(only.load) && !unquote(override.load)) do
        @doc """
          Load pool from datastore.
        """
        def load() do
          GenServer.call(__MODULE__, {:load})
        end
      end # end load

      # @status
      if (unquote(only.status) && !unquote(override.status)) do

        @doc """
          Retrieve pool status
        """
        def status() do
          GenServer.call(__MODULE__, {:status})
        end
      end # end status

      # @generate
      if (unquote(only.generate) && !unquote(override.generate)) do

        @doc """
          Generate unique nmid value.
        """
        def generate(:nmid) do
          # Assuming uncapped sequence, and no more than 99 nodes and 999 processes per node
          GenServer.call(__MODULE__, {:generate, :nmid})
        end
      end # end generate

      # @add!
      if (unquote(only.add!) && !unquote(override.add!)) do

        @doc """
          Add worker pool keyed by nmid. Worker must know how to load itself, and provide a load method.
        """
        def add!(nmid) do
          GenServer.call(__MODULE__, {:add_worker, nmid})
        end
      end # end add!

      # @remove!
      if (unquote(only.remove!) && !unquote(override.remove!)) do

        @doc """
          Remove worker process.
        """
        def remove!(nmid) do
          GenServer.call(__MODULE__, {:remove_worker, nmid})
        end
      end # end remove!

      # @fetch!
      if (unquote(only.fetch!) && !unquote(override.fetch!)) do

        @doc """
          Fetch information about worker. Exact information is class dependent.
        """
        def fetch!(nmid, details \\ :default) do
          case pid_or_spawn!(nmid) do
            {:ok, pid} -> GenServer.call(pid, {:fetch, details})
            _ -> :not_found
          end
        end
      end # end fetch!

      # @pid_or_spawn!
      if (unquote(only.pid_or_spawn!) && !unquote(override.pid_or_spawn!)) do

        @doc """
          return cached pid for process or spawn if dead and return newly created pid.
        """
        def pid_or_spawn!(nmid) do
          case alive?(nmid, :worker) do
            {false, :nil} -> add!(nmid)
            {true, pid} -> {:ok, pid}
          end
        end
      end # end pid_or_spawn!

      # @remove!
      if (unquote(only.remove!) && !unquote(override.remove!)) do

        @doc """
          Remove worker Asynch
        """
        def remove!(nmid, :asynch) do
          GenServer.cast(__MODULE__, {:remove_worker, nmid})
        end
      end # end remove!

      # @add!
      if (unquote(only.add!) && !unquote(override.add!)) do

        @doc """
          Add worker process Asynch
        """
        def add!(nmid, :asynch) do
          GenServer.cast(__MODULE__, {:add_worker, nmid})
        end
      end # end add!

      # @fetch!
      if (unquote(only.fetch!) && !unquote(override.fetch!)) do

        @doc """
          Fetch worker process Asynch. (Once worker ahs prepared requested data it performs a callback)
        """
        def fetch!(nmid, details, :asynch, caller \\ :current) do
          caller = if caller == :current do
            self()
          else
            caller
          end

          # Todo - Implement as macro
          case alive?(nmid, :worker) do
            {false, :nil} ->
              # Call Server to spawn worker and then fetch results.
              GenServer.cast(__MODULE__, {:fetch, nmid, details, caller})
            {true, pid} ->
              # Call worker directly
              GenServer.cast(pid, {:fetch, nmid, details, caller})
          end
        end
      end # end fetch!

      # @alive?
      if (unquote(only.alive?) && !unquote(override.alive?)) do

        #-----------------------------------------------------------------------------
        # alive/2
        #-----------------------------------------------------------------------------
        def alive?(:nil, :worker) do
          {false, :nil}
        end

        def alive?(nmid, :worker) when is_number(nmid) or is_tuple(nmid) do
          case :ets.info(@base.lookup_table()) do
            :undefined -> {false, :nil}
            _ ->
              case :ets.lookup(@base.lookup_table(), nmid) do
                 [{_key, pid}] ->
                    if Process.alive?(pid) do
                      {true, pid}
                    else
                      {false, :nil}
                    end
                  [] -> {false, :nil}
              end
          end
        end

        def alive?(nmid, :worker) when is_bitstring(nmid) do
          alive?(nmid |> Integer.parse() |> elem(0), :worker)
        end
      end # end alive?

        #-----------------------------------------------------------------------------
        # add/3
        #-----------------------------------------------------------------------------
      # @add
      if (unquote(only.add) && !unquote(override.add)) do
        defp add(nmid, :worker, sup) do
          case alive?(nmid, :worker) do
              {:false, :nil} -> start(nmid, :worker, sup)
              {:true, pid} -> {:ok, pid}
          end
        end
      end # end add

        #-----------------------------------------------------------------------------
        # start/3
        #-----------------------------------------------------------------------------
      # @start
      if (unquote(only.start) && !unquote(override.start)) do
        defp start(nmid, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          childSpec = @worker_supervisor.child(nmid)
          case Supervisor.start_child(sup, childSpec) do
            {:ok, pid} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            {:error, {:already_started, pid}} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            error ->
              Logger.error("#{__MODULE__} unable to start #{inspect nmid}")
              error
          end
        end

        defp start(nmid, :worker, sup) when is_bitstring(nmid) do
          start(nmid |> Integer.parse() |> elem(0), :worker, sup)
        end

      end # end start

        #-----------------------------------------------------------------------------
        # start/4
        #-----------------------------------------------------------------------------
        # @start
        if (unquote(only.start) && !unquote(override.start)) do
        defp start(nmid, arguments, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          childSpec = @worker_supervisor.child(nmid, arguments)
          case Supervisor.start_child(sup, childSpec) do
            {:ok, pid} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            {:error, {:already_started, pid}} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            error ->
              Logger.error("#{__MODULE__} unable to start #{inspect nmid}")
              error
          end
        end

        defp start(nmid, arguments, :worker, sup) when is_bitstring(nmid) do
          start(nmid |> Integer.parse() |> elem(0), arguments, :worker, sup)
        end
      end # end start

        #-----------------------------------------------------------------------------
        # remove/3
        #-----------------------------------------------------------------------------

        # @remove
        if (unquote(only.remove) && !unquote(override.remove)) do
        defp remove(nmid, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          Supervisor.terminate_child(sup, nmid)
          Supervisor.delete_child(sup, nmid)
          :ets.delete(@base.lookup_table(), nmid)
        end
        defp remove(nmid, :worker, sup) when is_bitstring(nmid) do
          remove(nmid |> Integer.parse() |> elem(0), :worker, sup)
        end

      end # end remove

      # @worker_pid!
      if (unquote(only.worker_pid!) && !unquote(override.worker_pid!)) do

        @doc """
          Add worker pool keyed by nmid. Worker must know how to load itself, and provide a load method.
        """
        def worker_pid!(nmid) when is_bitstring(nmid) do
          String.to_integer(nmid)
            |> worker_pid!()
        end

        def worker_pid!(nmid) when is_integer(nmid) or is_tuple(nmid) do
          case alive?(nmid, :worker) do
            {false, :nil} ->
              # Call Server to spawn worker and then fetch results.
              add!(nmid)
            {true, pid} ->
              # Call worker directly
              {:ok, pid}
          end
        end
      end # end worker_pid!

        #=========================================================================
        #=========================================================================
        # Common Call Handlers for SimpleServer
        #=========================================================================
        #=========================================================================
      # @call_load_complete
      if (unquote(only.call_load_complete) && !unquote(override.call_load_complete)) do
        def handle_call({:load_complete, {outcome, details}}, _from, state) do
            state = if outcome == :ok do
              %Noizu.SimplePool.Server.State{state| status: :online, status_details: details}
            else
              %Noizu.SimplePool.Server.State{state| status: :degrade, status_details: details}
            end
            {:reply, :ok, state}
        end
      end # end call_load_complete

      # @call_status
      if (unquote(only.call_status) && !unquote(override.call_status)) do
        def handle_call({:status}, _from, %Noizu.SimplePool.Server.State{status: status} = state) do
          {:reply, status, state}
        end
      end # end call_status

      # @call_generte
      if (unquote(only.call_generate) && !unquote(override.call_generate)) do
        def handle_call({:generate, :nmid}, _from, %Noizu.SimplePool.Server.State{nmid_generator: {{node, process}, sequence}} = state) do
          {nmid, state} = @base.generate_nmid(state)
          {:reply, nmid, state}
        end
      end # end call_generate

      # @call_add_worker
      if (unquote(only.call_add_worker) && !unquote(override.call_add_worker)) do
        def handle_call({:add_worker, nmid}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
          # Check if existing entry exists. If so confirm it is live and return {:exists, pid} or respawn
          response = add(nmid, :worker, sup)
          {:reply, response, state}
        end
      end # end call_add_worker

      # @call_remove_worker
      if (unquote(only.call_remove_worker) && !unquote(override.call_remove_worker)) do
        def handle_call({:remove_worker, nmid}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
          # Check if existing entry exists. If so confirm it is live and return {:exists, pid} or respawn
          response = remove(nmid, :worker, sup)
          {:reply, response, state}
        end
      end # end call_remove_worker

      # @call_fetch
      if (unquote(only.call_fetch) && !unquote(override.call_fetch)) do
        def handle_call({:fetch, nmid, details}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
              {:ok, pid} = add(nmid, :worker, sup)
              response = GenServer.call(pid, {:fetch, details})
              {:reply, response, state}
        end
      end # end call_fetch

        #=========================================================================
        #=========================================================================
        # Common Cast Handlers for SimpleServer
        #=========================================================================
        #=========================================================================

      # @call_load
      if (unquote(only.call_load) && !unquote(override.call_load)) do

        if (!unquote(asynch_load)) do
          def handle_cast({:load}, state) do
            IO.puts "INITIAL LOAD: #{inspect state}"
            {:noreply, state}
          end
        end

        if (unquote(asynch_load)) do
          def handle_cast({:load}, %Noizu.SimplePool.Server.State{status: status} = state) do
            if status == :uninitialized do
              state = %Noizu.SimplePool.Server.State{state| status: :initializing}

              spawn fn ->
                {status, details} = lazy_load(state)
                GenServer.call(__MODULE__, {:load_complete, {status, details}})
              end

              {:noreply, state}
            else
              {:noreply, state}
            end
          end
        end
      end # end call_load

      @before_compile unquote(__MODULE__)
    end # end quote
  end #end __using__

  defmacro __before_compile__(_env) do
    quote do
    end # end quote
  end # end __before_compile__

end

defmodule Noizu.SimplePool.ServerBehaviour do

  # Must be implemented
  @callback lazy_load(Noizu.SimplePool.Server.State.t) :: {any, Noizu.SimplePool.Server.State.t}

  # Provided
  @callback load() :: any

  @callback status() :: any

  @callback generate(:nmid) :: any

  @callback add!(any) :: any
  @callback add!(any, :asynch) :: any

  @callback remove!(any) :: any
  @callback remove!(any, :asynch) :: any
  @callback remove(any, :worker, any) :: any

  @callback fetch!(any, any)  :: any
  @callback fetch!(any, any, :asynch, any) :: any

  @callback pid_or_spawn(any) :: {:ok, any} | {:error, any}

  @callback start_link(any, any) :: any
  @callback init(any) :: any
  @callback terminate(any, any, any) :: any
  @callback alive?(any, :worker) :: any
  @callback add(any, :worker, any) :: any
  @callback start(any, :worker, any) :: any
  @callback start(any, any, :worker, any) :: any
  @callback worker_pid!(any) :: any

  @callback handle_call({:load_complete, {any, any}}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:status}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:generate, :nmid}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:add_worker, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:remove_worker, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_call({:fetch, any, any}, any, Noizu.SimplePool.Server.State.t) :: any
  @callback handle_cast({:load}, Noizu.SimplePool.Server.State.t) :: any

  @provided_methods [
    :start_link,
    :init,
    :terminate,
    :load,
    :status,
    :generate,
    :add!,
    :remove!,
    :fetch!,
    :pid_or_spawn!,
    :alive?,
    :add,
    :start,
    :remove,
    :worker_pid!,
    :call_load_complete,
    :call_status,
    :call_generate,
    :call_add_worker,
    :call_remove_worker,
    :call_fetch,
    :cast_load
  ]

  defmacro __using__(options) do
    global_verbose = Dict.get(options, :verbose, false)
    module_verbose = Dict.get(options, :server_verbose, false)
    only = Noizu.SimplePool.Behaviour.map_intersect(@provided_methods, Dict.get(options, :only, @provided_methods))
    override = Noizu.SimplePool.Behaviour.map_intersect(@provided_methods, Dict.get(options, :override, []))
    asynch_load = Dict.get(options, :asynch_load, false)

    quote do
      import unquote(__MODULE__)
      require Amnesia
      require Amnesia.Fragment

      @behaviour Noizu.SimplePool.ServerBehaviour
      @base Module.split(__MODULE__) |> Enum.slice(0..-2) |> Module.concat
      @worker_supervisor Module.concat([@base, "WorkerSupervisor"])
      #=========================================================================
      #=========================================================================
      # Genserver Lifecycle
      #=========================================================================
      #=========================================================================

    # @start_link
    if (unquote(only.start_link) && !unquote(override.start_link)) do
      def start_link(sup, nmid_generator) do
        if (unquote(global_verbose) || unquote(module_verbose)) do
          "************************************************\n" <>
          "* START_LINK #{__MODULE__} (#{inspect {sup, nmid_generator}})\n" <>
          "************************************************\n" |> IO.puts()
        end
        GenServer.start_link(__MODULE__, {sup, nmid_generator}, name: __MODULE__)
      end
    end # end start_link

    # @init
    if (unquote(only.init) && !unquote(override.init)) do
      def init({sup, {node, process}} = p) do
        if (unquote(global_verbose) || unquote(module_verbose)) do
          "************************************************\n" <>
          "* INIT #{__MODULE__} (#{inspect {sup, {node, process}}})\n" <>
          "************************************************\n" |> IO.puts()
        end
        {{node, process}, sequence} = @base.book_keeping_init()

        state = %Noizu.SimplePool.Server.State{
          pool: sup,
          nmid_generator: {{node, process}, sequence},
          status_details: nil,
          status: :uninitialized
        }

        {:ok, state}
      end
    end # end init

    # @terminate
    if (unquote(only.terminate) && !unquote(override.terminate)) do
      def terminate(reason, request, state) do
        :ok
      end
    end # end terminate

      #=========================================================================
      #=========================================================================
      #  Common Convienence Methods
      #=========================================================================
      #=========================================================================

      # @load
      if (unquote(only.load) && !unquote(override.load)) do
        @doc """
          Load pool from datastore.
        """
        def load() do
          GenServer.call(__MODULE__, {:load})
        end
      end # end load

      # @status
      if (unquote(only.status) && !unquote(override.status)) do

        @doc """
          Retrieve pool status
        """
        def status() do
          GenServer.call(__MODULE__, {:status})
        end
      end # end status

      # @generate
      if (unquote(only.generate) && !unquote(override.generate)) do

        @doc """
          Generate unique nmid value.
        """
        def generate(:nmid) do
          # Assuming uncapped sequence, and no more than 99 nodes and 999 processes per node
          GenServer.call(__MODULE__, {:generate, :nmid})
        end
      end # end generate

      # @add!
      if (unquote(only.add!) && !unquote(override.add!)) do

        @doc """
          Add worker pool keyed by nmid. Worker must know how to load itself, and provide a load method.
        """
        def add!(nmid) do
          GenServer.call(__MODULE__, {:add_worker, nmid})
        end
      end # end add!

      # @remove!
      if (unquote(only.remove!) && !unquote(override.remove!)) do

        @doc """
          Remove worker process.
        """
        def remove!(nmid) do
          GenServer.call(__MODULE__, {:remove_worker, nmid})
        end
      end # end remove!

      # @fetch!
      if (unquote(only.fetch!) && !unquote(override.fetch!)) do

        @doc """
          Fetch information about worker. Exact information is class dependent.
        """
        def fetch!(nmid, details \\ :default) do
          case pid_or_spawn!(nmid) do
            {:ok, pid} -> GenServer.call(pid, {:fetch, details})
            _ -> :not_found
          end
        end
      end # end fetch!

      # @pid_or_spawn!
      if (unquote(only.pid_or_spawn!) && !unquote(override.pid_or_spawn!)) do

        @doc """
          return cached pid for process or spawn if dead and return newly created pid.
        """
        def pid_or_spawn!(nmid) do
          case alive?(nmid, :worker) do
            {false, :nil} -> add!(nmid)
            {true, pid} -> {:ok, pid}
          end
        end
      end # end pid_or_spawn!

      # @remove!
      if (unquote(only.remove!) && !unquote(override.remove!)) do

        @doc """
          Remove worker Asynch
        """
        def remove!(nmid, :asynch) do
          GenServer.cast(__MODULE__, {:remove_worker, nmid})
        end
      end # end remove!

      # @add!
      if (unquote(only.add!) && !unquote(override.add!)) do

        @doc """
          Add worker process Asynch
        """
        def add!(nmid, :asynch) do
          GenServer.cast(__MODULE__, {:add_worker, nmid})
        end
      end # end add!

      # @fetch!
      if (unquote(only.fetch!) && !unquote(override.fetch!)) do

        @doc """
          Fetch worker process Asynch. (Once worker ahs prepared requested data it performs a callback)
        """
        def fetch!(nmid, details, :asynch, caller \\ :current) do
          caller = if caller == :current do
            self()
          else
            caller
          end

          # Todo - Implement as macro
          case alive?(nmid, :worker) do
            {false, :nil} ->
              # Call Server to spawn worker and then fetch results.
              GenServer.cast(__MODULE__, {:fetch, nmid, details, caller})
            {true, pid} ->
              # Call worker directly
              GenServer.cast(pid, {:fetch, nmid, details, caller})
          end
        end
      end # end fetch!

      # @alive?
      if (unquote(only.alive?) && !unquote(override.alive?)) do

        #-----------------------------------------------------------------------------
        # alive/2
        #-----------------------------------------------------------------------------
        def alive?(:nil, :worker) do
          {false, :nil}
        end

        def alive?(nmid, :worker) when is_number(nmid) or is_tuple(nmid) do
          case :ets.info(@base.lookup_table()) do
            :undefined -> {false, :nil}
            _ ->
              case :ets.lookup(@base.lookup_table(), nmid) do
                 [{_key, pid}] ->
                    if Process.alive?(pid) do
                      {true, pid}
                    else
                      {false, :nil}
                    end
                  [] -> {false, :nil}
              end
          end
        end

        def alive?(nmid, :worker) when is_bitstring(nmid) do
          alive?(nmid |> Integer.parse() |> elem(0), :worker)
        end
      end # end alive?

        #-----------------------------------------------------------------------------
        # add/3
        #-----------------------------------------------------------------------------
      # @add
      if (unquote(only.add) && !unquote(override.add)) do
        defp add(nmid, :worker, sup) do
          case alive?(nmid, :worker) do
              {:false, :nil} -> start(nmid, :worker, sup)
              {:true, pid} -> {:ok, pid}
          end
        end
      end # end add

        #-----------------------------------------------------------------------------
        # start/3
        #-----------------------------------------------------------------------------
      # @start
      if (unquote(only.start) && !unquote(override.start)) do
        defp start(nmid, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          childSpec = @worker_supervisor.child(nmid)
          case Supervisor.start_child(sup, childSpec) do
            {:ok, pid} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            {:error, {:already_started, pid}} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            error ->
              Logger.error("#{__MODULE__} unable to start #{inspect nmid}")
              error
          end
        end

        defp start(nmid, :worker, sup) when is_bitstring(nmid) do
          start(nmid |> Integer.parse() |> elem(0), :worker, sup)
        end

      end # end start

        #-----------------------------------------------------------------------------
        # start/4
        #-----------------------------------------------------------------------------
        # @start
        if (unquote(only.start) && !unquote(override.start)) do
        defp start(nmid, arguments, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          childSpec = @worker_supervisor.child(nmid, arguments)
          case Supervisor.start_child(sup, childSpec) do
            {:ok, pid} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            {:error, {:already_started, pid}} ->
              :ets.insert(@base.lookup_table(), {nmid, pid})
              {:ok, pid}
            error ->
              Logger.error("#{__MODULE__} unable to start #{inspect nmid}")
              error
          end
        end

        defp start(nmid, arguments, :worker, sup) when is_bitstring(nmid) do
          start(nmid |> Integer.parse() |> elem(0), arguments, :worker, sup)
        end
      end # end start

        #-----------------------------------------------------------------------------
        # remove/3
        #-----------------------------------------------------------------------------

        # @remove
        if (unquote(only.remove) && !unquote(override.remove)) do
        defp remove(nmid, :worker, sup) when is_number(nmid) or is_tuple(nmid) do
          Supervisor.terminate_child(sup, nmid)
          Supervisor.delete_child(sup, nmid)
          :ets.delete(@base.lookup_table(), nmid)
        end
        defp remove(nmid, :worker, sup) when is_bitstring(nmid) do
          remove(nmid |> Integer.parse() |> elem(0), :worker, sup)
        end

      end # end remove

      # @worker_pid!
      if (unquote(only.worker_pid!) && !unquote(override.worker_pid!)) do

        @doc """
          Add worker pool keyed by nmid. Worker must know how to load itself, and provide a load method.
        """
        def worker_pid!(nmid) when is_bitstring(nmid) do
          String.to_integer(nmid)
            |> worker_pid!()
        end

        def worker_pid!(nmid) when is_integer(nmid) or is_tuple(nmid) do
          case alive?(nmid, :worker) do
            {false, :nil} ->
              # Call Server to spawn worker and then fetch results.
              add!(nmid)
            {true, pid} ->
              # Call worker directly
              {:ok, pid}
          end
        end
      end # end worker_pid!

        #=========================================================================
        #=========================================================================
        # Common Call Handlers for SimpleServer
        #=========================================================================
        #=========================================================================
      # @call_load_complete
      if (unquote(only.call_load_complete) && !unquote(override.call_load_complete)) do
        def handle_call({:load_complete, {outcome, details}}, _from, state) do
            state = if outcome == :ok do
              %Noizu.SimplePool.Server.State{state| status: :online, status_details: details}
            else
              %Noizu.SimplePool.Server.State{state| status: :degrade, status_details: details}
            end
            {:reply, :ok, state}
        end
      end # end call_load_complete

      # @call_status
      if (unquote(only.call_status) && !unquote(override.call_status)) do
        def handle_call({:status}, _from, %Noizu.SimplePool.Server.State{status: status} = state) do
          {:reply, status, state}
        end
      end # end call_status

      # @call_generte
      if (unquote(only.call_generte) && !unquote(override.call_generte)) do
        def handle_call({:generate, :nmid}, _from, %Noizu.SimplePool.Server.State{nmid_generator: {{node, process}, sequence}} = state) do
          {nmid, state} = @base.generate_nmid(state)
          {:reply, nmid, state}
        end
      end # end call_generte

      # @call_add_worker
      if (unquote(only.call_add_worker) && !unquote(override.call_add_worker)) do
        def handle_call({:add_worker, nmid}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
          # Check if existing entry exists. If so confirm it is live and return {:exists, pid} or respawn
          response = add(nmid, :worker, sup)
          {:reply, response, state}
        end
      end # end call_add_worker

      # @call_remove_worker
      if (unquote(only.call_remove_worker) && !unquote(override.call_remove_worker)) do
        def handle_call({:remove_worker, nmid}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
          # Check if existing entry exists. If so confirm it is live and return {:exists, pid} or respawn
          response = remove(nmid, :worker, sup)
          {:reply, response, state}
        end
      end # end call_remove_worker

      # @call_fetch
      if (unquote(only.call_fetch) && !unquote(override.call_fetch)) do
        def handle_call({:fetch, nmid, details}, _from, %Noizu.SimplePool.Server.State{pool: sup} = state) do
              {:ok, pid} = add(nmid, :worker, sup)
              response = GenServer.call(pid, {:fetch, details})
              {:reply, response, state}
        end
      end # end call_fetch

        #=========================================================================
        #=========================================================================
        # Common Cast Handlers for SimpleServer
        #=========================================================================
        #=========================================================================

      # @call_load
      if (unquote(only.call_load) && !unquote(override.call_load)) do

        if (!unquote(asynch_load)) do
          def handle_cast({:load}, state) do
            IO.puts "INITIAL LOAD: #{inspect state}"
            {:noreply, state}
          end
        end

        if (unquote(asynch_load)) do
          def handle_cast({:load}, %Noizu.SimplePool.Server.State{status: status} = state) do
            if status == :uninitialized do
              state = %Noizu.SimplePool.Server.State{state| status: :initializing}

              spawn fn ->
                {status, details} = lazy_load(state)
                GenServer.call(__MODULE__, {:load_complete, {status, details}})
              end

              {:noreply, state}
            else
              {:noreply, state}
            end
          end
        end
      end # end call_load

      @before_compile unquote(__MODULE__)
    end # end quote
  end #end __using__

  defmacro __before_compile__(_env) do
    quote do
    end # end quote
  end # end __before_compile__

end