<p dir="auto">This issue is based on the suggestions in <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="44747268" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/1553" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/1553/hovercard" href="https://github.com/kubernetes/kubernetes/issues/1553">#1553</a>. I thought getting a clean overview for a renewed discussion was worth the additional issue.</p>
<p dir="auto">Similar to secret distribution <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="47034637" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/2030" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/2030/hovercard" href="https://github.com/kubernetes/kubernetes/issues/2030">#2030</a> there is a need to dynamicaly configure applications. Producing or changing configuration scripts via cat, sed or other "hacky" tooling is a method I dislike and I would love a cleaner way of supporting this.</p>
<h3 dir="auto">Option 1: <a href="https://github.com/hashicorp/envconsul">envconsul</a>/<a href="https://github.com/mattn/etcdenv">etcdenv</a></h3>
<h4 dir="auto">Pros:</h4>
<ul dir="auto">
<li>Injects etcd/consul values as ENVs</li>
<li>Easy way to switch from ENVs to etcd/consul</li>
</ul>
<h4 dir="auto">Cons:</h4>
<ul dir="auto">
<li>Doesn't actually solve config generation</li>
</ul>
<h3 dir="auto">Option 2: <a href="https://github.com/kelseyhightower/confd">confd</a></h3>
<h4 dir="auto">Pros:</h4>
<ul dir="auto">
<li>Supports various backends (consul, etcd, ENVs)</li>
<li>Supports almost any config structure as it is template based</li>
<li>Easy templates using Golang Text Templates</li>
<li>Support for one-time and watch (etcd/consul)</li>
<li>Command line flags to bootstrap confd config</li>
</ul>
<h4 dir="auto">Cons:</h4>
<ul dir="auto">
<li>Template based</li>
<li>No encrypted value support -&gt; crypt or just encrypted volumes <a class="issue-link js-issue-link" data-error-text="Failed to load title" data-id="47034637" data-permission-text="Title is private" data-url="https://github.com/kubernetes/kubernetes/issues/2030" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/2030/hovercard" href="https://github.com/kubernetes/kubernetes/issues/2030">#2030</a>?</li>
</ul>
<h3 dir="auto">Option 3: <a href="https://github.com/spf13/viper">viper</a></h3>
<h4 dir="auto">Pros:</h4>
<ul dir="auto">
<li>Supports encrypted values via crypt</li>
<li>No templates</li>
<li>Easy syntax</li>
<li>Command line flags to bootstrap viper config</li>
</ul>
<h4 dir="auto">Cons:</h4>
<ul dir="auto">
<li>No ini file support</li>
<li>Not as easy for complex configuration files</li>
</ul>
<h3 dir="auto">Option 4: <a href="http://augeas.net" rel="nofollow">augeas</a></h3>
<h4 dir="auto">Pros:</h4>
<ul dir="auto">
<li>Wide range of config formats supported</li>
</ul>
<h4 dir="auto">Cons:</h4>
<ul dir="auto">
<li>Complex syntax especially for complex configurations</li>
</ul>
<h3 dir="auto">Option5: <a href="https://github.com/markround/tiller">tiller</a></h3>
<h4 dir="auto">Pros:</h4>
<ul dir="auto">
<li>Supports almost any config structure as it is template based</li>
<li>Ability to "pipe" json configs into a container instead of ENV files</li>
</ul>
<h4 dir="auto">Cons:</h4>
<ul dir="auto">
<li>Template based</li>
<li>ERB templates -&gt; more complex than necessary in my opinion</li>
<li>Can use container provided .yaml files -&gt; could prevent image sharing</li>
</ul>
<h3 dir="auto">Clarification:</h3>
<h4 dir="auto">Confd:</h4>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/kelseyhightower/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/kelseyhightower">@kelseyhightower</a><br>
Is a config file needed for confd to work or could everything be done using commandline flags?<br>
Can multiple config files be generated, if yes how?</p>
<h4 dir="auto">Viper:</h4>
<p dir="auto"><a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/spf13/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/spf13">@spf13</a> <a class="user-mention notranslate" data-hovercard-type="user" data-hovercard-url="/users/bketelsen/hovercard" data-octo-click="hovercard-link-click" data-octo-dimensions="link_type:self" href="https://github.com/bketelsen">@bketelsen</a><br>
Is a config file needed for viper to work or could everything be done using commandline flags?<br>
-&gt; Config file and various other configuration possibilities for viper are optional. It can be used straight via commandline flags.</p>
<p dir="auto">Overall I think the way to go would be to use confd with ENV as default backend. Furthermore containers should be able to understand a backend switch ENV so that switching to etcd/consul is easily done on a per container level at runtime. This enables easy testing and environment switching. With the addition of watchable configuration via confd more dynamic setups are possible.</p>
<p dir="auto">Looking forward to your thoughts and suggestions.</p>
<p dir="auto">Kubernetes version 1.0.6</p>
<p dir="auto">This setup involves pods for Cassandra (2-nodes), Spark (master, 3-workers, driver), RabbitMQ(1-node). Some pods that are not under replication controller die after a day and never restarted automatically. I found out that  *influxDB' monitoring was using too much of CPU and the pods dying are from that node.</p>
<p dir="auto">Please notice the <em>restartPolicy: Always</em> is the default as I have not explicitly specified it. With this I was expecting manually started pods to restart on failure similar to the ones under replication controller</p>
<p dir="auto">Once the pod died, there was no way to run "kubectl.sh describe pod-is* on it. Could not find any debugging information on the failure. This is very much duplicatable if I switch on the  cluster monitoring (with ENABLE_CLUSTER_MONITORING; true by default)</p>
<p dir="auto">Related threads with more info/discussion<br>
<a href="https://github.com/kubernetes/kubernetes/issues/4455" data-hovercard-type="issue" data-hovercard-url="/kubernetes/kubernetes/issues/4455/hovercard">Influxdb is using 1 core on 4 core cluster</a><br>
<a href="http://stackoverflow.com/questions/33068639/kubernetes-pods-some-die-after-running-for-a-day" rel="nofollow">Kubernetes pods (some) die after running for a day</a></p>