title: Use TensorBoard with Keras Tuner
body: <p>I ran into an apparent circular dependency trying to use log data for TensorBoard during a hyper-parameter search done with <a href="https://github.com/keras-team/keras-tuner" rel="noreferrer">Keras Tuner</a>, for a model built with TF2. The typical setup for the latter needs to set up the Tensorboard callback in the tuner's <code>search()</code> method, which wraps the model's <code>fit()</code> method.</p>
  
  <pre><code>from kerastuner.tuners import RandomSearch
  tuner = RandomSearch(build_model, #this method builds the model
               hyperparameters=hp, objective='val_accuracy')
  tuner.search(x=train_x, y=train_y,
               validation_data=(val_x, val_y),
               callbacks=[tensorboard_cb]
  </code></pre>
  
  <p>In practice, the <code>tensorboard_cb</code> callback method needs to set up the directory where data will be logged <em>and this directory has to be unique to each trial</em>. A common way is to do this by naming the directory based on the current timestamp, with code like below. </p>
  
  <pre><code>log_dir = time.strftime('trial_%Y_%m_%d-%H_%M_%S')
  tensorboard_cb = TensorBoard(log_dir)
  </code></pre>
  
  <p>This works when training a model with known hyper-parameters. However, when doing hyper-parameters search, I have to define and specify the TensorBoard callback before invoking <code>tuner.search()</code>. This is the problem: <code>tuner.search()</code> will invoke <code>build_model()</code> multiple times and each of these trials should have its own TensorBoard directory. Ideally defining <code>log_dir</code> will be done inside <code>build_model()</code> but the Keras Tuner search API forces the TensorBoard to be defined outside of that function.</p>
  
  <p><strong>TL;DR:</strong> TensorBoard gets data through a callback and requires one log directory per <em>trial</em>, but Keras Tuner requires defining the callback once for the entire <em>search</em>, before performing it,  not per trial. How can unique directories per trial be defined in this case?</p>