Visualizing word embeddings
It seems there are a couple issues at work here.

First, apparently the PCA computation wasn't stalling out because of the absurd dimensions being reported - updating to tensorflow 1.12 and Keras 2.2.4 has eliminated that problem.

Second, it seems that Keras bases the generation of embedding data for tensorboard on the layer inputs rather than the actual layer weights. Maybe this is important for some other usage, but it makes things very complicated in a standard word/token embedding scenario. To work around this, I ended up skipping embeddings during training, exporting the embedding weights at the end, and then training a new throwaway model with a much simpler topology to generate the projections I'm interested in. Given an object model trained from the samples above:

w &lt;- model %&gt;% get_layer(index = 1) %&gt;% get_weights()

emodel &lt;- model &lt;- keras_model_sequential() %&gt;%
  layer_embedding(input_dim = max_features, output_dim = 128, input_length = 1) %&gt;%
  layer_global_max_pooling_1d() %&gt;%
  layer_dense(units = 1)

emodel %&gt;% get_layer(index = 1) %&gt;% set_weights(w) %&gt;% freeze_weights()

emodel %&gt;% compile(
  optimizer = "rmsprop",
  loss = "binary_crossentropy",
  metrics = c("acc")
)

callbacks = list(
  callback_tensorboard(
    log_dir = "my_log_dir",
    embeddings_freq = 1,
    embeddings_data = c(1:max_features)
  )
)

history &lt;- emodel %&gt;% fit(
  1:max_features, 1:max_features,
  epochs = 1,
  batch_size = 256,
  callbacks = callbacks
)

tensorboard("my_log_dir")             


Ideally there would be some simpler/cleaner way to achieve this, but this works for now.

edit

Another user asked in a now-deleted question about adding token labels to Projector. For posterity, that is as simple as adding a value for embeddings_metadata in the callback_tensorboard. This should be a path (relative to your log_dir) to a tab delimited file with the desired labels. Details are here: https://www.tensorflow.org/guide/embedding#metadata