How to create a feature vector given final set of feature maps?
After asking around about this, it seems the third option is the standard: take the mean of each feature map, and create a 2048-element feature vector. The search term for this is global pooling. That's what we are talking about that is the terminology I was missing.

Global average pooling is good b/c it reduces the dimensionality before classification. Also, by the time you have reached this level of abstraction in your feature extractor, you are probably not that interested in the finer-grained spatial aspects of the features because you are already representing a very abstract "object" (depending on how deep your feature extractor is) after a great deal of pooling of your original image.

That said, if you are worried about losing spatial information you can always try one of the other two options. I'd be curious if anyone has tried this if they have experiences to share about how they went about it.

Also, for a really nice summary from a paper that addresses this topic, see this answer from crossvalidated:
https://stats.stackexchange.com/a/308218/17624

IN particular:


  Conventional convolutional neural networks perform convolution in the
  lower layers of the network. For classification, the feature maps of
  the last convolutional layer are vectorized and fed into fully
  connected layers followed by a softmax logistic regression layer. This
  structure bridges the convolutional structure with traditional neural
  network classifiers. It treats the convolutional layers as feature
  extractors, and the resulting feature is classified in a traditional
  way.
  
  However, the fully connected layers are prone to overfitting, thus
  hampering the generalization ability of the overall network. Dropout
  is proposed by Hinton et al as a regularizer which randomly sets half
  of the activations to the fully connected layers to zero during
  training. It has improved the generalization ability and largely
  prevents overfitting.
  
  In this paper, we propose another strategy called global average
  pooling to replace the traditional fully connected layers in CNN. The
  idea is to generate one feature map for each corresponding category of
  the classification task in the last mlpconv layer. Instead of adding
  fully connected layers on top of the feature maps, we take the average
  of each feature map, and the resulting vector is fed directly into the
  softmax layer. One advantage of global average pooling over the fully
  connected layers is that it is more native to the convolution
  structure by enforcing correspondences between feature maps and
  categories. Thus the feature maps can be easily interpreted as
  categories confidence maps. Another advantage is that there is no
  parameter to optimize in the global average pooling thus overfitting
  is avoided at this layer. Futhermore, global average pooling sums out
  the spatial information, thus it is more robust to spatial
  translations of the input. We can see global average pooling as a
  structural regularizer that explicitly enforces feature maps to be
  confidence maps of concepts (categories). This is made possible by the
  mlpconv layers, as they makes better approximation to the confidence
  maps than GLMs.


Which is from this paper: https://arxiv.org/pdf/1312.4400.pdf