BUG: Transformer Second Layer Access
pretrained_weights='bert-base-uncased'
tokenizer = tokenizer_class.from_pretrained(pretrained_weights)
model = model_class.from_pretrained(pretrained_weights)

`features = []
input_ids = torch.tensor([self.tokenizer.encode(phrase, add_special_tokens=True)])
with torch.no_grad():
    # Models outputs are tuples
    outputs = self.model(input_ids)
    last_hidden_states = outputs[0]
    avg_pool_hidden_states = np.average(last_hidden_states[0], axis=0))
    return avg_pool_hidden_states`
I am working on Sentence Similarity problem, given sentence S, find similar sentences W. So I want to encode sentence S and find the closest top-K sentences from W.

I have read the documentations, but I want to confirm few things how do I do. the following:

How do I get the second to last layer?
Am I average pooling the last hidden states correctly?