RL's policy gradient (REINFORCE) pipeline clarification
For notation and visualizations please take a look at this excellent tutorial Policy Gradients.

For your questions:


The second is correct. In PGs we try to maximize the expected reward. In order to do this we approximate the expectation with the mean reward over samples of trajectories under a parametrized policy. In other words sample actions and get their respective rewards over a period of time within an episode. Compute the discounted return from last step back to the first step (this is your $V_t$ in your notation which usually you will find it as $R_t$ and is the discounted return). Multiply returns with logits and sum. Take a look at slides 8 and 9 of the (1) to see how the REINFORCE is being implemented along with this code examples (lines 59-75).
As you may have already realized, no. It's a return over an episode (multiple timesteps) and is calculated as a discounted sum of all the rewards that you collected. Even if you get 1 at the end and 0 everywhere else the reward is propagated back to the first step (doing it by hand helps a lot!) so all rewards at every timestep are converted into returns from that state and timestep.
Look at slide 13 of (1). Your intuition is correct. The maximum likelihood equation for a classification problem (cross-entropy) multiplied by the return equates the policy gradient loss. If you use a simple Neural Network with REINFORCE with two actions to perform a task you will notice that the gradients propagated back are the same as in a classification task but here are multiplied with their respective return (line 69 2) instead of the class label (0/1). 


A bit of intuition for the 3: This is with accordance with PG methods considered as Model-free methods and map states to actions. If you use a clustering technique to cluster the hidden representations of the last layer of your network (after dimensionality reduction) and color the data points according to which action your network chose, you will find out that the representations are naturally clustered into separate groups depending the action. At a high level, you can state that the REINFORCE performs a type of classification having as label the reward signal.

Hope it helps!