Should I multiply by time to determine Watt-hours?
The term "watt-hour" (Wh) is often not correctly understood. Watt (W) is a unit of power. Energy is measured in Joules (J) and is related to power by
$$
E = {P_\text{avg}}\times \Delta t,
$$
where \$\Delta t\$ is time in question and \$P_\text{avg}\$ is the average power spent during that time. 

With only two measurements, one at the start and one at the end, the best guess is that the average power was
$$
P_\text{avg} = \frac{P_\text{start} + P_\text{end}}{2}
$$

Notice that "watt-hours" is power multiplied with time, so it is a measure of energy. This is what power companies really sell you, and what they should charge you for; though that's for some reason not always the case.

Joules is the same as a "watt-second", so 
$$
1\;\text{Wh} = (1\;\text{W})\times(1\;\text{h}) = (1\;\text{W})\times(3,600\;\text{s}) = 3,600\;\text{J}=3.6\;\text{kJ}
$$

Sidenote

If you know calculus, then a more accurate formula for the relationship between power and energy is
$$
E = \int_{t_\text{start}}^{t_\text{end}} P(t)\;dt,
$$
or even
$$
P(t) = \frac{dE(t)}{dt},
$$
which says that power is the rate-of-change of energy consumption. You can make use of this, if you know the sample-rate of your measurements, for instance by using an ADC from a micro-controller. If the time between samples is dt, then the total energy can be calculated by:

loop() {
  current = currentADC.sample();
  voltage = voltageADC.sample();
  time = time_now();
  dt = time - time_last;
  power = current * voltage;
  energy += power * dt;
  time_last = time;
}