Often times I edit a file - think my work is done - then close the file and move on to the next thing. But then I realise I need to go back to that code and tweak it a bit more.  

If I open the file again, I can use the `.` mark and jump to the position of the last edit with `'.`.  

Now I'm trying to find a way to use vim's marks to jump to locations in files I have recently visited, but not changed.  

I know I can set a mark manually, but most of the time, I don't expect to come back and so I don't set a mark and need to use a mark that vim generates automatically.  

The `"` mark seems like this could be a way to achieve this, from `:h motion`. 

> 
> `"` To the cursor position when last exiting the current
>  buffer. Defaults to the first character of the first
>  line. See |last-position-jump| for how to use this
>  for each opened file.
>  Only one position is remembered per buffer, not one
>  for each window. As long as the buffer is visible in
>  a window the position won't be changed.
>  {not in Vi}.
> 
> 
> 

However if I

* close all windows showing a buffer (but the buffer is still present in the buffer list). or:
* kill all instances of the buffer

the mark doesn't seem to update, how can I use a mark to jump to a location of the last cursor position in the buffer/file?