How does an Image filter really sweep an Image?
I assume by image filter sweeping, you mean the image is convolved with a kernel.  In that case, the method is more similar to what you show in your second image.  

Conceptually, the NxM kernel is multiplied by an NxM block of pixels in the image, and the results are summed to a single value-- this is the first value in your 2D result array.  

Then the next NxM block of pixels is taken by offsetting the window one pixel to the right, the kernel is multiplied by this block, the results summed, and this is the second value (top row, 2nd from left) in your result array.  

This continues until you reach the end of the row, then you offset down one pixel and start again at the beginning of the row.  This result is then the first result in the second row of your result array.

And so forth.

Of course, it doesn't have to be done in order, and many optimizations or parallel programming techniques may perform better if you don't step through sequentially overlapped like this, but conceptually this is what's happening-- take a block, convolve, get one pixel output, put it into the result for that offset.

There is also the question of how to handle edges-- if you want the convolution result to be the same size as the input image, you need to assume values for pixels outside the border of the actual image.  Zero is a common assumption. 

Considering edges, and in reference to the question in the comments, where you start the kernel depends on what exactly you want out.  First, different kernels may have different "origins"-- typically the result of the kernel calculation is referenced to the center of the kernel, but that's not a requirement (and can't happen if the kernel is an even number of units across).  

Then the tradeoff needs to be resolved between the size of the output and the validity of the data.  If you want an output image of the same size as the input image, then starting (3,3) of the kernel at (1,1) in the image, assuming 1 based indexing, will give you that result.  The challenge is what value to use for parts of the kernel that don't overlap the image.  For example, if you are using a kernel to average a neighborhood of pixels, and you assume 0 for values outside the image, then the edge-most pixels of your result image will darken because you are averaging in those zeros with the existing data.  That may be reasonably harmless.  

Other kernels, however, may give bad data at the border if you assume values outside the image.  Assuming zero outside the image when you're taking a picture of sky could cause "phantom" edges to appear with an edge detection kernel, for example, and this could cause problems later in your processing pipeline.  In this case you might prefer to start with (3,3) of your kernel at (3,3) of the image, and then your result image will be 4 pixels narrower and 4 pixels shorter.