What are these little bugs?
These are very likely mites (notice their small size and 8 legs). Based on their size, appearance, and abundance, I'd guess they're bird mites (family Macronyssidae). Since they're near your window, I assume you either have a bird nest outside your window or frequent bird visitors to your window. 



 Credit: user Aewills on bugguide.com 

you can find existing posts about bird mites on Bio.Stackexchange below:


Help identify this mite?
Bug Identification: Tiny Bugs
Tiny white very hard cell bug! Please help to identify!