Is this mold in our windows?
The answer is - it doesn't matter.
Whether the debris is grime and dirt or mold...

It won't &quot;hurt&quot; you as there is a glass barrier in between.
There isn't a different way to handle it.
As a typical home owner without industrial tools, you really have no way of resealing your windows.

So basically a seal has failed on your window.   You can either replace that part of the window (frame and all).  For that part you will need to know manufacturing details to assure a proper guide fit.   More than likely only that manufacture (not all cases) can reproduce.
The other option is take that part of the window out, and either take it to a window or glass shop.   They can take window apart and reseal it.   Just FWIW I usually tend to order a replacement as I have found that when getting a window resealed for some reason I don't see the quality compared to the window going through the normal &quot;manufacturing&quot; process.   Usually the costs are close and obviously this is locality and opinion based.   If there was a glass shop down the street that would offer a 5 year warranty I would go with that - mine won't.