Preparing concrete ceiling for painting
Paint over it.

Both the wood and the concrete will pass moisture more or less evenly, so there is no reason to expect that wood chip to pop out later.  And if it does, it's not exactly the end of the world.  Doing the extra work to remove it now isn't going to make your life any easier...

You might hit it with some sand paper on a sanding block, as it loos like there is sawdusrt along with the wood chips.  I would expect some of it to fall out, but probably not the larger chips.  The sawdust falls into the category of "loose dirt" potentially, so knocking off loose dirt is a good thing.