Can I put small nails through plexiglass?
Fine nails are fine. But you do need to drill a clearance hole the full diameter of the nail shaft through the plastic. You may want to drill a smaller pilot hole at least partway into the wood to reduce the chance of splitting.

Any standard drill bit will do, but go very slowly to avoid melting the plastic as you drill.

Drill bits tend to walk on plastic, so you may want to put a piece of masking tape over each area to be drilled, which will give a bit of grip to avoid the slide.

Screws would also be fine (and a bit stronger; maybe overkill), but the same pilot hole rules apply.