How should one cross the IJ river with a bike?
The route that you highlighted on the map is the ferry route. It operates 24/7. It allows bikes and it is completely free.

More info about ferries in Amsterdam.

There is also bridge on the east of Amsterdam.

And there are plans to build bike tunnel close to IJ tunnel.