How do I use an entity reference field as a sidebar navigation on a view? I have 2 content types: category and project. On project, I have a reference field to category.

I would like to list all my nodes from project in a view and then use all the category nodes as links for filtering this list.

I know it is possible with a taxonomy term, but I'm trying to achieve it with a reference field. Is it possible with the view UI?

I'm using Drupal 8.8.2.
Yes. Doing this with content types is really no different than with taxonomies. You can leverage the reference value with a Views relationship.
You description sounds like it would use two Views, one for the block of navigational links in the sidebar, and the other for the lists of projects.

For the first you just want a view of the categories. I'd use fields and rewrite the output to link to each category title to a url like /projects/{{category_title}}. "Rewrite output" is an option available in the modal for the field. Alternatively, and perhaps more straightforward is to use the option to "output as a link."

The second page display View is based on the project type, and has a relationship to the category. Create a contextual filter to pull the category title from the URL, matching whatever you set up in the block.