Should I get CE mark on my Arduino project?
Simple answer is: YES.
If you want to sell a product in the European common market, then you need to have it CE certified. Since your product has a radio module in it, then the regulation you should abide to is the RED (Radio Equipment Directive).
You can find more information about the RED here.
There's links there for the applicable regulations and standards. The RED guide, which can be found here can also be useful.
Being very short you'll need to make three kinds of certification tests:

EMC (EN 301 489-17)
Safety (EN 60950-1 + EN 50371)
Radio (EN 300 328)

By the way, due to Brexit, since January this year, if you want to sell in the UK, now you also need the UKCA marking as well.
You have to find a way to perform these tests and provide proof of compliance. You can seek a certification company to help you out with the tests and the necessary documentation.
There are some exceptions depending if you want to sell your product as some kind of development platform. But that's a greyzone, there's no simple answer if want to follow that path. Besides, it's not without necessary due diligence and needed documentation.