Access denied after contribution (thank you page denied)
Ok, thanks a lot Demerit but I could not get what I should do in there and finally found the solution. 

So the trick was with the capabilities. 
I installed user role editor and found out that by default, a subscriber is granted way more capabilities than just "read", which are : 

access_all_custom_data
access_civimail_subscribe_unsubscribe_pages
access_uploaded_files
make_online_contributions
profile_create
profile_edit
profile_listings
profile_view
read
register_for_events
sign_civicrm_petition
view_event_info
view_public_civimail_content


So basicaly if your custom roles are not granted those capabilities, users will get denied the access to some civi page at some point. 

So in wordpress I just went Users > User role Editor and updated my custom roles with the right capabilities.

Hope it helps !