How likely is fragmentation on a table with 40000 products likely to affect performance
Based on the data that you have provided - 

there are 67 total pages. Index fragmentation for such a small table would not affect the performance.  I would not worry about index fragmentation for the table that you have mentioned.

You should update your table statistics so that sql server can generate better query plan.

I would start my troubleshooting by   


Running sp_Blitz: Overall Health Check


Read : Stop Worrying About SQL Server Fragmentation