SET ARITHABORT, .NET, and explanations inconsistency?
The sql engine is generating you a bad plan.  As a starting point make sure your regular maintenance is getting fresh statistics and you don't have fragmented indexes so the engine is trying to work with valid numbers.  If the basics are solid you are more than likely having parameter sniffing issues.  Put simply sometimes the engine is getting a good plan and sometimes it is making a very bad estimate based on a parameter which gives you a different execution plan done in a terrible way resulting in poor performance.

Arithabort setting does not create this performance issue as you would think looking at your fast execution.  .NET using the setting off and SSMS using the setting on just guarantees you won't use the same plan and it will come up with a new plan probably the correct one.  But if you just turn arithabort on for .NET you will probably run into the same bad estimate issue.  Rather what you want to do is leave arithabort off and then try a sample execution of the query with the setting OFF from SSMS to make it mimic running from code and setting ON to compare to help you discover when you are having a problem.  You may or may not be able to figure out based on the table data where the issue lies for instance we had a table with a pseudo-PK that was 0 half the time and had a value the rest of the time so queries had trouble using it as the key is should have been.

The best way to deal with this once you discover this is a problem is the add the I found a parameter sniffing issue specific hint optimize for unknown.  You should be able to search for that and find the syntax that fits your case.