Is it alright to skip problem solving sessions during coursework?
Yes, skipping the sessions is not only alright, but is in fact advisable. Will it be viewed adversely during grading, you bet yeah. The key thing to remember is that in grad school grades don't matter. If skipping the session means you can put an extra hour into your research, then that is better time spent.

The way it will affect you in grading is you might lose some participation points and the benefit of the doubt for borderline grades. The instructor may be less inclined to write you a letter of recommendation, but ideally your letter writers will be talking about your research.