Can I land my aircraft on the grass next to the runway at a public airport?
It depends on the airfield. Landing on grass is only recommended if the ground is maintained to a reasonable standard and clear of obstructions. Some airfields have grass strips maintained for that purpose, on others the grass has lights, wires and cables, or just lots of holes/ruts which will be far worse than the tarmac. 

Some airfields are only grass, tarmac is not a prerequisite for having an airfield.

You want to know before you go, how to find out depends on where you are. There are flight guides and websites in many countries that tell you what is available, but it always makes sense to call ahead and ask the condition of the available grass, and to find out if there are any unofficial grass areas you can use. 

There is also ATC, at a towered airfield you must land where you are cleared, you can'd decide to land on the grass nearby because you want to as it's a violation. If there's grass available you need to make sure you ask for it and they will give it to you if they can, most of the time ATC is helpful. If they say no there's probably a good reason. At an untowered airfield it's at your discretion, if there's grass available and it's in decent condition you would state your intention on the frequency and follow normal procedures.