What's this plastic thing along my sidewalk pavers?
That is called Paver Edging. It is used to keep the pavers tight or to separate one section from the other. It's usually covered to the point that you only see a small strip of it. However, it looks like the installer didn't quite finish the job. I would probably do as you suggested and bury the edging deeper. You could also take up those few bricks that are outside the gate, level that area a little better and place those bricks closer to the edging for a more uniform look and to stabilize that immediate area.