Is it legal to begin pilot training on a jet?
There is nothing in the US regulations and no references that I can find that prohibits a pilot from learning to fly in a jet aircraft.  There are multiple systems on a jet aircraft that are not found in small GA aircraft and the same systems are more complex.  A lot of professional pilots who fly the same jet for a career don't fully understand the different systems on their jet.

The issue isn't with the FAA though; it is with the instructor and insurance companies.

I certainly would not agree, as a CFII, to train a new student pilot in a jet.  I personally don't want to take on the liability for the payment I would receive for my services.

Most insurance companies will not ensure a student pilot to learn to fly in a jet.