Argyle reported the 27-year-old midfielder to the FA after claiming one of their ball boys was "shoved to the ground" in the 86th minute of the game.
The incident was not seen by the match officials, but was caught on video.
The FA has submitted a claim that the standard punishment, a three-match ban, would be "clearly insufficient".
Kelly has until 18:00 GMT on Friday, 17 February to respond to the charge.
Leyton Orient captain Liam Kelly has been charged with violent conduct by the Football Association for an incident in Tuesday's win at Plymouth.