Is it possible to create sub-menus for Services?
There is no API to perform this currently.

Items in the Services Menu


The items in the Services menu are commands categorized by what type of data they operate on. The application icon of the application providing a service appears to the left of the service’s name in the menu. If two or more applications provide a service with an identical name, the name of the application providing each one is appended in parentheses after the name of the service in the menu.
The Services menu is populated when the menu is opened.

I suppose you could hack the OS (assuming SIP is not enabled) to add that functionality, but it's not something that's publicly available or documented widely that a private API exists to do what you ask.