Is UL approval only required for the power supply?
UL does not approve anything. They publish standards, maintain a list of  products that they have tested and/or evaluated to determine that they meet those and other applicable standards, and authorize products that have been listed to be marked with the UL label. In the USA, they are one of several nationally recognized testing laboratories (NRTLs) that provide a similar service. Other NRTLs evaluate products to UL standards, list the products and authorize their own label. In the USA, "approved" means acceptable to the authority having jurisdiction.

There appears to be a standard for information technology equipment (ITE) that allows it to be powered by a listed power supply designed for such equipment without requiring the ITE to be listed.

Golf Carts

In the USA, the standard for golf carts seems to be ANSI Z130.1. There may be NRTLs that will certify conformance with that or manufacturers may assert conformance based on their own evaluation. The standard seems to apply to all aspects of electric motor and engine driven carts.