Running Power Functions Train Without IR Receiver/Remote
There is a fairly easy and cheap way to do this since you don't want real variable speed control. As you noted, connecting the AAA battery box directly to the PF train motor technically works, but the motor runs too fast.

The default AAA battery box provides 9v from 6 AAA cells. Lowering this voltage will lower the speed of your motor. If you'd like to get your hands dirty, you can modify the AAA battery box or buy a battery box that supplies another voltage and connect this up to the C0 and C1 lines of your motor.

However, the simplest solution I can think of is to purchase (or possibly build) AAA dummy batteries and use a few of these in the AAA battery box in conjunction with regular batteries. These look like AAA batteries, but are actually just metal conductors internally. By using a number of these in the battery box, you can have it provide 1.5v, 3v, 4.5v, 6v, 7.5v, or 9v.

Philo has a nice breakdown of speed vs voltage for PF train motors:

3V = 135 rpm (2 AAA batteries, 4 dummies)
4.5V = 468  rpm (3 AAA batteries, 3 dummies)
6V = 792 rpm (4 AAA batteries, 2 dummies)
7.5V = 1107 rpm (5 AAA batteries, 1 dummies)
9V = 1458 rpm (6 AAA batteries, 0 dummies)


You can make adjustments in 1.5 volt increments by adding or removing batteries from your setup to hopefully find a speed that works for you.

It's a bit more complicated/expensive to get proper variable speed control, but if you just want a constant speed that is slower than the default PF full speed, this is the simplest method I can think of.