Does my cell phone alarm run off the battery or does it need cell service
Yes, it will.

The default alarm is built into Android and uses the alarmManager to schedule when to sound the alarm.

It will always go off, unless the device is powered off or in silent mode (however some ROM's have separate volume controls for alarms and other volumes).