Does the weight of the battery count as "fuel" in a battery powered ultralight?
The jury (i.e. FAA) is still out on this question as of early June 2014.  FAA has indicated it will not give a gasoline "fuel allowance" to be credited against weight for batteries.  Given that fuel weighs 6 lbs. per gallon, or 30 lbs. for the maximum legal gas an ultralight can carry, it would be a somewhat negligible, but certainly welcome, allowance for would-be electric manufacturers.

One company, Chip Erwin of AeroMarine, has imported a barebones ultralight motorglider based on the Goat towed ultralight glider.  It's called the Zigolo, and carries batteries enough for 30-40 mins. of flight, without exceeding the 254-lb. weight limit of the part 103 category.

In time the FAA may well decide to grant the allowance.  It's still not going to amount to much more time, maybe 10 minutes given current battery energy density, which is a fraction of the energy density of gasoline, but research into electric storage is at a feverish pace worldwide.  

Look for sport, then commercial electric aircraft, within the next 5 to 10 years.  Several electric single and two-seat electrics are already flying.  The Airbus ducted fan, recently ballyhooed as the first electric aircraft, which it certainly isn't, is a promising look at one way commercial electric applications might come to market.