Can you use a home built helicopter for solo hours?
I assume that this is for a Private Pilot - Helicopter license. I also assume you mean something like larger kit aircraft, not ultra-light helicopters like the Mosquito.

You can do this, but there are other conditions that must be met:


The aircraft must have completed it's flight test phase (this is usually the first 25-40 hours of operation)
The CFI must be willing to provide instruction in the aircraft. It will be difficult because either the CFI is not familiar with the aircraft or insurance won't cover them, some just won't fly homebuilts.
Your aircraft insurance must allow primary flight training in the aircraft (not many do)
The aircraft must meet requirements such has having a radio, transponder, etc for being able to operate in required airspaces.


The big ones will be finding a CFI willing to do this, and getting insurance to cover you. The insurance alone may outweigh the cost of renting a certificated aircraft.