Is using /dev/hwrng for raw key material secure? Is using /dev/hwrng for raw key material secure (as long as you trust the hardware)?
I tested it using the rngtest tool and it has good output.
Isn't that a contradiction in terms?
/dev/hwrng could be a  block device to anything at all.  If you trust the hardware, then yes. If you don't' trust the hardware, then no. The thing is that you can't tell. Nano scale technology has to be taken on trust. See Can any part of Intel's (RdRand) random number generator be visually inspected?

rngtest

Irrelevant. The Mersenne Twister is a pseudo random number generator. And it will ace most randomness tests. Yet it's deterministic depending on the seed, and can be inverted by only looking at 624 outputs. You could simply reseed it with every restart and it will look really random. Like seeding with the time. Yet it will offer no cryptographic security. Prove that your hardware if not just:-
$$ \text{E}_{k  \oplus  id} (x) $$
where k = some shadowy agency’s key, id = a device identification/serial number and x is some counter and/or time. Perhaps using AES, or some other CSPRNG? Some form of NIST 800-90A HMAC_DRBG? It can’t be mathematically disproved by any analysis of the output data.
How can you confirm that what your generator is giving you is truly random? You can't. Last year's US intelligence budget was $60 billion. What did they spend it on?

Trumpet sounds! Build your own. It's relatively easy and the only way to have 100% certainty.