Does hash_df use binary or an ASCII hexadecimal representation for numbers to be passed to the hash function? I'm implementing the HASH_DRBG algorithm.
As per NIST-SP800 90 document for HASH DRBG Generation, section 10.4.1 "Derivation Function Using a Hash Function (Hash_df)", the Hash_df algorithm contains this step 4.1:


  temp = temp || Hash (counter || no_of_bits_to_return || input_string).


I believe || here represents "append" operation. According to the algorithm, do I have to make the append operation and hashing on an ASCII hex string or a binary string?

Example, I can do 
Hash  ("000102030405060708090A0B0C0D0E0F101112131415161718191A1B1") where the input is ASCII string characters. Or I can do Hash(&amp;bytearray) where bytearray stores the non-ASCII numbers in binary?

Which one is expected as per the algorithm? I am planning to use SHA-1 or SHA-256 as Hash here.
In step 4.1 of the algorithm:


Counter is expected to be a 1 byte binary value
no_of_bits_to_return is a 4 byte bigendian binary value
input_string is a variable length binary value
$||$ is concatination (just like you guessed)


So, Hash(&amp;bytearray) would be appropriate.

The trickiest part is the 'no_of_bits_to_return' -- that's bigendian (and so a value of 24 would be represented by the four bytes 0x00 0x00 0x00 0x18), which is the opposite of how the most common processors internally represent a 4 byte value.

Now, if you were to convert to (say) ASCII hex first, well, it probably won't affect the security of the algorithm; however, it won't be the HASH_DRBG algorithm, but something else.