Shouldn't a signature using ECDSA be exactly 96 bytes, not 102 or 103? Attempting to use openssl to create a signature is confusing on several levels:


If I'm using it to sign a hash that I've already created (HMAC-SHA-384-192, specifically),

a. why must I specify another hash algorithm?

b. What algorithm is consistent with the one in the HMAC and ECDSA-384?
Why does the signature size vary between 102 and 103, instead of being exactly 96?
If I'm supposed to append a binary signature onto a potentially binary message (UDP), how is the recipient going to figure out where a message of unspecified length ends and a signed digest of unspecified length begins?
1) If I'm using it to sign a hash that I've already created (HMAC-SHA-384-192, specifically)
  
  a) why must I specify another hash algorithm? 


HMAC is not a hash algorithm. It's a MAC, a message authentication code, or keyed hash. As the private key is used to create the signature there should not be any need for the secret key used to create a HMAC value. The hash itself is a integral part of ECDSA.


  b) What algorithm is consistent with the one in the HMAC and ECDSA-384? 


You'd expect something like SHA-384, not HMAC-SHA-384-192.


  2) Why does the signature size vary between 102 and 103, instead of being exactly 96? 


The signature consists of two integers. The format of X9.62 creates two ASN.1 INTEGER values for these and puts them within a SEQUENCE. The integer values are encoded as big endian signed values. So if the integers are 9, 17 or 25 bits smaller than the key size then the resulting encoding will be smaller.

Take for instance this 100(!) byte $r$ and $s$ value encoding for a 384 bit curve:

3062022e52884531f76504d28c0a60771025bce8881d33377fce5d62566d7d9a47da55066784b21b4bfcabd105dbe127ea6f0230560212a2b3078c55157842b92476dfaea7b6a58f2e4047d374de53b98629607b70e2ee06a1263c25db66f45572759c2d


decodes to

SEQUENCE(2 elem)
  INTEGER(367 bit) 1938306347072450679876740861736525521572671793660053577404161377925729…
  INTEGER(383 bit) 1323785767677292921898744777970756602439423500858315154875512504309946…


In principle they could also be slightly larger as a leading 00byte needs to be added for positive values where the very first bit is 1. This is because the initial bit is the sign bit, so if the first bit is set then the value would be interpreted as negative. The order of the curve however could well prevent this from happening.


  3) If I'm supposed to append a binary signature onto a potentially binary message (UDP), how is the recipient going to figure out where a message of unspecified length ends and a signed digest of unspecified length begins?


If you just concatenate both then you may not have a deterministic method of doing so. So basically you have to include a length of your message or signature.

You could also convert the signature to a signature that does contain two statically sized integers of course. The encoding of both "Plain Format" integers and "X9.62 Format" signatures is discussed in TR-03111 of the German BSI, section 5.2. You may want to reencode them back to ASN.1 DER format after the message is received and parsed of course, in case your API doesn't support Plain Format signatures.

In that case the signature size will be 96 bytes.