HMAC SHA256 for deriving session keys using random nonce
Yes, it is secure, as the KDF is secure, as the answers to the question you linked to indicates.

However, there are some things that you should keep in mind:


The result of the KDF is not validated; if the initial shared key is incorrect you will only find out once the session keys are used. This is tricky to implement if you want to use the session key for authentication (you may want to use explicit authentication rather than implicit authentication during the session).
Both parties now share a single session key, so you may need to be careful when using encrypt-then-MAC.
Both parties use share a single session key, possibly for communication in both directions, which means you can have messages replayed not just from $A$ to $B$, but also from $A$ to $B$ and then replayed back to $A$.
The scheme will not provide any forward secrecy that ephemeral Diffie-Hellman could provide.


You can e.g. use a single byte tag in addition to calculate both an encryption key and a MAC key, possibly a third one to create a value that can be exchanged to authenticate the entities.

You could include the identities $ID_A$ and $ID_B$ in the MAC calculation, e.g. $ID_A || ID_B || R_A || R_B$ for sending from $A$ to $B$.