Is there value in performing signature during key exchange? I looked around the net, however I couldn't see anything authoritative on this.

If one is using something like libsodium to perform key exchange (using x25519), is there value in using Ed25519/signature, to sign the public keys being exchanged for use in the key agreement scheme (assuming the ed25519 keys are correctly tied to an entities identity)?

Or is this over-engineering?

I have static keyPairs tied to user identity in my app, and see some value in signature despite the extra effort to sign public keys being exchanged during symmetric key agreement.
As for the first question: as long as you create a signature over some or all data that is required and unique in the key establishment protocol then you should get entity authentication for that session. If you sign entire messages then you may be able to avoid some attacks on the message format itself. Some protocols deliberately include the public key into that calculation, others don't. Including the public key is not strictly required of course, you can also sign over a nonce used in the protocol or even over the shared secret (although I would seriously frown on the latter, as you don't want to use the key material for anything other than what it should be used for).

I'd also make sure that the established secret is correct by validating the generated secret (derive a MAC key and use that to create a MAC over some or all data unique to the session); you don't want to be relying on the messages themselves (implicit authentication) to verify the data and shared secret, use explicit authentication instead. It might be a good idea to start hashing the messages immediately and perform the signing / MAC'-ing over the calculated hash for maximum efficiency.

Yes, you can achieve forward secrecy with libsodium, if - as always - you make sure you destroy the keys as soon as they are not needed anymore. Just performing ephemeral-static DH - using whatever scheme - doesn't cut it, as you must make sure that the key material is not available anymore. Fortunately, Daniel is known for trying to make his protocol implementations in such a way that key destruction is easy.