Does signing message weakens the address' security? Will signing message (without doing any $ transaction) reveal the private key to the blockchain just the same/similar way the private key is revealed to the blockchain when you do a $ transaction?

The point is, doing a transaction weakens the security of the address a tiny bit because the private key is revealed to the blockchain. That's one reason why change/new address is used with every transaction.

But what about just signing message? Will that weakens the security of the address a tiny bit too?
Public / Private key encryption requires that the private key is never revealed - the owner must keep it secret.

Message and transaction signing is done with the private key and verified with the associated public key. So the private key is never revealed to the blockchain. Note that public key here refers to the the full ECC public key, and not a Bitcoin address - which is a hashed and transformed representation of the public key. The public key can only be derived from a Bitcoin address if the SHA256 and ripeMD-160 hashing algorithms are ever broken. This question explores the challenge.

To answer your question: Yes, each new signature produced by a private key may bring an attacker, who already knows the associated public key, closer to discovering the private key.

Comments to this answer have informed an updated position:

Although there is no provable security for ECDSA, it is commonly assumed that, even with knowledge of multiple signatures produced by a particular key, an attacker will not have improved odds at deriving the associated private key or forging its signature.

It is an almost impossible task to discover a Bitcoin private key from its public key. An answer to this question illustrates the challenge mathematically.

Some people advise an attitude of "better safe than sorry" - that public keys should not be revealed, where possible. Yet, caution should not discourage practical utility - as pointed out in the comments below.

Best practice, as you say, is to  frequently change keys and to preferably use a new address for every receive transaction. If you want someone to be able to verify a signature they will, of course, need your public key.

As far as I'm aware, the only Bitcoin transaction type that reveals public keys is Pay-To-Public-Key-Hash (P2PKH) which contains the full public key in its script.