Can a client leave its mark on the blockchain? Would it be possible for a client (eg the core client) to leave its mark in the form of META data on the blockchain or even in the mempool so that we know that this client was used to create or send that transaction? 

This is very similar to what web browsers do (for example this website tells you all about your browser by reading some META info: https://www.whatismybrowser.com/)?
The OP_RETURN feature can be used to accompany a transaction with arbitrary data.  A standard transaction can contain an OP_RETURN output with up to 80 bytes of data (40 bytes prior to Bitcoin Core 0.11).

You could have a client that includes an OP_RETURN output with every transaction, containing whatever meta data is desired (client software version, info about client settings, friendly greetings, Bible verses, etc).  

However, since this would make each transaction larger, the users of this client software would on average pay higher transaction fees (depending on how often the extra data pushes your transaction past a 1 KB boundary, since the fee is per KB).  Also, since every full node has to store the extra data forever, some people might consider this feature to be "unfriendly" to the block chain, even though it is legal in the protocol.

Given these tradeoffs, one would probably not implement such a feature without a pretty compelling reason to do so, beyond simple curiosity.