Can a bitcoin transaction hold an arbitrary string or strings for reference? I'm not too sure how I can expand on this. 

Let's say that you want to tag the actual transaction with 'Rent' or 'Bill'. 

Is there a field where I can insert a string for the other party? Note that this doesn't need to be hashed, it can be clear text.
No such support exists in the bitcoin protocol. Use a different receiving address for every transaction to uniquely identify them, if you do this correctly then no external labels are necessary.