what is the primary factor responsible for the increase of block size? Blocks are made up mostly of transactions.

I understand that the larger the transaction output is in coins, the greater the transaction size must be because it must list all previous transaction inputs that lead up to this transaction and add up to the net output of this specific transaction.

Is there any other factor I might not have accounted for that increases transaction size or block size?
The primary driver of Block size is the number of transactions. Below is the structure of a block and the size of each part in the block.


  Magic no, 4 bytes
  
  Blocksize, 4 bytes
  
  Blockheader, 80 bytes
  
  Transaction counter, 1 - 9 bytes
  
  Transactions, -many transactions


The only part of a block with a variable size is the transactions part.

See: https://en.bitcoin.it/wiki/Blocks

If you're exploring the scalability of Bitcoin, see: https://en.bitcoin.it/wiki/Scalability#Storage