What are the security considerations for hosting *your own* Electrum server?
Electrum servers don't save any private information. All they do is create an interface to a bitcoind instance that is exposed in such a way the Electrum client, which stores the important and private data, can read it. 

The only 'evil' thing an Electrum server could do is lying about certain transactions, feeding you misinformation. When you run your own server this is the only thing you will gain. 

Encryption won't be needed, but is a good practice overall. No wallets will be saved on the server, except the wallet from Bitcoind. This wallet however will always be empty.