How to securely store data? / How to securely encrypt a file? How do I safely encrypt a file?

or formulated differentely:

How do I safely store the data of my application in a file?

Note: Using established tools isn't an option for this. The tasks need to be done at program level (using libraries and such).
Ask yourself if you even can securely store the user's data.
This includes to check that the user won't get sniffed by the root / admin 
user.Also make sure that your implementation is secure.Employ 
standard code review mechanisms, make sure that your implementation has 
countermeasures against timing attacks (like against AES).Also make sure 
that you use appropriate data types.Using a byte* allocated using malloc 
usually isn't very secure. If your library provides mechanism for data 
storage, use them! Your OS may also provide means to protect data 
while in memory, for example the data protection API in Windows only relies 
on the current user and can be used to conceal sensible keys in memory. 
Reading Schneier's Cryptography Engineering may be a very good idea for you, 
to ensure you don't have side-channel leaks.
Ask yourself what you can use to authenticate the user. The 
options are (it is explicitely allowed and recommended to combine as 
many of them as convenience and availability allow):

Passwords: Your user can supply a password or a passphrase. Do 
not limit him in length of this, to allow him to use good and long 
passphrases. This usually is a good choice, as it requires the user to know 
something and this hence can not be lost like an usb-stick.
Keyfiles: Allow the user to provide keyfiles. These files may 
look arbitrary, but should be rare. Allow the user to use the 
(cryptographical) random number generator of your application to generate 
strong keyfiles (of 64 bytes length or more). Examples for keyfiles are rare documents, like personal (word/) text documents or on-purpose generated files just containing random data (created using the application's random number generator).
Custom Hardware: If you are deploying in large scale you can force 
the user to be required to use a tamper-resistand hardware device. This 
device should support symmetric encryption with a device-held key.
User's Hardware: Allow the user to use his PKCS#11 or similar 
smartcard or cryptographic token, which provides public key encryption, to 
be used. Also allow the user to protect the key using the TPM (if 
available)
OS data: Allow the user to make the decryption dependant on 
something stored in the current OS, like the user credentials or OS 
provided secure key storage.

Inspect what algorithms you have at your disposal. You'll need (at 
worst) five classes of algorithms: Password-based key derivation functions, 
encryption algorithms, modes of operation, message authentication codes and 
cryptographically secure random number generators.At the time of 
writing the following algorithms are recommended (in descending order). If 
functionality is already provided more top-level don't use it.

Cryptographically secure random number generators:

Fortuna
An approved algorithm. This may either be the result of a 
contest or some approved algorithm by some trusted institute (e.g. ANSSI, 
NIST, BSI, ...). Avoid algorithms with "provable" security where the 
institute provides the parameters, these parameters may be chosen on 
purpose to get back-door access. For seeding use the cryptographically 
secure random number generator of your OS. They are usually a bit 
hidden.
Some "weak" standard approach. (i.e. run a blockcipher in CTR 
mode and delete key and IV as soon as possible) Get the key and the IV 
from the cryptographically secure random number generator of your OS. 
They are usually a bit hidden.

Password based key derivation functions:

The winner of the PHC-competition. If there are multiple 
winners, ask for the differences and use the one most suited for your needs.
scrypt
bcrypt
PBKDF2, using an efficient and secure hash function (on your 
platform)
implement your own PBKDF2! 

Ciphers:

The winner of the CAESAR-competition (2018+). If there are 
multiple winners, ask for the differences and choose the appropriate one.
AES. Note: For data that needs to be confidential unti 2070 or 
earlier, it is recommeded to use AES-128. In every else case use AES-192 or 
AES-256.
ChaCha(preferred) or Salsa20.
One of the following three: Threefish, Serpent or Twofish
3DES with 3 keys. (needs 168 bits of keying material)

Modes of operation (this is obsolete for CAESAR-winners):

OCB (If the patent has expired)
GCM
EAX (preferred) or CCM
CTR, below this a message authentication code is required 
(including)
One of the following: CFB or OFB
ECB, avoid at all cost, rather implement your own CTR!

Message authentication codes:

Poly-1305 using a standardized construction
HMAC, using a secure standardized hash function, preferably 
SHA-3 or a member of the SHA-2 family. Encrypt the data first, 
then authenticate the ciphertext!
implement your own HMAC!


Ask yourself on how to design the header. The header should contain: 

The salt, a random 64 byte or more value that will be newly 
chosen on each single creation of a new file. This will be fed into the 
password-based key derivation function. The salt can be open, but may be 
authenticated using the authentication mechanism.
The parameters for the password-based key derivation function. 
These parameters need to be open. They should be chosen such that key 
derivation needs approximiately 100ms to 1s on the target platform. A 
value indicating the password based key derivation function used is 
considered a parameter.
A nonce or an IV for the cipher / mode. This value should be chosen 
at random and be stored openly.
The remainder of the header should be encrypted and 
authenticated.The before-mentioned nonce/IV should be used along with 
the key derived from the password using the password-based key derivation 
function and the stored salt and parameters.
The master-keys for the data part of the file. These should be 
random and maximal length. This is part of the encrypted part of the 
header
A nonce / IV for the data part. This value should be random. This 
value can (and should) be part of the encrypted part of the header. 

The actual data is now encrypted using the above chosen 
cipher/mode/authentication triple and the key/IV/nonce pair of the encrypted 
part of the header.


Now some question that may arise:


Is it worth it to use multiple-encryption?No.The benefit 
by double encryption is only 1-bit due to the meet-in-the-middle attack. 
Streamciphers may be combined easier, but it is absolutely unneccessary as 
all ciphers are unbreakable for the forseeable future (50 years+)
What authentication methods should I use?As many as possible, 
but at least passwordYou need to have a very good reason not to use 
passwords because everything else can be attacked with admin rights or can 
be stolen.
You didn't mention the other four authentication methods in the header 
part, how should I incorporate them?

Keyfiles: Assume that each keyfile has low entropy. That being 
said, you can't really do alot against it. Derive a key from every keyfile 
using some digest-expansion function and your favorite secure hash function 
(SHA-2/3) and construct a big XOR consisting of all file's hashes and the 
user's password. This would look like this: $Key = PBKDF(PW) \oplus H
(File_1) \oplus H(File_2) \oplus ...$, where $\oplus$ denotes bitwise XOR.
Custom Hardware: At the creation of the file let the device 
generate a symmetric key that will be kept inside the device. Perform 
everything up to the keyfile / password level (see directly above point) and 
then send this to the device for encryption. The key being used doesn't have 
to be unique for each file. It is suggest that one key is being used for all 
files. It is also possible to add an identifier to the file to identify the 
associated hardware key.
User's Hardware: Let the device generate an OpenPGP key or a valid 
S/MIME encryption certificate. Send the key being the result of the step 
directly above (or 2 steps above) to the device for decryption. It will 
decrypt it and return the key used to decrypt the master-keys.
OS data: If you have user credentials (maybe some random strings), 
hash them and incorporate them at the same time in the same way as the 
keyfiles. If your OS provides you secure storage (or your TPM does), store a 
key and pass it in as "keyfile", thus hash it and incorporate it in the XOR 
step.

Should I use an open source library or some closed-source library?Open-SourceChances are very good that there aren't any security 
issues in well-known open-source libraries, whereas closed source puts 
restriction in regard of when you can use it, usually you have to pay to be 
able to use and you can't usually inspect the code on your own to make sure 
it's secure.
Why did you use XOR to include keyfiles?(relates to question 3)Convenience.If 
you don't use something that is commutative, like XOR, users will run into 
problem because files have to be supplied in the exact same order. This gets 
even harder if files are just re-named but their contents are kept. And from 
a security perspective this is no less secure as one basically constructs a 
"stream cipher" with the password based key derivation function output, the 
hashes of the keyfiles and the other outputs (TPM, credentials,...) which is secure as 
long as one source is secure.