I have a 20.2 GB dataset which is entirely consisting of 1 file. Need help on how to open the file
This file should be a plain text file with n rows and m columns separated by a tabulator. Here is another much smaller file which they offer for download and claim that it has the same data format.

Since the file is too big to fit in memory, you can use the data format HDF5. This will allow you to load slices of your dataset without having to load the whole file into memory. However, first, you need to convert the file to a .h5 file. This can be done with h5py. 

I put the following code together right now. It reads the text-file line by line and writes each line to the .h5-file. It works fine on the smaller file which they offer for download. I haven't tested it on a larger file, however. If you try it out, let me know how it goes.

import h5py
import numpy as np

N_COLS = 3  # adjust to your number of columns

n_rows_dataset = 1000

file = h5py.File('/path/to/your/output.h5', 'w')
dataset = file.create_dataset(
    'my_dataset', 
    (n_rows_dataset, N_COLS), 
    chunks=(5, N_COLS), 
    maxshape=(None, N_COLS))

current_row = 0
# this does not load the whole file but creates an iterator instead
for line in open("/path/to/the/large/file"):  
    if current_row == n_rows_dataset-1:
        n_rows_dataset = n_rows_dataset + 1000
        dataset.resize((n_rows_dataset, N_COLS))
    dset[current_row] = np.fromstring(line, sep=" ", dtype=np.float64)
    current_row += 1



Note: You can find out the number of colums of your dataset by printing the first lines of your file:

i = 0
for line in open("/path/to/the/large/file"):
    print(line)
    i += 1
    if i &gt;= 5:
        break