Python code to split csv into smaller csvs, not splitting IDs
If I understand correctly, you want to split a file into smaller files, based on size (no more than 1000000 lines) and ID (no ID should be split among files).

If that's the case, I think you're over-complicating things. You don't need pandas and you definitely don't need to keep all data in memory.

You just need two counters, one for the amount of lines you've written and one for the index of the file to write.

Sample code (of course, replace file names with what you need, or move the check after the write to start from 0 instead of 1):

current_id = ''
index = 0
written_lines = 0
max_lines = 1000000

with open('data.csv', 'r') as input_file:
    for line in input_file:
        values = line.split(',')
        if (current_id != values[0]) or (written_lines &gt; max_lines):
            index += 1
            current_id = values[0]
        with open('output_{:08d}.csv'.format(index), 'a') as output_file:
            output_file.write(line)
            written_lines += 1


EDIT: This works assuming the file is sorted or that at least the IDs are grouped together as you said in the comment.