Unable to successfully dump emails retrieved from gmail to mbox file using python
Question: I have written below 2 code snippets. Second one work's as expected but not
the first one.I am unable to find out why output list of print(msg_obj.keys())
at line #22 in first snippet doesn't contain 'Subject','From' keys while
msg_obj contains header fields like 'Subject','From'. When I dump emails to
mbox file using this script and later open that file using some
utility(mboxview.exe for windows) ,utility doesn't recognise any email
dumped.Please help me out of this. Any suggestions are most welcomed.

    
    
    import imaplib,email,mailbox
    
    M=imaplib.IMAP4_SSL('imap.gmail.com',993)
    status,data=M.login('someone@gmail.com', 'password')
    M.select()
    
    #create new mbox file if doesn't exist
    mbox_file=mailbox.mbox('gmail_mails.mbox')
    mbox_file.lock()
    
    #get all mails number   
    status,data=M.search(None, 'ALL')
    
    try:
        for mail_no in data[0].split():
    
            status,msg=M.fetch(mail_no,'(RFC822)')
    
            msg_obj=email.message_from_string(str(msg[0][1]))
    
            #print for debugging purpose
            print(msg_obj.keys())
            print(msg_obj["Subject"])
    
            mbox_msg_obj=mailbox.mboxMessage(msg_obj)
    
            mbox_file.add(mbox_msg_obj)
    
            mbox_file.flush()
    
    finally:
        mbox_file.unlock()
        mbox_file.close()
    
    M.close()
    M.logout()
    

Also what I found is in case of below code it works:

    
    
    from email.parser import Parser
    
    str="""Received: (qmail 8580 invoked from network); 15 Jun 2010 21:43:22      -0400\r\nReceived: from mail-fx0-f44.google.com (209.85.161.44) by ip-73-187-35-131.ip.secureserver.net with SMTP; 15 Jun 2010 21:43:22 -0400\r\nReceived: by fxm19 with SMTP id 19so170709fxm.3 for <username@domain.com>; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nMIME-Version: 1.0\r\nReceived: by 10.103.84.1 with SMTP id m1mr2774225mul.26.1276652853684; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nReceived: by 10.123.143.4 with HTTP; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nDate: Tue, 15 Jun 2010 20:47:33 -0500\r\nMessage-ID: <AANLkTikFsIjJ3KYW1HJWcAqQlGXNiXE2YMzrj39I0tdB@mail.gmail.com>\r\nSubject: TEST 12\r\nFrom: Full Name <username@sender.com>\r\nTo: username@domain.com\r\nContent-Type: text/plain; charset=ISO-8859-1 ONE\nTWO\nTHREE"""
    
    msg=Parser().parsestr(str)
    
    print (msg['Subject'])
    print (msg['From'])
    print (msg['to'])
    

Here output is

    
    
    TEST 12
    Full Name <username@sender.com>
    username@domain.com
    

Problem solved using email.parser.BytesParser().parsebytes() instead of
email.message_from_string().But not getting why?


Answer: From your example code, I'm trying the following:

    
    
    import email
    
    estr = """Received: (qmail 8580 invoked from network); 15 Jun 2010 21:43:22      -0400\r\nReceived: from mail-fx0-f44.google.com (209.85.161.44) by ip-73-187-35-131.ip.secureserver.net with SMTP; 15 Jun 2010 21:43:22 -0400\r\nReceived: by fxm19 with SMTP id 19so170709fxm.3 for <username@domain.com>; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nMIME-Version: 1.0\r\nReceived: by 10.103.84.1 with SMTP id m1mr2774225mul.26.1276652853684; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nReceived: by 10.123.143.4 with HTTP; Tue, 15 Jun 2010 18:47:33 -0700 (PDT)\r\nDate: Tue, 15 Jun 2010 20:47:33 -0500\r\nMessage-ID: <AANLkTikFsIjJ3KYW1HJWcAqQlGXNiXE2YMzrj39I0tdB@mail.gmail.com>\r\nSubject: TEST 12\r\nFrom: Full Name <username@sender.com>\r\nTo: username@domain.com\r\nContent-Type: text/plain; charset=ISO-8859-1 ONE\nTWO\nTHREE"""
    
    msg = email.message_from_string(estr)
    
    print (msg['Subject'])
    print (msg['From'])
    print (msg['to'])
    

That prints the results fine.

    
    
    TEST 12
    Full Name <username@sender.com>
    username@domain.com
    

So from your first code snippet, the function input `str(msg[0][1])` must
contain some unparsable contents. you'll have to have a closer look at the
contents of `str(msg[0][1])` that is failing to parse.

