Why is this program not writing the final variable string to it's destination text file?
Question: I'm almost there with using some Python code to pull down stock data strings
passed to it from SAS to build a text file to read back into SAS, but the
final output text file remains at 0 bytes, despite the code running without
error and the print statements I have put in the log seeming to display valid
output. Here is the code:

    
    
    import concurrent.futures
    import urllib.request
    import json
    
    with open("C:\\Python33\\NASDAQ Stock Strings\\NASDAQ_Config_File_1_a.txt", "r") as myurls:
        myurls2 = myurls.read().replace('\n', '')
    
    
    URLS = [myurls2]
    print('URLS =', URLS) 
    
    # Retrieve a single page and report the url and contents
    def load_url(url, timeout):
        conn = urllib.request.urlopen(url, timeout=timeout)
        return conn.readall()
    
    
    # We can use a with statement to ensure threads are cleaned up promptly
    with concurrent.futures.ThreadPoolExecutor(max_workers=50) as executor:
        # Start the load operations and mark each future with its URL
        future_to_url = {executor.submit(load_url, url, 60): url for url in URLS}
    
        c = 0
    
    
        for future in concurrent.futures.as_completed(future_to_url):
            url = future_to_url[future]
    
            a = ''
            b = ''
            c += 1
    
            mylen = (len(myurls2)) - 1
            print('String length = %s' % (len(myurls2)))
            a = myurls2[mylen:]
            print('a=', a)
    
    
            if a == 'a':
               b =  'Ask' 
            elif a == 'y': 
                 b = 'Dividend Yield'
            elif a == 'b':
                 b = 'Bid'
            elif a == 'd':
                 b = 'Dividend per Share'
            elif a == 'b2':
                 b = 'Ask (Realtime)'
            elif a == 'r1':
                 b = 'Dividend Pay Date'
            elif a == 'b3':
                 b = 'Bid (Realtime)'
            elif a == 'q':
                 b = 'Ex-Dividend Date'
            elif a == 'p':
                 b = 'Previous Close'
            elif a == 'o':
                 b = 'Open'
            elif a == 'c1':
                 b = 'Change'
            elif a == 'd1':
                 b = 'Last Trade Date'
            elif a == 'c':
                 b = 'Change & Percent Change'
            elif a == 'd2':
                 b = 'Trade Date'
            elif a == 'c6':
                 b = 'Change (Realtime)'
            elif a == 't1':
                 b = 'Last Trade Time'
            elif a == 'k2':
                 b = 'Change Percent (Realtime)'
            elif a == 'p2':
                 b = 'Change in Percent'    
            elif a == 'c8':
                 b = 'After Hours Change (Realtime)'
            elif a == 'm5':
                 b = 'Change From 200 Day Moving Average'
            elif a == 'c3':
                 b = 'Commission'
            elif a == 'm6':
                 b = 'Percent Change From 200 Day Moving Average'
            elif a == 'g':
                 b = 'Days Low'
            elif a == 'm7':
                 b = 'Change From 50 Day Moving Average'
            elif a == 'h':
                 b = 'Days High'
            elif a == 'm8':
                 b = 'Percent Change From 50 Day Moving Average'
            elif a == 'k1':
                 b = 'Last Trade (Realtime) With Time'
            elif a == 'm3':
                 b = '50 Day Moving Average'
            elif a == 'l':
                 b = 'Last Trade (With Time)'
            elif a == 'm4':
                 b = '200 Day Moving Average'
            elif a == 'l1':
                 b = 'Last Trade (Price Only)'
            elif a == 't8':
                 b = '1 yr Target Price'    
            elif a == 'w1': 
                 b = 'Days Value Change'
            elif a == 'g1':
                 b = 'Holdings Gain Percent'
            elif a == 'w4':
                 b = 'Days Value Change (Realtime)'
            elif a == 'g3':
                 b = 'Annualized Gain'
            elif a == 'p1':
                 b = 'Price Paid'
            elif a == 'g4':
                 b = 'Holdings Gain'
            elif a == 'm':
                 b = 'Days Range'
            elif a == 'g5':
                 b = 'Holdings Gain Percent (Realtime)'
            elif a == 'm2':
                 b = 'Days Range (Realtime)'
            elif a == 'g6':
                 b = 'Holdings Gain (Realtime)'
            elif a == 'k':
                 b = '52 Week High'
            elif a == 'v':
                 b = 'More Info'
            elif a == 'j':
                 b = '52 week Low'
            elif a == 'j1':
                 b = 'Market Capitalization'
            elif a == 'j5':
                 b = 'Change From 52 Week Low'
            elif a == 'j3':
                 b = 'Market Cap (Realtime)'
            elif a == 'k4':
                 b = 'Change From 52 week High'
            elif a == 'f6':
                 b = 'Float Shares'
            elif a == 'j6':
                 b = 'Percent Change From 52 week Low'
            elif a == 'n':
                 b ='Name'
            elif a == 'k5':
                 b = 'Percent Change From 52 week High'
            elif a == 'n4':
                 b = 'Notes'
            elif a == 'w':
                 b = '52 week Range'
            elif a == 's':
                 b = 'Symbol'
            elif a == 's1':
                 b = 'Shares Owned'
            elif a == 'x':
                 b = 'Stock Exchange'
            elif a == 'j2':
                 b = 'Shares Outstanding' 
            elif a == 'v':
                 b = 'Volume'
            elif a == 'a5':
                 b = 'Ask Size'
            elif a == 'b6':
                 b = 'Bid Size'
            elif a == 'k3':
                 b = 'Last Trade Size'
            elif a == 't7':
                 b = 'Ticker Trend'
            elif a == 'a2':
                 b = 'Average Daily Volume'
            elif a == 't6':
                 b = 'Trade Links'
            elif a == 'i5':
                 b = 'Order Book (Realtime)'
            elif a == 'l2':
                 b = 'High Limit'
            elif a == 'e':
                 b = 'Earnings per Share'
            elif a == 'l3':
                 b = 'Low Limit'
            elif a == 'e7':
                 b = 'EPS Estimate Current Year'
            elif a == 'v1':
                 b = 'Holdings Value'
            elif a == 'e8':
                 b = 'EPS Estimate Next Year'
            elif a == 'v7':
                 b = 'Holdings Value (Realtime)'
            elif a == 'e9':
                 b = 'EPS Estimate Next Quarter'
            elif a == 's6':
                 b = 'Revenue'
            elif a == 'b4':
                 b = 'Book Value'
            elif a == 'j4':
                 b = 'EBITDA'
            elif a == 'p5':
                 b = 'Price-Sales'
            elif a == 'p6':
                 b = 'Price-Book'
            elif a == 'r':
                 b = 'P-E Ratio'
            elif a == 'r2':
                 b = 'P-E Ratio (Realtime)'
            elif a == 'r5':
                 b = 'PEG Ratio'
            elif a == 'r6':
                 b = 'Price - EPS Estimate Current Year'
            elif a == 'r7':
                 b = 'Price - EPS Estimate Next Year'
            elif a == 's7':
                 b = 'Short Ratio'
    
    
    
            print('b =', b)
            print('c =', c)
            filename = "%s" % (b)
            filepath = "C:\\Python33\\Stock Data\\" + str(filename) + ".txt"
            print(filepath)
            print("future.result = ", future.result())
    
            try:
                data = future.result() 
                d = open(filepath,"wb")
                d.write(data)
                d.close
    
    
                # do json processing here
    
            except Exception as exc:
               for e in range(1,11):
                    if len(data) > 0:
                        print('Byte length = %d' % (len(data)))
                        print(e)
                        print('%r generated an exception: %s' % (url, exc))
                        print('retrying %r' % (url))
    
                        def load_url(url, timeout):
                            conn = urllib.request.urlopen(url, timeout=timeout)
                            return conn.readall()
                            time.sleep(10)
    
                            print("press ENTER to exit")
    
    
            else:
                print('%r page is %d bytes' % (url, len(data)))
    

The various print statement shows the source text file contents being picked
up correctly and passed through to the code ok. The line `print("future result
=", future.result())` shows the following result:

    
    
    b'31.90\r\n36.66\r\nN/A\r\n3.69\r\n25.52\r\n27.10\r\n525.33\r\n31.81\r\n56.90\r\n38.23\r\n23.86\r\n2.19\r\n66.93\r\n35.74\r\n21.74\r\n2.10\r\n26.08\r\n14.20\r\n26.73\r\n14.92\r\n48.42\r\n12.49\r\n19.31\r\n4.09\r\n3.37\r\n57.78\r\n45.85\r\n3.32\r\n60.02\r\n2.31\r\n18.50\r\n37.74\r\n3.42\r\n12.46\r\n14.03\r\n1.25\r\n15.13\r\n2.53\r\n1.73\r\n56.72\r\n44.98\r\n35.89\r\n1.05\r\n67.50\r\n17.35\r\n50.72\r\n20.72\r\n50.37\r\n6.27\r\n13.23\r\n77.50\r\n27.62\r\n24.49\r\n34.02\r\n24.56\r\n50.59\r\n25.50\r\n21.53\r\n31.33\r\n4.65\r\n4.65\r\n24.00\r\n52.04\r\n2.73\r\n24.78\r\n39.94\r\n20.57\r\n6.84\r\n2.97\r\n1.27\r\n24.08\r\n20.50\r\n7.44\r\n14.49\r\n13.22\r\n37.62\r\n4.39\r\n44.46\r\n44.46\r\n43.80\r\n22.58\r\n22.58\r\n48.92\r\n14.60\r\n50.12\r\n60.75\r\n2.36\r\n35.10\r\n8.47\r\n29.81\r\n53.13\r\n19.57\r\n12.95\r\n16.76\r\n59.70\r\n16.63\r\n4.74\r\n23.44\r\n37.52\r\n10.37\r\n52.81\r\n107.50\r\n6.64\r\n46.15\r\n15.50\r\n14.85\r\n72.06\r\n79.08\r\n14.25\r\n8.90\r\n1.91\r\n5.54\r\n35.43\r\n5.12\r\n177.09\r\n20.30\r\n20.60\r\n18.80\r\n28.30\r\n31.93\r\n31.93\r\n10.24\r\n1.65\r\n10.09\r\n1.83\r\n2.15\r\n74.23\r\n7.51\r\n14.38\r\n123.76\r\n12.89\r\n6.17\r\n23.22\r\n11.80\r\n19.70\r\n9.95\r\n17.93\r\n1.81\r\n4.18\r\n2.13\r\n42.81\r\n44.29\r\nN/A\r\n32.72\r\n373.95\r\n21.12\r\n1.85\r\n114.72\r\n20.25\r\n2.03\r\n16.89\r\n57.65\r\n13.28\r\n16.79\r\n42.24\r\n33.87\r\n77.08\r\n3.49\r\n7.26\r\nN/A\r\n33.95\r\n34.02\r\n32.33\r\n3.07\r\n2.42\r\n16.00\r\n2.87\r\n10.26\r\nN/A\r\nN/A\r\n13.45\r\n9.70\r\n17.36\r\n2.89\r\n14.61\r\n14.61\r\n29.00\r\n20.78\r\n11.39\r\n8.25\r\n71.81\r\n3.34\r\n22.15\r\n80.82\r\n47.80\r\n6.56\r\n26.67\r\n2.17\r\n28.43\r\n5.06\r\n48.16\r\nN/A\r\n6.00\r\n4.29\r\n20.20\r\n22.79\r\n17.75\r\n17.52\r\n17.52\r\n13.80\r\n'
    

This looks like the correct values to me, but just needing cleaning up. The
destination text file resolves correctly and the all OK statement below the
error handling returns the byte length of the URL submission correctly. The
text file though remains empty.

Can anyone spot an obvious mistake I have made?

Thanks


Answer: Move `d = open(filepath,"wb")` and `d.close()` outside of the for loop.

Every iteration of the loop deletes and overwrites the file.
<http://docs.python.org/3/tutorial/inputoutput.html#reading-and-writing-files>

EDIT: I didn't see that the filepath is calculated in the forloop. I'd do what
@sabbahillel pointed out, and open the file with "ab" mode to ensure it's not
overwritten.

