Pythonの文字列連結の方法について


+演算子を使って文字列連結すると速度が落ちるということだったので  
以下のように、joinをひたすら繰り返すだけのプログラムを試しました

**test.py**

[code]

    test = ''
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    print(test)
    
[/code]

すると、以下のようなエラーが出ます。

[code]

    Traceback (most recent call last):
      File "C:\Users\emoto\Desktop\test.py", line 10, in <module>
        test = test.join(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    MemoryError
    
[/code]

実行環境は、Windows10の64bit、AnacondaPrompt、Pythonのバージョンは3.7です。  
+演算子の場合は以下のように文字列連結してもエラーになりません。

**test2.py**

[code]

    test = ""
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    test += "aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"+"aaaa"
    print(test)
    
[/code]

また、以下のようにappendしてからjoinしてみるコードも試してみました。

**test3.py**

[code]

    test = []
    test.append(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test.append(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test.append(["aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    print(''.join(test))
    
[/code]

すると、以下のようなエラーが出ました。

[code]

    Traceback (most recent call last):
      File "C:\Users\emoto\Desktop\test.py", line 5, in <module>
        print(''.join(test))
    TypeError: sequence item 0: expected str instance, list found
    
[/code]

Pythonで複数の文字列を連結するにはどのように書くのがよいのでしょうか？  
この例では、"aaaa"という文字列だけを連結していますが  
実際の開発中のコードでは、途中でif文やfor文や変数等を組み合わせて文字列連結しています。




`test3.py`のエラー原因は @metropolis
さんコメントで良いとして、`test.py`がエラーになって`test2.py`が正常に動作するのは、その2つの処理が同じ結果では無く`test.py`はそれだけ実際のメモリを使っているからです。  
だからテストの方法が悪いわけです。  
自分で確認するのは大切だとしても、何をやっているかは意識した方が良いでしょう。

* * *

`test2.py`では単純に指定された文字列が連結されていくだけですが、`test.py`ではその直前に連結した処理結果が指定された文字列の間に挟まります。  
例えば文字の数を減らして、行ごとに文字を変えれば一目瞭然でしょう。`test.py`ではアッという間にサイズが肥大します。  
`test.py`簡略化

[code]

    test = ''
    test = test.join(["a","a","a","a","a","a","a","a","a","a","a","a","a","a","a"])
    test = test.join(["b","b","b","b","b","b","b","b","b","b","b","b","b","b","b"])
    test = test.join(["c","c","c","c","c","c","c","c","c","c","c","c","c","c","c"])
    print(len(test))
    
[/code]

これで結果の文字数が`3165`  
長いので"a"と"b"の連結だけの結果ではこうなります。

[code]

    baaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaab
    
[/code]

`test2.py`簡略化

[code]

    test = ""
    test += "a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"
    test += "b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"
    test += "c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"
    print(len(test))
    
[/code]

こちらは結果の文字数が`45`  
同様に"a"と"b"の連結だけの結果ではこうなります。

[code]

    aaaaaaaaaaaaaaabbbbbbbbbbbbbbb
    
[/code]

ちなみに`test.py`の元の長さで連結した場合は、8行分だけで`6811334100`=6.34G文字x2byte(Unicode)になります。  
9行目で`MemoryError`になりました。

* * *

`test.py`を`test2.py`と同様の条件にするとしたら、以下のようになるでしょう。  
各行毎に完了させる処理で代入となりますが、文字列の連結は`.join`だけで行われます。

[code]

    test = ''
    test = ''.join([test,"aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = ''.join([test,"aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
      ...
    
[/code]

* * *

例えばこの辺の記事を参考にしてみるとか。  
[文字列連結の常識、joinは本当に速いのか
Python3](https://qiita.com/GinRickey/items/d8ed9964820313340a93)  
[Pythonの処理速度を上げる方法
その１](https://aqvi.hatenadiary.org/entry/20080411/1207929090)  
[Pythonの処理速度を上げる方法
その２](https://aqvi.hatenadiary.org/entry/20080419/1208577161)  
[pythonの速度で気にするところ(高速化メモ)](http://nobunaga.hatenablog.jp/entry/2018/03/19/081425)




`test3.py`のエラー原因は @metropolis
さんコメントで良いとして、`test.py`がエラーになって`test2.py`が正常に動作するのは、その2つの処理が同じ結果では無く`test.py`はそれだけ実際のメモリを使っているからです。  
だからテストの方法が悪いわけです。  
自分で確認するのは大切だとしても、何をやっているかは意識した方が良いでしょう。

* * *

`test2.py`では単純に指定された文字列が連結されていくだけですが、`test.py`ではその直前に連結した処理結果が指定された文字列の間に挟まります。  
例えば文字の数を減らして、行ごとに文字を変えれば一目瞭然でしょう。`test.py`ではアッという間にサイズが肥大します。  
`test.py`簡略化

[code]

    test = ''
    test = test.join(["a","a","a","a","a","a","a","a","a","a","a","a","a","a","a"])
    test = test.join(["b","b","b","b","b","b","b","b","b","b","b","b","b","b","b"])
    test = test.join(["c","c","c","c","c","c","c","c","c","c","c","c","c","c","c"])
    print(len(test))
    
[/code]

これで結果の文字数が`3165`  
長いので"a"と"b"の連結だけの結果ではこうなります。

[code]

    baaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaabaaaaaaaaaaaaaaab
    
[/code]

`test2.py`簡略化

[code]

    test = ""
    test += "a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"+"a"
    test += "b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"+"b"
    test += "c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"+"c"
    print(len(test))
    
[/code]

こちらは結果の文字数が`45`  
同様に"a"と"b"の連結だけの結果ではこうなります。

[code]

    aaaaaaaaaaaaaaabbbbbbbbbbbbbbb
    
[/code]

ちなみに`test.py`の元の長さで連結した場合は、8行分だけで`6811334100`=6.34G文字x2byte(Unicode)になります。  
9行目で`MemoryError`になりました。

* * *

`test.py`を`test2.py`と同様の条件にするとしたら、以下のようになるでしょう。  
各行毎に完了させる処理で代入となりますが、文字列の連結は`.join`だけで行われます。

[code]

    test = ''
    test = ''.join([test,"aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
    test = ''.join([test,"aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa","aaaa"])
      ...
    
[/code]

* * *

例えばこの辺の記事を参考にしてみるとか。  
[文字列連結の常識、joinは本当に速いのか
Python3](https://qiita.com/GinRickey/items/d8ed9964820313340a93)  
[Pythonの処理速度を上げる方法
その１](https://aqvi.hatenadiary.org/entry/20080411/1207929090)  
[Pythonの処理速度を上げる方法
その２](https://aqvi.hatenadiary.org/entry/20080419/1208577161)  
[pythonの速度で気にするところ(高速化メモ)](http://nobunaga.hatenablog.jp/entry/2018/03/19/081425)

