title: Can gzip compress data without loading it all into memory, i.e. streaming/on-the-fly?
body: <p>Is it possible to gzip data via some amount of streaming, i.e. without loading all of the compressed data in memory at once?</p>
  <p>For example, can I gzip a file that will be 10gb gzipped, on a machine with 2gb of memory?</p>
  <p>At <a href="https://docs.python.org/3/library/gzip.html#gzip.compress" rel="nofollow noreferrer">https://docs.python.org/3/library/gzip.html#gzip.compress</a>, the <code>gzip.compress</code> function returns the bytes of the gzip, so must be all loaded in memory. But... it's not clear how <code>gzip.open</code> works internally: whether the zipped bytes will all be in memory at once. Does the gzip format itself make it particularly tricky to achieve a streaming gzip?</p>
  <p>[This question is tagged with Python, but non-Python answers welcome as well]</p>