title: Real-time scraping website with Django: stoppable scraping process and loading bar
body: <p><strong>What I would like to achieve</strong></p>
  <p>I would like to make a website that allows users to scrape information from other websites. My goal is to achieve something as follows:</p>
  <ol>
  <li>User presses &quot;scrape&quot;</li>
  <li>When the scraping process begins, Django renders a view with a loading message and a &quot;stop&quot; button</li>
  <li>If the user presses the &quot;stop&quot; button, or if the scraping process terminates, a view with the scraping results is shown</li>
  </ol>
  <p><strong>Current state</strong></p>
  <p>At the moment, I have a very simple scraping website with a horrible user experience. After the user presses scrape, I render a view similar to the following one:</p>
  <pre class="lang-py prettyprint-override"><code>def scraping_view(request, scraper_args):
      scrape_results = scrape(scraper_args) # Very long function!
      return render(request, 'scrape_view.html', scrape_results )
  </code></pre>
  <p>This is a bad idea, as the <code>scrape</code> function takes a long time, and the user experience is terrible.</p>
  <p><strong>My (probably bad) idea</strong></p>
  <p>My idea was to:</p>
  <ul>
  <li>launch a celery task and render an intermediary view with the loading message and a &quot;stop&quot; button</li>
  <li>let the celery task store the scrape results somewhere (where?)</li>
  <li>find a way to revoke the task if the user presses the &quot;stop&quot; button</li>
  <li>retrieve the results of the celery task when the task is finished or when &quot;stop&quot; is being pressed</li>
  <li>cancel the scrape results after the page is closed, as I am simply interested in showing them on the website</li>
  </ul>
  <p>As I am new to Django, I cannot evaluate if my idea is feasible or if it makes sense at all. Does anyone have a solution to propose, an alternative solution, or some examples to share? Thanks!</p>