title: How to hook go-colly to elasticsearch?
body: <p>What change do I make in below code to  index in elastic using go-colly? </p>
  
  <ol>
  <li><p>I want to get full text (strip html, strip js, render if needed), then </p></li>
  <li><p>Conform it to an avro schema {pageurl: , title:, content:}, </p></li>
  <li><p>Bulk-post to specific elastic-search 'mywebsiteindex-yyyymmdd' - perhaps use config file, and not hardcoding.</p></li>
  </ol>
  
  <p>Code snippets would be great. Is there an example go-colly code that shows "<strong>pipelining</strong>" output of crawl->scraping->yield to elastic (e.g as in python scrapy framework). I.e pipelining  <strong>framework</strong> support.</p>
  
  <p>For inserting to elastic, I'm considering: <a href="https://github.com/olivere/elastic" rel="nofollow noreferrer">https://github.com/olivere/elastic</a> ?</p>
  
  <pre class="lang-golang prettyprint-override"><code>func main() {
      c := colly.NewCollector(
          colly.AllowedDomains( "www.coursera.org"),
          colly.Async(true),
      )
  
      c.Limit(&amp;colly.LimitRule{
          DomainGlob: "*",
           Parallelism: 2,
      })
  
      c.OnHTML("a[href]", func(e *colly.HTMLElement) {
          link := e.Attr("href")
          e.Request.Visit(link)
      })
      pageCount :=0
      c.OnRequest(func(r *colly.Request) {
          r.Ctx.Put("url", r.URL.String())
      })
  
      // Set error handler
      c.OnError(func(r *colly.Response, err error) {
          log.Println("Request URL:", r.Request.URL, "failed with response:", r, "\nError:", err)
      })
  
      // Print the response
      c.OnResponse(func(r *colly.Response) {
          pageCount++
          urlVisited := r.Ctx.Get("url")
          log.Println(fmt.Sprintf("%d  DONE Visiting : %s", pageCount, urlVisited))
      })
  
      baseUrl := "https://www.coursera.org"
      c.Visit(baseUrl)
   }
  </code></pre>