title: Make Coroutine wait for Jsoup response in Kotlin
body: <p>I'm kind of new developing with Android, and have problems understanding how things works. I made an app to <strong>scrape</strong> content from a page, and download elements, first with AsyncTask and worked, but since AsyncTask doesn't let me communicate with the UI Activity on progress, i decided to change to coroutines, checked an example, and the same code i used doesn't seems to work.</p>
  <p>I used a few logs to try to determine the problem, and seems like it doesn't wait for the Jsoup request. The coroutine first calls a method <code>scrapePage()</code> to download the HTML and <strong>scrape</strong> the links, and then calls <code>downloadImages()</code> to add the links to Android's DownloadManager. In the logs <code>Log.d(&quot;action&quot;, &quot;Start Scraping&quot;)</code> is printed, but <code>Log.d(&quot;action&quot;, &quot;Page downloaded&quot;)</code> doesn't, still we get <code>Log.d(&quot;action&quot;, &quot;End&quot;)</code> from the coroutine, which makes me think that instead of waiting for the Jsoup request to answer, it goes with an empty response, causing the rest of the code to not work correctly.</p>
  <p>DownloadService.kt</p>
  <pre><code>object DownloadService {
  
      private val parentJob = Job()
      
      ...
  
      private val coroutineScope = CoroutineScope(Dispatchers.Main + parentJob +
              coroutineExceptionHandler)
  
      fun StartService(URL: String, location:String, contx:Context) {
  
          coroutineScope.launch(Dispatchers.Main) {
              Log.d(&quot;action&quot;, &quot;Start&quot;)
              val links = scrapePage(URL)
              val download = downloadImages(links, location, contx)
              Log.d(&quot;action&quot;, &quot;End&quot;)
          }
      }
  
      private suspend fun scrapePage(url: String): MainActivity.Scraped =
          withContext(Dispatchers.IO) {
              var URL = url
              var scrape = MainActivity.Scraped()
              try {
                  Log.d(&quot;action&quot;, &quot;Start Scraping&quot;)
                  var response = Jsoup.connect(URL).get()
                  Log.d(&quot;action&quot;, &quot;Page downloaded&quot;)
                  response.getElementsByClass(&quot;link&quot;).forEach {
                      /*Scrape URLs*/
                      Log.d(&quot;action&quot;, &quot;Add &quot;+link)
                  }
              } catch (e: Exception) {
                  when(e) {
                      is HttpStatusException -&gt; {
                          System.out.println(e.getStatusCode())
                          scrape.error = true
                          error = true
                      }
                  }
              }
              return@withContext scrape
          }
  
      ...
  }
  </code></pre>
  <p>MainActivity.kt</p>
  <pre><code>class MainActivity : AppCompatActivity() {
      ...
      fun makeRequest(URL : String) {
          WorkingURL = URL
          var uri = &quot;&quot;
          if (Build.VERSION.SDK_INT &gt;= Build.VERSION_CODES.Q) {
              uri = MediaStore.Downloads.EXTERNAL_CONTENT_URI.toString()
              log.text = uri
          } else {
              uri = getStoragePath()
              log.text = uri
          }
          DownloadService.StartService(URL, uri, this)
          Log.d(&quot;links&quot;, DownloadService.getError().toString())
      }
  }
  </code></pre>
  <p>I am not sure where the problem is, nor where to start searching. I know the code for the <strong>Scraping</strong> works, because i used it before with AsyncTask, so the problem seems to be passing it to coroutines.</p>