Page download with reattempts in Scala I wrote the following code to deal with possible network problems.

def getPage(browser: Browser, page: String): Document =
    {
      var attempts = 5
      while (attempts &gt; 0) {
        try {
          return browser.get(page)
        } catch {
          case ste: java.net.SocketTimeoutException if attempts == 1 =&gt; throw ste
          case ste: java.net.SocketTimeoutException =&gt; attempts -= 1
        }
      }
      return null
    }


I am new to scala and would appreciate if there is a better, "scala-way" of implementing similar

Document and Browser are classes from Scalascraper lib which is a wrapper around JSoup lib.
Your code uses a number of things seldom if ever seen in idiomatic Scala:


return
null
throw
var


the land of no return

The final statement in a code block is the return value of that block, so the keyword return isn't needed. If you structure your code so that each block/method/function has only one exit point, at the end, then your code is easier to read and understand, and less prone to unexpected results.

Also, under some circumstances (I won't detail here) using return will do unexpected things. Easier to just avoid it altogether.

failure is an option

Code that might fail to complete correctly usually does one of two things: return a nonsense value, like null, which the calling code will test for (hopefully), or throw an exception, indicating the source of the problem, which some routine somewhere will catch (hopefully) and process.

Scala offers a third and better option: put the failure in the type system. If a routine might fail, put that in its return type, not its return value. That way the calling code can't ignore the possibility of failure.

In this case, instead of returning a Document, return a Try[Document]. Much like an Option, which is Some(value) or None, a Try is expressed as Success(value) or Failure(exception).

iteration without mutation

Functional Programming avoids data structures with mutable state. Scala has vars but they are discouraged. There are a number of different ways to iterate without using a var. In this case here's what I'd do.

Seq.iterate(Try(browser get page), 5)(_ orElse Try(browser get page))


This is a sequence of 5 results from the page requests, but a new attempt is made only if the previous one failed. The first Success is passed down the line to all subsequent positions in the sequence.

putting it all together

import util.Try

def getPage(browser :Browser, page :String) :Try[Document] =
  Seq.iterate(Try(browser get page), 5)(_ orElse Try(browser get page))
     .last