title: Apache Livy - Rest API
body: <p>My Requirement : From web application trigger a Spark job in Yarn and display the result back in web page.
  The spark job accepts few arguments and computes a DataSet with values that needs to be returned to web application.</p>
  
  <p>After doing some browsing in the web , I figured Livy can be used for this.</p>
  
  <p>Livy was already installed with HDP 2.5.
  So I created new Livy session using POST/Sessions and including my jar file.</p>
  
  <pre><code>{"kind":"spark","name":"livy","jars":["/xyz.jar"],"proxyUser":"livy"}
  </code></pre>
  
  <p>(I had to include header 'x-requested-by' as csrfPrevention was enabled.)
  Note:- the jar had to be placed in HDFS for this to work</p>
  
  <p>As per Livy Examples :- <a href="https://livy.apache.org/examples/" rel="nofollow noreferrer">https://livy.apache.org/examples/</a>
  I can pass code snippets as "data = {'code': '1 + 1'}"
  I don't understand how I can invoke the method in my class.I do not have 'className' option as per Livy Rest API Documentation - <a href="https://livy.apache.org/docs/latest/rest-api.html" rel="nofollow noreferrer">https://livy.apache.org/docs/latest/rest-api.html</a></p>
  
  <p>If I use POST/Batch to create the session , I can specify a jar and my main class.But doing it this way I will not get my result back in my web application.</p>
  
  <p>Java Code in my jar file :</p>
  
  <pre><code>public class LivySample {
  
  
      public String executeSampleLivy(SparkContext sc,String input){
          JavaSparkContext jsc = new JavaSparkContext(sc);
          List&lt;String&gt; listNames = Arrays.asList("abc","def","ghi");
          JavaRDD&lt;String&gt; rdd =  jsc.parallelize(listNames);
          return rdd.filter(l-&gt;l.contains(input)).collect().get(0);
      }
  
  }
  </code></pre>
  
  <p>I tried to run the below code as POST on Livy url - sessions/20/statements
  '''</p>
  
  <pre><code>{
    "code": "import LivySample;LivySample lv = new LivySample();lv.executeSampleLivy(sc, \"abc\")"
  }
  </code></pre>
  
  <p>Error I got while invoking GET sessions/21/statements/0:</p>
  
  <pre><code>  {
  "id": 2,
  "state": "available",
  "output": {
  "status": "error",
  "execution_count": 2,
  "ename": "Error",
  "evalue": "&lt;console&gt;:1: error: '.' expected but ';' found. import LivySample;LivySample lv = new LivySample();lv.executeSampleLivy(sc, "chris"); ^",
  "traceback": [],
  }
  }
  </code></pre>
  
  <p>I am not able to debug this error.Can you please let me know what I am doing wrong here.</p>
  
  <p>Can I use Java in LivyRest API Like I have specified here.</p>
  
  <p>Thanks!</p>