title: Pass parameters from Cloud functions to Dataflow
body: <p>I'd like to pass the filename of the file uploaded on Google Cloud Storage from Cloud Functions to Dataflow so that I can process the uploaded file.</p>
  
  <p>The code that I've written for Cloud Functions is</p>
  
  <pre><code>const google = require('googleapis');
  
  exports.goWithTheDataFlow = function(event, callback) {
   const file = event.data;
   if (file.resourceState === 'exists' &amp;&amp; file.name) {
       console.log(file.name);
     google.auth.getApplicationDefault(function (err, authClient, projectId) {
       if (err) {
         throw err;
       }
  
       if (authClient.createScopedRequired &amp;&amp; authClient.createScopedRequired()) {
         authClient = authClient.createScoped([
           'https://www.googleapis.com/auth/cloud-platform',
           'https://www.googleapis.com/auth/userinfo.email'
         ]);
       }
  
       const dataflow = google.dataflow({ version: 'v1b3', auth: authClient });
  
       dataflow.projects.templates.create({
         projectId: '-------',
         resource: {
             parameters: {
             inputFile: `gs://${file.bucket}/${file.name}`
           },
           jobName: '-------',
           gcsPath: '-------'
         }
       }, function(err, response) {
         if (err) {
           console.error("problem running dataflow template, error was: ", err);
         }
         console.log("Dataflow template response: ", response);
         callback();
       });
  
     });
   }
  };
  </code></pre>
  
  <p>And the code of my Pipeline looks something like this:</p>
  
  <pre><code>public interface FruitOptions extends PipelineOptions {
        @Description("Path of the file to read from")
        @Validation.Required
        ValueProvider&lt;String&gt; getInputFile();
        void setInputFile(ValueProvider&lt;String&gt; value);
    }
  
  
  //Main Method
  FruitOptions options = PipelineOptionsFactory.fromArgs(args).withValidation()
        .as(FruitOptions.class);
        Pipeline p = Pipeline.create(options);
  </code></pre>
  
  <p>How to create a template of the above Pipeline code? I've been following the link <a href="https://shinesolutions.com/2017/03/23/triggering-dataflow-pipelines-with-cloud-functions/" rel="nofollow noreferrer">TRIGGERING DATAFLOW PIPELINES WITH CLOUD FUNCTIONS</a> to do this but I get several errors while running the maven commands such as:</p>
  
  <pre><code>[WARNING]
  java.lang.NoClassDefFoundError: com/google/cloud/dataflow/sdk/options/PipelineOp
  tions
          at java.lang.Class.getDeclaredMethods0(Native Method)
          at java.lang.Class.privateGetDeclaredMethods(Class.java:2701)
          at java.lang.Class.privateGetMethodRecursive(Class.java:3048)
          at java.lang.Class.getMethod0(Class.java:3018)
          at java.lang.Class.getMethod(Class.java:1784)
          at org.codehaus.mojo.exec.ExecJavaMojo$1.run(ExecJavaMojo.java:281)
          at java.lang.Thread.run(Thread.java:745)
  Caused by: java.lang.ClassNotFoundException: com.google.cloud.dataflow.sdk.optio
  ns.PipelineOptions
          at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
          at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
          ... 7 more
  [INFO] ------------------------------------------------------------------------
  [INFO] BUILD FAILURE
  [INFO] ------------------------------------------------------------------------
  [INFO] Total time: 12.411 s
  [INFO] Finished at: 2017-06-29T12:16:15+05:30
  [INFO] Final Memory: 11M/27M
  [INFO] ------------------------------------------------------------------------
  [ERROR] Failed to execute goal org.codehaus.mojo:exec-maven-plugin:1.4.0:java (d
  efault-cli) on project Common: An exception occured while executing the Java cla
  ss. com/google/cloud/dataflow/sdk/options/PipelineOptions: com.google.cloud.data
  flow.sdk.options.PipelineOptions -&gt; [Help 1]
  [ERROR]
  [ERROR] To see the full stack trace of the errors, re-run Maven with the -e swit
  ch.
  [ERROR] Re-run Maven using the -X switch to enable full debug logging.
  [ERROR]
  [ERROR] For more information about the errors and possible solutions, please rea
  d the following articles:
  [ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoExecutionE
  xception
  </code></pre>
  
  <p>And the command that I ran was:</p>
  
  <pre><code>mvn compile exec:java -Dexec.mainClass=Common.StarterPipeline -Dexec.args="--project=******** --stagingLocation=gs://******** --dataflowJobFile=gs://********* --runner=TemplatingDataflowPipelineRunner"
  </code></pre>
  
  <p>The POM file:</p>
  
  <pre><code>&lt;project xmlns="http://maven.apache.org/POM/4.0.0"
      xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
      xsi:schemaLocation="http://maven.apache.org/POM/4.0.0 http://maven.apache.org/xsd/maven-4.0.0.xsd"&gt;
    &lt;modelVersion&gt;4.0.0&lt;/modelVersion&gt;
  
    &lt;groupId&gt;Common&lt;/groupId&gt;
    &lt;artifactId&gt;Common&lt;/artifactId&gt;
    &lt;version&gt;0.0.1-SNAPSHOT&lt;/version&gt;
  
    &lt;properties&gt;
      &lt;project.build.sourceEncoding&gt;UTF-8&lt;/project.build.sourceEncoding&gt;
    &lt;/properties&gt;
  
    &lt;repositories&gt;
      &lt;repository&gt;
        &lt;id&gt;ossrh.snapshots&lt;/id&gt;
        &lt;name&gt;Sonatype OSS Repository Hosting&lt;/name&gt;
        &lt;url&gt;https://oss.sonatype.org/content/repositories/snapshots/&lt;/url&gt;
        &lt;releases&gt;
          &lt;enabled&gt;false&lt;/enabled&gt;
        &lt;/releases&gt;
        &lt;snapshots&gt;
          &lt;enabled&gt;true&lt;/enabled&gt;
        &lt;/snapshots&gt;
      &lt;/repository&gt;
    &lt;/repositories&gt;
  
    &lt;build&gt;
     &lt;plugins&gt;
        &lt;plugin&gt;
          &lt;groupId&gt;org.apache.maven.plugins&lt;/groupId&gt;
          &lt;artifactId&gt;maven-compiler-plugin&lt;/artifactId&gt;
          &lt;version&gt;3.5.1&lt;/version&gt;
          &lt;configuration&gt;
            &lt;source&gt;1.8&lt;/source&gt;
            &lt;target&gt;1.8&lt;/target&gt;
          &lt;/configuration&gt;
        &lt;/plugin&gt;
      &lt;/plugins&gt;
  
      &lt;pluginManagement&gt;
        &lt;plugins&gt;
          &lt;plugin&gt;
            &lt;groupId&gt;org.codehaus.mojo&lt;/groupId&gt;
            &lt;artifactId&gt;exec-maven-plugin&lt;/artifactId&gt;
            &lt;version&gt;1.4.0&lt;/version&gt;
            &lt;configuration&gt;
              &lt;cleanupDaemonThreads&gt;false&lt;/cleanupDaemonThreads&gt;
            &lt;/configuration&gt;
          &lt;/plugin&gt;
        &lt;/plugins&gt;
      &lt;/pluginManagement&gt;
    &lt;/build&gt;
  
    &lt;dependencies&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;com.google.cloud.dataflow&lt;/groupId&gt;
        &lt;artifactId&gt;google-cloud-dataflow-java-sdk-all&lt;/artifactId&gt;
        &lt;version&gt;2.0.0&lt;/version&gt;
      &lt;/dependency&gt;
  
      &lt;!-- slf4j API frontend binding with JUL backend --&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.slf4j&lt;/groupId&gt;
        &lt;artifactId&gt;slf4j-api&lt;/artifactId&gt;
        &lt;version&gt;1.7.14&lt;/version&gt;
      &lt;/dependency&gt;
      &lt;dependency&gt;
        &lt;groupId&gt;org.slf4j&lt;/groupId&gt;
        &lt;artifactId&gt;slf4j-jdk14&lt;/artifactId&gt;
        &lt;version&gt;1.7.14&lt;/version&gt;
      &lt;/dependency&gt;
    &lt;/dependencies&gt;
  &lt;/project&gt;
  </code></pre>
  
  <p>Can somebody please help me with this. And please let me know if I'm doing anything wrong. Thanks</p>