title: How to output multiple files dynamically using U-SQL custom Extractors?
body: <p>I have created a U-SQL query which reads the data from multiple files. Now I have to output the data in multiple files in some other folder. I have the below example,</p>
  
  <blockquote>
    <p><a href="https://stackoverflow.com/questions/42636855/u-sql-output-in-azure-data-lake/42676271#42676271">U-SQL Output in Azure Data Lake</a></p>
  </blockquote>
  
  <p>Used the below script,</p>
  
  <pre><code>        DECLARE @storagePath string = @"E:\";
          DECLARE @inputFileSetName string = @"dbo.file{*}.tsv"; 
          DECLARE @outputFileName string = @"dbo.files.csv";
  
          DECLARE @input string = String.Concat(@storagePath, 
          @inputFileSetName);
          DECLARE @output = string.Concat(@storagePath, @outputFileName);
          @searchlog =
          EXTRACT Id string,
          Name string,
          Address string
  
          FROM @input
          USING new USQLApplication3.SampleExtractor(Encoding.UTF8);
          @transactions =
          SELECT *,
          ROW_NUMBER() OVER(PARTITION BY Id ORDER BY Id DESC) AS 
          RowNumber_12345
          FROM @searchlog;
  
         @result =
         SELECT EXTRACT Id,
          Name,
          Address 
         FROM @transactions
         WHERE RowNumber_12345 == 1;
  
         OUTPUT @result 
         TO @output
         USING Outputters.Csv(encoding : Encoding.UTF8, quoting : false);
  </code></pre>
  
  <p>Below is the C# code for custom Extractors,</p>
  
  <pre><code>public class SampleExtractor : IExtractor
  {
      private Encoding _encoding;
      private byte[] _row_delim;
      private char _col_delim;
  
      public SampleExtractor(Encoding encoding, string row_delim = "\n", char col_delim = '\t')
      {
          this._encoding = ((encoding == null) ? Encoding.UTF8 : encoding);
          this._row_delim = this._encoding.GetBytes(row_delim);
          this._col_delim = col_delim;
      }
  
      public override IEnumerable&lt;IRow&gt; Extract(IUnstructuredReader input, IUpdatableRow output)
      {
          string line;
          //Read the input line by line
          foreach (Stream current in input.Split(_encoding.GetBytes("\n")))
          {
              using (StreamReader streamReader = new StreamReader(current, this._encoding))
              {
                  line = streamReader.ReadToEnd().Trim();
                  //Split the input by the column delimiter
                  string[] parts = line.Split(this._col_delim);
                  foreach (string part in parts)
                  {
                      string res = part;
                      if (res != null)
                      {
                          res = "\"" + res.Replace("\"", "\"\"") + "\"";
                      }
                      output.Set&lt;string&gt;(count, res);
                  }
  
              }
              yield return output.AsReadOnly();
  
          }
         yield break;
      }
  }
  </code></pre>
  
  <p>How can I <strong>dynamically create multiple files</strong> on output? As of now, it is creating only a single file.</p>
  
  <p><strong>Edited:</strong></p>
  
  <p>I have also tried with the multiple output in a single u-sql script,</p>
  
  <pre><code>      @x = SELECT * FROM (VALUES( "A", 10, 20), ("A", 11, 21), ("B", 10, 30), ("B", 100, 200)) AS T(name, value1, value2);
  
       // Generate the script to do partitioned output based on name column:
       DECLARE @out string ="E:/genscript.usql";
       @stmts = 
           SELECT "@res=SELECT value1, value2 FROM @x WHERE name == 
           \""+name+"\"; 
       OUTPUT 
            @res TO \"E:/"+name+".csv\" USING Outputters.Csv();" AS output 
            FROM (SELECT DISTINCT name FROM @x) AS x;
  
       OUTPUT @stmts TO @out
             USING Outputters.Text(delimiter:' ', quoting:false);
  
       @path = EXTRACT path string FROM @out USING Extractors.Text() ;
       OUTPUT @path  TO "E:/{*}.usql" 
             USING Outputters.Text(delimiter:' ', quoting:false);
  </code></pre>
  
  <p>But it is showing the following error,</p>
  
  <blockquote>
    <p>Severity   Code    Description Project File    Line    Suppression State
    Error       E_CSC_USER_READFROMPREVIOUSOUTPUT: The EXTRACT statement reads/references from location E:\genscript.usql that is the target of a previous OUTPUT statement.
    Description:
    Compiler does not support OUTPUT to and then reading/referencing from the same file in a single job.
    Resolution:
    Please make one job to OUTPUT to the file and a different job to read/reference the same file.  </p>
  </blockquote>
  
  <p><strong>What should I do now to output multiple files from a single u-sql or how I access another u-sql script file from current u-sql script?</strong></p>