title: Encog Neural Net : Normalization
body: <p>In Encog 3.x, how do you <strong>normalize</strong> data, <strong>use it</strong> for training, and <strong>denormalize</strong> results?</p>
  
  <p>There is no good documentation on this and a simple example that applies each of these would go a long way in reducing the learning curve on Encog. I haven't figured it all out yet, but here are some resources.</p>
  
  <p><strong>(1)</strong> *<em><a href="http://www.heatonresearch.com/content/how-does-encog-30-normalize" rel="nofollow">How does Encog 3.0 Normalize?</a></em>*</p>
  
  <p>This code is ok for saving a new normalized csv. It is not clear here though how to take the AnalystNormalizeCSV and convert it to an MLDataSet to actually use it.</p>
  
  <pre><code>EncogAnalyst analyst = new EncogAnalyst();
  AnalystWizard wizard = new AnalystWizard(analyst);
  wizard.wizard(sourceFile, true, AnalystFileFormat.DECPNT_COMMA);
  final AnalystNormalizeCSV norm = new AnalystNormalizeCSV();
  norm.analyze(sourceFile, true, CSVFormat.ENGLISH, analyst);
  norm.setOutputFormat(CSVFormat.ENGLISH);
  norm.setProduceOutputHeaders(true);
  norm.normalize(targetFile)
  </code></pre>
  
  <p><strong>(2)</strong> *<em><a href="http://www.heatonresearch.com/faq/4/2" rel="nofollow">How do I normalize a CSV file with Encog (Java)</a></em>*</p>
  
  <p>This code is, again, ok for producing a normalized csv output. But it is unclear on how to take the normalized data and actually apply it. There is a method for setting the target as an MLData, but it assumes all columns are inputs and doesn't leave room for any ideals. Furthermore, both of these options are difficult to use when the file has <strong>headers</strong> and/or <strong>unused columns</strong>.</p>
  
  <pre><code>try {
              File rawFile = new File(MYDIR, "iris.csv");
  
              // download Iris data from UCI
              if (rawFile.exists()) {
                  System.out.println("Data already downloaded to: " + rawFile.getPath());
              } else {
                  System.out.println("Downloading iris data to: " + rawFile.getPath());
                  BotUtil.downloadPage(new URL("http://archive.ics.uci.edu/ml/machine-learning-databases/iris/iris.data"), rawFile);
              }
  
              // define the format of the iris data
  
              DataNormalization norm = new DataNormalization();
              InputField inputSepalLength, inputSepalWidth, inputPetalLength, inputPetalWidth;
              InputFieldCSVText inputClass;
  
              norm.addInputField(inputSepalLength = new InputFieldCSV(true, rawFile, 0));
              norm.addInputField(inputSepalWidth = new InputFieldCSV(true, rawFile, 1));
              norm.addInputField(inputPetalLength = new InputFieldCSV(true, rawFile, 2));
              norm.addInputField(inputPetalWidth = new InputFieldCSV(true, rawFile, 3));
              norm.addInputField(inputClass = new InputFieldCSVText(true, rawFile, 4));
              inputClass.addMapping("Iris-setosa");
              inputClass.addMapping("Iris-versicolor");
              inputClass.addMapping("Iris-virginica");
  
              // define how we should normalize
  
              norm.addOutputField(new OutputFieldRangeMapped(inputSepalLength, 0, 1));
              norm.addOutputField(new OutputFieldRangeMapped(inputSepalWidth, 0, 1));
              norm.addOutputField(new OutputFieldRangeMapped(inputPetalLength, 0, 1));
              norm.addOutputField(new OutputFieldRangeMapped(inputPetalWidth, 0, 1));
              norm.addOutputField(new OutputOneOf(inputClass, 1, 0));
  
              // define where the output should go
              File outputFile = new File(MYDIR, "iris_normalized.csv");
              norm.setCSVFormat(CSVFormat.ENGLISH);
              norm.setTarget(new NormalizationStorageCSV(CSVFormat.ENGLISH, outputFile));
  
              // process
              norm.setReport(new ConsoleStatusReportable());
              norm.process();
              System.out.println("Output written to: " + rawFile.getPath());
  
          } catch (Exception ex) {
              ex.printStackTrace();
          }
  </code></pre>
  
  <p><strong>(3)</strong> *<em>Denormalizing</em>*</p>
  
  <p>I'm at a total loss for how to take all of this and denormalize according to the appropriate data-type's max's and min's.</p>