title: Clojure : parse a XML, remove head and tail lines first
body: <p>I come today with a problem I cannot solve.</p>
  
  <p><strong>Context</strong></p>
  
  <p>I have a catalog with this kind of input :</p>
  
  <pre><code>&lt;catalogue&gt;
    &lt;produit&gt;
      &lt;nom&gt;mince&lt;/nom&gt;
      &lt;sku&gt;25&lt;/sku&gt;
      &lt;criterias&gt;
        &lt;criteria&gt;65&lt;/criteria&gt;
        &lt;criteria&gt;25&lt;/criteria&gt;
      &lt;/criterias&gt;
    &lt;/produit&gt;
    &lt;produit&gt;
      &lt;nom&gt;gros&lt;/nom&gt;
      &lt;sku&gt;56&lt;/sku&gt;
      &lt;criterias&gt;
        &lt;criteria&gt;35&lt;/criteria&gt;
        &lt;criteria&gt;8&lt;/criteria&gt;
      &lt;/criterias&gt;
    &lt;/produit&gt;
  &lt;/catalogue&gt;
  </code></pre>
  
  <p>I want to transform it as EDN. In fact I succeed with sample data; This is the first step (then I do only lazy processing) :</p>
  
  <pre><code>(defn catalog-fr-to-edn []
    (let [content (slurp "catalog-fr.xml")]
      (-&gt;&gt; (xml/parse-str content)
           (into {}))))
  </code></pre>
  
  <p>But, as you can see - maybe I miss something - but I do not see other way to parse an XML from this library apart from slurping all the content as STR. The problem is that I have 700 mb of data !</p>
  
  <p>So I tought about something better</p>
  
  <p>1) First remove the "catalogue" lines</p>
  
  <pre><code>  &lt;produit&gt;
      &lt;nom&gt;mince&lt;/nom&gt;
      &lt;sku&gt;25&lt;/sku&gt;
      &lt;criterias&gt;
        &lt;criteria&gt;65&lt;/criteria&gt;
        &lt;criteria&gt;25&lt;/criteria&gt;
      &lt;/criterias&gt;
    &lt;/produit&gt;
    &lt;produit&gt;
      &lt;nom&gt;gros&lt;/nom&gt;
      &lt;sku&gt;56&lt;/sku&gt;
      &lt;criterias&gt;
        &lt;criteria&gt;35&lt;/criteria&gt;
        &lt;criteria&gt;8&lt;/criteria&gt;
      &lt;/criterias&gt;
    &lt;/produit&gt;
  </code></pre>
  
  <p>So that I have virtually N XML "files" that correspond to N products.</p>
  
  <p>2) Write records line by line like</p>
  
  <p>{:sku 25 ...}</p>
  
  <p><strong>Problem</strong></p>
  
  <p>I think the first step is ok (I did not see the tail of the file, the header is ok).He is the script for this first part </p>
  
  <pre><code>(defn remove-lines [input nskip]
    (let [path (-&gt;&gt; (decompose-filepath input)
                    (last)
                    (str "qsdqsdqsdqsd."))]
      (with-open [rdr (io/reader input)]
        (with-open [wrt (io/writer path)]
          (loop [n nskip]
            (let [line (.readLine rdr)]
              (cond (nil? line)
                      nil
                    (and (not (nil? line)) (not (empty? (re-find #"&lt;\\catalogue&gt;.*" line))))
                      nil
                    :else
                      (cond (pos? n)
                              (recur (dec n))
                            :else
                              (do (doto wrt (.write line) (.newLine))
                                            (recur n))))))))
      (io/delete-file input)
      (rename-file path input)))
  </code></pre>
  
  <p>Now I want to do the second step but I have no clue of how to do it. I could do this for 1 product but I don't know how to have something ion the file that save position in it.</p>
  
  <p>The XML parsing is ok so let's say I just want lines like that as output (I removes the newlines for visibility) :</p>
  
  <pre><code>"&lt;produit&gt;&lt;nom&gt;mince&lt;/nom&gt;&lt;sku&gt;25&lt;/sku&gt;&lt;criterias&gt;&lt;criteria&gt;65&lt;/criteria&gt;&lt;criteria&gt;25&lt;/criteria&gt;&lt;/criterias&gt;&lt;/produit&gt;"
  </code></pre>
  
  <p>i.e. read the content until  is reached, take the read article, writes it and then jump to the next article.</p>
  
  <p>Do I need other Java classes ? I can uwork with Java code if it's better.</p>
  
  <p>Or maybe I miss something from <code>clojure.data.xml</code> library ?</p>
  
  <p>Thanks</p>