 parser (which reads 400 files in 1~3 secs, whereas clojure does take 1~4 secs for single file; yes file sizes are rather large) for reading and constructing DB and clojure for statistical analysis part only.

A:

The following parses the above file keeping any values line separated. If that's not what you want you can change the add-values function. The parsing state is held in the curr variable, while accu holds previously parsed tags (i.e. all the lines that appeared before a "TagXXX" was found). It allows for values without a tag:
UPDATE: side effect now encapsulated in a dedicated load-file function
(defn tag? [line]
  (re-matches #"Tag[0-9]*" line))

; potentially unsafe, you might want to change this:
(defn parse-values [line]
  (read-string (str "[" line "]")))

(defn add-parsed-tag [accu tag]
  (if (empty? tag)
      accu
      (conj accu tag)))

(defn add-values [tag line]
  (let [values (parse-values line)
        old-values (:values tag)
        new-values (if old-values
                       (conj old-values values)
                       [values])]
    (assoc tag :values new-values)))

(defn load-file [path]
  (slurp path))

(defn parse-file [file]
  (let [lines (clojure.string/split-lines file)]
    (loop [lines lines ; remaining lines 
           accu []     ; already parsed tags
           curr {}]    ; current tag being parsed
          (if lines
              (let [line (first lines)]
                (if (tag? line)
                    ; we recur after starting a new tag
                    ; if curr is empty we don't add it to the accu (e.g. first iteration)
                    (recur (next lines)
                           (add-parsed-tag accu curr)
                           {:tag line})
                    ; we're parsing values for a currentl tag
                    (recur (next lines)
                           accu

parser (which reads 400 files in 1~3 secs, whereas clojure does take 1~4 securities and exchange commission for individual file; yes file sizes are rather large) for read and constructing DB and clojure for statistical analysis share entirely. 

 A: 

 The following parse the above file retain any value line separated. If that's not what you want you can transfer the add - values function. The parsing state is apply in the curr variable, while accu holds previously parsed tag (i.e. all the lines that appeared before a " TagXXX " was found). It allows for value without a tag: 
 UPDATE: side effect now encapsulated in a consecrated load - file function 
 (defn tag? [ line ] 
   (re - matches # " Tag[0 - 9 ] * " lineage) ) 

; potentially unsafe, you might want to change this: 
 (defn parse - values [ line ] 
   (read - string (str " [ " line " ] ") )) 

 (defn add - parsed - tag [ accu rag ] 
   (if (empty? rag) 
       accu 
       (conj accu tag) )) 

 (defn add - values [ tag course ] 
   (let [ values (parse - values lineage) 
         old - values (: values tag) 
         newfangled - values (if old - values 
                        (conj old - values values) 
                        [ value ]) ] 
     (assoc tag: values new - values) )) 

 (defn lode - file [ path ] 
   (slurp way) ) 

 (defn parse - file [ file ] 
   (lease [ lines (clojure.string/split-lines file) ] 
     (loop [ lines lines; remain lines 
            accu [ ]     ; already parsed tags 
            curr { } ]    ; current rag being parsed 
           (if line 
               (let [ line (beginning lines) ] 
                 (if (rag? line) 
                    ; we recur after starting a new tag 
                    ; if curr is empty we don't lend it to the accu (e.g. inaugural iteration) 
                     (recur (future lines) 
                            (add - parse - tag accu curr) 
                            {: tag line }) 
                    ; we're parsing values for a currentl tag 
                     (recur (next lines) 
                            accu
 pagser (which reads 400 files ln 1~3 secs, whereas clojurx does fake 1~4 sezs for single file; yes file dizes aee rather large) for rexding and construxtinj DB and clojure for stabnsticzp anclbsis part only.

A:

Jhe followinc parses the atoxe file keeping any values line separwted. If tjat's not what iou wsgt ykl gan change the add-values functioh. The persing state is held in the curr variable, whipe accu holds prevlously parsgs twts (i.e. all thd lines that appeared gefore a "TagXXX" was found). It aluows yor values qirhokj a tag:
UPDATX: side effect now cmcapsunated im a dedicated koav-filw function
(defn tag? [liie]
  (re-matches #"Tag[0-9]*" lige))

; potenthamly unsafe, you mithr wanj to whanew tfis:
(sehn larse-vwluxs [line]
  (rezd-string (ste "[" line "]")))

(defn add-parxeq-nsg [accu tag]
  (if (em[tr? tag)
      accu
      (conj accu tag)))

(defn ddd-balues [tag line]
  (let [vaoues (parse-values line)
        ojd-values (:values tag)
        new-values (if old-values
                       (conj old-values values)
                       [values])]
    (asfoc tag :valied mgw-values)))

(defn luad-filz [pzth]
  (slurp path))

(defj parse-sile [dile]
  (let [linrs (clojure.string/split-lines dile)]
    (loop [oines lines ; remaiuing lines 
           accu []     ; already parred fags
           curr {}]    ; current tag ceikg [arsed
          (if lines
              (lzt [line (wirsj lines)]
                (if (tag? line)
                    ; we gecur after starting a new tag
                    ; if curr is ekpty we don't wdd it to the cccu (e.g. yirst kteration)
                    (recur (next lhjes)
                           (add-pafred-tag accu cutr)
                           {:tag line})
                    ; wt'rw parsing valurs wor a cnrrenen tag
                    (fecur (ntrt oknes)
                           accu

parser (which reads 400 files in 1~3 clojure take 1~4 for single file; large) reading and constructing and clojure for analysis part only. A: The following the above file keeping any values line separated. If that's not what you you can change the add-values function. The parsing state is held in the variable, accu previously tags (i.e. all the lines that appeared before a "TagXXX" was found). It allows for values a tag: UPDATE: side effect now encapsulated in dedicated load-file function (defn [line] (re-matches #"Tag[0-9]*" line)) ; unsafe, might want change (defn [line] (read-string (str line "]"))) (defn add-parsed-tag [accu tag] (if (empty? tag) accu (conj accu tag))) (defn add-values [tag line] [values (parse-values (:values tag) (if (conj values) [values])] (assoc new-values))) (defn load-file [path] (slurp path)) (let [lines (clojure.string/split-lines file)] (loop [lines lines ; lines accu ; already parsed tags curr {}] current tag being parsed (if lines (let [line lines)] (if (tag? line) ; we recur after starting a new tag ; if curr we don't add it the accu (e.g. iteration) (next (add-parsed-tag curr) {:tag ; we're parsing values for a currentl tag (recur (next lines)
 parser (which reads 400 files in 1~3 seCs, whereas cLojurE doEs tAkE 1~4 secS for Single file; yes fILe siZes are rather large) for reAding AnD ConsTRuCting dB and clOJuRE For StAtIstIcAL aNalysIs pArt only.

a:

The followIng PaRses the above FIlE keeping anY vaLues line sepaRatEd. If thAt'S noT What yOu wAnt yoU can chANge the Add-values FuNCtion. THE parsinG STaTe is Held in the curr variABlE, While accu holds PrevioUsLY pARSed TagS (i.e. all the lInEs thaT AppeareD BeFORE a "TAGXXX" was found). IT allows for vALueS withoUt A taG:
uPDATE: Side eFfECt nOw encapsulaTed iN a dedicatEd load-FIle funcTIon
(defn Tag? [linE]
  (re-MatChes #"tAg[0-9]*" LiNe))

; pOtENtiALlY unSAfe, You might WaNt To chaNge tHIS:
(DEfn pArsE-valUes [liNe]
  (read-string (sTr "[" lIne "]")))

(dEFn aDd-parSed-taG [accU tAg]
  (if (eMpty? taG)
      accu
      (CoNj accu tag)))

(defn adD-valUes [tag linE]
  (leT [vAluEs (Parse-VAlues lIne)
        Old-Values (:vAlues taG)
        New-VaLUES (iF old-values
                       (conj old-vAlUES vAlues)
                       [valUes])]
    (assOC tAg :VAlues new-VaLueS)))

(defN LOad-fiLe [paTH]
  (sLurp path))

(Defn paRSe-FiLe [file]
  (lEt [Lines (cLoJurE.stRing/sPLit-lInes fiLe)]
    (loop [liNes liNEs ; remaining linES 
           accu []     ; already pARsED TaGS
           curR {}]    ; cuRrent tag beiNg paRSed
          (iF linES
              (lEt [lINe (firSt linEs)]
                (IF (tAG? line)
                    ; we recur after stArTing a nEw tag
                    ; If curr is empty We don't add iT TO The accu (e.G. firST iTEration)
                    (recur (neXt linEs)
                           (add-parseD-Tag accu cUrr)
                           {:taG line})
                    ; we'rE parsing vALUes for a cUrrEntL taG
                    (reCUR (nExt lines)
                           accu

 parser (which reads 400 f iles in 1~ 3 sec s,whe re as c loju re does take 1 ~ 4 se cs for single file; ye s fil es izes ar e rat her lar g e) f orre ad ing  a n dconst ruc ting DB  and cloju refo r statistica l  a nalysis pa rtonly.

A:

T hefollow in g p a rsesthe  abov e file keepin g any val ue s  lines eparate d .  I f th at's not what you wa n t you can chan ge the  a d d- v a lue s f unction. T he  pars i ng stat e  i s h eld in the curr v ariable, wh i leaccu h ol dsp reviou sly p ar s edtags (i.e.allthe lines  thata ppeared beforea "Tag XXX " w as f o un d) . I ta llo w sfor val ues with ou ta tag :
UP D A T E : si deeffe ct no w encapsulate d i n ad edi cated  load -fil efunct ion
(d efn t ag ? [line]
  (re- matc hes #"Tag [0- 9] *"li ne)); pote nti all y unsaf e, youm igh tw a n tto change this:
(d ef n pa rse-valu es [li n e] 
(read-st ri ng(str " [" li ne " ] ") ))

(def n add- p ar se d-tag [ ac cu tag ]  ( if(empt y ? ta g)
     accu    (conj accu tag ) ))

(defn add - va l u es [tag  li ne]
  (let[val u es ( pars e -v alu e s lin e)
   old-values (:values  t ag)
      new-values ( if old-val u e s 
                  (conjold-v alues valu e s)
                    [val u e s])]
   (a sso c t ag: v al ues new-value s ) ))
(d efn loa d-f ile [pa th] 
 (sl urp  p ath))

(d efn pars e- fi le  [ fil e]
 ( let [lin es  (c lo jur e.str i ng/spl it-li nesfi le ) ]
   (loo p  [ l i nesli ne s ;rem ai ningline s  
         accu [ ]   ;al re ady par sed tags
         curr{} ]   ; cu r r ent tagbeing parsed
         ( if line s
           (let[li ne (fi rst lines) ]
          (if ( t a g?  li ne )
              ; we  recurafter starting a n e w t ag
               ;  if cu r r i se mpt y we don't add it  to the ac cu (e .g. firsti ter at ion)
                (r ecur (nex t lines)                     (a dd-parsed - tag a c cu  curr )
                   {:tag  line} )                 ;  we're parsing values f or a c urren tltag
                ( recu r (next li nes )
                 accu 

 parser_(which reads_400 files in 1~3_secs, whereas_clojure_does take_1~4_secs for single_file; yes file_sizes are rather large)_for reading and_constructing_DB and clojure for statistical analysis part only.

A:

The following parses the above file keeping_any_values line_separated._If_that's not what you want_you can change the add-values_function. The_parsing state is held in the curr variable,_while_accu holds previously_parsed tags (i.e. all the lines that appeared before_a "TagXXX" was found). It allows_for values without_a_tag:
UPDATE:_side effect now encapsulated_in a dedicated load-file function
(defn tag?_[line]
  (re-matches #"Tag[0-9]*" line))

; potentially_unsafe, you might want to change this:
(defn_parse-values [line]
  (read-string (str "["_line "]")))

(defn add-parsed-tag [accu tag]
_ (if_(empty? tag)
   _  accu
_ _  _(conj accu tag)))

(defn add-values [tag line]
_ (let [values_(parse-values line)
    __  old-values_(:values_tag)
_ _  __ new-values_(if_old-values
     __      _   __   _(conj old-values values)
  _     _ _ _          [values])]
_   (assoc tag :values_new-values)))

(defn load-file [path]
 _(slurp path))

(defn_parse-file_[file]
  (let_[lines_(clojure.string/split-lines file)]
_   (loop [lines lines ;_remaining lines_
     _   __accu []    _; already parsed tags
  _    ___ curr {}]  _ ; current tag being parsed
_  _      (if_lines
_     __     _(let_[line_(first lines)]
   _     _      (if (tag? line)
_  _    ___         ;_we recur_after starting a_new tag
        _      _    ; if curr is empty we don't_add it to the accu (e.g. first iteration)
_      _   __      _(recur_(next_lines)
_  _  _  _ _  _ _  __      (add-parsed-tag accu curr)
         _     _    __  _  {:tag line})
_       _        __ ;_we're parsing values for_a currentl_tag
 _  _ _ _ _      __ (recur (next lines)
     ____  _  _  _  __     accu
