# Replacing matching text with a macro containing information from a CSV file

I have some data stored in a CSV file, in this manner:

000000001 @ name @ information @ more information @ some more information


I also have a information.tex file containing the numbers from the first column throughout (this file is \input into my main document):

\chapter{Some information about 000012311}
\chapter{Some other information}


I currently use a BASH script which replaces all numbers appearing information.tex matched to the first column of the CSV file with a macro, \mymacro{}{}{}{}, where each {} contains data from a column (second, third, fourth, and fifth).

For e.g., where the input file above contains This text is about 000000001., using the CSV file line above, it becomes This text is about \mymacro{name}{information}{more information}{some more information}., so when compiled, the text is displayed according to the definition of \mymacro. Essentially, I am using BASH to edit information.tex, after which I compile it.

Can I remove the need for BASH and do this directly with LuaTeX or some form of ConTeXt?

• Is it possible to create your CSV with commas to separate the contents? Also, does the CSV only contain unique entries? If so, this can most likely be done entirely within LaTeX using datatool. – Werner Jan 18 '12 at 6:27
• I used "@" because commas appear throughout the contents, however, commas with quotes around the content would be okay. The CSV file is easily changed. There are no duplicate rows and the items appearing in columns 1 and 2 are never identical. I tried datatool, but it seems to have difficulty with UTF-8. – Village Jan 18 '12 at 10:21

My approach is to read all lines from the file, split each line in it's arguments (divided by @) and store the arguments in a lua-table (tableItem) which is added to an other table (dataTable). We give the tableItem a name that we can access each item by name not by index. If we use the first argument of each line for the name we have direct access to the (sub)tables (tableItem) via the given number in the text using a macro called csvData. In this macro one can access the table and its arguments to format a latex-string or do whatever is needed.

\documentclass{book}

\def\csvData#1{%
\directlua{
%here one can access the stored arguments of the input line via the table item name
d=dataTable[#1];
tex.sprint(d.arg1..d.arg2..d.arg3..d.arg4..d.arg5)%
}
}

\directlua{%
local input = io.open('datafile.dat', 'r')
dataTable = {}
for line in input:lines() do
local split = string.explode(line,"@")
tableItem={split[1]}%item name is the first input argument of the line
tableItem.arg1=split[1]
tableItem.arg2=split[2]
tableItem.arg3=split[3]
tableItem.arg4=split[4]
tableItem.arg5=split[5]

table.insert(dataTable,tableItem)
end
input:close()
}%
}
\makeatother

\begin{document}
\noindent
\csvData{000000001}.\\
\csvData{000000004}.\\
\csvData{000000007}.\\
\end{document}


A simple approach is to call string.explode() for that. As you don't give the exact detail what goes where, I can only show the frame for that:

\documentclass{book}
\usepackage{luacode}
\begin{document}

\begin{luacode*}

for line in io.lines("somedata.dat") do
local line_t = string.explode(line,"@")

where somedata.dat looks like this:
000000001 @ name @ information @ more information @ some more information
{"extraction_info": {"found_math": true, "script_math_tex": 0, "script_math_asciimath": 0, "math_annotations": 0, "math_alttext": 0, "mathml": 0, "mathjax_tag": 0, "mathjax_inline_tex": 0, "mathjax_display_tex": 0, "mathjax_asciimath": 1, "img_math": 0, "codecogs_latex": 0, "wp_latex": 0, "mimetex.cgi": 0, "/images/math/codecogs": 0, "mathtex.cgi": 0, "katex": 0, "math-container": 0, "wp-katex-eq": 0, "align": 0, "equation": 0, "x-ck12": 0, "texerror": 0, "math_score": 0.5715088248252869, "perplexity": 2197.3013731450765}, "config": {"markdown_headings": true, "markdown_code": true, "boilerplate_config": {"ratio_threshold": 0.18, "absolute_threshold": 10, "end_threshold": 15, "enable": true}, "remove_buttons": true, "remove_image_figures": true, "remove_link_clusters": true, "table_config": {"min_rows": 2, "min_cols": 3, "format": "plain"}, "remove_chinese": true, "remove_edit_buttons": true, "extract_latex": true}, "warc_path": "s3://commoncrawl/crawl-data/CC-MAIN-2019-47/segments/1573496667262.54/warc/CC-MAIN-20191113140725-20191113164725-00181.warc.gz"}