JavaScript Website-Content Grabber I my firm a few people have following problem:

A Content Management System is hosted externally. The treaty doesn't include database-access.

In September the treaty will expire. So they have to get the content copied.

I have tried to figure out something. So this programming is a first-try. A prototype in the best-case. 

It works only with the HTML source-code of the articles from these news-site: heise.de

I could have used another site as well. It was just for having something to experiment with. 

Here's the code I have written so far: 



document.getElementById('exportText').addEventListener('click', function() {
  'use strict';
  var exportText = '';
  var re = /(data\-article\-type="meldung"&gt;)(.*)(&lt;\/article)/m;
  var reString  = '(&lt;h2 class="article__heading"&gt;)(.*?)(&lt;\/h2&gt;)|';
      reString += '(&lt;p class="meldung_anrisstext"&gt;)(.*?)(&lt;\/p&gt;)|';
      reString += '(&lt;h3 class="subheading"&gt;)(.*?)(&lt;\/h3&gt;)|';
      reString += '(&lt;p&gt;)(.*?)(&lt;\/p&gt;)';
  var matches  = [];
  var mainText = document.getElementById('source')
                         .value                            // Get the content from the first text-box.
                         .replace(/\r?\n|\r|\s{2,}/g, '')  // Remove line breaks; Multiple blanks to ONE blank.
                         .match(re)[0];                    // Restrict the area which is searched.

  re = new RegExp(reString, 'gm');

  // Search in the "meldung-wrapper"-container for h2-, h3-headlines and paragraphs.
  while ((matches = re.exec(mainText)) !== null) {
    exportText += ( matches[2] || matches[5] ||
                    matches[8] || matches[11] || '' ) + '\n\n';
  }

  document.getElementById('target').value = exportText;
});

document.getElementById('reset').addEventListener('click', function() {
  document.getElementById('source').value = '';
  document.getElementById('target').value = '';
});
* {
  background-color: #cdcdcd;
}

.wrap {
  width: 800px;
  margin: 10px auto;
}

label {
  font-weight: 900;
  margin: 10px 0;
  display: block;
}

textarea {
  background-color: #efefef;
  border-radius: 4px;
}

label,
button {
  font-family: verdana;
  sans-serif;
}

.control {
  display: inline-block;
  text-decoration: none;
  color: #fff;
  padding: 6px 8px;
  border: 1px solid;
  text-align: center;
  text-transform: uppercase;
  margin: 12px 5px;
  border-radius: 4px;
  background-color: #9a9a9a;
  font-size: .9rem;
}

.control:hover {
  background-color: #232323;
  color: #efefef;
}

.control:active {
  opacity: 0.6;
}

#target {
  padding: 4px 5px;
}
&lt;!doctype html&gt;
&lt;html&gt;

&lt;head&gt;
  &lt;title&gt;Exporter Script&lt;/title&gt;
  &lt;link rel="stylesheet" href="css/style.css" /&gt;
&lt;/head&gt;

&lt;body&gt;
  &lt;div class="wrap"&gt;
    &lt;div&gt;
      &lt;label for="target"&gt;HTML Source-Code&lt;/label&gt;
      &lt;textarea id="source" rows="10" cols="100"&gt;&lt;/textarea&gt;
    &lt;/div&gt;

    &lt;div&gt;
      &lt;label for="target"&gt;Exportierte Inhalte&lt;/label&gt;
      &lt;textarea id="target" rows="20" cols="100"&gt;&lt;/textarea&gt;
    &lt;/div&gt;

    &lt;div&gt;
      &lt;a class="control" href="#" id="exportText"&gt;Export starten&lt;/a&gt;
      &lt;a class="control" href="#" id="reset"&gt;Reset Textboxes&lt;/a&gt;
    &lt;/div&gt;
  &lt;/div&gt;
  &lt;script src="js/main.js"&gt;&lt;/script&gt;
&lt;/body&gt;

&lt;/html&gt;




You've got to paste the source-code of an article into the textbox "HTML Source Code". Then click the button. Then the second textbox contains the textual content of the article including inline-tags (hyperlinks, bold, italic etc. are desired to be exported too). 



What do you think about the approach?

- Is this a good approach? Or is there a better way to accomplish this task?

Please keep in mind that I don't have database-access.

- Are there points which can or must be improved in my programming? 

- Any ideas how to accomplish a better structure?

Looking forward to reading your hints, comments and recommendations.
Is this a good approach?


Sadly, nope.


  Or is there a better way to accomplish this task?


Yes.

Instead of manually sifting through pages and copying source code, use wget's site mirroring functionality to download the entire site. There are a lot of ways to do this with the various options wget has. HTTrack is also an option if you want the tool to mangle the structure, names, etc. But we only need to download the pages so wget is fine.

Next, instead of having a page that parses the page source for content, consider writing a Node.js script instead*. Use glob to grab the paths of all HTML files, cheerio to traverse and grab content from the HTML like jQuery, and Node's very own fs to read and write to files. Also, regular expressions are the worst tools to parse through HTML. That's a known fact, don't even go there.

In summary, you run wget to download the entire site, and the Node.js script to parse all the HTML that was downloaded. You'll probably spend half a day to a day tops to build this script and the next day you'll have parsed content in seconds.  You just saved 2 months worth of work right there.

With the answer above, the rest becomes irrelevant.



*Any language would actually do (Python, Ruby, PHP, etc). But since you wrote in JS, Node.js would probably make more sense.