d20pfsrd Discord Bot Scraper Abstraction and Deployment I wrote a discord bot that listens for links to the d20pfsrd rulebook in discord chat, checks if they are feats or magic, and then scrapes the page, formats it, and spits it back into chat as formatted text. The pages I'm checking are formulaic enough, so even though the cheerio selectors I'm using are probably fairly brittle, it seems to work in most cases. The text formatters are some ugly beasts though (though the logic is probably unique), and the axios calls could use some cleaning up as well.
My problem is as follows: I suspected a good 40% of the code can be refactored out of existence if I properly generalized it. My solution was to do something that could probably be described as an abuse of the spread operator:
async function getPage(msg, url){
  let message = &quot;placeholder&quot;;
  await axios.get(url).then( (response) =&gt; {
      message = responder(...selectResponder(url), response);
    } 
    ).catch((error) =&gt; {
      console.error(error.message);
    });
  sendMessage(message, msg);
  
}

A week later I'm now realizing I wrote the functions in a way that causes them to read backwards? Aesthetics aside, this is how it ends up working out:
function selectResponder(siteUrl){
  if (siteUrl.startsWith(&quot;https://www.d20pfsrd.com/feats/&quot;)){
      return feats.featsConfig;
    } else if (siteUrl.startsWith(&quot;https://www.d20pfsrd.com/magic/&quot;)){
      return magic.magicConfig;
  }
}

function responder(parser, formatter, messageFormatter, response){
    const replyData = parser(response.data);
    const formattedData = formatter(replyData);
    return messageFormatter(formattedData);
}

The idea is that as much unique logic as I could separate out exists in magic.js and feats.js as possible. How can I clean this section up? The github repo is here and I would love and feedback on the entire project. I did most of that because I thought it as weird to have a big if block here and duplicating the axios calls:
client.on('message', msg =&gt; {
  

  if(validateUrl(msg.content)){
    const siteUrl = encodeURI(msg.content);
    getPage(msg, siteUrl);
  }
  

});

Also I wrote up this gist on how I deployed it to a raspberry pi. That works, so suggestions on how to improve it would be greatly appreciated
In broad strokes this program:

Listens for a message
Validates the message
Gets the page content at the message url
Parses the page content
Re-formats the page content
Converts the page content into a message
Sends the message
Here's how I would do it. Move the logic about which pages a parser consumes into the code for said parser. Also organizing your helper files into objects could reduce complexity in terms of the number of variables you pass around but that's more a matter of personal preference.
class SpecificPageParser {
    match(url) {
        return (
           url.startswith('https://www.d20pfsrd.com/magic/') &amp;&amp;
           !url.endsWith('/magic/')
        );
    }
    format(response) {
        // do your parsing stuff
        const formatted = {...response,'extra':'info'};
        return formatted;
    }
}

const responders = [new SpecificPageParser(),new SomeOtherParser()];

async function getPage(msg, url){
    try {
        const responder = responders.find(responder=&gt;responder.match(url));
        if (!responder) return;
        const response = await axios.get(url);
        const message = responder.format(response);
        sendMessage(message, msg);
    }
    catch(error) {
        console.error(error);
    }
  }
```