Scanning directories and caching paths to an array What am I trying to do

I'm trying to scan a disk and save (cache) all the paths of files and directories into an array so that I could perform a quick file/directory search later on.

Implementation

I'm using a Node.js module called walkdir.

cacheDisk () {
  let searchDir = 'C:/'
  let cache = []

  let options = {
    "max_depth": 5,       // only recurse down to max_depth. if you need more than no_recurse
    "track_inodes": true  // should be used with max_depth to prevent infinite loop. On windows or with hardlinks some files are not emitted due to inode collision.
  } 

  let paths = walkdir(searchDir, options, (path) =&gt; {
    cache.push(path)
  })
  .on('end', () =&gt; {
    console.log(cache)
    this.findItem(cache)
  })
},

findItem (cache) {
  let filename = 'file_1.txt'
  const matches = cache.filter(element =&gt; element.includes(filename))
  console.log(matches)
}


Questions / Problems:

FYI: I only need it to work in the latest Chrome (it's an Electron app)


Is using cache.push(path) a good way (performance-wise) to save paths to the array or is the module already doing it for me and I'm just repeating this action on every iteration? It takes about 30 seconds and 500-800 MB of RAM on a core i7 to finish that caching operation for my ssd with 1 million paths





Is using cache.filter(element =&gt; element.includes(filename)) a good / safe way to find an element in an array that can potentially be ~10,000,000 paths in length?





When I'm trying to scan disk's root (e.g. C:/) I get 2 slashes after the disk letter in every path (e.g. C:\\Program Files\test). But when I'm scanning something like C:/test/ everything is fine. I guess that's Node's doing? How do I avoid that? (do I just replace all \\ with \ in all the paths afterwards?)
I have to ask why you would want to find a file anywhere on the filesystem. That seems like a pretty odd requirement. Most apps (take node module loading for example) check a limited number of folders which can be done very quickly.

To answer your questions:


the OS does not do this kind of caching.
No, I wouldn't consider this safe. For two reasons. There are many other names that could match. For example video_file_1.txt, file_1.txt.old or some_path/file_1.txt/cat.jpg.  I would check against the basename something like

function basename(path) {
   return path.match(/[^\\\/]+$/)[0];
}
const matches = cache.filter(element =&gt; basename(element) == filename) 

Someone already answered this in the comments. It is really just an artifact of attempting to handle windows and unix paths consistently. 


Also if retrieval efficiency is important then I would store the data as a hash not an array, i.e:

let cache = {};

let paths = walkdir(searchDir, options, (path) =&gt; {
  const name = basename(path);
  if (!cache[basename])
    cache[basename] = [];  
  cache[basename].push(path)
});


Then finding an element is a simple matter of:

findItem (cache) {
  let filename = 'file_1.txt';
  const matches = cache[filename];
  console.log(matches);
}


Note that, since you are on Windows, you should also consider case-insensitivity of filenames.