Performing a "mean blur" I perform a mean blur on a number[][]-array (called grid in the code). I iterate the array, and calculate average values from adjacent "cells".
The kernel size defines how many adjacent values are used for average calculation.

/**
 * Performs a mean blur with a given kernel size.
 * @param {number[][]} grid - Data grid holding values to blur. 
 * @param {number} kernelSize - Describes the size of the "blur"-cut for each cell.
 */
static MeanBlur(grid, kernelSize) {
    // Result
    let newGrid = [];

    // Iterate original grid
    for (let row = 0; row &lt; grid.length; row++) {
        let newRow = [];
        for (let col = 0; col &lt; grid[row].length; col++) {

            let adjacentValues = [];

            // Get all adjacent values
            for (let i = 1; i &lt;= kernelSize; i++) {
                adjacentValues = adjacentValues.concat(this._getAdjacentValues(row, col, i, grid));
            }


            // Calculate average value
            const average = adjacentValues.reduce((a, b) =&gt; a + b, 0) / adjacentValues.length;

            // add average value to the current row
            newRow.push(average);
        }
        newGrid.push(newRow);
    }
    return newGrid;
}

/**
 * Return all adjacent values of a cell in a `number[][]`-array.
 * The amount of adjacent value depends on the kernel size.
 * @param {number} row - Describes the cell's row position.
 * @param {number} col - Describes the cell's column position. 
 * @param {number} kernelSize - The kernel size.
 * @param {number[][]} grid - Original data grid.
 */
static _getAdjacentValues(row, col, kernelSize, grid) {

    if (kernelSize &lt; 1) {
        throw "Kernel size value should be at least 1.";
    }

    let adjacentValues = [];

    // north
    if (row - kernelSize &gt;= 0) {
        adjacentValues.push(grid[row - kernelSize][col]);
    }
    // north-east
    if (row - kernelSize &gt;= 0 &amp;&amp; col + kernelSize &lt; grid[row].length) {
        adjacentValues.push(grid[row - kernelSize][col + kernelSize]);
    }
    // east
    if (col + kernelSize &lt; grid[row].length) {
        adjacentValues.push(grid[row][col + kernelSize]);
    }

    // south-east
    if (row + kernelSize &lt; grid.length &amp;&amp; col + kernelSize &lt; grid[row].length) {
        adjacentValues.push(grid[row + kernelSize][col + kernelSize]);
    }

    // south
    if (row + kernelSize &lt; grid.length) {
        adjacentValues.push(grid[row + kernelSize][col]);
    }

    // south-west
    if (row + kernelSize &lt; grid.length &amp;&amp; col - kernelSize &gt;= 0) {
        adjacentValues.push(grid[row + kernelSize][col - kernelSize]);
    }

    // west
    if (col - kernelSize &gt;= 0) {
        adjacentValues.push(grid[row][col - kernelSize]);
    }

    // north-west
    if (row - kernelSize &gt;= 0 &amp;&amp; col - kernelSize &gt;= 0) {
        adjacentValues.push(grid[row - kernelSize][col - kernelSize]);
    }

    return adjacentValues;
}


What I am most concerned with is _getAdjacentValues. I think there has to be a more convenient way to get adjacent values without including values at implausible indices.



What I mean is that i.e. at the grid position (0, 0) even with a kernel size of 1, values like (-1, -1) will be checked automatically by a loop.

Edit:
I was asked for clarification on how the kernel size actually is supposed to work. 
Assuming that I want to calculate the value for the current position (black cell). With a kernel size of 1 I would include all green cells for average value calculation. With a kernel size of 2 I would include all green and blue cell values. And with a kernel size of 3 all orange, blue and green cell values will be used for average calculation.
Convolution Filter
In CG this type of processing is call a convolution filter and there are many strategies used to handle edges
As the previous answer points out for performance you are best to use typed arrays and avoid creating arrays for each cell you process.
In your example

for (let i = 1; i &lt;= kernelSize; i++) {
    adjacentValues = adjacentValues.concat(this._getAdjacentValues(row, col, i, grid));
}
const average = adjacentValues.reduce((a, b) =&gt; a + b, 0) / adjacentValues.length;


You can do the reduce in the loop you create the sub array in, and avoid the 9 new arrays you create and the need to iterate again.
Example substitute value at edges.
const edgeVal = 0;
var val, sum = 0;
for (let i = 0; i &lt; 9; i++) {
    val = grid[row + (i / 3 | 0) - 1];
    if (val !== undefined) {
         val = val[col + (i % 3) - 1];
         sum += val !== undefined ? val : edgeVal;
    } else {
         sum += edgeVal;
    }
}
const average = sum / 9;

Or ignore edges
var val;
var sum = 0;
var count = 0;
for (let i = 0; i &lt; 9; i++) {
    val = grid[row + (i / 3 | 0) - 1];
    if (val !== undefined) {
         val = val[col + (i % 3) - 1];
         if (val !== undefined) {
            sum += val;
            count ++;
         }
    }
}
const average = sum / count;

Better yet, work on flattened arrays to avoid the double indexing for 2D arrays, and create the result array at the start rather than pushing to it each cell.
Workers
Convolution filters are ideally suited for parallel processing with an almost n / p performance increase (n is number of cells, p is number of processing nodes).
Using web workers on a i7 4 core (8 virtual cores) can give you a 8 times performance increase (On chrome (maybe others) you can get a core count at window.clientInformation.hardwareConcurrency)
Attempting to use more workers than available cores will result in slower processing.
GPU
For the ultimate performance the GPU, via webGL is available and will provide realtime processing of very large data sets (16M and above), A common non CG use is processing layers in convolutional neural network.
An example of a convolution filter (About halfway down the page) using webGL can easily be adapted to most data types, however doubles will only be available on a small number of hardware setups.