Representation graph as Adjacency List in JavaScript
Strictly speaking, the list of vertices is not neccessary because the adjacency list already contains all (connected) vertices and disconnected ones can be represented as [x, null]. If you prefer to keep vertices, you'll have to synchronize them with edges:


in removeVertex you not only remove the vertex but also all edges connected to it
in addEdge you add input labels to the vertices list, if not already there (which makes one think that vertices should be a Set)


Minor notes:

In general, as of 2018, it's cleaner to use the class syntax to define classes.

In removeEdge you'd probably want &amp;&amp;, not ||: return startLabel === inputStartLabel &amp;&amp; endLabel === inputEndLabel. If the graph is undirected, you also have to check for reversed edges (start == inputEnd &amp;&amp; end == inputStart)

You didn't share the input data format and formatDataToAdjacencyList, but anyways, it would look better as a class member, not as an extra function.