Filtering numeric vectors by proximity to any elements of another vector in R
I do quite similar stuff.   I find your approach quite lengthy.  Especially the construction of the boolean matrix might be a bottleneck. 

First I would consider to write a second function - say filter_by_mz_ which just does the filtering and have filter_by_mz just take care of the input and call the new function - just to keep things modular.

In your algorithm you basically compare all masses pairwise.  A simpler approach might be to use expand.grid(q=query_mzs, t=target_mzs) this will calculate all pairs.  This list can be then easy checked (let's call it gr) using abs(gr$t-gr$q) &lt; ppm * gr$t - that would be fully vectorized.

A bool vector can usually be easy constructed using %in%.

Just FYI  I find it much easier to write with dplyr - especially since it is possible to keep much additional info in the table.  I often have the constraint that one mz can only be assigned once - extra data is often required to make such decisions.

Typically it looks like this:

Just the setup and some test data:

library(dplyr)

peaks &lt;- data.frame(mz=c(21.01,28.00,30,30.1,35, 21.02), val=c(1,2,3,4,5,6))
info &lt;- data.frame(mz=c(21.008, 28.003, 30.002, 32.02), str=c('wat', 'n2', 'aa' , 'bb'))


The magic happens here:

keyfun &lt;- function(x) sprintf('%.1f',x)
peaks &lt;- mutate(peaks, key=keyfun(mz))
info &lt;- mutate(info, key=keyfun(mz))

inner_join(peaks, info, by='key') %&gt;% filter( abs(mz.x - mz.y) &lt; 0.0005 * mz.x)


Basically I create an temporary key that is too wide - just that I only get to work on combinations that could be.  The inner_join will create all the permutations.  And in a second step I filter out the ones that actually match.

Of course usually I give more informative names and spend some time labelling stuff.  But this should suffice to give an idea of the method.

addition

It came to me that you could avoid the matrix if you could rewrite such that inside the loop you check for all query_mz instead of target_mz - then you could also put the any inside the loop.  I suppose it would come down to just switching the variable names in a number of lines. 
After this transformation the for loop should be replacable by an sapply.

As R stores logicals quite expensive with 4 bytes per logical. That shoud give you extensive savings. (For me these numbers would go into the 100s so I would have >99% less memory usage)