<p>I have a webapp development problem that I've developed one solution for, but am trying to find other ideas that might get around some performance issues I'm seeing.</p>

<p>problem statement: </p>

<ul>
<li>a user enters several keywords/tokens</li>
<li>the application searches for matches to the tokens</li>
<li>need one result for each token

<ul>
<li>ie, if an entry has 3 tokens, i need the entry id 3 times</li>
</ul></li>
<li>rank the results

<ul>
<li>assign X points for token match</li>
<li>sort the entry ids based on points</li>
<li>if point values are the same, use date to sort results</li>
</ul></li>
</ul>

<p>What I want to be able to do, but have not figured out, is to send 1 query that returns something akin to the results of an in(), but returns a duplicate entry id for each token matches for each entry id checked.</p>

<p>Is there a better way to do this than what I'm doing, of using multiple, individual queries running one query per token? If so, what's the easiest way to implement those?</p>

<p><strong>edit</strong><br>
I've already tokenized the entries, so, for example, "see spot run" has an entry id of 1, and three tokens, 'see', 'spot', 'run', and those are in a separate token table, with entry ids relevant to them so the table might look like this:</p>

<pre><code>'see', 1 
'spot', 1 
'run', 1 
'run', 2 
'spot', 3 
</code></pre>

[{'answer_id': 47796, 'author': 'Robin Barnes', 'author_id': 1349865, 'author_profile': 'https://Stackoverflow.com/users/1349865', 'pm_score': 4, 'selected': True, 'text': '<p>you could achive this in one query using \'UNION ALL\' in MySQL.</p>\n\n<p>Just loop through the tokens in PHP creating a UNION ALL for each token:</p>\n\n<p>e.g if the tokens are \'x\', \'y\' and \'z\' your query may look something like this</p>\n\n<pre><code>SELECT * FROM `entries` \nWHERE token like "%x%" union all \n    SELECT * FROM `entries` \n    WHERE token like "%y%" union all \n        SELECT * FROM `entries` \n        WHERE token like "%z%" ORDER BY score ect...\n</code></pre>\n\n<p>The order clause should operate on the entire result set as one, which is what you need.</p>\n\n<p>In terms of performance it won\'t be all that fast (I\'m guessing), however with databases the main overhead in terms of speed is often sending the query to the database engine from PHP and receiving the results. With this technique this only happens once instead of once per token, so performance will increase, I just don\'t know if it\'ll be enough.</p>\n'}, {'answer_id': 47853, 'author': 'Erik', 'author_id': 4484, 'author_profile': 'https://Stackoverflow.com/users/4484', 'pm_score': 1, 'selected': False, 'text': "<p>If you're using the UNION ALL pattern you may also want to include the following parts to your query:</p>\n\n<pre><code>SELECT COUNT(*) AS C\n...\nGROUP BY ID\nORDER BY c DESC\n</code></pre>\n\n<p>While this is a really trivial example it does get you the frequency of the matches for each result and this could be a pseudo rank to start with.</p>\n"}, {'answer_id': 49301, 'author': 'Matt Sheppard', 'author_id': 797, 'author_profile': 'https://Stackoverflow.com/users/797', 'pm_score': -1, 'selected': False, 'text': '<p>You\'ll probably get much better performance if you used a data structure designed for search tasks rather than a database. For example, you might try looking at building an <a href="http://en.wikipedia.org/wiki/Inverted_index" rel="nofollow noreferrer">inverted index</a>. Rather than writing it youself, however, you might also want to look into something like <a href="http://en.wikipedia.org/wiki/Lucene" rel="nofollow noreferrer">Lucene</a> which does most of the work for you.</p>\n'}, {'answer_id': 49388, 'author': 'David McLaughlin', 'author_id': 3404, 'author_profile': 'https://Stackoverflow.com/users/3404', 'pm_score': 2, 'selected': False, 'text': '<p>I know this isn\'t strictly an answer to the question you\'re asking <strong>but if your table is thousands rather than millions of rows</strong>, then a FULLTEXT solution might be the best way to go here.</p>\n\n<p>In MySQL when you use MATCH on your indexed column, each keyword you supply will be given a relevance score (calculated roughly by the number of times each keyword was mentioned) that will be more accurate than your method and certainly more effecient for multiple keywords.</p>\n\n<p>See here:\n<a href="http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html" rel="nofollow noreferrer">http://dev.mysql.com/doc/refman/5.0/en/fulltext-search.html</a></p>\n'}]
['https://Stackoverflow.com/questions/47762', 'https://Stackoverflow.com', 'https://Stackoverflow.com/users/4418/']