How to index "almost" unique columns?
An index doesn't need to be unique to be efficient, and in fact I've rarely found making my indexes unique made much of a difference in terms of performance, in my experience. (It does in certain cases, data size dependent, but not enough to be measurably important to me so far.) On that note, my experience is within Microsoft SQL Server, so knowing which database system, version, and implementation (when applicable) would be helpful here.
That said, in most modern Relational Database Management Systems (RDBMS), the standard index is a B-Tree index. That means your data is stored in a B-Tree logical data structure which is much more efficient for searching than the entire unindexed Table itself. It doesn't matter if your data is &quot;almost&quot; unique vs perfectly unique. If your data was completely redundant (every value for the column(s) you anticipate indexing are exactly the same) then indexing probably won't help you much, but in your case a regular B-Tree index sounds like it would work very good.