Should I drop my paper after finding a recent very similar paper?
First of all, you are absolutely right that you need to cite the paper. Citations should not be to "previous work you based your work on" but to "the state of the art" - whether or not you explicitly based your work on a previous article. The reader should be able to place your work in the context of what is already known on a topic.

Second, giving any advice will be hard for us, since we don't know the specific situation. If all you add to this previous article is a simpler design and collecting data in a different manner, then your paper likely does not offer a publication-worthy contribution to the state of the art. 

Your best bet is likely to rework your paper heavily. Identify weaknesses or open questions remaining in the previous article and address these. Then you can build on the previous article and expand upon it, and you will have a genuine contribution.

Of course, this will be a lot of work, and much of what you already did may turn out to be wasted effort. That sort of thing unfortunately happens in academia. You will need to analyze your specific situation to find out how much you can salvage.

The positive side is that you have found someone who can meaningfully review your work. And if, as you write, their work has so far mainly been self-cited, they will appreciate some external attention on it, so if you get them to review your paper, you may have a sympathy bonus. Alternatively, you could even contact the authors of the earlier paper and see whether they would be interested in collaborating with you.