Realised afterwards that I committed mild plagiarism in a paper I wrote as an undergraduate, what should I do?
but sometimes I would quote single sentences or parts of them directly from my sources without indicating that they were direct quotes (even if these unmarked quotes were always followed by a citation to the original source).


"You didn't use quotation marks where you should have in a paper you wrote as an undergrad? My god, you are despicable human being."

Nobody reading your question here will think this. 

Rather they will think something like:

"Wow, you wrote a journal paper on your own as an undergrad and it's still being cited?"

I would imagine that the vast majority of readers of your paper, even if they notice, would think likewise.



Though it's technically plagarism, I think it would be quite obvious to a reasonable reader that if your intent was to plagarise, you would not immediately cite the plagarised document you stole the quote from. Omitting quotation marks sounds like a minor transgression to me, particularly if your area is STEM where the (natural) language of the text is a means not an end. (If you did not have the citation(s) at all, I think it would be a different matter.)

Mistakes happen and this was clearly just a mistake.

(I would also add that I think it's pretty common for researchers to be somewhat embarrassed by earlier papers that they wrote when they were more junior. At least I am. But that's just a sign of improving.)


  I also don’t know what I could do about it. I have thought of putting something like a public admission and apology on my website (from which the paper can also be downloaded)


I would suggest two things:


Update the preprint on your homepage.
Keep a list of errata on your homepage.


In my area (CS) people mostly read the preprints (found through Google or Google Scholar). 


  public admission and apology


I don't think there's a need to apologise. I think you just need to admit your mistake and move on. Again, mistakes happen!