Does the FAA have a best practice for highlighting memory items on a checklist?
The short answer is to use color in some way.

I couldn't find anything very detailed on this: there's plenty of general checklist information out there from the FAA, NASA, universities and so on, but nothing that goes into specific details on how to use colors, fonts or the other things that you mentioned.

There is an FAA report called The Use and Design of Flightcrew Checklists and Manuals that was part of the response to the Northwest 255 crash. Unfortunately it's from 1991 so some content is already dated, and it's aimed at part 121 and 135 operators with multiple crew members, but it's an interesting report anyway.

Their main recommendation on memory items was simply to reduce them as much as possible because memory is very unreliable compared to checklists, but again that's in the context of multi-member crews. As for design, layout, formatting etc., they did make some specific recommendations (see section 4.1 and Appendix A) and the most applicable one to your question seems to be the use of color-coding. They strongly recommended the use of color as a general method, e.g. headings in yellow for "abnormal" items and in red for "emergency" items.

Interestingly, the report says that at that time (1991) airlines resisted using color because of the costs of printing, but today it seems to be the generally accepted way to highlight critical checklists and items. However, each checklist provider has their own approach to coloring headers, text, blocks or even entire pages.

I don't think there's any such thing as a standard 'style guide' for checklists, so if you're making your own then you can obviously do whatever you want that works for you. But the 1991 report noted that most checklist problems were caused by missing or inaccurate items, so make sure you get the content right before you break out your paintbrushes :-)