How should I group rare Magic cards when selling them online?
Primarily, you need to judge how much your time is worth. In the general case, you can get more money per card by selling them as singles, or as small groups (such as playsets of 4). However, that will take a lot of time, especially for a large collection. Dumping your whole collection at your LGS for a bulk price is extremely quick and easy, but you're not going to get anywhere near as much money as selling each card individually online... but making all of those online sales will take a long time and require a lot of effort. (Shipping costs should also be considered.)

Rares and old desirable cards are more liquid, meaning it's easy to find someone willing to buy them, which means less time and effort spent selling them. Foils may or may not be liquid, depending on the cards in question.

I do not recommend trying to sell loose old packs over the internet. In the very oldest sets, it is possible to determine the exact contents of the pack before opening it, meaning the savvy buyers will refuse to purchase any packaged old cards unless they're in a factory-sealed box/case. (The assumption being that the seller has determined the pack contains nothing of worth already.)

Even in the sets new enough where that isn't possible, box mapping has been a thing for years, so a seller could pull out all of the worthwhile packs from a box before selling the rest; Wizards has made efforts to counteract this over the years, but they have only really been effective in the most recent sets (roughly Avacyn and onwards -- mapping is still possible, but it is no longer financially beneficial). The possibility of repacks also makes online buyers wary of purchasing loose packs from older sets.