Which among an LoR for startup work and a "good student" LoR, will be more effective?
If the startup is showing evidence of success (e.g., it's actually raising significant venture capital or making money from clients), then if it were me, I would absolutely go with the letter from the startup.  They may be have been undergraduates, but now they are a Small Business Success Story, and you are part of that story.  It will help you stand out as unique, capable of taking initiative, and capable of working on ill-bounded problems as part of a team.

If, on the other hand, the startup is still little more than a collection of nebulous hopes, dreams, and semi-finished software, the professor (who is essentially neutral as far as recommendation value goes) is a safer bet.