Here's a cool little app I built with Pyllments. It allows you to compare the output of a large array of LLMs side by side. Just BYOAPIKEYS.
@litellm makes this a breeze.
@panel is responsible for the easily customizable frontend creation
Supermegaultraeasy deployment as well.