🔥 💪 🎥 use o1 BEAST MODE to make your code more efficient

>>> shared a post earlier where I saw the team from @every used o1 pro to save BIG on cloud costs

💡 had the idea to ask o1 PRO how to make my code more programmatically efficient to save big on TIME

I have a script that I'm working to get right - takes all my local files from obsidian, embeds, then indexes them in chroma

here's a walkthrough of how I used o1 Pro to make this code run MUCH faster and more efficiently

here are the suggestions o1 Pro made:

🎯 remove chunk level context embeddings

🎯 increase chunk size and remove overlap

🎯 batch embeddings instead of uploading to vector db one by one

🎯 parallelize the job using joblib

now this script to embed and index local files to chroma is running probably 5x faster than before... ⚡️