autogen/samples/tools/autogenbench/CONTRIBUTING.md
Task Instance Expansion Algorithm  Once the tasks have been defined, as per above, they must be "instantiated" before they can be run. This instantiation happens automatically when the user issues the `autogenbench run` command and involves creating a local folder to share with Docker. Each instance and repetition gets its own folder along the path: `./results/[scenario]/[task_id]/[instance_id]`. For the sake of brevity we will refer to this folder as the `DEST_FOLDER`.  The algorithm for populating the `DEST_FOLDER` is as follows:  1. Pre-populate DEST_FOLDER with all the basic starter files for running a scenario (found in `autogenbench/template`). 2. Recursively copy the template folder specified in the JSONL line to DEST_FOLDER (if the JSON `template` attribute points to a folder) If the JSONs `template` attribute instead points to a file, copy the file, but rename it to `scenario.py` 3. Apply any string replacements, as outlined in the prior section. 4. Write a run.sh file to DEST_FOLDER that will be executed by Docker when it is loaded. The `run.sh` is described below. 