autogen/website/docs/Use-Cases/enhanced_inference.md
Logging  When debugging or diagnosing an LLM-based system, it is often convenient to log the API calls and analyze them.  ### For openai >= 1  Logging example: [View Notebook](https://github.com/microsoft/autogen/blob/main/notebook/agentchat_logging.ipynb)  #### Start logging: ```python import autogen.runtime_logging  autogen.runtime_logging.start(logger_type="sqlite", config={"dbname": "YOUR_DB_NAME"}) ``` `logger_type` and `config` are both optional. Default logger type is SQLite logger, that's the only one available in autogen at the moment. If you want to customize the database name, you can pass in through config, default is `logs.db`.  #### Stop logging: ```python autogen.runtime_logging.stop() ```  #### LLM Runs  AutoGen logging supports OpenAI's llm message schema. Each LLM run is saved in `chat_completions` table includes: - session_id: an unique identifier for the logging session - invocation_id: an unique identifier for the logging record - client_id: an unique identifier for the Azure OpenAI/OpenAI client - request: detailed llm request, see below for an example - response: detailed llm response, see below for an example - cost: total cost for the request and response - start_time - end_time  ##### Sample Request ```json {   "messages":[     {       "content":"system_message_1",       "role":"system"     },     {       "content":"user_message_1",       "role":"user"     }   ],   "model":"gpt-4",   "temperature": 0.9 } ```  ##### Sample Response ```json {   "id": "id_1",   "choices": [     {       "finish_reason": "stop",       "index": 0,       "logprobs": null,       "message": {         "content": "assistant_message_1",         "role": "assistant",         "function_call": null,         "tool_calls": null       }     }   ],   "created": "<timestamp>",   "model": "gpt-4",   "object": "chat.completion",   "system_fingerprint": null,   "usage": {     "completion_tokens": 155,     "prompt_tokens": 53,     "total_tokens": 208   } } ```  Learn more about [request and response format](https://platform.openai.com/docs/api-reference/chat/create)  ### For openai < 1  `autogen.Completion` and `autogen.ChatCompletion` offer an easy way to collect the API call histories. For example, to log the chat histories, simply run: ```python autogen.ChatCompletion.start_logging() ``` The API calls made after this will be automatically logged. They can be retrieved at any time by: ```python autogen.ChatCompletion.logged_history ``` There is a function that can be used to print usage summary (total cost, and token count usage from each model): ```python autogen.ChatCompletion.print_usage_summary() ``` To stop logging, use ```python autogen.ChatCompletion.stop_logging() ``` If one would like to append the history to an existing dict, pass the dict like: ```python autogen.ChatCompletion.start_logging(history_dict=existing_history_dict) ``` By default, the counter of API calls will be reset at `start_logging()`. If no reset is desired, set `reset_counter=False`.  There are two types of logging formats: compact logging and individual API call logging. The default format is compact. Set `compact=False` in `start_logging()` to switch.  * Example of a history dict with compact logging. ```python {     """     [         {             'role': 'system',             'content': system_message,         },         {             'role': 'user',             'content': user_message_1,         },         {             'role': 'assistant',             'content': assistant_message_1,         },         {             'role': 'user',             'content': user_message_2,         },         {             'role': 'assistant',             'content': assistant_message_2,         },     ]""": {         "created_at": [0, 1],         "cost": [0.1, 0.2],     } } ```  * Example of a history dict with individual API call logging. ```python {     0: {         "request": {             "messages": [                 {                     "role": "system",                     "content": system_message,                 },                 {                     "role": "user",                     "content": user_message_1,                 }             ],             ... # other parameters in the request         },         "response": {             "choices": [                 "messages": {                     "role": "assistant",                     "content": assistant_message_1,                 },             ],             ... # other fields in the response         }     },     1: {         "request": {             "messages": [                 {                     "role": "system",                     "content": system_message,                 },                 {                     "role": "user",                     "content": user_message_1,                 },                 {                     "role": "assistant",                     "content": assistant_message_1,                 },                 {                     "role": "user",                     "content": user_message_2,                 },             ],             ... # other parameters in the request         },         "response": {             "choices": [                 "messages": {                     "role": "assistant",                     "content": assistant_message_2,                 },             ],             ... # other fields in the response         }     }, } ```  * Example of printing for usage summary ``` Total cost: <cost> Token count summary for model <model>: prompt_tokens: <count 1>, completion_tokens: <count 2>, total_tokens: <count 3> ```   It can be seen that the individual API call history contains redundant information of the conversation. For a long conversation the degree of redundancy is high. The compact history is more efficient and the individual API call history contains more details. 