Skip to content

Commit

Permalink
(docs) control proxy debug using env vars
Browse files Browse the repository at this point in the history
  • Loading branch information
ishaan-jaff committed Jan 8, 2024
1 parent 5d7646b commit fa74831
Show file tree
Hide file tree
Showing 2 changed files with 49 additions and 7 deletions.
29 changes: 22 additions & 7 deletions docs/my-website/docs/proxy/quick_start.md
Original file line number Diff line number Diff line change
Expand Up @@ -611,16 +611,31 @@ print(result)
</Tabs>

## Debugging Proxy
Run the proxy with `--debug` to easily view debug logs

Events that occur during normal operation
```shell
litellm --model gpt-3.5-turbo --debug
```

When making requests you should see the POST request sent by LiteLLM to the LLM on the Terminal output
Detailed information
```shell
litellm --model gpt-3.5-turbo --detailed_debug
```

### Set Debug Level using env variables

Events that occur during normal operation
```shell
export LITELLM_LOG=INFO
```

Detailed information
```shell
export LITELLM_LOG=DEBUG
```

No Logs
```shell
POST Request Sent from LiteLLM:
curl -X POST \
https://api.openai.com/v1/chat/completions \
-H 'content-type: application/json' -H 'Authorization: Bearer sk-qnWGUIW9****************************************' \
-d '{"model": "gpt-3.5-turbo", "messages": [{"role": "user", "content": "this is a test request, write a short poem"}]}'
export LITELLM_LOG=None
```

27 changes: 27 additions & 0 deletions litellm/proxy/proxy_server.py
Original file line number Diff line number Diff line change
Expand Up @@ -976,6 +976,33 @@ async def initialize(
verbose_router_logger.setLevel(level=logging.DEBUG) # set router logs to info
verbose_proxy_logger.setLevel(level=logging.DEBUG) # set proxy logs to debug
litellm.set_verbose = True
elif debug == False and detailed_debug == False:
# users can control proxy debugging using env variable = 'LITELLM_LOG'
litellm_log_setting = os.environ.get("LITELLM_LOG", "")
if litellm_log_setting != None:
if litellm_log_setting.upper() == "INFO":
from litellm._logging import verbose_router_logger, verbose_proxy_logger
import logging

# this must ALWAYS remain logging.INFO, DO NOT MODIFY THIS

verbose_router_logger.setLevel(
level=logging.INFO
) # set router logs to info
verbose_proxy_logger.setLevel(
level=logging.INFO
) # set proxy logs to info
elif litellm_log_setting.upper() == "DEBUG":
from litellm._logging import verbose_router_logger, verbose_proxy_logger
import logging

verbose_router_logger.setLevel(
level=logging.DEBUG
) # set router logs to info
verbose_proxy_logger.setLevel(
level=logging.DEBUG
) # set proxy logs to debug
litellm.set_verbose = True

dynamic_config = {"general": {}, user_model: {}}
if config:
Expand Down

0 comments on commit fa74831

Please sign in to comment.