Back-end debugging in Docker

[plankter]

Thank you for this project, it's very useful.

One of the missing things is a description of how to debug back-end services in Docker. I couldn't successfully manage how to debug in VSCode (via ptvsd) or PyCharm (via remote debug option). Probably some Traefik config should be changed in order to allow remote connections to interpreter.

[tiangolo]

Thanks for the report, I'll take a look into this.

[plankter]

It seems the solution is quite simple: one can add the following lines in main.py:

import ptvsd

ptvsd.enable_attach(redirect_output=True)

and then use such VSCode launch.json config as:

{
      "name": "Python: Remote Attach",
      "type": "python",
      "request": "attach",
      "port": 5678,
      "host": "localhost",
      "pathMappings": [
        {
          "localRoot": "${workspaceFolder}/backend/app",
          "remoteRoot": "."
        }
      ]
    }

[sepharg]

I've tried this approach and it's not working for me, at least using the latest ptvsd
i can connect to the debugger but breakpoints aren't hit
any help would be appreciated!

main.py

import ptvsd

from flask import Flask
app = Flask(__name__)

ptvsd.enable_attach(log_dir='/var/log/')

@app.route("/")
def hello():
    return "Hello World from Flask!"

if __name__ == "__main__":
    # Only for debugging while developing
    app.run(host='0.0.0.0', debug=True, port=80)

launch.json (i'm on windows)

{
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "port": 5678,
            "host": "localhost",
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}\\app",
                    "remoteRoot": "/app"
                }
            ]
}

Dockerfile

FROM tiangolo/uwsgi-nginx-flask:python3.6

COPY ./app /app
ADD requirements.txt /app/
WORKDIR /app

RUN pip install -r requirements.txt

ENV PTVSD_DEBUG 1

requirements.txt

ptvsd

[LukGerman]

I cannot help with VSCode, but I'd like to leave an instruction on how to do the same thing when using Pycharm. To debug backend app deployed locally using docker-compose, my preferred way is to use Pycharm's remote debugger.

Here is a documentation on how to set it up.

Few caveats that require a little explanation when debugging project deployed in a local docker environment:

1. In a 'run/debug configuration' window, as a Local host name use localhost. When creating a mapping, use the same mapping that is defined in docker-compose.dev.volumes.yml, which is <path in your host filesystem>/backend/app=/app.
2. Installation of the pydevd-pycharm~=<version of PyCharm on the local machine> can be added in backend/backend.dockerfile, just append "pydevd-pycharm~=193.6494" at the end of the pip install run. Version can be checked using Pycharm's menu bar -> help -> about.
3. Finally, add import pydevd_pycharm pydevd_pycharm.settrace('172.22.0.1', port=12345, stdoutToServer=True, stderrToServer=True) in backend/app/app/main.py. Replace IP address, with an address of your host. It can be checked using docker inspect <container-id-or-name> | grep Gateway (e.g. docker inspect projectslug_proxy_1 | grep Gateway).
   It would be even better to use some env var to enable debugging only in development.
   Now you can run your debugger, then docker-compose up and voila!

[cuzox]

What's up with this @tiangolo ?
This project is great, but not knowing how to debug it makes development a tedious headache.

[mashkovd]

Hello!
@tiangolo, i try instruction by @LukGerman, but can't debug backend via pycharm.
I need to put breakpoint to code,
Is it possible?
do you help me?
Thanks a lot!

[NMisko]

For pycharm i got it working by doing this:

1. Set up a remote interpreter: https://www.jetbrains.com/help/pycharm/using-docker-compose-as-a-remote-interpreter.html which is connected to the backend service
2. Add a file called run.py next to main.py

import uvicorn

if __name__ == "__main__":
    uvicorn.run(
        "main:app",
        host="0.0.0.0",
        port=8567,
        log_level="debug",
        reload=True,
        workers=1
    )

3. Add a pycharm configuration to run that file. Select your local main.py file and the remote python interpreter you created in step 2. Make sure the working directory is backend/app
4. Modify docker-compose-override.yml to expose backend port 8567, by adding the following line - "8567:8567" under - "8888:8888".
5. Modify your frontend to talk to that port, by adding :8567 to envApiUrl for dev in env.ts. (Skip this step if you don't need the frontend).
6. Add CORS for that port, by adding it to your .env.

Then to run it:

1. docker-compose up (as usual)
2. run or debug your configuration from step 3. It should now kill the backend service created by docker-compose and replace it with its own.

If anyone knows more about traefik and docker, there is probably an easier way that doesn't require me to add a new backend port and hardcode it in the frontend. Please tell me if there is :)

[nmdanny]

In light of @sepharg's suggestion, using debugpy which is the replacement of ptvsd(for VSCode users, or Vim/Emacs+DAP users), adapted to the current structure of the project:

1. Add the following to the settings at backend/app/core/config.py:

      DEBUG_MODE: bool = False
      WAIT_FOR_ATTACH: bool = False

2. Add the following dev depenency to backend/app/pyproject.toml, under [tool.poetry.dev-dependencies]:
   debugpy = "^1.2"

3. Add the following code to backend/app/main.py, right before the FastAPI constructor:

   if settings.DEBUG_MODE:
       import debugpy
       debugpy.listen(("0.0.0.0", 5432))
       if settings.WAIT_FOR_ATTACH:
           debugpy.wait_for_client()

   Optionally(not related to the debugger), add the argument debug=settings.DEBUG_MODE to the FastAPI constructor to have stacktraces upon API endpoint errors.

4. Add the port mapping

   - "5432:5432"

   (or any other external port you wish) to docker-compose.override.yml under services.backend.ports

5. Add DEBUG_MODE=True to the .env file and optionally WAIT_FOR_ATTACH=True

At this point, you should be able to connect to localhost:5432 via any debugger that supports DAP

For VSCode users, make the backend/app/.vscode/launch.json as follows: (assuming you open the VSCode editor at the backend/app folder)

{
    "configurations": [
        {
            "name": "Python: Remote Attach",
            "type": "python",
            "request": "attach",
            "connect": {
                "host": "localhost",
                "port": 5432
            },
            "pathMappings": [
                {
                    "localRoot": "${workspaceFolder}",
                    "remoteRoot": "/app"
                }
            ]
        }
    ]
}

This has worked for me, using Docker on Windows (WSL2 engine) and VSCode

[okz]

@nmdanny provided excelent instructions, unfortunately a typo on the port number, 5432 is for postgres, they should have been 5678, the default port for debugpy.

Another minor issue is in the pathmappings, if the IDE workspace, is the project root folder, should use:

                "localRoot": "${workspaceFolder}/backend/app",
                "remoteRoot": "."

[mshajarrazip]

@nmdanny's setup works for me - using docker on Ubuntu 20.04

[phil037]

Thanks @nmdanny and @okz for your solutions. It works for me.
Question for @okz :
Regarding 5432 being the default port for postgres -
Aren't we using port 5050 for postgres for this project? Why is it blocked then?
I'm newbie here. Sorry if the question is too naive