Debug Pytest in docker container using VS Code - docker

I have troubles to setup debugging of py.test code in docker container using VS Code.
After studying this: https://code.visualstudio.com/docs/python/debugging
And this: How to remote debug python code in a Docker Container with VS Code
I have setup following debug configuration in vscode:
{
"name": "Python: Attach",
"type": "python",
"request": "attach",
"localRoot": "${workspaceFolder}",
"remoteRoot": "/capi",
"port": 3000,
"secret": "secret_text",
"host": "localhost"
}
I have imported this bit into my test file:
import ptvsd
ptvsd.enable_attach("secret_text", address = ('0.0.0.0', 3000))
ptvsd.wait_for_attach()
And I made sure I open that 3000 port in docker-compose file:
ports:
- 3000:3000
I double checked that the port is open:
nmap -p 3000 localhost
Starting Nmap 7.60 ( https://nmap.org ) at 2018-07-19 10:53 CEST
Nmap scan report for localhost (127.0.0.1)
Host is up (0.000074s latency).
PORT STATE SERVICE
3000/tcp open ppp
Nmap done: 1 IP address (1 host up) scanned in 0.11 seconds
It seems to be the case. When I run pytest file from the container it starts and waits for debugger to be connected:
===================================================== test session starts =====================================================
platform linux2 -- Python 2.7.15, pytest-3.5.1, py-1.5.3, pluggy-0.6.0
rootdir: /capi, inifile:
plugins: requests-mock-1.5.0, xdist-1.14, metadata-1.7.0, html-1.16.1, cov-2.5.1
collecting 0 items
But when I run this configuration from VS Code nothing seems to happen.
It seems to hang. Nothing in the debug console or in the docker container.
I have setup remote debug for a simple hello-world.py console app just for testing and it seems to work. So my assumption is it has something to do with the fact that I'm trying to debug a pytest.
Have anyone managed to do this? I would appreciate some help.

I encountered the same issue and your post almost solves the problem.
When I tried implementing your solution I encountered the following issue:
ImportError while loading conftest '/app/tests/conftest.py'.
tests/conftest.py:36: in <module>
ptvsd.enable_attach("secret_text", address=("0.0.0.0", 5678))
E TypeError: enable_attach() got multiple values for argument 'address'
Removing the "secret_text" value allowed me to hit the wait_for_attach() point and successfully attach the debugger to the code. I was able to hit breakpoints in my tests. Thank you!
.vscode/launch.json
{
"name": "Python: Attach",
"type": "python",
"request": "attach",
"localRoot": "${workspaceFolder}/path/to/code",
"remoteRoot": "/app",
"port": 5678,
"host": "localhost"
}
docker-compose.yml
ports:
- "5678:5678"
conftest.py
import ptvsd
ptvsd.enable_attach(address=("0.0.0.0", 5678))
ptvsd.wait_for_attach()
Note: The ptvsd lines are placed after all the imports.
CLI command to execute tests:
import subprocess
import click
#click.command()
def cli():
return subprocess.call("pytest test", shell=True)
Sequence to debug tests:
docker-compose up (get container running)
docker-compose exec MODULE CONTAINER_NAME FUNCTION_THAT_EXECUTES_TEST
Attach debugger in VSCode
Tests will execute and hit whatever breakpoint you have setup.

I came across this question before discovering that Visual Studio Code has had built-in-support for remote debugging docker containers since 2019!
A quick guide I wrote to getting started
The official documentation

Related

VS code terminal process failed to launch as debugging in attached running container

I'm trying to setup Airflow debugging environment with VS code using Remote - containers plugin. What I did so far:
docker-compose up the image apache/airflow:2.2.4 with Docker Desktop community 2.4 on macOS 10.13.6
Attach to the running airflow-scheduler container with Remote - containers plugin
Compose a launch.json file with
"version": "0.2.0",
"configurations": [
{
"name": "Airflow Test",
"type": "python",
"request": "launch",
// $ which airflow
"program": "/home/airflow/.local/bin/airflow",
"console": "integratedTerminal",
"args": [
"dags",
"test",
"task_of_middleware",
"2022-04-08"
],
"justMyCode": true
}
]
VS code pops up error below when I start debugging:
The terminal process failed to launch: Path to shell executable "/sbin/nologin" does not exist.
The attached running container shows error:
[1382 ms] Start: Run in container: for pid in `cd /proc && ls -d [0-9]*`; do { echo $pid ; readlink /proc/$pid/cwd ; readlink /proc/$pid/ns/mnt ; cat /proc/$pid/stat | tr "
[1518 ms] Container server: Error: spawn /sbin/nologin ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
[1525 ms] Error: spawn /sbin/nologin ENOENT
at Process.ChildProcess._handle.onexit (node:internal/child_process:282:19)
at onErrorNT (node:internal/child_process:477:16)
at processTicksAndRejections (node:internal/process/task_queues:83:21)
[1530 ms] Container server: (node:11888) PromiseRejectionHandledWarning: Promise rejection was handled asynchronously (rejection id: 1)
(Use `node --trace-warnings ...` to show where the warning was created)
I think the container runs normally since I'm able to connect to container shell via Docker Desktop and run Airflow test command. Any advice would be appreciated.
Not sure if this is the best practice, I found a way to ease the error and make debugger run normally. Put a config into the Attached container configuration files by select Remote-Containers: Open Container Configuration File from the Command Palette after attaching.
// Container user VS Code should use when connecting
"remoteUser": "root"
Reference here.

VSCode & Delve dap: Can't get the vscode debugger to work when dlv is launched using 'dlv dap'

I'm running VSCode 1.66.0 and I'm trying to attach the golang debugger on a running instance of a specific golang-docker-image. I'm able to do this when using 'dlv' but not when using 'dlv dap' and I can't understand why.
I'm following the instructions provided here:
https://vscode-debug-specs.github.io/go/#debugging-running-remote-process
Versions used:
VSCode ver. 1.66.0
dlv ver. 1.8.2
Docker Desktop ver. 4.4.4
My launch.json looks like this:
{
"version": "0.2.0",
"configurations": [
{
"name": "Attach to Remote Process",
"type": "go",
"request": "attach",
"mode": "remote",
"port": 2345,
"host": "127.0.0.1",
"showLog": true,
"apiVersion": 2,
"dlvLoadConfig": {
"followPointers": true,
"maxVariableRecurse": 1,
"maxStringLen": 200,
"maxArrayValues": 64,
"maxStructFields": -1
}
}
]
}
My dockerfile looks like so (as you can see the resulting image is based on the official image "mcr.microsoft.com/vscode/devcontainers/go:0-1.18"):
# syntax=docker/dockerfile:experimental
FROM golang:1.18 as builder
[...]
RUN CGO_ENABLED=0 GOOS=linux GOARCH=$ARCH go build -gcflags "all=-N -l" -v -o main .
FROM mcr.microsoft.com/vscode/devcontainers/go:0-1.18
WORKDIR /app
CMD ["./main"]
The command I use to launch the 'dlv' process inside the running docker instance is this:
dlv dap --listen=:2345 --log=true --api-version=2 attach 1 ./app/main
The VSCode UI allegedly allows me to attach via:
> Debug: Select and Start Debugging -> Attach to Remote Process
However the debugger doesn't really get activated in the sense that the breakpoints are not honored and the 'dlv' command shown above doesn't print anything in the output.
If I remove the 'dap' part from the 'dlv' command:
dlv --listen=:2345 --log=true --headless=true --log-output=debugger,debuglineerr,gdbwire,lldbout,rpc --api-version=2 --accept-multiclient --headless attach 1 ./app/main
then the debugger attaches perfectly and works just fine (this is the legacy mode of the dlv debugger if I understand correctly). What gives? Am I missing something?

PHP Xdebug, VSCode, Docker & MacOS - debugger not disconnecting

I've encountered a weird problem that seems to have either started appearing when the Xdebug extension was updated or Visual Studio Code received an update.
When the debugger is active in VSCode the website works fine, when stopping the debugger, the Xdebug extension within the docker container seems to be unable to disconnect. And the website is stuck in a loading state.
I receive the following message in the xdebug.log when trying to access the site when the debugger is stopped in VSCode.
[318] Log opened at 2022-04-08 12:19:15.819009
[318] [Step Debug] INFO: Connecting to configured address/port: host.docker.internal:9001.
[318] [Step Debug] INFO: Connected to debugging client: host.docker.internal:9001 (through xdebug.client_host/xdebug.client_port). :-)
[318] [Step Debug] -> <init xmlns="urn:debugger_protocol_v1" xmlns:xdebug="https://xdebug.org/dbgp/xdebug" fileuri="file:///var/www/html/index.php" language="PHP" xdebug:language_version="7.4.0" protocol_version="1.0" appid="318" idekey="VSCODE"><engine version="3.1.4"><![CDATA[Xdebug]]></engine><author><![CDATA[Derick Rethans]]></author><url><![CDATA[https://xdebug.org]]></url><copyright><![CDATA[Copyright (c) 2002-2022 by Derick Rethans]]></copyright></init>
This seems like the Xdebug extension still tries to connect (successfully) to the listening port at 9001.
But the port is not listening on the host, when checking with lsof -i :9001.
PHP Xdebug settings:
zend_extension=xdebug.so
xdebug.mode=debug
xdebug.remote_handler=dbgp
xdebug.idekey=VSCODE
xdebug.start_with_request=yes
xdebug.log=/tmp/xdebug.log
xdebug.client_host=host.docker.internal
xdebug.client_port=9001
xdebug.discover_client_host=0
launch.json settings:
{
"version": "0.2.0",
"configurations": [{
"name": "Listen for XDebug",
"type": "php",
"request": "launch",
"port": 9001,
"log": false,
"externalConsole": false,
"pathMappings": {
"/var/www/html": "${workspaceFolder}",
},
"ignore": [
"shared/vendor/*"
]
},
]
}
I've also tried with a different client_host setting.
Added an alias to the en0 interface, to reach the host IP from Xdebug.
sudo ifconfig en0 alias 10.128.128.128 255.255.255.0
No variation of settings seems to eliminate this problem, anyone has any ideas?
The 4.8.0 build solves this issue.
The Docker for mac ticket: https://github.com/docker/for-mac/issues/6235
The builds that resolves the issue:
Intel: https://desktop-stage.docker.com/mac/main/amd64/78146/Docker.dmg
Apple silicon: https://desktop-stage.docker.com/mac/main/arm64/78146/Docker.dmg

How to debug my Startup.cs within docker in VsCode (debugger is attached too late)

I managed to debug an asp .net core 3.1 application within an Ubuntu WSL-2 distro (on Windows 10), using a docker-compose file thanks to this Vs code tutorial
But considering we start the container using a docker-compose up command, then we attach the debugger, all breakpoints set in Program.cs or Startup.cs are not hit. Indeed the app has already pocessed these files when I manually attach the debugger, it's too late.
All other breakpoints (in controllers or background process ...) are correctly hit.
When debugging the same project with Visual Studio 2019 Community, all my breakpoints are hit, even those in Program.cs or Startup.cs which is the expected behavior here.
What am I doing wrong ?
The breakpoint l.44 is never hit
docker-compose.debug.yml (launched by docker-compose -f "docker-compose.debug.yml" up -d --build)
version: '3.7'
services:
myapp:
image: myapp
container_name: myapp
build:
context: .
dockerfile: src/MyApp/src/Dockerfile
ports:
- "60713:80"
environment:
- ASPNETCORE_ENVIRONMENT=Development
- ASPNETCORE_URLS=http://+:80
volumes:
- ~/.vsdbg:/remote_debugger:rw
networks:
default:
external:
name: mynetwork
launch.json
{
"version": "0.2.0",
"configurations": [
{
"name": "Docker .NET Core Attach (Preview)",
"containerName": "myapp",
"type": "docker",
"request": "attach",
"platform": "netCore",
"sourceFileMap": {
"/src": "${workspaceFolder}"
}
}
]
}
Indeed if I launch the VS code debugger before the container is up, I get the following error message
Error: Process 'docker exec -i "myapp...' exited with code 1
Error: Error: No such container: myapp
I opened an Issue on Github and here the provided responses:
From Brandon Waterloo [MSFT]
Right now our only supported story for that is to launch it manually and then attach.
We have an issue to add "real" compose debugging (microsoft/vscode-docker#840) ...
From Gregg Miskelly
... the only way to debug startup code is to add something like: while (!Debugger.IsAttached) { Thread.Sleep(100); } to the beginning of your startup code so that it will not proceed until a debugger is attached
So to conclude, until a solution is provided by Microsoft on VSCode or via an adhoc extension, one solution is to add some waiting code :
public static IHostBuilder CreateHostBuilder(string[] args)
{
if (Environment.GetEnvironmentVariable("ASPNETCORE_ENVIRONMENT") == Environments.Development)
{
while (!Debugger.IsAttached) { Thread.Sleep(100); }
}
Now breakpoints in Startup.cs are correclty hit

selenium inside windows docker container fails with ff/chrome "session deleted because of page crash"

DOCKER WITH SELENIUM AND ASP.NET 4.5 MVC
What am I trying to do?
Attempting to have a docker image that does following. Yes, I'm aware there are other ways to accomplish the end game here but I have a specific request on this.
Runs an ASP.NET MVC 4.5 web app
Has selenium driver via a c# console exe testing the code
Current status - when test is run in regular Windows 10 or Windows 2016 it runs fine. When the test is run in a Windows docker container it blows out with "session deleted because of page crash". Note I'm focusing on chrome tests but we get similar results using FireFox as well.
I turned on verbose debugging for selenium chrome and trapped out logfiles. I have two logfiles. "goodrun_log.txt" is from a Windows 10 successful test. "docker_log.txt" is the log from a failed run inside a container.
About line 473 we can see the docker run fail. Up to that point the log file is exactly the same as the good run. Then boom. So what are we missing that makes the docker container fail at that point?
Posted to Selenium at https://github.com/SeleniumHQ/selenium/issues/7165
Posted to ASP.NET docker repo at https://github.com/Microsoft/aspnet-docker/issues/181
SEEMS SIMILAR TO
Docker issue (linux) on aug 11 2015 - /dev/shm sizing - https://github.com/elgalu/docker-selenium/issues/20 by kkochubey1
Docker issue (linux) march 2018 - https://github.com/pranavgore09/fabric8-planner/pull/3
ChromeDriver - https://github.com/rshf/chromedriver/issues/772
Chromium bug (linux) - https://bugs.chromium.org/p/chromium/issues/detail?id=522853
THINGS I TRIED
chrome flags (many more than this but...)
option.AddArgument("--disable-dev-shm-usage"); // https://github.com/elgalu/docker-selenium/issues/20#issuecomment-407101358
driver retry like this https://github.com/electron/electron/issues/9369#issuecomment-312234465
SHM mode. Command runs but did not resolve issue
docker run -d --name aspnet48testsrun --shm-size="1g" -p 5000:80 aspnet48testsd
memory
docker run -d --name aspnet48testsrun -m inf --memory-swap inf -p 5000:80 aspnet48tests
docker run -d --name aspnet48testsrun -m 2g -p 5000:80 aspnet48tests
DOCKER RUN FAIL
[1556732925.450][DEBUG]: DevTools WebSocket Event: DOM.documentUpdated 7FCEC12C5F4ADEA352BBA3DF3AF6075D {
}
[1556732925.450][DEBUG]: DevTools WebSocket Command: DOM.getDocument (id=15) 7FCEC12C5F4ADEA352BBA3DF3AF6075D {
}
[1556732925.451][DEBUG]: DevTools WebSocket Response: Runtime.evaluate (id=14) 7FCEC12C5F4ADEA352BBA3DF3AF6075D {
"result": {
"type": "string",
"value": "http://localhost/"
}
}
[1556732925.531][DEBUG]: DevTools WebSocket Event: Inspector.targetCrashed 7FCEC12C5F4ADEA352BBA3DF3AF6075D {
}
[1556732925.532][INFO]: Waiting for pending navigations...
[1556732925.532][DEBUG]: DevTools WebSocket Command: Runtime.evaluate (id=16) 7FCEC12C5F4ADEA352BBA3DF3AF6075D {
"expression": "1"
}
[1556732925.532][INFO]: Done waiting for pending navigations. Status: unknown error: cannot determine loading status
from tab crashed
[1556732925.552][INFO]: [464b2b630c39434969f9b90e11b7aa37] RESPONSE Navigate ERROR unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=74.0.3729.108)
[1556732925.552][DEBUG]: Log type 'driver' lost 0 entries on destruction
[1556732925.552][DEBUG]: Log type 'browser' lost 0 entries on destruction
WIN10 RUN GOOD
[1556733552.098][DEBUG]: DevTools WebSocket Event: DOM.documentUpdated 193B5CE9ACD5F7CE56919120C68276A7 {
}
[1556733552.098][DEBUG]: DevTools WebSocket Command: DOM.getDocument (id=15) 193B5CE9ACD5F7CE56919120C68276A7 {
}
[1556733552.104][DEBUG]: DevTools WebSocket Response: Runtime.evaluate (id=14) 193B5CE9ACD5F7CE56919120C68276A7 {
"result": {
"type": "string",
"value": "http://localhost:29657/"
}
}
[1556733552.104][DEBUG]: DevTools WebSocket Response: DOM.getDocument (id=15) 193B5CE9ACD5F7CE56919120C68276A7 {
"root": {
"backendNodeId": 6,
"baseURL": "http://localhost:29657/",
"childNodeCount": 1,
"children": [ {
"attributes": [ ],
"backendNodeId": 7,
"childNodeCount": 2,
... lots more ...
SESSION DELETED BECAUSE OF PAGE CRASH
PS C:\seleniumtests> .\SeleniumDockerTest.exe http://localhost
[chrome options:] =[--headless --no-sandbox --disable-gpu]
Starting ChromeDriver 74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}) on port 49160
Only local connections are allowed.
Please protect ports used by ChromeDriver and related test frameworks to prevent access by malicious code.
[0501/120039.381:ERROR:network_change_notifier_win.cc(156)] WSALookupServiceBegin failed with: 0
[0501/120039.428:ERROR:audio_device_listener_win.cc(46)] RegisterEndpointNotificationCallback failed: 80070424
DevTools listening on ws://127.0.0.1:49163/devtools/browser/f33a8cd9-6411-46f5-a9ab-d69901cd53c1
[0501/120039.772:ERROR:network_change_notifier_win.cc(156)] WSALookupServiceBegin failed with: 0
[exception caught] =[OpenQA.Selenium.WebDriverException: unknown error: session deleted because of page crash
from unknown error: cannot determine loading status
from tab crashed
(Session info: headless chrome=74.0.3729.108)
(Driver info: chromedriver=74.0.3729.6 (255758eccf3d244491b8a1317aa76e1ce10d57e9-refs/branch-heads/3729#{#29}),platform=Windows NT 10.0.17763 x86_64)
at OpenQA.Selenium.Remote.RemoteWebDriver.UnpackAndThrowOnError(Response errorResponse)
at OpenQA.Selenium.Remote.RemoteWebDriver.Execute(String driverCommandToExecute, Dictionary`2 parameters)
at OpenQA.Selenium.Remote.RemoteWebDriver.set_Url(String value)
at OpenQA.Selenium.Remote.RemoteNavigator.GoToUrl(String url)
at SeleniumDockerTest.Program.DoChromeTests() in C:\dev\docker-selenium-aspnet45.git\SeleniumDockerTest\Program.cs:line 60]
TRY IT YOURSELF
There is a docker image with Windows, IIS, Chrome, FF and some tests at https://cloud.docker.com/repository/docker/jhealy62/devfish .
Pull it down the repo and provision it
docker pull jhealy62/devfish
docker run -d --name aspnettest -p 5000:80 jhealy62/devfish
Powershell into the container
docker exec -it aspnettest powershell
Inside the docker container, see the web server working
curl http://localhost -UseBasicParsing
See the seleniumtest failing:
cd \
cd \seleniumtests
.\SeleniumDockerTests.exe http://localhost
Cry with me!
NEXT STEPS
POSTED ISSUE - Aspnet docker github - https://github.com/Microsoft/aspnet-docker/issues/181
WHAT HAPPENED TO THE PAGE TIMEOUT ISSUE?
WebDriver Timeout error resolved -Occurs with either FireFox or Chrome tests inside docker container. FIX (requires both items below):
Install websocket's into the docker container. Excerpt from dockerfile:
RUN powershell -Command Add-WindowsFeature Web-WebSockets
Pass a very interesting set of options to the chromedriver.
option.AddArguments( "--headless","--disable-gpu", "--no-sandbox" );
Just in case I will leave my solution here, maybe for someone, it will be helpful)
So, the idea is to run the 'selenium/standalone-chrome' image in a separate container.
First of all setup your 'docker-compose.yml' file, something like that:
version: '3.8'
services:
chrome:
image: <your_storage>/standalone-chrome
restart: always
ports:
- 4444:4444
networks:
front:
ipv4_address: 172.16.238.5
net-worker:
build: <your_storage>/<your_project>
depends_on:
- chrome
networks:
front:
ipv4_address: 172.16.238.10
networks:
front:
driver: bridge
ipam:
config:
- subnet: 172.16.238.0/24
And then in code just connect to the exist chrome instance
var options = new ChromeOptions();
options.AddArguments("--headless");
options.AddArgument("no-sandbox");
_chrome = new RemoteWebDriver(new Uri("http://172.16.238.5:4444/wd/hub"), options);
Now you can use this chrome instance for uploading your site for testing.

Resources