I can not send my spiders.
I use one of the following codes:
docker run -i -t --rm -v /home/raphael/Documents/entreprise/portia/portia-master/test:/app/data/projects:rw -v /home/raphael/Documents/entreprise/portia/res:/mnt:rw -p 9001:9001 scrapinghub/portia \
portiacrawl /app/data/projects/Oscaro www.oscaro.com -o /mnt/Oscaro.jl
docker run -i -t --rm -v /home/raphael/Documents/entreprise/portia/portia-master/test:/app/data/projects:rw -v /home/raphael/Documents/entreprise/portia/res:/mnt:rw -p 9001:9001 scrapinghub/portia \
portiacrawl /app/data/projects/Oscaro
The console me return:
+ action=portiacrawl
+ shift
+ '[' -z portiacrawl ']'
+ case $action in
+ exec portiacrawl /app/data/projects/Oscaro www.oscaro.com -o /mnt/Oscaro.jl
Usage: portiacrawl <project dir/project zip> [spider] [options]
Allow to easily run slybot spiders on console. If spider is not given, print a
list of available spiders inside the project
Options:
-h, --help show this help message and exit
--settings=SETTINGS Give specific settings module (must be on python path)
--logfile=LOGFILE Specify log file
-a NAME=VALUE Add spider arguments
-s NAME=VALUE Add extra scrapy settings
-o FILE, --output=FILE
dump scraped items into FILE (use - for stdout)
-t FORMAT, --output-format=FORMAT
format to use for dumping items with -o (default:
jsonlines)
-v, --verbose more verbose
However, this seems to be a good adaptation of the documentation code:
docker run -i -t --rm -v <PROJECTS_FOLDER>:/app/data/projects:rw -v <OUPUT_FOLDER>:/mnt:rw -p 9001:9001 scrapinghub/portia \
portiacrawl /app/data/projects/PROJECT_NAME SPIDER_NAME -o /mnt/SPIDER_NAME.jl
I am completely new to docker, portia and scrapy.
I have trouble idantifying the sorce of the problem.
By the way, I did not understand the solution proposed here:
https://emu.one/scrapy/823487/how-do-i-start-running-portia-spider-how-to-do-it.html
I do not know if this solution concerns me since it does not seem to be used docker.
I also have a question about the first part of the code. I would like to know what is done that I write:
-v /home/raphael/Documents/entreprise/portia/portia-master/test:/app/data/projects
I thank you in advance
It was necessary to put a relative path. :(
docker run -i -t --rm -v ~/Documents/entreprise/portia/portia-master/test:/app/data/projects:rw -v ~/Documents/entreprise/portia/res:/mnt:rw -p 9001:9001 scrapinghub/portia \
portiacrawl /app/data/projects/Oscaro www.oscaro.com -o /mnt/Oscaro.jl
Related
I have generated a openapi json file, and I wish to create a typescript client using docker.
I have tried to do something similar to what is on the openapi generator site (https://openapi-generator.tech/ - scroll down to docker part), but it doesn't work.
Command from site:
docker run --rm \
-v $PWD:/local openapitools/openapi-generator-cli generate \
-i /local/petstore.yaml \
-g go \
-o /local/out/go
What I have tried:
docker run --rm -v \
$PWD:/local openapitools/openapi-generator-cli generate -i ./openapi.json \
-g typescript-axios
No matter what I do, there is always a problem with the ./openapi.json file. The error which occours:
[error] The spec file is not found: ./openapi.json
[error] Check the path of the OpenAPI spec and try again.
I have tried the things below:
-i ~/compass_backend/openapi.json
-i openapi.json
-i ./openapi.json
-i $PWD:/openapi.json
cat openapi.json | docker run .... (error, -i is required)
I am out of ideas. The error is always the same. What am I doing wrong?
I was able to solve the problem by switching from bash to powershell. Docker uses windows path notation and I was trying to use bash notation. If you type pwd in bash you get this:
/c/Users/aniemirka/compass_backend
And if you type pwd in powershell you get this:
C:\Users\aniemirka\compass_backend
So docker was trying to mount a volume to /c/Users/aniemirka/compass_backend\local, and it couldn't read it because it is not windows notation, so the volume didn't exist.
//Runs android sdk on container
docker run -it — rm -v $(pwd)/sdk:/sdk thyrlian/android-sdk bash -c 'cp -a $ANDROID_HOME/. /sdk'
Throws below error
docker: invalid reference format.
See 'docker run --help'.
You're looking too far in the command for the error. In this case it's earlier. In this command:
docker run -it — rm -v $(pwd)/sdk:/sdk thyrlian/android-sdk bash -c 'cp -a $ANDROID_HOME/. /sdk'
The invalid image name is the long dash: —. The rm needs two normal dashes (since it's not a single character style arg like the -i and -t). Those are dashes that you'd type with the keyboard, and some editor that you've likely copy and pasted from converted that. The correct command is:
docker run -it --rm -v "$(pwd)/sdk:/sdk" thyrlian/android-sdk bash -c 'cp -a $ANDROID_HOME/. /sdk'
Note, you should always quote the path since it can contain a space that would also break the parsing of the args.
What's the exact command you need to run in the interactive terminal?
In your case,
"- rm" is incorrect, there should no space between - and rm, so it should be -rm
Secondly, it should be --rm and re try.
docker run -it --rm -v $(pwd)/sdk:/sdk thyrlian/android-sdk bash -c 'cp -a $ANDROID_HOME/. /sdk'
The issue happens when the docker image name given in the command is invalid. Please try this command
docker run -it — rm -v "$(pwd)/sdk:/sdk thyrlian/android-sdk" bash -c 'cp -a $ANDROID_HOME/. /sdk'
I have Firefox nightly running in a container. I'm looking for a solution to configure it as my default browser application(ubuntu 18.04).
So my question is, how to configure a Docker container as default system application in Ubuntu.
My docker command is:
docker run -d --net=host -v ~/:/home/firefox -v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=unix:0 -v /dev/shm:/dev/shm --device /dev/snd \
--group-add 29 -e PULSE_SERVER=unix:/run/user/1000/pulse/native \
-v /run/user/1000/pulse/native:/run/user/1000/pulse/native \
firefox-nightly
I suppose I must create a new mime file, but not sure how to do it, to be able to create the container with all these parameters.
Thanks
One alternative is to create a new .desktop file (e.g: /usr/share/applications/firefox-docker.desktop).
I just copied the existing firefox.desktop and changed Exec sections with the command using docker (*)
Then use xdg-utils (**) configure it as default browser application:
xdg-settings set default-web-browser firefox-docker.desktop.
*: To keep the .desktop file cleaner, you could create an executable file in system PATH (e.g: /usr/bin): docker-firefox:
xhost +
docker run --net=host -v ~/:/home/firefox -v /tmp/.X11-unix:/tmp/.X11-unix \
-e DISPLAY=unix:0 -v /dev/shm:/dev/shm --device /dev/snd \
--group-add 29 -e PULSE_SERVER=unix:/run/user/1000/pulse/native \
-v /run/user/1000/pulse/native:/run/user/1000/pulse/native \
firefox-nightly $#
Note the $# at the end. And make it executable so it can be executed as a normal application.
**: The link is from Arch documentation, but it works in Ubuntu as well.
I am new to Docker and trying to understand but I have noticed the Visual Studio does a lot of 'magic' behind the scenes. I have managed to figure out all my questions about the docker run command VS uses when you debug an ASP.NET Core app with Docker support except one.
docker run
-dt
-v "C:\Users\jnhaf\vsdbg\vs2017u5:/remote_debugger:rw"
-v "D:\ProtoTypes\WebAppDockerOrNot\WebAppDockerOrNot:/app"
-v "C:\Users\jnhaf\AppData\Roaming\ASP.NET\Https:/root/.aspnet/https:ro"
-v "C:\Users\jnhaf\AppData\Roaming\Microsoft\UserSecrets:/root/.microsoft/usersecrets:ro"
-v "C:\Users\jnhaf\.nuget\packages\:/root/.nuget/fallbackpackages2"
-v "C:\Program Files\dotnet\sdk\NuGetFallbackFolder:/root/.nuget/fallbackpackages"
-e "DOTNET_USE_POLLING_FILE_WATCHER=1"
-e "ASPNETCORE_ENVIRONMENT=Development"
-e "ASPNETCORE_URLS=https://+:443;http://+:80"
-e "ASPNETCORE_HTTPS_PORT=44328"
-e "NUGET_PACKAGES=/root/.nuget/fallbackpackages2"
-e "NUGET_FALLBACK_PACKAGES=/root/.nuget/fallbackpackages;/root/.nuget/fallbackpackages2"
-p 4800:80
-p 44328:443
--entrypoint tail webappdockerornot:dev -f /dev/null
The final argument --entrypoint tail webappdockerornot:dev -f /dev/null is the one that confuses me. I get that VS is overriding the entry point setup in the Dockerfile but what I do not understand nor can find online is what tail webappdockerornot:dev and the -f /dev/null. I figured out that webappdockerornot:dev is the docker image but can someone explain how this argument works or provide a link to something that explains it.
We can break down that command line a little differently as
docker run \
... some other arguments ... \
--entrypoint tail \
webappdockerornot:dev \
-f /dev/null
and match this against a general form
docker run [OPTIONS] [IMAGENAME:TAG] [CMD]
So the --entrypoint tail option sets the entry point to tail, and the "command" part is -f /dev/null. When Docker actually launches the container, it passes the command as additional arguments to the entrypoint. In the end, the net effect of this is
Ignore what the Dockerfile said to do; after setting up the container runtime environment, run tail -f /dev/null instead.
which in turn is a common way to launch a container that doesn't do anything but also stays running. Then you can use docker exec and similar debugging-oriented tools to do things inside the container.
Really new to docker and docker infrastructure. So from the official Zalenium github page I followed all the steps successfully until I get to the point:
docker run --rm -ti --name zalenium -p 4444:4444 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp/videos:/home/seluser/videos \
--privileged dosel/zalenium start
and after executing it I'm getting:
> docker: invalid reference format.
> See 'docker run --help'
While doing research here on SOF, I've landed here and here, but nothing seems to work in my situation. I've tried to replace --rm -ti to the end of the expression: didn't work
docker run --name zalenium -p 4444:4444 \
-v /var/run/docker.sock:/var/run/docker.sock \
-v /tmp/videos:/home/seluser/videos \
--privileged --rm -ti dosel/zalenium start
I'm sure there is a specific format that I have to follow, but can't find a clue to it (missing comma, wrong backslash, unnecessary whitespace). Appreciate for your help.
I just noticed that the command example we have does not work for Windows, I just changed the docs to show how to run a multiline command in Windows.
Here is the way:
docker run --rm -ti --name zalenium -p 4444:4444 ^
-v /var/run/docker.sock:/var/run/docker.sock ^
-v /c/Users/your_user_name/temp/videos:/home/seluser/videos ^
--privileged dosel/zalenium start