Use-case: copy a file containing some creds from local machine directory to existing and already created Docker container/volume
Per the documentation on using docker cp, I constructed my command line statement like this:
docker cp mynodered:/Users/<myUserName>/Documents/nodered-volume/creds.json /data/creds.json
However, I consistently get an error returned:
invalid output path: directory "/data" does not exist
Eventually, I found that changing the syntax of the docker cp statement to:
docker cp /Users/<myUserName>/Documents/nodered-volume/creds.json mynodered:/data/creds.json resolved the issue
troubleshooting tl;dr
I didnt see this documented anywhere, but the syntax that worked for me was docker cp <current local filepath> containerName:/<intended container filepath>
Make sure there is not a space between containerName: and /<intended container filepath>
However, I consistently get an error returned: invalid output path:
directory "/data" does not exist
Above error message you're getting since such directory doesn't exists
# ensure /data exists if not create directory
mkdir -pv /data
# now copy whatever from container to host directory
docker cp <container-id-or-name>:/absolute/path/of/your/file /data
When I run this command on my gitlab pipeline
docker build --build-arg NPM_TOKEN=${NPM_TOKEN} --tag $REGISTRY_IMAGE/web-public:$CI_COMMIT_SHA --tag $REGISTRY_IMAGE/web-public:$CI_COMMIT_REF_NAME packages/web-public
it fails with
build requires exactly 1 argument
It looks to me like I am actually passing one argument, the path; packages/web-public. Flags are not arguments as far as I know.
What am I missing here?
This is the structure of my project
Quote your variables. Something in those variables is expanding to be more than the single arg to the flag.
docker build --build-arg "NPM_TOKEN=${NPM_TOKEN}" --tag "$REGISTRY_IMAGE/web-public:$CI_COMMIT_SHA" --tag "$REGISTRY_IMAGE/web-public:$CI_COMMIT_REF_NAME" packages/web-public
You can also echo that command to see how the variables are expanding, e.g.
echo docker build ...
from https://docs.docker.com/engine/reference/commandline/build/
docker build [OPTIONS] PATH | URL | -
It looks like there's something wrong with your PATH. Try using the absolute path or change to the directory containing the Dockerfile and use .
see also: "docker build" requires exactly 1 argument(s)
My issue was that I had a multi line script entry, eg
script:
- >
docker build \
--network host \
-t ${CI_REGISTRY}/kylehqcom/project/image:latest \
....
As soon as I added to a single line, we were all ok. So I guess the line breaks got "entered" after the first line which meant that the subsequent lines were ignored and the error was returned. Also note, that I CI linted via the GitLab ui and all was syntactically correct.
I'm trying to use the tf-sentencepiece operation in my model found here https://github.com/google/sentencepiece/tree/master/tensorflow
There is no issue building the model and getting a saved_model.pb file with variables and assets. However, if I try to use the docker image for tensorflow/serving, it says
Loading servable: {name: model version: 1} failed:
Not found: Op type not registered 'SentencepieceEncodeSparse' in binary running on 0ccbcd3998d1.
Make sure the Op and Kernel are registered in the binary running in this process.
Note that if you are loading a saved graph which used ops from tf.contrib, accessing
(e.g.) `tf.contrib.resampler` should be done before importing the graph,
as contrib ops are lazily registered when the module is first accessed.
I am unfamiliar with how to build anything manually, and was hoping that I could do this without many changes.
One approach would be to:
Pull a docker development image
$ docker pull tensorflow/serving:latest-devel
In the container, make your code changes
$ docker run -it tensorflow/serving:latest-devel
Modify the code to add the op dependency here.
In the container, build TensorFlow Serving
container:$ tensorflow_serving/model_servers:tensorflow_model_server && cp bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server /usr/local/bin/
Use the exit command to exit the container
Look up the container ID:
$ docker ps
Use that container ID to commit the development image:
$ docker commit $USER/tf-serving-devel-custom-op
Now build a serving container using the development container as the source
$ mkdir /tmp/tfserving
$ cd /tmp/tfserving
$ git clone https://github.com/tensorflow/serving .
$ docker build -t $USER/tensorflow-serving --build-arg TF_SERVING_BUILD_IMAGE=$USER/tf-serving-devel-custom-op -f tensorflow_serving/tools/docker/Dockerfile .
You can now use $USER/tensorflow-serving to serve your image following the Docker instructions
Ran into this Docker error with one of my projects:
invalid reference format: repository name must be lowercase
What are the various causes for this generic message?
I already figured it out after some effort, so I'm going to answer my own question in order to document it here as the solution doesn't come up right away when doing a web search and also because this error message doesn't describe the direct problem Docker encounters.
A "reference" in docker is a pointer to an image. It may be an image name, an image ID, include a registry server in the name, use a sha256 tag to pin the image, and anything else that can be used to point to the image you want to run.
The invalid reference format error message means docker cannot convert the string you've provided to an image. This may be an invalid name, or it may be from a parsing error earlier in the docker run command line if that's how you run the image.
If the name itself is invalid, the repository name must be lowercase means you use upper case characters in your registry or repository name, e.g. YourImageName:latest should be yourimagename:latest.
With the docker run command line, this is often the result in not quoting parameters with spaces, missing the value for an argument, and mistaking the order of the command line. The command line is ordered as:
docker ${args_to_docker} run ${args_to_run} image_ref ${cmd_to_exec}
The most common error in passing args to the run is a volume mapping expanding a path name that includes a space in it, and not quoting the path or escaping the space. E.g.
docker run -v $(pwd):/data image_ref
Where if you're in the directory /home/user/Some Project Dir, that would define an anonymous volume /home/user/Some in your container, and try to run Project:latest with the command Dir:/data image_ref. And the fix is to quote the argument:
docker run -v "$(pwd):/data" image_ref
Other common places to miss quoting include environment variables:
docker run -e SOME_VAR=Value With Spaces image_ref
which docker would interpret as trying to run the image With:latest and the command Spaces image_ref. Again, the fix is to quote the environment parameter:
docker run -e "SOME_VAR=Value With Spaces" image_ref
With a compose file, if you expand a variable in the image name, that variable may not be expanding correctly. So if you have:
version: 2
services:
app:
image: ${your_image_name}
Then double check that your_image_name is defined to an all lower case string.
In my case was the -e before the parameters for mysql docker
docker run --name mysql-standalone -e MYSQL_ROOT_PASSWORD=hello -e MYSQL_DATABASE=hello -e MYSQL_USER=hello -e MYSQL_PASSWORD=hello -d mysql:5.6
Check also if there are missing whitespaces
Let me emphasise that Docker doesn't even allow mixed characters.
Good:
docker build -t myfirstechoimage:0.1 .
Bad:
docker build -t myFirstEchoImage:0.1 .
had a space in the current working directory and usign $(pwd) to map volumes. Doesn't like spaces in directory names.
In my case, the image name defined in docker-compose.yml contained uppercase letters. The fact that the error message mentioned repository instead of image did not help describe the problem and it took a while to figure out.
In my case the problem was in parameters arrangement. Initially I had --name parameter after environment parameters and then volume and attach_dbs parameters, and image at the end of command like below.
docker run -p 1433:1433 -e sa_password=myComplexPwd -e ACCEPT_EULA=Y --name sql1 -v c:/temp/:c:/temp/ attach_dbs="[{'dbName':'TestDb','dbFiles':['c:\\temp\\TestDb.mdf','c:\\temp\\TestDb_log.ldf']}]" -d microsoft/mssql-server-windows-express
After rearranging the parameters like below everything worked fine (basically putting --name parameter followed by image name).
docker run -d -p 1433:1433 -e sa_password=myComplexPwd -e ACCEPT_EULA=Y --name sql1 microsoft/mssql-server-windows-express -v C:/temp/:C:/temp/ attach_dbs="[{'dbName':'TestDb','dbFiles':['C:\\temp\\TestDb.mdf','C:\\temp\\TestDb_log.ldf']}]"
On MacOS when your are working on an iCloud drive, your $PWD will contain a directory "Mobile Documents". It does not seem to like the space!
As a workaround, I copied my project to local drive where there is no space in the path to my project folder.
I do not see a way you can get around changnig the default path to iCloud which is ~/Library/Mobile Documents/com~apple~CloudDocs
The space in the path in "Mobile Documents" seems to be what docker run does not like.
If you encounter this problem in go-swagger (Windows).
#echo off
echo.
docker run --rm -it --env GOPATH=/go -v %CD%:/go/src -w /go/src quay.io/goswagger/swagger %*
Use this instead: (add quote)
#echo off
echo.
docker run --rm -it --env GOPATH=/go -v "%CD%:/go/src" -w /go/src quay.io/goswagger/swagger %*
A reference in Docker is what points to an image. This could be in a remote registry or the local registry. Let me describe the error message first and then show the solutions for this.
invalid reference format
This means that the reference we have used is not a valid format. This means, the reference (pointer) we have used to identify an image is invalid. Generally, this is followed by a description as follows. This will make the error much clearer.
invalid reference format: repository name must be lowercase
This means the reference we are using should not have uppercase letters. Try running docker run Ubuntu (wrong) vs docker run ubuntu (correct). Docker does not allow any uppercase characters as an image reference. Simple troubleshooting steps.
1) Dockerfile contains a capital letters as images.
FROM Ubuntu (wrong)
FROM ubuntu (correct)
2) Image name defined in the docker-compose.yml had uppercase letters
3) If you are using Jenkins or GoCD for deploying your docker container, please check the run command, whether the image name includes a capital letter.
Please read this document written specifically for this error.
sometimes you miss -e flag while specific multiple env vars inline
e.g.
bad: docker run --name somecontainername -e ENV_VAR1=somevalue1 ENV_VAR2=somevalue2 -d -v "mypath:containerpath" <imagename e.g. postgres>
good: docker run --name somecontainername -e ENV_VAR1=somevalue1 -e ENV_VAR2=somevalue2 -d -v "mypath:containerpath" <imagename e.g. postgres>
In my case I had a naked --env switch, i.e. one without an actual variable name or value, e.g.:
docker run \
--env \ <----- This was the offending item
--rm \
--volume "/home/shared:/shared" "$(docker build . -q)"
Replacing image: ${DOCKER_REGISTRY}notificationsapi
with image:notificationsapi
or image: ${docker_registry}notificationsapi
in docker-compose.yml did solves the issue
file with error
version: '3.4'
services:
notifications.api:
image: ${DOCKER_REGISTRY}notificationsapi
build:
context: .
dockerfile: ../Notifications.Api/Dockerfile
file without error
version: '3.4'
services:
notifications.api:
image: ${docker_registry}notificationsapi
build:
context: .
dockerfile: ../Notifications.Api/Dockerfile
So i think error was due to non lower case letters it had
For me the issue was with the space in volume mapping that was not escaped. The jenkins job which was running the docker run command had a space in it and as a result docker engine was not able to understand the docker run command.
Indeed, the docker registry as of today (sha 2e2f252f3c88679f1207d87d57c07af6819a1a17e22573bcef32804122d2f305) does not handle paths containing upper-case characters. This is obviously a poor design choice, probably due to wanting to maintain compatible with certain operating systems that do not distinguish case at the file level (ie, windows).
If one authenticates for a scope and tries to fetch a non-existing repository with all lowercase, the output is
(auth step not shown)
curl -s -H "Authorization: Bearer $TOKEN" -X GET https://$LOCALREGISTRY/v2/test/someproject/tags/list
{"errors":[{"code":"UNAUTHORIZED","message":"authentication required","detail":[{"Type":"repository","Class":"","Name":"test/someproject","Action":"pull"}]}]}
However, if one tries to do this with an uppercase component, only 404 is returned:
(authorization step done but not shown here)
$ curl -s -H "Authorization: Bearer $TOKEN" -X GET https://docker.uibk.ac.at:443/v2/test/Someproject/tags/list
404 page not found
I solve this changing some uppercase words on my Dockerfile like:
FROM Base as Build
RUN npm run Build:prod
to
FROM base as build
RUN npm run build:prod
Another place:
FROM Base as Release
COPY --from=Build /usr/path/here/dist/ ./dist
to
FROM base as Release
COPY --from=build /usr/path/here/dist/ ./dist
I've encountered the same issue while using docker with mlflow.
In my case, the directory name containing my Dockerfile was "My Project" which I changed to myproject or my_project and It worked for me.
Also, follow the same naming format for all the root/super directories under which, the Dockerfile resides.
Not only for docker, but it's also good practice (especially in Unix based OS) to avoid the following while defining a directory name:-
white spaces
camel-case
upper-case
I had the same error, and for some reason it appears to have been cause by uppercase letters in the Jenkins job that ran the docker run command.
This is happening because of the spaces in the current working directory that came from $(pwd) for map volumes. So, I used docker-compose instead.
The docker-compose.yml file.
version: '3'
services:
react-app:
build:
context: .
dockerfile: Dockerfile.dev
ports:
- "3000:3000"
volumes:
- /app/node_modules
- .:/app
"docker build -f Dockerfile -t SpringBoot-Docker ."
As in the above commend, we are creating an image file for docker container. commend says create image use file(-f refer to docker file) and -t for the target of the image file we are going to push to docker. the "." represents the current directory
solution for the above problem: provide target image name in lowercase
Docker can build images automatically by reading the instructions from a Dockerfile. A Dockerfile is a text document that contains all the commands a user could call on the command line to assemble an image.
example:
FROM python:3.7-alpine
The 'python' should be in lowercase
In my case I was trying to run postgres through docker. Initially I was running as :
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=test_password POSTGRES_USER=test_user POSTGRES_DB=test_db --rm -v ~/docker/volumes/postgres:/var/lib/postgresql/data --name pg-docker postgres
I was missing -e after each environment variable. Changing the above command to the one below worked
docker run -d -p 5432:5432 -e POSTGRES_PASSWORD=test_password -e POSTGRES_USER=test_user -e POSTGRES_DB=test_db --rm -v ~/docker/volumes/postgres:/var/lib/postgresql/data --name pg-docker postgres
I wish the error message would output the problem string. I was getting this due to a weird copy and paste problem of a "docker run" command. A space-like character was being used before the repo and image name.
Most of the answers above did not work for my case, so I will document this in case somebody finds it helpful. The first line in the dockerfile FROM node:10 for my case, the word node should not be uppercase i.e FROM NODE:10. I made that change and it worked.
In my case DockerFile contained the image name in mixed case instead of lower case.
Earlier line in my DockerFile
FROM CentOs
and when I changed above to FROM centos, it worked smoothly.
You need to enter the Name of the Docker-Image and not your File Name :P
$ docker run {your image}
Another possible cause of this error is that in your Dockerfile you have mixed capitalization in the syntax declaration itself. For example:
# syntax=docker/Dockerfile:1
instead of
# syntax=docker/dockerfile:1
If you come here after encountering this error in your GitHub Actions worflows…
Make sure to use docker/metadata-action action to handle repository naming for you. Just call it before docker/build-push-action:
# Add this
- id: docker-metadata
uses: docker/metadata-action#v4
with:
images: ghcr.io/${{ github.repository }}
# Use the extracted metadata
- uses: docker/build-push-action#v3
with:
tags: ${{ steps.docker-metadata.outputs.tags }}
labels: ${{ steps.docker-metadata.outputs.labels }}
… other properties …
I am trying to save a docker image in windows so that I can load to another Linux box,in between while saving the images in windows, I got an error stating access is denied to rename the docker temp file.
I checked the permission everything looks fine, in fact I can able to edit. Any help here is highly appreciable. I am using docker 1.11.0
docker save -o . <imgID>
rename .docker_temp_742575903 .: Access is denied.
Never mind, along with path I need to give my new file name that docker wanted to create and it don't happen implicitly, in my cases I gave
docker save -o ./<tar name that you wanted docker to create> <imgID>
For the similar issue but on unix:
root#linux:/opt/docker# docker save -o ./presto.tar starburstdata/presto
open .docker_temp_359214587: permission denied
You can use different syntax to save the image as a workaround:
root#linux:/opt/docker# docker save starburstdata/presto > presto.tar
root#linux:/opt/docker# ls -l
razem 1356196
-rw-r--r-- 1 root root 1388737024 maj 23 11:16 presto.tar
I workaround the problem on linux by changing the current forlder permission to 777. Make sure your current direct :
mkdir ~/docker-images
cd ~/docker-images
chmod 777 ./
sudo docker save <img_id> -o ./<filename>
You are not giving image name and it's creating some temp name by default but, while renaming that it's throwing error. you can use this command to solve this problem.
docker save -o <some custom name with path> <imgID or REPOSITORY:TAG>
something like this if you want to create in present directory
docker save -o ./ubuntu_image.tar ubuntu:latest
or
docker save -o ./ubuntu_image.tar eat546t
If you want to create in specific location
docker save -o path/of/image/ubuntu_image.tar ubuntu:latest