I am prototyping an azure batch sample using windows container workload in batch task. This container expects an input file, executes an algorithm and writes an output file st01_demo_results.csv inside the container at path C:\app\output (this directory is created in dockerfile build). On batch task success, the output file will be uploaded to azure storage container using SAS URL. I have mostly used this sample to build my prototype, just that instead of using exe, I am using windows container.
I am trying to transfer my output file generated inside windows container to batch task working directory by mounting a volume with source as batch task working directory and target my container output file path.
string taskCommandLine = $"{inputMediaFile} C:\\app\\output\\{outputMediaFile}";
string containerRunOptions = "-v %AZ_BATCH_TASK_WORKING_DIR%:c:\\app\\output:rw";
var task = new CloudTask(taskId, taskCommandLine)
{
UserIdentity = new UserIdentity(autoUserSpec),
ContainerSettings = containerSettings,
};
task.ResourceFiles = new List<ResourceFile> { inputFiles[i] };
With this, I get exception
Unhandled exception. System.UnauthorizedAccessException: Access to the path 'c:\app\output\st01_demo_results.csv' is denied.
If, I don't provide volume binding in containerRunOptions then my container task executes fine but then I don't get the file transferred to my batch task working directory.
I suspect, that azure batch somewhere ignores rw option and forcefully makes the volume binding read-only - because when I tried running the same windows container locally in docker desktop with ro option, I get same unauthorized exception.
You need to ensure that c:\\app\\output is accessible by the container user that is currently executing within the container context, typically ContainerUser, ContainerAdministrator, or some custom user that you may have added as part of your DOCKERFILE.
Related
I am new to docker, and I have created a docker image for sparq spatial reasoning toolbox using pull docker pull dwolter/sparq:latest, (Gethub: https://github.com/dwolter/SparQ).
The sparq catains set of calculus in form of lisp files, which can be used to do spatial reasoning, using the Sparq docker image in the windows docker.
The thing that I have developed my own calculi and I need to add it to the image.
I have tried to do that using the cp command but I could not. Because I don't know the path of the file indside the image, in otherwords, where I should place the file inside the image, also when place it in the main root of a container, and applied the command commit, it generated error: access denied by ther resource.
first question is :
Does the path in the image has the same path in the sparq application folder which I have already downloaded?
Also, How I can add this culculi (lisp file) to the image in docker ?
P.s. I have also downloaded the folder which contains the application (sparq and all its files and folders) and I have placed my calculi inside the appropriate folder ( caculi folder and it works fine).
I run it using Linux command line and it works fine, Now I need to use this application through the docker.
As I have the application on folder.
Can I create an image on my own based on the folder that contains the application ?
The Sparq Dockerfile indicates the working directory is set to /root/sparq. That means, you should be able to run the following copy command in your own Dockerfile to place your lisp file in the same place you have locally, the place where all other Calculi lisp files are located:
FROM dwolter/sparq
COPY ./path/to/my/Calculi/file.lisp ./Calculi
Then run docker build . to build a Docker image containing sparq and your file. Then, it should be ready to run.
NOTE: I am not familiar with lisp. If it needs to be compiled, the compile command will need to be added to the Dockerfile after the COPY.
I have a .NetCore web API and and need to connect with a C++ shared library (libCppAppOutput.so). For this I am using DllImport inside the controller of the Web API as in the below code:
[DllImport("libCppAppOutput.so", CallingConvention = CallingConvention.Cdecl, CharSet = CharSet.Ansi, EntryPoint = "methodNameinCppProject")]
public static extern int MethodNameInWebAPI(string param1, string param2);
When I am debugging it in Visual studio with Windows environment, I am getting an exception like
An attempt was made to load a program with an incorrect format.
(0x8007000B).
This may be because of trying to access a Linux file in Windows environment, if I am using a CppAppOutput.dll in place of libCppAppOutput.so I can successfully call the methods inside the dll. But my actual requirement is to run the entire application in Linux environment and to be proceed with .so file only, so I have created a docker image with target OS as Linux and tried to call the methods inside it from running docker with the below command :
docker run -p 8081:80 name_of_docker_image:tagname
Then I am getting an exception like:
Unable to load shared library 'libCppAppOutput.so' or one of its dependencies. In order to help diagnose loading problems, consider
setting the LD_DEBUG environment variable: libCppAppOutput.so: cannot
open shared object file: No such file or directory.
But I can see the file, libCppAppOutput.so is present inside the docker image by using the following commands,
docker create --name name_of_container name_of_docker_image:tagname
docker start name_of_container
docker exec -ti name_of_container /bin/bash
root#xyzabc:/app# ls
I am using below URL to test the application
http://localhost:8081/launchUrl/MethodNameInWebAPI/param1/param2
Please help me in this regard like how can I call a .so file from the Linux docker.
if this issue to be elaborated more, kindly comment and I will do.
Thanks in advance.
I could able to solve this issue with help from my colleagues and friends.
There was a dependency with libCppAppOutput.so (an XML parser). Even though that dependency package is installed in the container, it was missed to configure while creating the libCppAppOutput.so.
This was identified by executing ldl libCppAppOutput.so command in bash.
Thank you all for spend time on reading and analyzing the question !
Have more N more great coding days!
I hope you are having a great day!
I'm new to docker. I think my problem is related to docker's directory tree
My app writes to a file to /home/user directory and then after some time reads that file again.
I got this error from my app.
[error] a.a.OneForOneStrategy - /home/user/bkjw_eqvfohygvkaxoxc-small.jpg
java.nio.file.NoSuchFileException: /home/user/bkjw_eqvfohygvkaxoxc-small.jpg
My dockerized app is unable to create the file and read. I'm thinking that the Docker considers the directory /home/user/ as a absolute directory of host.
I thought that the container would write to /home/user directory within the container's directory tree.
So the question is :
How can I specify the path to write the file inside the containers directory tree?
Your understanding about the directory tree is correct. Application running inside a docker container would write to /home/user/ in the container's directory tree.
Your issue seems to be with permissions, your java application probably doesn't have the rights to write to /home/user/ within the container. Either you should change the ownership/rights of the directory you're wanting to write in, or a simple solution I did in such case was to create the directory I wanted to write in, within the java code.
like:
// Create volume directories explicitly so that they are created with correct owner
Files.createDirectories(Paths.get(dirPath));
You can set dirPath String to something like /home/user/mydir IF your requirement is not to write in /home/user/ specifically.
I'm creating some Windows Container images that I need but the source file I want to ADD are in a network share \\myserver\myshare\here.
I've tried in any possible way but I always get the message error The system cannot find the path specified.
Is it because I have not yet found the right way to set it or is it that it is just not possible?
From the Docker site:
Multiple resource may be specified but if they are files or directories then they must be relative to the source directory that is being built (the context of the build).
Is that why I can't accomplish what I need?
Full error message: GetFileAttributesEx \\myserver\myshare\here\: The system cannot find the path specified.
Whatever you ADD or COPY must be in the docker build context.
When you do this:
docker build .
That directory param (the . in the example) is the context that is copied and sent to the Docker daemon. Then the docker daemon use those files to COPY or ADD. It won't use any file that is not in that context.
That is the issue that you are experiencing. I'm not sure how you can solve it anything than copying the files from \\myserver to your build directory.
ADD is capable of download files by providing an URL (should investigate if it supports Windows' shares)
I am using ANT task Copy to copy a zip file from one share to another
<copy file="\\server_share\nightly\xyz08022012.zip" todir="Z:\output\Nightly"/>
when this gets executed, I am getting the below exception
Failed to copy \server_share\nightly\xyz08022012.zip to Z:\output\Nightly\xyz08022012.zip due to java.io.FileNotFoundException Z:\output\Nightly\xyz08022012.zip (The system cannot find the path specified.)`
When I change the Z:\output\Nightly to C:\temp, the copy works
Z:\ points to a server share which is mounted on the server with different user credentials and the drive is made persistent. This work around is because of the fact that when the build runs, the build user does not have access to the output share
Hence I mapped the server share as a network drive with different credentials (the user who has read/write permission) and made this drive persistent
This is on a Windows 7 machine where the build is running.
I tried doing a copy manually and that worked
I looked into Ant Copy Task: Failed to copy due to java.io.FileNotFoundException but doesn't help me