In XCode 11.7, when I add a Copy Files build phase, it only allows me to select files, not directories, and while I can select files from within sub-directories, they all get mashed together into the top-level.
I can work around it by adding multiple Copy Files phases with a different subpath so that the structure ends up correct, but this is tedious and feels wrong.
Is there a better way to make the output directory structure of Copy Files match the input?
I figured out how to do it - instead of a Copy Files phase, I added a Run Script phase and used the cp -R command.
In my case, I wanted to copy a TargetApp/www folder to the "Wrapper" (root of the app), so my full script is:
echo "copying $SRCROOT/TargetApp/www to $TARGET_BUILD_DIR/$CONTENTS_FOLDER_PATH"
cp -R "$SRCROOT/TargetApp/www" "$TARGET_BUILD_DIR/$CONTENTS_FOLDER_PATH"
Related
In my Dockerfile, I have the following:
COPY . /var/task
...which copies my app code into the image.
I need to exclude the vendor/ directory when performing this copy.
I cannot add vendor/ to .dockerignore, because that directory needs to be part of the image when it gets built within the image with a RUN composer install.
I cannot specify every file and directory that should be copied, because they may change and I can't rely on other developers to keep the list updated.
I've tried the following, with the following errors:
COPY [^vendor$]* /var/task
When using COPY with more than one source file, the destination must be a directory and end with a /
COPY [^vendor$]*/ /var/task
COPY failed: no source files were specified
It is actually enough to add the vendor directory to the .dockerignore file.
You can broadly follow the flow of files through docker build in three phases:
docker build reads files from the directory you name, ignoring things in the .dockerignore file, and sends them to the Docker daemon as the build context.
The COPY instruction copies files from the build context into the container filesystem.
RUN instructions do further transformation or processing.
If you put vendor in the .dockerignore file, it prevents the directory from being included in the build context. The build will go somewhat faster, and COPY won't have the files to copy into the image. It won't prevent a RUN composer install step later on from creating its own vendor directory in the image.
I don't think there is an easy solution to this problem.
If you need vendor for RUN composer install and you're not using a multistage build then it doesn't matter if you remove the vendor folder in the copy command. If you've copied it into the build earlier then it's going to be present in your final image, even if you don't copy it over in your COPY step.
One way to get around this is with multi-stage builds, like so:
FROM debian as base
COPY . /var/task/
RUN rm -rf /var/task/vendor
FROM debian
COPY --from=base /var/task /var/task
If you can use this pattern in your larger build file then the final image will contain all the files in your working directory except vendor.
There's still a performance hit though. You're still going to have to copy the entire vendor directory into the build, and depending on what docker features you're using that will still take a long time. But if you need it for composer install then there's really no way around this.
Dockerfile copy command allows golang regex. But with the regex, I am not able to omit a particular folder.
For example, if the directory has:-
public
dist
webapp
somefile.txt
anotherfile.txt
Now, how should I write the expression for COPY such that it omits 'webapp' and copy all other files and folders?
NOTE: I know I can put it to .dockerignore, but in later build stage in the same Dockerfile, I want to copy that folder - 'webapp'
You have two choices:
List all directories you want to copy directly:
COPY ["foldera", "folderc", "folderd", ..., "/dstPath]
Try to exclude some paths but also make sure that all paths patterns are not including the path we want to exclude:
COPY ["folder[^b]*", "file*", "/dstPath"]
Also you can read more about available solutions in this issue: https://github.com/moby/moby/issues/15771
COPY with exclusions work around
I have a PHP + Node app, with both node_modules and vendor directories, with layer caching in place.
I was looking to exclude my dependencies, by excluding some files from being copied, but since Docker COPY does not support exclusions, I took a different approach, to get my dependencies cached in a different layer.
It took a combination of 3 different steps:
Step 1
Script the tarring the node_modules and vendor directories in my build process:
tar -cf ./node_modules.tgz --directory=./src/node_modules .
tar -cf ./vendor.tgz --directory=./src/vendor .
docker build ...
rm node_modules.tgz vendor.tgz
docker push ...
Step 2
Use .dockerignore to ignore the node_modules and vendor directories:
src/node_modules
src/vendor
Step 3
Add the tar files to the project, before copying the rest of my source code:
ADD node_modules.tgz /var/www/node_modules
ADD vendor.tgz /var/www/vendor
COPY ./src /var/www
Obviously, the first build is slow while the layer gets cached, and whenever the cache is invalidated (e.g. new packages).
Credit to jason-kane from here for inspiration: https://github.com/moby/moby/issues/15771#issuecomment-207113714
Something else to note: my vendor and node_modules directories are in the same folder as the source code.
IN TFS 2017 on-premises site, I've put together a TFS build that generates several database builds and SSIS package builds. it produces the desired dacpac and ispac files. However, when created, these files are placed in a hierarchy, based on the particular project structure. It looks something like this:
Database1
\bin
\Release
\database1.dacpac
Database2
\bin
\Release
\database2.dacpac
ssisPackage
\bin
\Development
\ssispackage.ispac
I would like to copy all of these files (*.dacpac and *.ispac) to a single
directory (flattened) when pushing them out to my team. However, the Copy Files task is copying them and preserving the folder structure.
The Contents block of the "Copy Files" task is:
**\bin\$(BuildConfiguration)\**\*.dacpac
**\bin\Development\*.ispac
and the Target Folder is
$(build.artifactstagingdirectory)
Is there a way to move this files to the target folder without the folder hierarchy, resulting in:
OutputFolder
\database1.dacpac
\database2.dacpac
\ssispackage.ispac
Thanks for the advice
Refer to these steps to achieve your requirement:
Create a bat file with code below and add to your project, then check into source control (%1 means first argument, %2 means second argument)
Code:
pushd %1
for /r %%a in (*.pdb) do (
copy "%%a" %2
)
popd
Add Command Line build step to your build definition
Note: This code is used to copy pdb files, you need to modify it per to requirements.
I need to replace the Copy Files build phase with the manual copy files via Run Script build phase.
Copy Files stage looks like this now:
The replacement script looks this way:
but the destination dir is not correct. What is the correct absolute path for the Products Directory? Thanks
The Products Directory is a ${BUILT_PRODUCTS_DIR} in xcodebuild
I want to copy a complete directory (with all subdirectories) from a workspace in a job a into another job b's workspace.
I try with artifact but I don't find a way to copy all subdirectories and there is no option to preserve directory structure.
For artifact archiving, use **/* to copy all workspace files and subdirectories
For the Copy Artifacts step in other job, you can leave it blank to copy all artifacts, or you can use **/* syntax again
If the directory is very large or has a lot of files, it might be better to archive the source workspace directory using something like zip and archive the resulting zip file. Jenkins is notoriously slow at archiving artifacts, so even though you should be able to do it all with individual files, I myself have found moving a single (often much smaller) zip file has much better performance.