I am trying to zip files and directories in Groovy using AntBuilder. I have the following code:
def ant = new AntBuilder()
ant.zip(basedir: "./Testing", destfile:"${file}.zip",includes:file.name)
This zips the file "blah.txt", but not the file "New Text Document.txt". I think the issue is the spaces. I've tried the following:
ant.zip(basedir: "./Testing", destfile:"${file}.zip",includes:"${file.name}")
ant.zip(basedir: "./Testing", destfile:"${file}.zip",includes:"\"${file.name}\"")
Neither of the above resolved the issue. I'm using Ant because it will zip directories, and I don't have access to org.apache.commons.io.compression at work.
If you look at the docs for the ant zip task, the includes parameter is described as:
comma- or space-separated list of patterns of files that must be included
So you're right, that it is the space separator that's breaking it...
You need to use the longer route to get this to work:
new AntBuilder().zip( destFile: "${file}.zip" ) {
fileset( dir: './Testing' ) {
include( name:file.name )
}
}
Related
Imagine I have a java_binary target triggered by a custom rule that generates source code and places the generated sources under a directory, let's call it "root".
So after the code generation we will have something like this:
// bazel-bin/...../src/com/example/root
root:
-> Foo.java
-> Bar.java
-> utils
-> Baz.java
Now, I have another target, a java_library, that depends on the previously generated sources, so it depends on the custom rule.
My custom rule definition currently looks something like this:
def _code_generator(ctx):
outputDir = ctx.actions.declare_directory("root")
files = [
ctx.actions.declare_file("root/Foo.java"),
ctx.actions.declare_file("root/Bar.java"),
ctx.actions.declare_file("root/utils/Baz.java"),
// and many,
// many other files
]
outputs = []
outputs.append(outputDir)
outputs.extend(files)
ctx.actions.run(
executable = // executable pointing to the java_binary
outputs = outputs
// ....
)
This works. But as you can see, every anticipated file that is to be generated, is hard-coded in the rule definition. This makes it very fragile, should the code generation produce a different set of files in the future (which it will).
(Without specifying each of the files, as shown above, Bazel will fail the build saying that the files have no generating action)
So I was wondering, is there a way to read the content of the root directory and automatically, somehow, declare each of the files as an output?
What I tried:
The documentation of declare_directory says:
The contents of the directory are not directly accessible from Starlark, but can be expanded in an action command with Args.add_all().
And add_all says:
[...] Each directory File item is replaced by all Files recursively contained in that directory.
This sounds like there could be a way to get access to the individual files in the directory, but I am not sure how.
I tried:
outputDir = ctx.actions.declare_directory("root")
//...
args = ctx.actions.args()
args.add_all(outputDir)
with the intention to access the individual files later from args, but the build fails with: "Error in add_all: expected value of type sequence or depset for values, got File".
Any other ideas on how to implement the rule, so that I don't have to hard-code each and every file that will be generated?
How do I replace the new File method with a secure one? Is it possible to create a python script and connect it?
Part of the code where I have a problem:
def template Name = new File(file: "${template}").normalize.name.replace(".html", "").replace(".yaml", "")
But when I run my pipeline, I get the error
java.lang.SecurityException: Unable to find constructor: new java.io .File java.util.LinkedHashMap
This method is prohibited and is blacklisted. How do I replace it and with what?
If you're reading the contents of the file, you can replace that "new File" with "readFile".
See https://www.jenkins.io/doc/pipeline/steps/workflow-basic-steps/#readfile-read-file-from-workspace
readFile: Read file from workspace
Reads a file from a relative path (with root in current directory, usually > workspace) and returns its content as a plain string.
file : String
Relative (/-separated) path to file within a workspace to read.
encoding : String (optional)
The encoding to use when reading the file. If left blank, the platform default encoding will be used. Binary files can be read into a Base64-encoded string by specifying "Base64" as the encoding.
I'm trying to require files in Lua, in one case it is working, but when I want to simplify the requirements in updating the LUA PATH the file is not found, but it is in the trace!
To reproduce my require problem I did the test with the package.searchpath function, which takes the required key and the Lua path in arguments.
So the code :
print('MY LUA PATH')
local myLuaPath = "?;?.lua;D:\\Projets\\wow-addon\\HeyThere\\?;D:\\Projets\\wow-addon\\HeyThere\\src\\HeyThere\\?;D:\\Projets\\wow-addon\\HeyThere\\test\\HeyThere\\?"
print(myLuaPath)
print('package search core.add-test')
print(package.searchpath('core.add-test', myLuaPath))
print('package search test.HeyThere.core.add-test')
print(package.searchpath('test.HeyThere.core.add-test', myLuaPath))
The result :
MY LUA PATH
?;?.lua;D:\Projets\wow-addon\HeyThere\?;D:\Projets\wow-addon\HeyThere\src\HeyThere\?;D:\Projets\wow-addon\HeyThere\test\HeyThere\?
package search core.add-test
nil no file 'core\add-test'
no file 'core\add-test.lua'
no file 'D:\Projets\wow-addon\HeyThere\core\add-test'
no file 'D:\Projets\wow-addon\HeyThere\src\HeyThere\core\add-test'
no file 'D:\Projets\wow-addon\HeyThere\test\HeyThere\core\add-test'
package search test.HeyThere.core.add-test
test\HeyThere\core\add-test.lua
So the first try with 'core.add-test' should work with the 'D:\Projets\wow-addon\HeyThere\test\HeyThere\?' value in the path but fails...
In the trace, there is the file I want!
no file 'D:\Projets\wow-addon\HeyThere\test\HeyThere\core\add-test'
But with the same LUA PATH but starting in a parent folder the file is found... Second test with 'test.HeyThere.core.add-test' found from the 'D:\Projets\wow-addon\HeyThere\?'
-> test\HeyThere\core\add-test.lua
Can someone explains to me why it doesn't work the first time?
EDIT :
My current directory is D:\Projets\wow-addon\HeyThere
My lua.exe is in D:\Projets\wow-addon\HeyThere\bin\lua but is added to my PATH variable (I'm on Windows)
I set the LUA_PATH environment variable and execute
lua "test\test-suite.lua" -v
The code inside test-suite.lua is the test code described above
As #EgorSkriptunoff suggested, adding file extansion in the path resolve the problem...
Ex:
Wrong path D:\Projets\wow-addon\HeyThere\?
Good path D:\Projets\wow-addon\HeyThere\?.lua
The extension should be in the path variable because in the require the dot is replace and used as a folder separator.
I've got a Java project I'm converting to Bazel.
As is typical with Java projects, there are property files with placeholders that need to be resolved/substituted at build time.
Some of the values can be hardcoded in a BUILD or BZL file:
BUILD_PROPERTIES = { "pom.version": "1.0.0", "pom.group.id": "com.mygroup"}
Some of the variables are "stamps" (e.g. BUILD_TIMESTAMP, GIT_REVISION, etc): The source for these variables are volatile-status.txt and stable-status.txt
I must generate a POM for publish, so I use #bazel_common//tools/maven:pom_file in BUILD
(assume that I need ALL the values described above for my pom template):
_local_build_properties = {}
_local_build_properties.update(BUILD_PROPERTIES)
# somehow add workspace status properties?
# add / override
_local_build_properties.update({
"pom.project.name": "my-submodule",
"pom.project.description": "My submodule description",
"pom.artifact.id": "my-submodule",
})
# Variable placeholders in the pom template are wrapped with {}
_pom_substitutions = { '{'+k+'}':v for (k,v) in _local_build_properties.items()}
pom_file(
name = "my_submodule_pom",
targets = [
"//my-submodule",
],
template_file = "//:pom_template.xml",
substitutions = _pom_substitutions,
)
So, my questions are:
How do I get key-value pairs from volatile/stable -status.txt into the
dictionary I need for pom_file.substitutions?
pom_file depends on java_library so that it can write its dependencies
into the POM. How do I update the jar generated by java_library with the
pom?
Once I have the pom and the updated jar containing the pom, how do I publish to a Maven repo?
When I look at existing code, for example rules_docker, it seems that the implementation always bails to a local executable (shell | python | go) to do the real work of substitution, jar manipulation and image publication. Am I trying to do too much in BUILD and BZL files? Should I be thinking, "Ultimately, what do I need to pass to local shell/python/go scripts to get real build work done?
(Answered on bazel-discuss group)
Hi,
You can't get these values from Starlark. You need a genrule to read the stable/volatile files and do the substitutions using an
external tool like 'sed'.
A file cannot be both an input and output of an action, i.e. you can't update the .jar from which you generate the pom. The action has
to produce a new .jar file.
I don't know -- how would you publish outside of Bazel, is there a tool to do so? Can you write a genrule / Starlark rule to wrap this
tool?
Cheers, László
I want to edit project.pbxproj straight up using command line (for CI server script)
what tools can allow me to do this?
I used to use PlistBuddy to edit the output Info.plist; however, what i really want to do is to edit this user defined field, which is used in multiple places, and i really don't want to have to hunt that down in every plist location
project.pbxproj is an old-style ASCII property list file, too. So you can use /usr/libexec/PlistBuddy to edit it.
Print some User-Defined key's value like this,
# Get the key A83311AA20DA4A80004B8C0E in your project.pbxproj
# LZD_NOTIFICATION_SERVICE_BUNDLE_ID is defined by me,
# Replace key paths with your own.
/usr/libexec/PlistBuddy -c 'print :objects:A83311AA20DA4A80004B8C0E:buildSettings:LZD_NOTIFICATION_SERVICE_BUNDLE_ID' LAAppAdapter.xcodeproj/project.pbxproj
Set its value like this,
/usr/libexec/PlistBuddy -c 'set :objects:A83311AA20DA4A80004B8C0E:buildSettings:LZD_NOTIFICATION_SERVICE_BUNDLE_ID com.dawnsong.notification-service' LAAppAdapter.xcodeproj/project.pbxproj
UPDATE
PlistBuddy will automatically convert project.pbxproj into a xml-format plist file since macOS Catalina (or some earlier version). It's better to move the setting item into xcconfig file instead since xcconfig is much smaller and simpler than project.pbxproj and not easy to make mistakes when editing with perl script.
I know this has been answered for a while, but since the original question is about tools supporting the manipulation of .pbxproj files, and many other people may be looking for the same information, here's how I do it. It took me quite a while to figure this out because I was very unfamiliar with Xcode when I started attempting this, so I hope this saves others the hours of grief I had to put in.
You can use the plutil command to transform the .pbxproj file from the legacy .plist format into an XML or JSON format you will be able to manipulate more easily. I'm using JSON. To do so, just run:
plutil -convert json project.pbxproj
This will convert the format of project.pbxproj, but be aware that -contrary to common sense- the output won't be another file with a JSON extention such as project.json. What will happen is that project.pbxproj will be converted to JSON format, but retain it's cryptic .pbxproj extension. So even though the file's format has been changed, Xcode will still pick it up and use it in its new JSON format.
Then you can change project.pbxproj with ease using any JSON manipulation tool of your choosing. I'm using Groovy's JsonSlurper class in a Groovy script.
Note I also explored the XML option, but I found the project.pbxproj file in XML format to be cumbersome to parse. The elements are not properly nested to allow for traversing the tree with ease. It's plagued with:
<key>someKey</key>
<dict>
<!--More elements which provide configuration for the key above-->
</dict>
So it's positional in nature. You have to look for the key element corresponding to the setting you want to manipulate and then jump to the dict element just after it. Which means you have to mount the children of each XML element into an array, in order to index them.
Here are 3 open-source tools which implement .pbxproj file editing:
https://github.com/CocoaPods/Xcodeproj (Ruby based)
https://github.com/apache/cordova-node-xcode (NodeJS based)
https://github.com/kronenthaler/mod-pbxproj (Python based)
Personally, I made the best experience with the NodeJS based tool. So far it has covered all our needs reliably.
In the following is listed an example javascript file update-project.js which sets the developer team ID, app entitlements, adds a GoogleService-Info.plist file to the project and checks it as part of the build target. Take it as an inspiration and adapt the scripts and its paths to your needs:
const fs = require('fs')
const xcode = require('xcode')
if (process.argv.length !== 3) {
console.error("Please pass the development team ID as the first argument")
process.exit(1)
}
const developmentTeamId = process.argv[2]
const path = 'ios/App/App.xcodeproj/project.pbxproj'
const project = xcode.project(path)
project.parse(error => {
const targetKey = project.findTargetKey('App')
const appGroupKey = project.findPBXGroupKey({path: 'App'})
project.addBuildProperty('CODE_SIGN_ENTITLEMENTS', 'App/App.entitlements')
project.addBuildProperty('DEVELOPMENT_TEAM', developmentTeamId)
project.addFile('App.entitlements', appGroupKey)
project.removeFile('GoogleService-Info.plist', appGroupKey)
const f = project.addFile('GoogleService-Info.plist', appGroupKey, {target: targetKey})
f.uuid = project.generateUuid()
project.addToPbxBuildFileSection(f)
project.addToPbxResourcesBuildPhase(f)
fs.writeFileSync(path, project.writeSync())
})
Above script can be executed with
yarn run update-project <arguments...>
given that update-project is registered in package.json:
{
...,
"scripts": {
...
"update-project": "node update-project.js"
},
...
}