Ant build and empty directories - ant

For some reason, when I use scp to move files from my current machine to the server using ant it always ignores empty directories.
<scp todir="ec2-user#xx.x.x.xxx:/var/www/html" keyfile="${basedir}\..\keys\prod_default.pem" trust="true">
<fileset dir="${staging.dir}" />
</scp>
Why is the case?
Full Update verbose log:
D:\BUILD\SCRIPTS>ant -v -f getcom.xml update
Apache Ant(TM) version 1.8.2 compiled on December 20 2010
Buildfile: D:\BUILD\SCRIPTS\getcom.xml
Detected Java version: 1.7 in: C:\Program Files (x86)\Java\jdk1.7.0_02\jre
Detected OS: Windows Server 2008 R2
parsing buildfile D:\BUILD\SCRIPTS\getcom.xml with URI = file:/D:/BUILD/SCRIPTS/getcom.xml
Project base dir set to: D:\BUILD\SCRIPTS
parsing buildfile jar:file:/C:/ant/lib/ant.jar!/org/apache/tools/ant/antlib.xml
with URI = jar:file:/C:/ant/lib/ant.jar!/org/apache/tools/ant/antlib.xml from a zip file
[property] Loading D:\BUILD\SCRIPTS\build.properties
Build sequence for target(s) `update' is [update]
Complete build sequence is [update, upload, cleanup, staging, export, init, ]
update:
[exec] Current OS is Windows Server 2008 R2
[exec] Executing 'C:\\Program Files (x86)\\subversion\\bin\\\svn' with arguments:
[exec] 'update'
[exec] 'D:\BUILD\SCRIPTS\..\_STAGING\GETOM'
[exec] '--username'
[exec] 'uuuu'
[exec] '--password'
[exec] 'uuuu'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
[exec] Updating 'D:\BUILD\_STAGING\GETOM':
[exec] At revision 134.
[scp] Connecting to xx.0.0.100:22
[scp] done.
[scp] Connecting to xx.0.2.100:22
[scp] done.
BUILD SUCCESSFUL
Total time: 8 seconds
Properties:
svn.repoBaseURL="https://uuu#uuuu.uuuu.com"
svn.username="uuuu"
svn.password="uuuu"
svn.bin=C://Program Files (x86)//subversion//bin//
MSBuildPath=C://WINDOWS//Microsoft.NET//Framework//v4.0.30319//MSBuild.exe
lib.path.ant-contrib=C:/ant/lib/ant-contrib-1.0b3.jar
Please note, folder names have been changed.

You'll have to come up with a workaround. The way that Ant implements the scp means that empty directories are always skipped.
The gist of it is that the Scp class does this:
DirectoryScanner scanner = set.getDirectoryScanner(getProject());
Directory root = new Directory(scanner.getBasedir());
String[] files = scanner.getIncludedFiles();
if (files.length != 0) {
...
}
In other words, it only processes the directory if it contains at least one file.
There is a patch for the Scp task code you might consider, but that means a bespoke Ant build.
The two other approaches that come to mind are
make the directories non-empty by populating with placeholder files (which you can delete once deployed)
create the empty directories in a separate step

Related

Malformed Collection Error

Moving on from the first issue I reported here :
Running an OS command using apache-ant to IIS config
I am trying to run the second part of the IIS config.
<exec executable="cmd.exe" dir="C:/Windows/System32/inetsrv"
failonerror="true">
<arg line="/c appcmd set config /section:handlers
/[name=&apos;ExtensionlessUrlHandler-Integrated-
4.0&apos;].verb:GET,POST,PUT,DELETE,OPTIONS,CONNECT"/>
</exec>
I have no double quotes in this line, and I am using apos; to indicate my single quotes. However I'm getting the following error which Im struggling to find an explanation for :
confIIS:
[echo] Configuring IIS
[exec] ERROR ( message:Malformed collection indexer; format is
[#position,name='value',name2='value2',...]. The #position specifier is
optional, and be '#start', '#end', or
#N' where N is a numeric index into the collection. )
quick Update :
I've tried dbl quotes but no luck. I've done some more digging and the issue is that the single quotes are being dropped.
If I run the cmd line in the dos prompt without the single quotes it fails in the same way.
cmd line :
appcmd set config /section:handlers /[name='ExtensionlessUrlHandler-
Integrated-4.0'].verb:GET
Ant script (debug) output
[exec] 'appcmd'
[exec] 'set'
[exec] 'config'
[exec] '/section:handlers'
[exec] '/[name=ExtensionlessUrlHandler-Integrated-
4.0].verb:GET,POST,PUT,DELETE,OPTIONS,CONNECT'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
ute:Java13CommandLauncher: Executing 'cmd.exe' with arguments:

Sonar and TFS 2013 : No ProjectInfo.xml files were found

When I run TFS 2013 with sonar I get the following error: No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
Message: TF270015: 'SonarQube.MSBuild.Runner.exe' returned an unexpected exit code. Expected '0'; actual '1'.
I use :
sonarqube-5.1.zip
sonar-csharp-plugin-4.0.jar
SonarQube.MSBuild.Runner-0.9.zip
sonar-runner-dist-2.4.zip
I've followed instructions found in "SonarQube Installation Guide for Existing TFS Environment.pdf".
Any help ?
Thank you.
Log :
SonarQube Analysis Summary
Analysis failed for SonarQube project "WpfApplication2", version 1.0
Product projects: 0, test projects: 0
Invalid projects: 0, skipped projects: 0, excluded projects: 0
Résumé
Debug | x86
0 erreur(s), 1 avertissement(s)
$/Essai2/WpfApplication2/WpfApplication2.sln - 0 erreur(s), 1 avertissement(s),
Afficher le fichier journal
C:\Builds\1\Essai2\WpfApplication2\src\documents\visual studio 2013\Projects\WpfApplication2\WpfApplication2.sln.metaproj : The specified solution configuration "Debug|x86" is invalid. Please specify a valid solution configuration using the Configuration and Platform properties (e.g. MSBuild.exe Solution.sln /p:Configuration=Debug /p:Platform="Any CPU") or leave those properties blank to use the default solution configuration.
$/Essai2/WpfApplication2/WpfApplication2.sln compilé
Aucun résultat des tests
Aucun résultat de couverture du code
Autres erreurs et avertissements
2 erreur(s), 0 avertissement(s)
01:45:52: No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
Exception Message: TF270015: 'SonarQube.MSBuild.Runner.exe' returned an unexpected exit code. Expected '0'; actual '1'. See the build logs for more details. (type UnexpectedExitCodeException)
Exception Stack Trace: at System.Activities.Statements.Throw.Execute(CodeActivityContext context)
at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager)
at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)
Here is the Diagnostic (I truncated the source check out part) :
Compile, Test and Publish00:00:46
Run optional script before MSBuild00:00:08
InputsEnvironmentVariables:
Enabled: True
Arguments: /key:WpfApplication2 /name:WpfApplication2 /version:1.0
FilePath: C:\sonarqube\bin\SonarQube.MSBuild.Runner.exe
OutputsResult: 0
C:\sonarqube\bin\SonarQube.MSBuild.Runner.exe /key:WpfApplication2 /name:WpfApplication2 /version:1.0
Pre-processing (3 arguments passed)
Using environment variables to determine the download directory...
Using environment variable 'TF_BUILD_BUILDDIRECTORY', value 'C:\Builds\1\Essai2\WpfApplication2'
Creating the analysis bin directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin
SonarQube server url: http://localhost:9000
Downloading SonarQube.MSBuild.Runner.Implementation.zip from http://localhost:9000/static/csharp/SonarQube.MSBuild.Runner.Implementation.zip to C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin\SonarQube.MSBuild.Runner.Implementation.zip
Executing file C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin\SonarQube.MSBuild.PreProcessor.exe
Args: "/key:WpfApplication2" "/name:WpfApplication2" "/version:1.0"
Working directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin
Timeout (ms):300000
Process id: 4168
01:45:11: The path to the sonar-runner.properties file was not supplied on the command line. Attempting to locate the file...
01:45:11: Located the runner properties file: C:\sonar-runner-2.4\conf\sonar-runner.properties
01:45:11: Legacy TeamBuild environment detected
01:45:11: Creating config and output folders...
01:45:11: Removing the existing directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\conf
01:45:11: Creating directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\conf
01:45:11: Removing the existing directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\out
01:45:11: Creating directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\out
Generating the FxCop ruleset...
01:45:14: Saving the config file to C:\Builds\1\Essai2\WpfApplication2\sqtemp\conf\SonarQubeAnalysisConfig.xml
Process returned exit code 0
Run MSBuild00:00:12
InputsProjectsToBuild: String[] Array
MSBuildMultiProc: True
Verbosity: Normal
ToolPlatform: Auto
Targets:
RunCodeAnalysis: AsConfigured
CommandLineArguments: /p:SkipInvalidConfigurations=true /p:Configuration=Debug /p:Platform="Any CPU"
ConfigurationsToBuild: String[] Array
OutputLocation: SingleFolder
Enabled: True
ToolVersion:
CleanBuild: True
OutDir:
RestoreNuGetPackages: True
C:\Program Files\Microsoft Team Foundation Server 12.0\Tools\nuget.exe restore "C:\Builds\1\Essai2\WpfApplication2\src\documents\visual studio 2013\Projects\WpfApplication2\WpfApplication2.sln" -NonInteractive
C:\Program Files (x86)\MSBuild\12.0\bin\amd64\MSBuild.exe /nologo /noconsolelogger "C:\Builds\1\Essai2\WpfApplication2\src\documents\visual studio 2013\Projects\WpfApplication2\WpfApplication2.sln" /nr:False /fl /flp:"logfile=C:\Builds\1\Essai2\WpfApplication2\src\documents\visual studio 2013\Projects\WpfApplication2\WpfApplication2.log;encoding=Unicode;verbosity=normal" /p:SkipInvalidConfigurations=true /p:Configuration=Debug /p:Platform="Any CPU" /m /p:OutDir="C:\Builds\1\Essai2\WpfApplication2\bin\\" /p:Configuration="Debug" /p:Platform="x86" /p:VCBuildOverride="C:\Builds\1\Essai2\WpfApplication2\src\documents\visual studio 2013\Projects\WpfApplication2\WpfApplication2.sln.x86.Debug.vsprops" /dl:WorkflowCentralLogger,"C:\Program Files\Microsoft Team Foundation Server 12.0\Tools\Microsoft.TeamFoundation.Build.Server.Logger.dll";"Verbosity=Normal;BuildUri=vstfs:///Build/Build/52;IgnoreDuplicateProjects=False;InformationNodeId=12;TargetsNotLogged=GetNativeManifest,GetCopyToOutputDirectoryItems,GetTargetPath;LogProjectNodes=True;LogWarnings=True;TFSUrl=http://localhost:8080/tfs/DefaultCollection;"*WorkflowForwardingLogger,"C:\Program Files\Microsoft Team Foundation Server 12.0\Tools\Microsoft.TeamFoundation.Build.Server.Logger.dll";"Verbosity=Normal;" /p:BuildId="dbda6e4d-d5bc-4eed-8b3e-6cc79e3721cc,vstfs:///Build/Build/52" /p:BuildLabel="WpfApplication2_20150615.3" /p:BuildTimestamp="Mon, 15 Jun 2015 11:44:59 GMT" /p:BuildSourceVersion="LWpfApplication2_20150615.3#$/Essai2" /p:BuildDefinition="WpfApplication2"
Run optional script after MSBuild00:00:00
InputsEnvironmentVariables:
Enabled: True
Arguments:
FilePath:
OutputsResult: 0
Run optional script before Test Runner00:00:00
InputsEnvironmentVariables:
Enabled: True
Arguments:
FilePath:
OutputsResult: 0
Run VS Test Runner00:00:00
InputsTestSpecs: BuildParameter[] Array
Enabled: False
ConfigurationsToTest: String[] Array
OutDir:
This activity was not run since its Enabled property was set to False.
Run optional script after Test Runner00:00:25
InputsEnvironmentVariables:
Enabled: True
Arguments:
FilePath: C:\sonarqube\bin\SonarQube.MSBuild.Runner.exe
OutputsResult: 0
C:\sonarqube\bin\SonarQube.MSBuild.Runner.exe
Post-processing (no arguments passed)
Using environment variables to determine the download directory...
Using environment variable 'TF_BUILD_BUILDDIRECTORY', value 'C:\Builds\1\Essai2\WpfApplication2'
Executing file C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin\SonarQube.MSBuild.PostProcessor.exe
Args:
Working directory: C:\Builds\1\Essai2\WpfApplication2\sqtemp\bin
Timeout (ms):3600000
Process id: 6548
01:45:28: Legacy TeamBuild environment detected
01:45:28: Loading the SonarQube analysis config from C:\Builds\1\Essai2\WpfApplication2\sqtemp\conf\SonarQubeAnalysisConfig.xml
01:45:28: Legacy TeamBuild environment detected
01:45:28: Attempting to locate the CodeCoverage.exe tool...
01:45:28: Code coverage command line tool: C:\Program Files (x86)\Microsoft Visual Studio 12.0\Team Tools\Dynamic Code Coverage Tools\CodeCoverage.exe
01:45:28: Fetching code coverage report information from TFS...
01:45:28: Connecting to TFS...
01:45:29: Fetching build information...
01:45:30: Fetch code coverage report info...
01:45:52: Operation timed out, Elapsed time (ms): 20358
01:45:52: ...done.
01:45:52: No code coverage reports were found for the current build.
01:45:52: Generating SonarQube project properties file to C:\Builds\1\Essai2\WpfApplication2\sqtemp\out\sonar-project.properties
01:45:52: No ProjectInfo.xml files were found. Check that the analysis targets are referenced by the MSBuild projects being built.
01:45:52: Writing processing summary to C:\Builds\1\Essai2\WpfApplication2\sqtemp\out\ProjectInfo.log
01:45:52: Generation of the sonar-properties file failed. Unable to complete SonarQube analysis.
01:45:52: Updating the TeamBuild summary...
Process returned exit code 1
Exception Message: TF270015: 'SonarQube.MSBuild.Runner.exe' returned an unexpected exit code. Expected '0'; actual '1'. See the build logs for more details. (type UnexpectedExitCodeException) Exception Stack Trace: at System.Activities.Statements.Throw.Execute(CodeActivityContext context) at System.Activities.CodeActivity.InternalExecute(ActivityInstance instance, ActivityExecutor executor, BookmarkManager bookmarkManager) at System.Activities.Runtime.ActivityExecutor.ExecuteActivityWorkItem.ExecuteBody(ActivityExecutor executor, BookmarkManager bookmarkManager, Location resultLocation)
Handle Exception
Troubleshooting issues with the MSBuild.Runner v0.9
The following information is provided to help troubleshooting issues with version 0.9 of the MSBuild.Runner. The behaviour may change in later versions.
Most serious configuration issues will result in warnings or errors that will appear on the Build Summary page. In cases in which there are no errors or warnings reported, check the TFS TeamBuild diagnostic output (click on "Diagnostics" link at the top of the build summary report in Visual Studio). Both the pre- and post-processor steps log quite a lot of information to help diagnose issues.
1. Check the SonarQube Analysis Summary section appears in the Build Summary Report
The section should look like this:
SonarQube Analysis Summary
Analysis failed for SonarQube project "Simple console app", version 1.0
Product projects: 4, test projects: 2
Invalid projects: 0, skipped projects: 0, excluded projects: 0
If the section does not appear on the build summary page then there are configuration issues with the build agent, build definition, or both. Read the steps below to diagnose these further.
1.1 Check the number of projects that are reported in the summary
If no projects were found then it suggests that the integration targets are not being imported and used during the build.
If all of the projects are marked as invalid, it is likely that you are building more than one configuration e.g. [release or debug] | [x86 or x64]. You should also get an error saying no ProjectInfo.xml files could be found.
You can only analyse one configuration at a time. If you want to analyze both release and debug builds, then you will need to create a separate build definition for each, using different SonarQube project ids.
The diagnostic log will contain more infomation about why a project was invalid, which is normally because the project doesn't have a guid, or the guid is not unique. The normal reason for a non-unique guid is that you are building multiple configurations e.g. release and debug.
2. Check that the pre-build step is being executed
The diagnostic output should contain output similar to the following:
Run optional script before MSBuild
C:\SonarQube\bin\SonarQube.MSBuild.Runner.exe /key:MyProject /name:"My Project" /version:1.0
Pre-processing (3 arguments passed)
If it doesn't, check that the you are correctly calling SonarQube.MSBuild.Runner.exe in the "Pre-build script" step in the build definition. An easy mistake is to put the script parameters in the wrong box by setting the "Post-build" properties instead (unhelpfully, the Post-build fields in the UI appear before the Pre-build fields).
2. Check that the post-build step is being executed
The diagnostic output should contain output similar to the following:
Run optional script after Test Runner
C:\SonarQube\bin\SonarQube.MSBuild.Runner.exe
Post-processing (no arguments passed)
If it doesn't, check that the you are correctly calling SonarQube.MSBuild.Runner.exe in the "Post-test script" step in the build definition.
3. Check the rest of the diagnostic log
If the pre- and post- build steps are being executed correctly, check the rest of the diagnostic output for any clues as to what isn't working. Pay particular attention to messages from the pre- and post-processor about the directories and files they are writing to.
4. Check the files that have been created on the build agent
The analysis process creates temporary files under the build directory under a folder called SQTemp in v0.9.
Examining the contents of the folders can help determine the stage at which the analysis is failing.
The expected folders are as follows:
bin: this folder contains the analysis targets and binaries. It is created by the SonarQube.MSBuild.Runner.exe which downloads the files from the SonarQube server. If this folder does not exist or is empty then the analysis failed at a very early stage and there should warnings or errors. Check that the C# plugin v4 is installed correctly on the SonarQube server, and that the SonarQube has been restarted since the plugin was installed.
conf: this folder is created during pre-processing and contains the settings downloaded from the SonarQube server. If there are any FxCop rules active in the SonarQube the folder should contain an FxCop ruleset.
out: this folder is populated during the MSBuild phase. It should contain one folder per project being built. It will also contain the generated sonar-project.properties file that is passed to the sonar-runner. If the folder is empty then it is possible the SonarQube.Integration.targets are not being imported correctly.
The error message "No ProjectInfo.xml files were found" will normally appear in this case. The ProjectInfo.xml files are generated by the target "WriteSonarQubeData" in the SonarQube.Integration.targets, so the next thing to check is that SonarQube.Integration.targets are being imported correctly and the expected targets are being executed.
5. Checking the SonarQube.Integration.targets are imported and called
Firstly, double-check the installation of the SonarQube.Integration.ImportBefore.targets file:
check it is in the correct folder for the version of MSBuild you are using i.e. %ProgramFiles(x86)%\MSBuild\[12.0 or 14.0]\Microsoft.Common.Targets\ImportBefore
check for typos in the folder name "ImportBefore" (should be singular, not plural)
(Thanks to Richard from BlackMarble for this tip).
If the targets are installed correctly then you will need to increase the verbosity of the MSBuild logging to diagnostic and build again so you can check the detailed MSBuild output. Unfortunately, there doesn't seem to be a simple way to increase the verbosity of the MSBuild logs when running under TeamBuild 2013. You can produce an additional log file with more detail, but you will need access to the build agent machine to pick up the log file.
To create an MSBuild log with more detailed info:
select "Queue New Build" from the BUILD menu in Visual Studio
click on the "Parameters" tab.
Expand the "2. Build" section and the "5. Advanced" sections
Set the "MSBuild arguments" property to the following:
/m:1 /fl2 /flp2:"verbosity=diagnostic"
The "/m:1" tells MSBuild not to build in parallel which can make the log easier to read. The other two parameters tell MSBuild to log to a file. See the MSBuild command line help for more info.
Click "Queue" and wait for the build to finish.
Find and open the log on the build agent. It should be called msbuild2.log and be under the source directory for your build definition.
If the SonarQube.Intergration.ImportBefore.targets are being executed, there will be a log entry saying the "SonarQubeImportBeforeInfo" target was executed. There should be some output message saying whether the file "SonarQube.Integration.targets" was located or not.
The ProjectInfo.xml files are written by the target "WriteSonarQubeProjectData". The log should show that this target has been executed, and the "WriteProjectInfoFile" task has executed. If not, the log should give an idea of why the targets were skipped.

Ant: failed to create task or type runtarget

I am running into the following error when trying to run ant:
Problem: failed to create task or type runtarget
I am building on a mac 10.8.3.
Prior research has suggested that I add ant-contrib-0.3.jar to my ANT_HOME installation directory, which I have done (that had actually gotten rid of another 'failed to create task or type' error)
I used ant-contrib-0.3.jar because research suggested that this jar is mapped to the line:
< taskdef resource="net/sf/antcontrib/antcontrib.properties" />
which is in the build.xml file I am using.
The project builds on windows machines ( I even got it to build using https://code.google.com/p/winant/ ) but am trying to get it built on a mac. I am thus not looking to change the build.xml file.
An example of the run target line is:
<target name="setPASProps" depends="" description="setup the properties">
<property name="systemname" value="PAS"/>
<runtarget target="setSystemProps"/>
</target>
Here is some info from running ant -diagnostics
-------------------------------------------
ANT PROPERTIES
-------------------------------------------
ant.version: Apache Ant(TM) version 1.8.2 compiled on June 20 2012
ant.java.version: 1.7
Is this the Apache Harmony VM? no
Is this the Kaffe VM? no
Is this gij/gcj? no
ant.core.lib: /usr/share/ant/lib/ant.jar
ant.home: /usr/share/ant
-------------------------------------------
ANT_HOME/lib jar listing
-------------------------------------------
ant.home: /usr/share/ant
ant-antlr.jar (5756 bytes)
ant-contrib-0.3.jar (17708 bytes)
ant-jmf.jar (6745 bytes)
ant-junit.jar (102350 bytes)
ant-junit4.jar (7126 bytes)
ant-launcher.jar (12321 bytes)
ant-swing.jar (7563 bytes)
ant-testutil.jar (15198 bytes)
ant.jar (1937098 bytes)
Thanks !
It would be helpful if you posted your build.xml too.
You usually get this error if Ant sees a task, but there's a problem with the definition.
Here's my recommendation:
In your project create a directory antlib/ant-contrib.
Download this zip file. Ant-contrib is a wee bit strange is that there is a separate jar for C compiling and for all of the other Ant tasks. The latest version is 1.0b3. When you unzip this zip file, you will see ant-contrib-1.0b3.jar inside this folder.
Put that ant-contrib-1.0b3.jar inside the antlib/ant-contrib folder.
Now, in your build.xml, use the following <taskdef/>:
<taskdef resource="net/sf/antcontrib/antlib.xml">
<classpath>
<fileset dir="${basedir}/antlib/ant-contrib"/>
</classpath>
</taskdef>
Make sure this is not inside a target. You want this to be executed before any targets are executed.
I like doing the definition this way because the Ant Contrib jar file becomes part of my project, and anyone who needs to run my project will not have to manually install the Ant Contrib jar before they can use my project.
Take a look at your build.xml and see where that <Runtarget> task is being used. I've never used it, and the documentation for this task is so clear and helpful. Actually, I'm not even sure if it works. If you are still having problems, you try to see if you can remove the defined <target/> that contains this task, and see if that gets rid of the issue.
Removed ant-contrib-0.3.jar and added ant-contrib-1.0b3.jar and ant-contrib.jar to my ANT_HOME directory since these are the jars installed with https://code.google.com/p/winant/ (and it was working on windows machines).
This did the trick.

CruiseControl.net ndk-build on Windows 64bit without Cygwin

Latest Android NDK (version 8) allows to call ndk-build without additional cygwin installed.
My question is: can I plug this into CruiseControl.Net for periodical native build?
I assume that most likely I would need to use Ant as a build method and then add it to ccnet server config.
So the questions are:
How to call ndk-build.cmd from an Ant command
How to configure build.xml file?
I tried several approaches, but all of those failed. Could you give me some clues if it is possible at all?
I ran into the same problem (the CreateProcess failures, etc) when trying to invoke "ndk-build.cmd" directly from build.xml, using CC.net on Windows. I figured out how to get this to work, and so I felt compelled to share my solution.
First, create a 1-line Windows command file as follows (I named it "ndkwrapper.cmd"):
sh ndkwrap.sh %1 %2 %3
Next, create a simple bash shell script as follows (named "ndkwrap.sh"):
#!/usr/bin/sh
# Wrapper around ndk-build script for Windows
#
NDK_PROJECT_PATH=/cygdrive/c/workspace/agnc_android
export NDK_PROJECT_PATH
/cygdrive/c/Android/android-ndk-r8b/ndk-build $1 $2 $3
Of course, your exact paths (for your workspace and NDK directories) may be different, but note, in particular, that "/cygdrive/c" refers to "C:".
One very important step here, after you create the script above, is to convert the line endings from Windows to UNIX. You can do this with a utility called "dos2unix.exe" which is freely available. I don't remember where I got it, but it was part of some open source package of GNU/Linux tools ported to Windows. (Google on "UNIX file utilities for Windows".) If you don't convert the line endings, then sh or bash will have trouble reading the script, and you'll get all kinds of erros about "\r" ...
So, to invoke the equivalent of "ndk-build.cmd clean", for example, you'd type "ndkwrapper.cmd clean" to delete your intermediate and output NDK-related build files, in preparation for a full NDK build.
In your build.xml file for CC.net on Windows, you can invoke the NDK makefile as follows:
<tasks>
<exec>
<executable>cmd.exe</executable>
<baseDirectory>C:\Android</baseDirectory>
<buildArgs>/C ndkwrapper.cmd clean</buildArgs>
</exec>
Hope this helps!
Ben
i observed problems with running ndk-build as an CCNET task as well.
It took me a while, but at the end i noticed, that you have to provide HOST_OS and HOST_ARCH as ndk-build parameters to let it run.
<exec>
<executable>cmd</executable>
<buildArgs>/C ndk-build HOST_OS=windows HOST_ARCH=x86 -B NDK_PROJECT_PATH=[PROJECT] APP_BUILD_SCRIPT=[ANDROIDMKFILE] NDK_APPLICATION_MK=[APPLICATIONMKFILE] NDK_LOG=1</buildArgs>
<buildTimeoutSeconds>120</buildTimeoutSeconds>
</exec>
hope it helps anyone to save time.
Ok I got some progress, I am able to build the jni code via ant or nant but in both cases plugging it to the cc.net server config gives me strane error:
but now CC.net gives me strange errors:
<builderror>
<type>NAnt.Core.BuildException</type>
<message><![CDATA[External Program Failed: G:\\android-ndk-r8b\\ndk-build.cmd (return code was 2)]]></message>
<location>
<filename>G:/MYPath/project.build</filename>
<linenumber>7</linenumber>
<columnnumber>4</columnnumber>
</location>
<stacktrace><![CDATA[ at NAnt.Core.Tasks.ExternalProgramBase.ExecuteTask()
at NAnt.Core.Tasks.ExecTask.ExecuteTask()
at NAnt.Core.Task.Execute()
at NAnt.Core.Target.Execute()
at NAnt.Core.Project.Execute(String targetName, Boolean forceDependencies)
at NAnt.Core.Project.Execute()
at NAnt.Core.Project.Run()]]></stacktrace>
</builderror>
</failure>
<duration>296.40000000000003</duration>
</buildresults>Buildfile: file:///G:/MYPath/project.build
Target framework: Microsoft .NET Framework 4.0
Target(s) specified: build
clean:
[echo] Starting Android ndk CLEAN ...
[exec] Android NDK: Unable to determine HOST_OS from uname -s:
[exec] Android NDK: Please define HOST_OS in your environment.
[exec] process_begin: CreateProcess(NULL, uname -s, ...) failed.
[exec] G:/android-ndk-r8b/build/core/init.mk:131: *** Android NDK: Aborting. . Stop.
BUILD FAILED - 0 non-fatal error(s), 2 warning(s)
My project in cc.net config:
<project>
<name>MY_PROJECT_NAME</name>
<workingDirectory>PATH_TO_MY_PROJECT</workingDirectory>
<tasks>
<nant>
<executable>G:\nant-0.92\bin\Nant.exe</executable>
<baseDirectory>PATH_TO_MY_PROJECT</baseDirectory>
<buildFile>MYPROJECTNAME.build</buildFile>
<buildArgs>build</buildArgs>
<buildTimeoutSeconds>1200</buildTimeoutSeconds>
</nant>
</tasks>
</project>
And my NAnt build file:
<project name="my name" default="build" basedir=".">
<description>My project Build Files.</description>
<target name="clean" description="remove all generated files">
<echo message="Starting Android ndk CLEAN ..."/>
<exec program="G:\\android-ndk-r8b\\ndk-build.cmd" failonerror="true"
basedir="MY PROJECT PATH">
<arg value="clean" />
</exec>
<echo message="Android ndk CLEAN done."/>
</target>
<target name="build" description="remove all generated files" depends="clean">
<echo message="Starting Android ndk BUILD ..."/>
<exec program="G:/android-ndk-r8b/ndk-build.cmd" failonerror="true" />
<echo message="Android ndk BUILD done."/>
</target>
</project>
As I said I can run a Nant.exe for my project and it cleans and build correctly.
It looks like the cc.net tries to run the other ndk-build commend which is used for linux and is missing some commands as uname.
Do you have any idea what cI could be doing wrong ?
The original question is asking about ant builds. This answer is related to a problem in CC.NET and ant and gradle builds are going to be affected in the same way.
We are using gradle and with some custom gradle tasks it is possible to compile the native code of your project as well by calling ndk-build.cmd (from a path that has no spaces in it).
After the gradle file is prepared properly shell initiated gradle builds will work but cc.net initiated builds will fail.
Our CC.NET task is defined as follows:
<exec executable=".\gradlew.bat">
<baseDirectory>$(projSrcDir)</baseDirectory>
<buildArgs>clean assemblePlayRelease assembleProRelease</buildArgs>
<buildTimeoutSeconds>900</buildTimeoutSeconds>
</exec>
The problem is related to a CC.NET bug that causes the shell variable names to be in lower case. Windows shell scripts do not care about the case of shell variable names but because the NDK fires up make variable case becomes an issue. The init.mk make file that is part of the build process relies on shell variable names to identify the host OS type. In particular the shell variable OS. Normally the OS value is defined like this:
OS=Windows_NT
But when the variable is passed to gradle from CC.NET it gets transformed into this:
os=Windows_NT
This causes the detection mechanism in init.mk to fail and tries to execute the unix detection sequence and looks for the Unix uname utility that is not present on Windows resulting in:
Android NDK: Unable to determine HOST_OS from uname -s:
Android NDK: Please define HOST_OS in your environment.
make.exe: Entering directory `D:/CC.NET/svncheckout/pes/src/project/src/main/jni'
make.exe: Leaving directory `D:/CC.NET/svncheckout/pes/src/project/src/main/jni'
And ultimately:
process_begin: CreateProcess(NULL, uname -s, ...) failed.
c:/ndk/build/core/init.mk:160: *** Android NDK: Aborting. . Stop.
FAILURE: Build failed with an exception.
The workaround is less then ideal but it gets the job done. The idea is to edit the ndk-build.cmd and change the case of the passed variables only when we are building from CC.NET.
Simply insert this code after the #echo off line in ndk-build.cmd:
IF DEFINED ccnetproject (
ECHO Applying Environment variable hack for CC.NET.
SET OS2=%os%
SET OS=
REM SET PATH=%path%
SET TEMP2=%temp%
SET TEMP=
)
IF DEFINED ccnetproject (
SET OS=%OS2%
SET OS2=
SET TEMP=%TEMP2%
SET TEMP2=
)
This script first makes temporary copies of values in the lower case OS and TEMP variables. Next it undefines them by clearing their values. An finally the reverse is done.
Two steps are needed because just saying
SET OS=%os%
doesn't do much since scripts are case insensitive. It first locates the OS variable, finds a lower case version and assigns its own value back to itself.
This was tested on CC.NET v1.6.7981.1, gradle v1.1.0 and Android NDK v10 (64bit).

Can't run "preverify.exe" using Ant's <exec> task

I am having trouble getting my Ant script (for BlackBerry build) to run the preverify.exe command & pass the correct parameters to it.
In the command prompt (Windows 7), this works 100% - the parameters as given work properly:
preverify -verbose -classpath C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar -d build\classes\preverified build\classes\preverified build\classes\unverified
I tried to put this into my Ant script using the following target - trying to use the same parameters:
<target name="preverify">
<mkdir dir="${dest.dir}/classes/preverified" />
<exec executable="${jde.home}/bin/preverify">
<arg value="-verbose" />
<arg value="-classpath C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar" />
<arg value="-d build\classes\preverified" />
<arg value="build\classes\unverified" />
</exec>
</target>
This does not work. I get the following error:
Illegal option
-classpath C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar
this classpath was perfectly acceptable from the command line (often java commands accept JAR files as directories, since they are basically ZIP files).
How can I get Ant to send the correct parameters to this command, as in the command line version? There must be something about exec that I'm missing?
Here is the full Ant output from running this target in verbose mode, if it helps:
Apache Ant(TM) version 1.8.2 compiled on December 20 2010
Trying the default build file: build.xml
Buildfile: C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi\build.xml
Detected Java version: 1.6 in: C:\Java\jdk1.6.0_24\jre
Detected OS: Windows 7
parsing buildfile C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi\build.xml with URI = file:/C:/development/ant/test_using_javac_jar_preverify_then_rapc/Cobi/build.xml
Project base dir set to: C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi
parsing buildfile jar:file:/C:/development/tools/apache-ant/lib/ant.jar!/org/apache/tools/ant/antlib.xml with URI = jar:file:/C:/development/tools/apache-ant/lib/ant.jar!/org/apache/tools/ant/antlib.xml from a zip file
Importing file C:\development\ant\common\constants.xml from C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi\build.xml
Overriding previous definition of reference to ant.projectHelper
parsing buildfile C:\development\ant\common\constants.xml with URI = file:/C:/development/ant/common/constants.xml
parsing buildfile jar:file:/C:/development/tools/bb-ant-tools/bb-ant-tools.jar!/bb-ant-defs.xml with URI = jar:file:/C:/development/tools/bb-ant-tools/bb-ant-tools.jar!/bb-ant-defs.xml from a zip file
Overriding previous definition of reference to ant.projectHelper
[property] Loading C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi\project.properties
[property] Loading C:\development\ant\common\jde5.0.properties
[property] Loading C:\development\ant\common\common.properties
[pathconvert] Set property net_rim_api.jar.dos = C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar
Build sequence for target(s) `preverify' is [preverify]
Complete build sequence is [preverify, javac, build, sign, clean, ]
preverify:
[mkdir] Skipping C:\development\ant\test_using_javac_jar_preverify_then_rapc\Cobi\build\classes\preverified because it already exists.
[exec] Current OS is Windows 7
[exec] Executing 'C:\development\tools\bb-jde\jde5.0\components\bin\preverify' with arguments:
[exec] '-verbose'
[exec] '-classpath C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar'
[exec] '-d build\classes\preverified'
[exec] 'build\classes\unverified'
[exec]
[exec] The ' characters around the executable and arguments are
[exec] not part of the command.
[exec] preverify: Illegal option -classpath C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar
[exec]
[exec] Usage: preverify [options] classnames|dirnames ...
[exec]
[exec] where options include:
[exec] -classpath <directories separated by ';'>
[exec] Directories in which to look for classes
[exec] -d <directory> Directory in which output is written (default is ./output/)
[exec] -cldc1.0 Checks for existence of language features prohibited
[exec] by CLDC 1.0 (native methods, floating point and finalizers)
[exec] -nofinalize No finalizers allowed
[exec] -nonative No native methods allowed
[exec] -nofp No floating point operations allowed
[exec] #<filename> Read command line arguments from a text file
[exec] Command line arguments must all be on a single line
[exec] Directory names must be enclosed in double quotes (")
[exec]
[exec] Result: 1
BUILD SUCCESSFUL
Total time: 1 second
This doesn't look like an ANT issue. The error message is being returned by the preverify command, proving that ANT is executing it...
I don't understand what this command is supposed to be doing, however the usage message gives a hint as to the root cause:
[exec] Usage: preverify [options] classnames|dirnames ...
[exec]
[exec] where options include:
[exec] -classpath <directories separated by ';'>
[exec] Directories in which to look for classes
You haven't specified a list of directories as the "classpath" parameter.... You've supplied a jar file. Is the command able support jar files?
The way you are passing the parameters is incorrect. The space between the -classpath tag, and the JAR name is not allowed.
You must break that line (and the -d below it) onto 2 lines. This works:
<exec executable="${jde.home}/bin/preverify">
<arg value="-verbose" />
<!-- classpath to the RIM api -->
<arg value="-classpath" />
<arg value="C:\development\tools\bb-jde\jde5.0\components\lib\net_rim_api.jar" />
<!-- destination folder -->
<arg value="-d" />
<arg value="build\classes\preverified" />
<!-- source folder -->
<arg value="build\classes\unverified" />
</exec>
I solved this problem by including the jdk\bin directory path in environment variable PATH.

Resources