How can I compile apache PIG in IntelliJ? - ant

I'm trying to get apache pig running locally from source in IntelliJ. I'd like to step through the unit tests.
I've done the following steps:
git clone https://github.com/apache/pig.git
cd pig;
git checkout release-0.9.1
ant
Does anyone have instructions after that to get it imported into IntelliJ? It compiles fine from the cmd line with ant.
When I do FILE -> IMPORT PROJECT and then build I get complication errors complaining of duplicate classes. All I'm trying to do is BUILD -> MAKE PROJECT in intellij :/
thanks!

found the answer:
git clone https://github.com/apache/pig.git
cd pig;
git checkout release-0.9.1
ant
ant eclipse-files
Open IntelliJ then do
FILE -> IMPORT
as type "ECLIPSE" project and away you go!

Related

Copy all output jars with dependencies into one folder

I've got a parent POM.xml defining 4 modules, 3 of which reference the first one (this project to be precise).
I am trying to copy all output jars with their runtime dependencies into one folder using the following commands:
mvn package
mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.1:copy-dependencies
-DoutputDirectory=deps -DincludeScope=runtime
-DexcludeGroupIds=com.acadiasoft.im
The first command succeeds and the last fails with the error:
[ERROR] Failed to execute goal on project simm: Could not resolve dependencies for project com.acadiasoft.im:simm:jar:1.0.0-SNAPSHOT: Could not find artifact com.acadiasoft.im:base:jar:1.0.0-SNAPSHOT -> [Help 1]
Note the last parameter -DexcludeGroupIds=com.acadiasoft.im which is how I tried working around this error. I also tried excludeArtifactIds to the same effect.
Question 1: Why doesn't it work? Why is it not skipping artefacts with group id com.acadiasoft.im?
Question 2: Is there a way to make maven 'see' the the jars produced in the mvn package step and copy them over along with their dependencies?
Version info:
Maven 3.6.1
JDK 1.8
Please note that I would prefer a command line based solution rather than having to modify a POM. Thanks.
working spell is
mvn clean install
mvn org.apache.maven.plugins:maven-dependency-plugin:3.1.1:copy-dependencies -DoutputDirectory=deps -DincludeScope=runtime -DexcludeGroupIds=com.acadiasoft.im
note package -> install

How to debug the Jenkins plugin

I have built a new "Hello World" plugin for Jenkins and I was able to upload it onto Jenkins and it works successfully. I am now trying to make change to the plugin and debug it on Jenkins.
Note:
My Jenkins is deployed in my local tomcat, i build the hpi file and copy it to jenkins plugins folder, it works.
Now i want to ask how can i debug my plugin code? Suggestions with any IDE is welcome.
I also have questions about:
do i need to create a project for jenkins source in my IDE and develop my plugin base on it?
Currently i only create a project for the plugin, build a hpi file and put it to Jenkins.
I am not sure if this step is correct.
Thanks in advance!
I imagine you ran
mvn package
to create your package
To debug you can do
mvnDebug hpi:run
and this will copy all the dependencies down (rather than in your jenkins install) and run it in place
If you are using an IDE then this can be done from within it.
More help can be found in the plugin tutorial
Follow the tutorial to set MAVEN_OPTS, and using IntelliJ to add a Jetty Server to point the port to it.
execute from cmd:
mvn clean
set MAVEN_OPTS=-Xdebug -Xrunjdwp:transport=dt_socket,server=y,address=8000,suspend=n
mvn hpi:run
add a debug local Jetty Server from intelliJ and point it to port 8000 (port number is specified in above cmd)
In case of IntelliJ IDEA the easiest way is to use maven's Run/Debug configuration.
Just add hpi:run in "Command line" filed and start Debug as usual.
When the console says that your "Jenkins is fully up and running", open a browser and go to http://localhost:8080/jenkins/ and your code will stopped at break point (if any).

JSchException while using Jgit api

I'm trying to clone my git repository using Jgit and I get this exception:
`java.lang.NoClassDefFoundError: com/jcraft/jsch/JSchException
at org.eclipse.jgit.transport.Transport.<clinit>(Transport.java:112)
at org.eclipse.jgit.api.FetchCommand.call(FetchCommand.java:118)
at org.eclipse.jgit.api.CloneCommand.fetch(CloneCommand.java:178)
at org.eclipse.jgit.api.CloneCommand.call(CloneCommand.java:125)
`
I downloaded and copied Jsch in /lib and restarted my application (since Jgit has dependencies on this jar), I still get the same error. I'm running a java application build using ant from command line. Any other details please let me know.

How to execute sbt tasks inside Play projects cloned from git to a single directory under Jenkins?

Currently the HudsonPluginForPlay doesn't support Play 2.x and the author hasn't updated the plugin for quite some time. So I'm trying to figure out a way to get the build automated and tested on my own using sbt-launcher plugin as highlighted by Play framework 2.0 continuous integration setup.
However, I've run into a problem where my git checkout structure is like this,
project/
project1/ (Play project 1)
project2/ (Play project 2)
Now sbt seems to run on the root under project/ and it doesn't do anything.
Is there a way to get it to say, run the sbt commands under project1/ and then project2/? I tried using shell command to cd into the directories but that doesn't seem to do anything in particular.
You may be quite successful doing the following:
(cd project1; sbt package); (cd project2; sbt package)
to execute package command inside the projects. It has in fact nothing to do with sbt since I've only leveraged the fact you build the projects on Unix and just cd to each project.
There's not much you can do with sbt since defining the projects as submodules or a parent project would disable project-part of each Play project's build, i.e. sbt can also be defined via a Scala code inside project/ directory inside a sbt project, but it can only be inside the root project. If you've defined the directory project as the root project the submodules may be damaged.

Cannot run program "gradle" in Jenkins

I've installed the Gradle plugin for Jenkins. When I try to build the project I get the following error:
[workspace] $ gradle clean -b build/build.gradle
FATAL: command execution failed
java.io.IOException: Cannot run program "gradle" (in directory "/Users/Shared/Jenkins/Home/jobs/test/workspace"): error=2, No such file or directory
at java.lang.ProcessBuilder.start(ProcessBuilder.java:460)
at hudson.Proc$LocalProc.<init>(Proc.java:244)
at hudson.Proc$LocalProc.<init>(Proc.java:216)
at hudson.Launcher$LocalLauncher.launch(Launcher.java:707)
at hudson.Launcher$ProcStarter.start(Launcher.java:338)
at hudson.Launcher$ProcStarter.join(Launcher.java:345)
at hudson.plugins.gradle.Gradle.performTask(Gradle.java:201)
at hudson.plugins.gradle.Gradle.perform(Gradle.java:97)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:19)
at hudson.model.AbstractBuild$AbstractRunner.perform(AbstractBuild.java:695)
at hudson.model.Build$RunnerImpl.build(Build.java:178)
at hudson.model.Build$RunnerImpl.doRun(Build.java:139)
at hudson.model.AbstractBuild$AbstractRunner.run(AbstractBuild.java:465)
at hudson.model.Run.run(Run.java:1404)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:46)
at hudson.model.ResourceController.execute(ResourceController.java:88)
at hudson.model.Executor.run(Executor.java:238)
Caused by: java.io.IOException: error=2, No such file or directory
at java.lang.UNIXProcess.forkAndExec(Native Method)
at java.lang.UNIXProcess.<init>(UNIXProcess.java:53)
at java.lang.ProcessImpl.start(ProcessImpl.java:91)
at java.lang.ProcessBuilder.start(ProcessBuilder.java:453)
... 16 more
Build step 'Invoke Gradle script' changed build result to FAILURE
Build step 'Invoke Gradle script' marked build as failure
Finished: FAILURE
Running the task $ gradle clean -b build/build.gradle from the command line works as expected.
I had the same issue and found that the problem for me was the gradle version in the project configuration. It was set to Default and when I set it to the gradle version I was pointing to in the plugin configuration in the Manage Jenkins > Configuration Options, It found gradle and worked properly.
This is a difficult issue to debug and I hope this saves someone else some time.
Solution Mastering_the_Object pointed out worked form me too, just to clarify steps there:
just installing Gradle plugin in Jenkins is not enough, you must also go to:
Jenkins->Manage Jenkins->Configure Jenkins->Configure system.
Under "Gradle Installations" type in name (it appears as version in project configuration), check "Install automatically" and select version. Then you can select that "Gradle version" in project configuration.
I was getting this error using the Gradle wrapper, was able to fix my broken build as follows:
Go to Manage Jenkins -> Global Tool Configuration -> Gradle -> Add Gradle, give it a name
Go to Jenkins -> (your job) -> Configure -> Build, choose "Invoke Gradle" and change the Gradle version from (Default) to the named version
When running Gradle on a CI machine such as Jenkins, it's most convenient to use the Gradle Wrapper.
On your development machine, stand in your root project directory and run
gradle wrapper
Then, check in the resulting files into your source control system. After that, you don't have to install anything on your Jenkins server if you need to change Gradle versions. Just configure your Jenkins job like this:
Quoting from the Gradle User Guide:
By distributing the wrapper with your project, anyone can work with it
without needing to install Gradle beforehand. Even better, users of
the build are guaranteed to use the version of Gradle that the build
was designed to work with. Of course, this is also great for
continuous integration servers (i.e. servers that regularly build your
project) as it requires no configuration on the server.
The jenkins can not find the gradle executable. I have never made gradle wrapper to work. Please follow the following steps:
Download gradle ( http://gradle.org/downloads),
unpack it to, e.g., /usr/local/lib/gradle ,
open /etc/profile and add the following two lines:
export GRADLE_HOME=/usr/local/lib/gradle
export PATH=$PATH:$GRADLE_HOME/bin
It works for my jenkins installation.
ps. I answer this old question, because it is a common issue when setting up gradle with jenkins. I have spent some time trying to make the gradle wrapper work without success before.
Used both #Skarab & a number of other solutions here to get it to work.
Download gradle ( http://gradle.org/downloads),
unpack it to, e.g., /usr/local/lib/gradle ,
open /etc/profile and add the following two lines:
export GRADLE_HOME=/usr/local/lib/gradle
export PATH=$PATH:$GRADLE_HOME/bin
Then under Manage Jenkins > Configure system > Gradle, defined GRADLE_HOME and gave that gradle installation a name
Then, {this could be a bug} for the project, change gradle version from Default to the one I defined above
As others have noted, this is because Jenkins can't find the gradle executable.
None of the published solutions worked for me, but I was able to fix the problem by adding the gradle bin dir to the path set in the .bashrc for the build account on the build slave. Modifying the .profile failed, and setting PATH in the jenkins node configuration also failed.
Some posts will suggest setting the gradle path in the tools menu, but no gradle entry was available there either (perhaps due to regressions / design changes in the gradle plugin?).
In any case, the best test I found (short of running the build again and watching for failure) was to run env over ssh:
ssh <host> env
and check the PATH variable defined that way; if gradle isn't in that path, you probably won't be able to run it from jenkins.

Resources