Polling for Jobs issue on Jenkins initiated from AWS pipeline - jenkins

I am facing following issue on Jenkins.
What I am doing is that I have setup a pipeline on AWS and it is all working fine with deployments etc.
I have setup a Jenkins project already on a server separately for the testing purpose of APIs test cases.
What I am doing connecting the Jenkins server with the Build stage of the AWS. All seem to go smooth but on the Jenkins server I found the following issue.
To make it simple, I just tried to execute the date in a variable and print it (echo) and comment out the original running script command. In build execute shell area only following two command are written.
now='date +"%Y-%m-%d-%H%M%S"'
echo $now
[AWS CodePipeline Plugin] Polling for jobs for action type id: [Owner:
Custom, Category: Test, Provider: Jenkins_November, Version: 1,
ProjectName: Chiragh_Tests] ERROR: Failed to record SCM polling for
hudson.model.FreeStyleProject#5f5daea6[Chiragh_Tests]
com.amazonaws.AmazonServiceException: Unable to unmarshall exception
response with the unmarshallers provided (Service: AWSCodePipeline;
Status Code: 404; Error Code: null; Request ID: null) at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleErrorResponse(AmazonHttpClient.java:1799)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleServiceErrorResponse(AmazonHttpClient.java:1383)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1359)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1139)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:796)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:764)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:738)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:698)
at
com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:680)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:544)
at
com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:524)
at
com.amazonaws.services.codepipeline.AWSCodePipelineClient.doInvoke(AWSCodePipelineClient.java:3040)
at
com.amazonaws.services.codepipeline.AWSCodePipelineClient.invoke(AWSCodePipelineClient.java:3007)
at
com.amazonaws.services.codepipeline.AWSCodePipelineClient.invoke(AWSCodePipelineClient.java:2996)
at
com.amazonaws.services.codepipeline.AWSCodePipelineClient.executePollForJobs(AWSCodePipelineClient.java:1990)
at
com.amazonaws.services.codepipeline.AWSCodePipelineClient.pollForJobs(AWSCodePipelineClient.java:1961)
at
com.amazonaws.codepipeline.jenkinsplugin.AWSCodePipelineSCM.pollForJobs(AWSCodePipelineSCM.java:255)
at
com.amazonaws.codepipeline.jenkinsplugin.AWSCodePipelineSCM.compareRemoteRevisionWith(AWSCodePipelineSCM.java:191)
at hudson.scm.SCM.poll(SCM.java:418) at
hudson.model.AbstractProject._poll(AbstractProject.java:1388) at
hudson.model.AbstractProject.poll(AbstractProject.java:1291) at
hudson.triggers.SCMTrigger$Runner.runPolling(SCMTrigger.java:606) at
hudson.triggers.SCMTrigger$Runner.run(SCMTrigger.java:652) at
hudson.util.SequentialExecutionQueue$QueueEntry.run(SequentialExecutionQueue.java:119)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266) at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)

It seems to me that either the Jenkins Server is unable to connect to the CodePipeline service or the Jenkins server has some missing dependency with respect to AWS SDK for Java.
Can you SSH into the Jenkins Server and run the following AWS CLI command to confirm connectivity with CodePipeline service:
aws codepipeline poll-for-jobs --action-type-id=category=Test,owner=Custom,provider=Jenkins_November,version=1
SSH Into a Linux Server
Installing AWS CLI

Related

Jenkins build over ssh times out on first attempt

I have successfully set up a build with Jenkins (version 2.375.1) that is triggered by a GitHub web-hook. Dockerized Jenkins is running on an Ubuntu VM locally.
If I push from my local machine to GitHub, then this will initiate a build using build step: "Execute shell script on remote host using ssh" on the target AWS and run some steps to install the application. However, if I leave it too long the job times out. If I make a change and push again, or if I just hit build now, then the build is successful.
It seems like the connection from Jenkins to AWS is going to sleep and it requires the first attempt to "wake it up". I can't find any reference to this behaviour anywhere.
At the end of the console output....
[SSH] executing...
[SSH] Exception:Session.connect: java.net.SocketTimeoutException: Read timed out
com.jcraft.jsch.JSchException: Session.connect: java.net.SocketTimeoutException: Read timed out
at com.jcraft.jsch.Session.connect(Session.java:565)
at org.jvnet.hudson.plugins.CredentialsSSHSite.createSession(CredentialsSSHSite.java:132)
at org.jvnet.hudson.plugins.CredentialsSSHSite.executeCommand(CredentialsSSHSite.java:208)
at org.jvnet.hudson.plugins.SSHBuilder.perform(SSHBuilder.java:104)
at hudson.tasks.BuildStepMonitor$1.perform(BuildStepMonitor.java:20)
at hudson.model.AbstractBuild$AbstractBuildExecution.perform(AbstractBuild.java:818)
at hudson.model.Build$BuildExecution.build(Build.java:199)
at hudson.model.Build$BuildExecution.doRun(Build.java:164)
at hudson.model.AbstractBuild$AbstractBuildExecution.run(AbstractBuild.java:526)
at hudson.model.Run.execute(Run.java:1900)
at hudson.model.FreeStyleBuild.run(FreeStyleBuild.java:44)
at hudson.model.ResourceController.execute(ResourceController.java:107)
at hudson.model.Executor.run(Executor.java:449)
Build step 'Execute shell script on remote host using ssh' marked build as failure
Finished: FAILURE

Jenkins not able to connect sonarqube server

I'm running Ubuntu on Oracle VM from my Windows 10 machine.
On Ubuntu, I'm running Jenkins in a docker container.
In this I've created a job which in which a I've added a SonarScanner build step.
In sonar configurations, I've mentioned a remote URL of my sonar server. Same URL can be opened in web browser without any problem.
I've also mentioned sonar project properties in the build step.
But when I execute the build, I get error, that jenkins is not able to connect sonar server.
But it's fishy since, I'm able to open same URL from my browser.
The jenkins setup & sonar configurations also seems to correct as same I've used in my native window's jenkins setup & it worked.
So I think it is something to do with VM, but not sure exactly what.
Any suggestions.
Also, on Ubuntu I'm working as a default user with admin access. (Not as root user)
Thank you
Error Trace:
10:34:27.358 ERROR: Error during SonarQube Scanner execution
org.sonarsource.scanner.api.internal.ScannerException: Unable to execute SonarQube
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:85)
at java.security.AccessController.doPrivileged(Native Method)
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:74)
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.createLauncher(IsolatedLauncherFactory.java:70)
at org.sonarsource.scanner.api.EmbeddedScanner.doStart(EmbeddedScanner.java:167)
at org.sonarsource.scanner.api.EmbeddedScanner.start(EmbeddedScanner.java:113)
at org.sonarsource.scanner.cli.Main.execute(Main.java:73)
at org.sonarsource.scanner.cli.Main.main(Main.java:61)
Caused by: java.lang.IllegalStateException: Fail to get bootstrap index from server
at org.sonarsource.scanner.api.internal.BootstrapIndexDownloader.getIndex(BootstrapIndexDownloader.java:42)
at org.sonarsource.scanner.api.internal.JarDownloader.getScannerEngineFiles(JarDownloader.java:58)
at org.sonarsource.scanner.api.internal.JarDownloader.download(JarDownloader.java:53)
at org.sonarsource.scanner.api.internal.IsolatedLauncherFactory.lambda$createLauncher$0(IsolatedLauncherFactory.java:76)
... 7 more
Caused by: java.net.UnknownHostException: sonarqube.intranet.myhost.com: Name or service not known
at java.net.Inet4AddressImpl.lookupAllHostAddr(Native Method)
at java.net.InetAddress$2.lookupAllHostAddr(InetAddress.java:928)
at java.net.InetAddress.getAddressesFromNameService(InetAddress.java:1323)
at java.net.InetAddress.getAllByName0(InetAddress.java:1276)
at java.net.InetAddress.getAllByName(InetAddress.java:1192)
at java.net.InetAddress.getAllByName(InetAddress.java:1126)
at org.sonarsource.scanner.api.internal.shaded.okhttp.Dns$1.lookup(Dns.java:39)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RouteSelector.resetNextInetSocketAddress(RouteSelector.java:171)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RouteSelector.nextProxy(RouteSelector.java:137)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.RouteSelector.next(RouteSelector.java:82)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.StreamAllocation.findConnection(StreamAllocation.java:171)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.StreamAllocation.findHealthyConnection(StreamAllocation.java:121)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.StreamAllocation.newStream(StreamAllocation.java:100)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.connection.ConnectInterceptor.intercept(ConnectInterceptor.java:42)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.cache.CacheInterceptor.intercept(CacheInterceptor.java:93)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.BridgeInterceptor.intercept(BridgeInterceptor.java:93)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RetryAndFollowUpInterceptor.intercept(RetryAndFollowUpInterceptor.java:120)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:92)
at org.sonarsource.scanner.api.internal.shaded.okhttp.internal.http.RealInterceptorChain.proceed(RealInterceptorChain.java:67)
at org.sonarsource.scanner.api.internal.shaded.okhttp.RealCall.getResponseWithInterceptorChain(RealCall.java:185)
at org.sonarsource.scanner.api.internal.shaded.okhttp.RealCall.execute(RealCall.java:69)
at org.sonarsource.scanner.api.internal.ServerConnection.callUrl(ServerConnection.java:113)
at org.sonarsource.scanner.api.internal.ServerConnection.downloadString(ServerConnection.java:98)
at org.sonarsource.scanner.api.internal.BootstrapIndexDownloader.getIndex(BootstrapIndexDownloader.java:39)
... 10 more
WARN: Unable to locate 'report-task.txt' in the workspace. Did the SonarScanner succedeed?
ERROR: SonarQube scanner exited with non-zero code: 1
Finished: FAILURE

DirectPipelineRunner in Dataflow to read from Local machine to Google Cloud storage

I tried running a Dataflow pipeline to read from Local machine(windows) and write to Google cloud storage using a DirectPipelineRunner. The job failed with the error below specifying FileNotFoundException(so i believe the dataflow job is unable to read my location). I am running the job from my local machine to run the GCP based template that i created. I am able to see it in the GCP Dataflow dashboard, but fails with the below error. Please help. I also tried IP or hostname of my local machine along with my local location, but faced this FileNotFoundException?
Error:
java.io.FileNotFoundException: No files matched spec: C:/data/sampleinput.txt
at org.apache.beam.sdk.io.FileSystems.maybeAdjustEmptyMatchResult(FileSystems.java:172)
at org.apache.beam.sdk.io.FileSystems.match(FileSystems.java:158)
at org.apache.beam.sdk.io.FileBasedSource.split(FileBasedSource.java:261)
at com.google.cloud.dataflow.worker.WorkerCustomSources.splitAndValidate(WorkerCustomSources.java:275)
COMMAND TO RUN THE TEMPLATE:
gcloud dataflow jobs run jobname --gcs-location gs://<somebucketname of template>/<templatename> --parameters inputFilePattern=C:/data/sampleinput.txt,outputLocation=gs://<bucketname>/output/outputfile,runner=DirectPipelineRunner
CODE:
PCollection<String> textData =pipeline.apply("Read Text Data", TextIO.read().from(options.getInputFilePattern()));
textData.apply("Write Text Data",TextIO.write().to(options.getOutputLocation()));
The gcloud dataflow jobs run command runs your job on Cloud Dataflow. That means the Dataflow workers will try to find C:/data/sampleinput.txt, which does not exist on these workers, obviously.
You can fix this by uploading sampleinput.txt to a bucket and supply the URI gs://<bucketname>/sampleinput.txt as inputFilePattern. Then the Dataflow workers will be able to find your input file and the job should succeed.

Jenkins job status failing despite running correctly

I have a jenkins jobs that executes with 'Publish over SSH'. The job connects to the remote server, transfers files and runs and ansible playbook.
The playbook runs as intended, confirmed by the logs. However at the end of the job an error is returned, failing the job. It's causing problems as it's preventing the pipeline from working correctly.
SSH: EXEC: completed after 402,593 ms
SSH: Disconnecting configuration [server] ...
ERROR: Exception when publishing, exception message [Exec exit status not zero. Status [2]]
Build step 'Send files or execute commands over SSH' changed build result to UNSTABLE
[Run Playbook] $ /bin/sh -xe /tmp/jenkins1528195779014969962.sh
+ echo Finished
Finished
Finished: UNSTABLE
Is there a setting missing to allow this to pass?
never used the 'Publish over SSH' you are referingto, but I can recommend Jenkins Ansible Plugin. I am running several playbooks in pipeline stages here successfully from labeled build slaves (have one dedicated slave that has Ansible installed) targeting Linux hosts on cloud infrastructure via SSH.
Especially in combination with the ANSI color plugin the output very readable.
If you cannot try that plugin, check whats the return code of the playbook run shell call.

sonar runner authorization error

I'm trying to send a job to sonarqube from jenkins. However I'm getting an error saying te following when I run the job
ERROR: Error during Sonar runner execution
org.sonar.runner.impl.RunnerException: Unable to execute Sonar
at org.sonar.runner.impl.BatchLauncher$1.delegateExecution(BatchLauncher.java:91)
at org.sonar.runner.impl.BatchLauncher$1.run(BatchLauncher.java:75)
at java.security.AccessController.doPrivileged(Native Method)
at org.sonar.runner.impl.BatchLauncher.doExecute(BatchLauncher.java:69)
at org.sonar.runner.impl.BatchLauncher.execute(BatchLauncher.java:50)
at org.sonar.runner.api.EmbeddedRunner.doExecute(EmbeddedRunner.java:102)
at org.sonar.runner.api.Runner.execute(Runner.java:100)
at org.sonar.runner.Main.executeTask(Main.java:70)
at org.sonar.runner.Main.execute(Main.java:59)
at org.sonar.runner.Main.main(Main.java:53)
Caused by: Not authorized. Please check the properties sonar.login and sonar.password.
The system I'm using is as follows. I have jenkins 2.7.1, SonarQube 5.6 and SonarQube Runner 2.4 is installed on my server as well.
To log in to jenkins and sonarqube I'm using a ldap that only holds ldap users, no local users exists.
In my jenkins settings I set up SonarQube as follows
the token was created in a admin account in SonarQube
When I created the job I made a new build step for a sonarqube scanner. Inside my project I created a sonar-project.properties file that holds the following information
sonar.projectKey=org.sonarqube:php-ut-sq-scanner
sonar.projectName=vanbenthem :: PHPUnit :: SonarQube Scanner
sonar.projectVersion=1.0
sonar.sources=src
sonar.tests=tests
sonar.language=php
sonar.sourceEncoding=UTF-8
# Reusing PHPUnit reports
sonar.php.coverage.reportPath=reports/phpunit.coverage.xml
sonar.php.tests.reportPath=reports/phpunit.xml
Can anyone tell me what I'm doing wrong? To my understanding I shouldn't be giving a login and password since SonarQube 5.4
Hope someone will be able to help me out
per nicloas' request
Here are the console output from jenkins and my sonar settings regarding my ldap.
Well, I faced the same issue with Bitbucket pipelines and Sonarqube. It turned out that the Sonarqube authentication token I was using was incorrect. It is configured as SONAR_TOKEN in Bitbucket's repository variables and correcting it solved the issue.
Please make sure to double check the Server authentication token you are entering here.
Make sure the token provided by Sonarqube and stored in Jenkins is correct

Resources