aws_cdk events rule target for cdk pipelines fails - aws-cdk

below error pops when I try to target a CDK pipeline using events targets.
jsii.errors.JavaScriptError:
Error: Resolution error: Supplied properties not correct for "CfnRuleProps"
targets: element 0: supplied properties not correct for "TargetProperty"
arn: required but missing.
code is below
from aws_cdk import (
core,
aws_codecommit as codecommit,
aws_codepipeline as codepipeline,
aws_events as events,
aws_events_targets as targets
)
from aws_cdk import pipelines
from aws_cdk.pipelines import CodePipeline, CodePipelineSource, ShellStep
class BootStrappingStack(core.Stack):
def __init__(self, scope: core.Construct, construct_id: str, **kwargs) -> None:
super().__init__(scope, construct_id, **kwargs
repo = codecommit.Repository(
self, 'Repository',
repository_name='Repository'
)
source_artifact = codepipeline.Artifact()
cloud_assembly_artifact = codepipeline.Artifact()
pipeline = CodePipeline(self, 'Pipeline',
synth=ShellStep("Synth",
input=CodePipelineSource.code_commit(
repository=repo,
branch='master'),
commands=[
'pip install -r requirements',
'npm install -g aws-cdk',
'cdk synth']
)
)
rule = events.Rule(self, 'TriggerPipeline',
schedule=events.Schedule.expression('rate(1 hour)')
)
rule.add_target(targets.CodePipeline(pipeline))
The documentation for aws_cdk.events_targets works with codepipeline construct, however doesn't work as documented for cdk pipelines.
This needs to be addressed in the documentation, once I get to know what's the fix. Please help.

As mentioned by #Otavio, you need to use the codepipeline.IPipeline.
You can use the pipeline property from CDK CodePipeline construct but in order to use that first you need to construct the pipeline using build_pipeline() method:
pipeline = CodePipeline(self, 'Pipeline',
synth=ShellStep("Synth",
input=CodePipelineSource.code_commit(
repository=repo,
branch='master'),
commands=[
'pip install -r requirements',
'npm install -g aws-cdk',
'cdk synth']
)
)
# You need to construct the pipeline before passing it as a target in rule
pipeline.build_pipeline()
rule = events.Rule(self, 'TriggerPipeline',
schedule=events.Schedule.expression('rate(1 hour)')
)
# Using the pipeline property from CDK Codepipeline
rule.add_target(targets.CodePipeline(pipeline.pipeline))

The problem is that targets.CodePipeline receives a codepipeline.IPipeline as a parameter. But what you are using instead is a pipelines.CodePipeline, which is a different thing. CodePipeline is more abstract construct, built on top of the codepipeline module.

You can try this:
const pipeline = new CodePipeline(self, 'Pipeline' ....
Then:
rule.addTarget(new targets.CodePipeline(pipeline))

Related

Add Jenkins Shared Libraries with classes reference in a Jenkinsfile dynamically

I have constructed a Shared Library in which I want to have class definitions, below is the example shared lib code.
package api
class api implements Serializable {
private final Map data
private final DSL steps
api(Map data, DSL steps) {
this.data = data
this.steps = steps
}
}
I have placed this class definition in a shared library git repo, with the path api/src/api/api.groovy. I am importing the shared library using the following step at the top of my jenkinsfile, outside any node or pipeline directive:
library identifier: 'api#master', retriever: modernSCM(
scm: [
$class: 'GitSCMSource',
credentialsId: '<my-credentials-id>',
remote: '<my-git-remote>',
traits: [gitBranchDiscovery()]
],
libraryPath: 'api/'
)
import api.*
When I use the above class from the shared library it's throwing an exception saying that the class is not found.
WorkflowScript: 75: unable to resolve class api
# line 75, column 29.
api foo = new api(env.data, steps)
^
at org.codehaus.groovy.control.ErrorCollector.failIfErrors(ErrorCollector.java:309)
at org.codehaus.groovy.control.CompilationUnit.applyToSourceUnits(CompilationUnit.java:981)
at org.codehaus.groovy.control.CompilationUnit.doPhaseOperation(CompilationUnit.java:626)
at org.codehaus.groovy.control.CompilationUnit.compile(CompilationUnit.java:575)
at groovy.lang.GroovyClassLoader.doParseClass(GroovyClassLoader.java:323)
at groovy.lang.GroovyClassLoader.parseClass(GroovyClassLoader.java:293)
at org.jenkinsci.plugins.scriptsecurity.sandbox.groovy.GroovySandbox$Scope.parse(GroovySandbox.java:163)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.doParse(CpsGroovyShell.java:142)
at org.jenkinsci.plugins.workflow.cps.CpsGroovyShell.reparse(CpsGroovyShell.java:127)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.parseScript(CpsFlowExecution.java:562)
at org.jenkinsci.plugins.workflow.cps.CpsFlowExecution.start(CpsFlowExecution.java:514)
at org.jenkinsci.plugins.workflow.job.WorkflowRun.run(WorkflowRun.java:336)
at hudson.model.ResourceController.execute(ResourceController.java:107)
at hudson.model.Executor.run(Executor.java:449)
Finished: FAILURE
As i read in the doccument that loading library dynamically does not import classes directly to the pipeline's classpath.
So need a little help to get this class into my job execution.
Note: I have tested it and it's working when adding the global libraries under jenkins global settings, however i want this library setting to be dynamic from jenkins file.

How to configure tests stage in the codepipeline to use BatchBuild from Codebuild and create multiple instances

I am using CDK based code pipeline from https://github.com/awslabs/aws-simple-cicd . I would like to enable batchbuild in the test stage as per https://docs.cypress.io/guides/continuous-integration/aws-codebuild#Using-the-Cypress-Dashboard-with-AWS-CodeBuild
How can batchbuild be enabled in CDK codepipeline to have tests run in parallel?
As far as I see BatchBuild can be enabled in the CodeBuildAction:
const testAction = new CodeBuildAction ({
actionName: 'Test',
outputs: [testOutputArtifact],
input: buildOutputArtifact,
project: testProject,
executeBatchBuild: true
})
The docs states:
/**
* Trigger a batch build.
*
* Enabling this will enable batch builds on the CodeBuild project.
*
* #default false
*/
readonly executeBatchBuild?: boolean;
If You have any commands to run then it is possible to create pre/post-StageSteps.
I would also recommend You to use the simple aws-cdk-lib.pipelines module instead of aws-cdk-lib.aws_codepipeline module.

Specifying a GitHub project URL in a Jenkins Multibranch pipeline

I'm trying to specify the GithubProjectProperty in a Jenkins Multibranch pipeline. I've been unsuccessful attempting to set an entry in the option block to control this value.
The pipeline syntax snippet generator suggests:
properties([
$class: 'GithubProjectProperty',
displayName: '',
projectUrlStr: 'https://myServer/myOrg/myRepo'
])
None of the following appear to work:
1) Try to put the properties directly in the options block
options {
// Set the URL for the GitHub project option
properties([
$class: 'GithubProjectProperty',
displayName: '',
projectUrlStr: 'https://myServer/myOrg/myRepo'
])
}
ERROR: The ‘properties’ section has been renamed as of version 0.8. Use ‘options’ instead
2) Remove the properties keyword but leave the option in the options block
options {
// Set the URL for the GitHub project option
[
$class: 'GithubProjectProperty',
displayName: '',
projectUrlStr: 'https://myServer/myOrg/myRepo'
]
}
ERROR: Options cannot be defined as maps
3) Treat the GitHubProjectProperty as if it can be instantiated (like office365ConnectorWebhooks)
options {
// Set the URL for the GitHub project option
GithubProjectProperty('https://myServer/myOrg/myRepo')
}
ERROR: Invalid option type "GithubProjectProperty". Valid option types: [authorizationMatrix, buildDiscarder, catchError, checkoutToSubdirectory, disableConcurrentBuilds, disableResume, durabilityHint, newContainerPerStage, office365ConnectorWebhooks, overrideIndexTriggers, parallelsAlwaysFailFast, preserveStashes, quietPeriod, rateLimitBuilds, retry, script, skipDefaultCheckout, skipStagesAfterUnstable, timeout, waitUntil, warnError, withContext, withCredentials, withEnv, ws]
4) Treat the GitHubProjectProperty as if it can be instantiated but inside a script block (because script is supposed to be valid according to attempt #3)
options {
script {
// Set the URL for the GitHub project option
GithubProjectProperty('https://myServer/myOrg/myRepo')
}
}
ERROR: Options definitions cannot have blocks
The office-365-connector-plugin is a working plugin that is supported in the options block of a Jenkinsfile. I compared its code with the github-plugin source on GitHub and noticed the following line:
#Extension
public static final class DescriptorImpl extends JobPropertyDescriptor {
The code is lacking a #Symbol directive that the office365ConnectorWebhooks appears to provide in its code:
#Extension
#Symbol("office365ConnectorWebhooks")
public final class WebhookJobPropertyDescriptor extends JobPropertyDescriptor {
Is there some special syntax to use to add the GitHub URL to a multibranch pipeline or does that plugin just not support managing it through a Jenkinsfile?
The ability to specify options in a pipeline using a Jenkinsfile requires a Symbol. There is a proposed fix in the Jenkins github-plugin that adds the necessary Symbol directive but it is not currently part of the plugin as of version 1.30.0.
See: https://issues.jenkins-ci.org/browse/JENKINS-62339
It's possible for a developer to build their own updated plugin in the meantime by updating the following file:
src/main/java/com/coravy/hudson/plugins/github/GithubProjectProperty.java in the source found at: https://github.com/jenkinsci/github-plugin
Add the Symbol:
import org.jenkinsci.Symbol;
...
#Extension
#Symbol("githubProjectProperty")
public static final class DescriptorImpl extends JobPropertyDescriptor {
...
And for good measure, make sure the code specifies the correct function signature for newInstance:
#Override
public JobProperty<?> newInstance(#Nonnull StaplerRequest req, JSONObject formData)
throws Descriptor.FormException {
The updated plugin may be installed by a Jenkins admin using the Advanced option in the Plugin Manager to upload an .hpi file from outside the central plugin repository
I had the same problem and was able to solve it with:
options{
githubProjectProperty(displayName: '', projectUrlStr: 'your_project_url')
}

Unable to run shell command in Groovy in Jenkins

I am trying to get certain values from the slave by running shell commands such as :
git rev-parse HEAD
git config --get remote.origin.url
The method that I have tried to write for this is :
def executeCommand(String command) {
stdout = sh script: command, returnStdout: true
return stdout.trim()
}
Now when I try to run the first command :
output = executeCommand('git rev-parse HEAD')
I get the ERROR :
[Running] groovy "/Users/user-a/Documents/cmd.groovy"
Caught: groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
groovy.lang.MissingMethodException: No signature of method: cmd.sh() is applicable for argument types: (LinkedHashMap) values: [[script:git rev-parse HEAD, returnStdout:true]]
Possible solutions: is(java.lang.Object), use([Ljava.lang.Object;), run(), run(), any(), tap(groovy.lang.Closure)
at cmd.executeCommand(cmd.groovy:2)
at cmd.run(cmd.groovy:6)
I also tried:
output = command.execute().text
But this returns nothing.
Im running out of ideas on how to run shell commands in Groovy in Jenkins and record the output.
MORE DETAILS
I am working with Jenkins shared Libraries. I have exposed a method in for my Jenkinsfile by the name getLatestBuildDetails(). This method is defined within my library. One of the actions within the method is to execute the git commands locally. So inorder to run any shell command locally, I have created the executeCommand function which takes the actual command to run as a String and executes it and returns the output to be used later by getLatestBuildDetails()
Library classes cannot directly call steps such as sh or git. They can however implement methods, outside of the scope of an enclosing class, which in turn invoke Pipeline steps, for example:
// src/org/foo/Zot.groovy
package org.foo;
def checkOutFrom(repo) {
git url: "git#github.com:jenkinsci/${repo}"
}
return this
Which can then be called from a Scripted Pipeline:
def z = new org.foo.Zot()
z.checkOutFrom(repo)
This approach has limitations; for example, it prevents the declaration of a superclass.
Alternately, a set of steps can be passed explicitly using this to a library class, in a constructor, or just one method:
package org.foo
class Utilities implements Serializable {
def steps
Utilities(steps) {this.steps = steps}
def mvn(args) {
steps.sh "${steps.tool 'Maven'}/bin/mvn -o ${args}"
}
}
When saving state on classes, such as above, the class must implement the Serializable interface. This ensures that a Pipeline using the class, as seen in the example below, can properly suspend and resume in Jenkins.
#Library('utils') import org.foo.Utilities
def utils = new Utilities(this)
node {
utils.mvn 'clean package'
}
If the library needs to access global variables, such as env, those should be explicitly passed into the library classes, or methods, in a similar manner.
Instead of passing numerous variables from the Scripted Pipeline into a library,
package org.foo
class Utilities {
static def mvn(script, args) {
script.sh "${script.tool 'Maven'}/bin/mvn -s ${script.env.HOME}/jenkins.xml -o ${args}"
}
}
The above example shows the script being passed in to one static method, invoked from a Scripted Pipeline as follows:
#Library('utils') import static org.foo.Utilities.*
node {
mvn this, 'clean package'
}
In your case you should write something like:
def getLatestBuildDetails(context){
//...
executeCommand(context, 'git rev-parse HEAD')
//...
}
def executeCommand(context, String command) {
stdout = script.sh(script: command, returnStdout: true)
return stdout.trim()
}
Jenkins file:
#Library('library_name') _
getLatestBuildDetails(this)
For more info see jenkins shared library documentation: https://jenkins.io/doc/book/pipeline/shared-libraries/
I am also using shared libraries. This is how I have used in my code:
String getMavenProjectName() {
echo "inside getMavenProjectName +++++++"
// mavenChartName = sh(
// script: "git config --get remote.origin.url",
// returnStdout: true
// ).trim()
def mavenChartName = sh returnStdout:true, script: '''
#!/bin/bash
GIT_LOG=$(env -i git config --get remote.origin.url)
basename "$GIT_LOG" .git; '''
echo "mavenChartName: ${mavenChartName}"
return mavenChartName
}
PS: Ignore the commented lines of code.
Try out the sh step instead of execute. :)
EDIT:
I would go with execute() or which I think it is even better, grgit.
I think you are not getting any output when you run cmd.execute().text because .text returns the standard output of the command and your command might only use the standard error as its output, you can check both:
def process = cmd.execute()
def stdOut = process.inputStream.text
def stdErr = process.errorStream.text

How to set environment variables when testing DSL scripts against dummy jenkins?

I am trying to automate testing Jenkins groovy dsl scripts, like here:
https://github.com/sheehan/job-dsl-gradle-example
The idea I think is very straight forward, what I'm having issues with is setting environment variables for the dummy Jenkins. I followed the instructions here:
https://wiki.jenkins-ci.org/display/JENKINS/Unit+Test
Specifically "How to set env variables" section and added the following to my test executor:
import hudson.slaves.EnvironmentVariablesNodeProperty
import hudson.EnvVars
/**
* Tests that all dsl scripts in the jobs directory will compile.
*/
class JobScriptsSpec extends Specification {
#Shared
#ClassRule
JenkinsRule jenkinsRule = new JenkinsRule()
EnvironmentVariablesNodeProperty prop = new EnvironmentVariablesNodeProperty();
EnvVars envVars = prop.getEnvVars();
#Unroll
void 'test script #file.name'(File file) {
given:
envVars.put("ENVS", "dev19");
jenkinsRule.jenkins.getGlobalNodeProperties().add(prop);
JobManagement jm = new JenkinsJobManagement(System.out, [:], new File('.'))
when:
new DslScriptLoader(jm).runScript(file.text)
then:
noExceptionThrown()
where:
file << jobFiles
}
However when I run the actual tests for one of the scripts, I still see the following:
Failed tests
test script Build.groovy
Expected no exception to be thrown, but got 'javaposse.jobdsl.dsl.DslScriptException'
at spock.lang.Specification.noExceptionThrown(Specification.java:118)
at com.dslexample.JobScriptsSpec.test script #file.name(JobScriptsSpec.groovy:40)
Caused by: javaposse.jobdsl.dsl.DslScriptException: (script, line 3) No such property: ENVS for class: script
The script Build.groovy uses the variable "${ENVS}" (as if it were provided by parameter in seed job of Jenkins), which works as expected when actually running in Jenkins... So any way to set these "parameters" or env variables in the test jenkins context?
Example of how I use the ENVS variable in the Build.groovy:
def envs = '-'
"${ENVS}".eachLine{
def env = it
envs+=env+'-'
}
envs.substring(0,envs.length()-1)
job('Build'+envs) {
...
}
The second argument of the JenkinsJobManagement constructor is a map of environment variables which will be available in the DSL scripts.
Map<String, String> envVars = [
FOO: 'BAR'
]
JobManagement jm = new JenkinsJobManagement(System.out, envVars, new File('.'))

Resources