CDK and batch build - aws-cdk

I am trying to have cypress tests run in parallel in codepipeline/codebuild as documented here: https://docs.cypress.io/guides/continuous-integration/aws-codebuild#Parallelizing-the-build
I am reading aws docs here:
https://docs.aws.amazon.com/codebuild/latest/userguide/batch-build-buildspec.html#build-spec.batch.build-list
This is what I have got so far:
import * as cdk from '#aws-cdk/core';
import * as codebuild from '#aws-cdk/aws-codebuild';
import * as iam from '#aws-cdk/aws-iam';
import { defaultEnvironment, NODE_JS_VERSION } from './environments/base-environment';
import { projectEnvironmentVars } from './environments/e2e-tests-project-environment';
// import { createReportGroupJsonObject } from '../../utils/utils';
// import { Duration } from '#aws-cdk/core'
interface RunTestsProjectProps {
testsBucketName: string,
testsBucketArn: string,
targetEnv: string,
repoType: string,
role: iam.Role,
codeCovTokenArn: string,
}
export class RunTestsProject extends codebuild.PipelineProject {
constructor(scope: cdk.Construct, id: string, props: RunTestsProjectProps) {
const { testsBucketName, testsBucketArn, targetEnv, repoType, codeCovTokenArn } = props
super(scope, id, {
projectName: id,
role: props.role,
environment: defaultEnvironment,
environmentVariables: projectEnvironmentVars({ testsBucketName, testsBucketArn, targetEnv, repoType, codeCovTokenArn }),
timeout: cdk.Duration.hours(3),
buildSpec: codebuild.BuildSpec.fromObject({
version: '0.2',
phases: {
install: {
'runtime-versions': {
nodejs: NODE_JS_VERSION
}
},
build: {
commands: [
'if [ ! -f "${CODEBUILD_SRC_DIR}/scripts/assume-cross-account-role.env" ]; then echo "assume-cross-account-this.role.env not found in repo" && aws s3 cp s3://${ARTIFACTS_BUCKET_NAME}/admin/cross-account/assume-cross-account-role.env ${CODEBUILD_SRC_DIR}/scripts/; else echo "Overriding assume-cross-account-role.env from repo"; fi',
'. ${CODEBUILD_SRC_DIR}/scripts/assume-cross-account-role.env',
'bash ${CODEBUILD_SRC_DIR}/scripts/final-tests.sh'
],
batch: {
'fail-fast': false,
'build-list': []
}
},
},
artifacts: {
files: '**/*'
},
})
});
}
}
what should I add in the build-list part to have multiple builds running
I tried
'build-list': { identifier: 'build1', identifier: 'build2' }
but this looks like incorrect syntax.
The number of builds should ideally be based on the cypress grouping. Can it be dynamic or has to be defined statically?

Related

CDK GraphqlApi with Function Using Typescript Causes Undefined or Not Exported

I have a aws_appsync.GraphqlApi with a lambda resolver:
import * as cdk from 'aws-cdk-lib';
import {aws_appsync} from 'aws-cdk-lib';
import {Construct} from 'constructs';
import {AttributeType, BillingMode, Table} from "aws-cdk-lib/aws-dynamodb";
import * as path from "path";
import {FieldLogLevel} from "aws-cdk-lib/aws-appsync";
import {RetentionDays} from "aws-cdk-lib/aws-logs";
import {Code, Function, Runtime} from "aws-cdk-lib/aws-lambda";
export class RelmStack extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
const relmTable = new Table(this, 'relm', {
billingMode: BillingMode.PAY_PER_REQUEST,
tableName: 'relm',
partitionKey: {
name: 'pk',
type: AttributeType.STRING
}, sortKey: {
name: 'sk',
type: AttributeType.STRING
}
})
const api = new aws_appsync.GraphqlApi(this, 'relm-api', {
name: 'relm-api',
logConfig: {
fieldLogLevel: FieldLogLevel.ALL,
excludeVerboseContent: true,
retention: RetentionDays.TWO_WEEKS
},
schema: aws_appsync.SchemaFile.fromAsset(path.join(__dirname, 'schema.graphql')),
authorizationConfig: {
defaultAuthorization: {
authorizationType: aws_appsync.AuthorizationType.API_KEY,
apiKeyConfig: {
name: 'relm-api-key'
}
}
}
})
const createLambda = new Function(this, 'dialog-create', {
functionName: 'dialog-create',
runtime: Runtime.NODEJS_14_X,
handler: 'index.handler',
code: Code.fromAsset('src/lambda'),
memorySize: 3008,
})
const createDataSource = api.addLambdaDataSource('create-data-source', createLambda)
createDataSource.createResolver('create-resolver', {
typeName: 'Mutation',
fieldName: 'dialogCreate'
});
relmTable.grantWriteData(createLambda);
}
}
The sources lives under src/lambda/index.ts and the code is as follows:
exports.handler = async (event: any) => {
console.log('event: ', event)
};
Super simple. When the file extension is index.js everything works. When I change it to index.ts I get an error:
"index.handler is undefined or not exported"
I've looked at many examples on how to do this and all of them seem to be using the ts extension with no problems. What am I doing wrong here?
You should use the NodejsFunction which includes transpiling TypeScript to JavaScript.
https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_lambda_nodejs-readme.html
You can write your handler in typescript but you'll have to invoke a build script to transpile it into javascript to serve as your lambda handler.
This project uses cdk and tsc:
https://github.com/mavi888/cdk-typescript-lambda/blob/main/package.json
This line discusses using esbuild to transpile:
https://docs.aws.amazon.com/lambda/latest/dg/typescript-package.html

Pipeline stack which uses cross-environment actions must have an explicitly set region

I am trying to create an AWS Code Pipeline to create a DynamoDB table in one of the stages. I was able to successfully deploy the same code with CDK v1. now trying to replicate the same on CDK v2. I am getting an error Pipeline stack which uses cross-environment actions must have an explicitly set region
Here's the complete code:
import { Stack, StackProps, Stage, StageProps } from "aws-cdk-lib";
import { AttributeType, Table } from "aws-cdk-lib/aws-dynamodb";
import {
CodePipeline,
CodePipelineSource,
ShellStep,
} from "aws-cdk-lib/pipelines";
import { Construct } from "constructs";
export class DdbStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
new Table(this, "TestTable", {
partitionKey: { name: "id", type: AttributeType.STRING },
});
}
}
class MyApplication extends Stage {
constructor(scope: Construct, id: string, props?: StageProps) {
super(scope, id, props);
new DdbStack(this, `${id}-ddb`, {});
}
}
export class PipelineStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const pipeline = new CodePipeline(this, `${id}-PipelineStack-`, {
crossAccountKeys: true,
selfMutation: false,
pipelineName: "MangokulfiCDK",
synth: new ShellStep("Synth", {
input: CodePipelineSource.connection("gowtham91m/mango-cdk", "main", {
connectionArn:
"arn:aws:codestar-connections:us-west-2:147866640792:connection/4b18bea2-9eb6-47b1-bbdc-adb3bf6fd2a9",
}),
commands: ["npm ci", "npm run build", "npx cdk synth"],
}),
});
pipeline.addStage(
new MyApplication(this, `Staging`, {
env: {
account: "123456789123",
region: "us-west-2",
},
})
);
pipeline.buildPipeline();
}
}

How add rollupNodePolyFill to electron.vite.configs

I need excute 'twilio-client' on electron project
import Twilio from 'twilio-client';
// this line broken when run electron-vite preview or builded app version
const device = new Twilio.Device();
device.setup('xyls', {
debug: true
});
console.log(device);
When I run my app with electron-vite preview or after build, I got an error:
Uncaught TypeError: Failed to resolve module specifier "events"
This also happened when it was executed:
electron-vite dev --watch
I added nodePolifill plugins now it works in dev mode, but on preview or build doesn't
The plugin 'rollup-plugin-node-polyfills' doesn't works on build.rollupOptions.plugins
I need help
My electron.vite.configs.ts:
import { resolve, normalize, dirname } from 'path'
import { defineConfig } from 'electron-vite'
import injectProcessEnvPlugin from 'rollup-plugin-inject-process-env'
import tsconfigPathsPlugin from 'vite-tsconfig-paths'
import reactPlugin from '#vitejs/plugin-react'
import { NodeGlobalsPolyfillPlugin } from '#esbuild-plugins/node-globals-polyfill'
import { NodeModulesPolyfillPlugin } from '#esbuild-plugins/node-modules-polyfill'
import rollupNodePolyFill from "rollup-plugin-node-polyfills";
import { main, resources } from './package.json'
const [nodeModules, devFolder] = normalize(dirname(main)).split(/\/|\\/g)
const devPath = [nodeModules, devFolder].join('/')
const tsconfigPaths = tsconfigPathsPlugin({
projects: [resolve('tsconfig.json')],
})
export default defineConfig({
main: {
plugins: [tsconfigPaths],
build: {
rollupOptions: {
input: {
index: resolve('src/main/index.ts'),
},
output: {
dir: resolve(devPath, 'main'),
},
},
},
},
preload: {
plugins: [tsconfigPaths],
build: {
outDir: resolve(devPath, 'preload'),
},
},
renderer: {
define: {
'process.env.NODE_ENV': JSON.stringify(process.env.NODE_ENV),
'process.platform': JSON.stringify(process.platform),
},
server: {
port: 4927,
},
publicDir: resolve(resources, 'public'),
plugins: [
tsconfigPaths,
reactPlugin(),
],
resolve: {
alias: {
util: 'rollup-plugin-node-polyfills/polyfills/util',
sys: 'util',
stream: 'rollup-plugin-node-polyfills/polyfills/stream',
path: 'rollup-plugin-node-polyfills/polyfills/path',
querystring: 'rollup-plugin-node-polyfills/polyfills/qs',
punycode: 'rollup-plugin-node-polyfills/polyfills/punycode',
url: 'rollup-plugin-node-polyfills/polyfills/url',
string_decoder: 'rollup-plugin-node-polyfills/polyfills/string-decoder',
http: 'rollup-plugin-node-polyfills/polyfills/http',
https: 'rollup-plugin-node-polyfills/polyfills/http',
os: 'rollup-plugin-node-polyfills/polyfills/os',
assert: 'rollup-plugin-node-polyfills/polyfills/assert',
constants: 'rollup-plugin-node-polyfills/polyfills/constants',
_stream_duplex: 'rollup-plugin-node-polyfills/polyfills/readable-stream/duplex',
_stream_passthrough: 'rollup-plugin-node-polyfills/polyfills/readable-stream/passthrough',
_stream_readable: 'rollup-plugin-node-polyfills/polyfills/readable-stream/readable',
_stream_writable: 'rollup-plugin-node-polyfills/polyfills/readable-stream/writable',
_stream_transform: 'rollup-plugin-node-polyfills/polyfills/readable-stream/transform',
timers: 'rollup-plugin-node-polyfills/polyfills/timers',
console: 'rollup-plugin-node-polyfills/polyfills/console',
vm: 'rollup-plugin-node-polyfills/polyfills/vm',
zlib: 'rollup-plugin-node-polyfills/polyfills/zlib',
tty: 'rollup-plugin-node-polyfills/polyfills/tty',
domain: 'rollup-plugin-node-polyfills/polyfills/domain',
events: 'rollup-plugin-node-polyfills/polyfills/events',
buffer: 'rollup-plugin-node-polyfills/polyfills/buffer-es6', // add buffer
process: 'rollup-plugin-node-polyfills/polyfills/process-es6', // add process
}
},
optimizeDeps: {
esbuildOptions: {
// Node.js global to browser globalThis
define: {
global: 'globalThis'
},
// Enable esbuild polyfill plugins
plugins: [
NodeGlobalsPolyfillPlugin({
process: true,
buffer: true,
}),
NodeModulesPolyfillPlugin(),
]
}
},
worker: {
format: "es",
},
build: {
outDir: resolve(devPath, 'renderer'),
rollupOptions: {
plugins: [
injectProcessEnvPlugin({
NODE_ENV: 'production',
platform: process.platform,
}),
rollupNodePolyFill() //this line doesn't works
],
input: {
index: resolve('src/renderer/index.html'),
},
output: {
format: "esm",
dir: resolve(devPath, 'renderer'),
},
},
},
},
})
and the is my project repostitory:
https://github.com/caioregatieri/app-electron
I tried add many polyfills, add twilio import on preload and export to render with
contextBridge.exposeInMainWorld('Twilio', Twilio)
I solved it with:
electron.vite.config.ts
build: {
outDir: resolve(devPath, 'renderer'),
rollupOptions: {
plugins: [
injectProcessEnvPlugin({
NODE_ENV: 'production',
platform: process.platform,
}),
rollupNodePolyFill,
],
},
},
and this:
index.html
<script>
global = globalThis;
</script>

esbuild How to build a product without a filesystem

I want esbuild to automatically handle the duplicate imports in the file for me, and I also want to use its tree shaking capabilities to help me optimise my code and then output it to a file, I wrote the following code to try and do the above, but I could never get it right
export interface ConfigSource extends Partial<Options> {
code: string
name: string
loader: Loader
}
export interface Config {
sources: ConfigSource[]
options?: BuildOptions
}
function babelBuildWithBundle(main: ConfigSource, config: Config) {
const buildModuleRuntime: Plugin = {
name: 'buildModuleRuntime',
setup(build) {
build.onResolve({ filter: /\.\// }, (args) => {
return { path: args.path, namespace: 'localModule' }
})
build.onLoad({ filter: /\.\//, namespace: 'localModule' }, (args) => {
const source = config.sources.find(
(source) =>
source.name.replace(/\..+$/, '') === args.path.replace(/^\.\//, '')
)
const content = source?.code || ''
return {
contents: content,
loader: source?.loader || 'js',
}
})
},
}
return build(
{
stdin: {
contents: main.code,
loader: main.loader,
sourcefile: main.name,
resolveDir: path.resolve('.'),
},
bundle: true,
write: false,
format: 'esm',
outdir: 'dist',
plugins: [buildModuleRuntime],
}
)
}
const foo = `
export const Foo = FC(() => {
return <div>gyron</div>
})
`
const app = `
import { Foo } from './foo'
function App(): any {
console.log(B)
return <Foo />
}
`
const bundle = await babelBuildWithBundle(
{
loader: 'tsx',
code: app,
name: 'app.tsx',
},
{
sources: [
{
loader: 'tsx',
code: foo,
name: 'foo.tsx',
},
],
}
)
Then the only answer I got in the final output was
console.log(bundle.outputFiles[0].text)
`
// localModule:. /foo
var Foo = FC(() => {
return /* #__PURE__ */ React.createElement("div", null, "gyron");
});
`
console.log(bundle.outputFiles[1])
`
undefined
`
I have just tried setting treeShaking to false and can pack the app into the product.

Adding a stage to CodePipeline throws errors

I am creating a code pipeline as follows -
import * as cdk from "aws-cdk-lib";
import { CodeBuildStep, CodePipeline, CodePipelineSource } from "aws-cdk-lib/pipelines";
...
export class Pipeline extends cdk.Stack {
constructor(scope: Construct, id: string, props?: cdk.StackProps) {
super(scope, id, props);
...
const pipeline = new CodePipeline(this, "Pipeline", {
pipelineName: "pipeline",
synth: new CodeBuildStep("SynthStep", {
input: CodePipelineSource.codeCommit(repo, "mainline"),
buildEnvironment: {
computeType: CodeBuild.ComputeType.MEDIUM,
buildImage: CodeBuild.LinuxBuildImage.STANDARD_5_0,
},
partialBuildSpec: buildSpec,
commands: [],
role: codeBuildSynthRole,
primaryOutputDirectory: "cdk/cdk.out",
}),
crossAccountKeys: true,
selfMutation: true,
dockerEnabledForSelfMutation: true,
});
const review = new ReviewPipelineStage(this, "Review", {
sourceFileSet: pipeline.cloudAssemblyFileSet,
});
const reviewStage = pipeline.addStage(review);
const gitleaksReviewAction = new GitleaksReviewAction(
this,
"GitleaksReviewAction",
{
sourceFileSet: pipeline.cloudAssemblyFileSet,
}
);
reviewStage.addPost(gitleaksReviewAction.action);
gitleaksReviewAction.gitleaksImage.repository.grantPull(
gitleaksReviewAction.action.project
);
}
}
I am trying to add a stage for Gitleaks and the GitleaksReviewAction construct is as follows -
export interface GitleaksReviewActionProps {
sourceFileSet: FileSet;
}
export class GitleaksReviewAction extends Construct {
public readonly action: CodeBuildStep;
public readonly gitleaksImage: DockerImageAsset;
constructor(scope: Construct, id: string, props: GitleaksReviewActionProps) {
super(scope, id);
this.gitleaksImage = new DockerImageAsset(this, "gitleaksDockerAsset", {
directory: path.join(__dirname, "../assets/gitleaks"),
});
this.action = new CodeBuildStep("Gitleaks", {
input: props.sourceFileSet,
commands: [
"find . -type d -exec chmod 777 {} \\;",
"find . -type f -exec chmod 666 {} \\;",
`aws ecr get-login-password --region $AWS_REGION | docker login -u AWS --password-stdin ${this.gitleaksImage.imageUri}`,
`docker run -v $(pwd):/repo ${this.gitleaksImage.imageUri} --path=/repo --repo-config-path=config/gitleaks/gitleaks.toml --verbose`,
],
buildEnvironment: {
buildImage: codebuild.LinuxBuildImage.STANDARD_5_0,
privileged: true,
},
});
}
}
The ReviewPipelineStage is as follows -
export interface ReviewPipelineStageProps extends StageProps {
sourceFileSet: FileSet;
}
export class ReviewPipelineStage extends Stage {
constructor(scope: Construct, id: string, props: ReviewPipelineStageProps) {
super(scope, id, props);
new GitleaksReviewAction(this, "GitleaksReviewAction", {
sourceFileSet: props.sourceFileSet,
});
}
}
As i do a cdk synth i get an error -
throw new Error(`${construct.constructor?.name ?? 'Construct'} at '${Node.of(construct).path}' should be created in the scope of a Stack, but no Stack found`);
I am not sure if i should be using the other construct aws_codepipeline to define the stages or is this the right way to create a stage. Any ideas?
The issue is that you're creating a Construct in the scope of a Stage. You can't do that, you can only create Stacks in the scope of a Stage. Constructs have to be created in the scope of a Stack.
The issue is here:
export class ReviewPipelineStage extends Stage {
constructor(scope: Construct, id: string, props: ReviewPipelineStageProps) {
super(scope, id, props);
new GitleaksReviewAction(this, "GitleaksReviewAction", {
sourceFileSet: props.sourceFileSet,
});
}
}
this is a Stage, not a Stack.
To fix, this, you have to create a Stack in your stage, and create your Constructs in there.

Resources