cdk watch command forces full deploy with unrelated error on file change - aws-cdk

I'm developing a little CDKv2 script to instantiate a few AWS services.
I have some lambda code deployed in the lambda/ folder and the frontend stored in a bucket populated using the frontend/ folder in the source.
I've noticed that whenever I make a change to any of the file inside these two, cdk watch return the following error and falls back to perform a full redeploy (which is significantly slow).
Could not perform a hotswap deployment, because the CloudFormation template could not be resolved: Parameter or resource 'DomainPrefix' could not be found for evaluation
Falling back to doing a full deployment
Is there any way to make changes in these folders only trigger updating the related bucket content or the related lambda?
Following here the stack.ts for quick reference, just in case here you can take a look at the repo.
export class CdkAuthWebappStack extends Stack {
constructor(scope: Construct, id: string, props?: StackProps) {
super(scope, id, props);
const domainPrefixParam = new CfnParameter(this, 'DomainPrefix', {
type: 'String',
description: 'You have to set it in google cloud as well', //(TODO: add link to explain properly)
default: process.env.DOMAIN_NAME || ''
})
const googleClientIdParam = new CfnParameter(this, 'GoogleClientId', {
type: 'String',
description: 'From google project',
noEcho: true,
default: process.env.GOOGLE_CLIENT_ID || ''
})
const googleClientSecretParam = new CfnParameter(this, 'GoogleClientSecret', {
type: 'String',
description: 'From google project',
noEcho: true,
default: process.env.GOOGLE_CLIENT_SECRET || ''
})
if(!domainPrefixParam.value || !googleClientIdParam.value || !googleClientSecretParam.value){
throw new Error('Make sure you initialized DomainPrefix, GoogleClientId and GoogleClientSecret in the stack parameters')
}
const s3frontend = new s3.Bucket(this, 'Bucket', {
bucketName: domainPrefixParam.valueAsString+'-frontend-bucket',
blockPublicAccess: s3.BlockPublicAccess.BLOCK_ALL,
encryption: s3.BucketEncryption.S3_MANAGED,
enforceSSL: true,
versioned: false,
removalPolicy: cdk.RemovalPolicy.DESTROY,
autoDeleteObjects: true,
websiteIndexDocument: "index.html",
});
//TODO: fare in modo che questa origin access identity non sia legacy quando deployo
const cfdistributionoriginaccessidentity = new cloudfront.OriginAccessIdentity(this, 'CFOriginAccessIdentity', {
comment: "Used to give bucket read to cloudfront"
})
const cfdistribution = new cloudfront.CloudFrontWebDistribution(this, 'CFDistributionFrontend', {
originConfigs: [
{
s3OriginSource: {
s3BucketSource: s3frontend,
originAccessIdentity: cfdistributionoriginaccessidentity
},
behaviors: [{
isDefaultBehavior: true,
allowedMethods: cloudfront.CloudFrontAllowedMethods.GET_HEAD_OPTIONS,
forwardedValues: {
queryString: true,
cookies: { forward: 'all' }
},
minTtl: cdk.Duration.seconds(0),
defaultTtl: cdk.Duration.seconds(3600),
maxTtl: cdk.Duration.seconds(86400)
}]
}
]
})
s3frontend.grantRead(cfdistributionoriginaccessidentity)
const cfdistributionpolicy = new iam.PolicyStatement({
effect: iam.Effect.ALLOW,
actions: ['cloudfront:CreateInvalidation'],
resources: [`"arn:aws:cloudfront::${this.account}:distribution/${cfdistribution.distributionId}"`]
});
const userpool = new cognito.UserPool(this, 'WebAppUserPool', {
userPoolName: 'web-app-user-pool',
selfSignUpEnabled: false
})
const userpoolidentityprovidergoogle = new cognito.UserPoolIdentityProviderGoogle(this, 'WebAppUserPoolIdentityGoogle', {
clientId: googleClientIdParam.valueAsString,
clientSecret: googleClientSecretParam.valueAsString,
userPool: userpool,
attributeMapping: {
email: cognito.ProviderAttribute.GOOGLE_EMAIL
},
scopes: [ 'email' ]
})
// this is used to make the hostedui reachable
userpool.addDomain('Domain', {
cognitoDomain: {
domainPrefix: domainPrefixParam.valueAsString
}
})
const CLOUDFRONT_PUBLIC_URL = `https://${cfdistribution.distributionDomainName}/`
const client = userpool.addClient('Client', {
oAuth: {
flows: {
authorizationCodeGrant: true
},
callbackUrls: [
CLOUDFRONT_PUBLIC_URL
],
logoutUrls: [
CLOUDFRONT_PUBLIC_URL
],
scopes: [
cognito.OAuthScope.EMAIL,
cognito.OAuthScope.OPENID,
cognito.OAuthScope.PHONE
]
},
supportedIdentityProviders: [
cognito.UserPoolClientIdentityProvider.GOOGLE
]
})
client.node.addDependency(userpoolidentityprovidergoogle)
// defines an AWS Lambda resource
const securedlambda = new lambda.Function(this, 'AuhtorizedRequestsHandler', {
runtime: lambda.Runtime.NODEJS_14_X,
code: lambda.Code.fromAsset('lambda'),
handler: 'secured.handler'
});
const lambdaapiintegration = new apigw.LambdaIntegration(securedlambda)
const backendapigw = new apigw.RestApi(this, 'AuthorizedRequestAPI', {
restApiName: domainPrefixParam.valueAsString,
defaultCorsPreflightOptions: {
"allowOrigins": apigw.Cors.ALL_ORIGINS,
"allowMethods": apigw.Cors.ALL_METHODS,
}
})
const backendapiauthorizer = new apigw.CognitoUserPoolsAuthorizer(this, 'BackendAPIAuthorizer', {
cognitoUserPools: [userpool]
})
const authorizedresource = backendapigw.root.addMethod('GET', lambdaapiintegration, {
authorizer: backendapiauthorizer,
authorizationType: apigw.AuthorizationType.COGNITO
})
const s3deploymentfrontend = new s3deployment.BucketDeployment(this, 'DeployFrontEnd', {
sources: [
s3deployment.Source.asset('./frontend'),
s3deployment.Source.data('constants.js', `const constants = {domainPrefix:'${domainPrefixParam.valueAsString}', region:'${this.region}', cognito_client_id:'${client.userPoolClientId}', apigw_id:'${backendapigw.restApiId}'}`)
],
destinationBucket: s3frontend,
distribution: cfdistribution
})
new cdk.CfnOutput(this, 'YourPublicCloudFrontURL', {
value: CLOUDFRONT_PUBLIC_URL,
description: 'Navigate to the URL to access your deployed application'
})
}
}

Recording the solution from the comments:
Cause:
cdk watch apparently does not work with template parameters. I guess this is because the default --hotswap option bypasses CloudFormation and deploys instead via SDK commands.
Solution:
Remove the CfnParamters from the template. CDK recommends not using parameters in any case.
Perhaps cdk watch --no-hotswap would also work?

Related

How to Cache Additional Items in a Workbox Plugin

I'd like to write a Workbox plugin that intercepts a particular API call, does some logic (omitted from the code below) that determines other URL's that can be cached, then adds them to the runtime cache.
export class PreCachePlugin implements WorkboxPlugin {
fetchDidSucceed: WorkboxPlugin["fetchDidSucceed"] = async ({
response,
}) => {
if (response.ok) {
const clonedResponse = await response.clone();
const json = await clonedResponse.json();
const urls: string[] = getUrlsToCache(json); // Omitting this logic.
await Promise.all(urls.map(url => {
const response = await fetch(url);
// TODO: How do I fetch these URL's and put them in the cache configured
// below while still respecting the expiration.maxEntries setting below?
}));
}
}
}
I am using GenerateSW with workbox-webpack-plugin and adding my plugin:
new GenerateSW({
include: [/\.js$/, /\.json$/, /\.html$/],
runtimeCaching: [
{
urlPattern: ({ url }) => url.toString().endsWith("/foo"),
handler: "StaleWhileRevalidate",
options: {
cacheName: "Foo",
expiration: { maxEntries: 100 },
plugins: [
new PreCachePlugin(),
],
},
},
],
swDest: "./sw.js",
}
How do I fetch the above new URL's and put them in the configured runtime cache while still respecting the expiration.maxEntries settings?

CDK get a webACLId of WAF

How to get a webACLId of WAF
I'm trying to create Cloudfront distribution with WAF.
I tried to create custom resource to get it, but no luck
new cr.AwsCustomResource(this, 'GetParameter', {
onUpdate: { // will also be called for a CREATE event
service: 'WAF',
action: 'ListWebACLs',
// parameters: {
// Limit: 10,
// NextMarker: 'cloudfrontwebacl'
// },
physicalResourceId: cr.PhysicalResourceId.of('Date.now().toString()'),
},
policy: cr.AwsCustomResourcePolicy.fromSdkCalls({
resources: cr.AwsCustomResourcePolicy.ANY_RESOURCE,
}),
});
Ohh. The webACLId should be extract by using Ref
Example:
// 👇 Create WAF Rule
const rule = new CfnRule(this,'rule',{
metricName:'rule',
name:'rule',
predicates:[
...
]
})
...
// 👇 Create WAF WebACL
const CloudfrontWebACL = new CfnWebACL(this, "cloudfrontwebacl", {
name: "cloudfrontwebacl",
defaultAction: { type: 'ALLOW' },
metricName: 'cloudfrontwebacl',
rules:[
{
priority: 0,
ruleId: rule.ref,
action:{
type: 'BLOCK',
}
}
]
})
...
// 👇 Create Cloudfront distribution for web
...
webAclId: CloudfrontWebACL.ref,
...
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-waf-webacl.html#aws-resource-waf-webacl-return-values
https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/intrinsic-function-reference-ref.html

How to get the URL of the ALB after creating in CDK?

I'm creating an Application Load Balancer using the AWS CDK v2.
This is my code:
const lb = new elb.ApplicationLoadBalancer(this, 'LB', {
vpc: ec2.Vpc.fromLookup(this, 'vpc-lookup', {
isDefault: true
}),
internetFacing: true
});
const listener = lb.addListener('Listener', {
port: 80,
});
My question is how do I get the URL (DNS name) of the load balancer? I need it in the CDK after to update something
TL;DR The name's actual value is resolved at deploy-time. At synth-time, you can pass loadBalancerDnsName to other constructs and CDK will create the necessary references.
Resource identifiers like DNS addresses are generally known only only at deploy-time. The CDK uses Tokens to "represent values that can only be resolved at a later time in the lifecycle of an app" . ApplicationLoadBalancer's loadBalancerDnsName: string property is one of those properties whose value resolves to a string Token placeholder
at synth-time and an actual value at deploy-time.
Here's an example of passing the loadBalancerDnsName between constructs:
export class AlbStack extends cdk.Stack {
constructor(scope: Construct, id: string, props: cdk.StackProps) {
super(scope, id, props);
const alb = new elb.ApplicationLoadBalancer(this, 'MyALB', {
vpc: ec2.Vpc.fromLookup(this, 'DefaultVpc', { isDefault: true }),
});
// WON'T WORK: at synth-time, the name attribute returns a Token, not the expected DNS name
console.log(alb.loadBalancerDnsName); // ${Token[TOKEN.220]}
// WILL WORK - CDK will wire up the token in CloudFormation as
new ssm.StringParameter(this, 'MyAlbDns', {
stringValue: alb.loadBalancerDnsName,
});
}
}
The CDK's CloudFormation template output has Fn::GetAtt placeholder for the DNS name that resolves at deploy-time:
// CDK CloudFormation stack template
// Resources section
"MyAlbDnsFD44EB27": {
"Type": "AWS::SSM::Parameter",
"Properties": {
"Type": "String",
"Value": { "Fn::GetAtt": [ "MyALB911A8556", "DNSName" ] } // this will resolve to the string at deploy
},
"Metadata": {
"aws:cdk:path": "TsCdkPlaygroundAlbStack/MyAlbDns/Resource"
}
},

Convert API Gateway Cloudformation template to Swagger file

There is an existing API described in a Coludformation template. Now I want to document the API using Swagger. Is there a way to parse the Cloudformation template to create the swagger.yaml specification file? I would like to avoid writing the API a second time, if possible.
Note: I am aware that you can define your API using Swagger, then import the API configuration in your Cloudformation template. This is not what I need. The Cloudformation already exists and will not be changed. Hence, I need the opposite: a Swagger configuration file based on an existing Cloudformation template.
There is no way to convert the template to a swagger file that I know about. But if you are looking for a way to keep service-spec in one place only (template) and you have it deployed, you can take swagger or OAS file from the stage (so to do it you must have a stage as well) in two ways at least:
By Web console. Use Amazon API Gateway->
APIs->Your API->Stages>Your Stage -> Export tab. See the picture: exporting Swagger or OAS as a file by Web console
aws apigateway get-export ... Here is an example:
aws apigateway get-export --rest-api-id ${API_ID} --stage-name ${STAGE_NAME} --export-type swagger swagger.json
I just made this, it is not setup for perfect plug/play, but will give you an idea what you need to adjust to get it working (also need to make sure you CF template is setup so it has the needed info, on mine I had to add some missing requestParams I was missing, also use this site to test your results from this code to see it works with swagger):
const yaml = require('js-yaml');
const fs = require('fs');
// Get document, or throw exception on error
try {
// loads file from local
const inputStr = fs.readFileSync('../template.yaml', { encoding: 'UTF-8' });
// creating a schema to handle custom tags (cloud formation) which then js-yaml can handle when parsing
const CF_SCHEMA = yaml.DEFAULT_SCHEMA.extend([
new yaml.Type('!ImportValue', {
kind: 'scalar',
construct: function (data) {
return { 'Fn::ImportValue': data };
},
}),
new yaml.Type('!Ref', {
kind: 'scalar',
construct: function (data) {
return { Ref: data };
},
}),
new yaml.Type('!Equals', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::Equals': data };
},
}),
new yaml.Type('!Not', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::Not': data };
},
}),
new yaml.Type('!Sub', {
kind: 'scalar',
construct: function (data) {
return { 'Fn::Sub': data };
},
}),
new yaml.Type('!If', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::If': data };
},
}),
new yaml.Type('!Join', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::Join': data };
},
}),
new yaml.Type('!Select', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::Select': data };
},
}),
new yaml.Type('!FindInMap', {
kind: 'sequence',
construct: function (data) {
return { 'Fn::FindInMap': data };
},
}),
new yaml.Type('!GetAtt', {
kind: 'scalar',
construct: function (data) {
return { 'Fn::GetAtt': data };
},
}),
new yaml.Type('!GetAZs', {
kind: 'scalar',
construct: function (data) {
return { 'Fn::GetAZs': data };
},
}),
new yaml.Type('!Base64', {
kind: 'mapping',
construct: function (data) {
return { 'Fn::Base64': data };
},
}),
]);
const input = yaml.load(inputStr, { schema: CF_SCHEMA });
// now that we have our AWS yaml copied and formatted into an object, lets pluck what we need to match up with the swagger.yaml format
const rawResources = input.Resources;
let guts = [];
// if an object does not contain a properties.path object then we need to remove it as a possible api to map for swagger
for (let i in rawResources) {
if (rawResources[i].Properties.Events) {
for (let key in rawResources[i].Properties.Events) {
// console.log(i, rawResources[i]);
if (rawResources[i].Properties.Events[key].Properties.Path) {
let tempResource = rawResources[i].Properties.Events[key].Properties;
tempResource.Name = key;
guts.push(tempResource);
}
}
}
} // console.log(guts);
const defaultResponses = {
'200': {
description: 'successful operation',
},
'400': {
description: 'Invalid ID supplied',
},
};
const formattedGuts = guts.map(function (x) {
if (x.RequestParameters) {
if (
Object.keys(x.RequestParameters[0])[0].includes('path') &&
x.RequestParameters.length > 1
) {
return {
[x.Path]: {
[x.Method]: {
tags: [x.RestApiId.Ref],
summary: x.Name,
parameters: [
{
name: Object.keys(x.RequestParameters[0])[0].split('method.request.path.')[1],
in: 'path',
type: 'string',
required: Object.values(x.RequestParameters[0])[0].Required,
},
{
name: Object.keys(x.RequestParameters[1])[0].split('method.request.path.')[1],
in: 'path',
type: 'string',
required: Object.values(x.RequestParameters[1])[0].Required,
},
],
responses: defaultResponses,
},
},
};
} else if (Object.keys(x.RequestParameters[0])[0].includes('path')) {
return {
[x.Path]: {
[x.Method]: {
tags: [x.RestApiId.Ref],
summary: x.Name,
parameters: [
{
name: Object.keys(x.RequestParameters[0])[0].split('method.request.path.')[1],
in: 'path',
type: 'string',
required: Object.values(x.RequestParameters[0])[0].Required,
},
],
responses: defaultResponses,
},
},
};
} else if (Object.keys(x.RequestParameters[0])[0].includes('querystring')) {
return {
[x.Path]: {
[x.Method]: {
tags: [x.RestApiId.Ref],
summary: x.Name,
parameters: [
{
name: Object.keys(x.RequestParameters[0])[0].split(
'method.request.querystring.'
)[1],
in: 'query',
type: 'string',
required: Object.values(x.RequestParameters[0])[0].Required,
},
],
responses: defaultResponses,
},
},
};
}
}
return {
[x.Path]: {
[x.Method]: {
tags: [x.RestApiId.Ref],
summary: x.Name,
responses: defaultResponses,
},
},
};
});
const swaggerYaml = yaml.dump(
{
swagger: '2.0',
info: {
description: '',
version: '1.0.0',
title: '',
},
paths: Object.assign({}, ...formattedGuts),
},
{ noRefs: true }
); // need to keep noRefs as true, otherwise you will see "*ref_0" instead of the response obj
// console.log(swaggerYaml);
fs.writeFile('../swagger.yaml', swaggerYaml, 'utf8', function (err) {
if (err) return console.log(err);
});
} catch (e) {
console.log(e);
console.log('error above');
}

Getting "bad-precaching-response " error after service worker registeration is successful

In my project, I am using NextJS+KOA+Apollo. My nextJS app is inside client in root directory. I am using next-offline to convert it to PWA.
Nextjs app is inside client directory. koa server is inside server directory.
when i am building the app via below command:
next build client && tsc --project tsconfig.server.json
it creates a build directory inside client for nextjs and dist directory at the top level for koa server.
i run the code in production via below command
NODE_ENV=production node dist/server/index.js
ISSUE
Service worker is getting registered properly. But I am getting below error:
PrecacheController.mjs:194
Uncaught (in promise) bad-precaching-response: bad-precaching-response :: [{"url":"https://my-domain/_next/bo.svg?__WB_REVISION__=e02afe0476bb357aebde18136fda06e0","status":404}]
at l.o (https://storage.googleapis.com/workbox-cdn/releases/4.3.1/workbox-precaching.prod.js:1:1749)
at async Promise.all (index 0)
at async l.install (https://storage.googleapis.com/workbox-cdn/releases/4.3.1/workbox-precaching.prod.js:1:1221)
Below is my build file that gets generated:
tsconfig.server.json
{
"extends": "./tsconfig.json",
"compilerOptions": {
"module": "commonjs",
"outDir": "dist",
"target": "es2017",
"isolatedModules": false,
"noEmit": false
},
"include": ["server/**/*.ts"]
}
Below is my next.config.js (inside client direcotry)
/* eslint-disable #typescript-eslint/no-var-requires */
const withPlugins = require("next-compose-plugins");
const offline = require("next-offline");
const pino = require("next-pino");
const withTypescript = require("#zeit/next-typescript");
const withCSS = require("#zeit/next-css");
const withLess = require("#zeit/next-less");
const Dotenv = require("dotenv-webpack");
const path = require("path");
const _ = require("lodash");
const nextConfig = {
distDir: "build",
webpack(config) {
config.module.rules.push({
test: /\.(eot|woff|woff2|ttf|svg|png|jpg|gif)$/,
use: {
loader: "url-loader",
options: {
limit: 100000,
name: "[name].[ext]",
},
},
});
config.plugins.push(
new Dotenv({
path: path.resolve(process.cwd(), ".env"),
systemvars: true,
}),
);
return config;
},
// overwrites values given in the .env file with the current
// process.env value
env: _.omitBy(
{
GRAPHQL_SERVER: process.env.GRAPHQL_SERVER,
},
_.isUndefined,
),
workboxOpts: {
globPatterns: ["static/**/*"],
globDirectory: "client",
runtimeCaching: [
{
urlPattern: /^https?.*/,
handler: "NetworkFirst",
options: {
cacheName: "offlineCache",
expiration: {
maxEntries: 200,
},
},
},
],
},
};
const cssConfig = {
cssModules: true,
cssLoaderOptions: {
importLoaders: 1,
localIdentName: "[local]",
},
};
const lessConfig = cssConfig;
module.exports = withPlugins(
[
[offline],
[pino],
[withTypescript],
[withCSS, cssConfig],
[withLess, lessConfig],
],
nextConfig,
);
And below is my file to start koa server
import Router from "koa-router";
const server = new Koa();
const dev = !["production", "staging"].includes(process.env.NODE_ENV || "");
const app = next({ dir: "./client", dev });
const publicRouter = new Router();
const handle = app.getRequestHandler();
publicRouter.get("/service-worker.js", async ctx => {
const pathname = await join(
__dirname,
"../../../client/build",
"service-worker.js",
);
ctx.body = await app.serveStatic(ctx.req, ctx.res, pathname);
ctx.respond = false;
});
publicRouter.get("*", async ctx => {
if (!ctx.path.match(/graphql/)) {
await handle(ctx.req, ctx.res);
ctx.respond = false;
}
});
server.use(async (ctx, next) => {
ctx.res.statusCode = 200;
await next();
});
server.use(publicRouter.routes()).use(publicRouter.allowedMethods());
server.listen({ port: 3000 });
================================================================
I have done a dirty fix for now. I am not sure how to handle it properly. I will really appreciate if anyone can put forth their view on this.
As bo.svg, firfox.svg, all these static files are throwing 404,
Ex - (/_next/bo.svg?WB_REVISION=e02afe0476bb357aebde18136fda06e0)
in my file to start koa server, added a condition to check this URL and serve static file from build directory like below:
publicRouter.get("*", async ctx => {
if (ctx.path.match(/\_next/) && ctx.path.match(/\.svg/)) {
const pathname = await join(
__dirname,
"../../../client/build",
ctx.path.replace("_next/", ""),
);
ctx.body = await app.serveStatic(ctx.req, ctx.res, pathname);
ctx.respond = false;
} else if (!ctx.path.match(/graphql/)) {
await handle(ctx.req, ctx.res);
ctx.respond = false;
}
});
It served my prupose for now, but not sure how to handle this properly.

Resources