Kia ora.
We have a very specific tagging specification for all our AWS resources at work. Obviously I can add "static" tags to all resources within the stack/app, like this:
var app = new App();
new MyStack(app, "MyStack", new StackProps
{
// ...
});
// Required, "static" tags, as per our tagging specification
Tags.Of(app).Add("uuid", "unique-internal-identifier");
Tags.Of(app).Add("env.name", "testish");
Tags.Of(app).Add("managed:ownership:division", "Flibble");
Tags.Of(app).Add("managed:ownership:portfolio", "Flooble");
Tags.Of(app).Add("managed:ownership:product-group", "Wibble");
// ... and so on
app.Synth();
The way I understand the semantics of this is that the CDK is being told that the resources declared within the construct (app) are required to have these tags applied in order for the CDK's "model" to be considered consistent with the deployed resources in AWS. And so the applicable resources will be created/updated, and have the applicable tabs applied. If the "static" value of a tag is changed, then all constructs that have that tag applied will be redeployed during the "deploy" phase of the CDK.
There are, however, other tags that our tagging specification requires which contain more dynamic content. An example would be a git commit hash; or date/time of deployment. For these, we want to update the tags' values if and only if the CDK effects a change within AWS for any other reason. That is: the definition of the tag should not form part of the evaluation of the consistency of the CDK's "model"; but if the CDK makes any change to the affected resource, then a new value for the tag should be set.
How would I accomplish this? Is there an escape hatch in the "plan" stage of the CDK to exclude certain tags, but then have the tag values applied during a "deploy" stage?
Related
In order to provide hermetic builds and runtime, we currently build custom flex template and worker images. As part of deployment, we build the flex template and we can specify the custom flex template image but not the custom worker image - we have to pass it to dataflow separately when invoking the flex template. This creates hidden dependencies that we need to track as part of the release process elsewhere (and seems to go against the design of flex templates of being self-contained). Is there a way to bake at least a default for the worker image ( sdk_container_image ) into the template?
In general, if you want to have a default for a template parameter, you can just use the the #Default annotation on it and change the default value for every template version you build. Though in this particular case, since sdkContainerImage parameter is declared in DataflowPipelineOptions in Java SDK (or WorkerOptions in Python SDK), and you can't control its default from the user code, I'd try to set the parameter value programmatically in the template code.
I think something like this should work, but I haven't tested it:
DataflowPipelineOptions options =
PipelineOptionsFactory.fromArgs(args).as(DataflowPipelineOptions.class);
if (options.getSdkContainerImage() == null || options.getSdkContainerImage().isEmpty()) {
// Set the default if not already set by the template runner.
options.setSdkContainerImage("...");
}
Pipeline pipeline = Pipeline.create(options);
// ...
This is for Java, but you can do a similar thing with Python SDK.
In the AWS CDK, it's straight forward to create a Pinpoint Service. But how do you get the Project ID (also referred to as the Pinpoint App ID or Application ID) for use in subsequent CDK code.
Create a Pinpoint project:
const pinpointProject = new pinpoint.CfnApp(this, 'PinpointNotificationProject', {
name: 'myProject',
});
In the AWS CloudFormation docs it says:
"When you pass the logical ID of this resource to the intrinsic Ref function, Ref returns the unique identifier (ApplicationId) for the Amazon Pinpoint application."
However, the following CDK code returns the project name not the id. The value of logicalId = myProject.
cdk.Fn.ref(pinpointProject.logicalId); // This returns 'myProject'
pinpointProject.ref; // This also returns 'myProject'
This is confirmed fixed in the latest CDK version 1.130.0. The ref property now returns the Pinpoint ProjectId.
The problem you are running into is that pinpoint is not a finished module. You can see this that all the functions within are prefixed with Cfn - cloudformation. This means that they are barebones and not tied into all the interface hooks that the rest of CDK is making use of to toss things around.
First, the logical ID is NOT the project name. the Logical Id is part of the Cloudformation Template that is generated for any resource Cloudformation is going to stand up. It links the given resource to the stack, so that any changes under the same logical id will be applied to the same stood up resource. It is only referenced internally to the cloudformation stack and never known outside. CDK uses the LogicalID to generate the name of the resource if you do not specify one.
Second, Taking a look at the documentation shows that CfnApp has the following property: attrArn. Meaning in your code, you would reference this by pinpointProject.attrArn - the arn of a pinpoint resource is something like: arn:aws:mobiletargeting:region:accountId:apps/projectId. with, as you guessed it, the projectId as the last value. you can split the string and get that value out, or use the arn manipulation methods provided as part of the core module to extract what you need.
Finally, even though the Pinpoint module is pretty much just barebones, it may still be possible to pass the variable storing your Pinpoint Construct Object to whatever other resource requires it. I say may because, as mentioned, most of the Cfn prefixed functions do not have the proper hooks to do this well - but some do, and Ive never worked with Pinpoint directly.
I recommend spending some time to understand how the CDK Documentation is laid out. Its bare bones in places, but once you understand how they structured it, these kinds of questions are readily answered within.
This question piggybacks this github issue. However, I have ran into this issue in one other context.
Context
Within Bazel, there are two repository rules, maven_jar and maven_server.
maven_jar(name, artifact, repository, server, sha1)
maven_server(name, repository, settings)
The maven_jar rule's server attribute is a label pointing to some maven_server target.
Currently, whenever the server attribute is provided, the maven_jar rule errors out.
What I would like to accomplish
Within maven_jar's implementation function, I would like to access the maven_server's attributes. Specifically, I would like to do something along the lines of:
def _impl(rtx):
settings_attr = rtx.attr.server.getSettings()
# alternatively
settings_attr = rtx.attr.server.getAttributes().settings
Is this behavior supported? If not, any way I can approximate it?
The server attribute is a label, so I'm not sure if one can obtain these values using its providers/aspects.
Repository rules are macros, so they do not have providers the same way "normal" rules do. Thus, if you specify a label attribute, it basically has to be a source file.
As settings.xml isn't supposed to be project-specific, I think it mgiht make more sense for maven_jar to use the users/system's settings.xml, as described in the Maven docs:
There are two locations where a settings.xml file may live:
The Maven install: ${maven.home}/conf/settings.xml
A user’s install: ${user.home}/.m2/settings.xml
The former settings.xml are also called
global settings, the latter settings.xml are referred to as user
settings. If both files exists, their contents gets merged, with the
user-specific settings.xml being dominant.
A custom plugin we wrote for an older version of Jenkins uses an EnvironmentContributingAction to provide environment variables to the execution so they could be used in future build steps and passed as parameters to downstream jobs.
While attempting to convert our build to workflow, I'm having trouble accessing these variables:
node {
// this step queries an API and puts the results in
// environment variables called FE1|BE1_INTERNAL_ADDRESS
step([$class: 'SomeClass', parameter: foo])
// this ends up echoing 'null and null'
echo "${env.FE1_INTERNAL_ADDRESS} and ${env.BE1_INTERNAL_ADDRESS}"
}
Is there a way to access the environment variable that was injected? Do I have to convert this functionality to a build wrapper instead?
EnvironmentContributingAction is currently limited to AbstractBuilds, which WorkflowRuns are not, so pending JENKINS-29537 which I just filed, your plugin would need to be modified somehow. Options include:
Have the builder add a plain Action instead, then register an EnvironmentContributor whose buildEnvironmentFor(Run, …) checks for its presence using Run.getAction(Class).
Switch to a SimpleBuildWrapper which defines the environment variables within a scope, then invoke it from Workflow using the wrap step.
Depend on workflow-step-api and define a custom Workflow Step with comparable functionality but directly returning a List<String> or whatever makes sense in your context. (code sample)
Since PR-2975 is merged, you are able to use new interface:
void buildEnvVars(#Nonnull Run<?, ?> run, #Nonnull EnvVars env, #CheckForNull Node node)
It will be used by old type of builds as well.
Can I store completely different images in single Docker repository just by using different tags?
e.g username/myrepo:software, username/myrepo:code
If so, how?
Yes. The tags are of the form [url]/[user]/image[:tag]. If you can authenticate as 'user' at 'url' you can store any number of repos there with any number of tags.
In the usual case, your 'code' and 'software' tags will be related in some way, say by having a common ancestor, but there's nothing that enforces that rule.