AWS CDK Lambda function_from_arn not working as expected - aws-cdk

I have this Lambda and this DynamoDB table
my_lambda = lambda_.Function(
self,
"my_lambda",
function_name="my_lambda",
description="A Lambda to test permissions",
code=lambda_code,
memory_size=512,
handler="my_lambda.main",
runtime=lambda_.Runtime.PYTHON_3_9,
architecture=lambda_.Architecture.ARM_64,
timeout=Duration.minutes(1),
)
table = dynamodb.Table(
self,
'test_table',
partition_key=dynamodb.Attribute(
name="id",
type=dynamodb.AttributeType.STRING,
),
)
Now, if I want to give the Lambda access to write in the DynameDB table I do this.
table.grant_full_access(my_lambda)
This works perfectly.
Now, if I want to give this same Lambda access to the table be getting a reference to it it doesn't work.
lambda_by_arn = lambda_.Function.from_function_arn(
self,
"my lambda by arn",
my_lambda.function_arn
)
table.grant_full_access(lambda_by_arn)
The above doesn't work and the Lambda has no access whatsoever to the DynamoDB function.
If you have the DynamoDB creation in a different stack than the Lambda, you cannot do it any other way (at least, to my knowledge) than by the function_from_arn method.
I tried getting the Lambda from a different method: function_from_attributes but this resulted in the same way: No access.

What from_function_arn does is import an existing Lambda function so that you can reference it from your CDK application. But it is not actually part of the application, so you can't do much with it. In particular, you can't grant any access to it.
If you have the DynamoDB creation in a different stack than the Lambda, you cannot do it any other way (at least, to my knowledge) than by the function_from_arn method.
If both stacks are in the same app, you can pass the Lambda function reference to the other stack, and CDK will know how to deal with it.

TL;DR - The iam.Grant methods like grant_full_access *sometimes* work on externally referenced resources returned from Something.fromSomethingAttributes methods. Unfortunately, *not* for DynamoDB Table resources.
You should be seeing a warning produced by the CDK CLI when you synth the app:
[Warning at /my_stack/my_lambda_by_arn] Add statement to this resource's role: ...
This is telling you the CDK didn't grant access - do it yourself! The CDK made a design decision to warn, but not to throw an error1.
Under what conditions can an externally reference ISomething construct successfully be granted IAM privileges?
The granting resource type must support Resource-based Policies (e.g. s3.Bucket, sqs.Queue), and
The ISomething's role reference must be passed to Something.fromSomethingAttributes
This table summarizes what happens in various case. Your case is on the bottom right:
Method
Granter has Resource Policy
Granter has no Resource Policy
fromSomethingAttributes + role: IRole
✅ Role ARN added to Resource Policy
❌ UnknownPrincipal assigned, CLI Warning
fromSomethingAttributes, no role
❌ Synth error: resource imported without a role
❌ UnknownPrincipal assigned, CLI Warning
fromSomethingArn
❌ Synth error: resource imported without a role
❌ UnknownPrincipal assigned, CLI Warning
You can force the CDK to fail on such synth warnings with cdk synth --strict

Related

How do you authenticate with an API key inside an IBM Cloud Function?

I am writing an IBM Cloud Function which uses the python SDK to interface with a Cloudant service. I have the Cloudant service up, the databases populated, and service credentials / API key ready. However when I try to instantiate the CloudantV1 service inside my Function I get a runtime error "must provide authenticator".
I looked up the error in their git repos and it seems like it is trying to setup an authenticator object by looking up values from environment variables, which do not exist in the Function. I just want to pass my API key directly, but I have not found a method to do this. I am using basic code from the examples so I think my calls are correct.
I have considered injecting the environment variables inside the Function, but that sounds like a major hack. I must be doing something incorrectly. Please help me understand what it is. Here is basic Function python code which reproduces the error:
from ibmcloudant.cloudant_v1 import CloudantV1
def main(params_dict):
service = CloudantV1.new_instance()
# unreachable
return { "message" : "hello world" }
There is an example for programmatic authentication at https://cloud.ibm.com/apidocs/cloudant?code=python#programmatic-authentication - it basically looks like this:
from ibmcloudant.cloudant_v1 import CloudantV1
from ibm_cloud_sdk_core.authenticators import IAMAuthenticator
authenticator = IAMAuthenticator('yourAPIkey')
service = CloudantV1(authenticator=authenticator)
service.set_service_url('https://yourserviceurl.example')

import python functions into serversless

I h got an issue with the following serverless config.
this is my handler and the files/folders structure.
the issue is that after uploading my project to AWS when I test my lambda I got an error as follows:
lambda execution fails: "errorMessage": "Unable to import module 'app_monitor': No module named 'monitoring'"
{
"errorMessage": "Unable to import module 'src/app_monitor': No module named 'monitoring'",
"errorType": "Runtime.ImportModuleError",
"requestId": "bca3f67d-815f-452b-a2a6-c713ad2c6baa",
"stackTrace": []
}
have you got any clue how can I add this into serverless config.?
First, a quick tip on troubleshooting: When I ran into such issues it was helpful to go to the AWS console, look at the lambda function, and see what the uploaded file structure looks like on that end. Is the monitoring folder there?
Moreover, in order to specify how a specific function is packaged, you have to explicitly state that you want it to be individually packaged and not follow the general rules of the project as a whole.
You should try to add:
app_monitoring:
package:
individually: true
patterns:
- 'src/**'
More documentation on packaging configuration here
You may also have better luck with explicitly stating the patterns you need, I know I've had issues with globs in the past. So for example you can try:
patterns:
- 'src/app_monitoring.py'
- 'src/monitoring/get_lb.py'

What is the correct way to identify production & development environment in AWS CDK?

I am learning AWS Cloud Development Kit (CDK).
As part of this learning, I am trying to understand how I am supposed to correctly handle production and development environment.
I know AWS CDK provides the environment parameter to allow deploying stacks to specific account.
But then, how to have specific options for development versus production stacks ? It does not seem to be provided by default by AWS CDK or am I missing/misunderstanding something ?
A very simple example could be that I want a S3 bucket called my-s3-bucket-dev for my development account and one named my-s3-bucket-prod for my production account. But then how to have e.g. a variable stage correctly handled in AWS CDK ?
I know I can add parameters in the cdk.json file but again, I don't know how to correctly use this file to depend upon the deployed stack i.e. production vs development.
Thanks for the support
Welcome to AWS CDK.
Enjoy the ride. ;)
Actually, there is no semantic (in your case the stage) in an account itself.
This has nothing to do with CDK or Cloud Formation.
You need to take care of this.
You're right, that you could use the CDK context in the cdk.json.
There's no schema enforcement in the context, except for some internally used variables by CDK.
You could define your dev and prod objects within.
There are other ways of defining the context.
Here is an example, what it could look like:
{
"app": "node app",
// usually there's some internal definition for your CDK project
"context": {
"dev": {
"accountId" : "prod_account",
"accountRegion" : "us-east-1",
"name": "dev",
"resourceConfig":
{
// here you could differentiate the config per AWS resource-type
// e.g. dev has lower hardware specs
}
},
"prod": {
"accountId" : "prod_account",
"accountRegion" : "us-east-1",
"name": "prod",
"resourceConfig":
{
// here you could differentiate the config per AWS resource-type
// prod has higher hardware specs or more cluster nodes
}
}
}
}
With this being defined, you need to run your CDK application with the -c flag to specify which configuration object (dev or prod), you want to have.
For instance, you could run it with cdk synth -c stage=prod.
This sets the stage variable in your context and makes it available.
When it was successful, you can re-access the context again and fetch the appropriate config object.
const app = new cdk.App();
const stage = app.node.tryGetContext('stage');
// the following step is only needed, if you have a different config per account
const stageConfig = app.node.tryGetContext(stage );
// ... do some validation and pass the config to the stacks as constructor argument
As I said, the context is one way of doing this.
However, there are drawbacks to it.
It's JSON and no code.
What I prefer is to have TypeScript types per resource configuration (e.g. S3) and wire them all together as a plain object.
The object maps the account/region information and the corresponding resource configurations.

How does one inject dependencies like a logger, database connection, or SHA256 generator in Iron? [duplicate]

In writing my tests, I'd like to be able to inject a connection into the request so that I can wrap the entire test case in a transaction (even if there is more than one request in the test case).
I've attempted to do this using a BeforeMiddleware which I can link in my test cases to insert a connection, as such:
pub type DatabaseConnection = PooledConnection<ConnectionManager<PgConnection>>;
pub struct DatabaseOverride {
conn: DatabaseConnection,
}
impl BeforeMiddleware for DatabaseOverride {
fn before(&self, req: &mut Request) -> IronResult<()> {
req.extensions_mut().entry::<DatabaseOverride>().or_insert(self.conn);
Ok(())
}
}
However, I'm encountering a compile error in trying to do this:
error: the trait bound `std::rc::Rc<diesel::pg::connection::raw::RawConnection>: std::marker::Sync` is not satisfied [E0277]
impl BeforeMiddleware for DatabaseOverride {
^~~~~~~~~~~~~~~~
help: run `rustc --explain E0277` to see a detailed explanation
note: `std::rc::Rc<diesel::pg::connection::raw::RawConnection>` cannot be shared between threads safely
note: required because it appears within the type `diesel::pg::PgConnection`
note: required because it appears within the type `r2d2::Conn<diesel::pg::PgConnection>`
note: required because it appears within the type `std::option::Option<r2d2::Conn<diesel::pg::PgConnection>>`
note: required because it appears within the type `r2d2::PooledConnection<r2d2_diesel::ConnectionManager<diesel::pg::PgConnection>>`
note: required because it appears within the type `utility::db::DatabaseOverride`
note: required by `iron::BeforeMiddleware`
error: the trait bound `std::cell::Cell<i32>: std::marker::Sync` is not satisfied [E0277]
impl BeforeMiddleware for DatabaseOverride {
^~~~~~~~~~~~~~~~
help: run `rustc --explain E0277` to see a detailed explanation
note: `std::cell::Cell<i32>` cannot be shared between threads safely
note: required because it appears within the type `diesel::pg::PgConnection`
note: required because it appears within the type `r2d2::Conn<diesel::pg::PgConnection>`
note: required because it appears within the type `std::option::Option<r2d2::Conn<diesel::pg::PgConnection>>`
note: required because it appears within the type `r2d2::PooledConnection<r2d2_diesel::ConnectionManager<diesel::pg::PgConnection>>`
note: required because it appears within the type `utility::db::DatabaseOverride`
note: required by `iron::BeforeMiddleware`
Is there a way around this with diesel's connections? I've found several examples on Github to do this using the pg crate, but I'd like to keep using diesel.
This answer will certainly solve the problem, but it's not optimal. As mentioned, you can't share a single connection as it's not thread safe. However, while wrapping it in a Mutex makes it thread-safe, it would force all the server threads to use a single connection. Instead, you want to use a connection pool.
You can accomplish this with the r2d2 and r2d2-diesel crates. This will establish multiple connections as needed, and reuse them when possible in a thread safe manner.
Since there isn't enough code provided for me to reproduce your issue, I've made this:
use std::cell::Cell;
trait Middleware: Sync {}
struct Unsharable(Cell<bool>);
impl Middleware for Unsharable {}
fn main() {}
which has the same error:
error: the trait bound `std::cell::Cell<bool>: std::marker::Sync` is not satisfied [E0277]
impl Middleware for Unsharable {}
^~~~~~~~~~
help: run `rustc --explain E0277` to see a detailed explanation
note: `std::cell::Cell<bool>` cannot be shared between threads safely
note: required because it appears within the type `Unsharable`
note: required by `Middleware`
You can solve the problem by changing the type to make it cross-thread compatible:
use std::sync::Mutex;
struct Sharable(Mutex<Unsharable>);
impl Middleware for Sharable {}
Note that Rust has done a very good thing for you: it prevented you from using a type that is unsafe to be called in multiple threads.
In writing my tests, I'd like to be able to inject a connection into the request so that I can wrap the entire test case in a transaction (even if there is more than one request in the test case).
I'd suggest that it's possible an architectural change would be even better. Separate the domains of "web framework" from your "database". The authors of Growing Object-Oriented Software, Guided by Tests (a highly recommended book) advocate for this style.
Pull apart your code such that there is a method that simply accepts some type that can start / end a transaction, write the interesting stuff there, and test it thoroughly. Then have just enough glue code in the web layer to create a transaction object, then call the next layer down.

:rpc.call fails with global registered node alias

I have an Elixir project that globally registers a node using the Erlang global module:
:global.register_name(:my_node, self)
From another node in the cluster I can get the pid of the registered node using the global alias:
:global.whereis_name :my_node
However I'm unable to issue an rpc call using the global alias. The following fails:
:rpc.call(:my_node, Foo.Bar, :new, ["hello"])
>> {:badrpc, :nodedown}
I'm able to issue the rpc call if I use the full node name. The following works:
:rpc.call(:"mynode#127.0.0.1", Foo.Bar, :new, ["hello"])
Is it possible to use global aliases with rpc calls?
The global registry is for registering processes. So when you call:
:global.register_name(:my_node, self)
You are registering the current process (given by self) globally.
There is no node name registering because their names are already available globally. So the last snippet in your post is the correct way of doing so.

Resources