I bumped into a problem where we want to make sure some conventions are being followed in the naming of CloudFormation resources. The idea is that we use CDK Aspects to process resources. A simple example:
export class BucketConvention implements cdk.IAspect {
private readonly ctx: Context;
constructor(ctx: Context) {
this.ctx = ctx;
}
public visit(node: cdk.IConstruct): void {
if (cdk.CfnResource.isCfnResource(node) && node.cfnResourceType == 'AWS::S3::Bucket') {
const resource = node as s3.CfnBucket;
const resourceId = resource.bucketName ? resource.bucketName : cdk.Stack.of(node).getLogicalId(node);
resource.addPropertyOverride('BucketName', `${ctx.project}-${ctx.environment}-${resourceId}`);
}
}
}
The Context interface simply holds some variables used to create names. The problem with this snippet is that we are trying to interpolate the bucket name if it has been set, if not use the logical ID. Now the method to obtain the logical ID works, however resource.BucketName will return a token of which the resolved value could be undefined (i.e. the user didn't pass a bucket name in constructing the bucket, which happens a lot in high level constructs). So the logical ID will actually never trigger, since a token is always defined. If you would log the interpolation output you could get something like
myproject-myenvironment-${Token[TOKEN.104]}
My question, how can we make this work such that the interpolation happens with the bucket name if it has been supplied and if not use the logical ID? Is there a way to peek whether the token will give an undefined value during synthesis time?
And found the answer to my problem... similar to the logical ID you can use
cdk.Stack.of(resource).resolve(resource.bucketName)
Related
So I am trying to get into app development and following this guys yt-tutorial (yt-vid from The Net Ninja). However I stumbled over the following error:
Non-nullable instance field ‘{0}’ must be initialized. (error on dart.dev explained). The whole code is shown in this image
img coding: comparison mine with yt:
The issue I am facing is that I don't understand the origin of that error. I understand that you can't use a variable that was not initialized yet and that this error is trying to tell you that before you run the code and run into an exception in runtime.
I also know that I can fix this using "late" in front of the variables, but as far as I'm concerned that's just "ignoring" the error.
So my main question is: why do I have this error while the yt doesn't even though we have (except for some names and assignments) the same code?
I appreciate every answer and hope you have a great day
the links:
error on dart.dev explained
You should change your User class to:
class User {
String username;
int age;
User(this.username, this.age);
}
By doing so, the variables will be initialized with values (since we are then able to use this values in the constructor body) as part of initialization of the object.
Dart does object creation in two steps:
Initialize object.
Run constructor body.
So when the constructor body is running, the object must be initialized with values (well, the non-nullable values at least). To do so, we have a code block which is executed before { } like:
class User {
String username;
int age;
User(String username, int age)
: this.username = username,
this.age = age;
}
But since this is kinda redundant, we have the shortcut form as I first suggested.
When initializing an object, we are not allowed to use variables from the object since the object is ongoing its creation.
my objective is to call a first level cdk command starting from an interface: IBucket.
I can get the bucket reference starting from this:
const sourceBucket = props.glueTable.bucket;
Afterwords, I need to call:
cfnBucket.replicationConfiguration = {
The procedure is exactly as for the script below:
https://github.com/rogerchi/cdk-s3-bucketreplication/blob/main/src/index.ts
But, as you can see, this script requires:
readonly sourceBucket: s3.Bucket;
Since it is needed to call:
const sourceAccount = cdk.Stack.of(props.sourceBucket).account;
Finally, are there really no other ways to call a cloudformation level 1 command starting from a reference?
It looks odd.
Thank you in advance
Marco
There is an example on exactly this in the aws docs:
If a Construct is missing a feature or you are trying to work around an issue, you can modify the CFN Resource that is encapsulated by the Construct.
All Constructs contain within them the corresponding CFN Resource. For example, the high-level Bucket construct wraps the low-level CfnBucket construct. Because the CfnBucket corresponds directly to the AWS CloudFormation resource, it exposes all features that are available through AWS CloudFormation.
The basic approach to get access to the CFN Resource class is to use construct.node.defaultChild (Python: default_child), cast it to the right type (if necessary), and modify its properties. Again, let's take the example of a Bucket.
// Get the CloudFormation resource
const cfnBucket = bucket.node.defaultChild;
// Change its properties
cfnBucket.analyticsConfiguration = [
{
id: 'Config'
// ...
}
];
From https://docs.aws.amazon.com/cdk/latest/guide/cfn_layer.html
For you, it wouldn't be analyticsconfiguration but bucketreplication of course.
I'm developing a multitenant application and would like to have to option to remove a tenant. This however seems to be less trivial than one would assume.
My goal is to delete all references to the tenant everywhere in the database. I understand that Tenant is Soft-Delete, but I since I don't want my database to fill up with old meaningless data I've tried disabling the soft-delete filter.
Here is some code that I've tried:
using (_unitOfWorkManager.Current.DisableFilter(AbpDataFilters.SoftDelete))
{
await TenantRepository.DeleteAsync(x => x.Id == tenantId);
}
This did not work. The tenant is marked as "IsDeleted" but not removed.
Then I figured that maybe it has something to do with UnitOfWork so I made sure no UnitOfWork was active and then manually controlled it:
using (var unitOfWork = _unitOfWorkManager.Begin())
{
// the codeblock above went here
unitOfWork.Complete();
}
This did not work, same result. And this is just the AbpTenant table. I'm also trying to delete from all other tables. For example AbpSettings and AbpLanguages. It's very unclear to me how to do that at all - the "managers" doesn't contain any Delete functions.
I tried creating IRepository for these entities but it does not work. The error reads
The type Abo.Configuration.Setting cannot be used as a type parameter TEntity in the generic type or method IRepository. There is no implicit reference conversion from Abp.Configuration.Setting to Abo.Domain.Entities.IEntity.
That leaves me with the option to use the DataContext directly:
using (EntityFramework.MyDbContext db = new EntityFramework.MyDbContext())
{
List<PermissionSetting> perms = await db.Permissions.Where(x => x.TenantId == tenantId).ToListAsync();
for (int i=0; i<perms.Count(); i++)
{
db.Permissions.Remove(perms[i]);
}
// I also tried deleting them in bulk at first
// ((DbSet<PermissionSetting>)db.Permissions).RemoveRange(db.Permissions.Where(x => x.TenantId == tenantId));
await db.SaveChangesAsync();
}
I tried that with and without UnitOfWork.
But it simply does not get deleted from the database. I'm getting no errors or Exceptions.
Why does it not get deleted? How can I delete it? Surely it must be possible?
since I don't want my database to fill up with old meaningless data I've tried disabling the soft-delete filter.
From the question on Disable SoftDelete for AbpUserRole:
protected override void CancelDeletionForSoftDelete(EntityEntry entry)
{
if (IsSoftDeleteFilterEnabled)
{
base.CancelDeletionForSoftDelete(entry);
}
}
The type Abo.Configuration.Setting cannot be used as a type parameter TEntity in the generic type or method IRepository. There is no implicit reference conversion from Abp.Configuration.Setting to Abo.Domain.Entities.IEntity.
Inject IRepository<Setting, long> instead of IRepository<Setting>.
That leaves me with the option to use the DataContext directly
...
But it simply does not get deleted from the database. I'm getting no errors or Exceptions.
From the documentation on Data Filters:
using (_unitOfWorkManager.Current.DisableFilter(AbpDataFilters.MayHaveTenant))
{
using (var db = new ...)
{
// ...
}
}
That said, there is no way to easily delete related tenant data completely. Consider writing SQL.
I have a pipeline that successfully outputs an Avro file as follows:
#DefaultCoder(AvroCoder.class)
class MyOutput_T_S {
T foo;
S bar;
Boolean baz;
public MyOutput_T_S() {}
}
#DefaultCoder(AvroCoder.class)
class T {
String id;
public T() {}
}
#DefaultCoder(AvroCoder.class)
class S {
String id;
public S() {}
}
...
PCollection<MyOutput_T_S> output = input.apply(myTransform);
output.apply(AvroIO.Write.to("/out").withSchema(MyOutput_T_S.class));
How can I reproduce this exact behavior except with a parameterized output MyOutput<T, S> (where T and S are both Avro code-able using reflection).
The main issue is that Avro reflection doesn't work for parameterized types. So based on these responses:
Setting Custom Coders & Handling Parameterized types
Using Avrocoder for Custom Types with Generics
1) I think I need to write a custom CoderFactory but, I am having difficulty figuring out exactly how this works (I'm having trouble finding examples). Oddly enough, a completely naive coder factory appears to let me run the pipeline and inspect proper output using DataflowAssert:
cr.RegisterCoder(MyOutput.class, new CoderFactory() {
#Override
public Coder<?> create(List<? excents Coder<?>> componentCoders) {
Schema schema = new Schema.Parser().parse("{\"type\":\"record\,"
+ "\"name\":\"MyOutput\","
+ "\"namespace\":\"mypackage"\","
+ "\"fields\":[]}"
return AvroCoder.of(MyOutput.class, schema);
}
#Override
public List<Object> getInstanceComponents(Object value) {
MyOutput<Object, Object> myOutput = (MyOutput<Object, Object>) value;
List components = new ArrayList();
return components;
}
While I can successfully assert against the output now, I expect this will not cut it for writing to a file. I haven't figured out how I'm supposed to use the provided componentCoders to generate the correct schema and if I try to just shove the schema of T or S into fields I get:
java.lang.IllegalArgumentException: Unable to get field id from class null
2) Assuming I figure out how to encode MyOutput. What do I pass to AvroIO.Write.withSchema? If I pass either MyOutput.class or the schema I get type mismatch errors.
I think there are two questions (correct me if I am wrong):
How do I enable the coder registry to provide coders for various parameterizations of MyOutput<T, S>?
How do I values of MyOutput<T, S> to a file using AvroIO.Write.
The first question is to be solved by registering a CoderFactory as in the linked question you found.
Your naive coder is probably allowing you to run the pipeline without issues because serialization is being optimized away. Certainly an Avro schema with no fields will result in those fields being dropped in a serialization+deserialization round trip.
But assuming you fill in the schema with the fields, your approach to CoderFactory#create looks right. I don't know the exact cause of the message java.lang.IllegalArgumentException: Unable to get field id from class null, but the call to AvroCoder.of(MyOutput.class, schema) should work, for an appropriately assembled schema. If there is an issue with this, more details (such as the rest of the stack track) would be helpful.
However, your override of CoderFactory#getInstanceComponents should return a list of values, one per type parameter of MyOutput. Like so:
#Override
public List<Object> getInstanceComponents(Object value) {
MyOutput<Object, Object> myOutput = (MyOutput<Object, Object>) value;
return ImmutableList.of(myOutput.foo, myOutput.bar);
}
The second question can be answered using some of the same support code as the first, but otherwise is independent. AvroIO.Write.withSchema always explicitly uses the provided schema. It does use AvroCoder under the hood, but this is actually an implementation detail. Providing a compatible schema is all that is necessary - such a schema will have to be composed for each value of T and S for which you want to output MyOutput<T, S>.
I'm trying to avoid referencing the concrete type library in my main project, but I'm getting this error:
No default instance or named instance 'Default' for requested plugin type StackExchangeChatInterfaces.IClient
1.) Container.GetInstance(StackExchangeChatInterfaces.IClient ,{username=; password=; defaultRoomUrl=; System.Action`2[System.Object,System.Object]=System.Action`2[System.Object,System.Object]})
I've setup my container to scan for assemblies, like so:
var container = new Container(x =>
{
x.Scan(scan =>
{
scan.AssembliesFromApplicationBaseDirectory();
scan.ExcludeNamespace("StructureMap");
scan.WithDefaultConventions();
scan.AddAllTypesOf<IMessageHandlers>();
});
//x.For<IClient>().Use<Client>(); //GetInstance will work if this line is not commented out.
});
When I try to get an instance, I get the error, my code for getting an instance is here:
chatInterface = container
.With("username").EqualTo(username)
.With("password").EqualTo(password)
.With("defaultRoomUrl").EqualTo(roomUrl)
.With<Action<object, object>>(delegate(object sender, object messageWrapper)
{
string message = ((dynamic)messageWrapper).Message;
Console.WriteLine("");
Console.WriteLine(message);
foreach (var item in messageHandlers)
{
item.MessageHandler.Invoke(message, chatInterface);
}
}).GetInstance<IClient>();
If I explicitly map the concrete class to the interface, everything works hunky dory, but that means I need to reference the project that Client is in, which I don't want to do.
This is really interesting. Looks like default conventions are not able to register types with such constructor (tried on both versions 2.6.3 and 3+). I was only registered when only parameterless constructor was specified. Looking at sources of both versions it is really suspicious as it should be registered. Deeper dive into the code would be needed...
Anyway try using custom registration convention:
public class ClientConvention : IRegistrationConvention
{
public void Process(Type type, Registry registry)
{
if (type.IsClass && !type.IsAbstract && !type.IsGenericType &&
type.GetInterfaces().Contains(typeof(IClient)))
{
registry.For(typeof(IClient)).Use(type);
}
}
}
Configure it like this:
var container = new Container(
c => c.Scan(
s =>
{
s.ExcludeNamespace("StructureMap");
s.WithDefaultConventions();
s.Convention<ClientConvention>();
s.AddAllTypesOf<IMessageHandlers>();
}));
and this should work just fine.
The default type scanning will not pick up concrete types whose constructor functions contain primitive arguments like strings, numbers, or dates. The thinking is that you'd effectively have to explicitly configure those inline dependencies anyway.
"but that means I need to reference the project that Client is in, which I don't want to do."
Does that actually matter? I think you're making things harder than they have to be by trying to eliminate the assembly reference.