I h got an issue with the following serverless config.
this is my handler and the files/folders structure.
the issue is that after uploading my project to AWS when I test my lambda I got an error as follows:
lambda execution fails: "errorMessage": "Unable to import module 'app_monitor': No module named 'monitoring'"
{
"errorMessage": "Unable to import module 'src/app_monitor': No module named 'monitoring'",
"errorType": "Runtime.ImportModuleError",
"requestId": "bca3f67d-815f-452b-a2a6-c713ad2c6baa",
"stackTrace": []
}
have you got any clue how can I add this into serverless config.?
First, a quick tip on troubleshooting: When I ran into such issues it was helpful to go to the AWS console, look at the lambda function, and see what the uploaded file structure looks like on that end. Is the monitoring folder there?
Moreover, in order to specify how a specific function is packaged, you have to explicitly state that you want it to be individually packaged and not follow the general rules of the project as a whole.
You should try to add:
app_monitoring:
package:
individually: true
patterns:
- 'src/**'
More documentation on packaging configuration here
You may also have better luck with explicitly stating the patterns you need, I know I've had issues with globs in the past. So for example you can try:
patterns:
- 'src/app_monitoring.py'
- 'src/monitoring/get_lb.py'
I have the following three files as below:
main.tf, variables.tf and dev.auto.tfvars
Snippet from main.tf
module "sql_vms" {
source = "git::git#github.com:xxxxxxxxxxxx/terraform-modules//azure/"
rg_name = var.resource_group_name
location = module.resource_group.external_rg_location
vnet_name = var.virtual_network_name
subnet_name = var.sql_subnet_name
app_nsg = var.application_nsg
vm_count = var.count_vm
base_hostname = var.sql_host_basename
sto_acc_suffix = var.storage_account_suffix
vm_size = var.virtual_machine_size
vm_publisher = var.virtual_machine_image_publisher
vm_offer = var.virtual_machine_image_offer
vm_sku = var.virtual_machine_image_sku
vm_img_version = var.virtual_machine_image_version
username = var.username
password = var.password
}
Snippet from variables.tf
variable "app_subnet_name" {
type = string
}
variable "sql_subnet_name" {
type = string
}
Snippet from dev.auto.tfvars
app_subnet_name = "subnet_1"
sql_subnet_name = "subnet_2"
application_nsg = "test_nsg"
However, I'm getting error like below
Error: Unsupported argument
on main.tf line 7, in module "sql_vms":
7: subnet_name = var.sql_subnet_name
An argument named "subnet_name" is not expected here.
Error: Unsupported argument
on main.tf line 8, in module "sql_vms":
8: app_nsg = var.application_nsg
An argument named "app_nsg" is not expected here.
My modules directory structure looks like below
$ ls -R terraform-modules/
terraform-modules/:
aws azure gcp
terraform-modules/aws:
alb ec2-instance-rhel
terraform-modules/aws/alb:
terraform-modules/aws/ec2-instance-rhel:
main.tf
terraform-modules/azure:
compute resourcegroup sqlserver
terraform-modules/azure/compute:
main.tf README.md variable.tf
terraform-modules/azure/resourcegroup:
data.tf outputs.tf variables.tf
terraform-modules/azure/sqlserver:
main.tf README.md variables.tf
terraform-modules/gcp:
compute
terraform-modules/gcp/compute:
main.tf
Any idea what is going wrong here?
If you are starting out with Terraform, you will get that error message ("An argument named "example" is not expected here") if your module arguments refer to the resource properties and not to variable names, see below for an example:
Example of a Terraform module "example_mod.tf" you want to call from your module:
variable "sg_name" { } # Usually in a separate file
variable "sg_desc" { } # called variables.tf
resource "example_resource" "example_name" {
name = var.sg_name
description = var.sg_desc
...
}
CORRECT WAY:
module "my_module" {
source = "./modules/example_mod.tf"
sg_name = "whatever" # NOTE the left hand side "sg_name" is the variable name
sg_desc = "whatever"
...
}
INCORRECT WAY: (Gives the error "An argument named "name" is not expected here" )
module "my_module" {
source = "./modules/example_mod.tf"
name = "whatever" # WRONG because the left hand side "name" is a resource property
description = "whatever" # WRONG for the same reason
...
}
I think the issue is that you do not refer to the exact module with the source. I see you have three modules in the source:
source = "git::git#github.com:xxxxxxxxxxxx/terraform-modules//azure/"
They are compute, resourcegroup and sqlserver. But you want to load them in one module. So it cannot find the related variables for the modules. I also don't think it's the right way to load all the modules like that. I would recommend you load the modules one by one like below:
module "compute" {
source = "git::git#github.com:xxxxxxxxxxxx/terraform-modules//azure/compute"
...
}
module "resourcegroup" {
source = "git::git#github.com:xxxxxxxxxxxx/terraform-modules//azure/resourcegroup"
...
}
module "sqlserver" {
source = "git::git#github.com:xxxxxxxxxxxx/terraform-modules//azure/sqlserver"
...
}
Without knowing the details about the module it is usually hard to say what's the reason for an error, but in this particular case it seems that there isn't a requirement in the module you're importing to use those two arguments (subnet_name and app_nsg), or rather that you are using a version of the module that doesn't require them to be present. What helps with that type of error is to check if there is a version of the module that does have such a requirement. The syntax for using a particular module version from Github is explained in Terraform Module Sources documentation, Selecting a Revision section:
module "vpc" {
source = "git::https://example.com/vpc.git?ref=v1.2.0"
}
You are probably using SSH to fetch the module, so the recommended way to do that is:
When using Git over SSH, we recommend using the ssh://-prefixed URL form for consistency with all of the other URL-like git address forms.
In your example, this translates to:
module "sql_vms" {
source = "git::ssh://git#github.com/org/terraform-modules-repo.git//azure/module-name?ref=v1.2.0"
where org is your organisation's (or your private) Github account, terraform-modules-repo is the repo where modules reside, module-name is the module you are using and ref=v1.2.0 represents the module revision number.
The error An argument named "example" is not expected here. means that the module doesn't expect to see an input argument with that name. Think about Terraform modules as functions in a programming language: in order to have a function provide a result, you pass the function a set of required arguments. If you provide more (or less) input arguments than required by that function call, you will get an error. (There are special cases but it is out of the scope of this question.)
Another similarity between modules and functions is that Terraform modules can also provide output values, besides creating resources that are specified. That can be handy in cases where output can be used as input in other modules or resources. The line module.resource_group.external_rg_location is doing exactly that: getting the output value from another module and using it to assign a value to an argument location.
I had a similar issue when working with AWS Eventbridge and Terraform.
When I run terraform plan I get the error below:
Error: Unsupported argument
│
│ on ../../modules/aws/eventbridge/main.tf line 37, in resource "aws_cloudwatch_event_target" "ecs_cloudwatch_event_target":
│ 37: maximum_age_in_seconds = var.maximum_age_in_seconds
│
│ An argument named "maximum_age_in_seconds" is not expected here.
Here's how I solved it:
The issue was that I was not using the correct attribute for the AWS Eventbridge resource block.
The attribute should have been maximum_event_age_in_seconds and not maximum_age_in_seconds.
Another issue that could this is not defining a variable in your terraform script that is already defined in a module.
That's all
It could be happening due to plenty of reasons.
I'd suggest some verification:
Check if you are using the correct source URL, path or revision branch/tag.
I'm not sure about your implementation, but you probably want to double check the revision you are referencing contains theses variable declarations.
GitHub Modules addressing allows ref argument.
Refer to the GitHub Module Addressing for Terraform Documentation and how to specify a revision.
Check if all necessary variables are declared on every module, including the root module.
Did you declare those variables both in a variables.tf file on your root directory and on the module context/path?
I know that's exhausting and repetitive, but every module should be designed as an "independent project". Each module **MUST have its own declared variables.tf**, which work as inputs for that module, and it is also desirable that it has its own mapped outputs.tf, provider.tf, backend.tf, etc., though these last ones are not required.
FYI: Doing so you guarantee scalability, reusability, as well as reliability to work with different tfstate files and even different repositories for each module in order to guarantee atomicity and minimum permissions, hence preventing your infrastructure from being destroyed by undesired code changes.
I highly recommend this read to understand the importance of independent modularization design.
Furthermore, tools like Terragrunt, Terratest can make this job less painful by keeping your code DRY ( Don't Repeat Yourself ).
Check if the **type constraints of the related variables match.**
If that's not your case, try looking if the type constraints match between all declarations of the variables used both as arguments ( on your root variables.tf ) and inputs ( on your module level variables.tf ).
I'll share my pain as well.
Writing block configuration like this
vpc_config = {
subnet_ids = [aws_subnet.example1.id, aws_subnet.example2.id]
}
Instead of (Notice the = Equal Sign):
vpc_config {
subnet_ids = [aws_subnet.example1.id, aws_subnet.example2.id]
}
Will give an error of An argument named "vpc_config" is not expected here and will waste you a few good hours.
I am working on a Jira/Rally (CA Agile Central) integration and can get a basic sync to work, however some fields require a more complex transformation when syncing them between Jira and Rally.
For this I can see that the CA Agile Connector (https://help.rallydev.com/jira-installation-user-guide) provides some support for "custom field handlers" which are written in Ruby and follow a format like:
# Copyright 2015 CA Technologies. All Rights Reserved.
require 'rallyeif/wrk/field_handlers/field_handler'
module RallyEIF
module WRK
module FieldHandlers
class MyCustomFieldHandler < OtherFieldHandler
def initialize(field_name = nil)
super(field_name)
end
# ... more code here ...
end
end
end
end
However when I create that file and add the following to my connector config:
...
<Connector>
<FieldMapping>
<Field>
<Rally>Description</Rally>
<Other>Description</Other>
<Direction>TO_RALLY</Direction>
</Field>
...
</FieldMapping>
<OtherFieldHandlers>
<MyCustomFieldHandler>
<FieldName>Description</FieldName>
</MyCustomFieldHandler>
</OtherFieldHandlers>
</Connector>
...
When running the connector I get the following error:
[2017-08-22 20:25:39 Z] ERROR : RallyEIF::WRK::Connector.rescue in block in read_field_handlers - For RallyEIF::WRK::JiraConnection: Could not find class for MyCustomFieldHandler
The documentation does not mention how to use the custom handlers at all, so I'm wondering if anyone has used this feature and can share some information on how to declare and use the custom field handlers.
I tested connector version 4.7.2 and it worked. Things to check:
within the folder where the connector is invoked, there must be a folder named "field_handlers"
within this "field_handlers" folder, there must be a file (as you have shown above) with "class MyCustomFieldHandler < ...."
There is an example on the help pages. It's for HP-ALM (QC) not for JIRA, but the concept is the same:
https://help.rallydev.com/QC-custom-fieldhandler.pxml
End of ideas so far.
I am very new to Kafka and Samza. I tried the hello-samza exampe and it is working. What I am looking for is to create a samza task that reads the message from a kafka topic.The task I added does not throw any error, and is not reading any message from the topic. Yarn UI shows task as accepted.Not sure what I am doing wrong here.
Here is the Class
public class MyTask implements StreamTask {
#Override
public void process(IncomingMessageEnvelope incomingMessageEnvelope, MessageCollector messageCollector, TaskCoordinator taskCoordinator) throws Exception {
System.out.println(" key - " + incomingMessageEnvelope.getKey() + " | message " + incomingMessageEnvelope.getMessage());
}
}
Here is the properties file
# Job
job.factory.class=org.apache.samza.job.yarn.YarnJobFactory
job.name=addresses
# YARN
yarn.package.path=file://${basedir}/target/${project.artifactId}-${pom.version}-dist.tar.gz
# Task
task.class=samza.examples.wikipedia.task.MyTask
task.inputs=addressestopic
# Serializers
serializers.registry.json.class=org.apache.samza.serializers.JsonSerdeFactory
# Kafka System
systems.kafka.samza.factory=org.apache.samza.system.kafka.KafkaSystemFactory
systems.kafka.samza.msg.serde=json
systems.kafka.consumer.zookeeper.connect=localhost:2181/
systems.kafka.producer.bootstrap.servers=localhost:9092
# Job Coordinator
job.coordinator.system=kafka
# Add configuration to disable checkpointing for this job once it is available in the Coordinator Stream model
# See https://issues.apache.org/jira/browse/SAMZA-465?focusedCommentId=14533346&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-14533346 for more details
job.coordinator.replication.factor=1
If the Yarn UI shows that your job is in "ACCEPTED" state and not "RUNNING", then it is possible that Yarn has not yet found resources to run your Samza job.
Usually, I have noticed this to happen when you run out of disk space on your local box on which you are executing.
Can you check the Yarn UI (localhost:8088) and verify that "Active Nodes" (on the top) == 1 ?
Additionally, you can cross-check the number of "Lost Nodes" to be equal to zero.
If there are Lost Nodes, you can click on the link to see why it is not available for use.
I've created a custom Symfony execution filter to manage the integrity of a POST request by extending the sfFilter class. The filter works OK but some times I need to disable it and the fact is the parameter condition specified in the filter definition in filters.yml file(see below) is not working.
# filters.yml
rendering: ~
security: ~
SSL:
class: SSLFilter
condition: %APP_ENABLE_SSLFILTER%
integrity:
class: IntegrityFilter
param:
integrity_header: x-integrity-cnonce
condition: %APP_ENABLE_INTEGRITYCHECK%
cache: ~
execution: ~
As you can see the condition for the IntegrityFilter requires the configuration entry ENABLE_INTEGRITYCHECK to be ON in the app.yml file to execute the filter.
# app.yml
all:
sf_guard_plugin:
algorithm_callable: md5
success_signout_url: /
success_signin_url: /
enable_sslfilter: off
# Enables de IntegrityFilter check
enable_integritycheck: off
Well the thing is that's not working and it's becoming since I use a script to enable/disable some settings along with other tasks and that one isn't disabling. The funny thing is that enabling/disabling works fine for the the SSL Filter shown in both YAML files. Any ideas...?