Terraform for_each iteration with the file function - foreach

My requirement is to create a dynamic resource Confluent Schema. Below is the schema.tf file.
Basically, need to include map type object and will be creating different Schema by passing name and file attributes. What changes to be done on below highlighted "schema" file parameter so that it can be included in the for_each block?
resource "confluent_schema" "sample_avro_schema" {
schema_registry_cluster {
id = confluent_schema_registry_cluster.essentials.id
}
rest_endpoint = confluent_schema_registry_cluster.essentials.rest_endpoint
for_each = toset(var.subject_name_avro)
subject_name = each.key
format = "AVRO"
**schema = file("modules/confluent_kafka_cluster_dedicated/schemas/sample_schema_avro.avsc")**
credentials {
key = confluent_api_key.env-manager-schema-registry-api-key.id
secret = confluent_api_key.env-manager-schema-registry-api-key.secret
}
}
Variable declaration as below: variable.tf file
variable "subject_name_avro" {
description = "AVRO Schema Name"
type = list(string)
default = ["avro-topic-value"]
}
And I am running this execution using .tfvars file:
subject_name_avro = ["avro-topic-1-value"]
My requirement is to include below changes in .tfvars file. Kindly suggest what resource and variable level changes to be done to include schema file parameter dynamically.
subject_name_avro = [
{
subject_name_avro = "avro-topic-1-value"
schema = file("modules/confluent_kafka_cluster_dedicated/schemas/sample_schema_avro1.avsc")
},
{
subject_name_avro = "avro-topic-2-value"
schema = file("modules/confluent_kafka_cluster_dedicated/schemas/sample_schema_avro2.avsc")
},
]
Sample file content "sample_schema_avro.avsc"
{
"type": "record",
"namespace": "io.confluent.developer.avro",
"name": "Purchase",
"fields": [
{
"name": "item",
"type": "string"
},
{
"name": "amount",
"type": "double"
},
{
"name": "customer_id",
"type": "string"
}
]
}

You can't use file in a variabiles. You can use only path in your case:
subject_name_avro = [
{
subject_name_avro = "avro-topic-1-value"
schema = "./modules/confluent_kafka_cluster_dedicated/schemas/sample_schema_avro1.avsc"
},
{
subject_name_avro = "avro-topic-2-value"
schema = "./modules/confluent_kafka_cluster_dedicated/schemas/sample_schema_avro2.avsc"
},
]
To iterate over this you can use count or for_each. With for_each it would be:
resource "confluent_schema" "sample_avro_schema" {
for_each = {for idx, val in var.subject_name_avro: idx => val}
schema_registry_cluster {
id = confluent_schema_registry_cluster.essentials.id
}
rest_endpoint = confluent_schema_registry_cluster.essentials.rest_endpoint
subject_name = each.value.subject_name_avro
format = "AVRO"
**schema = file(each.value.schema)
credentials {
key = confluent_api_key.env-manager-schema-registry-api-key.id
secret = confluent_api_key.env-manager-schema-registry-api-key.secret
}
}

Related

Ruby make put / post http call with array of object

i have this http call code, the type is form
param = {
form: {
"creatives[]" => [
{
is_visible: params[:creative_banner_is_visible],
type: "banner",
value_translations: {
id: params[:creative_banner_value_id],
en: params[:creative_banner_value_en]
}
},
{
is_visible: params[:creative_video_is_visible],
type: "video",
value_translations: {
id: params[:creative_video_value_id],
en: params[:creative_video_value_en]
}
}
]
}
}
http = HTTP.headers(headers)
http.put(base_url, param)
but somehow this is translated to this on the target server
"creatives"=>[
"{:is_visible=>\"true\", :type=>\"banner\", :value_translations=>{:id=>\"Banner URL ID\", :en=>\"Banner URL EN\"}}",
"{:is_visible=>\"true\", :type=>\"video\", :value_translations=>{:id=>\"12345ID\", :en=>\"12345EN\"}}"
]
do you know how to make this http call not stringified? i used same schema on postman and work just fine
"creatives": [
{
"is_visible": true,
"type": "banner",
"value_translations": {
"id": "http://schroeder.info/elinore",
"en": "http://wehner.info/dusti"
}
},
{
"is_visible": true,
"type": "video",
"value_translations": {
"id": "85177e87-6b53-4268-9a3c-b7f1c206e002",
"en": "5134f3ca-ead7-4ab1-986f-a695e69ace96"
}
}
]
i'm using this gem https://github.com/httprb/http
EDIT
First, replace your "creatives[]" => [ ... with creatives: [ ... so the end result should be the following.
creatives = [
{
is_visible: params[:creative_banner_is_visible],
type: "banner",
value_translations: {
id: params[:creative_banner_value_id],
en: params[:creative_banner_value_en]
}
},
{
is_visible: params[:creative_video_is_visible],
type: "video",
value_translations: {
id: params[:creative_video_value_id],
en: params[:creative_video_value_en]
}
}
]
http = HTTP.headers(headers)
http.put(base_url, creatives.to_json)
Second, I don't see any problem with what you get in your target server, you just have to parse it to JSON, so if you also have a Rails app there use JSON.parse on the body.
somehow this approach fixed the issue
create_params = {}.compare_by_identity
create_params["creatives[][is_visible]"] = params[:creative_banner_is_visible]
create_params["creatives[][type]"] = 'banner'
create_params["creatives[][value_translations][id]"] = params[:creative_banner_value_id]
create_params["creatives[][value_translations][en]"] = params[:creative_banner_value_en]
create_params["creatives[][is_visible]"] = params[:creative_video_is_visible]
create_params["creatives[][type]"] = 'video'
create_params["creatives[][value_translations][id]"] = params[:creative_video_value_id]
create_params["creatives[][value_translations][en]"] = params[:creative_video_value_en]

Robot allowed the website but being identified and rejected

I need to do web scraping for a website which has allowed robot access. Below is the robot.txt file's content.
User-agent: *
Disallow:
Sitemap:https://www.sample.com/sitemap-index.xml
But when I try to fetch the website's content using nokogiri, it's being detected.
Nokogiri::HTML(open('https://www.sample.com/search?q=test', :ssl_verify_mode => OpenSSL::SSL::VERIFY_NONE))
Here the output:
> (Document:0x3fda40e7cf70 {
name = "document",
children = [
#(DTD:0x3fda40e9591c { name = "html" }),
#(Element:0x3fda40e8c95c {
name = "html",
attributes = [ #(Attr:0x3fda4071a598 { name = "style", value = "height:100%" })],
children = [
#(Element:0x3fda3fefa28c {
name = "head",
children = [
#(Element:0x3fda401a3088 {
name = "meta",
attributes = [ #(Attr:0x3fda40ebd7a0 { name = "name", value = "ROBOTS" }), #(Attr:0x3fda40ebd778 { name = "content", value = "NOINDEX, NOFOLLOW" })]
}),
#(Element:0x3fda4074faf4 {
name = "meta",
attributes = [ #(Attr:0x3fda3ff0beec { name = "name", value = "format-detection" }), #(Attr:0x3fda3ff0bed8 { name = "content", value = "telephone=no" })]
}),
#(Element:0x3fda401ca700 {
name = "meta",
attributes = [ #(Attr:0x3fda401c2050 { name = "name", value = "viewport" }), #(Attr:0x3fda401c217c { name = "content", value = "initial-scale=1.0" })]
}),
#(Element:0x3fda4079a284 {
name = "meta",
attributes = [ #(Attr:0x3fda4078bfb8 { name = "http-equiv", value = "X-UA-Compatible" }), #(Attr:0x3fda4078bf04 { name = "content", value = "IE=edge,chrome=1" })]
})]
}),
#(Element:0x3fda407e2e6c {
name = "body",
attributes = [ #(Attr:0x3fda430205f0 { name = "style", value = "margin:0px;height:100%" })],
children = [
#(Element:0x3fda4072e2a0 {
name = "iframe",
attributes = [
#(Attr:0x3fda3ff45214 {
name = "src",
value = "/_Incapsula_Resource?SWUDNSAI=28&xinfo=5-66719320-0%200NNN%20RT%281543054979096%20247%29%20q%280%20-1%20-1%20-1%29%20r%280%20-1%29%20B12%284%2c315%2c0%29%20U2&incident_id=245000650118470008-256430953704260629&edet=12&cinfo=04000000"
}),
#(Attr:0x3fda3ff451d8 { name = "frameborder", value = "0" }),
#(Attr:0x3fda3ff451b0 { name = "width", value = "100%" }),
#(Attr:0x3fda3ff45188 { name = "height", value = "100%" }),
#(Attr:0x3fda3ff45174 { name = "marginheight", value = "0px" }),
#(Attr:0x3fda3ff4514c { name = "marginwidth", value = "0px" })],
children = [ #(Text "Request unsuccessful. Incapsula incident ID: 245000650118470008-256430953704260629")]
})]
})]
})]
})
How can I achieve this web scraping?

Post method with multiple parameter

I am unable to insert multiple rows in database using Post method in MVC web API. I have written code for it but when i am testing by inserting multiple rows through postman it is giving error. At line first the variable "delegatetable" shows null due to which error is coming. i am not doing database connection through entity framework, i have created a DelegateTable class.
public HttpResponseMessage Post(List<DelegateTable> delegatetable)
{
try
{
using (var delegateContext = new ShowContext())
{
foreach (DelegateTable item in delegatetable)
{
DelegateTable delegates = new DelegateTable();
delegates.Salutation__c = item.Salutation__c;
delegates.First_Name__c = item.First_Name__c;
delegates.Last_Name__c = item.Last_Name__c;
delegates.Account_Name__c = item.Account_Name__c;
delegates.Contact_Email__c = item.Contact_Email__c;
delegates.Category__c = item.Category__c;
delegates.Conference_Type__c = item.Conference_Type__c;
delegates.Conference_Selection__c = item.Conference_Selection__c;
delegates.Payment_Statuss__c = item.Payment_Statuss__c;
delegates.Barcode__c = item.Barcode__c;
delegateContext.SaveChanges();
}
var message = Request.CreateResponse(HttpStatusCode.Created, delegatetable);
message.Headers.Location = new Uri(Request.RequestUri.ToString());
return message;
}
}
catch (Exception ex)
{
return Request.CreateErrorResponse(HttpStatusCode.BadRequest, ex);
}
}
Json data that i am passing is below
[
{
"attributes": {
"type": "Registration__c",
"url": "/services/data/v43.0/sobjects/Registration__c/a3h8E0000009VuVQAU"
},
"Salutation__c": "Dr.",
"First_Name__c": "Test",
"Last_Name__c": "Test",
"Account_Name__c": "Test",
"Contact_Email__c": "test123#gmail.com",
"Category__c": "Test",
"Conference_Type__c": null,
"Conference_Selection__c": null,
"Payment_Statuss__c": null,
"Barcode__c": "Test"
},
{
"attributes": {
"type": "Registration__c",
"url": "/services/data/v43.0/sobjects/Registration__c/a3hD0000001kEfOIAU"
},
"Salutation__c": "Mr.",
"First_Name__c": "Demo",
"Last_Name__c": "Demo",
"Account_Name__c": "Demo",
"Contact_Email__c": "Demo#gmail.com",
"Category__c": "Demo",
"Conference_Type__c": null,
"Conference_Selection__c": null,
"Payment_Statuss__c": null,
"Barcode__c": null
}
]
You may try to reformat your payload as a JSON array, as the problem might be that the payload cannot be converted to a List.
Try this:
{
"delegates" :
[
{
"attributes": ..., ...
},
{ "attributes": ..., ...
},
...
]
}

Add bpmn:Resource in propertiesPanel

Is it possible to add "bpmn:Resource" element in propertiesPanel and edit its attribute? how to do it?
I've already added some properties to property panel based on the property-panel[link] example.
But I have a challenge in adding 'bpmn:Resource' to the properties panel. I don't want it to be added as an extensionElement.
I'd like it to be inside bpmn:Definitions (beside bpmn:Process). Also I'd like to extend the original bpmn:Resource to add some parameters.
So in MyModdle.json I added:
{
"name": "Resource",
"extends": [
"bpmn:Resource"
],
"properties": [
{
"name": "parameters",
"isMany": true,
"type": "MyParameter"
}
]
}, {
"name": "MyParameter",
"properties": [
{
"name": "myParameterType",
"isAttr": true,
"type": "String"
}
{
"name": "myParameterName",
"isAttr": true,
"type": "String"
},
{
"name": "myParameterValue",
"isAttr": true,
"type": "String"
}
]
}
now for example in newElement function of MyResource.js
var newElement = function (type, prop, factory) {
return function (element, extensionElements, value) {
var commands = [];
var resource = getResource(element);
if (!resource) {
var parent = extensionElements;
resource = createResource(parent, bpmnFactory);
console.log('resource', resource);
commands.push(cmdHelper.addAndRemoveElementsFromList(
element,
extensionElements,
'values',
'extensionElements',
[resource],
[]
));
}
var newElem = createResourceParameter(type, resource, bpmnFactory, {
resourceId: 'id-' + value
});
commands.push(cmdHelper.addElementsTolist(element, parameters, prop, [newElem]));
return commands;
};
}
I know this cmdHelper adds 'bpmn:Resource' to extensionElements but I don't know what to use instead!

How to access URI query parameters from a Resource class using Apigility?

I'm developing an Apigility driven application based on the Zend Framework 2.
I want my application to provide nested responses for both -- single items and lists:
/projects/1
{
"id": "1",
"title": "...",
...
"_embedded": {
"images": [
{
"id": "1",
"project_id": "1",
"title": "...",
...
},
{
"id": "2",
"project_id": "1",
"title": "...",
...
}
]
},
...
}
/projects
{
...
"_embedded": {
"projects": [
{
"id": "1",
"title": "...",
...
"_embedded": {
"images": [
{
"id": "1",
"project_id": "1",
"title": "...",
...
},
...
]
},
...
},
...
]
},
"total_items": 2
}
Since I've not found an apigility conform solution for implementing lists with nested lists (in this case projects with a list of images for every project list item, see here). I have to deal with the Paginator and DbAdapter and provide the page parameter manually:
class ProjectResource extends AbstractResourceListener {
...
public function fetchAll($params = array()) {
$pageNumber = $this->getEvent()->getRouteMatch()->getParam('page', 1); <-- that doesn't work
$projectService = $this->getProjectService();
$offset = $pageNumber > 0 ? $pageNumber - 1 : 0;
$config = $this->getServiceManager()->get('Config');
$itemCountPerPage = $config['zf-rest']['Portfolio\\V2\\Rest\\Project\\Controller']['page_size'];
$projects = $projectService->getProjects($offset, $itemCountPerPage);
return $projects;
}
...
}
The problem is, that $this->getEvent()->getRouteMatch()->getParam('page', 1) doesn't work. Instead of the page parameter, $this->getEvent()->getRouteMatch()->getParams() returns
Array
(
[controller] => Portfolio\V2\Rest\Project\Controller
[version] => 2
)
How to access request parameters?
Request parameters have first to be added onto the whitelist. It can be done over the Apigility GUI or directly in the config:
module.config.php
return array(
...
'zf-rest' => array(
...
'Portfolio\\V2\\Rest\\Project\\Controller' => array(
...
'collection_query_whitelist' => array('page'),
...
),
...
),
);
Then the parameter can be accessed over the arguments of the end point methods of the Resource class:
public function fetchAll($params = array()) {
$projectService = $this->getProjectService();
$config = $this->getServiceManager()->get('Config');
$itemCountPerPage = $config['zf-rest']['Portfolio\\V2\\Rest\\Project\\Controller']['page_size'];
$pageNumber = isset($params['page']) && intval($params['page']) > 0
? $params['page']
: 1
;
$offset = ($pageNumber - 1) * $itemCountPerPage;
$projects = $projectService->getProjects($offset, $itemCountPerPage);
return $projects;
}
See also the Apiglity documentation: ZF REST -> Configuration -> User Configuration -> collection_query_whitelist.

Resources