I need to send an input in the below format to an API, I'm running into problems producing the desired format which is below.
{ "application" :
"{\"attributes\":{\"type\":\"genesis__Applications__c\"},\"genesis__Days_Convention__c\":\"ACTUAL/ACTUAL\", \"RecordTypeID\":\"012260000004RHF\",
\"genesis__Interest_Calculation_Method__c\":\"Flat\",
\"genesis__Interest_Rate__c\":10.0000,
\"genesis__Loan_Amount__c\":22120.00,
\"Application_Completed__c\":false,
\"genesis__Payment_Frequency__c\":\"WEEKLY\",
\"genesis__Product_Type__c\":\"LOAN\", \"genesis__Term__c\":24,
\"genesis__Interest_Only_Period__c\":2,
\"genesis__Balloon_Payment__c\":100.00}", "relatedObjects" : "{
\"genesis__Account__c\" : {\"attributes\":
{\"type\":\"Account\"}, \"Name\":\"LONDON METALS HOLDINGS
LIMITED\" }, \"Loan_Product_Purpose__c\" : {\"attributes\":
{\"type\":\"Loan_Product_Purpose__c\"}, \"Name\":\"Equipment
Purchase\" }}" }
API is accepting above format as input.
Input specifications as shown.
We tried JSON builder that did not help instead producing the below format
which is not valid for API.
"\"{:attributes:{:type:\\"genesis__Applications__c\\"},
:genesis__Days_Convention__c:\\"ACTUAL/ACTUAL\\",
:RecordTypeID:\\"012260000004RHF\\",
:genesis__Interest_Calculation_Method__c:\\"Flat\\",
:genesis__Interest_Rate__c:10.0, :genesis__Loan_Amount__c:22120.0,
:Application_Completed__c:false,
:genesis__Payment_Frequency__c:\\"WEEKLY\\",
:genesis__Product_Type__c:\\"LOAN\\", :genesis__Term__c:24,
:genesis__Interest_Only_Period__c:2,
:genesis__Balloon_Payment__c:100.0}\""
edit1: Input Hash object
{:application=> {:attributes=>{:type=>"genesis__Applications__c"},
:genesis__Days_Convention__c=>"ACTUAL/ACTUAL",
:RecordTypeID=>"012260000004RHF",
:genesis__Interest_Calculation_Method__c=>"Flat",
:genesis__Interest_Rate__c=>10.0,
:genesis__Loan_Amount__c=>22120.0,
:Application_Completed__c=>false,
:genesis__Payment_Frequency__c=>"WEEKLY",
:genesis__Product_Type__c=>"LOAN", :genesis__Term__c=>24,
:genesis__Interest_Only_Period__c=>2,
:genesis__Balloon_Payment__c=>100.0}, :relatedObjects=>
{:genesis__Account__c=>{:attributes=>{:type=>"Account"},
:Name=>"LONDON METALS HOLDINGS LIMITED"},
:Loan_Product_Purpose__c=>{:attributes=>{:type=>"Loan_Product_Purpose__c"}, :Name=>"Equipment Purchase"}}}
How are you using JSON? It should work just fine
require 'json'
h = {:application=>
{:attributes=>{:type=>"genesis__Applications__c"},
:genesis__Days_Convention__c=>"ACTUAL/ACTUAL",
:RecordTypeID=>"012260000004RHF",
:genesis__Interest_Calculation_Method__c=>"Flat",
:genesis__Interest_Rate__c=>10.0,
:genesis__Loan_Amount__c=>22120.0,
:Application_Completed__c=>false,
:genesis__Payment_Frequency__c=>"WEEKLY",
:genesis__Product_Type__c=>"LOAN", :genesis__Term__c=>24,
:genesis__Interest_Only_Period__c=>2,
:genesis__Balloon_Payment__c=>100.0}, :relatedObjects=>
{:genesis__Account__c=>{:attributes=>{:type=>"Account"}, :Name=>"LONDON METALS HOLDINGS LIMITED"},
:Loan_Product_Purpose__c=>{:attributes=>{:type=>"Loan_Product_Purpose__c"}, :Name=>"Equipment Purchase"}}}
puts h.to_json
prints:
{"application":{"attributes":{"type":"genesis__Applications__c"},"genesis__Days_Convention__c":"ACTUAL/ACTUAL","RecordTypeID":"012260000004RHF","genesis__Interest_Calculation_Method__c":"Flat","genesis__Interest_Rate__c":10.0,"genesis__Loan_Amount__c":22120.0,"Application_Completed__c":false,"genesis__Payment_Frequency__c":"WEEKLY","genesis__Product_Type__c":"LOAN","genesis__Term__c":24,"genesis__Interest_Only_Period__c":2,"genesis__Balloon_Payment__c":100.0},"relatedObjects":{"genesis__Account__c":{"attributes":{"type":"Account"},"Name":"LONDON METALS HOLDINGS LIMITED"},"Loan_Product_Purpose__c":{"attributes":{"type":"Loan_Product_Purpose__c"},"Name":"Equipment Purchase"}}}
Ok, now I see the value of application and relatedObjects are strings (with json content). So try this instead:
require 'json'
application = {:attributes=>{:type=>"genesis__Applications__c"},
:genesis__Days_Convention__c=>"ACTUAL/ACTUAL",
:RecordTypeID=>"012260000004RHF",
:genesis__Interest_Calculation_Method__c=>"Flat",
:genesis__Interest_Rate__c=>10.0,
:genesis__Loan_Amount__c=>22120.0,
:Application_Completed__c=>false,
:genesis__Payment_Frequency__c=>"WEEKLY",
:genesis__Product_Type__c=>"LOAN", :genesis__Term__c=>24,
:genesis__Interest_Only_Period__c=>2,
:genesis__Balloon_Payment__c=>100.0}
relatedObjects = {:genesis__Account__c=>{:attributes=>{:type=>"Account"}, :Name=>"LONDON METALS HOLDINGS LIMITED"},
:Loan_Product_Purpose__c=>{:attributes=>{:type=>"Loan_Product_Purpose__c"}, :Name=>"Equipment Purchase"}}
h = {:application=> application.to_json,
:relatedObjects=> relatedObjects.to_json}
puts h.to_json
Related
Using Krakend as api gateway.
I have an endpoint configured in krakend.json:
"endpoint":"/call",
"extra_config":{
"github.com/devopsfaith/krakend-lua/proxy":{
"sources":[
"/function.lua"
],
"pre":"pre_backend(request.load())",
"live":true,
"allow_open_libs":true
}
},
"method":"POST",
"output_encoding":"json",
"headers_to_pass":[
"*"
],
"backend":[
{
"url_pattern":"/api/v1/get_client_id",
[...]]
},
The endopont "/api/v1/get_client_id" recives just a param:
{"user_mail_1":"test#test.es"}
I want, whith the lua script my endopoint "/call" recives:
{"email":"test#test.es"}
and transform on before send:
{"user_mail_1":"test#test.es"}
I tried with gsub, but use body() as "string" is no efficient.
function pre_backend( req )
print('--Backend response, pre-logic:');
local r = req;
r:params('test','test');
r:query('lovelyquery')
r:body('test','test');
lolcal v = r:body():gsub('email', 'user_mail_1')
...
Is a way to parse "req" as a table, dict or something i can transform data?
Is another way to transform REQUEST data?
EXAMPLE WORKING WITH GSUB:
function pre_backend( req )
print('--Backend response, pre-logic:');
print('--req');
print(req);
print(type(req));
local r = req;
print('--body');
print(type(r:body()));
print(r:body())
local body_transformed = r:body():gsub('email', 'user_mail_1');
print('--body_transformed');
print(body_transformed);
print(type(body_transformed));
end
Console output:
2022/02/11 09:59:52 DEBUG: [http-server-handler: no extra config]
--Backend response, pre-logic:
--req
userdata: 0xc0004f9b60
userdata
--body
string
{"email" : "test#test.es","test_field":"email"}
--body_transformed
{"user_mail_1" : "test#test.es","test_field":"user_mail_1"}
string
As we can see the gsub is not efficient becouse replace all strings.
If I can work with req as table, dict or something similar, I can replace dict key/value. ex: req['xxx] = 'xxx' or iterate req.keys
gsub stands for global substitution. It replaces all occurances of the pattern in the string.
If you just want to replace "email" infront of an email address simply use a pattern that takes this into account.
print((r:body():gsub('(")(email)("%s-:%s-"%w+#%w+%.%w+")', "%1user_mail_1%3")))
Alternatively if you knwo that you only want to replace the first occurance of email you can simply do this:
print((r:body():gsub("email", "user_mail_1", 1)))
The thrid parameter will stop gsub after the first replacement.
I’m new to telegraf and influxdb, and currently looking forward to exploring telegraf, but unfortunetly, I have some difficulty getting started, I will try to explain my problem below:
Objectif: parsing JSON file using telegraf input plugin.
Input : https://wetransfer.com/downloads/0abf7c609d000a7c9300dc20ee0f565120200624164841/ab22bf ( JSON file used )
The input json file is a repetition of the same structure that starts from params and ends at it.
you find below the main part of the input file :
{
"events":[
{
"params":[
{
"name":"element_type",
"value":"Home_Menu"
},
{
"name":"element_id",
"value":""
},
{
"name":"uuid",
"value":"981CD435-E6BC-01E6-4FDC-B57B5CFA9824"
},
{
"name":"section_label",
"value":"HOME"
},
{
"name":"element_label",
"value":""
}
],
"libVersion":"4.2.5",
"context":{
"locale":"ro-RO",
"country":"RO",
"application_name":"spresso",
"application_version":"2.1.8",
"model":"iPhone11,8",
"os_version":"13.5",
"platform":"iOS",
"application_lang_market":"ro_RO",
"platform_width":"320",
"device_type":"mobile",
"platform_height":"480"
},
"date":"2020-05-31T09:38:55.087+03:00",
"ssid":"spresso",
"type":"MOBILEPAGELOAD",
"user":{
"anonymousid":"6BC6DC89-EEDA-4EB6-B6AD-A213A65941AF",
"userid":"2398839"
},
"reception_date":"2020-06-01T03:02:49.613Z",
"event_version":"v1"
}
Issue : Following the documentation, I tried to define a simple telegraf.conf file as below:
[[outputs.influxdb_v2]]
…
[[inputs.file]]
files = ["/home/mouhcine/json/file.json"]
json_name_key = "My_json"
#... Listing all the string fields in the json.(I put only these for simplicity reason).
json_string_fields = ["ssid","type","userid","name","value","country","model"]
data_format = "json"
json_query= "events"
Basically declaring string fields in the telegraf.conf file would do it, but I couldn’t get all the fields that are subset in the json file, like for example what’s inside ( params or context ).
So finally, I get to parse fields with the same level of hierarchy as ssid, type, libVersion, but not the ones inside ( params, context, user).
Output : Screen2 ( attachment ).
OUTPUT
By curiosity, I tried to test the documentation’s example, in order to verify whether I get the same expected result, and the answer is no :/, I don’t get to parse the string field in the subset of the file.
The doc’s example below:
Input :
{
"a": 5,
"b": {
"c": 6,
"my_field": "description"
},
"my_tag_1": "foo",
"name": "my_json"
}
telegraf.conf
[[outputs.influxdb_v2]]
…
[[inputs.file]]
files = ["/home/mouhcine/json/influx.json"]
json_name_key = "name"
tag_keys = ["my_tag_1"]
json_string_fields = ["my_field"]
data_format = "json"
Expected Output : my_json,my_tag_1=foo a=5,b_c=6,my_field="description"
The Result I get : "my_field" is missing.
Output: Screen 1 ( attachement ).
OUTPUT
By the way, I use the influxdb cloud 2, and I apologize for the long description of this little problem, I would appreciate some help please :), Thank you so much in advance.
I'm getting same error as this question, but with XQuery:
SaxonApiException: The context item for axis step ./CLIENT is absent
When running from the command line, all is good. So I don't think there is a syntax problem with the XQuery itself. I won't post the input file unless needed.
The XQuery is displayed with a Console.WriteLine before the error appears:
----- Start: XQUERY:
(: FLWOR = For Let Where Order-by Return :)
<MyFlightLegs>
{
for $flightLeg in //FlightLeg
where $flightLeg/DepartureAirport = 'OKC' or $flightLeg/ArrivalAirport = 'OKC'
order by $flightLeg/ArrivalDate[1] descending
return $flightLeg
}
</MyFlightLegs>
----- End : XQUERY:
Error evaluating (<MyFlightLegs {for $flightLeg in root/descendant::FlightLeg[DepartureAirport = "OKC" or ArrivalAirport = "OKC"] ... return $flightLeg}/>) on line 4 column 20
XPDY0002: The context item for axis step root/descendant::FlightLeg is absent
I think that like the other question, maybe my input XML file is not properly specified.
I took the samples/cs/ExamplesHE.cs run method of the XQuerytoStream class.
Code there for easy reference is:
public class XQueryToStream : Example
{
public override string testName
{
get { return "XQueryToStream"; }
}
public override void run(Uri samplesDir)
{
Processor processor = new Processor();
XQueryCompiler compiler = processor.NewXQueryCompiler();
compiler.BaseUri = samplesDir.ToString();
compiler.DeclareNamespace("saxon", "http://saxon.sf.net/");
XQueryExecutable exp = compiler.Compile("<saxon:example>{static-base-uri()}</saxon:example>");
XQueryEvaluator eval = exp.Load();
Serializer qout = processor.NewSerializer();
qout.SetOutputProperty(Serializer.METHOD, "xml");
qout.SetOutputProperty(Serializer.INDENT, "yes");
qout.SetOutputStream(new FileStream("testoutput.xml", FileMode.Create, FileAccess.Write));
Console.WriteLine("Output written to testoutput.xml");
eval.Run(qout);
}
}
I changed to pass the Xquery file name, the xml file name, and the output file name, and tried to make a static method out of it. (Had success doing the same with the XSLT processor.)
static void DemoXQuery(string xmlInputFilename, string xqueryInputFilename, string outFilename)
{
// Create a Processor instance.
Processor processor = new Processor();
// Load the source document
DocumentBuilder loader = processor.NewDocumentBuilder();
loader.BaseUri = new Uri(xmlInputFilename);
XdmNode indoc = loader.Build(loader.BaseUri);
XQueryCompiler compiler = processor.NewXQueryCompiler();
//BaseUri is inconsistent with Transform= Processor?
//compiler.BaseUri = new Uri(xqueryInputFilename);
//compiler.DeclareNamespace("saxon", "http://saxon.sf.net/");
string xqueryFileContents = File.ReadAllText(xqueryInputFilename);
Console.WriteLine("----- Start: XQUERY:");
Console.WriteLine(xqueryFileContents);
Console.WriteLine("----- End : XQUERY:");
XQueryExecutable exp = compiler.Compile(xqueryFileContents);
XQueryEvaluator eval = exp.Load();
Serializer qout = processor.NewSerializer();
qout.SetOutputProperty(Serializer.METHOD, "xml");
qout.SetOutputProperty(Serializer.INDENT, "yes");
qout.SetOutputStream(new FileStream(outFilename,
FileMode.Create, FileAccess.Write));
eval.Run(qout);
}
Also two questions regarding "BaseURI".
1. Should it be a directory name, or can it be same as the Xquery file name?
2. I get this compile error: "Cannot implicity convert to "System.Uri" to "String".
compiler.BaseUri = new Uri(xqueryInputFilename);
It's exactly the same thing I did for XSLT which worked. But it looks like BaseUri is a string for XQuery, but a real Uri object for XSLT? Any reason for the difference?
You seem to be asking a whole series of separate questions, which are hard to disentangle.
Your C# code appears to be compiling the query
<saxon:example>{static-base-uri()}</saxon:example>
which bears no relationship to the XQuery code you supplied that involves MyFlightLegs.
The MyFlightLegs query uses //FlightLeg and is clearly designed to run against a source document containing a FlightLeg element, but your C# code makes no attempt to supply such a document. You need to add an eval.ContextItem = value statement.
Your second C# fragment creates an input document in the line
XdmNode indoc = loader.Build(loader.BaseUri);
but it doesn't supply it to the query evaluator.
A base URI can be either a directory or a file; resolving relative.xml against file:///my/dir/ gives exactly the same result as resolving it against file:///my/dir/query.xq. By convention, though, the static base URI of the query is the URI of the resource (eg file) containing the source query text.
Yes, there's a lot of inconsistency in the use of strings versus URI objects in the API design. (There's also inconsistency about the spelling of BaseURI versus BaseUri.) Sorry about that; you're just going to have to live with it.
Bottom line solution based on Michael Kay's response; I added this line of code after doing the exp.Load():
eval.ContextItem = indoc;
The indoc object created earlier is what relates to the XML input file to be processed by the XQuery.
I am writing a csv reader to generate Genbank files to capture annotations with sequence.
First I used a Bio.SeqRecord and got correctly formatted output but the SeqRecord class lacks fields that I need.
Blockquote
FEATURES Location/Qualifiers
HCDR1 27..35
HCDR2 50..66
HCDR3 99..109
I switched to Bio.GenBank.Record and have the needed fields except now the annotation formatting is wrong. It can't have the extra "type:" "location:" and "qualifiers:" text and the information should all be on one line.
Blockquote
FEATURES Location/Qualifiers
type: HCDR1
location: [26:35]
qualifiers:
type: HCDR2
location: [49:66]
qualifiers:
type: HCDR3
location: [98:109]
qualifiers:
The code for pulling annotations is the same for both versions. Only the class changed.
# Read csv entries and create a container with the data
container = Record()
container.locus = row['Sample']
container.size = len(row['Seq'])
container.residue_type="PROTEIN"
container.data_file_division="PRI"
container.date = (datetime.date.today().strftime("%d-%b-%Y")) # today's date
container.definition = row['FullCloneName']
container.accession = [row['Vgene'],row['HCDR3']]
container.version = getpass.getuser()
container.keywords = [row['ProjectName']]
container.source = "test"
container.organism = "Homo Sapiens"
container.sequence = row['Seq']
annotations = []
CDRS = ["HCDR1", "HCDR2", "HCDR3"]
for CDR in CDRS:
start = row['Seq'].find(row[CDR])
end = start + len(row[CDR])
feature = SeqFeature(FeatureLocation(start=start, end=end), type=CDR)
container.features.append(feature)
I have looked at the source code for Bio.Genbank.Record but can't figure out why the SeqFeature class has different formatting output compared to Bio.SeqRecord.
Is there an elegant fix or do I write a separate tool to reformat the annotations in the Genbank file?
After reading the source code again, I discovered Bio.Genbank.Record has its own Features method that takes key and location as strings. These are formatted correctly in the output Genbank file.
CDRS = ["HCDR1", "HCDR2", "HCDR3"]
for CDR in CDRS:
start = row['Seq'].find(row[CDR])
end = start + len(row[CDR])
feature = Feature()
feature.key = "{}".format(CDR)
feature.location = "{}..{}".format(start, end)
container.features.append(feature)
I'm converting some forms within my Rails 3.2 app to use AngularJS so that I can do live calculations and such. In my rails app I use money-rails to handle the currency. This treats all currency fields as integers made of of cents.
This becomes a problem when I send all the information through JSON over to my AngularJS template. Now my form is all in cents when I want them in dollars and cents.
I've put the conversion in my AngularJS controller so when I get the data from the server I convert it from cents to dollars & cents and vice vesa just before updating. Here is the code:
# Edit Investor
window.InvestorEditCtrl = ($scope, $routeParams, $location, Investor, Common) ->
console.log 'InvestorEditCtrl'
# Setup variable for common services(factory)
$scope.common = Common
$scope.master = {} # Initialise our main object hash
investor_id = $routeParams.investor_id # Get the ID of the investor we are editing from the URL
# Get the investor information & assign it to the scope master object
console.log 'Get JSON'
$scope.investor = new Investor.show({investor_id: investor_id}, (resource) ->
# copy the response from server response (JSON format) to the scopes master
$scope.master = angular.copy(resource)
# Convert price_cents to dollars
$scope.investor.price_cents /= 100
)
# Update the investor passing the JSON back to the server.
$scope.update = (investor) ->
# Convert price_cents to cents
investor.price_cents *= 100
$scope.master = angular.copy(investor)
investor.$update({investor_id: investor_id}, (t) ->
$location.path('/investors/' + t.id)
)
Is there a better way to do this?
You could write either a filter or a directive that converts it to the form you want in your HTML. The filter would look something like this:
app.filter('centsToDollars', function() {
return function(input) {
var out = input / 100;
return out;
}
});
Then, in your html, wherever you want the cents displayed dollars and cents, call it like this:
<p>{{investor.price_cents | centsToDollars}}</p>
The filter would only impact the display of the data and wouldn't modify the underlying data from being cents.
If you needed to modify the display of an input field, the better route would probably be a directive. You could do something like what's referenced here
app.directive('myCentsToDollars', function() {
return {
restrict: 'A',
require: 'ngModel',
link: function(scope, elem, attrs, ngModel) {
var toDollars = function(text) {
var text = (text || "0");
return (parseFloat(text) / 100);
}
var toCents = function(text) {
var text = (text || "0");
return (parseFloat(text) * 100);
}
ngModel.$parsers.push(toDollars);
ngModel.$formatters.push(toCents);
}
}
});
Then, in your html, do:
<input type="text" my-cents-to-dollars ng-model="investor.price_cents" />