I am new to Kraken but quite excited about it. What I am trying to do currently is to have two sequential backends where I would like to use parts of the response from the first backend to populate the body of the second request using Lua. the only issue I am having is to get a hold of the data that I can get via {resp0} if I use it outside the Lua scripts.
This is the part of the krakend.json where I would like to access the previous response:
"extra_config": {
"modifier/lua-backend": {
"sources": [ "/etc/krakend/config/lua/enrichRequestBody.lua" ],
"pre": " populate_request_body(request.load(), {resp0});",
"live": false
}
Any input or suggestions would be much appreciated!
Thanks
I have been trying finding {resp0} or the corresponding value from debugging but no luck. I haven't found any documentation or examples for this in the Kraken documentation
Related
I have recently starting doing things with a raspberry pi zero W and wanted to be able to call a number from it.
Unfortunately it seems very hard to use the normal Twilio library because the pi uses MicroPython so I have to use the raw API.
I also gathered that the API uses the content type x-www-form-urlencoded which urequests seems to have a hard time interacting with.
This is my code so far
url = f"https://api.twilio.com/2010-04-01/Accounts/{TWILIO_USER}/Calls.json"
def call() -> None:
body = {"To": CALLING_NUMBER, "From": CALLER_NUMBER}
response = urequests.post(url, json=body, auth=(TWILIO_USER, TWILIO_KEY))
print("Status Code", response.status_code)
print("JSON Response ", response.json())
however I get the error
{'code': 21201, 'more_info': 'https://www.twilio.com/docs/errors/21201', 'message': "No 'To' number is specified", 'status': 400}
I have tried a ton of stuff like url encoding the body myself, using it as the parameter for data, json.dumpsing it, nothing seemed to work. Any help would be greatly appreciated.
Please remember I am limited by the picos standard libraries.
It seems I was close. All I had to do was copy paste urllib's urlencode and use it on body, then it ended up working.
I am trying to follow this youtube tutorial.
I am getting stuck at inserting the first piece of data. Ania demonstrates it at 20.46 as follows:
mutation insertGenres {
action: insertreference_list(value: {label: "genre", value: "action"}) {
value{
value
},
}
When I try this, I get an error that says:
{
"errors": [
{
"message": "Validation error of type FieldUndefined: Field 'insertreference_list' in type 'Mutation' is undefined # 'insertreference_list'",
"locations": [
{
"line": 2,
"column": 3
}
],
"extensions": {
"classification": "ValidationError"
}
}
]
}
When I google the error, a lot of responses tell people to use mutations instead of queries - but I've started from a mutation. I would like to know how to resolve the error, but I'd also like to find the skills to improve my search strategy for finding answers.
When I look at the documentation for using GraphQL with DataStax, I see a different format to the write structure, which is as follows:
insertbook(value: bookInput!, ifNotExists: Boolean, options:
UpdateOptions): bookMutationResult
It has a colon and a fragment of text after it. It also explicitly states the ifNotExists: Boolean and options. I don't know if there may have been a change to how to use DataStax since the time Ania recorded the tutorial that means it is no longer a current demonstration of how to use the tool, or if there is an answer for this and I just haven't found it yet.
You didn't provide details of how you've configured your Astra DB for Ania's Netflix Clone tutorial so I'm going to assume that you've named your keyspace as netflix.
It seems as though you haven't followed the instructions correctly and have missed steps. I can replicate the error you reported if I skip at least one of the steps in the tutorial.
In step 5 of the tutorial, you needed to do the following:
✅ In graphQL playground, change tab to now use graphql. Edit the end of the URl to change from system to the name of your keyspace: netflix
✅ Populate HTTP HEADER variable x-cassandra-token on the bottom of the page with your token as shown below (again !! yes this is not the same tab)
Switch tabs
In order to insert data, you need to switch to the graphql tab.
If you try to insert the data in the graphql-schema tab, you will get the error you reported.
Set keyspace
You need to update the URI of your GraphQL playground in the graphql tab to use the keyspace name netflix instead of system. For example:
https://db_id-us-west1.apps.astra.datastax.com/api/graphql/system
change to:
https://db_id-us-west1.apps.astra.datastax.com/api/graphql/netflix
If you try to insert data into the system keyspace, you will get the error you reported because the reference_list table does not exist in that keyspace. Cheers!
The problem:
I have used rest-client to pull news articles from newsapi.org through my Rails backend. I can display that data on localhost:3000/articles. I do not (necessarily) want to save this data to a database and would like to simply display it on my Ember frontend after calling it on my Rails backend. I get this error:
Error while processing route: article Assertion Failed: You must include an 'id' for article in an object passed to 'push'
I understand this to mean that my data has no 'id' - which it doesn't. I have no control over the data coming in.
{
"status": "ok",
"totalResults": 6868,
"articles": [
{
"source": {
"id": "mashable",
"name": "Mashable"
},
"author": "Lacey Smith",
"title": "Cannabis may alter the genetic makeup of sperm",
"description": "A new study suggests that marijuana use can not
only lower sperm count, but also affect sperm DNA. Read more... More
about Marijuana, Cannabis, Sperm, Science, and Health",
"url": "https://mashable.com/video/cannabis-sperm-dna/",
"content": null
},
Ember-Data doesn't like this because it's not in JSONAPI format and I can't change it to JSONAPI. How do I display this data on my ember frontend without using Ember-Data and without having to save it into my database? The tutorials I've found all have examples using databases and Ember-Data.
What I've already tried:
I tried to save it to my database and load from there, but I'm thinking that might complicate the idea, since I would need to scrape the data and save it to my database and that has proven difficult to do. I'm not experienced with backend stuff and haven't found many answers. If you have an answer for doing it this way, feel free to answer here:
How to automatically save data from url to database
I'm willing to try it out. I thought originally this would be the best way to go. I could save the data and only add to the database when new articles were returned. I'm not having any luck this way, so I thought I'd approach it from another direction.
Thanks for any help!
If you want to do some processing outside of ember-data, create a service. In your service, you will need to then make an ajax/fetch request to your backend. Let's say you are using ember-ajax to make your requests:
import Service, { inject } from '#ember/service';
export default Service.extend({
// inject the ember-ajax ajax service into your service
ajax: inject(),
//this returns a promise
getNewsItems(){
var options = {
// request options (aka anything needed to authenticate against your api, etc)
};
let url = "yourBackendEndpoint/for/this/resource";
return request(url, options).then(response => {
// `response` is the data from the server
// here you want to convert the JSON into something useable
return response;
});
}
});
Then, let's say in a model hook for some route, you needed this service (which we will say is defined in your-app/services/my-service.js) to fetch the news feed items:
import Route from '#ember/route';
import { inject } from '#ember/service';
export default Route.extend({
myService: inject(),
model(){
let myService = this.get('myService');
return myService.getNewsItems();
}
)
I personally use ember-fetch which avoids the jQuery dependency and is useable in all target browsers for my app. I do not use ember-data in any of my apps.
I cannot consistently successfully send form variables that may/may not include special characters e.g. ? & #
Depending on where I try to escape the chars I encounter different errors when reading the data server-side.
I am aware that an update is due for React Native 0.7 to include formdata but wondered if I could safely post objects without needing this.
Someone has already posted a similar issue but no example code was posted to illustrate the POST working:
How to post a form using fetch in react native?
I have tried - amongst other things :
fetch(APIURL, {
method: 'POST',
body: JSON.stringify({
object1: {
param1a: "value 1a",
param1b: "value 1b - with bad chars & # ?",
},
object2:
{
param2a: "value 2a",
param2b: 0,
}
})
})
but it groups the data into a single unnamed parameter (changing the API to accept this is not an option).
also this:
fetch(APIURL, {
method: 'GET',
accessPackage: JSON.stringify({
accessToken: "abc123",
tokenType: 2,
}),
taggData: JSON.stringify({
title: "test",
wishlistID: 0,
anotherVar: "anotherVal"
})
})
I wish to receive the data as two strings that can be parsed as json objects at the other end.
Looking at the the fetch repo https://github.com/github/fetch hasn't helped as this assumes the post with be a full JSON post (which it isn't) or uses FormData which isn't available to React Native yet.
Another solution may be to safely encode/serialise all of the data to URL parameters but this has also proven inconsistent so far especially with the # char.
What's the best way to do this?
"it groups the data into a single unnamed parameter (changing the API
to accept this is not an option)."
It would, because you've set the post body. This is how it's supposed to work.
I wish to receive the data as two strings that can be parsed as json objects at the other end.
You can do whatever you want, but there's no magic happening here. You will receive a single string, the string you set body to. Likewise, the post body can contain anything but shouldn't get confused with "special" characters. Most likely it is your server-side that is causing the problems.
If you want to use FormData then I think it'll be in v0.7.0 which should be out any day now, or you could probably just include the JS file in your own project. You can find it here. Usage examples are in the UIExplorer demo.
Im trying to create a table on BigQuery - I have a single dataset and need to use the api to add a table and import data (json.tar.gz) from cloud storage. I need to be able to use the ruby client to automate the whole process. I have two questions:
I have read the docs and tried to get it to upload (code below) and have not been successful and have absolutely no idea what Im doing wrong. Could somebody please enlighten me or point me in the right direction?
Once I make the request, how do I know when the job has actually finished? From the API, I presume Im meant to use a jobs.get request? Having not completed the first part I have been unable to get to look at this aspect.
This is my code below.
config= {
'configuration'=> {
'load'=> {
'sourceUris'=> ["gs://person-bucket/person_json.tar.gz"],
'schema'=> {
'fields'=> [
{ 'name'=>'person_id', 'type'=>'integer' },
{ 'name'=> 'person_name', 'type'=>'string' },
{ 'name'=> 'logged_in_at', 'type'=>'timestamp' },
]
},
'destinationTable'=> {
'projectId'=> "XXXXXXXXX",
'datasetId'=> "personDataset",
'tableId'=> "person"
},
'createDisposition' => 'CREATE_IF_NEEDED',
'maxBadRecords'=> 10,
}
},
'jobReference'=>{'projectId'=>XXXXXXXXX}
}
multipart_boundary="xxx"
body = "--#{multipart_boundary}\n"
body += "Content-Type: application/json; charset=UTF-8\n\n"
body += "#{config.to_json}\n"
body += "--#{multipart_boundary}\n"
body +="Content-Type: application/octet-stream\n\n"
body += "--#{multipart_boundary}--\n"
param_hash = {:api_method=> bigquery.jobs.insert }
param_hash[:parameters] = {'projectId' => 'XXXXXXXX'}
param_hash[:body] = body
param_hash[:headers] = {'Content-Type' => "multipart/related; boundary=#{multipart_boundary}"}
result = #client.execute(param_hash)
puts JSON.parse(result.response.header)
I get the following error:
{"error"=>{"errors"=>[{"domain"=>"global", "reason"=>"wrongUrlForUpload", "message"=>"Uploads must be sent to the upload URL. Re-send this request to https://www.googleapis.com/upload/bigquery/v2/projects/XXXXXXXX/jobs"}], "code"=>400, "message"=>"Uploads must be sent to the upload URL. Re-send this request to https://www.googleapis.com/upload/bigquery/v2/projects/XXXXXXXX/jobs"}}
From the request header, it appears to be going to the same URI the error says it should go to, and I am quite at a loss for how to proceed. Any help would be much appreciated.
Thank you and have a great day!
Since this is a "media upload" request, there is a slightly different protocol for making the request. The ruby doc here http://rubydoc.info/github/google/google-api-ruby-client/file/README.md#Media_Upload describes it in more detail. I'd use resumable upload rather than multipart because it is simpler.
Yes, as you suspected, the way to know when it is done is to do a jobs.get() to look up the status of the running job. The job id will be returned in the response from jobs.insert(). If you want more control, you can pass your own job id, so that in the event that the jobs.insert() call returns an error you can find out whether the job actually started.
Thank you for that. Answer resolved. Please see here :
How to import a json from a file on cloud storage to Bigquery
I think that the line of code in the docs for the resumable uploads section (http://rubydoc.info/github/google/google-api-ruby-client/file/README.md#Media_Upload) should read:
result = client.execute(:api_method => drive.files.insert,
Otherwise, this line will throw an error with 'result' undefined:
upload = result.resumable_upload