Transform request data in Krakrnd with lua - lua

Using Krakend as api gateway.
I have an endpoint configured in krakend.json:
"endpoint":"/call",
"extra_config":{
"github.com/devopsfaith/krakend-lua/proxy":{
"sources":[
"/function.lua"
],
"pre":"pre_backend(request.load())",
"live":true,
"allow_open_libs":true
}
},
"method":"POST",
"output_encoding":"json",
"headers_to_pass":[
"*"
],
"backend":[
{
"url_pattern":"/api/v1/get_client_id",
[...]]
},
The endopont "/api/v1/get_client_id" recives just a param:
{"user_mail_1":"test#test.es"}
I want, whith the lua script my endopoint "/call" recives:
{"email":"test#test.es"}
and transform on before send:
{"user_mail_1":"test#test.es"}
I tried with gsub, but use body() as "string" is no efficient.
function pre_backend( req )
print('--Backend response, pre-logic:');
local r = req;
r:params('test','test');
r:query('lovelyquery')
r:body('test','test');
lolcal v = r:body():gsub('email', 'user_mail_1')
...
Is a way to parse "req" as a table, dict or something i can transform data?
Is another way to transform REQUEST data?
EXAMPLE WORKING WITH GSUB:
function pre_backend( req )
print('--Backend response, pre-logic:');
print('--req');
print(req);
print(type(req));
local r = req;
print('--body');
print(type(r:body()));
print(r:body())
local body_transformed = r:body():gsub('email', 'user_mail_1');
print('--body_transformed');
print(body_transformed);
print(type(body_transformed));
end
Console output:
2022/02/11 09:59:52 DEBUG: [http-server-handler: no extra config]
--Backend response, pre-logic:
--req
userdata: 0xc0004f9b60
userdata
--body
string
{"email" : "test#test.es","test_field":"email"}
--body_transformed
{"user_mail_1" : "test#test.es","test_field":"user_mail_1"}
string
As we can see the gsub is not efficient becouse replace all strings.
If I can work with req as table, dict or something similar, I can replace dict key/value. ex: req['xxx] = 'xxx' or iterate req.keys

gsub stands for global substitution. It replaces all occurances of the pattern in the string.
If you just want to replace "email" infront of an email address simply use a pattern that takes this into account.
print((r:body():gsub('(")(email)("%s-:%s-"%w+#%w+%.%w+")', "%1user_mail_1%3")))
Alternatively if you knwo that you only want to replace the first occurance of email you can simply do this:
print((r:body():gsub("email", "user_mail_1", 1)))
The thrid parameter will stop gsub after the first replacement.

Related

Different output from encodeURIComponent vs URLSearchParams

I built an oauth2 url with query params using URLSearchParmas API. However, the output URL didn't return an expected url. Can anyone help me understand the difference between those two APIs and how can I get the same result as the output of encodeURIComponent, using URLSearchParams? Thanks!
const expected = encodeURIComponent('code id_token'); // code%20id_token
const search = new URLSearchParams();
search.set('response_type', 'code id_token');
search.toString(); // code+id_token
According to WHATWG, URLSearchParams uses application/x-www-form-urlencoded format. While it's suitable for decoding URL queries, for encoding it can lead to unexpected results such as spaces being encoded as + and extra characters such as ~ being percent-encoded. It's better to use encodeURIComponent instead:
Having an object:
const params = {
text1: 'aaa bbb',
text2: '-._*~()'
}
Instead of:
url.search = (new URLSearchParams(params)).toString()
Use:
url.search = Object.entries(params)
.map(([key, value]) => `${encodeURIComponent(key)}=${encodeURIComponent(value)}`)
.join('&')
Also, according to MDN even encodeURIComponent doesn't conform to newer RFC 3986 which defines more characters to escape, for example *. While it's probably safe not to escape these additional characters if you aren't using them as field separators, if you want to be strictly conformant to latest RFC, use this updated implementation from MDN:
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16).toUpperCase();
});
}
A playground for experimenting:
const params = {
text1: 'aaa bbb',
text2: '-._*~()'
}
const url1 = new URL('http://example.com')
const search1 = new URLSearchParams(params)
url1.search = search1 // Incorrect
console.log('URLSearchParams', url1.toString())
function fixedEncodeURIComponent(str) {
return encodeURIComponent(str).replace(/[!'()*]/g, function(c) {
return '%' + c.charCodeAt(0).toString(16).toUpperCase()
})
}
const url2 = new URL('http://example.com')
const search2 = Object
.entries(params)
.map(([key, value]) => `${fixedEncodeURIComponent(key)}=${fixedEncodeURIComponent(value)}`)
.join('&')
url2.search = search2 // Correct
console.log('fixedEncodeURIComponent', url2.toString())

hmac authentication in vcl

I am trying to authenticate URL using hamc. I can do the following to verify.My question is how do I parse the URL to extract only part of the URL excluding the hmac parameter. I tried using local variables in vcl but it threw an error.
Any suggestions on how to extract the hmac value and URL query parameters as shown below.
http://localhost/zzz/?q1=xxx&q2=yyy&hmac=hash
if (digest.hmac_md5("key", "q1=xxx&q2=yyy") != "value")
{
return (synth(401, digest.hmac_md5("key", "http://localhost/zzz/?q1=xxx&q2=yyy")));
}
Thanks
You'll want to use the [querystring][1] vmod. As far as I know it does not come pre-packaged, so you will need to build it but it should do exactly what you need.
With that you can define regexes/static values to match querystring arguments, and then filter those out or in.
there is no need for a external plugin in that case you can just strip out the hmac=XXX query string parameter, from req.url and store the result in a new variable req.http.url_without_hmac and req.http.hmac to the digest.hmac_md5
see a sample test case:
varnishtest "Strip query parameter"
server s1 {
rxreq
txresp
rxreq
txresp
} -start
varnish v1 -vcl+backend {
import std;
sub vcl_recv {
# Strip out HMAC parameter
# get only the query string, ignore uri
set req.http.qs = regsuball(req.url, ".*\?(.*?)$", "?\1");
# strip hmac= from the qs
set req.http.url_without_hmac = regsuball(req.http.qs,"\?hmac=[^&]+$",""); # strips when QS = "?hmac=AAA"
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"\?hmac=[^&]+&","?"); # strips when QS = "?hmac=AAA&foo=bar"
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"&hmac=[^&]+",""); # strips when QS = "?foo=bar&hmac=AAA" or QS = "?foo=bar&hmac=AAA&bar=baz"
# remove the leading ? from the url_without_hmac
set req.http.url_without_hmac = regsuball(req.http.url_without_hmac,"^\?(.*)$", "\1");
# extract the hmac= value from the req.http.qs
set req.http.hmac = regsuball(req.http.qs, ".*[?&]hmac=([^&]*).*", "\1");
# NOW USE req.http.url_without_hmac for your digest validation and req.http.hmac as the value
}
sub vcl_deliver {
set resp.http.url_without_hmac = req.http.url_without_hmac;
set resp.http.hmac = req.http.hmac;
}
} -start
client c1 {
txreq -url "/1?a=1&hmac=2&b=1"
rxresp
expect resp.http.url_without_hmac == "a=1&b=1"
expect resp.http.hmac == "2"
} -run
client c2 {
txreq -url "/1?hmac=hello&a=1&b=1"
rxresp
expect resp.http.url_without_hmac == "a=1&b=1"
expect resp.http.hmac == "hello"
} -run

Capture full response in Lua Socket call

I am trying to call a REST API through LUA. However, I am not able to capture full raw response returned by the API. Below is the code sample:
local http_socket = require("socket.http")
local pretty_print = require("pl.pretty")
local header = {
["x-device-type"] = "M",
["authorization"] = "ashdjkashd",
["x-app-secret"] = "asdasda",
["x-user-id"] = "asdasdasd"
}
r, c, h = http_socket.request {
method = "GET", -- Validation API Method
url = "http://google.com", -- Validation API URL
headers = header
}
print(r .. c)
pretty_print.dump(h)
I'm using lua 5.3, and luarocks version=2.4.1.
In variable c i am getting code, and in h there are a few headers. I need to capture full response returned by the API.
As you may know, luasocket's http.request supports two forms of usage. I'm assuming you need the second form to customize the resty request for that particular API.
In this case to capture the response body you'll need to use the sink field with ltn12.sink module. For example
local ltn12 = require 'ltn12'
-- ...
local res = {}
r, c, h, s = http_socket.request
{
method = "GET", -- Validation API Method
url = "http://google.com", -- Validation API URL
headers = header,
sink = ltn12.sink.table(res)
}
res = table.concat(res)
print(res)
The table.concat is needed since the response could be comprised of multiple chunk sizes(appended to res as it's received).
You can also write it out to file by replacing above with ltn12.sink.file, eg. using ltn12.sink.file(io.stdout) will dump the response to standard output.

How to convert string into a table in Lua

I am having a table data in string form. Sample is given below:
{"engName1":"HOLDER","validDurPeriod":3,"engName2":"INFORMATION","appStatus":2,"stayExpDate":"01/10/2012","engName3":"","appExpDate":"12/04/2010"}
How can I convert it into a proper table type variable so that I can access keys.I am new to lua and I am not aware if there is any existing method to do so.
There is plenty of JSON parsers available for Lua, for example dkjson:
local json = require ("dkjson")
local str = [[
{
"numbers": [ 2, 3, -20.23e+2, -4 ],
"currency": "\u20AC"
}
]]
local obj, pos, err = json.decode (str, 1, nil)
if err then
print ("Error:", err)
else
print ("currency", obj.currency)
for i = 1,#obj.numbers do
print (i, obj.numbers[i])
end
end
Output:
currency €
1 2
2 3
3 -2023
4 -4
Try this code to start with
J=[[
{"engName1":"HOLDER","validDurPeriod":3,"engName2":"INFORMATION","appStatus":2,"stayExpDate":"01/10/2012","engName3":"","appExpDate":"12/04/2010"}
]]
J=J:gsub("}",",}")
L={}
for k,v in J:gmatch('"(.-)":(.-),') do
L[k]=v
print(k,v)
end
You'll still need to convert some values to number and remove quotes.
Alternatively, you can let Lua do the hard work, if you trust the source string. Just replace the loop by this:
J=J:gsub('(".-"):(.-),','[%1]=%2,\n')
L=loadstring("return "..J)()

Lua String Split

Hi I've got this function in JavaScript:
function blur(data) {
var trimdata = trim(data);
var dataSplit = trimdata.split(" ");
var lastWord = dataSplit.pop();
var toBlur = dataSplit.join(" ");
}
What this does is it take's a string such as "Hello my name is bob" and will return
toBlur = "Hello my name is" and lastWord = "bob"
Is there a way i can re-write this in Lua?
You could use Lua's pattern matching facilities:
function blur(data) do
return string.match(data, "^(.*)[ ][^ ]*$")
end
How does the pattern work?
^ # start matching at the beginning of the string
( # open a capturing group ... what is matched inside will be returned
.* # as many arbitrary characters as possible
) # end of capturing group
[ ] # a single literal space (you could omit the square brackets, but I think
# they increase readability
[^ ] # match anything BUT literal spaces... as many as possible
$ # marks the end of the input string
So [ ][^ ]*$ has to match the last word and the preceding space. Therefore, (.*) will return everything in front of it.
For a more direct translation of your JavaScript, first note that there is no split function in Lua. There is table.concat though, which works like join. Since you have to do the splitting manually, you'll probably use a pattern again:
function blur(data) do
local words = {}
for m in string.gmatch("[^ ]+") do
words[#words+1] = m
end
words[#words] = nil -- pops the last word
return table.concat(words, " ")
end
gmatch does not give you a table right away, but an iterator over all matches instead. So you add them to your own temporary table, and call concat on that. words[#words+1] = ... is a Lua idiom to append an element to the end of an array.

Resources