Can't get "Location" attribute from response header in karate [duplicate] - response

This question already has an answer here:
How to get id in responseHeaders location?
(1 answer)
Closed last year.
I am not able to fetch location attribute from response header in karate, while other headers are getting displayed when using below code also from postman i am able to see all the headers including location
Background:
* url APAuthorizationURL
* def sessionTokenPayload = call read('DIGI-3049.feature')
* def authToken = sessionTokenPayload.session_Token
* configure cors = true
* print authToken
#Authorization
Scenario: Authorization_100200
Given header Content-Type = 'application/json'
* print authToken
* def query = {client_id: '0oajtiuveymO1ES7G0h7',response_type : 'token',response_mode: 'okta_post_message', redirect_uri: 'https%3A%2F%2Fqa2-sc.adviser.ad.mlclife.com.au', scope:'advisor',prompt:'none',state:'203475b6-836d-4dcb-8a12-7b2b2b3a2cf9', sessionToken : 'authToken', nonce:'158261204839000', scope : 'advisor' }
And params query
When method get
Then status 200
* print responseHeaders / location header is not printing
* print 'headers:', karate.prevRequest.headers / location header is not printing

This works fine for me. Please paste this into a new Scenario and try:
* url 'http://httpbin.org'
* path 'redirect-to'
* param url = 'http://httpbin.org/get'
* param status_code = 200
* method get
And the first request-response is:
19:04:20.612 [main] DEBUG com.intuit.karate - request:
1 > GET http://httpbin.org/redirect-to?url=http%3A%2F%2Fhttpbin.org%2Fget&status_code=200
1 > Accept-Encoding: gzip,deflate
1 > Connection: Keep-Alive
1 > Host: httpbin.org
1 > User-Agent: Apache-HttpClient/4.5.11 (Java/1.8.0_231)
19:04:21.287 [main] DEBUG com.intuit.karate - response time in milliseconds: 670.62
1 < 302
1 < Access-Control-Allow-Credentials: true
1 < Access-Control-Allow-Origin: *
1 < Connection: keep-alive
1 < Content-Length: 0
1 < Content-Type: text/html; charset=utf-8
1 < Date: Tue, 25 Feb 2020 13:34:21 GMT
1 < Location: http://httpbin.org/get
1 < Server: gunicorn/19.9.0
So you must be missing something or making the wrong request to the wrong end-point. Also make sure you read about the followRedirects configuration in the docs.

Related

Display image on browser, where image get from the API response

I am working on sample application, where I have to show generated QR code. I am using http://goqr.me/api/ for generating the API.
Here is my sample code
class HomeController < ApplicationController
def index
end
def submit_qr
code = params[:code]
url = "https://api.qrserver.com/v1/create-qr-code/?data=#{params[:code]}&size=220x220&margin=0"
#response = HTTParty.get(url).parsed_response
end
end
When I check the call from rails console I am getting following
irb(main):004:0> url = "https://api.qrserver.com/v1/create-qr-code/?data=test&size=220x220&margin=0"
=> "https://api.qrserver.com/v1/create-qr-code/?data=test&size=220x220&margin=0"
irb(main):005:0> response = HTTParty.get(url)
=> #<HTTParty::Response:0x8efada0 parsed_response="\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00\xDC\x00\x00\x00\xDC\x01\x03\x00\x00\x00!\x94]\xC2\x00\x00\x00\x06PLTE\xFF\xFF\xFF\x00\x00\x00U\xC2\xD3~\x00\x00\x01\x02IDATX\x85\xED\xD7\xBB\x11\x840\f\x04\xD0e\b\b)\xC1\xA5P\x1A\x94\xE6R(\x81\x90\x80A\x87\xFC\x05\x06\xCE\xC7\xA5\xAC\"\x8F\x1F\xD1\x0E\xB2lH\xAC\x01\xB5\x00\x8D\xA0\x8B;\x13\x88\x8A\x16Z\xB5\ff\x01\xCC\ft\xB3\xDBAO\f\xD8jRu\x88\x0F\x8A\xD5\xB6\xB3\x12\x9Fa\xCE\x96x\x13\x1F\\\xB3\x12\x9Faj\xDEmk\xD4\xE3\xED\xB2\xB3_\x8Ci\x00l\x7F\x9F\xC7\xDD\x00 \x16\xF0P\xBA\x85\xBBz+\x86\xA4j\xF1\xD0\x88\xD8fE7U\xDA\xB0\xC4\"Z\xE8i\xB6\xA0\x1F\xE1\xFF>(\xA6\xE3\x8D\xB8U\x15\xE6\xA7\x068\xC7\xAF\xC5M\x04b\tS\x8C&6n\xA3\xB4\xBA%1\xA1\xBF\xC2\xE6\xF8|\x7F\x12\x8B\x98\xCAH(\xBD~\xB8[\e\xF1t\xC3\x90\xF8\b?M\a\xE2\x17\xB4.\xD8\xF0\xC44!\xDB\xCE7/\xD1\xE1\xEE\x89\x99\xE7g\x8C\x8F\xF8\x17~\xCD\xF6\xCD\x18\a\x80m\x89?bl^\x97\xAD\xAE\xF5\x15#\xCC\x98\a\xC0!>9M\a\xE2\x15~\x00\xA9\xB7\xB6\x8EW\xEAb\xBE\x00\x00\x00\x00IEND\xAEB`\x82", #response=#<Net::HTTPOK 200 OK readbody=true>, #headers={"date"=>["Fri, 20 Mar 2020 10:33:39 GMT"], "server"=>["Apache/2"], "access-control-allow-origin"=>["*"], "access-control-allow-methods"=>["POST, GET, OPTIONS, DELETE, PUT"], "access-control-max-age"=>["7200"], "access-control-allow-headers"=>["Accept, Accept-Language, Content-Language, Content-Type, Origin, Authorization, X-Requested-With, Client-Security-Token"], "x-powered-by"=>["PHP/5.4.16"], "content-length"=>["333"], "connection"=>["close"], "content-type"=>["image/png"]}>
So how can I show the image on browser.
response = "\x89PNG\r\n\x1A\n\x00\x00\x00\rIHDR\x00\x00\x00\xDC\x00\x00\x00\xDC\x01\x03\x00\x00\x00!\x94]\xC2\x00\x00\x00\x06PLTE\xFF\xFF\xFF\x00\x00\x00U\xC2\xD3~\x00\x00\x01\x02IDATX\x85\xED\xD7\xBB\x11\x840\f\x04\xD0e\b\b)\xC1\xA5P\x1A\x94\xE6R(\x81\x90\x80A\x87\xFC\x05\x06\xCE\xC7\xA5\xAC\"\x8F\x1F\xD1\x0E\xB2lH\xAC\x01\xB5\x00\x8D\xA0\x8B;\x13\x88\x8A\x16Z\xB5\ff\x01\xCC\ft\xB3\xDBAO\f\xD8jRu\x88\x0F\x8A\xD5\xB6\xB3\x12\x9Fa\xCE\x96x\x13\x1F\\\xB3\x12\x9Faj\xDEmk\xD4\xE3\xED\xB2\xB3_\x8Ci\x00l\x7F\x9F\xC7\xDD\x00 \x16\xF0P\xBA\x85\xBBz+\x86\xA4j\xF1\xD0\x88\xD8fE7U\xDA\xB0\xC4\"Z\xE8i\xB6\xA0\x1F\xE1\xFF>(\xA6\xE3\x8D\xB8U\x15\xE6\xA7\x068\xC7\xAF\xC5M\x04b\tS\x8C&6n\xA3\xB4\xBA%1\xA1\xBF\xC2\xE6\xF8|\x7F\x12\x8B\x98\xCAH(\xBD~\xB8[\e\xF1t\xC3\x90\xF8\b?M\a\xE2\x17\xB4.\xD8\xF0\xC44!\xDB\xCE7/\xD1\xE1\xEE\x89\x99\xE7g\x8C\x8F\xF8\x17~\xCD\xF6\xCD\x18\a\x80m\x89?bl^\x97\xAD\xAE\xF5\x15#\xCC\x98\a\xC0!>9M\a\xE2\x15~\x00\xA9\xB7\xB6\x8EW\xEAb\xBE\x00\x00\x00\x00IEND\xAEB`\x82"
decoded_image = Base64.strict_encode64(response)
base64_img = "data:image/png;base64," + decoded_image
Something like this

Lua POST request contains garbage

I'm trying to send a POST request but noticed the endpoint I'm trying to access doesn't seem to like the request, so to investigate the situation I redirected the request to localhost while listening with nc and saw the following request:
nc -vlp 444
Connection from 127.0.0.1:53812
POST / HTTP/1.1
Host: localhost:4444
TE: trailers
Cookie:
Content-Type: application/x-www-form-urlencoded
Connection: close, TE
User-Agent: Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0
27 -- this line shouldn't be there
username=username&password=password
0 -- also this one
The code I'm using, just in case:
local http = require("socket.http") -- even tried ssl.https
...
function Session:post(url, payload) -- payload = "username=username&password=password"
local response = Response
local body = { }
local r, c, h, s = http.request{
url = url,
method = "POST",
sink = ltn12.sink.table(body),
source = ltn12.source.string(payload),
headers = {
["User-Agent"] = "Mozilla/5.0 (X11; Linux x86_64; rv:67.0) Gecko/20100101 Firefox/67.0",
["Content-Type"] = "application/x-www-form"
},
}
-- you can ignore this
response.code = c
response.status = s
response.body = table.concat(body)
self.cookies:parse(h["set-cookie"])
return response
end
Thinking it was a problem of ltn12 I copied the source code of it directly in my project but soon found out it only returned the payload without actually changing anything
BLOCKSIZE = 2048
function generate_payload(s)
if s then
local i = 1
return function()
local chunk = string.sub(s, i, i+BLOCKSIZE-1)
i = i + BLOCKSIZE
if chunk ~= "" then return chunk
else return nil end
end
else return source.empty() end
end
Managed to fix the issue by adding the Content-Length header.

ESP 8266 MicroPython getting partial data for POST

I'm trying to set up a simple HTTP web server on ESP8266-01 (1MB flash), which has latest 1.9.3 MicroPython firmware. The purpose is to be able to configure the credentials for the home network, to which the STA interface would ultimately connect.
So the code at high level does this:
Turns on the AP interface
Some one would connect to 192.168.0.1/index.html, which would have a form for username and password. We just need to put admin/admin. Clicking on Submit button should do a POST to 192.168.0.1/configure.html
Configure.html is a web page with a form, where SSID and password would be input. I hope you can follow more details by looking at the code below
I am facing two problems:
The total number of bytes that I receive when index.html form is submitted are not complete. I get to around Referrer (that too partial), all a total of around 560 bytes. This is when I do it from mobile browser. Funny thing, it always gets exactly those many bytes. I can share what I get if that's going to help.
The watch dog timer sometimes reboots my module. I'm doing most of the suggested changes in my code - using small sleeps. Is there any way in MicroPython on ESP8266, through which I can 'feed' the WDT so that it does not 'time-out' and reboot my module?
The following is my code:
import gc
import network
gc.collect()
import machine
gc.collect()
import ubinascii
gc.collect()
import ujson
gc.collect()
import uos
gc.collect()
import utime
gc.collect()
import socket
gc.collect()
import select
gc.collect()
html = """<!DOCTYPE html>
<html>
<head> <title>Ouroboros IoT Login</title> </head>
<body>
<form action="configure.html" method="POST">
Username : <input type="text" name="username"></br>
Password: <input type="password" name="password" ></br>
<input type="submit" value="submit" name="submit">
</form>
</body>
</html>
"""
login_fail_html = """<!DOCTYPE html>
<html>
<head> <title>Ouroboros IoT Login</title> </head>
<body>
<h2>Incorrect Credentials!</h2><br>Please login<br>
<form action="configure.html" method="POST">
Username : <input type="text" name="username"></br>
Password: <input type="password" name="password" ></br>
<input type="submit" value="submit" name="submit">
</form>
</body>
</html>
"""
# Check if file exists
def fileExists(fileName):
try:
uos.stat(fileName)
print("File " + fileName + " found!")
return True
except OSError:
print("No file " + fileName + " found!")
return False
# Turns WiFi ON for configuration
def turn_wifi_on():
# Setup the AP interface
ap_if = network.WLAN(network.AP_IF)
ap_if.active(False)
ap_if.active(True)
# Get the MACADDRESS - without any spaces
macaddress = ubinascii.hexlify(ap_if.config('mac'),'').decode()
ap_if.config(essid="OUB1_"+macaddress, password="12345678")
#ap_if.config(essid="OUB1_"+macaddress)
ap_if.ifconfig(('192.168.0.1', '255.255.255.0', '192.168.0.1', '192.168.0.1'))
# Configure the AP to static IPs
def turn_wifi_off():
ap_if = network.WLAN(network.AP_IF)
ap_if.active(False)
# Find out the stored IoT secret content
def get_iot_secret():
fileName = 'alpha.txt'
if fileExists(fileName):
f = open(fileName)
content_str = f.read()
f.close()
return content_str
else:
return 'asdasrefwefefergf9rerf3n4r23irn1n32f'
# Find out the stored home network credential if exist
def get_wifi_config():
fileName = 'wifi.conf'
if fileExists(fileName):
f = open(fileName)
content_str = f.read()
f.close()
content = ujson.loads(content_str)
return content
else:
return None
# Set the home network credentials
def save_wifi_config(essid, passphrase):
f = open('wifi.conf', 'w')
config = {'essid':essid, 'passphrase':passphrase}
config_str = ujson.dumps(config)
f.write(config_str)
f.close()
# Find out the stored login credentials
def get_login_config():
fileName = 'login.conf'
if fileExists(fileName):
f = open(fileName)
content_str = f.read()
f.close()
content = ujson.loads(content_str)
return content
else:
# No file exists so far, so use the admin/admin credentials
return {'user':'admin','password':'admin'}
# Set the login credentials
def save_login_config(user, password):
f = open('login.conf', 'w')
config = {'user':user, 'password':password}
config_str = ujson.dumps(config)
f.write(config_str)
f.close()
def turn_gpio_on(device_num):
# Device Num to Pin Mapping
if device_num == 0:
pin_num = 0
elif device_num == 1:
pin_num = 2
# Check Pin
pin = machine.Pin(pin_num)
if pin.value() == 0:
pin.on()
# else it is already at HIGH state, nothing to do
def turn_gpio_off(device_num):
# Device Num to Pin Mapping
if device_num == 0:
pin_num = 0
elif device_num == 1:
pin_num = 2
# Check Pin
pin = machine.Pin(pin_num)
if pin.value() == 1:
pin.off()
# else it is already at LOW state, nothing to do
def init_pin(device_num):
# Device Num to Pin Mapping
if device_num == 0:
pin_num = 0
elif device_num == 1:
pin_num = 2
#open GPIO0 in output mode & turn it off by default
pin = machine.Pin(pin_num, machine.Pin.OUT)
# Turn off both GPIO initially
turn_gpio_off(device_num)
# Find out the post parameters in a dictionary
def get_post_params(req):
print("Inside GET POST PARAMS : req = " + req)
post_params = req.split('\r\n')[-1:][0]
# Check if the post body contains the necessary fields
# Split the post_params by &
# params : ['username=', 'password=', 'method=POST', 'url=http%3A%2F%2Ftwig-me.com%2Fv1%2Fusergroups%2FWKMUYXELA9LCC', 'jsondata=', 'submit=submit']
print("post_params : " + post_params)
params = post_params.split('&')
print("Params")
print(params)
# Initialize the key value pair dict
post_dict = {}
# Iterate on each param
for param in params:
# Each param would be like 'method=POST', etc
key_val = param.split('=')
print("Key Val :")
print(key_val)
key = key_val[0]
val = key_val[1]
# Update post_dict
post_dict[key] = val
return post_dict
# This web server takes care of the WiFi configuration
# max_run_sec
def web_server(max_run_sec = None):
# Turn wifi interface ON
turn_wifi_on()
# Create server socket
addr = socket.getaddrinfo('0.0.0.0', 80)[0][-1]
s = socket.socket()
# TODO : If both the wifi and sta are operating simultaneously, then bind only to WiFi
s.setsockopt(socket.SOL_SOCKET, socket.SO_REUSEADDR, 1)
s.bind(addr)
s.listen(1)
# s.settimeout(1)
poller = select.poll()
poller.register(s, select.POLLIN)
# Get the current time since epoch
startTimeEpoch = utime.time()
while True:
events = poller.poll(200) # time in milliseconds
if events:
try:
gc.collect()
res = s.accept()
client_s = res[0]
client_addr = res[1]
req = ''
#while True:
# data = client_s.recv(200)
# if data:
# req += str(data, 'utf8')
# else:
# break
# utime.sleep_ms(50)
req = client_s.recv(4096)
req = req.decode()
print(req)
req = str(req)
# Came here means that there has been some connection!
# Reset the start time epoch in such a case:
startTimeEpoch = utime.time()
# Check route now
if req.find('configure.html') != -1:
print("Got configure request!\r\n")
# Check if the username and password are correct, if not, configure:
login_config = get_login_config()
username = login_config['user']
pwd = login_config['password']
print("Username : " + username + ", pwd : " + pwd)
# Find the POST PARAMETERS sent
# There would be just one entry in the array, so get the 0th index directly
# post_params : 'username=&password=&method=POST&url=http%3A%2F%2Ftwig-me.com%2Fv1%2Fusergroups%2FWKMUYXELA9LCC&jsondata=&submit=submit'
print("Came here A")
post_dict = get_post_params(req)
# Now check if the post_dict has the key and value for username and password as needed?
username_post = post_dict['username']
password_post = post_dict['password']
print("Came here B")
# Check if the password is same as expected
if (username_post == username) and (password_post == pwd):
hidden_input = '<input type="hidden" name="username" value="' + username + '"><input type="hidden" name="passphrase" value="' + pwd + '">'
# Send the login username and password inside the hidden input field
configure_html = "<!DOCTYPE html><html><head> <title>Ouroboros IoT WiFi Configuration Page</title> </head><body><form action=\"configure_wifi.html\" method=\"POST\">WiFi SSID : <input type=\"text\" name=\"essid\"></br>WiFi Password: <input type=\"password\" name=\"passphrase\" ></br>" + hidden_input + "<input type=\"submit\" value=\"submit\" name=\"submit\"></form></body></html>"
# TODO : Also show link to webpage, where from we can change the login credentials
client_s.send(configure_html)
else:
client_s.send(login_fail_html)
elif req.find('configure_wifi.html') != -1:
# Check if the username and password are correct, if not, configure:
login_config = get_login_config()
username = login_config['user']
pwd = login_config['password']
# Get post parameters
post_dict = get_post_params(req)
# Now check if the post_dict has the key and value for username and password as needed?
username_post = post_dict['username']
password_post = post_dict['password']
# Check if the password is same as expected
if (username_post == username) and (password_post == pwd):
# Do some sanity check for handling the new wifi ssid and password
new_wifi_ssid = post_dict['essid']
new_wifi_passphrase = post_dict['passphrase']
# Set the wifi credentials
save_wifi_config(new_wifi_ssid, new_wifi_passphrase)
client_s.send('<!DOCTYPE html><html><head> <title>Ouroboros IoT WiFi Configuration Success</title> </head><body>Configuration successful!<br>Device would go into reboot now!</body></html>')
# Reboot device now
machine.reset()
else:
client_s.send(login_fail_html)
elif req.find('index.html') != -1:
print("Got index.html request!\r\n")
client_s.send(html)
else :
# Do nothing
print("Invalid request received! Show the login page again!\r\n")
client_s.send(html)
client_s.close()
machine.idle()
except OSError:
# Got no request and it timedout!
print("Timed-out, no request received!\r\n")
except Exception as e:
print("Got some exception\r\n")
print(str(e))
finally:
if max_run_sec is not None:
elapsedTime = utime.time() - startTimeEpoch
if elapsedTime > max_run_sec:
# Max run time of web server has elapsed, time to exit this mode!
break
utime.sleep_ms()
machine.idle()
# When while loop ends!
s.close()
# Turn wifi interface OFF
turn_wifi_off()
# Starts a thread which runs the web server to handle WiFi
def start_web_server(max_run_sec = None):
# start_new_thread(web_server, (max_run_sec))
web_server(max_run_sec)
############# MAIN ##########################
# Initialize two pins to INPUT and OFF by default
init_pin(0)
init_pin(1)
#turn_wifi_off()
# Check if the home wifi network has been setup
# Check if home wifi config is valid, if so, connect to it
# If home wifi is not configured, then use the Web server all the time.
if get_wifi_config() is None:
# Came here means the wifi is not configured
# Start the web server
print("Starting web server")
start_web_server()
EDIT 1:
I'm able to setup WDT and feed it. So no more WDT reboots. However, POST issue is still there:
FYI, the following is the response:
POST /configure.html HTTP/1.1
Host: 192.168.0.1
Connection: keep-alive
Content-Length: 43
Cache-Control: max-age=0
Origin: http://192.168.0.1
Upgrade-Insecure-Requests: 1
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (Linux; Android 5.1.1; Redmi Note 3 Build/LMY47V) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.123 Mobile Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Referer: http://192.168.0.1/index.html
Accept-Encoding: g
As could be seen, the packet received is partial, the Content-Length header says 43 byte payload. But its not received. When using "nc", and running server locally, the packet received is follows:
POST /configure.html HTTP/1.1
Host: 192.168.0.1
Connection: keep-alive
Content-Length: 43
Cache-Control: max-age=0
Upgrade-Insecure-Requests: 1
Origin: null
Content-Type: application/x-www-form-urlencoded
User-Agent: Mozilla/5.0 (X11; Linux x86_64) AppleWebKit/537.36 (KHTML, like Gecko) Ubuntu Chromium/65.0.3325.181 Chrome/65.0.3325.181 Safari/537.36
Accept: text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8
Accept-Encoding: gzip, deflate, br
Accept-Language: en-GB,en-US;q=0.9,en;q=0.8
username=admin&password=admin&submit=submit
Here, the 43 byte length payload could be easily seen.
So my question is, is the almost 800 byte payload too much to handle for ESP8266? Is there anything that could be done to remove the unnecessary headers that browser is sending? Any way to get all the data if its fragmented?
I faced a similar problem, my configuration is slightly different though.
I import my html from html_files.py as shown below.
cred_prompt = """
<!DOCTYPE html>
<html>
<head>
<title>ESP8266 connection</title>
</head>
<body>
<h2>
Enter the UID and Password to connect to WiFi
</h2>
<form action="/post">
uid: <input type="text" name="uid">
password: <input type="text" name="password">
<input type="submit" value="Submit">
</form><br>
</body>
</html>
"""
This is the function I use to get the credentials. I later save them into a file (Successive boots will use them)
def get_new_creds():
sta_if.disconnect()
print("Setting up webpage to get new credentials")
addr = socket.getaddrinfo('0.0.0.0', 80)[0][-1]
soc = socket.socket()
soc.bind(addr)
soc.listen(5)
print("Listening on", addr)
while True:
client, addr = soc.accept()
print("Client connected from", addr)
request = client.recv(1024)
request = request.decode().split()
uid, pwd = '', ''
if 'uid' in request[1]:
uid = request[1].split('&')[0].split('=')[1]
pwd = request[1].split('&')[1].split('=')[1]
write_new_creds(uid, pwd)
connect_to_wifi()
print("The UID is", uid, "and the Password is", pwd)
client.send('HTTP/1.1 200 OK\r\nContent-type: text/html\r\n\r\n')
client.send(html_files.connection_response.format(sta_if.ifconfig()[0]))
return uid, pwd
print(request)
client.send('HTTP/1.1 200 OK\r\nContent-type: text/html\r\n\r\n')
client.send(html_files.cred_prompt)
client.close()
This is the whole boot.py code if that helps.

Flickraw & VCR give unexpected results

I´m using VCR to test a method which gets a list of photosets from Flickr
require 'rails_helper'
RSpec.describe Photoset, type: :model do
# ...
describe '.import' do
it "gets photosets" do
VCR.use_cassette('photosets_getList') do
expect(Photoset.import(user_id: ENV['FLICKR_TEST_USER_ID']).first).to be_a Photoset
end
end
it "sets title" do
VCR.use_cassette('photosets_getList') do
expect(Photoset.import(user_id: ENV['FLICKR_TEST_USER_ID']).first.title).to eq 'Showcase'
end
end
it "sets uid" do
VCR.use_cassette('photosets_getList') do
expect(Photoset.import(user_id: ENV['FLICKR_TEST_USER_ID']).first.uid).to eq '72157647753138397'
end
end
end
end
The method itself is pretty straight forward and uses flickraw to get the photos from
flickr.photosets.getList and them maps to Photoset objects.
class Photoset
include Mongoid::Document
include Mongoid::Timestamps
field :title, type: String
field :uid, type: String
validates_uniqueness_of :uid
belongs_to :user
# Get photosets from Flickr
# #param [Hash] opts
# #return [Enumerable]
def self.import(opts = {})
res = flickr.photosets.getList(opts)
res.map do |item|
new(
uid: item["id"],
title: item["title"]
)
end
end
end
However the specs fail intermittently as Flickraw also does a request to flickr.reflection.getMethods and
flickr.photosets.getList(opts).map { |x| x }
Sometimes returns the request methods instead of Photosets
[("flickr.activity.userComments", "flickr.activity.userPhotos", "flickr.auth.checkToken"...]
Am I using VCR wrong or is this a bug in Flickraw?
These are the recorded HTTP interactions:
---
http_interactions:
- request:
method: post
uri: https://api.flickr.com/services/rest/
body:
encoding: US-ASCII
string: method=flickr.reflection.getMethods&format=json&nojsoncallback=1
headers:
Accept-Encoding:
- gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept:
- "*/*"
User-Agent:
- FlickRaw/0.9.8
Content-Type:
- application/x-www-form-urlencoded
response:
status:
code: 200
message: OK
headers:
Date:
- Mon, 30 Mar 2015 14:57:38 GMT
Content-Type:
- application/json
Content-Length:
- '1428'
P3p:
- policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV
TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY
ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Cache-Control:
- private
X-Served-By:
- www235.flickr.bf1.yahoo.com
Vary:
- Accept-Encoding
Age:
- '0'
Via:
- http/1.1 fts124.flickr.bf1.yahoo.com (ApacheTrafficServer/4.0.2 [cMsSf ]),
http/1.1 r05.ycpi.dea.yahoo.net (ApacheTrafficServer [cMsSf ])
Server:
- ATS
Strict-Transport-Security:
- max-age=259200
Connection:
- keep-alive
body:
encoding: UTF-8
string: '{"methods":{"method":[{"_content":"flickr.activity.userComments"},{"_content":"flickr.activity.userPhotos"},{"_content":"flickr.auth.checkToken"},{"_content":"flickr.auth.getFrob"},{"_content":"flickr.auth.getFullToken"},{"_content":"flickr.auth.getToken"},{"_content":"flickr.auth.oauth.checkToken"},{"_content":"flickr.auth.oauth.getAccessToken"},{"_content":"flickr.blogs.getList"},{"_content":"flickr.blogs.getServices"},{"_content":"flickr.blogs.postPhoto"},{"_content":"flickr.cameras.getBrandModels"},{"_content":"flickr.cameras.getBrands"},{"_content":"flickr.collections.getInfo"},{"_content":"flickr.collections.getTree"},{"_content":"flickr.commons.getInstitutions"},{"_content":"flickr.contacts.getList"},{"_content":"flickr.contacts.getListRecentlyUploaded"},{"_content":"flickr.contacts.getPublicList"},{"_content":"flickr.contacts.getTaggingSuggestions"},{"_content":"flickr.favorites.add"},{"_content":"flickr.favorites.getContext"},{"_content":"flickr.favorites.getList"},{"_content":"flickr.favorites.getPublicList"},{"_content":"flickr.favorites.remove"},{"_content":"flickr.galleries.addPhoto"},{"_content":"flickr.galleries.create"},{"_content":"flickr.galleries.editMeta"},{"_content":"flickr.galleries.editPhoto"},{"_content":"flickr.galleries.editPhotos"},{"_content":"flickr.galleries.getInfo"},{"_content":"flickr.galleries.getList"},{"_content":"flickr.galleries.getListForPhoto"},{"_content":"flickr.galleries.getPhotos"},{"_content":"flickr.groups.browse"},{"_content":"flickr.groups.discuss.replies.add"},{"_content":"flickr.groups.discuss.replies.delete"},{"_content":"flickr.groups.discuss.replies.edit"},{"_content":"flickr.groups.discuss.replies.getInfo"},{"_content":"flickr.groups.discuss.replies.getList"},{"_content":"flickr.groups.discuss.topics.add"},{"_content":"flickr.groups.discuss.topics.getInfo"},{"_content":"flickr.groups.discuss.topics.getList"},{"_content":"flickr.groups.getInfo"},{"_content":"flickr.groups.join"},{"_content":"flickr.groups.joinRequest"},{"_content":"flickr.groups.leave"},{"_content":"flickr.groups.members.getList"},{"_content":"flickr.groups.pools.add"},{"_content":"flickr.groups.pools.getContext"},{"_content":"flickr.groups.pools.getGroups"},{"_content":"flickr.groups.pools.getPhotos"},{"_content":"flickr.groups.pools.remove"},{"_content":"flickr.groups.search"},{"_content":"flickr.interestingness.getList"},{"_content":"flickr.machinetags.getNamespaces"},{"_content":"flickr.machinetags.getPairs"},{"_content":"flickr.machinetags.getPredicates"},{"_content":"flickr.machinetags.getRecentValues"},{"_content":"flickr.machinetags.getValues"},{"_content":"flickr.panda.getList"},{"_content":"flickr.panda.getPhotos"},{"_content":"flickr.people.findByEmail"},{"_content":"flickr.people.findByUsername"},{"_content":"flickr.people.getGroups"},{"_content":"flickr.people.getInfo"},{"_content":"flickr.people.getLimits"},{"_content":"flickr.people.getPhotos"},{"_content":"flickr.people.getPhotosOf"},{"_content":"flickr.people.getPublicGroups"},{"_content":"flickr.people.getPublicPhotos"},{"_content":"flickr.people.getUploadStatus"},{"_content":"flickr.photos.addTags"},{"_content":"flickr.photos.comments.addComment"},{"_content":"flickr.photos.comments.deleteComment"},{"_content":"flickr.photos.comments.editComment"},{"_content":"flickr.photos.comments.getList"},{"_content":"flickr.photos.comments.getRecentForContacts"},{"_content":"flickr.photos.delete"},{"_content":"flickr.photos.geo.batchCorrectLocation"},{"_content":"flickr.photos.geo.correctLocation"},{"_content":"flickr.photos.geo.getLocation"},{"_content":"flickr.photos.geo.getPerms"},{"_content":"flickr.photos.geo.photosForLocation"},{"_content":"flickr.photos.geo.removeLocation"},{"_content":"flickr.photos.geo.setContext"},{"_content":"flickr.photos.geo.setLocation"},{"_content":"flickr.photos.geo.setPerms"},{"_content":"flickr.photos.getAllContexts"},{"_content":"flickr.photos.getContactsPhotos"},{"_content":"flickr.photos.getContactsPublicPhotos"},{"_content":"flickr.photos.getContext"},{"_content":"flickr.photos.getCounts"},{"_content":"flickr.photos.getExif"},{"_content":"flickr.photos.getFavorites"},{"_content":"flickr.photos.getInfo"},{"_content":"flickr.photos.getNotInSet"},{"_content":"flickr.photos.getPerms"},{"_content":"flickr.photos.getRecent"},{"_content":"flickr.photos.getSizes"},{"_content":"flickr.photos.getUntagged"},{"_content":"flickr.photos.getWithGeoData"},{"_content":"flickr.photos.getWithoutGeoData"},{"_content":"flickr.photos.licenses.getInfo"},{"_content":"flickr.photos.licenses.setLicense"},{"_content":"flickr.photos.notes.add"},{"_content":"flickr.photos.notes.delete"},{"_content":"flickr.photos.notes.edit"},{"_content":"flickr.photos.people.add"},{"_content":"flickr.photos.people.delete"},{"_content":"flickr.photos.people.deleteCoords"},{"_content":"flickr.photos.people.editCoords"},{"_content":"flickr.photos.people.getList"},{"_content":"flickr.photos.recentlyUpdated"},{"_content":"flickr.photos.removeTag"},{"_content":"flickr.photos.search"},{"_content":"flickr.photos.setContentType"},{"_content":"flickr.photos.setDates"},{"_content":"flickr.photos.setMeta"},{"_content":"flickr.photos.setPerms"},{"_content":"flickr.photos.setSafetyLevel"},{"_content":"flickr.photos.setTags"},{"_content":"flickr.photos.suggestions.approveSuggestion"},{"_content":"flickr.photos.suggestions.getList"},{"_content":"flickr.photos.suggestions.rejectSuggestion"},{"_content":"flickr.photos.suggestions.removeSuggestion"},{"_content":"flickr.photos.suggestions.suggestLocation"},{"_content":"flickr.photos.transform.rotate"},{"_content":"flickr.photos.upload.checkTickets"},{"_content":"flickr.photosets.addPhoto"},{"_content":"flickr.photosets.comments.addComment"},{"_content":"flickr.photosets.comments.deleteComment"},{"_content":"flickr.photosets.comments.editComment"},{"_content":"flickr.photosets.comments.getList"},{"_content":"flickr.photosets.create"},{"_content":"flickr.photosets.delete"},{"_content":"flickr.photosets.editMeta"},{"_content":"flickr.photosets.editPhotos"},{"_content":"flickr.photosets.getContext"},{"_content":"flickr.photosets.getInfo"},{"_content":"flickr.photosets.getList"},{"_content":"flickr.photosets.getPhotos"},{"_content":"flickr.photosets.orderSets"},{"_content":"flickr.photosets.removePhoto"},{"_content":"flickr.photosets.removePhotos"},{"_content":"flickr.photosets.reorderPhotos"},{"_content":"flickr.photosets.setPrimaryPhoto"},{"_content":"flickr.places.find"},{"_content":"flickr.places.findByLatLon"},{"_content":"flickr.places.getChildrenWithPhotosPublic"},{"_content":"flickr.places.getInfo"},{"_content":"flickr.places.getInfoByUrl"},{"_content":"flickr.places.getPlaceTypes"},{"_content":"flickr.places.getShapeHistory"},{"_content":"flickr.places.getTopPlacesList"},{"_content":"flickr.places.placesForBoundingBox"},{"_content":"flickr.places.placesForContacts"},{"_content":"flickr.places.placesForTags"},{"_content":"flickr.places.placesForUser"},{"_content":"flickr.places.resolvePlaceId"},{"_content":"flickr.places.resolvePlaceURL"},{"_content":"flickr.places.tagsForPlace"},{"_content":"flickr.prefs.getContentType"},{"_content":"flickr.prefs.getGeoPerms"},{"_content":"flickr.prefs.getHidden"},{"_content":"flickr.prefs.getPrivacy"},{"_content":"flickr.prefs.getSafetyLevel"},{"_content":"flickr.push.getSubscriptions"},{"_content":"flickr.push.getTopics"},{"_content":"flickr.push.subscribe"},{"_content":"flickr.push.unsubscribe"},{"_content":"flickr.reflection.getMethodInfo"},{"_content":"flickr.reflection.getMethods"},{"_content":"flickr.stats.getCollectionDomains"},{"_content":"flickr.stats.getCollectionReferrers"},{"_content":"flickr.stats.getCollectionStats"},{"_content":"flickr.stats.getCSVFiles"},{"_content":"flickr.stats.getPhotoDomains"},{"_content":"flickr.stats.getPhotoReferrers"},{"_content":"flickr.stats.getPhotosetDomains"},{"_content":"flickr.stats.getPhotosetReferrers"},{"_content":"flickr.stats.getPhotosetStats"},{"_content":"flickr.stats.getPhotoStats"},{"_content":"flickr.stats.getPhotostreamDomains"},{"_content":"flickr.stats.getPhotostreamReferrers"},{"_content":"flickr.stats.getPhotostreamStats"},{"_content":"flickr.stats.getPopularPhotos"},{"_content":"flickr.stats.getTotalViews"},{"_content":"flickr.tags.getClusterPhotos"},{"_content":"flickr.tags.getClusters"},{"_content":"flickr.tags.getHotList"},{"_content":"flickr.tags.getListPhoto"},{"_content":"flickr.tags.getListUser"},{"_content":"flickr.tags.getListUserPopular"},{"_content":"flickr.tags.getListUserRaw"},{"_content":"flickr.tags.getMostFrequentlyUsed"},{"_content":"flickr.tags.getRelated"},{"_content":"flickr.test.echo"},{"_content":"flickr.test.login"},{"_content":"flickr.test.null"},{"_content":"flickr.urls.getGroup"},{"_content":"flickr.urls.getUserPhotos"},{"_content":"flickr.urls.getUserProfile"},{"_content":"flickr.urls.lookupGallery"},{"_content":"flickr.urls.lookupGroup"},{"_content":"flickr.urls.lookupUser"}]},"stat":"ok"}'
http_version:
recorded_at: Mon, 30 Mar 2015 14:57:37 GMT
- request:
method: post
uri: https://api.flickr.com/services/rest/
body:
encoding: US-ASCII
string: user_id=62829091%40N05&method=flickr.photosets.getList&format=json&nojsoncallback=1
headers:
Accept-Encoding:
- gzip;q=1.0,deflate;q=0.6,identity;q=0.3
Accept:
- "*/*"
User-Agent:
- FlickRaw/0.9.8
Content-Type:
- application/x-www-form-urlencoded
response:
status:
code: 200
message: OK
headers:
Date:
- Mon, 30 Mar 2015 14:57:38 GMT
Content-Type:
- application/json
Content-Length:
- '878'
P3p:
- policyref="http://info.yahoo.com/w3c/p3p.xml", CP="CAO DSP COR CUR ADM DEV
TAI PSA PSD IVAi IVDi CONi TELo OTPi OUR DELi SAMi OTRi UNRi PUBi IND PHY
ONL UNI PUR FIN COM NAV INT DEM CNT STA POL HEA PRE LOC GOV"
Cache-Control:
- private
X-Served-By:
- www35.flickr.bf1.yahoo.com
Vary:
- Accept-Encoding
Age:
- '0'
Via:
- http/1.1 fts109.flickr.bf1.yahoo.com (ApacheTrafficServer/4.0.2 [cMsSf ]),
http/1.1 r14.ycpi.dea.yahoo.net (ApacheTrafficServer [cMsSf ])
Server:
- ATS
Strict-Transport-Security:
- max-age=259200
Connection:
- keep-alive
body:
encoding: UTF-8
string: '{"photosets":{"page":1,"pages":1,"perpage":10,"total":10,"photoset":[{"id":"72157647753138397","primary":"13362508473","secret":"73945d3830","server":"7300","farm":8,"photos":"10","videos":0,"title":{"_content":"Showcase"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"0","count_comments":"0","can_comment":0,"date_create":"1411992696","date_update":"1420398843"},{"id":"72157642806447225","primary":"13362324935","secret":"792aed9e58","server":"7072","farm":8,"photos":"11","videos":0,"title":{"_content":"Offpistcamp,
Mars 2014"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"18","count_comments":"0","can_comment":0,"date_create":"1395604793","date_update":"1420398843"},{"id":"72157635592985815","primary":"9789159454","secret":"68e3687ebf","server":"7304","farm":8,"photos":"49","videos":0,"title":{"_content":"Sara
& Adam\u00b4s wedding "},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"36","count_comments":"0","can_comment":0,"date_create":"1379447652","date_update":"1379448808"},{"id":"72157633225753747","primary":"8643524170","secret":"d31134cfac","server":"8387","farm":9,"photos":"11","videos":0,"title":{"_content":"\u00c5re
Park"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"0","count_comments":"0","can_comment":0,"date_create":"1365784604","date_update":"1365784720"},{"id":"72157632691639823","primary":"8446828475","secret":"c27f229c79","server":"8079","farm":9,"photos":"5","videos":0,"title":{"_content":"\u00c5resj\u00f6n"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"0","count_comments":"0","can_comment":0,"date_create":"1360071999","date_update":"1360072052"},{"id":"72157632228500139","primary":"8266649775","secret":"bea16448f6","server":"8337","farm":9,"photos":"36","videos":0,"title":{"_content":"Renskiljning"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"8","count_comments":"0","can_comment":0,"date_create":"1355334260","date_update":"1365536767"},{"id":"72157632141702707","primary":"8234153651","secret":"82b1bababb","server":"8487","farm":9,"photos":"11","videos":0,"title":{"_content":"Ozone
Demodag 2012"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"30","count_comments":"0","can_comment":0,"date_create":"1354372862","date_update":"1354373227"},{"id":"72157629510040228","primary":"6956421098","secret":"fe55925613","server":"7197","farm":8,"photos":"14","videos":0,"title":{"_content":"Slutskjutet
p\u00e5 Skutskjutet"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"4","count_comments":"0","can_comment":0,"date_create":"1335109758","date_update":"1335109926"},{"id":"72157626719466442","primary":"5719219791","secret":"5966d75b4d","server":"2128","farm":3,"photos":"4","videos":0,"title":{"_content":"Trondheim"},"description":{"_content":""},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"1","count_comments":"0","can_comment":0,"date_create":"1305400311","date_update":"1354372544"},{"id":"72157626594657331","primary":"5719668226","secret":"d5c9c6e1bd","server":"2593","farm":3,"photos":"15","videos":0,"title":{"_content":"Rope
Swing Session"},"description":{"_content":"Oslo, norway"},"needs_interstitial":0,"visibility_can_see_set":1,"count_views":"8","count_comments":"0","can_comment":0,"date_create":"1305398192","date_update":"1305398195"}]},"stat":"ok"}'
http_version:
recorded_at: Mon, 30 Mar 2015 14:57:38 GMT
recorded_with: VCR 2.9.3
flickr.reflection.getMethods is called to build all the objects and methods. Flickraw does not know what methods are available when initiallzed. If you want a deterministic behavior, you can use flickraw-cached which embeds a list of all methods.
https://github.com/hanklords/flickraw/issues/86#issuecomment-96834525

ruby mailman remove header from email body

I am trying to display an email body in my RoR project.
class IncomingMail
def initialize(message, params)
if person = Person.find_by_email(message.from)
changeMessage = Message.where({person_id: person.id})
#message = message.subject.force_encoding("UTF-8")
message = message.body.encoded
changeMessage.first.text = message
changeMessage.first.backInMinutes = 0
changeMessage.first.showText = 1
changeMessage.first.doNotDisturb = 0
changeMessage.first.save
end
end
but i also get the email header
> --e89a8ff1c0465030f204c082e054 Date: Mon, 21 May 2012 04:45:12 +0200 Mime-Version: 1.0 Content-Type: text/plain; charset=ISO-8859-1
> Content-Transfer-Encoding: 7bit Content-ID:
> <4fb9ac38c71d2_1e1343dd8042105b#ubuntu.mail> Text of the mail
how can I remove the header ?
with the subject it works like this
message = message.subject.force_encoding("UTF-8")
But not with the body.
I have found the solution!!!
change:
message = message.body.encoded
to:
message = message.text_part.body.decoded
that cuts all the header details away and gives me only the TEXT of the email.
It took a long time but it worked I hope it also helps other users

Resources