I have this Telegram bot written in Lua that I am doing as a hobby for a language network. And I have been reading new messages via the getUpdates API call all the time. Now I want to rewrite it to use webhooks, but I have no experience with that whatsoever. I have googled but didn't find anything certain. I kinda feel that WSAPI is the library to use, but I am not sure. Moreover, I am not really sure I need any special library just for reading POST requests (which is all that the Telegram bot API uses). I tried using sockets:
socket = require 'socket'
server = assert(socket.bind("*", 9000))
function read(client, pattern, prefix)
local data, emsg, partial = client:receive(pattern, prefix)
if data then
return data
end
if partial and #partial > 0 then
return partial
end
return nil, emsg
end
while true do
local client = server:accept()
client:settimeout(3)
local msg, err = read(client, '*a')
if not err then
print(msg)
client:close()
end
end
The print(msg) here gives me the full POST request including headers, which I am probably able to parse (the body is supposed to always be a JSON). I am not really that familiar with HTTP requests though and I'm not sure I can just throw away everything that goes before the first {.
My setup is Lua 5.2, Ubuntu x64 16.04 and Nginx. What I need to do is to receive and read POST requests, nothing more.
TL;DR: is it okay to parse the POST request I receive from the code above or am I missing something, like a library that'd make my life easier?
Thanks!
Related
I have a Lua proxy that needs to route requests. Each request destination is established based on the response from another HTTP request with a header from the initial request. My understanding is that HAProxy is an event-driven software, so blocking system calls are absolutely forbidden and my code is blocking because is doing an HTTP request.
I read about yielding after the request but I think it won't help since the HTTP request is already started. The library for doing the request is https://github.com/JakobGreen/lua-requests#simple-requests
local requests = require('requests')
core.register_fetches('http_backend', function(txn)
local dest = txn.sf:req_fhdr('X-dest')
local url = "http://127.0.0.1:8080/service";
local response = requests.get(url.."/"+dest);
local json = response.json()
return json.field
end )
How do I convert my code to be non-blocking?
You should consider using HAProxy's SPOE which was created exactly for these blocking scenarios.
I managed to do it using Lua. The thing I was making wrong was using require('requests') this is blocking. Ideally for HA never use a Lua external library. I have to deal with plain sockets and do an HTTP request and very important to use HA core method core.tcp() instead of Lua sockets.
This is more of a conceptual understanding gap rather than technical one. I am new to web socket\messaging api's.
I ran a chat application using faye ruby server and everything works fine between two browsers.I want to send a message from a stand alone ruby client to a browser client which is sending messages to same server. Is it possible to send a message from a client like the one below to a browser whose script is also given below ?
This is not related to the application I created, but I was trying to understand the use of WS client api. Or specifically put , can I send message from a server client to browser client ? I guess I am lacking the understanding of the word 'client' here.
I see the messages on the server console, but the browser doesn't get the message sent by the stand alone client.
Also I see this when i run the client :
Started GET "/faye/test123" for 127.0.0.1 at 2015-04-09 07:17:46 -0400
require 'faye'
require 'eventmachine'
EM.run {
ws = Faye::WebSocket::Client.new('ws://localhost:9292/faye/test123')
ws.onopen = lambda do |event|
p [:open, ws.headers]
ws.send('987654321')
end
ws.on :open do |event|
p [:open]
ws.send('123 123 123 123')
p [:sent]
end
}
Browser script :
window.client = new Faye.Client('http://localhost:9292/faye');
client.subscribe('/test123', function(payload){
if(payload.message)
{
console.log('I am in here 77777.......'+payload.message);
return $("#incomingText").append(payload.message);
}
}
Looking at your code I think it might be useful to highlight the difference between websockets and faye.
Faye is a framework that supports a number of transports, websockets being just one of them. It can also do long polling for example. One of the benefits of Faye is that it can select the right transport that both the client and server understand. It also implements a simple pub/sub protocol on top of that transport, giving you a nice API to build off of.
Doing a pure websocket implementation is totally doable, but if you're going to go with Faye it's probably a good idea to use Faye's publish/subscribe API and not muck with Faye's websockets directly.
To answer your specific question:
Is it possible to send a message from a client like the one below to a browser whose script is also given below ?
Yes, absolutely but I would suggest doing it with Faye::Client. Here's what your server side code might look like:
client = Faye::Client.new('http://localhost:9292/faye')
client.publish('/test123', 'message' => 'Hello world')
With much more info here:
http://faye.jcoglan.com/ruby/clients.html
I'm trying to upload to my server (on Heroku) a file stored in a password protected FTP.
The problem is that this FTP also dont contain my production IP address on his whitelist (and i cant add it..) so i should use a proxy to connect my rails app this FTP.
I tried this code :
proxy_uri = URI(ENV['QUOTAGUARDSTATIC_URL'] || 'http://login:password#myproxy.com:9293')
Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port,"login","password").start('ftp://login:password#ftp.website.com') do |http|
http.get('/path/to/myfile.gz').body
end
But my http.get returns me lookup ftp: no such host.
I also got this code for FTP download, but i dont know how to make it works with a proxy :
ftp = Net::FTP.new('ftp.myftp.com', 'login', 'password')
ftp.chdir('path/to')
ftp.getbinaryfile('myfile.gz', 'public/myfile.gz', 1024)
ftp.close
Thanks in advance.
I realise that you asked this question over 6 months ago, but I recently had a similar issue and found that this (unanswered) question is the top Google result, so I thought I would share my findings.
mudasobwa's comment below your original post has a link to the net/ftp documentation which explains how to use a SOCKS proxy...
Although you don't mention a specific requirement for a HTTP proxy in your original post, it seems obvious to me that is what you were trying to use. As I'm sure you're aware, this makes the SOCKS documentation totally irrelevant.
The following code has been tested on ruby-1.8.7-p357 using an HTTP proxy that does not require authentication:
file = File.open('myfile.gz', 'w')
http = Net::HTTP.start('myproxy.com', '9293')
resp, data = http.get('ftp://login:password#ftp.website.com')
file.write(data) if resp.code == "200"
file.close unless file.nil?
Source
This should give you a good starting point to figure the rest out for yourself.
To get you going, I would guess that you could use user:pass#myproxy.com for basic auth, or perhaps sending a Proxy-Authorization header in your GET request.
I want to build a small script in python which needs to fetch an url. The server is a kind of crappy though and replies pure ASCII without any headers.
When I try:
import urllib.request
response = urllib.request.urlopen(url)
print(response.read())
I obtain a http.client.BadStatusLine: 100 error because this isn't a properly formatted HTTP response.
Is there another way to fetch an url and get the raw content, without trying to parse the response?
Thanks
It's difficult to answer your direct question without a bit more information; not knowing exactly how the (web) server in question is broken.
That said, you might try using something a bit lower-level, a socket for example. Here's one way (python2.x style, and untested):
#!/usr/bin/env python
import socket
from urlparse import urlparse
def geturl(url, timeout=10, receive_buffer=4096):
parsed = urlparse(url)
try:
host, port = parsed.netloc.split(':')
except ValueError:
host, port = parsed.netloc, 80
sock = socket.create_connection((host, port), timeout)
sock.sendall('GET %s HTTP/1.0\n\n' % parsed.path)
response = [sock.recv(receive_buffer)]
while response[-1]:
response.append(sock.recv(receive_buffer))
return ''.join(response)
print geturl('http://www.example.com/') #<- the trailing / is needed if no
other path element is present
And here's a stab at a python3.2 conversion (you may not need to decode from bytes, if writing the response to a file for example):
#!/usr/bin/env python
import socket
from urllib.parse import urlparse
ENCODING = 'ascii'
def geturl(url, timeout=10, receive_buffer=4096):
parsed = urlparse(url)
try:
host, port = parsed.netloc.split(':')
except ValueError:
host, port = parsed.netloc, 80
sock = socket.create_connection((host, port), timeout)
method = 'GET %s HTTP/1.0\n\n' % parsed.path
sock.sendall(bytes(method, ENCODING))
response = [sock.recv(receive_buffer)]
while response[-1]:
response.append(sock.recv(receive_buffer))
return ''.join(r.decode(ENCODING) for r in response)
print(geturl('http://www.example.com/'))
HTH!
Edit: You may need to adjust what you put in the request, depending on the web server in question. Guanidene's excellent answer provides several resources to guide you on that path.
What you need to do in this case is send a raw HTTP request using sockets.
You would need to do a bit of low level network programming using the socket python module in this case. (Network sockets actually return you all the information sent by the server as it as, so you can accordingly interpret the response as you wish. For example, the HTTP protocol interprets the response in terms of standard HTTP headers - GET, POST, HEAD, etc. The high-level module urllib hides this header information from you and just returns you the data.)
You also need to have some basic information about HTTP headers. For your case, you just need to know about the GET HTTP request. See its definition here - http://djce.org.uk/dumprequest, see an example of it here - http://en.wikipedia.org/wiki/HTTP#Example_session. (If you wish to capture live traces of HTTP requests sent from your browser, you would need a packet sniffing software like wireshark.)
Once you know basics about socket module and HTTP headers, you can go through this example - http://coding.debuntu.org/python-socket-simple-tcp-client which tells you how to send a HTTP request over a socket to a server and read its reply back. You can also refer to this unclear question on SO.
(You can google python socket http to get more examples.)
(Tip: I am not a Java fan, but still, if you don't find enough convincing examples on this topic under python, try finding it under Java, and then accordingly translate it to python.)
urllib.urlretrieve('http://google.com/abc.jpg', 'abc.jpg')
We have a function in our Rails code that accepts a JSON POST body:
contacts = ActiveSupport::JSON.decode(request.raw_post.gsub("+", ""))
(I'm aware that I can get this from params["_json"] as well, but we have extremely large (MBs) POST bodies that do not get put into params["_json"] for some reason (and + throws errors too).
Since the JSON is usually sent from a mobile client, it's important to us to optimize the upload size. We want to switch to having the POST body gzipped.
However, no matter what we do, we get the same error with no line number:
MultiJson::DecodeError (743: unexpected token at ''):
We have tried:
gzipped_contacts = Zlib::GzipReader.new(StringIO.new(request.raw_post)).read
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
This:
gzipped_contacts = ActiveSupport::Gzip.decompress(request.raw_post)
contacts = ActiveSupport::JSON.decode(gzipped_contacts.gsub("+", ""))
And the solution found here: Rails: how to unzip a compressed xml request body?
I'm pretty sure this is not occurring at the controller level because I can't log anything there, so it needs to be done in the middleware or at the server (but I can't find anything for Nginx that lets us deflate). Please assist!
Ok, turns out the iPhone client was sending the wrong headers. So the solution for anyone encountering this is to see the advice here:
Rails: how to unzip a compressed xml request body?
And verify that you are sending Content-Type: gzip/json.