I'm trying to upload to my server (on Heroku) a file stored in a password protected FTP.
The problem is that this FTP also dont contain my production IP address on his whitelist (and i cant add it..) so i should use a proxy to connect my rails app this FTP.
I tried this code :
proxy_uri = URI(ENV['QUOTAGUARDSTATIC_URL'] || 'http://login:password#myproxy.com:9293')
Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port,"login","password").start('ftp://login:password#ftp.website.com') do |http|
http.get('/path/to/myfile.gz').body
end
But my http.get returns me lookup ftp: no such host.
I also got this code for FTP download, but i dont know how to make it works with a proxy :
ftp = Net::FTP.new('ftp.myftp.com', 'login', 'password')
ftp.chdir('path/to')
ftp.getbinaryfile('myfile.gz', 'public/myfile.gz', 1024)
ftp.close
Thanks in advance.
I realise that you asked this question over 6 months ago, but I recently had a similar issue and found that this (unanswered) question is the top Google result, so I thought I would share my findings.
mudasobwa's comment below your original post has a link to the net/ftp documentation which explains how to use a SOCKS proxy...
Although you don't mention a specific requirement for a HTTP proxy in your original post, it seems obvious to me that is what you were trying to use. As I'm sure you're aware, this makes the SOCKS documentation totally irrelevant.
The following code has been tested on ruby-1.8.7-p357 using an HTTP proxy that does not require authentication:
file = File.open('myfile.gz', 'w')
http = Net::HTTP.start('myproxy.com', '9293')
resp, data = http.get('ftp://login:password#ftp.website.com')
file.write(data) if resp.code == "200"
file.close unless file.nil?
Source
This should give you a good starting point to figure the rest out for yourself.
To get you going, I would guess that you could use user:pass#myproxy.com for basic auth, or perhaps sending a Proxy-Authorization header in your GET request.
Related
I have this Telegram bot written in Lua that I am doing as a hobby for a language network. And I have been reading new messages via the getUpdates API call all the time. Now I want to rewrite it to use webhooks, but I have no experience with that whatsoever. I have googled but didn't find anything certain. I kinda feel that WSAPI is the library to use, but I am not sure. Moreover, I am not really sure I need any special library just for reading POST requests (which is all that the Telegram bot API uses). I tried using sockets:
socket = require 'socket'
server = assert(socket.bind("*", 9000))
function read(client, pattern, prefix)
local data, emsg, partial = client:receive(pattern, prefix)
if data then
return data
end
if partial and #partial > 0 then
return partial
end
return nil, emsg
end
while true do
local client = server:accept()
client:settimeout(3)
local msg, err = read(client, '*a')
if not err then
print(msg)
client:close()
end
end
The print(msg) here gives me the full POST request including headers, which I am probably able to parse (the body is supposed to always be a JSON). I am not really that familiar with HTTP requests though and I'm not sure I can just throw away everything that goes before the first {.
My setup is Lua 5.2, Ubuntu x64 16.04 and Nginx. What I need to do is to receive and read POST requests, nothing more.
TL;DR: is it okay to parse the POST request I receive from the code above or am I missing something, like a library that'd make my life easier?
Thanks!
I'm working on a Rails app and here are two important pieces of the error message I get when I try to seed data in my database:
HTTP::ConnectionError: failed to connect: Operation timed out - SSL_connect
Errno::ETIMEDOUT: Operation timed out - SSL_connect
Here is my code, where I'm pulling data from a file, and creating Politician objects:
politician_data.each do |politician|
photo_from_congress = "https://theunitedstates.io/images/congress/original/" + politician["id"]["bioguide"] + ".jpg"
HTTP.get(photo_from_congress).code == 200 ? image = photo_from_congress : image = "noPoliticianImage.png"
Politician.create(
name: "#{politician["name"]["first"]} #{politician["name"]["last"]}",
image: image
)
end
I put in a pry, and the iteration works for the first loop, so the code is OK. After several seconds, the loop breaks, and I get that error, so I think it has something to do with the number of HTTP.get requests I'm making?
https://github.com/unitedstates/images is a Git repo. Perhaps that repo can't handle that many get requests?
I did some Google'ing and saw it may have something to do with "Request timed out" error? My having to set up a proxy servers? I'm a junior programmer so please be very specific when responding.
*EDIT TO ADD THIS:
I found this blurb on the site where I'm making get requests to cull photos (https://github.com/unitedstates/images), that may help?
Note: Our HTTPS permalinks are provided through CloudFlare's Universal SSL, which also uses "Flexible SSL" to talk to GitHub Pages' unencrypted endpoints. So, you should know that it's not an end-to-end encrypted channel, but is encrypted between your client use and CloudFlare's servers (which at least should dissociate your requests from client IP addresses).
by the way, using "Net::HTTP" instead of the "HTTP" Ruby gem worked. Instead of checking the status code, i just checked to see if the body contained key text:
photo_from_congress = "https://theunitedstates.io/images/congress/original/" + politician["id"]["bioguide"] + ".jpg"
photo_as_URI = URI(photo_from_congress)
Net::HTTP.get_response(photo_as_URI ).body.include?("File not found") ? image = "noPoliticianImage.png" : image = photo_from_congress
I am working on a project whereby we have sites (developed with ruby on rails) hosted on an Ubuntu server using tomcat. We want these sites to make HTTP calls to a service developed using Nancy. We have this working locally whereby the service is hosted on a machine that we can call within our network. We cannot however get it working when live. Here is an example call:
def get_call(routePath)
started_at = Time.now
enc_url = URI.encode("#{settings.service_endpoint}#{routePath}")
uri = URI.parse(enc_url)
http = Net::HTTP.new(uri.host, uri.port)
req = Net::HTTP::Get.new(uri.request_uri)
resp = http.request(req)
logger.bench 'SERVICE - GET', started_at, routePath
return resp if response_ok?(resp)
end
When working locally the settings are as follows:
settings.service_endpoint = http://10.10.10.27:7820
routePath = /Customers
When we upload it to the server we use the following:
settings.service_endpoint = http://127.0.0.1:24099
routePath = /Customers
We currently get the following error:
SocketError at /register
initialize: name or service not know
with the following line being highlighted:
resp = http.request(req)
Are we completely wrong with the IP being called. Should it be 127.0.0.1, localhost. 10.10.10.27 or something entirely different? The strange thing is we can do a GET call via telnet in our Ubuntu server (telnet 127.0.0.1 24099) so that must mean the server can make the calls but the site hosted on the server cannot. Do we need to include a HTTP proxy (have read some reference to that but dont really know if its needed).
Apologies if its obvious but we have never tried anything like this before so its all very perplexing. Any further information required just let me know.
We changed the service_endpoint to localhost and it worked. Not sure if this is because it didnt like "http://" or some other reason. Any explanation as to why this is the case would be much appreciated, just so we know. Thanks!
I'm currently hosting both my rails app and a faye-server app on Heroku. The faye server has been cloned from here (https://github.com/ntenisOT/Faye-Heroku-Cedar) and seems to be running correctly. I have disabled websockets, as they are not supported on Heroku. Despite the claim on Faye's site that:
"Faye clients and servers transparently support cross-domain communication, so your client can connect to a server on any domain you like without further configuration."
I am still running into this error when I try to post to a faye channel:
XMLHttpRequest cannot load http://MYFAYESERVER.herokuapp.com. Origin http://MYAPPURL.herokuapp.com is not allowed by Access-Control-Allow-Origin.
I have read about CORS and tried implementing some solutions outlined here: http://www.tsheffler.com/blog/?p=428 but have so far had no luck. I'd love to hear from someone who:
1) Has a rails app hosted on Heroku
2) Has a faye server hosted on Heroku
3) Has the two of them successfully communicating with each other!
Thanks so much.
I just got my faye and rails apps hosted on heroku communicating within the past hour or so... here are my observations:
Make sure your FAYE_TOKEN is set on all of your servers if you're using an env variable.
Disable websockets, which you've already done... client.disable(...) didn't work for me, I used Faye.Transport.WebSocket.isUsable = function(_,c) { c(false) } instead.
This may or may not apply to you, but was the hardest thing to track down for me... in my dev environment, the port my application is running on will be tacked onto the end of the specified hostname for my faye server... but this appeared to cause a failure to communicate in production. I worked around that by creating a broadcast_server_uri method in application_controller.rb that handles inclusion of a port when necessary, and then use that anywhere I spin up a new channel.
....
class ApplicationController < ActionController::Base
def broadcast_server
if request.port.to_i != 80
"http://my-faye-server.herokuapp.com:80/faye"
else
"http://my-faye-server.herokuapp.com/faye"
end
end
helper_method :broadcast_server
def broadcast_message(channel, data)
message = { :ext => {:auth_token => FAYE_TOKEN}, :channel => channel, :data => data}
uri = URI.parse(broadcast_server)
Net::HTTP.post_form(uri, :message => message.to_json)
end
end
And in my app javascript, including
<script>
var broadcast_server = "<%= broadcast_server %>"
var faye;
$(function() {
faye = new Faye.Client(broadcast_server);
faye.setHeader('Access-Control-Allow-Origin', '*');
faye.connect();
Faye.Transport.WebSocket.isUsable = function(_,c) { c(false) }
// spin off your subscriptions here
});
</script>
FWIW, I wouldn't stress about setting Access-Control-Allow-Origin as it doesn't seem to be making a difference either way - I see XMLHttpRequest cannot load http://... regardless, but this should still works well enough to get you unblocked. (although I'd love to learn of a cleaner solution...)
Can't say I have used Rails/Faye on Heroku but have you tried setting the Access-Control-Allow-Origin header to something like Access-Control-Allow-Origin: your-domain.com?
For testing you could also do Access-Control-Allow-Origin: * to see if that helps
Custom headers
Some services require the use of additional HTTP headers to connect to
their Bayeux server. You can add these headers using the setHeader()
method, and they will be sent if the underlying transport supports
user-defined headers (currently long-polling only).
client.setHeader('Authorization', 'OAuth abcd-1234');
Source: http://faye.jcoglan.com/browser.html
So try client.setHeader('Access-Control-Allow-Origin', '*');
I have been trying to connect to a web service that is using digest authentication.
I am able to connect in Safari using user:password#service.site.com/endpoint
I have tried in Ruby and Rails to connect using HTTParty and Net:HTTP using the "basic"auth" options, but have not had any luck.
Wondering if the HTTParty/Net:HTTP "basic_auth" option is not going to be compatible with a "digest auth" service?
If not, is there another way that I might connect?
HTTParty basic auth is apparently not compatible with digest_auth. I found this Net:HTTP extension: https://codesnippets.joyent.com/posts/show/1075 and am writing a method to handle this, with the help of the Crack gem http://github.com/jnunemaker/crack:
def self.decode vin
url = URI.parse(APP_CONFIG[:vinlink_url])
Net::HTTP.start(url.host) do |http|
res = http.head(url.request_uri)
req = Net::HTTP::Get.new("/report?type=basic&vin=#{vin}")
req.digest_auth(APP_CONFIG[:vinlink_login], APP_CONFIG[:vinlink_password], res)
#response = http.request(req)
end
if #response.code == "200"
hash = Crack::XML.parse(#response.body).recursive_downcase_keys!.recursive_symbolize_keys!
end
end
Wasn't able to get to the codesnippets link given above today, but code is also available here https://gist.github.com/73102. I've used this successfully for digest authentication, but ran into problems with multiple request, getting 'Stale client nonce' errors - resolved by generating a new nonce within the digest_auth function each time it was called. Didn't find much on that when I looked, so hope this helps someone.