I have an FTP server which only accepts connections through running FTPS (explicit FTP over TLS). I need to be able to connect to this using a Ruby on Rails app.
Does anybody know of a method to do this? I have tried the Net::FTP library but this does not appear to support FTPS connections.
How about using Net::FTPTLS ?
Since Ruby 2.4, TLS over FTP has been available with Net::FTP... this has caused gems like double-bag-ftps to become archived and all your google searches to yield outdated answers.
If you can do explicit FTP over TLS (Connects to FTP normally, then issues a command AUTH TLS to switch to TLS Mode), then great... that should be able to use Ruby's Net::FTP out of the box by just passing {ssl: true} in the options.
Implicit FTP over TLS (runs over TLS from the get-go) does not work out of the box, however, and you must override Net::FTP's connection method to establish an SSL socket and then optionally send commands to the FTP server.
Inidka K posted a Github Gist, but since those are bad form (can go stale), I've posted my version that works against a ShareFile Implicit FTP setup (which seems to only support Implicit FTP):
require 'net/ftp'
class ImplicitFtp < Net::FTP
FTP_PORT = 990
def connect(host, port = FTP_PORT)
synchronize do
#host = host
#bare_sock = open_socket(host, port)
begin
ssl_sock = start_tls_session(Socket.tcp(host, port))
#sock = BufferedSSLSocket.new(ssl_sock, read_timeout: #read_timeout)
voidresp
if #private_data_connection
voidcmd("PBSZ 0")
voidcmd("PROT P")
end
rescue OpenSSL::SSL::SSLError, Net::OpenTimeout
#sock.close
raise
end
end
end
end
Then, in your code:
ftp_options = {
port: 990,
ssl: true,
debug_mode: true, # If you want to see what's going on
username: FTP_USER,
password: FTP_PASS
}
ftp = ImplicitFtp.open(FTP_HOST, ftp_options)
puts ftp.list
ftp.close
I done something like this with Implicit/Explicit FTPS, I used double-bag-ftps gem that I patched to support reuse of ssl session. It's a requirement for a lot of ftps servers.
I put the code on github here : https://github.com/alain75007/double-bag-ftps
EDIT: I figured out how to get it running locally, but am having issues getting it to work on Heroku. That's a bit of a departure from this question, so I've created a new one:
Heroku with FTPTLS - Error on SSL Connection
require 'net/ftptls'
ftp = Net::FTPTLS.new()
ftp.passive = true
#make sure you define port_number
ftp.connect('host.com', port_number)
ftp.login('Username', 'Password')
ftp.gettextfile('filename.ext', 'where/to/save/file.ext')
ftp.close
If you want to use Implicit FTPS, please try this gist.
For Explicit FTPs, you can use the ruby gem ftpfxp.
Support for implicit FTPS was merged into ruby/net-ftp in January 2022 (in this PR). If you want to make use of this straight away, you can include the latest version directly in your Gemfile:
gem "net-ftp", github: "ruby/net-ftp", branch: "master"
Then you just need:
options = {
ssl: true,
port: 990,
implicit_ftps: true,
username: "your-user",
password: "*********",
debug_mode: true
}
Net::FTP.open("yourhost.com", options) do |ftp|
ftp.list.map{ |f| puts f }
end
I implemented an ftps solution using double-bag-ftps
double-bag-ftps
Related
We've setup Cloudfront in front of our application, but unfortunately it strips the Upgrade header required for ActionCable to run.
We'd like to have a different subdomain that points to the same servers, but bypasses Cloudfront (socket.site.com, for instance). We've done this and it's somewhat working, but it seems like a persistent connection can't be made. ActionCable continues to retry to make the connection every 10s and seems unable to hold the connection open:
Any advice related to Cloudfront or different domains for ActionCable is appreciated.
To all who follow, hopefully this helps.
As of the time of me writing this (Oct. 2018), it doesn't appear that you can use ActionCable behind Cloudfront at all. CF will discard the upgrade header which will prevent a secure socket connection from ever being made.
Our setup was CF -> Application Load Balancer (ALB) -> EC2. On the AWS side, we began by making a subdomain (socket.example.com) that pointed directly to the same ALB and bypassed CF entirely. Note that Classic Load Balancers absolutely will not work. You can only use ALBs.
This alone did not fix the issue. On your Rails config, you have to add the following lines to your production.rb:
config.action_cable.url = 'wss://socket.example.com:28080/cable'
config.action_cable.allowed_request_origins = ['https://example.com'] # Not the subdomain
You may also need to update your CSP to include wss://socket.example.com/cable for connect_src.
If at this point you're getting a message about failing to upgrade, you need to ensure that your NGINX config is correct. This answer may help.
You will also need to reflect this change in your cable.js. This following snippet works for me with local development as well as production, but you may need to alter it. I wrote it with pre-ES6 in mind because this file never hit Babel in our configuration.
(function() {
this.App || (this.App = {})
var wsUrl
if(location.host.indexOf('localhost') != -1) {
wsUrl = '/cable'
} else {
var host = location.host
var protocol = location.protocol
wsUrl = protocol + '//socket.' + host + '/cable'
}
App.cable = ActionCable.createConsumer(wsUrl)
}).call(this)
That may be all you need, depending on your authentication scheme. However, I was using cookies shared between the main application and ActionCable and this caused a difficult bug. The connection would appear to be made correctly, but it would actually fail and ActionCable would retry every 10s. The final step was to ensure the auth cookies being set would work across the socket subdomain. I updated my cookie as such:
cookies.signed[:cookie_name] = {
value: payload,
domain: ['.socket.example.com', '.example.com']
# Some people have to specify tld_length, but I was fine without it
}
I have some remote REST API running through HTTP2. It runs through SSL using certificate. The goal is to send and receive data via HTTP2 with SSL certificate via Proxy.
There are http-2 & net-http2 gems that allow to send requests with HTTP2. But what about proxy? In a standard Net::HTTP library, there is a child class, Net::HTTP::Proxy that duplicates behavior of parent's Net::HTTP class except the fact it sends requests via proxy-server. But HTTP2 gems does not support it.
The closes idea I came up is to make something similar to Proxy implementation of http1.1 - to write "Host:" and "Proxy-Authorization: " fields to the socket, that Net-Http2 gem uses:
#socket.writeline sprintf('CONNECT %s:%s HTTP/%s',
#address, #port, HTTPVersion)
#socket.writeline "Host: #{#address}:#{#port}"
if proxy_user
credential = ["#{proxy_user}:#{proxy_pass}"].pack('m')
credential.delete!("\r\n")
#socket.writeline "Proxy-Authorization: Basic #{credential}"
end
#socket.writeline ''
But it ends up with:
SSL_connect SYSCALL returned=5 errno=0 state=SSLv2/v3 read server hello A
I might miss some technical knowledge to achieve this, so any help related to direction of research is appreciated.
After all, I finished my idea with an example present in net/http standard library and created a pull request for Net-http2 gem: https://github.com/ostinelli/net-http2/pull/11
The idea was correct, all we have to do is to send proxy "CONNECT" message using any tcp socket, with an address where we want to connect, so it creates a TCP tunnel which bypass all the data in and out, doesn't matter if it is HTTP1.1 or HTTP2, or anything else.
Here is a part of code:
def self.proxy_tcp_socket(uri, options)
proxy_addr = options[:proxy_addr]
proxy_port = options[:proxy_port]
proxy_user = options[:proxy_user]
proxy_pass = options[:proxy_pass]
proxy_uri = URI.parse("#{proxy_addr}:#{proxy_port}")
# create a regular TCP socket (with or w/o SSL, if needed)
proxy_socket = tcp_socket(proxy_uri, options)
# The majority of proxies do not explicitly support HTTP/2 protocol,
# while they successfully create a TCP tunnel
# which can pass through binary data of HTTP/2 connection.
# So we’ll keep HTTP/1.1
http_version = '1.1'
buf = "CONNECT #{uri.host}:#{uri.port} HTTP/#{http_version}\r\n"
buf << "Host: #{uri.host}:#{uri.port}\r\n"
if proxy_user
credential = ["#{proxy_user}:#{proxy_pass}"].pack('m')
credential.delete!("\r\n")
buf << "Proxy-Authorization: Basic #{credential}\r\n"
end
buf << "\r\n"
proxy_socket.write(buf)
validate_proxy_response!(proxy_socket)
proxy_socket
end
I'm trying to upload to my server (on Heroku) a file stored in a password protected FTP.
The problem is that this FTP also dont contain my production IP address on his whitelist (and i cant add it..) so i should use a proxy to connect my rails app this FTP.
I tried this code :
proxy_uri = URI(ENV['QUOTAGUARDSTATIC_URL'] || 'http://login:password#myproxy.com:9293')
Net::HTTP::Proxy(proxy_uri.host, proxy_uri.port,"login","password").start('ftp://login:password#ftp.website.com') do |http|
http.get('/path/to/myfile.gz').body
end
But my http.get returns me lookup ftp: no such host.
I also got this code for FTP download, but i dont know how to make it works with a proxy :
ftp = Net::FTP.new('ftp.myftp.com', 'login', 'password')
ftp.chdir('path/to')
ftp.getbinaryfile('myfile.gz', 'public/myfile.gz', 1024)
ftp.close
Thanks in advance.
I realise that you asked this question over 6 months ago, but I recently had a similar issue and found that this (unanswered) question is the top Google result, so I thought I would share my findings.
mudasobwa's comment below your original post has a link to the net/ftp documentation which explains how to use a SOCKS proxy...
Although you don't mention a specific requirement for a HTTP proxy in your original post, it seems obvious to me that is what you were trying to use. As I'm sure you're aware, this makes the SOCKS documentation totally irrelevant.
The following code has been tested on ruby-1.8.7-p357 using an HTTP proxy that does not require authentication:
file = File.open('myfile.gz', 'w')
http = Net::HTTP.start('myproxy.com', '9293')
resp, data = http.get('ftp://login:password#ftp.website.com')
file.write(data) if resp.code == "200"
file.close unless file.nil?
Source
This should give you a good starting point to figure the rest out for yourself.
To get you going, I would guess that you could use user:pass#myproxy.com for basic auth, or perhaps sending a Proxy-Authorization header in your GET request.
I am a beginner in Rails. I got to know exciting feature of geocoder from railscasts
[ http://railscasts.com/episodes/273-geocoder ]
But same source code also downloaded from it not working behind proxy. as it doesn't populate any longitudes or latitudes.
How to deal with with proxyserver of my workspace network?
else from another machine having direct internet connection things work fine.
geocoder has http proxy support, but it's not obvious from the documentation for where to configure it.
you can find it when looking at the initializer, that should get created for your rails generate call: https://github.com/alexreisner/geocoder/blob/master/lib/generators/geocoder/config/templates/initializer.rb
Geocoder.configure(
[...]
# :http_proxy => nil, # HTTP proxy server (user:pass#host:port)
# :https_proxy => nil, # HTTPS proxy server (user:pass#host:port)
)
I'm currently hosting both my rails app and a faye-server app on Heroku. The faye server has been cloned from here (https://github.com/ntenisOT/Faye-Heroku-Cedar) and seems to be running correctly. I have disabled websockets, as they are not supported on Heroku. Despite the claim on Faye's site that:
"Faye clients and servers transparently support cross-domain communication, so your client can connect to a server on any domain you like without further configuration."
I am still running into this error when I try to post to a faye channel:
XMLHttpRequest cannot load http://MYFAYESERVER.herokuapp.com. Origin http://MYAPPURL.herokuapp.com is not allowed by Access-Control-Allow-Origin.
I have read about CORS and tried implementing some solutions outlined here: http://www.tsheffler.com/blog/?p=428 but have so far had no luck. I'd love to hear from someone who:
1) Has a rails app hosted on Heroku
2) Has a faye server hosted on Heroku
3) Has the two of them successfully communicating with each other!
Thanks so much.
I just got my faye and rails apps hosted on heroku communicating within the past hour or so... here are my observations:
Make sure your FAYE_TOKEN is set on all of your servers if you're using an env variable.
Disable websockets, which you've already done... client.disable(...) didn't work for me, I used Faye.Transport.WebSocket.isUsable = function(_,c) { c(false) } instead.
This may or may not apply to you, but was the hardest thing to track down for me... in my dev environment, the port my application is running on will be tacked onto the end of the specified hostname for my faye server... but this appeared to cause a failure to communicate in production. I worked around that by creating a broadcast_server_uri method in application_controller.rb that handles inclusion of a port when necessary, and then use that anywhere I spin up a new channel.
....
class ApplicationController < ActionController::Base
def broadcast_server
if request.port.to_i != 80
"http://my-faye-server.herokuapp.com:80/faye"
else
"http://my-faye-server.herokuapp.com/faye"
end
end
helper_method :broadcast_server
def broadcast_message(channel, data)
message = { :ext => {:auth_token => FAYE_TOKEN}, :channel => channel, :data => data}
uri = URI.parse(broadcast_server)
Net::HTTP.post_form(uri, :message => message.to_json)
end
end
And in my app javascript, including
<script>
var broadcast_server = "<%= broadcast_server %>"
var faye;
$(function() {
faye = new Faye.Client(broadcast_server);
faye.setHeader('Access-Control-Allow-Origin', '*');
faye.connect();
Faye.Transport.WebSocket.isUsable = function(_,c) { c(false) }
// spin off your subscriptions here
});
</script>
FWIW, I wouldn't stress about setting Access-Control-Allow-Origin as it doesn't seem to be making a difference either way - I see XMLHttpRequest cannot load http://... regardless, but this should still works well enough to get you unblocked. (although I'd love to learn of a cleaner solution...)
Can't say I have used Rails/Faye on Heroku but have you tried setting the Access-Control-Allow-Origin header to something like Access-Control-Allow-Origin: your-domain.com?
For testing you could also do Access-Control-Allow-Origin: * to see if that helps
Custom headers
Some services require the use of additional HTTP headers to connect to
their Bayeux server. You can add these headers using the setHeader()
method, and they will be sent if the underlying transport supports
user-defined headers (currently long-polling only).
client.setHeader('Authorization', 'OAuth abcd-1234');
Source: http://faye.jcoglan.com/browser.html
So try client.setHeader('Access-Control-Allow-Origin', '*');