I'm experiencing same problem as described here Strange public listening ports in Passenger.
Phusion Passenger is randomly opening ports and I have no idea how to further investigate it.
Passenger Version : 6.0.14
ruby-2.4.1
nginx version: nginx/1.20.2
Ubuntu 18.04.6 LTS
Linux 4.15.0-192-generic
netstat -lp
tcp 0 0 localhost:45363 0.0.0.0:* LISTEN 13226/Passenger Rub
tcp 0 0 localhost:34515 0.0.0.0:* LISTEN 7653/Passenger Ruby
ps -p 7653 -o pid,vsz=MEMORY -o user,group=GROUP -o comm,args=ARGS
7653 427776 user1 user1 ruby Passenger RubyApp: /home/user1/apps/application1/current (production)
Thank you in advance for any tips how to solve this problem.
It looks like this is by design. They can be used to manage the associated backend apps in some ways. See here: https://github.com/phusion/passenger/pull/2173 for a discussion.
Related
For some reason whenever I suspend my VM and resume it, I can no longer connect to the docker container that is hosted within the VM. Usually, I pass -p 3000:3000 to the docker container so that I can access the rails instance within it and this works fine, but when I suspend the VM and resume it later, I can no longer connect to port 3000 even though it's listening within the docker image.
This results in me having to reboot the VM as service docker restart does not change anything.
Is there something else I should be looking at to resolve this issue? I've been suspending/resuming my VM with docker in it for quite awhile and have never run into this issue before.
EDIT
To reproduce this issue, I simply resumed my VM and tried connecting to localhost port 3000 from the VM itself (not within the docker image) and it cannot connect. However, below shows that port 3000 is listening:
[root:kali:~/app]# curl http://localhost:3000
curl: (56) Recv failure: Connection reset by peer
[root:kali:~/app]# netstat -antp | grep -i listen
tcp 0 0 127.0.0.1:43050 0.0.0.0:* LISTEN 84770/autossh
tcp 0 0 0.0.0.0:111 0.0.0.0:* LISTEN 1/systemd
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN 20478/sshd
tcp6 0 0 :::3000 :::* LISTEN 32731/docker-proxy
tcp6 0 0 :::3001 :::* LISTEN 32715/docker-proxy
tcp6 0 0 :::111 :::* LISTEN 1/systemd
tcp6 0 0 :::22 :::* LISTEN 20478/sshd
From within docker, I can see that rails is working:
[root:77f444beafff:~/app]# rails s --binding 0.0.0.0
=> Booting Puma
=> Rails 5.2.3 application starting in development
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.12.1 (ruby 2.5.1-p57), codename: Llamas in Pajamas
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://0.0.0.0:3000
Use Ctrl-C to stop
And here's the netstat from within docker:
[root:77f444beafff:~/app]# netstat -antp | grep -i listen
tcp 0 0 0.0.0.0:6379 0.0.0.0:* LISTEN 478/redis-server *:
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 765/puma 3.12.1 (tc
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN -
tcp6 0 0 :::6379
If I curl from within the docker image, I can see it hits the rails app just fine:
[root:77f444beafff:~/app]# curl http://localhost:3000/ -I
HTTP/1.1 200 OK
X-Frame-Options: SAMEORIGIN
X-XSS-Protection: 1; mode=block
X-Content-Type-Options: nosniff
X-Download-Options: noopen
X-Permitted-Cross-Domain-Policies: none
Referrer-Policy: strict-origin-when-cross-origin
Content-Type: text/html; charset=utf-8
ETag: W/"5078d30a6c1a5f6fc5cb7f9a82cd89f5"
Cache-Control: max-age=0, private, must-revalidate
Set-Cookie: _vspm_session=Cace%2FN0zB%2F6QJOiietbuHxTHOMZUMuRmEukYqQTNaHQ91hskaN%2BPJzev0KdGUAAtYx9a35Mqdkr8eRkPdH4qOl6vOaCcPU0gy8s7IMfkb9VhRGPPbecepmI%2F9leA2dnD694P8ctXSBklOCnjhN0%3D--SglWrWvx3BFEAI3z--IkylACdXbR6eF27Hgn0Cgg%3D%3D; path=/; HttpOnly
X-Request-Id: 29aa7251-f29a-4309-adec-6af479e7bd9b
X-Runtime: 12.241723
I'm having exactly the same issue with my VMWare virtual machine (VMWare running on Windows).
The only workaround that is working for me is:
docker stop $(docker ps -aq) && sudo systemctl restart NetworkManager docker
If I had to guess I would say it may be related to some firewall rules docker setup on start, maybe when you resume the virtual machine a change in the network configuration breaks those rules.
Similar issue: https://github.com/docker/for-mac/issues/1990 (Doesn't seem specific to docker for mac).
I was able to solve this issue with the hint given by lannox in the comment. It's necessary to mark the network interfaces of the docker containers as unmanaged by NetworkManager.
To do that, create a new file /etc/NetworkManager/conf.d/10-unmanage-docker-interfaces.conf with the following content:
[keyfile]
unmanaged-devices=interface-name:docker*;interface-name:veth*;interface-name:br-*;interface-name:vmnet*;interface-name:vboxnet*
This configures NetworkManager to ignore all interfaces with names docker*,
veth*,
br-*,
vmnet*, and
vboxnet* interfaces.
Then restart NetworkManager with sudo systemctl restart NetworkManager.
Next time the host suspends and resumes, the docker containers keep their network connectivity.
Several questions here that might help you solve this :
Is your docker container still running? Run docker ps and find your container
Since the -p 3000:3000 option is set I guess the port is exposed, but you might want to check you really have run your container with this option this time
Is your app really listening? Run lsof -np | grep listen and find your app listening on port 3000
Connect to your container with docker exec -it <your_container> bash and try running lsof -np | grep listen to see if this is a docker issue or your app
It seems that when you run netstat on your VM you get the following line :
tcp6 0 0 :::3000 :::* LISTEN 32731/docker-proxy
On Docker you get :
tcp 0 0 0.0.0.0:3000 0.0.0.0:* LISTEN 765/puma 3.12.1 (tc
There are two differences here:
:::3000 vs 0 0.0.0.0:3000, the first means it is listening on IPv6 and the second on IPv4 (found the info on this question).
tcp6 vs tcp, again IPv6 vs IPv4.
According to this other question, it seems you have to run rails with -b :: option.
The -b option binds Rails to the specified IP, by default it is localhost. You can run a server as a daemon by passing a -d option.
Please do
sudo docker ps
If you do not got your container do
sudo docker ps -a
Does your container is stopped?
If its true so do
sudo docker start CONTAINER_ID
I am triyn to deploy two different rails app in one Ec2 i can run one each time and work ok, but i need the 2 app running at same time and that can be accessed from anywhere not only localhost,I enable (add rule) two tcp port 3000 and 3001, this is my try:
/path/app1$ rails s -d
/path/app2$ rails s -b0.0.0.0 -p 3001 -d
this is the ps -ef command output
dev+ 3028 1 0 17:10 ? 00:00:00 puma 3.11.2 (tcp://localhost:3000) [/]
dev+ 3160 1 0 17:14 ? 00:00:00 puma 3.11.3 (tcp://0.0.0.0:3001) [/]
also try to run app1 with -b0.0.0.0 so it can listen from anywhere, but same result: only 1 app is listening on 0.0.0.0.
What I am missing? How can I run two servers at same time and listen both on 0.0.0.0.
thanks
By default, according to the Rails documentation, the server will only listen on the localhost / loopback interface. This is actually confirmed in the output snippet that you posted.
In the first command for app1, you aren't telling it to listen on 0.0.0.0, so you'd need to change your first command to:
/path/app1$ rails s -b0.0.0.0 -p 3000 -d
Both applications can listen on 0.0.0.0, but they can't share the same port. You have already configured app1 to listen on port 3000 (Rails default), and app2 to listen on port 3001, so they should both coexist peacefully, once you make the change above.
See also: What does binding a Rails Server to 0.0.0.0 buy you?
I am getting this error when attempting to start up a rails 4.1.1 server:
Listening on 0.0.0.0:3000, CTRL+C to stop
Exiting
/Users/darrenburgess/.rvm/gems/ruby-2.1.2#myflix/gems/eventmachine-1.0.0/lib/eventmachine.rb:526:in `start_tcp_server': no acceptor (port is in use or requires root privileges) (RuntimeError)
I have tried the following commands to find and kill the process, however none of them reveal any servers running on 3000
ps ax | grep rails
ps ax | grep ruby
lsof -i TCP | grep LISTEN
lsof -i :3000
These, from my research on stack overflow, seem to be all of the available methods for discovering running ports.
In a rails 5 application I am getting the following similar error:
Listening on tcp://0.0.0.0:3000
Exiting
/Users/darrenburgess/.rvm/gems/ruby-2.3.1/gems/puma-3.7.0/lib/puma/binder.rb:269:in `initialize': Address already in use - bind(2) for "0.0.0.0" port 3000 (Errno::EADDRINUSE)
Note that I can start rails servers on other ports.
This error persists even after machine reboot. Seems I have exhausted all avenues for finding and killing ports in use. What other things can I try?
UPDATE:
#hjpotter92 suggests running:
netstat -lntp | grep 3000
This however does not work as an option is required for the p argument. According to man netstat the list of protocols is found in etc/protocols.
I looked in that file and found that tcp is a listed protocol. However, this command does not return any output:
netstat -lntp tcp | grep 3000
Nor does this command return anything either:
netstat -lnt | grep 3000
You can try scanning the port like this lsof -i :3000 and then kill the process using sudo kill -9 <PID>.
Well, it turns out the answer to this is fairly obscure. The Node instance of FileMaker server 16 is running on port 3000. I was running a FileMaker server on the my Rails development machine.
This command helped to discover that:
sudo lsof -P -i :3000
Result
node 562 fmserver 20u IPv6 0x3ef1908b38776fe5 0t0 TCP *:3000 (LISTEN)
I could kill that process, however elected instead to disable the Node instance (The FileMaker REST/Data API).
Documentation here shows that FileMaker 16 is using that port.
http://help.filemaker.com/app/answers/detail/a_id/16319
Excuse me for the noob question.
I have a Ubuntu 16.04.1 LTS (GNU/Linux 4.4.0-63-generic x86_64) server and my public IP is something like 183.xx.xxx.xx (curl -4 icanhazip.com).I am running my rails app using the command:
rails s -b0.0.0.0 -p3002
and puma server has started successfully
=> Booting Puma
=> Rails 5.0.3 application starting in development on http://localhost:3002
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 3.9.1 (ruby 2.4.0-p0), codename: Private Caller
* Min threads: 5, max threads: 5
* Environment: development
* Listening on tcp://0.0.0.0:3002
Use Ctrl-C to stop
I tried to access my app in the browser using:
http://183.xx.xxx.xx:3002
I am getting
This site can’t be reached
172.xx.xxx.xx took too long to respond.
ERR_CONNECTION_TIMED_OUT
netstat -plnt
tcp 0 0 0.0.0.0:45109 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:22 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:631 0.0.0.0:* LISTEN -
tcp 0 0 127.0.0.1:5432 0.0.0.0:* LISTEN -
tcp 0 0 0.0.0.0:3002 0.0.0.0:* LISTEN 25689/0.0.0.0:3002)
tcp 0 0 127.0.0.1:6010 0.0.0.0:* LISTEN -
A process is running with a pid 25689
Any help is highly Appreciate.Thanks in Advance!
rails s -b 192.168.1.23 -p 3000
then
run
192.168.1.23:3000
You should edit your /etc/hosts file for 0.0.0.0 to point to your public ip(183.xx.xxx.xx)
Tested:
#/etc/hosts
127.0.0.1 localhost
127.0.1.1 pavan
0.0.0.0 192.xxx.x.xxx #my ip
Now I ran the rails server with rails s -b 0.0.0.0 -p 3002 and I can access it with my IP (192.xxx.x.xxx:3002)
For booting your server on public IP first you need to find-out your public ip which can be done by running command on Ubuntu terminal ifconfig
copy the ip mentioned under eth0 key as:
eth0 XXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
inet addr:192.168.X.XXX
For booting your rails server on this public Ip you have to run command:
rails s -b PUBLIC_IP_OF_SYSTEM -p PORT_TO_BOOT_ON
in your case it would be like:
rails s -b 183.xx.xxx.xx -p 3002
I have multiple Flask based webapps running in docker containers (their processes need to be isolated from the host OS). To run these apps I use uWSGI servers inside the containers. Incoming requests should hit a uWSGI FastRouter with a subscription server (as described here: http://uwsgi-docs.readthedocs.org/en/latest/SubscriptionServer.html). When starting a container, the uWSGI should announce itself based on some internal configuration as a subdomain.
So the setup looks like this:
Request ---> FastRouter ----> container | myapp1 |
|
----> container | myapp2 |
I'm trying to test this on a single host running both the fast router as well as some docker containers.
The FastRouter is started using
uwsgi --fastrouter :1717 --fastrouter-subscription-server localhost:2626 --fastrouter-subscription-slot 1000
Question 1 Do I need to do anything else to get the subscription server running? Is this started together with the fastrouter process?
The containers have two ports mapped from the host to the container: 5000 (the webapp) and 2626 (to subscribe to the fast router).
So they're started like this:
docker run -d -p 5000:5000 -p 2626:2626 myImage $PATH_TO_START/start.sh
Where in start.sh the uWSGI is started as
uwsgi --http :5000 -M --subscribe-to 127.0.0.1:2626:/test --module server --callable env --enable-threads
The output looks good, it prints at the end:
spawned uWSGI master process (pid: 58)
spawned uWSGI worker 1 (pid: 73, cores: 1)
spawned uWSGI http 1 (pid: 74)
subscribing to 127.0.0.1:2626:/test
On the host I can do
curl localhost:5001
And I see the Webserver greeting me from inside the container. However, doing
curl localhost:1717/test
gets no response.
Question 2
Am I getting anything fundamentally wrong here? Should I test differently?
Question 3
How can I debug the FastRouter?
Edit:
Still struggling with this setup. I'm using a separate VPS now for the fastrouter. It is started using
uwsgi --uid fastrouter --master --fastrouter :80 --fastrouter-subscription-server :2626 --daemonize uwsgi.log --pidfile ./uwsgi.pid --chmod-socket --chown-socket fastrouter
WARNING: Think before copying above call for your project since it opens up the subscription service publicly - my plan is to secure it afterwards using the key signing facilities provided by uwsgi since the VPS doesn't have a private network available.
netstat -anp shows
tcp 0 0 0.0.0.0:80 0.0.0.0:* LISTEN 843/uwsgi
udp 0 0 0.0.0.0:2626 0.0.0.0:* 843/uwsgi
unix 3 [ ] STREAM CONNECTED 9089 843/uwsgi
unix 3 [ ] STREAM CONNECTED 9090 843/uwsgi
unix 3 [ ] SEQPACKET CONNECTED 8764 843/uwsgi
unix 3 [ ] SEQPACKET CONNECTED 8763 843/uwsgi
Anyway, using uwsgi nodes with --http :5000 --module server --callable env --enable-threads --subscribe-to [Subscription-Server-IP-Address]2626:/test --socket-timeout 120 --listen 128 --buffer-size 32768 --honour-stdin still leads to the same result - uwsgi logs 'subscribing to', but http://[Subscription-Server-IP-Address]/test is not reachable. Is this kind of routing even possible? Every example I can find only assigns subdomains like [mysub].example.com, root domains, or root domains with some port number. This page includes a hint that the subscription server should be part of a routable address: http://projects.unbit.it/uwsgi/wiki/Example.
So I have a follow-up question:
Is the FastRouter even meant to let nodes announce new routes that haven't yet been set statically in a DNS zone file? I don't really care whether it's http://[key].example.com or http://example.com/[key], what's important is that these keys can be generated from inside a Docker container at setup time of the uwsgi server.
Generally the "dockered" apps run in a different network namespace, so loopback of a docker app is not the same of the fastrouter.
Use unix sockets for subscriptions, they are a great way for inter-namespace communication.
Your commands are good, the fastrouter is pretty verbose so if you do not see messages in its logs it is not working (and eventually you can strace it)