mysql-proxy for a remote MySQL server - mysql-proxy

My program works with a remote MySQL server, and I want to create a local proxy server on my computer in order to make things faster (pool connections, cache queries etc.). I went by the documentation: http://dev.mysql.com/doc/refman/5.1/en/mysql-proxy.html
I ran, on one console:
erelsgl#ubuntu:~$ mysql-proxy --proxy-backend-addresses=$MYSQL --log-level=debug2011-11-22 09:43:30: (message) mysql-proxy 0.8.2 started
2011-11-22 09:43:30: (debug) max open file-descriptors = 1024
2011-11-22 09:43:30: (message) proxy listening on port :4040
2011-11-22 09:43:30: (message) added read/write backend: qa-srv:3308
And on another console:
erelsgl#ubuntu:~$ mysql --user root --host=localhost --port=4040
Welcome to the MySQL monitor. Commands end with ; or \g.
Your MySQL connection id is 81
Server version: 5.1.41-3ubuntu12.10 (Ubuntu)
Type 'help;' or '\h' for help. Type '\c' to clear the current input statement.
mysql> show databases;
However, instead of seeing the databases on the remote server (qa-srv) I saw the databases on my local server!
I also tried this:
erelsgl#ubuntu:~$ mysql --user root --host=qa-srv --port=4040
ERROR 2003 (HY000): Can't connect to MySQL server on 'qa-srv' (111)
I also tried running the proxy instead of the mysql daemon:
erelsgl#ubuntu:~$ sudo service mysql stop
mysql stop/waiting
erelsgl#ubuntu:~$ mysql-proxy --proxy-backend-addresses=$MYSQL --log-level=debug --proxy-address=0.0.0.0:3306
2011-11-22 14:14:25: (message) mysql-proxy 0.8.2 started
2011-11-22 14:14:25: (debug) max open file-descriptors = 1024
2011-11-22 14:14:25: (message) proxy listening on port 0.0.0.0:3306
2011-11-22 14:14:25: (message) added read/write backend: qa-srv:3308
And on another console:
erelsgl#ubuntu:~$ mysql --user root
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/var/run/mysqld/mysqld.sock' (2)
How can I tell my mysql client to connect to the remote server via the local proxy?

Hurray! I found the solution.
On console A:
erelsgl#ubuntu:~$ mysql-proxy --proxy-backend-addresses=qa-srv:3308 --log-level=debug --proxy-address=127.0.0.1:3306
2011-11-22 18:55:02: (message) mysql-proxy 0.8.2 started
2011-11-22 18:55:02: (debug) max open file-descriptors = 1024
2011-11-22 18:55:02: (message) proxy listening on port 127.0.0.1:3306
2011-11-22 18:55:02: (message) added read/write backend: qa-srv:3308
On console B:
erelsgl#ubuntu:~$ mysql -u <username-on-remote-database> -h 127.0.0.1
Or, alternatively, if you still want your local database active:
On console A:
erelsgl#ubuntu:~$ mysql-proxy --proxy-backend-addresses=qa-srv:3308 --log-level=debug
2011-11-22 18:55:02: (message) mysql-proxy 0.8.2 started
2011-11-22 18:55:02: (debug) max open file-descriptors = 1024
2011-11-22 18:55:02: (message) proxy listening on port 127.0.0.1:3306
2011-11-22 18:55:02: (message) added read/write backend: qa-srv:3308
On console B:
erelsgl#ubuntu:~$ mysql -u <username-on-remote-database> -h 127.0.0.1 -P 4040

Related

connection to server on socket "/var/run/postgresql/.s.PGSQL.5432" failed

I'm new to this so... my understanding is that postgres is listening on 5433 but the rails app is trying to connect on 5432
$ pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
14 main 5433 online postgres /var/lib/postgresql/14/main /var/log/postgresql/postgresql-14-main.log
$ sudo netstat -nlp | grep 5433
tcp 0 0 127.0.0.1:5433 0.0.0.0:* LISTEN -
unix 2 [ ACC ] STREAM LISTENING 340041 - /var/run/postgresql/.s.PGSQL.5433
I have read answers to similar questions but I haven't understood them well enough to fix this problem. Am I supposed to try to get postgres to listen on 5432? Or can I direct the app to try to connect on 5433 by editing config files?
-- This is on Codespace so it's running in a container.

MongoDB 5.0 Replication: Mongo::Error::NoServerAvailable

I would like to know how to fix the following error when enabling replication on MongoDB: <Server address=db-master:27017 UNKNOWN> It only happens when enabling replication.
Error
Mongo::Error::NoServerAvailable (No nearest server is available in cluster: #<Cluster topology=ReplicaSetNoPrimary[db-master:27017,db-node2:27017,db-node1:27017,name=rs0,v=6,e=7fffffff0000000000000017]
servers=[#<Server address=db-master:27017 UNKNOWN>,#<Server address=db-node2:27017 UNKNOWN>,#<Server address=db-node1:27017 UNKNOWN>]> with timeout=30, LT=0.015)
Is this issue due to DNS resolution? Is there a way to specify the IP address instead of the alias (from hosts file) to the cluster topology?
When ssh'ing to the primary and secondary nodes pinging seems to work.
[db-node2 server] $ ping db-master
PING db-master ([IP_IS_HERE]) 56(84) bytes of data.
64 bytes from db-master ([IP_IS_HERE]): icmp_seq=1 ttl=63 time=0.153 ms
64 bytes from db-master ([IP_IS_HERE]): icmp_seq=2 ttl=63 time=0.150 m
mongo.conf
net:
port: 27017
bindIp: 0.0.0.0,localhost,127.0.0.1,db-master,[IP_IS_HERE]
bindIpAll: true
replication:
replSetName: "rs0"
ubuntu hosts
$ cat /etc/hosts
127.0.0.1 localhost
[IP_IS_HERE] db-master
[IP_IS_HERE] db-node1
[IP_IS_HERE] db-node2
Did you start the mongod service on the other hosts?
Your config does not make much sense. If you want to permit connection from any host, then use
net:
port: 27017
bindIpAll: true
or
net:
port: 27017
bindIp: 0.0.0.0
If you want to limit the connection from specific host then use
net:
port: 27017
bindIp: localhost,db-master,[IP_IS_HERE]
ssh uses port 22 but your MongoDB uses port 27017, so your firewall may block the connection (the same applies for ping). If you like to check whether connection is available you can use curl command:
curl --connect-timeout 3 --silent --show-error db-node1:27017
If you get as response
It looks like you are trying to access MongoDB over HTTP on the native driver port.
then connection is possible.
An response of
Connection timed out after 3000 milliseconds
indicates that your firewall blocks the connection attempt.

Can not connect docker mysql that's forwarding to 3306

When I'm trying to connect to a docker MySQL that's running and forwarding to my local TCP:3306 I get the following answer
mysql -u root -pPASSWORD
mysql: [Warning] Using a password on the command line interface can be insecure.
ERROR 2002 (HY000): Can't connect to local MySQL server through socket '/tmp/mysql.sock' (2)
but when I do mysql -u root -pPASSWORD -h127.0.0.1 it connects wonderfully.
Any clue?
[UPDATE]
Considering the comments and this post I create ~/.my.cnf
with this content
[Mysql]
user=root
password=yourpass
host=127.0.0.1
port=3306
Giving these changes I could connect the localhost with the 127.0.0.1 address.
If you don't specify a host with -h (or a host directive in your .my.cnf), then MySQL defaults to connect to localhost. Connections to localhost use the UNIX domain socket interface, not TCP/IP. So it's not connecting to a TCP/IP port, and therefore does not forward to your docker container.
This distinction between localhost and 127.0.0.1 is a historical oddity of MySQL. Normally localhost and 127.0.0.1 are assumed to be equivalent. But MySQL treats the hostname "localhost" as special, using it to invoke the UNIX domain socket interface. This is a bit faster than using TCP/IP, but of course only works if the connection is on the local computer.

why can't i access the docker based zookeeper port

on OS X i started kafka docker image successfully,but it seems that i can't access it on localhost
➜ ~ docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
1f931da3d661 wurstmeister/zookeeper:3.4.6 "/bin/sh -c '/usr/..." About an hour ago Up About an hour 22/tcp, 2888/tcp, 3888/tcp, 0.0.0.0:2181->2181/tcp docker_zookeeper_1
8bc36bcf8fdf wurstmeister/kafka:0.10.1.1 "start-kafka.sh" About an hour ago Up About an hour 0.0.0.0:9092->9092/tcp docker_kafka_1
➜ ~ telnet 0.0.0.0:2181
0.0.0.0:2181: nodename nor servname provided, or not known
➜ ~ telnet 0.0.0.0 2181
Trying 0.0.0.0...
telnet: connect to address 0.0.0.0: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 192.168.43.193 2181
Trying 192.168.43.193...
telnet: connect to address 192.168.43.193: Connection refused
telnet: Unable to connect to remote host
➜ ~ telnet 127.0.0.1 2181
Trying 127.0.0.1...
telnet: connect to address 127.0.0.1: Connection refused
telnet: Unable to connect to remote host
my docker file is here kafka.yml and use this command to up:
docker-compose -f src/main/docker/kafka.yml up -d
when i use
./mvnw
the console is:
2017-09-15 17:05:46.433 WARN 15871 --- [localhost:2181)] org.apache.zookeeper.ClientCnxn : Session 0x0 for server null, unexpected error, closing socket connection and attempting reconnect
java.net.ConnectException: Connection refused
at sun.nio.ch.SocketChannelImpl.checkConnect(Native Method)
at sun.nio.ch.SocketChannelImpl.finishConnect(SocketChannelImpl.java:717)
at org.apache.zookeeper.ClientCnxnSocketNIO.doTransport(ClientCnxnSocketNIO.java:361)
at org.apache.zookeeper.ClientCnxn$SendThread.run(ClientCnxn.java:1141)
how can i access the 2181 port
EDIT
docker logs 8bc36bcf8fdf
[2017-09-15 08:14:13,386] FATAL Fatal error during KafkaServerStartable startup. Prepare to shutdown (kafka.server.KafkaServerStartable)
java.lang.RuntimeException: A broker is already registered on the path /brokers/ids/1001. This probably indicates that you either have configured a brokerid that is already in use, or else you have shutdown this broker and restarted it faster than the zookeeper timeout so it appears to be re-registering.
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:393)
at kafka.utils.ZkUtils.registerBrokerInZk(ZkUtils.scala:379)
at kafka.server.KafkaHealthcheck.register(KafkaHealthcheck.scala:70)
at kafka.server.KafkaHealthcheck.startup(KafkaHealthcheck.scala:51)
at kafka.server.KafkaServer.startup(KafkaServer.scala:270)
at kafka.server.KafkaServerStartable.startup(KafkaServerStartable.scala:39)
at kafka.Kafka$.main(Kafka.scala:67)
at kafka.Kafka.main(Kafka.scala)
[2017-09-15 08:14:13,393] INFO [Kafka Server 1001], shutting down (kafka.server.KafkaServer)
docker logs 1f931da3d661
2017-09-14 08:53:05,878 [myid:] - WARN [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#357] - caught end of stream exception
EndOfStreamException: Unable to read additional data from client sessionid 0x15e7ea74c8e0000, likely client has closed socket
at org.apache.zookeeper.server.NIOServerCnxn.doIO(NIOServerCnxn.java:228)
at org.apache.zookeeper.server.NIOServerCnxnFactory.run(NIOServerCnxnFactory.java:208)
at java.lang.Thread.run(Thread.java:745)
2017-09-14 08:53:05,887 [myid:] - INFO [NIOServerCxn.Factory:0.0.0.0/0.0.0.0:2181:NIOServerCnxn#1007] - Closed socket connection for client /172.18.0.2:54222 which had sessionid 0x15e7ea74c8e0000
Have you tried using host networking as in this example? https://docs.confluent.io/current/cp-docker-images/docs/quickstart.html#zookeeper
That looks like it will simplify and solve this. I'd also recommend checking out these images instead of the custom ones it looks like you are using because these are being run in production for people so they are known to work well.

Gitlab on archlinux not working -> API not reachable

I have tried to install gitlab on archlinux following https://wiki.archlinux.org/index.php/gitlab
As 8080 is a well current port, I have switched to 8033.
When I try to connect to the website, it prints me 402 error.
If I have a look to nginx/gitlab_errors.log I have :
2015/03/23 21:16:00 [error] 29748#0: *1081 connect() failed (111: Connection refused) while connecting to upstream, client: 5.51.59.153, server: gitlab.floth.fr, request: "GET / HTTP/1.1", upstream: "http://127.0.0.1:8033/", host: "gitlab.floth.fr"
If I open /var/lib/gitlab/gitlab-shell.log I get
# Logfile created on 2015-03-23 21:09:06 +0100 by logger.rb/47272
W, [2015-03-23T21:09:06.321779 #30833] WARN -- : Failed to connect to internal API <GET http://localhost:8033/api/v3/internal/check>: #<Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 8033>
W, [2015-03-23T21:17:48.059769 #31230] WARN -- : Failed to connect to internal API <GET http://localhost:8033//api/v3/internal/check>: #<Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 8033>
W, [2015-03-23T21:22:01.846281 #31548] WARN -- : Failed to connect to internal API <GET http://localhost:8033//api/v3/internal/check>: #<Errno::ECONNREFUSED: Connection refused - connect(2) for "localhost" port 8033>
And if I run sudo -u gitlab bundle exec rake gitlab:check RAILS_ENV=production I get :
hooks directories in repos are links: ... can't check, you have no projects
Running /usr/share/webapps/gitlab-shell/bin/check
Check GitLab API access: FAILED: Failed to connect to internal API
gitlab-shell self-check failed
Try fixing it:
Make sure GitLab is running;
Check the gitlab-shell configuration file:
sudo -u gitlab -H editor /usr/share/webapps/gitlab-shell/config.yml
Please fix the error above and rerun the checks.
Checking GitLab Shell ... Finished
If I do netstat -a | grep 8033, nothing listening on that port...
Does anyone have an idea where to look for ? What service is not running because not started or failed?
Thank you for your help.
Edit
Content of gitlab-shell/config.yml
user: gitlab
gitlab_url: "http://localhost:8033/"
repos_path: "/srv/git/gitlab"
auth_file: "/var/lib/gitlab/.ssh/authorized_keys"
redis:
bin: /usr/bin/redis-cli
host: 127.0.0.1
port: 6379
database: 0
namespace: resque:gitlab
log_file: "/var/log/gitlab/gitlab-shell.log"
log_level: INFO
audit_usernames: false
git_annex_enabled: false
TADA !
I found where my configuration was bad.
All comes from the fact I have chosen an other port than 8080.
In such a case, it is important not only to modify gitlab-shell configuration (that is only the client) but the server part gitlab/config/unicorn.rb:
# Listen on both a Unix domain socket and a TCP port.
# If you are load-balancing multiple Unicorn masters, lower the backlog
# setting to e.g. 64 for faster failover.
listen "/run/gitlab/gitlab.socket", :backlog => 1024
listen "127.0.0.1:8033", :tcp_nopush => true

Resources