Background
Ubuntu 15.10
Apache/2.4.10 (Ubuntu)
Module dependencies are all enabled :
mod_rewrite
mod_proxy
mod_proxy_http
GitLab Community Edition 8.0.3
GitLab docroot : /opt/gitlab/embedded/service/gitlab-rails/public
I install GitLab this way : https://about.gitlab.com/downloads/#ubuntu1404
Problem
On my server I have a website running with apache2 (exemple-site.com).
I have GitLab running with the bundled nginx (exemple-gitlab.com)
I have 1 server, 1 IP, and multiple FQDN.
Like this, all my domain names are pointing to GitLab.
So exemple-gitlab.com point to GitLab as wanted but exemple-site.com point to GitLab too, and all others FQDN too.
Ways of solution
I think I have to (and I tried) to :
Disable the bundled nginx and configure gitlab with apache2 (hard to do for me)
Configure the bundled nginx as a reverse proxy of apache2 (hard to do for me)
MAJ : in fact the problem is Apache and bundled-nginx run on the same IP with the same port (80). And I don't want to run a website on port 81 or whatever, only port 80.
I prefer to use apache2 for all my PHP websites and I don't mind if gitlab use apache2 or bundled nginx, all I want is using all my FQDN for each of my websites and not all my FQDN redirectinf to gitlab.
Understanding
I don't understand how omnibus or rails or reverse proxy work.
I tried disabling bundled nginx in /etc/gitlab/gitlab.rb
nginx['enable'] = false
# For GitLab CI, use the following:
ci_nginx['enable'] = false
add www-data to gitlab-www group and modify :
web_server['external_users'] = ['www-data']
and adding a modified vhost.conf to apache2
I take it from https://gitlab.com/gitlab-org/gitlab-recipes/blob/master/web-server/apache/gitlab-apache24.conf
<VirtualHost *:80>
ServerName exemple-gitlab.com
ServerSignature Off
ProxyPreserveHost On
AllowEncodedSlashes NoDecode
<Location />
Require all granted
#Allow forwarding to gitlab-git-http-server
ProxyPassReverse http://127.0.0.1:8181
#Allow forwarding to GitLab Rails app (Unicorn)
ProxyPassReverse http://127.0.0.1:8080
ProxyPassReverse http://exemple-gitlab.com/
</Location>
#apache equivalent of nginx try files
RewriteEngine on
#Forward these requests to gitlab-git-http-server
RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/api/v3/projects/.*/repository/archive.* [OR]
RewriteCond %{REQUEST_URI} ^/[\w\.-]+/[\w\.-]+/(info/refs|git-upload-pack|git-receive-pack)$
RewriteRule .* http://127.0.0.1:8181%{REQUEST_URI} [P,QSA]
#Forward any other requests to GitLab Rails app (Unicorn)
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads
RewriteRule .* http://127.0.0.1:8080%{REQUEST_URI} [P,QSA,NE]
# needed for downloading attachments
/opt/gitlab/embedded/service/gitlab-rails/public
#Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
ErrorDocument 404 /404.html
ErrorDocument 422 /422.html
ErrorDocument 500 /500.html
ErrorDocument 503 /deploy.html
# /var/log/apache2.
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /var/log/apache2/logs/gitlab.example.com_error.log
CustomLog /var/log/apache2/logs/gitlab.example.com_forwarded.log common_forwarded
CustomLog /var/log/apache2/logs/gitlab.example.com_access.log combined env=!dontlog
CustomLog /var/log/apache2/logs/gitlab.example.com.log combined
</VirtualHost>
But this conf bug my apache2 :
~# systemctl status apache2.service
● apache2.service - LSB: Apache2 web server
Loaded: loaded (/etc/init.d/apache2)
Active: failed (Result: exit-code) since mar. 2015-11-10 15:41:08 CET; 1min 9s ago
Docs: man:systemd-sysv-generator(8)
Process: 18315 ExecStop=/etc/init.d/apache2 stop (code=exited, status=0/SUCCESS)
Process: 18342 ExecStart=/etc/init.d/apache2 start (code=exited, status=1/FAILURE)
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: * The apache2 configtest failed.
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: Output of config test was:
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: (2)No such file or directory: AH02291: Cannot access di...f:10
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: AH00014: Configuration check failed
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: Action 'configtest' failed.
nov. 10 15:41:08 vpsxxx.ovh.net apache2[18342]: The Apache error log may have more information.
nov. 10 15:41:08 vpsxxx.ovh.net systemd[1]: apache2.service: Control process exited, code=exited status=1
nov. 10 15:41:08 vpsxxx.ovh.net systemd[1]: Failed to start LSB: Apache2 web server.
nov. 10 15:41:08 vpsxxx.ovh.net systemd[1]: apache2.service: Unit entered failed state.
nov. 10 15:41:08 vpsxxx.ovh.net systemd[1]: apache2.service: Failed with result 'exit-code'.
Hint: Some lines were ellipsized, use -l to show in full.
The simplest way would be to have omnibus install and configure nginx on a loopback address, but then use apache in front as a reverse proxy.
Simply re-enable nginx and add the following to your /etc/gitlab/gitlab.rb:
nginx['listen_addresses'] = ['127.0.1.1']
Then reconfigure your gitlab installation.
Then, in your apache site configuration, use something similar to this:
<VirtualHost *:80>
ServerName exemple-gitlab.com
ProxyPreserveHost On
ProxyPass / http://127.0.1.1/
ProxyPassReverse / http://127.0.1.1/
</VirtualHost>
By allowing omnibus to manage the nginx configuration, you don't have to deal with modifying the web server configuration every time they move a service from the unicorn workers to the gitlab-git-http-server Go server. Apache will seamlessly proxy requests to nginx (on the loopback) which will be automatically keep up to date by omnibus during updates.
Sorry I posted it on ServerFault and found the solution :
https://serverfault.com/questions/735270/gitlab-8-0-3-with-apache2-nginx/735273#735273
I found why apache2 crashed by commenting each line (damm logs were
useless).
I just had to create /var/log/apache2/logs
ErrorLog /var/log/apache2/logs/gitlab.example.com_error.log
CustomLog /var/log/apache2/logs/gitlab.example.com_forwarded.log common_forwarded
CustomLog /var/log/apache2/logs/gitlab.example.com_access.log combined env=!dontlog
CustomLog /var/log/apache2/logs/gitlab.example.com.log combined
apache2 was crashing because the folder/directory was missing ...
So now gitlab is working on is domain and my drupal too like this :
gitlab.com:80
drupal.com:80
Exactrly as I wanted :)
Related
I am facing the following issue to start a Rails application with Apache - Passenger in Azure platform (CentOS):
App 18106 stdout:
App 18106 stderr: Rails Error:
Unable to access log file. Please ensure that /path/to/production.log exists and is writable (ie, make it writable for user and group: chmod 0664 /path/to/production.log).
The log level has been raised to WARN and the output directed to STDERR until the problem is fixed.
App 18106 stdout:
[ 2017-06-19 11:37:13.5635 18014/7f7826db7700 age/Cor/App/Implementation.cpp:304 ]: Could not spawn process for application /var/www/my_rails_app: An error occurred while starting up the preloader.
Error ID: f684beca
Error details saved to: /tmp/passenger-error-JB9Dio.html
Message from application: could not connect to server: Permission denied
Is the server running on host "localhost" (::1) and accepting
TCP/IP connections on port 5432?
could not connect to server: Permission denied
Is the server running on host "localhost" (127.0.0.1) and accepting
TCP/IP connections on port 5432?
(PG::ConnectionBad)
/usr/local/rvm/gems/ruby-2.3.1/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:654:in `initialize'
/usr/local/rvm/gems/ruby-2.3.1/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:654:in `new'
/usr/local/rvm/gems/ruby-2.3.1/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:654:in `connect'
/usr/local/rvm/gems/ruby-2.3.1/gems/activerecord-4.2.7.1/lib/active_record/connection_adapters/postgresql_adapter.rb:242:in `initialize'
I have tried chmod 0664 /path/to/production.log but nothing changed in the error as specified above.
The same setup is working fine in AWS.
Apache conf.d files:
/etc/httpd/conf.d/my_app.conf:
<VirtualHost *:80>
ServerName localhost
DocumentRoot /var/www
ErrorLog logs/rails_app_error_log
<Directory "/var/www/">
Allow from all
Options -MultiViews
</Directory>
Alias /my_rails_app /var/www/my_rails_app/public/
<Location /my_rails_app>
PassengerBaseURI /my_rails_app
PassengerAppRoot /var/www/my_rails_app
</Location>
<Directory /var/www/my_rails_app/public/ >
Options Indexes ExecCGI FollowSymLinks MultiViews
Order allow,deny
Allow from all
AllowOverride all
</Directory>
<IfModule mod_passenger.c>
PassengerUser apache
PassengerGroup apache
</IfModule>
</VirtualHost>
/etc/httpd/conf.d/passenger.conf:
LoadModule passenger_module /usr/lib64/httpd/modules/mod_passenger.so
<IfModule mod_passenger.c>
PassengerRoot /usr/local/rvm/gems/ruby-2.3.1/gems/passenger-5.1.2
PassengerDefaultRuby /usr/local/rvm/gems/ruby-2.3.1/wrappers/ruby
</IfModule>
I have also set chown -R apache:apache my_rails_app.
Note: If I run Passenger in standalone mode without using Apache, then everything works just fine.
I have searched all other relevant posts and tried to avoid the could not connect to server: Permission denied in the log, but nothing seems to be working.
Solution:
I think the issue was with SElinux and issue got resolved after running the following command:
/usr/sbin/setsebool -P httpd_can_network_connect 1
Accoring to the Passenger offical troubleshooting The Rails application reports that it's unable to start because of a permission error for Passenger + Apache and Ruby, I think the issue was caused by the permissions of your Rails application's directory.
So please first check the owner of your apache process via ps -ef|grep apache, then to check the owner & group ownership of your Rails application's directory via ls -l <path of rails>, as below.
$ ps -ef |grep apache
root 7226 1 0 16:10 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 7229 7226 0 16:10 ? 00:00:00 /usr/sbin/apache2 -k start
www-data 7230 7226 0 16:10 ? 00:00:00 /usr/sbin/apache2 -k start
<user> 7756 30915 0 16:12 pts/2 00:00:00 grep --color=auto apache
$ ls -l
total 4
drwxrwxr-x 13 <user> <group> 4096 Jun 20 16:22 myrails
Then try to change the owner & group ownership of your Rails application's directory with root or www-data via chown -R <OWNER>:<GROUP> myrails with root or www-data.
And as reference, there is a similar SO thread What permissions are needed for apache Passenger which you can refer to.
Hope it helps.
Run
chmod 777 <directory_with_logs>
This will give you execute/read/write privileges for whole dir. It possible that problem is with directory, not the file.
You can play with the numbers further to finely tune app.
I have a Rails application deployed on Apache-Passenger which runs fine when access from localhost, but doesn't run via remote access.
Let's say the server name is server.name.com. The server info is -
[kbc#server KBC]$ uname -a
Linux server.name.com 2.6.32-431.el6.x86_64 #1 SMP Fri Nov 22 03:15:09 UTC 2013 x86_64 x86_64 x86_64 GNU/Linux
[kbc#server KBC]$ cat /etc/issue
CentOS release 6.5 (Final)
Kernel \r on an \m
When I do
[kbc#server ]$ curl http://localhost:3000/, it returns the home page for the application.
But when I try to access the Rails app from my laptop, I get the following error -
→ curl http://server.name.com:3000/
curl: (7) Failed to connect to server.name.com port 3000: Connection refused
To check if I can access the server, I tried -
→ ping server.name.com:3000
ping: cannot resolve server.name.com:3000: Unknown host
But, I can ping the server by -
→ ping server.name.com
PING server.name.com (#.#.#.#): 56 data bytes
64 bytes from #.#.#.#: icmp_seq=0 ttl=61 time=1.526 ms
64 bytes from #.#.#.#: icmp_seq=1 ttl=61 time=6.624 ms
Here is the Passenger configuration -
<VirtualHost *:3000>
ServerName server.name.com
ServerAlias server.name.com
DocumentRoot /home/kbc/KBC/public
<Directory /home/kbc/KBC/public>
AllowOverride all
Options -MultiViews
</Directory>
ErrorLog /var/log/httpd/kbc_error.log
CustomLog /var/log/httpd/kbc_access.log common
</VirtualHost>
NameVirtualHost *:3000
PassengerPreStart https://server.name.com:3000/
and
LoadModule passenger_module /home/kbc/.rvm/gems/ruby-2.3.0#kbc/gems/passenger-5.0.30/buildout/apache2/mod_passenger.so
<IfModule mod_passenger.c>
PassengerRoot /home/kbc/.rvm/gems/ruby-2.3.0#kbc/gems/passenger-5.0.30
PassengerDefaultRuby /home/kbc/.rvm/wrappers/ruby-2.3.0/ruby
PassengerRuby /home/kbc/.rvm/wrappers/ruby-2.3.0/ruby
PassengerMaxPoolSize 5
PassengerPoolIdleTime 90
PassengerMaxRequests 10000
</IfModule>
Passenger-status info -
[kbc#server ]$ passenger-status
Version : 5.0.30
Date : 2016-10-17 11:30:08 -0400
Instance: bKUJ0ptp (Apache/2.2.15 (Unix) DAV/2 Phusion_Passenger/5.0.30)
----------- General information -----------
Max pool size : 5
App groups : 1
Processes : 1
Requests in top-level queue : 0
----------- Application groups -----------
/home/kbc/KBC:
App root: /home/kbc/KBC
Requests in queue: 0
* PID: 5696 Sessions: 0 Processed: 1 Uptime: 1m 45s
CPU: 0% Memory : 38M Last used: 1m 45s ago
What am I doing wrong? Please let me know if you need more information.
This sounds like a connectivity problem, not a Passenger/Apache problem. The host you're running the server on may not accept inbound connections on port 3000 (due to iptables, firewall, or security group access control rules).
Take a look at apache not accepting incoming connections from outside of localhost and Apache VirtualHost and localhost, for instance.
#Jatin, could you please post the apache main configuration ? (/etc/apache2/apache2.conf or similar)
Also, please provide the output of the following :
sudo netstat -nl
sudo iptables -L
Just for the record, the ping utility can only test connectivity at the IP layer, meaning that it can tell you whether the host at a given IP is responding. It cannot, however, tell you if a specific TCP port is open on the remote system.
Testing TCP connectivity can be achieved easily with telnet or netcat :
telnet server.name.com 3000
If you get something like :
Trying #.#.#.#...
Connected to server.name.com.
Escape character is '^]'.
then this means you can correctly access the TCP endpoint, eliminating any possibility of network-related issues. In other words, if this works, you probably have a configuration problem with Apache.
When I restart my gitlab server with sudo gitlab-ctl restart (Debian Jessie), while the server is loading, I get the 502 error, instead of the "Deploying" page. Then everything works fine. I don't understand why this happens.
Please be aware that this appears only when loading, then everything works fine. I'm not satisfied because I used to get the better "Deploying" page.
This started happening after I configured my gitlab to work through gitlab-workhorse to go through my apache server proxy (prior to that I had huge problems with web interface communication. Snapshots downloads didn't work, but now everything is fine). Before fixing that, I used to proxy gitlab's nginx.
The configuration I currently use for apache is the following, which complies to the standard configuration of Gitlab:
<VirtualHost *:443>
ServerName git.example.com
ServerSignature Off
ProxyPreserveHost On
# Ensure that encoded slashes are not decoded but left in their encoded state.
# http://doc.gitlab.com/ce/api/projects.html#get-single-project
AllowEncodedSlashes NoDecode
<Location />
# New authorization commands for apache 2.4 and up
# http://httpd.apache.org/docs/2.4/upgrading.html#access
Require all granted
#Allow forwarding to gitlab-workhorse
ProxyPassReverse http://127.0.0.1:8283
ProxyPassReverse https://git.example.com/
</Location>
RewriteEngine on
#Don't escape encoded characters in api requests
RewriteCond %{REQUEST_URI} ^/api/v3/.*
RewriteRule .* http://127.0.0.1:8283%{REQUEST_URI} [P,QSA,NE]
#Forward all requests to gitlab-workhorse except existing files like error documents
RewriteCond %{DOCUMENT_ROOT}/%{REQUEST_FILENAME} !-f [OR]
RewriteCond %{REQUEST_URI} ^/uploads/.*
RewriteRule .* http://127.0.0.1:8283%{REQUEST_URI} [P,QSA]
# needed for downloading attachments
DocumentRoot /opt/gitlab/embedded/service/gitlab-rails/public
#Set up apache error documents, if back end goes down (i.e. 503 error) then a maintenance/deploy page is thrown up.
ErrorDocument 404 /404.html
ErrorDocument 422 /422.html
ErrorDocument 500 /500.html
ErrorDocument 503 /deploy.html
#... ssl config
RequestHeader set X_FORWARDED_PROTO 'https'
RequestHeader set X-Forwarded-Ssl on
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b" common_forwarded
ErrorLog /home/myuser/logs/gitlab_error.log
CustomLog /home/myuser/logs/gitlab_forwarded.log common_forwarded
CustomLog /home/myuser/logs/gitlab_access.log combined env=!dontlog
CustomLog /home/myuser/logs/gitlab.log combined
</VirtualHost>
The only configuration I changed, is my /etc/gitlab/gitlab.rb, which I did for the proxy to work.
external_url 'https://git.example.com'
nginx['enable'] = false
web_server['external_users'] = ['www-data']
gitlab_workhorse['listen_network'] = "tcp"
gitlab_workhorse['listen_addr'] = "127.0.0.1:8283"
What is it that I'm doing wrong to get that 502 error instead of the nice "Deploying" page that I used to get before doing these changes?
If you require any additional information, please ask.
Usually I see this page after gitlab-ctl start.
My Solution is
sudo gitlab-ctl start
sudo gitlab-ctl reconfigure
refresh browser :)
Well, technically this is the way it should work, it's a 502 error after all. If you want to show the deploy page there is a manual way to set it up before each restart:
gitlab-ctl deploy-page up
Wait a minute and then again:
gitlab-ctl deploy-page down
Otherwise you can add to your Apache config:
ErrorDocument 502 /deploy.html
which will show the deploy page every time a 502 error occurs.
config server firewall to allow http and https:
sudo ufw enable
sudo ufw allow http
sudo ufw allow https
sudo ufw allow OpenSSH
sudo ufw status
Output
Status: active
To Action From
-- ------ ----
OpenSSH ALLOW Anywhere
80/tcp ALLOW Anywhere
443/tcp ALLOW Anywhere
OpenSSH (v6) ALLOW Anywhere (v6)
80/tcp (v6) ALLOW Anywhere (v6)
443/tcp (v6) ALLOW Anywhere (v6)
i am running my ruby on rails application on a mac os x 10.7 on Apache Web Server with Passenger plugin. My httpd.conf file is as follows.
LoadModule passenger_module /Users/Ahmad/.rvm/gems/ruby-1.9.3-p392/gems/passenger-3.0.19/ext/apache2/mod_passenger.so
PassengerRoot /Users/Ahmad/.rvm/gems/ruby-1.9.3-p392/gems/passenger-3.0.19
PassengerRuby /Users/Ahmad/.rvm/wrappers/ruby-1.9.3-p392/ruby
NameVirtualHost *:80
<VirtualHost *:80>
ServerName qchext.local
# !!! Be sure to point DocumentRoot to 'public'!
DocumentRoot "/Users/Ahmad/Sites/redbytes_projects/qchext/public"
<Directory "/Users/Ahmad/Sites/redbytes_projects/qchext/public">
# This relaxes Apache security settings.
AllowOverride all
# MultiViews must be turned off.
Options -MultiViews
</Directory>
</VirtualHost>
ServerName qchext.local
error_log file
[notice] caught SIGTERM, shutting down
[notice] Digest: generating secret for digest authentication ...
[notice] Digest: done
[warn] mod_bonjour: Skipping user 'Ahmad' - no valid index file.
[notice] Apache/2.2.22 (Unix) Phusion_Passenger/3.0.19 DAV/2 configured -- resuming normal operations
Please guide me what wrong i ve done.
The SIGTERM you see is used to restart Apache, did you install the latest Apache with mod_passenger and not the Apache that comes bundled with OSX (Assuming that this is what you are running RoR on).
I'm trying to configure SSL on EC2onrails with no luck. At present I am unable to even telnet into my server at port 443, it simply says trying MY.IP.ADDRESS... and stays there indefinitely. Telnet into 80 works fine.
This was my starting point. I followed the instructions exactly, and because I have a go daddy cert I created this custom default-ssl file so I could add the SSLCertificateChainFile directive:
NameVirtualHost *:443
<VirtualHost *:443>
Include /etc/apache2/sites-available/app.custom
Include /etc/apache2/sites-available/app.common
ErrorLog /mnt/log/apache2/error.log
LogLevel warn
CustomLog /mnt/log/apache2/access.log combined
# see http://httpd.apache.org/docs/2.2/ssl/ssl_intro.html and http://httpd.apache.org/docs/2.2/mod/mod_ssl.html
SSLEngine On
SSLCertificateFile /etc/ec2onrails/ssl/cert/ec2onrails-default.crt
SSLCertificateKeyFile /etc/ec2onrails/ssl/private/ec2onrails-default.key
SSLCertificateChainFile /etc/ec2onrails/ssl/cert/ec2onrails-chain.crt
RequestHeader set X_FORWARDED_PROTO 'https'
ServerName MY_SERVER_NAME
</VirtualHost>
Note that I had to add
ServerName MY_SERVER_NAME
Or else I saw the following warning at apache startup in the error.log file:
[Wed May 27 19:46:20 2009] [warn] RSA server certificate CommonName (CN) ` MY_SERVER_NAME' does NOT match server name!?
I have run cap ec2onrails:server:enable_ssl, apache boots up cleanly, regular access over port 80 works, and apache access logs indicate no request activity to port 443. I know apache is loading my default-ssl config files because if I type gobbledygook in them it complains at startup.
Has anyone else successfully gotten SSL working with EC2onRails? What else can I do to debug this issue? Right now I am using ec2onRails version 0.9.9.1 which is based on a version of Ubuntu.
OK I figured it out. Amazon's EC2 has it's own firewall as part of its "security group" concept. This firewall was blocking port 443.