While attempting to deploy a non-default service to gcloud application with the command:
$ gcloud app deploy
I get the error (right after all gems finish installing):
...
Step #1: /opt/rbenv/versions/2.4.1/lib/ruby/gems/2.4.0/gems/google-api-client-
0.36.4/lib/google/apis/core/http_command.rb:228:in `check_status': forbidden: The caller does
not have permission (Google::Apis::ClientError)
....
Step #1: from /opt/rbenv/versions/2.4.1/bin/rcloadenv:22:in `<main>'
Step #1: The command '/bin/sh -c gem install rcloadenv && rbenv rehash && rcloadenv flex-env-
config >> .env' returned a non-zero code: 1
Finished Step #1
ERROR
ERROR: build step 1 "gcr.io/cloud-
builders/docker#sha256:853164af9dfd5ea4613401df48ad77b61bcb2c101b4e5de261c8300ecdd6eb47"
failed: exit status 1
Note that I performed this process for another service (default) minutes before this one, which tells me my local configuration is working to some extent.
Could I be missing any extra step specific to a Rails Application ?
app.yaml
entrypoint: bundle exec rackup --port $PORT
env: flex
runtime: ruby
service: api
runtime_config:
dotenv_config: flex-env-config
automatic_scaling:
min_instances: 0
min_idle_instances: 0
env_variables:
SECRET_KEY_BASE: <redacted-key>
beta_settings:
cloud_sql_instances: <redacted-instance-info>
The error got fixed and deployment succeeded when removed this lines from app.yaml:
...
runtime_config:
dotenv_config: flex-env-config
...
Not sure why this lines were important in the first place, investigate further and update with more meaningful explanation.
Related
We have recently updated our ruby/elasticbeanstalk platform to AWS Linux 2 / Ruby (Ruby 2.7 running on 64bit Amazon Linux 2/3.2.0)
A part of our Ruby deployment is a delayed_job (daemon gem)
After many attempts to have a bash script from the .platform/hooks/postdeploy/ folder, I have offically declared I am stuck. Here is the error from eb-engine.log:
2020/12/08 04:18:44.162454 [INFO] Running platform hook: .platform/hooks/postdeploy/restart_delayed_job.sh
2020/12/08 04:18:44.191301 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPostDeployHooks]. Stop running the command. Error: Command .platform/hooks/postdeploy/restart_delayed_job.sh failed with error exit status 127
2020/12/08 04:18:44.191327 [INFO] Executing cleanup logic
2020/12/08 04:18:44.191448 [INFO] CommandService Response: {"status":"FAILURE","api_version":"1.0","results":[{"status":"FAILURE","msg":"Engine execution has encountered an error.","returncode":1,"events":[{"msg":"Instance deployment failed. For details, see 'eb-engine.log'.","timestamp":1607401124,"severity":"ERROR"}]}]}```
Here is one of many scripts I have attempted:
#!/bin/bash
#Using similar syntax as the appdeploy pre hooks that is managed by AWS
exec 3>&1 4>&2
trap 'exec 2>&4 1>&3' 0 1 2 3
exec 1>delayed_job_err.out 2>&1
# Loading environment data
# source /etc/profile.d/sh.local #created from other .ebextension file
EB_APP_USER=$(/opt/elasticbeanstalk/bin/get-config platformconfig -k AppUser)
EB_APP_CURRENT_DIR=$(/opt/elasticbeanstalk/bin/get-config platformconfig -k AppDeployDir)
#EB_APP_PIDS_DIR=/home/webapp/pids
/opt/elasticbeanstalk/bin/get-config environment | jq -r 'to_entries | .[] | "export \(.key)=\"\(.value)\""' > /tmp/envvars
source /tmp/envvars
cd /var/app
cd $EB_APP_CURRENT_DIR
su -s /bin/bash -c "bin/delayed_job restart" $EB_APP_USER```
Here is the delayed_job file:
#!/usr/bin/env ruby
require File.expand_path(File.join(File.dirname(__FILE__), '..', 'config', 'environment'))
require 'delayed/command'
Delayed::Command.new(ARGV).daemonize
As you can see I'm doing my best to load up the env variables. The delayed_job seems to run just fine as root from within the EB Linux 2 host with the env vars loaded.
total 12
-rwxrwxr-x 1 webapp webapp 179 Dec 8 04:15 001_load_envs.sh
-rw-r--r-- 1 root root 251 Dec 8 04:46 delayed_job_err.out
-rwxrwxr-x 1 webapp webapp 1144 Dec 8 04:15 restart_delayed_job.sh
[root#ip-172-16-100-178 postdeploy]# cat delayed_job_err.out
/var/app/current/vendor/bundle/ruby/2.7.0/gems/json-1.8.6/lib/json/common.rb:155: warning: Using the last argument as keyword parameters is deprecated
delayed_job: warning: no instances running. Starting...
delayed_job: process with pid 5292 started.
Any help would be appreciated..
I am also using elasticbeanstalk on Amazon Linux 2
I am using resque which needs a restart postdeploy. Following is my postdeploy hook which restarts resque workers
.platform/hooks/postdeploy/0020_restart_resque_workers.sh
#!/usr/bin/env bash
. /opt/elasticbeanstalk/deployment/env
cd /var/app/current/
su -c "RAILS_ENV=production bundle exec rake resque:restart_workers" webapp ||
echo "resque workers restarted."
true
Notice the environment variable setup. It simply executes /opt/elasticbeanstalk/deployment/env which will give you the environment.
Hope you are able to use above script by simply replcaing command to restart delayed job instead of resque workers.
I am using Travis CI to deploy github pages using latest Hugo version
v0.59.1
however the build fails with the following error.
1.36s$ curl -LO https://github.com/gohugoio/hugo/releases/download/v0.59.1/hugo_0.59.1_Linux-64bit.deb install.2
1.29s$ sudo dpkg -i hugo_0.55.4_Linux-64bit.deb
0.08s$ hugo
Building sites … ERROR 2019/11/11 10:54:14 error: failed to transform
resource: TOCSS: failed to transform "style.scss" (text/x-sass): this
feature is not available in your current Hugo version
Total in 27 ms
Error: Error building site: logged 1 error(s)
The command "hugo" exited with 255.
Below is My .travis.yml
---
install:
- wget -O /tmp/hugo.deb
https://github.com/gohugoio/hugo/releases/download/v0.59.1/hugo_0.59.1_Linux-64bit.deb
- sudo dpkg -i /tmp/hugo.deb
script:
- hugo
deploy:
- provider: script
script: bash deploy.sh
skip_cleanup: true
on:
branch: source
Any suggestions as to how to address this issue? thanks in advance.
For anyone facing similar issue, It worked for me after replacing hugo_extended_0.59.1_Linux-64bit.deb instead of hugo_0.59.1_Linux-64bit.deb
I wanted to install pgAudit on my CentOS 6 32-bit, I already installed PostgreSQL 10 + pgAdmin4 from PostgreSQL Linux run installer that I downloaded from DBEnterprise
I followed the guide from here, here are the steps I followed :
git clone https://github.com/postgres/postgres.git
go into the folder by cd postgres and then git checkout REL_10_STABLE
./configure after that make install -s
cd contrib
Clone the pgAudit extension by git clone https://github.com/pgaudit/pgaudit.git
cd pgaudit
git checkout REL_10_STABLE
make -s check
make install
I'm stuck at step 8. Here's the result :
[root#localhost pgaudit]# make -s check
============== creating temporary instance ==============
============== initializing database system ==============
pg_regress: initdb failed
Examine /root/postgres/contrib/pgaudit/log/initdb.log for the reason.
Command was: "initdb" -D "/root/postgres/contrib/pgaudit/./tmp_check/data" --no-clean --no-sync > "/root/postgres/contrib/pgaudit/log/initdb.log" 2>&1
make: *** [check] Error 2
I opened the initdb.log at /root/postgres/contrib/pgaudit/log/initdb.log like how it told me to, and it says :
Running in no-clean mode. Mistakes will not be cleaned up.
initdb: cannot be run as root
Please log in (using, e.g., "su") as the (unprivileged) user that will
own the server process.
So I tried it again after I logged in as postgres, here's the result :
bash-4.1$ make -s check
make[3]: stat: ../../src/include/utils/errcodes.h: Permission denied
/bin/sh: line 0: cd: utils/: Not a directory
make[3]: *** [../../src/include/utils/errcodes.h] Error 1
make[2]: *** [submake-errcodes] Error 2
make[1]: *** [submake-libpgport] Error 2
make: *** [submake] Error 2
I'm really new to both Linux and PostgreSQL, so I don't know why it failed and what is the solution for this. Thanks in advance!
Look like your db user not having a login shell.
You can try with:
sudo -u postgres psql template
Or:
sudo -u postgres bash
I am doing a cron tab in AWS - Elastic Beanstalk with Ruby on Rails 3, but I don't know what is wrong.
I have this code in my .ebextensions/default.config
container_commands:
01remove_old_cron_jobs:
command: "crontab -r || exit 0"
02send_test_email:
command: crontab */2 * * * * rake send_email:test
leader_only: true
I receive this error:
Failed on instance with return code: 1 Output: Error occurred during build: Command 02send_test_email failed .
UPDATE 1
I tried next:
crontab.txt
*/2 * * * * rake send_email:test > /dev/null 2>&1
default.config
02_crontab:
command: "cat .ebextensions/crontab.txt | crontab"
leader_only: true
RESULT: No errors, but it does not work.
UPDATE 2
crontab.sh
crontab -l > /tmp/cronjob
#CRONJOB RULES
echo "*/2 * * * * /usr/bin/wget http://localhost/crontabs/send_test_email > /dev/null 2>&1" >> /tmp/cronjob
#echo "*/2 * * * * rake send_email:test > /dev/null 2>&1" >> /tmp/cronjob
crontab /tmp/cronjob
rm /tmp/cronjob
echo 'Script successful executed, crontab updated.'
default.config
02_crontab:
command: "/bin/bash .ebextensions/crontab.sh"
leader_only: true
RESULT: Works with url, but not with rake task.
Updated for 2018
In order to get this to work on the latest version of Elastic Beanstalk, I added the following to my .ebextensions:
.ebextensions/0005_cron.config
files:
"/etc/cron.d/mycron":
mode: "000644"
owner: root
group: root
content: |
56 11 * * * root . /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
commands:
remove_old_cron:
command: "rm -f /etc/cron.d/*.bak"
How I got there:
There are four main issues to confront when trying to cron a rake task in AWS EB:
The first hurdle is making sure all of your EB and Rails environment variables are loaded. I beat my head against the wall a while on this one, but then I discovered this AWS forum post (login may be required). Running . /opt/elasticbeanstalk/support/envvars loads all of your environment variables.
Then we need to make sure we cd into the current app directory using cd /var/app/current.
Next we need to know where to find the bundle and rake executables. They are not installed in the normal bin directories, but are located in a directory specific to your ruby version. To find out where your executables are located, ssh into your EB server (eb ssh) and then type the following:
$ cd /var/app/current
$ which bundle
/opt/rubies/ruby-2.3.4/bin/bundle
$ which rake
/opt/rubies/ruby-2.3.4/bin/rake
You could probably guess the directory based on your ruby version, but the above commands will let you know for sure. Based on the above, your can build your rake command as:
/opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test
NOTE: If you update your ruby version, you will likely need to update your cron config as well. This is a little brittle. I'd recommend making a note in your README on this. Trust me, six months from now, you will forget.
The fourth thing to consider is logging. I'd recommend logging to the same location as your other rails logs. We do this by tacking on >> /var/app/current/log/cron.log 2>&1 to the end of our command string.
Putting all of this together leads to a cron command string of:
. /opt/elasticbeanstalk/support/envvars && cd /var/app/current && /opt/rubies/ruby-2.3.4/bin/bundle exec /opt/rubies/ruby-2.3.4/bin/rake send_email:test >> /var/app/current/log/cron.log 2>&1
Finally, I referenced the latest AWS documentation to build an .ebextensions config file for my cron command. The result was the .ebextensions/0005_cron.config file displayed at the top of this answer.
I am having the same issue. Though I figured out that the reason that rake task doesn't run correctly on eb is because of RACK_ENV, RAILS_ENV and BUNDLE_WITHOUT
Defaults of eb:
RACK_ENV: production
RAILS_ENV: production
BUNDLE_WITHOUT: test:development
When the cron runs rake task, it runs in development mode, and gives gem not found error, as gems grouped in development are not installed.
you can see this by changing your cron a bitfrom:
*/2 * * * * rake send_email:test > /dev/null 2>&1
to:
*/2 * * * * cd /var/app/current/ && /usr/bin/bundle exec /usr/bin/rake send_email:test > /tmp/cron_log 2>&1
and then checking the /tmp/cron_log file
To know the location of bundle and rake, run
which bundle
which rake
I tried setting RAILS_ENV in command in cron, but that didn't work aswell
One quick fix is to set
BUNDLE_WITHOUT to null
EDIT:
Finally I got it to work,
.ebextensions/.config
files:
"/tmp/cron_jobs" :
mode: "000777"
content: |
1 10 * * * cd /var/app/current/ && RACK_ENV=production rake some:task >> /var/app/current/log/cron_log 2>&1
encoding: plain
container_commands:
01_delete_cron_jobs:
command: "crontab -r -u webapp || exit 0"
02_add_cron_jobs:
command: "crontab /tmp/cron_jobs -u webapp"
leader_only: true
option_settings:
- option_name: RAILS_ENV
value: production
- option_name: RACK_ENV
value: production
Notice the '-u webapp' when removing and adding cron, this will run this cron under user webapp. The above will also run in production mode. And the output will be dumped in log/cron_log file.
If the above wont work then adding 'Bundle exec' before 'rake some:task' might work.
I've seen these used with separate files in .ebextensions, such as:
02send_test_email:
command: "cat .ebextensions/crontab | crontab"
leader_only: true
I haven't gotten around to it yet, but I took note of this along the way. Let us know if this works.
This stackoverflow post has much more information
After Update 1/2:
Cron doesn't know where rake is. Your application runs from /var/app/current, and you need to be running bundle exec rake from that directory.
Elastic beanstalk is horrible with logging errors, to get this right, ssh to the machine and experiment until you have the commands right, then put this back into your cron script. You can even try and re-run some of the eb scripts as found in the logs, then reverse that into your ebextensions files.
Trying to give permissions to /public/uploads with deploy hook, during cloud66 deploy to digitalocean, to get carrierwave file uploads working.
I'm receiving the following error:
Error during deployment: Error during after_rails hook: Execution of sudo
/tmp/open_folder_permissions.sh returned a non-zero exit code. Output was:
/tmp/open_folder_permissions.sh:5: syntax error, unexpected tGVAR, expecting keyword_do or
'{' or '(' sudo chmod 0775 -R $RAILS_STACK_PATH/public/uploads ^
open_folder_permissions.sh
#! /usr/bin/env ruby
#load environment variables
source /var/.cloud66_env
#assign desired permissions
sudo chmod 0775 -R $RAILS_STACK_PATH/public/uploads
deploy_hooks.yml
production:
after_rails:
- source: /.cloud66/files/open_folder_permissions.sh
destination: /tmp/open_folder_permissions.sh
target: rails
execute: true
run_on: all_servers
apply_during: all
sudo: true
What could cause the error?
Your open_folder_permissions.sh doesn't contain valid Ruby code. From the looks of it, it's supposed to be a shell script, not a Ruby script, so you should change the shebang line to a shell interpreter instead of a Ruby interpreter.
#! bin/bash is the right interpreter.