Google Cloud build fails due to rails db:migrate - ruby-on-rails

I'm following the 'Running Rails on the Cloud Run environment' instructions and have hit a snag. I used their provided github repo and the google cloud shell and I had success in launching the working application.
Now, I am trying to integrate Cloud Run into my rails template. While 'Deploying the app to Cloud Run' using the cloudbuild.yaml file provided, the build crashes during database migration. I am using postgreSQL. Here are the error details:
The error
"bundle exec rails db:migrate" ->
"ActiveRecord::ConnectionNotEstablished: could not connect to server: No such file or directory"
I think I've traced it to database.yml file where Google recommends this host:
production:
  <<: *default
  database: <%= ENV["PRODUCTION_DB_NAME"] %>
  username: <%= ENV["PRODUCTION_DB_USERNAME"] %>
  password: <%= Rails.application.credentials.gcp[:db_password] %>
  host: "<%= ENV.fetch("DB_SOCKET_DIR") { '/cloudsql' } %>/<%= ENV["CLOUD_SQL_CONNECTION_NAME"] %>"
It is unclear where this ENV.fetch("DB_SOCKET_DIR") comes from [at least to me, I'm new]. Their git repo holds a folder where I found templates for another build that included an app.standard.yaml and a config/database_unix.yml that I've tried integrating.
app.standard.yaml:
entrypoint: bundle exec rackup --port $PORT
runtime: ruby27
env_variables:
SECRET_KEY_BASE: <SECRET_KEY>
RAILS_ENV: production
INSTANCE_UNIX_SOCKET: /cloudsql/<PROJECT-ID>:<INSTANCE-REGION>:<INSTANCE-NAME>
DB_USER: <YOUR_DB_USER_NAME>
DB_PASS: <YOUR_DB_PASSWORD>
DB_NAME: <YOUR_DB_NAME>
beta_settings:
cloud_sql_instances: <PROJECT-ID>:<INSTANCE-REGION>:<INSTANCE-NAME>
database_unix.yml:
# [START cloud_sql_postgres_activerecord_connect_unix]
unix: &unix
adapter: postgresql
# Configure additional properties here.
# [END cloud_sql_postgres_activerecord_connect_unix]
pool: 5
timeout: 5000
# [START cloud_sql_postgres_activerecord_connect_unix]
# Note: Saving credentials in environment variables is convenient, but not
# secure - consider a more secure solution such as
# Cloud Secret Manager (https://cloud.google.com/secret-manager) to help
# keep secrets safe.
username: <%= ENV["DB_USER"] %> # e.g. "my-database-user"
password: <%= ENV["DB_PASS"] %> # e.g. "my-database-password"
database: <%= ENV.fetch("DB_NAME") { "vote_development" } %>
# Specify the Unix socket path as host
host: "<%= ENV["INSTANCE_UNIX_SOCKET"] %>"
# [END cloud_sql_postgres_activerecord_connect_unix]
development:
<<: *unix
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *unix
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 1 } %>
database: <%= ENV.fetch("DB_NAME") { "vote_test" } %>
production:
<<: *unix
database: <%= ENV.fetch("DB_NAME") { "vote_production" } %>
Some other solutions I've seen mention this instead of host:
socket: “/cloudsql/project_id:us-central1:photo-album-production”
I tried this with no luck. I cloned the repo to my machine and re-tried the Cloud Run instructions. No success this time as it's getting the same migration error. Am I thinking about this wrong?

Check this out: https://cloud.google.com/sql/docs/postgres/connect-build.
In short, you'll need to run the Cloud SQL Auth Proxy to create a Unix socket in Cloud Build so your app can connect.
If you're using a private IP instance, you'll need to make sure you're using private pools.

Related

Incorrect credentials response in Rails 6

Rails 6
I created the credentials file, as follows:
EDITOR=vi rails credentials:edit
production:
mysql:
db: acme-production
user: deploy
password: xxxxxxxxxxxx
smartagent:
token: lnroftb7sgr8c7f1ogqvij24xl
test:
mysql:
db: acme-test
user: deploy
password: xxxxxxxxxxxx
smartagent:
token: lnroftb7sgr8c7f1ogqvij24xl
secret_key_base: xxxxxxxxxxxxxx
master.key is in the correct place (in the config folder, locally, and as an environment setting on the server).
I am having trouble with the smartagent token.
When I do:
rails c
and
Rails.application.credentials.dig(Rails.env.to_sym, :mysql, :db)
I get:
=> "acme-test"
However, when I do:
Rails.application.credentials.dig(Rails.env.to_sym, :smartagent, :token)
I get:
=> nil
Any idea why this might be happening?
Solution:
I deleted the credentials file, re-populated it, and restarted the Rails app. That fixed the problem, so this must have been either an indentation issue (spaces vs tabs), or that the server had not restarted properly, after earlier changes were made

Rails is not sending custom metrics to NewRelic

I'm trying to send custom metrics to NewRelic insights, but unfortunately it is not working for my Rails app that is currently sending default data to New Relic.
Steps to reproduce
I just logged in the console of the working application and ran the following command:
NewRelic::Agent.record_metric('/Custom/MyCategory/MyMetric', 5)
Unfortunately it never appeared in the Insights Data Explorer.
The configuration in the application is the following:
common: &default_settings
license_key: <MY_KEY>
app_name: my_app
log_level: info
development:
<<: *default_settings
app_name: executive_alerts (development)
monitor_mode: false
test:
<<: *default_settings
app_name: executive_alerts (test)
monitor_mode: false
staging:
<<: *default_settings
app_name: executive_alerts (staging)
production:
<<: *default_settings
Thank you!
From the documentation for the New Relic agent:
http://www.rubydoc.info/github/newrelic/rpm/NewRelic/Agent#record_metric-instance_method
https://github.com/newrelic/rpm/blob/c252fad410a5d41a65a827d633338af609f8dff6/lib/new_relic/agent.rb#L143-L144
metric_name should follow a slash separated path convention. Application specific metrics should begin with Custom/.
Change your command from:
NewRelic::Agent.record_metric('/Custom/MyCategory/MyMetric', 5)
to:
NewRelic::Agent.record_metric('Custom/MyCategory/MyMetric', 5)

Postgree too many connections in rails console

I am developing a Ruby on Rails app using postgre gem and this is how my database.yml looks like:
development:
adapter: postgresql
encoding: utf-8
pool: 5
username: "hytxlzju"
password: "xxxxx"
host: "jumbo.db.elephantsql.com"
port: "5432"
database: "hytxlzju"
production:
adapter: postgresql
encoding: utf-8
pool: 5
username: "hytxlzju"
password: "xxxxxx"
host: "jumbo.db.elephantsql.com"
port: "5432"
database: "hytxlzju"
Whenever I am connecting to this db locally, from the rails console I am getting too many connections. How can I kill a connection in the code, once the user logged out, in the code, and how can I kill one in my rails console, after I finished altering the tables?
[EDIT]
This is the error message:
C:/RailsInstaller/Ruby2.2.0/lib/ruby/gems/2.2.0/gems/activerecord-3.2.22.5/lib/active_record/connection_adapters/postgresql_adapter.rb:12
22:in `initialize': FATAL: too many connections for role "hytxlzju" (PG::ConnectionBad)
[EDIT] I added my initilizer, still no success:
Rails.application.config.after_initialize do
ActiveRecord::Base.connection_pool.disconnect!
ActiveSupport.on_load(:active_record) do
config = ActiveRecord::Base.configurations[Rails.env] ||
Rails.application.config.database_configuration[Rails.env]
config['pool'] = ENV['DB_POOL'] || ENV['RAILS_MAX_THREADS'] || 5
ActiveRecord::Base.establish_connection(config)
end
end
You can try the below approach
Active Record limits the total number of connections per application
through a database setting pool; this is the maximum size of the
connections your app can have to the database
in config/datbase.yml
pool: <%= ENV['RAILS_MAX_THREADS'] || 5 %>
If you are using puma then use ENV['RAILS_MAX_THREADS'] more here
It might solve the problem.
[SOLVED]
Somehow my demo app was not finding the entries in the tables, so it was creating multiple pools connections, without closing them, because before closing the connection, a 500 error was getting thrown, hence that bit of code where I closed the pool was never closed. More abut postgre session here
https://devcenter.heroku.com/articles/concurrency-and-database-connections#connection-pool

Do we need to add security.yml on heroku for deployment?

I have developed application in ROR,
My security.yml file have following entries,
admin_name: *****
admin_email: user#example.com
admin_password: *****
domain_name: ******
secret_key_base: *****
email_provider_username: ****
email_provider_password: ******
How to add these entries on heroku configration ?
I would read sensitive data from the environment.
Change your yaml files to something like this:
admin_name: <%= ENV['ADMIN_NAME'] %>
Then set the config with the Heroku cmd:
$ heroku config:set ADMIN_NAME=your_admin_name
Read the Heroku docs about config vars.

How to set up database.yml

I am trying to set up ruby on rails to develop locally on a 64 bit windows 7 machine using SQL server 2005. We have an existing database and it does not conform to the ruby way so I created a table to test conectivity.
Unable to find information on properly configuring the database.yml file I suspect my problem is there. I have both 32-bit and 64-bit DSNs defined. When I run the console I get "Table doesn't exist"
Here's my database.yml. I don't know
default: &default
host: 111.222.333.444
adapter: sqlserver
mode: odbc
dsn: DSN_Name
database: ERP
username: sa
password: ********
development:
<<: *default
host: 111.222.333.444
adapter: sqlserver
mode: odbc
dsn: DSN_Name
database: ERP
username: sa
password: ********
# Warning: The database defined as "test" will be erased and
# re-generated from your development database when you run "rake".
# Do not set this db to the same as development or production.
test:
<<: *default
database: db/test.sqlite3
production:
<<: *default
database: db/production.sqlite3
Does anyone know how the default works? I can't find any information on it. Specifically "<<: *default"
I updated my database.yml file. Created a table for testing; used the plural and ID for the primary key:
CREATE TABLE [dbo].[Cars](
[ID] [int] NULL,
[Name] [nchar](10) COLLATE SQL_Latin1_General_CP1_CI_AS NULL
) ON [PRIMARY]
Created a model
class Car < ActiveRecord::Base
end
But I get "Table doesn't exist" after connecting to Car and entering 'Car'. Anyone have any idea what the next step would be to debug this?
The &default in the line default: &default, means make the key/value pairs under this YAML namespace available as the variable *default.
The <<: *default lines are taking those key/value pairs, and making them part of the other groups, so you only need to change values that are different from the default: group.
If all your databases are on the same machine, then your database.yml should look a lot like this:
default: &default
host: 111.222.333.444
adapter: sqlserver
mode: odbc
dsn: DSN_Name
username: sa
password: ********
development:
<<: *default
database: ERP_development
test:
<<: *default
database: ERP_test
production:
<<: *default
database: ERP_production
Since you are on sqlserver, you'll probably have to create all those databases yourself (instead of letting Rails do it for you).
from: https://github.com/rails-sqlserver/activerecord-sqlserver-adapter
Force Schema To Lowercase
Although it is not necessary, the Ruby convention is to use lowercase
method names. If your database schema is in upper or mixed case, we
can force all table and column names during the schema reflection
process to be lowercase. Add this to your config/initializers file for
the adapter.
ActiveRecord::ConnectionAdapters::SQLServerAdapter.lowercase_schema_reflection = true
Schemas & Users
Depending on your user and schema setup, it may be needed to use a
table name prefix of dbo.. So something like this in your initializer
file for ActiveRecord or the adapter.
ActiveRecord::Base.table_name_prefix = 'dbo.'

Resources