I have just configured database replication in a Rails 6.0 application. It works smoothly, but I am trying to confirm that the right database/role is used for the queries I am expecting it to (long story short: manually calling ActiveRecord::Base.connected_to for business logic reasons).
I Googled and Stack-Overflow-ed that question, but can't seem to find an answer.
Is it possible to have the SQL queries debug entries display the database name and role used?
2021-08-06T13:26:02.186960+00:00 app[web.1]: D, [2021-08-06T13:26:02.186858 #46] DEBUG -- : [c132c761-c02c-4690-a8b0-f8c29d999xxx] Shop Load (1.6ms) SELECT "shops".* FROM "shops" WHERE "shops"."slug" = 'some slug' LIMIT 1
If not, what other ways are at my disposal to assert that my replica/follower is being hit instead of the leader/master database?
If it is of any help to answer my question: I am running this on Heroku.
Thank you in advance!
Related
I have a Rails application where there is a request to create elements on a table and another one, waiting, that reads if such element was created or not.
Right now, I check the live Data and the waiting process never sees any new record for the table.
Any ideas how to force a reconnection or anything that will update the model reference?
The basic idea:
One user, through a POST, is creating a record (that i can see directly on the database).
Another piece of code, that was running before the requests, is waiting for that record, but does not find it.
I'd appreciate any insights.
The waiting request is probably using the ActiveRecord cache. Rails caches queries for the duration of a request, so if you run the exact same query multiple times in the same request, it will only hit the database the first time.
You can simulate this in the console:
Entity.cache { Entity.first; Entity.first }
In the logs, you'll see something like this (notice the second request says CACHE):
[2018-12-06T10:51:02.436 DEBUG (1476) #] Entity Load (4.9ms) SELECT "entities".* FROM "entities" LIMIT 1
[2018-12-06T10:51:02.450 DEBUG (1476) #] CACHE (0.0ms) SELECT "entities".* FROM "entities" LIMIT 1
To bypass the cache, you can use:
Entity.uncached { Entity.where(key: #key)&.last }
Using uncached will disable the cache inside the block, even if the enclosing scope is running inside a cached block.
It seems that, for a running Rails application, if a running piece of code is looking for anything that has been updated by a new request, after the first began its execution, the updated data would not show, using active_record models.
The solution: Just run a raw query.
sql = "SELECT * FROM entities WHERE key = '"+key+"' ORDER BY ID DESC LIMIT 1;"
records_array = ActiveRecord::Base.connection.execute(sql).values
I know it's a huge oversimplification and that there is probably an underlying problem, but it did solved it.
I know i'm mis-understanding how ActiveRecord maintains integrity/works: hoping someone can clarify for me explaining why the following is not working?
We have an abnormal situation: our rails app calls a compiled binary that accesses (and creates a new table rows, it does not update existing) in a shared database. We therefore run config.use_transactional_fixtures = false in rails_helper (other wise we get savepoint errors).
The data needs to commit within the scenario so this legacy app can access the data in the database during the test.
During a test we are setting up data via eval(rubyexpression) (see below for full code)
"provider = Provider.create({:provider_reseller_phone_number => '0200000000', :provider_registered_business_name => 'ProviderReseller', :provider_name => 'providerwithzero'})"
NOTE:
i know we should be using factorygirl for this, thats a different
long story
there is no additional provider model code e.g. callbacks,
hooks are anything
using debugger to pause the test (line 22), the data is not saved to the database, but it is there once the rspec completes.
We cannot figure out why!? surely data is committed after each transaction e.g. eval?
appreciate any guidance / learnings?
we've tried
using new + "save" on the provider variable but it isn't populated by the eval.
config.use_transactional_fixtures = true but this breaks our other need e.g. external process accessing DB
searching for ways to flush (But only seems to apply to "transactions')
tried searching for ways of committing "save_points" (no luck)
provider.create!
based on ()Rails 3: ActiveRecord observer: after_commit callback doesn't fire during tests, but after_save does fire run_callbacks(:commit)
spec_test_eval.rb
require 'rails_helper'
describe 'trying to test using rails populated data for external process' do
it 'populates provider and tests external process' do
initial_data = "provider = Provider.create({:provider_reseller_phone_number => '0200000000', :provider_registered_business_name => 'ProviderReseller', :provider_name => 'providerwithzero'})"
eval(initial_data)
debugger
expect Provider.all.count.eql?(1)
# using mysql to check providers table its empty
exec_path_str = "#{EXTERNALPROCESS} 1 1"
stdop_content = `#{exec_path_str}`
end
end
test.log output
ActiveRecord::SchemaMigration Load (0.2ms) SELECT `schema_migrations`.* FROM `schema_migrations`
(0.1ms) BEGIN
(0.1ms) SAVEPOINT active_record_1
SQL (0.2ms) INSERT INTO `providers` (`created_at`, `provider_name`, `provider_registered_business_name`, `provider_reseller_phone_number`, `updated_at`) VALUES ('2014-12-27 03:33:21', 'providerwithzero', 'ProviderReseller', '0200000000', '2014-12-27 03:33:21')
(0.1ms) RELEASE SAVEPOINT active_record_1
(0.2ms) SELECT COUNT(*) FROM `providers`
(0.4ms) ROLLBACK
So it seems that its the DatabaseCleaner gem thats causing this behaviour.
Having understood truncation, transaction etc. differences (database cleaner strategies we had transaction enabled which forces the surrounding transaction and rollback. But using DatabaseCleaner.strategy = :truncation) allows each ActiveRecord action to commit to the database. At a speed cost.
Given this does slow up the tests and we only need on some special tests, now searching solutions for different strategies based on flags/attributes.
A simple Rails 3 application tries to talk to SQL Server 2000 using activerecord-jdbc-adapter. I tried both microsoft jdbc driver and jtds driver. seems to connect to database OK.
when it is time to SHOW data I get this error:
ActiveRecord::StatementInvalid in PencilsController#show
ActiveRecord::JDBCError: 'ROW_NUMBER' is not a recognized function name.: SELECT t.* FROM (SELECT ROW_NUMBER() OVER(ORDER BY [pencils].id) AS _row_num, [pencils].* FROM [pencils] WHERE [pencils].[id] = 1) AS t WHERE t._row_num BETWEEN 1 AND 1
The real problem here is the DB do not support proper LIMIT and OFFSET functions. Rails 2 would have the same problem.
For one of my old projects I had to use Sybase15, which is quite similar to old SQL Server. To make limit and offset work with that DB I had to write my own adapter:
https://github.com/arkadiyk/ar-sybase-jdbc-adapter .
It uses scrollable cursors to simulate offset. You can try to use it as it is with SQL SERVER 2000 or feel free to clone it and modify for your specific needs.
Update:
The ROW_NUMBER function is called at https://github.com/jruby/activerecord-jdbc-adapter/blob/master/lib/arjdbc/mssql/limit_helpers.rb line 82 (SqlServerReplaceLimitOffset)
There is no replacement for this function. There are other ways of implementing OFFSET but there is no straight forward one.
This is kind a old, but if anyone is passing through here, i put together another solution that uses activerecord-sqlserver-adapter that can be used to connect a rails 3.2 app to sqlserver 2000
https://bitbucket.org/jose_schmidt/rails-sqlserver-adapter-sql-server-2000-friendly
I'm trying to figure out where a whole pile of extra queries are being generated by my rails app. I need some ideas on how to tackle it. Or, if someone can give me some hints, I'd be grateful.
I get these:
SQL (1.0ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
SQL (0.8ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
SQL (0.8ms) SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
repeated over and over on every request to the DB (as much as 70 times for a single request)
I tried installing a plugin that traced the source of the queries, but it really didn't help at all. I'm using the hobofields gem, dunno if that is what's doing it but I'm somewhat wedded to it at the moment
Any tips on hunting down the source of these extra queries?
Have a look at ActiveRecord gem in connection_adapters/sqlite_adapter.rb on line 173 (I'm using ActiveRecord 3.0.7) you have a function called tables that generates the exact query you posted:
SELECT name
FROM sqlite_master
WHERE type = 'table' AND NOT name = 'sqlite_sequence'
In development mode this function gets called for every table from your database, on every request.
In production this function gets called only once, when the server starts, and the result is cached.
In development for each request rails looks into your database to see what columns each table has so it can generate methods on your models like "find_by_name". Since in production is unlikely that your database changes, rails does this lookup only on server start.
It's very difficult to tell this without look into the Code.
but i am sure you write your query in a certain loop
for i in 0..70
SqliteMaster.find(:all, :conditions=>["type=? and not name=?", 'table', 'sqlite_sequesnce'])
end
So my advice is that to check all the methods which gets called after requesting certain method and look whether the query called in a loop.
I've just seen this appearing in my logs when I do a search with the metasearch gem but ONLY in development mode.
I believe that it is caused by the plugin acts-as-taggable-on.
It will check if the table or the cache column exists.
I am working on porting a Rails app to JRuby and HSQLDB. My goal is to embed a database and the site within a single JAR file for deployment at customer sites. I have the site working quite well from the JAR, with a few notable problems.
When I do the following with a pretty mundane ActiveRecord model:
#total = SessionLog.count(:id)
I get the following exception:
ActiveRecord::StatementInvalid (ActiveRecord::ActiveRecordError: Not
in aggregate function or group by clause: org.hsqldb.Expression#7be117eb
in statement [SELECT count(session_logs.id) AS count_id
FROM session_logs WHERE (created_at >= '2010-02-06' AND created_at <=
'2010-03-09' AND session_type = 'tunnel_client') ORDER BY id DESC ]:
SELECT count(session_logs.id) AS count_id FROM session_logs WHERE
(created_at >= '2010-02-06' AND created_at <= '2010-03-09' AND
session_type = 'tunnel_client') ORDER BY id DESC )
It seems clear to me that the COUNT statement is causing the trouble in HSQLDB, but I'm not sure what the solution is to fix this. SQLite3 and MySQL both process this SQL statement without issue.
I'm open to using a different database other than HSQLDB, but it needs to be embeddable into our application on the JVM. That is the appeal of HSQLDB.
You've probably found a bug in the ActiveRecord adapter - activerecord-jdbchsqldb-adapter I assume.
Can you try run the SQL directly in some non-ruby SQL session? Then maybe you can see where it's going wrong and submit a bug or (better), submit a patch.
You can try H2 Database, wire it like so. From wikipedia:
The database engine is written by Thomas Mueller. He also developed the Java database engine Hypersonic SQL [1]. In 2001, the Hypersonic SQL was stopped, and the HSQLDB Group was formed to continue work on the Hypersonic SQL code. The name H2 stands for Hypersonic 2, however H2 does not share any code with Hypersonic SQL or HSQLDB. H2 is built from scratch.