I have code which where I manually checkout a connection like
connection = User.connection_pool.checkout
Now, I wish to use this connection to query any record in DB
is there a way to achieve this.
I'm not sure what I'm suppose to do with the connection, I just obtained other then checking it back in
So ideally I want the code do this.
connection = User.connection_pool.checkout
... ## query
User.connection_pool.checkin(connection)
Note: Please don't answer me to use with_connection, the point of this question is to manage checkout/checkin manually.
Any clue ??
Well in Rails 4.2 the way to achieve this is ..
connection = User.connection_pool.checkout
User.connection_pool.instance_variable_get(:'#reserved_connections')[Thread.current.object_id] ||= conn
User.first
User.connection_pool.checkin(conn)
This Works.
conn = Model.connection_pool.checkout
#records = conn.execute("call some_stored_procedure(args)")
Model.connection_pool.checkin(conn)
I specifically gave the example of executing a stored procedure because no other methods were able to execute SPs. This question actually helped my task out!
You can also execute other queries as well, like:
conn.execute("select attr1, attr2 from models")
Related
In our app, we need to switch to read replica database and read from it for some read-only APIs.
We decided to use the around_action filter for that:
Switch DB to read_replica before the action
Yield
Switching back to master.
We decided to use establish_connection for switching, which did the job but later we noticed that it's not thread-safe i.e it causes our other threads to face "#<ActiveRecord::ConnectionNotEstablished: No connection pool with 'primary' found.>" issue. So this solution would have worked in the case of single-threaded servers.
Later we tried to create a new connection pool, as below which is thread-safe:
databases = Rails.configuration.database_configuration
resolver = ActiveRecord::ConnectionAdapters::ConnectionSpecification::Resolver.new(databases)
spec = resolver.spec(:read_replica)
pool = ActiveRecord::ConnectionAdapters::ConnectionPool.new(spec)
pool.with_connection { |conn|
execute SQL query here.
}
The only problem with the above approach is, we can only execute queries using execute method like conn.execute(sql_query) any AR ORM query we execute inside this with_connection block run on the original DB and not read_replica.
Seems like ActiveRecord do have its default connection and it's using it when we run AR ORM queries.
Not sure how can we execute the AR ORM query inside the with_connection block as User.where(id: 1..10).
Please note:
I am aware that we can do this natively in rails 6, need to skip that for now.
I am also aware of the Octopus gem, again need to skip on that.
Appreciate any help, Thanks.
I am using mongoid orm and trying to establish connection dynamically.
def establish_mongo_connection publish_detail
Mongoid.disconnect_clients
Mongoid.clients.clear
Mongoid.configure do |config|
a = config.connect_to publish_detail.server.database_name, auth_source: 'admin', auth_mech: :scram
end
Mongoid.clients[:default][:hosts] = "mongodb://#{publish_detail.server.database_user}:#{publish_detail.server.database_password}##{publish_detail.server.database_connection}"
end
After establishing the connection I am doing business logic and finally saving the records (through MongoID model objects).
This whole processing and saving objects takes somewhere around 10-12 seconds.
The problem is sometimes records are saved in different mongo server than the one it was intended for.
I am running 2 puma server instances.
Locally everything works fine but misbehaves on server.
Please let me know if I m doing something wrong and if there's a more correct way of doing this.
Thanks in advance :)
I have a set of ActiveRecord models which I am pre-loading in order to avoid N+1 issues. In order to help that process along, I'd like to be able to do some manual testing where I have (conceptually) blocks that look like the following:
# Prepare some data
data = Foo.includes(:bar, {baz: :quux}).find(27)
ActiveRecord.raise_error_if_any_queries_occur
perform_some_work_with_data(data)
ActiveRecord.back_to_normal
This seems like there should be a way to do it, but I haven't had any luck. Has anyone else done this?
You can disconnect ActiveRecord from the database and reconnect it when you are finished:
data = Foo.includes(:bar, {baz: :quux}).find(27)
Foo.connection.disconnect!
perform_some_work_with_data(data)
Foo.connection.reconnect!
This will raise an error if anything tries to run a query against the database while ActiveRecord is disconnected.
You can disconnect from database by following:
ActiveRecord::Base.connection.disconnect!
Rails.logger.info('Disconnected from database')
perform_some_work_with_data(data)
ActiveRecord::Base.establish_connection
Rails.logger.info('Connected to database')
But I think it's a little bit weird. And I prefer using bullet gem for N+1 query detection.
Queries in rails will at some point call the select_all(arel, name = nil, binds = []) method as defined here:
https://github.com/rails/rails/blob/89a7187cc0d893da67f53d3215a33043905d68ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb#L22
You can see that this method will handle the query to the select(sql, name = nil, binds = []) method. Furthermore, this method handles the query to his parent method because of the use of undef_method :select which you can se in it's definition:
https://github.com/rails/rails/blob/89a7187cc0d893da67f53d3215a33043905d68ed/activerecord/lib/active_record/connection_adapters/abstract/database_statements.rb#L354
Finally, this means that the select method that is executed is the ConnectionAdapter method, for example the posgresql adapter, as you can see in line 946 of:
https://github.com/rails/rails/blob/89a7187cc0d893da67f53d3215a33043905d68ed/activerecord/lib/active_record/connection_adapters/postgresql_adapter.rb
So, first option:
-You could edit one of those files to include some information that they were called, in a log for example.
Second, I don't know if it would work:
- You could rewrite the select_all(arel, name = nil, binds = []) to raise an exception like:
def select_all(arel, name = nil, binds = [])
raise "error"
end
and when you want to cancel this overwrite you would undef your method with:
undef_method :select_all
So you would pass the select_all to the parent.
But Again, I don't know if the second works, but the first certainly works, although it's very invasive to edit rails core to include a log functionality.
If you remove find(27) and use where(id: 27) you can append .to_sql to the end.
data = Foo.includes(:bar, {baz: :quux}).where(id: 27).to_sql
and then later execute it with:
Foo.find_by_sql(data)
You can also accomplish the same thing with Arel. Arel will let you build more complex queries which you can execute when you so desire.
I am using rails application, my rails applications logs are should be stroed in mongoDB.
I am logging each and every controller's method call and its params its date of call etc.
Here is my code in my application controller, to log the information
db = Mongo::ReplSetConnection.new([MONGODB_PROP['host'],MONGODB_PROP['port']],:refresh_mode => :sync).db(MONGODB_PROP['database'])
au = db.authenticate(MONGODB_PROP['username'],MONGODB_PROP['password'])
if au
coll = db.collection("log_info")
doc = { :tab_name => "#{params[:controller}",:date =>"#{Time.now}"}
coll.insert(doc)
end
Obviously, my code has need some standard issues. From my implementation each time the method called happend the mongoDB connection is established .So automatically connection object is increased & it will become performance issue. i want the Single DB connection whenever it requires i need to get the connection object and perform the insert operation. How can i do this.
Please help me on this.
The easiest way is to use Mongoid and create a LogInfo class. Let mongoid handle your database connections, and you simply call:
LogInfo.create(:tab_name => "#{params[:controller}",:date =>"#{Time.now}")
Ok, I have been suck on it for hours. I thought net/imap.rb with ruby 1.9 supported the idle command, but not yet.
Can anyone help me in implementing that? From here, I though this would work:
class Net::IMAP
def idle
cmd = "IDLE"
synchronize do
tag = generate_tag
put_string(tag + " " + cmd)
put_string(CRLF)
end
end
def done
cmd = "DONE"
synchronize do
put_string(cmd)
put_string(CRLF)
end
end
end
But imap.idle with that just return nil.
I came across this old question and wanted to solve it myself. The original asker has disappeared - oh well.
Here's how you get IMAP idle working on Ruby (this is super cool). This uses the quoted block in the original question, and the documentation here.
imap = Net::IMAP.new SERVER, :ssl => true
imap.login USERNAME, PW
imap.select 'INBOX'
imap.add_response_handler do |resp|
# modify this to do something more interesting.
# called every time a response arrives from the server.
if resp.kind_of?(Net::IMAP::UntaggedResponse) and resp.name == "EXISTS"
puts "Mailbox now has #{resp.data} messages"
end
end
imap.idle # necessary to tell the server to start forwarding requests.
Are you sure it isn't working? Have you looked at the strings it has sent over the socket?
After doing some digging, it looks like put_string returns nil unless you have debug enabled, which is why imap.idle returns nil.
So your idle method might very well be working since it isn't throwing errors.
Does that help explain the behavior?
If you want to use debug, use Net::IMAP.debug = true
#Peter
I've done some research on how to scale an IDLE IMAP solution. I'm now essentially thinking of 2 options.
Option 1: Run a daemon that checks the mail for all accounts on a continuous loop.
Option 2: Open an IDLE connection for every account and receive updates.
Since my app is dealing with multiple (perhaps thousands or hundreds of thousands of accounts) option 2 seems like an impossibility. I think my best bet is to go with option one, and then break the server into multiple workers after hitting some sort of maximum.
The basic code/idea is outlined here http://railspikes.com/2007/6/1/rails-email-processing
with Ruby 2.x:
the solution is described by mzolin's code chunk here:
https://stackoverflow.com/a/21345164/1786393
I just wrote a complete (but still draft) script to fetch unseen mails here
https://gist.github.com/solyaris/b993283667f15effa579
btw, comments welcome.