I have to duplicate a BLOB field from one table into another and I want to use a INSERT-SELECT query to achive this.
INSERT INTO target_table (key, data, comment)
SELECT 'my key', data, 'some comment' FROM source_table
Can this be done with the Rails API?
Of course I could always use ActiveRecord::Base.connection to send a native query to the database, but I'm hoping to find a "Rails way" to do this. (One which doesn't involve actually loading the data in my Rails application)
This is a typical scenario where using the SQL directly using ActiveRecord::Base.connection makes sense and sensibility. There can't possibly be any rails way to it as you described. Even if there were to be one, it has to load it in memory and insert it into the target table involving two models; this is insanity.
Related
Let's say I need to implement a search algorithm for a product catalog database. This would include multiple joins across the products table, manufacturers table, inventory table, etc. etc.
In .NET / MSSQL, I would isolate such logic in a DB stored procedure, then write a wrapper method in my data access layer of my .NET app to simply call this stored procedure.
How does something like this work in RoR? From my basic understanding, RoR uses its ORM by default. Does this mean, I have to move my search logic into the application layer, and write it using its ORM? The SQL stored proc is pretty intense... For performance, it needs to be in the stored procedure.
How does this work in RoR?
Edit: From the first two responses, I gather that ActiveRecord is the way to do things in Ruby. Does this mean that applications that require large complex queries with lots of joins, filtering and even dynamic SQL can (should) be re-written using ActiveRecord classes?
Thanks!
While it is possible to run raw SQL statements in Rails, using the execute method on a connection object, by doing so you will forfeit all the benefits of ActiveRecord. If you still want to go down this path, you can use it like so:
ActiveRecord::Base.connection.execute("call stored_procedure_name")
Another option to explore might be to create a "query object" to encapsulate your query logic. Inside, you could still use ActiveRecord query methods. ActiveRecord has become fairly proficient in optimizing your SQL queries, and there is still some manual tweaking you could do.
Below is a simple scaffoold for such an object:
# app/queries/search_products.rb
class SearchProducts
def initialize(params)
#search = search
end
def call
Product.where(...) # Plus additional search logic
end
end
The third option would be to go with something like elasticsearch-rails or sunspot. This will require some additional setup time and added complexity, but might pay off down the line, if your search requirements change.
Stored procedure is one of way to make the apps can be faster sometimes but it will need high costs and time to debug code for developer. Rails is using ActiveRecord ORM so if you want to use stored procedure that will lead to the main function ActiveRecord unused well.
There are some explains about rails
stored-procedures-in-ruby-on-rails and using stored procedure
Mongoid allows one to query both references and embedded documents using methods like find_by or where.
Say I have a model, Project, which embeds many Tasks. When I want to find a specific task, I am currently performing the following operation:
project = Project.find(...)
task = project.tasks.find_by(:some_field => 'Some Value')
My question is this, does the find_by query on the embedded tasks above issue another query to the Mongo database, or does it recognize that tasks are embedded in the given project and instead filter the tasks locally?
Thank you
Nope, it does not do another query.
Embedded documents are basically denormalized.
http://docs.mongodb.org/manual/core/data-model-design/#data-modeling-embedding
If you are interested in seeing the queries yourself, you can turn it on:
How can I see raw mongoDB queries with Mongoid
I have a Rails 3 application where I need to ingest an XML file provided by an external system into a Postgres database. I would like to use something like ActiveRecord-Import but this does not appear to handle upsert capabilities for Postgres, and some of the records I will be ingesting will already exist, but will need to be updated.
Most of what I'm reading recommends writing SQL on the fly, but this seems like a problem that may have been solved already. I just can't find it.
Thanks.
You can do upserting on MySQL and PostgreSQL with upsert.
If you're looking for raw speed, you could use nokogiri and upsert.
It might be easier to import the data using data_miner, which uses nokogiri and upsert internally.
If you are on PostgreSQL 9.1 you should use writeable common table expressions. Something like:
WITH updates (id) AS (
UPDATE mytable SET .....
WHERE ....
RETURNING id
)
INSERT INTO mytable (....)
SELECT ...
FROM mytemptable
WHERE id NOT IN (select id from updates);
In this case you bulk process thins in a temp table first, then it will try to update the records from the temptable according to your logic, and insert the rest.
Its a two step thing. First you need to fetch the XML File. If its provided by a user via a form that luck for you otherwise you need to fetch it using the standard HTTP lib of ruby or otherwise some gem like mechanize (which is actually really great)
The second thing is really easy. You read all the XML into a string and then you can convert it into a hash with this pice of code:
Hash.from_xml(xml_string)
Then you can parse and work with the data...
I have a standard rails application, that uses a mysql database through Active Record, with data loaded through a separate parsing process from a rather large XML file.
This was all well and good, but now I need to load data from an Oracle database, rather than the XML file.
I have no control how the database looks, and only really need a fraction of the data it contains (maybe one or two columns out of a few tables). As such, what I really want to do is make a call to the database, get data back, and put the data in the appropriate locations in my existing, Rails friendly mysql database.
How would I go about doing this? I've heard* you can (on a model by model basis) specifiy different databases for Rails Models to use, but that sounds like they use them in their entirety, (that is, the database is Rails friendly). Can I make direct Oracle calls? Is there a process that makes this easier? Can Active Record itself handle this?
A toy example:
If I need to know color, price, and location for an Object, then normally I would parse a huge XML file to get this information. Now, with oracle, color, price, and location are all in different tables, indexed by some ID (there isn't actually an "Object" table). I want to pull all this information together into my Rails model.
Edit: Sounds like what I'd heard about was ActiveRecord's "establish_connection" method...and it does indeed seem to assume one model is mapped to one table in the target database, which isn't true in my case.
Edit Edit: Ah, looks like I might be wrong there. "establish_connection" might handle my situation just fine (just gotta get ORACLE working in the first place, and I'll know for sure... If anyone can help, the question is here)
You can create a connection to Oracle directly and then have ActiveRecord execute a raw SQL statement to query your tables (plural). Off the top of my head, something like this:
class OracleModel < ActiveRecord::Base
establish_connection(:oracle_development)
def self.get_objects
self.find_by_sql("SELECT...")
end
end
With this model you can do OracleModel.get_objects which will return a set of records whereby the columns specified in the SELECT SQL statement are attributes of each OracleModel. Obviously you can probably come up with a more meaningful model name than I have!
Create an entry named :oracle_development in your config/database.yml file with your Oracle database connection details.
This may not be exactly what you are looking for, but it seems to cover you situation pretty well: http://pullmonkey.com/2008/4/21/ruby-on-rails-multiple-database-connections/
It looks like you can make an arbitrarily-named database configuration in the the database.yml file, and then have certain models connect to it like so:
class SomeModel < ActiveRecord::Base
establish_connection :arbitrary_database
#other stuff for your model
end
So, the solution would be to make ActiveRecord models for just the tables you want data out of from this other database. Then, if you really want to get into some sql, use ActiveRecord::Base.connection.execute(sql). If you need it as a the actual active_record object, do SomeModel.find_by_sql(sql).
Hope this helps!
I don't have points enough to edit your question, but it sounds like what you really need is to have another "connection pool" available to the second DB -- I don't think Oracle itself will be a problem.
Then, you need to use these alternate connections to "simply" execute a custom query within the appropriate controller method.
If you only need to pull data from your Oracle database, and if you have any ability to add objects to a schema that can see the data you require . . . .
I would simplify things by creating a view on the Oracle table that projects the data you require in a nice friendly shape for ActiveRecord.
This would mean maintaining code to two layers of the application, but I think the gain in clarity on the client-side would outweigh the cost.
You could also use the CREATE OR REPLACE VIEW Object AS SELECT tab1., tab2. FROM tab1,tab2 syntax so the view returned every column in each table.
If you need to Insert or Update changes to your Rails model, then you need to read up on the restrictions for doing Updates through a view.
(Also, you may need to search on getting Oracle to work with Rails as you will potentially need to install the Oracle client software and additional Ruby modules).
Are you talking about an one-time data conversion or some permanent data exchange between your application and the Oracle database? I think you shouldn't involve Rails in. You could just make a SQL query to the Oracle database, extract the data, and then just insert it into the MySQL database.
I have a number of custom find_by_sql queries in Rails. I would like to use eager loading with them but there doesn't seem to be a good way to do this.
I have seen the eager_custom.rb file floating around and it doesn't seem to work with Rails now. It appear Rails does eager loading differently now, using 2 queries (the regular query plus a query where the 'id IN' the ids from the first query), instead of the single join query used in the past.
My question is if I do a custom SQL query, then do 'id IN' query, is there a way to add back associated objects into the initial query results?
For example I have topics loaded with find_by_sql, then I find topic images where the topic id is in the topics ids, is there a way to add the images manually back to the topics?
Thanks
As you noticed, in Rails 2.1 a new kind of eager/pre-loading was introduced which uses multiple queries with id IN (...). This method is usually faster, especially when there are multiple associations being pre-loaded. You can use this functionality manually with find_by_sql by using the preload_associations class method inherited from ActiveRecord (not recommended). For example:
class Person
def self.find_a_special_group
people = find_by_sql("...")
preload_associations(people, [:jobs, :addresses])
return people
end
end
The preload_associations method is protected, so you must call it from within the class, and it takes (1) an array of objects, (2) an array, hash, or symbol of associations (same format as find's :include option), and (3) an options hash. See the documentation for the ActiveRecord::AssociationPreload::ClassMethods module for more details.
However, having said all of that, this technique is certainly undesirable as the Rails documentation discourages programmers from using preload_associations directly. Are you sure you have to use find_by_sql? Are you sure you know all of the options find takes? (:select, :from, :joins, :group, :having, etc) I'm not saying you don't need find_by_sql, but it might be worth a few minutes to make sure.