I have a scenario in which I need to dump/transfer data of one user from my Rails App to another database of same configuration and tables. For instance, The application is built as
class Company < ActiveRecord::Base
has_many :depots
has_many :users
end
class Depot < ActiveRecord::Base
belongs_to :company
has_many :products
end
class User < ActiveRecord::Base
belongs_to :company
end
class Product < ActiveRecord::Base
belongs_to :depot
end
My requirement is, if companyA stops paying, I want to dump their data into another DB (databse2 for instance) to keep my actual DB clean and once they come back and start paying, I want this data back.
Second requirement is, database2 can already have some data in it. So I need to retain all the records and I want to change the IDs of companyA (as there can already be a company with the same ID) while saving in to database2 keeping associations intact. It might seem silly to do this, but that is my requirement.
I am using Postgres as my application DB.
Any helps???
You have a few options here worth investigating:
Output everything in a singular JSON file that encodes everything the client had in the database, complete with ID fields.
Dump out a series of CSV files that can be imported on the destination server.
Dump out a single .sql file that will properly restore the data simply by running it.
The first option is the most elegant, but probably requires the most work. It gives you the ability to archive your data in a neat, tidy file that's easily parsed.
The second option might be fine or could be severely ugly depending on the sorts of data you have. If there's any binary data involved that's probably not going to work, but for clean, text-only columns and tabular data it's usually fairly efficient. The advantage here is you can selectively load in parts of your data without having to commit to parsing all of it.
The third option isn't easily parsed, you need to restore it to be able to use it, but it does make insertion really, really simple. You will only have to write an archiver, no specific restoration tool is required.
Whatever approach you take you'll need to be absolutely certain that ID numbers are never, ever reissued. Do not reset your sequence generators to fill in holes, and when moving databases port these over as well and test that they're set correctly. The last thing you need is ID conflicts.
If you're really worried about ID conflicts you might want to switch to non-numeric IDs, like use a UUID for everything where conflicts are basically irrelevant, though this does not come without a cost.
Related
I'm currently working on a product where for some reaasons we decided to destroy the email column from a specific table and then delegate that column to an associated table using active record delegate method
https://apidock.com/rails/Module/delegate
My question here will be about what is the approach to follow in order to make sure that all the where clauses that uses the email colum are also delegated as well. Because it's basically need a lot of time to check all the places where we used table.where(dropped_column: value)
Example
class Station < ActiveRecord::Base
has_one :contact
delegate :email, to: :contact
end
class Contact < ActiveRecord::Base
belongs_to :station
end
I know that it is possible to switch all the queries from Station.where(email: value) to Station.joins(:contact).where('contacts.email': value) but this approach will take very long and also where clause can be written in many different ways so searching throught the code source and updating all of them is not efficient enough to cover all the cases.
If anyone faced a similar situation and managed to solved in way that saves us time and bugs I will be very glad to hear what are the approaches you followed.
Thanks.
Rails version: '5.2.3'
what is the approach to follow in order to make sure that all the where clauses that uses the email column are also delegated as well
You cannot "delegate" SQL commands. You need to update them all.
I know that it is possible to switch all the queries from Station.where(email: value) to Station.joins(:contact).where('contacts.email': value) but this approach will take very long
Yep, that's what you'll need to do, sorry!
If anyone face a similar situation and managed to solved in way that saves us time and bugs I will be very glad to hear what are the approaches you followd.
Before dropping the column, you can first do this:
class Station < ActiveRecord::Base
self.ignored_columns = ['email']
has_one :contact
delegate :email, to: :contact
# ...
end
This way, you can stop the application from being able to access the column, without actually deleting it from the database (like a 'soft delete'), which makes it easy to revert.
Do this on a branch, and make all the specs green. (If you have good test coverage, you're done! But if your coverage is poor, there may be errors after deploying...)
If it goes wrong after deploying, you could revert the whole PR, or just comment out the ignored_columns again, temporarily, while you push fixes.
Then finally, once your application is running smoothly without errors, drop the column.
I'm wondering if anyone could shed some light on the best way to test code that works with database models, but does not care what that model is.
Our app has a kind of synchronisation engine that keeps data on a remote server in sync with data on our system, and operates on a number of database models. It is also designed to work with multiple remote systems, and as such contains a number of internal data models which later get mapped to our actual database records.
As an example, we have something like this:
class RemoteResource < Syncable
maps_to ::InternalDbModel
....
end
Here, the sync engine would pull data from any of the remote servers and convert it into a RemoteResource instance, and later this would get translated into an InternalDbModel for persistence.
Due to the generic design, the sync engine doesn't care what database models it is converting to. So long as they meet certain criteria, they are valid.
My question revolves around testing this sort of code. Due to time constraints, I've had to write tests that I simply don't like - the tests involve creating records of actual models used by the app, but I don't like this for a number of reasons:
Each test has to know about the model in order to create the records
If the model has other requirements, they must be fulfilled first, which is completely irrelevant for the test itself
It is not the model that I want to test
It makes the tests brittle to any changes in those models
The syncable class is fine, as I am able to simply stub a class in the test and that works fine. But how can I do something similar for the active record model? Is there a way that I can create a kind of anonymous model class for these kind of tests? I will need to be able to persist the data as some of the sync engine process tests will require that data already exists, but i want this data to be completely independent of the main app itself.
I hope this makes sense
Example
Sync engine resource
class Integration::Resources::Account < Syncable
maps_to ::Account
string :name
string :description
belongs_to :company, Integration::Resources::Company, required: true
belongs_to :currency, Integration::Resources::Currency, required: true
end
Part being tested
I want to test a bit of code that checks that the required belongs_to relations of the Account resource as present. If they're not, it simply ignores the resources, but otherwise it continues.
Part of this involves loading the database records that are associated with the resources. So if the currency association has been set to an instance of the Integration::Resources::Account class, then the engine will retrieve that record from the database. If that record is not found, then the association is considered missing and thus the resource is ignored as it is required.
In this scenario, I currently have to:
Create a currency record
Create a company record
Instantiate a currency resource
Instantiate a company resource
THEN instantiate the account resource with the other resources, and run the test
To me, there is far too much knowledge here about the Account resource, as this is not what I am testing. I simply want to test the code that is ignoring resources with a missing association.
What I want to do is
- Create a dummy model class
- Only a single association
- Create a dummy resource class (this is ok)
- Only a single association
- Test
This way the test only knows what it needs to know. Any changes to any other code would then not break this test which stops it being so brittle.
The default behaviour for Ruby on Rails is to save changes made to collection associations.
Is there any way to change this behaviour, so that I can modify the collections in memory without the changes being written to the database.
So if I have two classes:
class Project < ActiveRecord::Base
has_many :tasks
class Task < ActiveRecord::Base
belongs_to :project
and write some code like:
Project.tasks.clear
Project.tasks << task1
Project.tasks << task2
then it automatically deletes all tasks associated with the Project and automatically writes the changes to the db.
This is a contrived example of what I'm trying to achieve. I know I could use Project.tasks.build() to add a new task to the collection without it being saved automatically, but the the tasks that I'm adding are not new tasks.They are links to a limited set of tasks defined in db. You could think of them as entries in an enumeration of tasks. In addition Project.tasks.clear immediately hits the db.
In java world, using Hibernate, I would disconnect the entity from the session and be able to modify the entity in memory until reconnecting and saving.
Thanks
Have you tried using the task_ids attribute instead?
Change your code to:
Project.tasks_ids = []
Project.tasks_ids << task1.id
Project.tasks_ids << task2.id
I know this question is a bit old, but since I tried to search a similar problem on Google I thought this might be useful to other people.
I have an app where the client will provide a list of pre-generated codes. When someone purchases a license, one of these codes will be served up.
There is a Product model that has_many :codes, and a Code model that belongs_to :product. The code model has a state which is either "assigned" or "unassigned".
How do I ensure that each code gets used only once, even if multiple processes are trying to fetch it? In the bad old days, I would lock the record, and if I couldn't lock it, move to the next one, but I'm not even sure I can do something like that in Rails.
The "bad old days" where ACID existed are still today. Read more on Rails/AR's locking on railsguides.
Item.transaction do
i = Item.lock("LOCK IN SHARE MODE").find(id)
# get rollin
i.save!
end
This is how my model looks like
User
belongs_to :computer
Computer
has_many :user
Users are created when people register for an account on the web site but computers are pre-loaded data that I create in seeds.rb/some .rake file. All is fine and good when the app is first launched and people start registering and get associated with the right computer_id. However, suppose I want to add another computer to the list
Computer.destroy_all
Computer.create({:name => "Akane"})
Computer.create({:name => "Yoda"})
Computer.create({:name => "Mojito"}) #newly added
running the rakefile the second time around will mess up the associations because computer_id in the User table refer to the old id in Computer table. Since I have run the script above, the id inside the computer table keeps incrementing without any regard to the association that user has to it. All the users now will have no reference back to its computer and all the computers now have new IDs.
Question: Is there a better way for me to pre-load data without screwing up my association? I want to be able to add new Computer without having to destroy the user's table. Destroying the computer table is fine with me and rebuilding it again but the old association that the existing users have must stay intact.
Have you thought about just running down the migration for the computers table?
lets say your migration's called db/migrate/1_create_computers.rb
$ rake db:migrate:down version=1
from the seed remove the destroy method.
You'd always keep the users table...in which by the way you must have a column computer_id
and of course the association goes
ComputerClass < ActiveRecord::Base
has_many :users #(don't forget the 's')
end