Yesterday i was absolutely sertain that all migrations data for EF placed in classes, placed in my solution as nested from DbMigration. But today i was dig a slightly deeper(just try to fallback to old migration with enable data loss not with nu-get and visual studio, but with code())
DbMigrator fg = new DbMigrator(new Settings() { AutomaticDataLossEnabled = true});
fg.Update("MigrationName");
And get exception, smth like "string should be truncated", those means that migrator tried to update column from big to small MaxLength attribute. So, i had excluded migration that caused this update and move this changes to migration, those create tables. The error still was occured. I got to intellitrace and it said that those(deleted) migration still was called. Looking to requests told me things like this:
SELECT [Extent1].[MigrationId] AS [MigrationId] FROM [dbo].[__MigrationHistory] AS [Extent1]
Looking to a table __MigrationsHistory and get my deleted migration there with field model that contains crypted data(don't decrypt this yet) . I was realy shocked. Does this means that all code, have written in classes is just the fake and really executed code placed here? And does anyone know, how to work with this table, register projections of migration classes to it etc. Or the once way to work with migrations is nu-get console?
I am not fully sure what your primary question is, so I will first try to answer last part about __MigrationHistory table.
Code in classes is not fake, your code in classes is compiled and run.
This table, however, really contains your database model, but it is not encrypted, it is compressed. The reason why Migrations API needs to store your model, is to be able to compare it against your current actual model and track changes for you (for example when you add a new property it will be able to tell what property you added and to perform automatic db migration).
In previous version of EF there was an EdmMetadata table where hash of your model was stored, and EF was able to detect if you made some changes to model by comparing stored and current model hash value. New version when migrations are enabled stores entire model as compressed blob, so it can do diff between the model that was used to create database and current model you are using, and make automatic migrations accordingly.
You should not work directly with this table, it is automatically populated by migrations API, but nuget console is not the only way to do migrations, you can check this resource for some insights how to do it from code.
Now, regarding your question from question title (where they are stored?), migrations are stored in code, in a class inheriting from DbMigration class that migrations API creates for you when you do Add-Migration command in nuget console. When you perform an migration (Update-Database), either from nuget package manager console or from code, API will compare your current model with versions in __MigrationsHistory to find initial version (if you have not specified it) and perform all migrations in between initial and target version (if not specified otherwise target is latest version).
I'm not really clear how you did exclude your migration that causes problems, as you need to migrate your database to version before that migration, and then delete and recreate all subsequent migrations from there.
Maybe you could solve your fallback to old version problem by implementing public override void Down() method in your migration that causes problems when trying to rollback? This method can be used to execute code which performs inverse any operations for migration.
Not directly related to question but worth mentioning, there is also pretty detailed tutorial here for EF CF.
Related
I have developed a application with Grails earlier.Now as per new requirement there is a need to modify the existing domain classes as well as adding a new classes and changing / establishing new relationship between the as well.
Now the new requirement has been implemented and I am going to deploy to production environment. however, the DBA want a script change the production database DDL. DBA is not allowing the auto create / update of database schema while bootstrapping the application.
I know how to export DDL of for creating tables. but that script will drop tables which means all data will be lost.
But I don't know how to export DDL for DDL-update (no drop tables/recreate tables). Anybody has good suggestion ?
You can not expect the existing data to get stored according to the new database schema as is.
For example, you have a table Sample with the contactNumber field with the nullable : true constraint in your existing schema and in your new schema this constraint has been changed to nullable : true & unique : true.
In such cases database will fail to keep the existing data intact or adapt to new schema.
To preserve the existing data, you may have to go through a tedious process like -
Take backup of the existing database.
Make a note of the modification you have made to the existing Domain classes.
Find out which modifications may lead to failure / data loss.
Drop the earlier database schema & Deploy the new application and let it create the database schema.
Write a script or utility which will process & store the data from database backup according to new database schema.Make sure the utility you have written has the capalibility to handle the modification(constraint, field added, field removed) done to the database schema.
The EF7 migration add command (to date, beta5) compares the model classes defined in the DbContext against the current model snapshot, creates a new migration class, and updates the model snapshot.
I need to modify the migration to make it generate different DDL SQL. As an example, EF7 uses sequences for SQL Server auto-increment values, and I would like it to use identity. However it could be any other reason. The migration remove command would physically delete the migration files and revert the model snapshot, so it's useless in this case.
There are 3 files that contain related code that look like they need to be edited:
The primary migration class: The Up and Down methods should be modified.
The DbContextModelSnapshot file contains Annotations that need to be modified
The secondary migration partial class: The badly named [migration].Designer.cs file also contains a model snapshot for the migration. I'm assuming this snapshot needs to match the model snapshot in item 2, but am not certain. The only information I have about the purpose of it is from Brice's blog , which says "It’s there in case you or a provider need to inspect the model for additional information during a migration."
Specific questions:
Do the two model snapshots need to be kept in sync in order to correctly perform migrations?
Is modifying 3 separate files the only way to edit the migration? (Although depending on the changes, the model snapshots may not have to be touched in some cases.)
Is there some EF command that would regenerate just the model snapshots, but not the migration methods?
Specific answers:
Do the two model snapshots need to be kept in sync in order to correctly perform migrations?
No, the snapshot in the migration is a last resort for provider writers. For example, SQLite can't rename columns so it could use migration's model snapshot to do a table rebuild for this operation. 99% of the time, it won't ever be used.
Is modifying 3 separate files the only way to edit the migration?
Most of the time, you should only ever edit the main migration file. In rare cases, you may need to edit the model snapshot if you're working in a team environment and you encounter a merge conflict. You can ignore the designer file; it just captures some metadata about the migration.
You may not have to edit anything if you configure your model correctly. For example, to use identity instead of sequences, override DbContext.OnModelCreating() and add modelBuilder.ForSqlServer().UseIdentity().
Is there some EF command that would regenerate just the model snapshots, but not the migration methods?
No, it shouldn't be needed since you almost never edit these files.
I'm a junior Rails developer and at work we faced the following problem:
Needed to update the value of a column only for one record.
What we did is creating a migration like this:
class DisableAccessForUser < ActiveRecord::Migration
def change
User.where(name: "User").first.update_column(:access, false)
end
end
Are migrations only for schema changes?
What other solutions do you suggest?
PS: I can only change it with code. No access to console.
The short version is, since migrations are only for schema changes, you wouldn't want to use them to change actual data in the database.
The main issue is that your data-manipulating migration(s) might be ignored by other developers if they load the DB structuring using either rake db:schema:load or rake db:reset. Both of which merely load the latest version of the structure using the schema.rb file and do not touch the migrations.
As Nikita Singh also noted in the comments, I too would say the best method of changing row data is to implement a simple rake task that can be run as needed, independent of the migration structure. Or, for a first time installation, the seed.rb file is perfect to load initial system data.
Hope that rambling helps.
Update
Found some documentation in some "official" sources:
Rails Guide for Migrations - Using Models in your Migrations. This section gives a description of a scenario in which data-manipulation in the migration files can cause problems for other developers.
Rails Guide for Migrations - Migrations and Seed Data. Same document as above, doesn't really explain why it is bad to put seed or data manipulation in the migration, merely says to put all that in the seed.rd file.
This SO answer. This person basically says the same thing I wrote above, except they provide a quote from the book Agile Web Development with Rails (3rd edition), partially written by David Heinemeier Hansson, creator of Rails. I won't copy the quote, as you can read it in that post, but I believe it gives you a better idea of why seed or data manipulation in migrations might be considered a bad practice.
Migrations are fine for schema changes. But when you work on much collaborated projects like pulling code everyday from lot of developers.
Chances are you might miss some migrations(Value update migrations..No problem for schema changes) Because migrations depends on the timestamps.
So what we do is create a rake task in a single namespace to update some table values( Be careful it does not overwrites)
And invoke all the rake task in that NameSpace whenever we update the code from Git.
Making data changes using classes in migrations is dangerous because it's not terribly future proof. Changes to the class can easily break the migration in the future.
For example, let's imagine you were to add a new column to user (sample_group) and access that column in a Rails lifecycle callback that executes on object load (e.g. after_initialize). That would break this migration. If you weren't skipping callbacks and validations on save (by using update_column) there'd be even more ways to break this migration going forward.
When I want to make data changes in migrations I typically fall back to SQL. One can execute any SQL statement in a migration by using the execute() method. The exact SQL to use depends on the database in use, but you should be able to come up with a db appropriate query. For example in MySQL I believe the following should work:
execute("UPDATE users SET access = 0 WHERE id IN (select id from users order by id limit 1);")
This is far more future proof.
There is nothing wrong with using a migration to migrate the data in your database, in the right situation, if you do it right.
There are two related things you should avoid in your migrations (as many have mentioned), neither of which preclude migrating data:
It's not safe to use your models in your migrations. The code in the User model might change, and nobody is going to update your migration when that happens, so if some co-worker takes a vacation for 3 months, comes back, and tries to run all the migrations that happened while she was gone, but somebody renamed the User model in the mean time, your migration will be broken, and prevent her from catching up. This just means you have to use SQL, or (if you are determined to keep even your migrations implementation-agnostic) include an independent copy of an ActiveRecord model directly in your migration file (nested under the migration class).
It also doesn't make sense to use migrations for seed data, which is, specifically, data that is to be used to populate a new database when someone sets up the app for the first time so the app will run (or will have the data one would expect in a brand new instance of the app). You can't use migrations for this because you don't run migrations when setting up your database for the first time, you run db:schema:load. Hence the special file for maintaining seed data: seeds.rb. This just means that if you do need to add data in a migration (in order to get production and everyone's dev data up to speed), and it qualifies as seed data (necessary for the app to run), you need to add it to seeds.rb too!
Neither of these, however, mean that you shouldn't use migrations to migrate the data in existing databases. That is what they are for. You should use them!
A migrations is simply a structured way to make database changes, both schema and data.
In my opinion there are situations in which using migrations for data changes is legitimate.
For example:
If you are holding data which is mostly constant in your database but changes annually, it is fine to make a migration each year to update it. For example, if you list the teams in a soccer league a migration would be a good way to update the current teams in each year.
If you want to mass-alter an attribute of a large table. For example if you had a slug column in your user and the name "some user" would be translated to the slug "some_user" and now you want to change it to "some.user". This is something I'd do with a migration.
Having said that, I wouldn't use a migration to change a single user attribute. If this is something which happens occasionally you should make a dashboard which will allow you to edit this data in the future. Otherwise a rake task may be a good option.
This question is old and I think rails approach changed over time here. Based on https://edgeguides.rubyonrails.org/active_record_migrations.html#migrations-and-seed-data it's OK to feed new columns with data here. To be more precise your migration code should contain also "down" block:
class DisableAccessForUser < ActiveRecord::Migration
def up
User.where(name: "User").first.update_column(:access, false)
end
def down
User.where(name: "User").first.update_column(:access, true)
end
end
If you use seeds.rb to pre-fill data, don't forget to include new column value there, too:
User.find_or_create_by(id: 0, name: 'User', access: false)
If I remember correctly, changing particular records may work, but I'm not sure about that.
In any case, it isn't a good practice, migrations should be user for schema changes only.
For updating one record I would use console. Just type 'rails console' in terminal and input code to change attributes.
Am trying to learn ASP.NET MVC and search the internet in and out including SO. A lot of questions and answers about |DataDirectory| as where and how.
When debugging the site breaks:
"The model backing the 'ASPNETDBContext' context has changed since the database was created...".
Then I checked the path: |DataDirectory|ASPNETDBContext.sdf
string path = AppDomain.CurrentDomain.GetData( "DataDirectory" ).ToString();
It points to the App_Data of the project but no file there.
The DB is entirely empty so deleting the thing is all I need.
My actual question I need to be answered: How to fix this properly?
made a follow-up: And additionally: Where is that file!
The answer to your problem lies in reading Scott Guthrie's blog
For those who are seeing this exception:
"The model backing the 'Production' context has changed since the database was created. Either manually delete/update the database, or call Database.SetInitializer with an IDatabaseInitializer instance."
Here is what is going on and what to do about it:
When a model is first created, we run a DatabaseInitializer to do things like create the database if it's not there or add seed data. The default DatabaseInitializer tries to compare the database schema needed to use the model with a hash of the schema stored in an EdmMetadata table that is created with a database (when Code First is the one creating the database). Existing databases won’t have the EdmMetadata table and so won’t have the hash…and the implementation today will throw if that table is missing. We'll work on changing this behavior before we ship the fial version since it is the default. Until then, existing databases do not generally need any database initializer so it can be turned off for your context type by calling:
Database.SetInitializer<Production>(null);
Hope this will help you resolve the issue.
I'm building a Rails application using MongoDB as the back-end and MongoMapper as the ORM tool. Suppose in version 1, I define the following model:
class SomeModel
include MongoMapper::Document
key :some_key, String
end
Later in version 2, I realize that I need a new required key on the model. So, in version 2, SomeModel now looks like this:
class SomeModel
include MongoMapper::Document
key :some_key, String
key :some_new_key, String, :required => true
end
How do I migrate all my existing data to include some_new_key? Assume that I know how to set a reasonable default value for all the existing documents. Taking this a step further, suppose that in version 3, I realize that I really don't need some_key at all. So, now the model looks like this
class SomeModel
include MongoMapper::Document
key :some_new_key, String, :required => true
end
But all the existing records in my database have values set for some_key, and it's just wasting space at this point. How do I reclaim that space?
With ActiveRecord, I would have just created migrations to add the initial values of some_new_key (in the version1 -> version2 migration) and to delete the values for some_key (in the version2 -> version3 migration).
What's the appropriate way to do this with MongoDB/MongoMapper? It seems to me that some method of tracking which migrations have been run is still necessary. Does such a thing exist?
EDITED: I think people are missing the point of my question. There are times where you want to be able to run a script on a database to change or restructure the data in it. I gave two examples above, one where a new required key was added and one where a key can be removed and space can be reclaimed. How do you manage running these scripts? ActiveRecord migrations give you an easy way to run these scripts and to determine what scripts have already been run and what scripts have not been run. I can obviously write a Mongo script that does any update on the database, but what I'm looking for is a framework like migrations that lets me track which upgrade scripts have already been run.
Check out Mongrations... I just finished reading about it and it looks like what you're after.
http://terrbear.org/?p=249
http://github.com/terrbear/mongrations
Cheers! Kapslok
One option is to use the update operation to update all of your data at once. Multi update is new in the development releases so you'll need to use one of those.
You can try this contraption I just made, but it only works with mongoid and rails 3 (beta 3) at the moment. http://github.com/adacosta/mongoid_rails_migrations . It'll be upgraded to rails 3 when it goes final.
Also another gem for MongoMapper migrations https://github.com/alexeypetrushin/mongo_mapper_ext
Mongrations is a super old gem, completely deprecated. I recommend NOT using it.
Exodus is a really cool migration framework for Mongo, that might be what you want:
https://github.com/ThomasAlxDmy/Exodus
We just build this one: https://github.com/eberhara/mongration - it is a regular node module (you can find it on npm).
We needed a good mongodb migration framework, but could not find any - so we built one.
It has lot's of better features than the regular migration frameworks:
Checksum (issues an error when a previosuly ran migration does not match its old version)
Persists migration state to mongo (there is no regular state file)
Full support to replica sets
Automatic handle rollbacks (developers must specify the rollback procedures)
Ability to run multiple migrations (sync or async) at the same time
Ability to run migrations against different databases at the same time
Hope it helps!
Clint,
You can write code to do updates -- though it seems that for updating a record based on its own fields is not supported.
In such a case, I did the following and ran it against the server:
------------------------------
records = Patient.all()
records.each do |p|
encounters = p.encounters
if encounters.nil? || encounters.empty?
mra = p.updated_at
#puts "\tpatient...#{mra}"
else
mra = encounters.last.created_at
#puts "\tencounter...#{mra}"
end
old = p.most_recent_activity
p.most_recent_activity = mra
p.save!
puts "#{p.last_name} mra: #{old} now: #{mra}"
end
------------------------------
I bet you could hook into Activerecord::Miration to automate and track your "migration" scripts.
MongoDB is a schema-less database. That's why there are no migrations. In the database itself, it doesn't matter whether the objects have the key :some_key or the key :some_other_key at any time.
MongoMapper tries to enforce some restrictions on this, but since the database is so flexible, you will have to maintain those restrictions yourself. If you need a key on every object, make sure you run a script to update those keys on pre-existing objects, or handle the case of an object that doesn't have that key as you come across them.
I am fairly new to MongoDB myself, but as far as I can see, due to the flexibility of the schema-less db this is how you will need to handle it.