EF migration Seeding with large dataset - asp.net-mvc

Question

I am using EF Code First Migration with ASP.NET MVC5 project. I want to seed the database with about 5000 records. The examples on the asp.net web site and blogs all have the seed function within the configuration method.
internal sealed class Configuration : DbMigrationsConfiguration<foo.ApplicationDbContext>
{
public Configuration()
{
AutomaticMigrationsEnabled = false;
MigrationsDirectory = #"Migrations\ApplicationDbContext";
}
protected override void Seed(foo.ApplicationDbContext context)
{
SeedProducts(context);
SeedFoods(context);
// This method will be called after migrating to the latest version.
// You can use the DbSet<T>.AddOrUpdate() helper extension method
// to avoid creating duplicate seed data. E.g.
//
// context.People.AddOrUpdate(
// p => p.FullName,
// new Person { FullName = "Andrew Peters" },
// new Person { FullName = "Brice Lambson" },
// new Person { FullName = "Rowan Miller" }
// );
//
}
}
What are some options to seed large data sets when using EF Migration?
Currently The configuration file is over 5000 lines long and is different to manage.
I would prefer to store them in another database or excel spreadsheet and then import them in using the seed function. I am not sure how to go about importing data from external sources within the Seed method.
I also tried to break up the data set into several files but when I try to call the function
SeedProducts(context);
SeedFoods(context);
outside of the Configuration Class I get a build error: "The name does not exists in the current context". (I am not sure what this means?

Answer 1

You can store the sql file in the base directory and use it. You can create multiple sql files. I used following code to execute all sql files stored on base directory.
protected override void Seed(CRM.Data.DbContexts.CRMContext context)
{
var sqlfiles = Directory.GetFiles(AppDomain.CurrentDomain.BaseDirectory+"\\initialdata", "*.sql");
sqlfiles.ToList().ForEach(x=> context.Database.ExecuteSqlCommand(File.ReadAllText(x)));
}

Answer 2

Why we need to have a seed data for 5000 records. If you prefer this way it will take lot of manual work also. where as, its not required here.
Instantly you can Create Scripts and execute that into you db. via Code as well as Manually.
Db Scripts can be taken for entire db as well as each table, Store Procedure wise also. So, that you will get records particularly.
Follow the steps from this link OR MSDN
Note: After Creating the Database Script. You can read the file from Seed Function and Execute the query from function itself. Or Manually you can go and execute when ever you need it.

Answer 3

I ended up using a CSV (comma delimited file) and storing it as a domain resource. Then reading the CSV file and adding database records:
I am able to Seed the database using EF Migration Seed method and a CSV file as defined as follows in the Migration.cs file. Note: the CSV file in the project in Visual Studio are set to the Build Action to Embedded Resource.
public void SeedData(WebApp.Models.ApplicationDbContext Context)
{
Assembly assembly = Assembly.GetExecutingAssembly();
string resourceName = "WebApp.SeedData.Name.csv";
using (Stream stream = assembly.GetManifestResourceStream(resourceName))
{
using (StreamReader reader = new StreamReader(stream, Encoding.UTF8))
{
CsvReader csvReader = new CsvReader(reader);
csvReader.Configuration.WillThrowOnMissingField = false;
var records = csvReader.GetRecords<Products>().ToArray();
foreach (Product record in records)
{
Context.Products.AddOrUpdate(record);
}
}
}
Context.SaveChanges();
}