Bulk Inserting Excel Data to Database Using EF Code First - asp.net-mvc

I use asp.net mvc ef code first. I upload the file to the server all i need is inserting that excel data to code first database. What is the best way to bulk insert excel data to database? Would appreciate your consultation.
Thanks in advance.

You can user LinqToExcel to get data from the Excel file and map it in your entity class.
If you are looking for alternative methods, these are some:
OLEDB, see an example
OPEN XML 2.0
Some third party DLL like ExcelDataReader or EPPlus

Using an ORM like Entity Framework is not efficient to perform bulk operations. To bulk insert efficiently, the SqlBulkCopy class must be used.
To insert a generic list it must be converted to a DataTable:
To insert a generic list it must be converted to a DataTable:
public static DataTable ConvertToDataTable<T>(IList<T> list)
{
PropertyDescriptorCollection propertyDescriptorCollection = TypeDescriptor.GetProperties(typeof(T));
DataTable table = new DataTable();
for (int i = 0; i < propertyDescriptorCollection.Count; i++)
{
PropertyDescriptor propertyDescriptor = propertyDescriptorCollection[i];
Type propType = propertyDescriptor.PropertyType;
if (propType.IsGenericType && propType.GetGenericTypeDefinition() == typeof(Nullable<>))
{
table.Columns.Add(propertyDescriptor.Name, Nullable.GetUnderlyingType(propType));
}
else
{
table.Columns.Add(propertyDescriptor.Name, propType);
}
}
object[] values = new object[propertyDescriptorCollection.Count];
foreach (T listItem in list)
{
for (int i = 0; i < values.Length; i++)
{
values[i] = propertyDescriptorCollection[i].GetValue(listItem);
}
table.Rows.Add(values);
}
return table;
}
Then the SqlBulkCopy can be used. In the example the user table is bulk inserted:
DataTable dt = new DataTable();
using (SqlConnection connection = new SqlConnection(connectionString))
{
connection.Open();
using (SqlBulkCopy sqlBulkCopy = new SqlBulkCopy(connection))
{
sqlBulkCopy.ColumnMappings.Add("UserID", "UserID");
sqlBulkCopy.ColumnMappings.Add("UserName", "UserName");
sqlBulkCopy.ColumnMappings.Add("Password", "Password");
sqlBulkCopy.DestinationTableName = "User";
sqlBulkCopy.WriteToServer(dt);
}
}

Related

MVC ADO without using EF

i wonder how to implement M-V-C ADO without using EF.
just like a pure ADO implementation. any suggestions and sample are appreciated Thanks guys.
Basic ADO.NET connections haven't really changed at all with MVC coming around. They still rely on things like SqlConnection objects and their associated commands.
If you wanted to build a simply query, it might look like the following :
// Build your connection
using(var connection = new SqlConnection("{your-connection-string-here"}))
{
// Build your query
var query = "SELECT * FROM YourTable WHERE foo = #bar";
// Create a command to execute your query
using(var command = new SqlCommand(query,connection))
{
// Open the connection
connection.Open();
// Add any parameters if necessary
command.Parameters.AddWithValue("#bar", 42);
// Execute your query here (in this case using a data reader)
using(var reader = command.ExecuteReader())
{
// Iterate through your results
while(reader.Read())
{
// The current reader object will contain each row here, so you
// can access the values as expected
}
}
}
}
You can use the type of ADO commands and paramaterized SQL seen here to retrieve data:
conn.Open();
cmd.CommandText = "SELECT id, desc FROM mytable WHERE id = #id";
cmd.Parameters.AddWithValue("#id", myid);
using (var reader = cmd.ExecuteReader())
{
if (!reader.Read())
{
return null;
}
return new myItem
{
Id = reader.GetInt32(reader.GetOrdinal("id")),
Desc = reader.GetString(reader.GetOrdinal("desc")),
}
}
There are lot of examples on MSDN for CRUD.

Linq query returns sql and not data from database

I'm not sure what the problem is but I cant seem to get my program to display the correct information. The requirements for my application are to take a pdf and populate the form fields with information from a database. The problem is that it does not return the information from the database and instead returns the sql generated from entity framework.
(text) SELECT
[Extent1].[Applicant_ID] AS [Applicant_ID]
FROM [dbo].[W4] AS [Extent1]
This is what is displayed in the pdf textbox.
This is my query
public class Query
{
ApplicatoinContext context = new ApplicatoinContext();
public List<W4> GetId()
{
return (from p in context.w4
select new W4 { Applicant_ID = p.Applicant_ID }).ToList();
}
}
My controller
public class ApplicationController : Controller
{
// GET: Application
public ActionResult Index()
{
string template = #"c:\users\carisch\documents\visual studio 2013\Projects\Idea\Idea\fw4.pdf";
string newFile = #"c:\users\carisch\documents\visual studio 2013\Projects\Idea\Idea\Newfw4.pdf";
PdfReader reader = new PdfReader(template);
PdfStamper stamper = new PdfStamper(reader, new FileStream(newFile, FileMode.Create));
AcroFields fields = stamper.AcroFields;
Query num = new Query();
var query = num.GetId();
fields.SetField("f1_15_0_", query.ToString());
stamper.FormFlattening = false;
stamper.Close();
return File(#"c:\users\carisch\documents\visual studio 2013\Projects\Idea\Idea\Newfw4.pdf", "application/pdf");
}
}
I'm pretty new to programming so any help would be greatly appreciated.

LINQ to Entities StringConvert(double)' cannot be translated to convert int to string

Problem
Need to convert int to string using EF4 + SQL CE4. The recommended option of using SqlFunctions.StringConvert(double) still gives me errors.
Option 1 (original). This gives me error:
public static IEnumerable<SelectListItem> xxGetCustomerList()
{
using (DatabaseEntities db = new DatabaseEntities())
{
var list = from l in db.Customers
orderby l.CompanyName
select new SelectListItem { Value = l.CustomerID.ToString(), Text = l.CompanyName };
return list.ToList();
}
}
Option 2 (most suggested). Then as many posts suggests, I used the SqlFunctions.StringConvert() function from the library System.Data.Objects.SqlClient:
public static IEnumerable<SelectListItem> GetCustomerList()
{
using (DatabaseEntities db = new DatabaseEntities())
{
var list = from l in db.Customers
orderby l.CompanyName
select new SelectListItem { Value = SqlFunctions.StringConvert((double)l.CustomerID), Text = l.CompanyName };
return list.ToList();
}
}
Which now shows below error:
The specified method 'System.String StringConvert(System.Nullable`1[System.Double])' on the type 'System.Data.Objects.SqlClient.SqlFunctions' cannot be translated into a LINQ to Entities store expression.
Option 3 (for very specific case). Then anoter post shows a smart solution using Dictionary, which finally works:
public static IEnumerable<SelectListItem> xGetCustomerList()
{
using (DatabaseEntities db = new DatabaseEntities())
{
var customers = db.Customers.ToDictionary(k => k.CustomerID, k => k.CompanyName);
var list = from l in customers
orderby l.Value
select new SelectListItem { Value = l.Key.ToString(), Text = l.Value };
return list.ToList();
}
}
But only work for simple pair values (key, value). Can someone help me with another solution or what I'm doing wrong with option 2?
And I hope Microsoft will soon make EF right before pushing us to move from L2S which is already stable and much more mature. I actually using EF4 just because want to use SQL CE, otherwise I stay with L2S.
EF is database independent at upper layers but the part dealing with conversion of linq query to SQL is always database dependent and SqlFunctions are dependent on SQL Server Provider but you are using SQL Server CE provider which is not able to translate functions from SqlFunctions class.
Btw. third option is also not a solution because it will select whole customer table to memory and use linq-to-objects after that. You should use this:
public static IEnumerable<SelectListItem> xxGetCustomerList()
{
using (DatabaseEntities db = new DatabaseEntities())
{
// Linq to entities query
var query = from l in db.Customers
orderby l.CompanyName
select new { l.CustomerID, l.CompanyName };
// Result of linq to entities transformed by linq to objects
return query.AsEnumerable()
.Select(x => new SelectListItem
{
Value = x.CustomerID.ToString(),
Test = x.CompanyName
}).ToList();
}
}
Here is a simplified version I'm using now (specific for SQL CE):
public static IEnumerable<SelectListItem> GetBlogCategoryList()
{
using (SiteDataContext db = new SiteDataContext())
{
var list = from l in db.BlogCategories.AsEnumerable()
orderby l.CategoryName
select new SelectListItem { Value = l.CategoryID.ToString(), Text = l.CategoryName };
return list.ToList();
}
}
Note the db.BlogCategories.AsEnumerable() part

EF4 Strip table prefixes on import

I am attempting to have table names automatically renamed to strip off a leading prefix in EF4. I know it can be done in the GUI, however, my company created the DB schema in Visio and use that to create the DB creation script in SQL. We do this often, and sometimes have a lot of tables, so using the GUI is not an ideal solution.
Is there a way to modify the properties on the .edmx file to strip off a defined prefix from the DB table so the Entity class is how we desire it?
The simplest solution we came up with was to make a new console application that did the low level work of renaming everything in the edmx file; the application can be added to the tools menu of Visual Studio (add external tool) with 'arguments' = $(ItemPath), 'initial directory' = $(ItemDir), 'prompt for arguments' = true and 'use output window' = true, then can be executed with the EDMX file selected. In our case, we needed to strip underscores and transform the names to camel case (interpreting the underscore as a word separator), in addition to stripping the prefix.
I'll omit the rest of the code (like the Transform.Standard method, error handling and the stripping of additional parameters) because it's too badly written to share and it's too late at night for refactoring :P; but it's easy enough to interpret the rest of the arguments after the first as strings to be stripped from names and so on - this main code is only about the modifications needed on the EDMX file. In case you use this as an External Tool from VS, you can specify the rest of the strings after $(ItemPath) in 'arguments'.
Please note this wasn't tested extensively, and there may be additional information in other EDMX files that we failed to account for (but I doubt it); also, I got part of the code from somewhere else on the net but failed to write down where exactly, so sorry! Credit should have been given accordingly. Also, naturally this would have been much better as an extension for VS2010, but I simply did not have the time to do it - if you do link it somewhere and I'll use that instead ;)
if (_args.Count < 1) return;
string file = _args.First();
if (!File.Exists(file))
{
wait("Could not find specified file.");
return;
}
if (Path.GetExtension(file) != ".edmx")
{
wait("This works only on EDMX files.");
return;
}
//processing:
Console.WriteLine("Creating backup: " + Path.ChangeExtension(file, ".bak"));
File.Copy(file, Path.ChangeExtension(file, ".bak"), true);
Console.WriteLine("Reading target document...");
XDocument xdoc = XDocument.Load(file);
const string CSDLNamespace = "http://schemas.microsoft.com/ado/2008/09/edm";
const string MSLNamespace = "http://schemas.microsoft.com/ado/2008/09/mapping/cs";
const string DiagramNamespace = "http://schemas.microsoft.com/ado/2008/10/edmx";
XElement csdl = xdoc.Descendants(XName.Get("Schema", CSDLNamespace)).First();
XElement msl = xdoc.Descendants(XName.Get("Mapping", MSLNamespace)).First();
XElement designerDiagram = xdoc.Descendants(XName.Get("Diagram", DiagramNamespace)).First();
//modifications for renaming everything, not just table names:
string[] CSDLpaths = new string[]
{
"EntityContainer/EntitySet.Name",
"EntityContainer/EntitySet.EntityType",
"EntityContainer/AssociationSet/End.EntitySet",
"EntityType.Name",
"EntityType/Key/PropertyRef/Name",
"EntityType/Property.Name",
"EntityType/NavigationProperty.Name",
"Association/End.Type",
"Association//PropertyRef.Name",
};
#region CSDL2
Console.WriteLine("Modifying CSDL...");
Console.WriteLine(" - modifying entity sets...");
foreach (var entitySet in csdl.Element(XName.Get("EntityContainer", CSDLNamespace)).Elements(XName.Get("EntitySet", CSDLNamespace)))
{
entitySet.Attribute("Name").Value = Transform.Standard(entitySet.Attribute("Name").Value);
entitySet.Attribute("EntityType").Value = Transform.Standard(entitySet.Attribute("EntityType").Value);
}
Console.WriteLine(" - modifying association sets...");
foreach (var associationSet in csdl.Element(XName.Get("EntityContainer", CSDLNamespace)).Elements(XName.Get("AssociationSet", CSDLNamespace)))
{
foreach (var end in associationSet.Elements(XName.Get("End", CSDLNamespace)))
{
end.Attribute("EntitySet").Value = Transform.Standard(end.Attribute("EntitySet").Value);
}
}
Console.WriteLine(" - modifying entity types...");
foreach (var entityType in csdl.Elements(XName.Get("EntityType", CSDLNamespace)))
{
entityType.Attribute("Name").Value = Transform.Standard(entityType.Attribute("Name").Value);
foreach (var key in entityType.Elements(XName.Get("Key", CSDLNamespace)))
{
foreach (var propertyRef in key.Elements(XName.Get("PropertyRef", CSDLNamespace)))
{
propertyRef.Attribute("Name").Value = Transform.Standard(propertyRef.Attribute("Name").Value);
}
}
foreach (var property in entityType.Elements(XName.Get("Property", CSDLNamespace)))
{
property.Attribute("Name").Value = Transform.Standard(property.Attribute("Name").Value);
}
foreach (var navigationProperty in entityType.Elements(XName.Get("NavigationProperty", CSDLNamespace)))
{
navigationProperty.Attribute("Name").Value = Transform.Standard(navigationProperty.Attribute("Name").Value);
}
}
Console.WriteLine(" - modifying associations...");
foreach (var association in csdl.Elements(XName.Get("Association", CSDLNamespace)))
{
foreach (var end in association.Elements(XName.Get("End", CSDLNamespace)))
{
end.Attribute("Type").Value = Transform.Standard(end.Attribute("Type").Value);
}
foreach (var propref in association.Descendants(XName.Get("PropertyRef", CSDLNamespace)))
{
//propertyrefs are contained in constraints
propref.Attribute("Name").Value = Transform.Standard(propref.Attribute("Name").Value);
}
}
#endregion
#region MSL2
Console.WriteLine("Modifying MSL...");
Console.WriteLine(" - modifying entity set mappings...");
foreach (var entitySetMapping in msl.Element(XName.Get("EntityContainerMapping", MSLNamespace)).Elements(XName.Get("EntitySetMapping", MSLNamespace)))
{
entitySetMapping.Attribute("Name").Value = Transform.Standard(entitySetMapping.Attribute("Name").Value);
foreach (var entityTypeMapping in entitySetMapping.Elements(XName.Get("EntityTypeMapping", MSLNamespace)))
{
entityTypeMapping.Attribute("TypeName").Value = Transform.Standard(entityTypeMapping.Attribute("TypeName").Value);
foreach
(var scalarProperty in
(entityTypeMapping.Element(XName.Get("MappingFragment", MSLNamespace))).Elements(XName.Get("ScalarProperty", MSLNamespace))
)
{
scalarProperty.Attribute("Name").Value = Transform.Standard(scalarProperty.Attribute("Name").Value);
}
}
}
Console.WriteLine(" - modifying association set mappings...");
foreach (var associationSetMapping in msl.Element(XName.Get("EntityContainerMapping", MSLNamespace)).Elements(XName.Get("AssociationSetMapping", MSLNamespace)))
{
foreach (var endProperty in associationSetMapping.Elements(XName.Get("EndProperty", MSLNamespace)))
{
foreach (var scalarProperty in endProperty.Elements(XName.Get("ScalarProperty", MSLNamespace)))
{
scalarProperty.Attribute("Name").Value = Transform.Standard(scalarProperty.Attribute("Name").Value);
}
}
}
#endregion
#region Designer
Console.WriteLine("Modifying designer content...");
foreach (var item in designerDiagram.Elements(XName.Get("EntityTypeShape", DiagramNamespace)))
{
item.Attribute("EntityType").Value = Transform.Standard(item.Attribute("EntityType").Value);
}
#endregion
Console.WriteLine("Writing result...");
using (XmlTextWriter writer = new XmlTextWriter(args[0], Encoding.Default))
{
writer.Formatting = Formatting.Indented;
xdoc.WriteTo(writer);
}
Edit: adding the Transform class used by the code above. Also, please note that this works for Entity Framework 4.0 - later versions might have slightly different EDMX structure (I'm not sure) so the code might need to be modified to account for that.
public class Transform
{
public static string Standard(string initial, IEnumerable<string> eliminations = null)
{
Regex re = new Regex(#"(\w+)(\W*?)$", RegexOptions.Compiled);
Regex camelSplit = new Regex(#"(?<!^)(?=[A-Z])", RegexOptions.Compiled);
return re.Replace(initial, new MatchEvaluator((Match m) =>
{
string name = m.Groups[1].Value;
var parts = name.Split('_').AsEnumerable();
if (parts.Count() == 1 && IsMixedCase(name))
{
string result = string.Concat(camelSplit.Split(name).Except(eliminations, StringComparer.CurrentCultureIgnoreCase));
return result + m.Groups[2];
}
else
{
parts = parts.Select(s => CultureInfo.CurrentCulture.TextInfo.ToTitleCase(s.ToLower()));
parts = parts.Except(eliminations, StringComparer.CurrentCultureIgnoreCase);
return string.Concat(parts) + m.Groups[2];
}
}));
}
public static bool IsMixedCase(string name)
{
int lower = 0, total = 0;
for (int i = 0; i < name.Length; i++)
{
if (char.IsLower(name, i)) lower++;
if (char.IsLetter(name, i)) total++;
}
return lower != 0 && lower != total;
}
}
Now you could write a little bit more code in the Main method that could take the arguments (except the first one which will be the name of the file) and pass them onward to the eliminations parameter; these could be strings that need to be erased from the names, in my case the prefixes. You can then specify these strings from the visual studio tools interface when invoking the tool. Or I suppose you can just hardcode them if you don't care that much about reusing the tool :)
I do not know about any build in feature for stripping table prefixes but you can open .edmx file as XML (use Open With in VS) and use simple replacing to replace prefix with nothing.

asp mvc plain sql possible?

Is there a way to use simple sql queries on ASP MVC without using LINQ thing?
any link is welcome :)
Of course, you can always drop down to use regular ol' ADO.NET :-)
The Data Access Application Block is also commonly used to simplify the execution of raw sql and stored procedures.
Sure, you can embed plain ADO.NET objects within your controller's action methods or in a custom business logic library. A bit of an example. WARNING: DEMONSTRATION CODE ONLY. DO NOT ATTEMPT TO USE IN PRODUCTION SCENARIO.
public ActionResult Index()
{
using(SqlConnection conn = new SqlConnection(ConfigurationManager.ConnectionStrings["CNTax"].ConnectionString))
{
using(SqlCommand command = conn.CreateCommand())
{
command.CommandText = "select * from Names";
command.CommandType = CommandType.Text;
conn.Open();
var table = new DataTable();
table.load(command.ExecuteReader());
return View("Index", table);
}
}
}
A simple code snippet to select all names from the database and return the index view with the model set as a DataTable.
You sure can. And it is done the same way as you normally would in an ASP.NET application just like the other answers here have indicated.
....HOWEVER, I prefer to use a tool to generate my data access layer. Some of the top choices right now are nHibernate, and LLBLGen Pro, or even Microsoft's Entity Framework
I personally would go with nHibernate or LLBLGen Pro, depending on if you want to have your data access layer driven by "domain driven design" (nHibernate) or "data driven design" (LLBLGen Pro)
Building off of #user102220's answer:
Set up a data access layer (just a separate class or series of classes), then call these from your controller. Apply ADO.NET as necessary.
If you are using Entity Framework you can use SqlQuery like this:
using(var db = new DbContext())
{
var result = db.Database.SqlQuery<your object>("Your Query",params));
}
Your object : your expected result Type (object or class Type) and
Your query and params : sql command with params you should pass to query or not
try this solution:
var results = DynamicCall.DynamicListFromSql(_entities, "select * from users", null).ToList();
public static IEnumerable<dynamic> DynamicListFromSql(this DbContext db, string Sql, Dictionary<string, object> Params)
{
using (var cmd = db.Database.Connection.CreateCommand())
{
cmd.CommandText = Sql;
if (cmd.Connection.State != ConnectionState.Open) { cmd.Connection.Open(); }
using (var dataReader = cmd.ExecuteReader())
{
while (dataReader.Read())
{
var row = new ExpandoObject() as IDictionary<string, object>;
for (var fieldCount = 0; fieldCount < dataReader.FieldCount; fieldCount++)
{
row.Add(dataReader.GetName(fieldCount), dataReader[fieldCount]);
}
yield return row;
}
}
}
}

Resources