How can this function be modified.
I want to use it to fill in the dataset from sqllite.
error
public void fillDATASET( DataSet ds, string tablename, string query)
{
string dbPath = Path.Combine(
System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal),
"Department.db3");
var conn = new SQLite.SQLiteConnection(dbPath);
using (Mono.Data.Sqlite.SqliteCommand cmd = new SqliteCommand(query, conn))// error conn
{
using (var DataAdapterd = new SqliteDataAdapter(cmd))
{
ds.Clear();
DataAdapterd.Fill(ds, tablename);
}
}
}
This is because you use two different libraries.
var conn = new SQLite.SQLiteConnection(dbPath);
here you used the method in sqlite-net-pcl nuget,
Mono.Data.Sqlite.SqliteCommand cmd = new SqliteCommand(query, conn)
here you want use the method in System.Data.SQLite.Core nuget.
So you need to use a unified.
For example(use System.Data.SQLite.Core nuget):
using System.Data;
using System.Data.SQLite;
public void fillDATASET(DataSet ds, string tablename, string query)
{
string dbPath = Path.Combine(System.Environment.GetFolderPath(System.Environment.SpecialFolder.Personal),
"Department.db3");
var conn = new SQLiteConnection(dbPath);
using (SQLiteCommand cmd = new SQLiteCommand(query, conn))// error conn
{
using (var DataAdapterd = new SQLiteDataAdapter(cmd))
{
ds.Clear();
DataAdapterd.Fill(ds, tablename);
}
}
}
I am following the blog http://www.venkatbaggu.com/signalr-database-update-notifications-asp-net-mvc-usiing-sql-dependency/ to get a signalR push message out to connected clients.
My debugger never hits the onchange event.
my Global.asax.cs:
string connString = ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
protected void Application_Start()
{
// basic stuff
SqlDependency.Start(connString);
var repo = new Repositories.MarkerRepository();
repo.GetAllMarkers(); // to register the dependency
}
My MarkerRepository.cs:
readonly string _connString = ConfigurationManager.ConnectionStrings["DefaultConnection"].ConnectionString;
private MarkersHub _mh = new MarkersHub(); // my signalr hub class
public IEnumerable<House> GetAllMarkers()
{
var markers = new List<House>();
using (var connection = new SqlConnection(_connString))
{
connection.Open();
using (var command = new SqlCommand(#"SELECT * FROM [dbo].Houses", connection))
{
command.Notification = null;
var dependency = new SqlDependency(command);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
if (connection.State == ConnectionState.Closed)
connection.Open();
var reader = command.ExecuteReader();
while (reader.Read())
{
markers.Add(item: new House {
ID = (int)reader["ID"],
Name = (string)reader["Name"],
Code = reader["Code"] != DBNull.Value ? (string)reader["Code"] : "",
Latitude = Convert.ToDouble(reader["Latitude"]),
Longitude = Convert.ToDouble(reader["Longitude"])
});
}
}
}
return markers;
}
private void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
if (e.Type == SqlNotificationType.Change)
{
_mh.SendMarkers();
}
}
I have had a hit once but it was no change, only a notification for subscribe. I have read a lot about resubscribe, but when it hit this event the sql:
select * from sys.dm_qn_subscriptions
still returns no rows. Not on my db or master. So I think that there is an error in the blog post with the re-subscribe to the on change event? This sample https://msdn.microsoft.com/en-us/library/a52dhwx7(VS.80).aspx does unregister in the onchange event and calls the method which registers a new event. Can someone verify my assumption?
These were the values for the SqlNotificationEventArgs e in my event and told me that my query to depend on, was invalid.
SqlNotificationEventArgs.Type --> SqlNotificationType.Subscribe
SqlNotificationEventArgs.Info --> SqlNotificationInfo.Invalid
SqlNotificationEventArgs.Source --> SqlNotificationSource.Statement
The statement may not use the asterisk () or table_name. syntax to specify columns.
source https://msdn.microsoft.com/en-us/library/ms181122.aspx
I have an Asp.Net Mvc application. In this application i have a functionality to download data from Database to Excel file (with OpenXml Sdk). It works now. But when data is large, time from user request to response with download window becomes 10+ minutes. This is because of two long process:
Taking data from MSSQL server.
Generating Excel document in memory on server. (Downloading begins only when Excel document completed)
First problem was solved through using of DataReader. Now generating of excel file begins just after user request becomes to webserver.
For solving second problem we need to generate Excel document on HttpResponse.OutputStream, but this stream is not Seekable and generation fails before begining.
Does anyone knows any workaround that can help to work with this problem?
Sample of my generating function:
public void GenerateSpreadSheetToStream(IDataReader dataReader, Stream outputStream)
{
var columnCaptions = FillColumnCaptionsFromDataReader(dataReader.GetSchemaTable());
//fails on next line with exception "Cannot open package because FileMode or FileAccess value is not valid for the stream."
using (var spreadsheetDocument = SpreadsheetDocument.Create(outputStream, SpreadsheetDocumentType.Workbook))
{
spreadsheetDocument.AddWorkbookPart();
var workSheetPart = spreadsheetDocument.WorkbookPart.AddNewPart<WorksheetPart>();
OpenXmlWriter writer;
using (writer = OpenXmlWriter.Create(workSheetPart))
{
using (writer.Write(new Worksheet()))
{
using (writer.Write(new SheetData()))
{
using (writer.Write(w =>
w.WriteStartElement(new Row(), new[] {new OpenXmlAttribute("r", null, 1.ToString(CultureInfo.InvariantCulture))})))
{
var cells =
columnCaptions.Select(caption => new Cell()
{
CellValue = new CellValue(caption.Item2),
DataType = CellValues.String
});
foreach (var cell in cells)
{
writer.WriteElement(cell);
}
}
var i = 2;
while (dataReader.Read())
{
var oxa = new[] { new OpenXmlAttribute("r", null, i.ToString(CultureInfo.InvariantCulture)) };
using (writer.Write(w => w.WriteStartElement(new Row(), oxa)))
{
var cells =
columnCaptions.Select(
(c, j) =>
new Cell
{
CellValue = new CellValue(dataReader[c.Item1].ToString()),
DataType = CellValues.String,
CellReference = new StringValue(GetSymbolByCellNumber(j))
});
foreach (var cell in cells)
{
writer.WriteElement(cell);
}
}
i++;
}
}
}
}
using (writer = OpenXmlWriter.Create(spreadsheetDocument.WorkbookPart))
{
using (writer.Write(new Workbook()))
{
using (writer.Write(new Sheets()))
{
var sheet = new Sheet
{
Id = spreadsheetDocument.WorkbookPart.GetIdOfPart(workSheetPart),
SheetId = 1,
Name = SheetName
};
writer.WriteElement(sheet);
}
}
}
}
}
private static string GetSymbolByCellNumber(int number)
{
var r = number/26;
var s = (char) ((number%26) + 65);
return new string(s, r);
}
My FileStreamResultWithTransformation (for working with HttpResponse.OutputStream):
public class FileStreamResultWithTransformation : FileResult
{
private readonly Action<Stream> _action;
public FileStreamResultWithTransformation(Action<Stream> action, string contentType, string fileName) : base(contentType)
{
_action = action;
FileDownloadName = fileName;
}
protected override void WriteFile(HttpResponseBase response)
{
response.BufferOutput = false;
_action(response.OutputStream); ->> it fails there
}
}
StackTrace:
[IOException: Cannot open package because FileMode or FileAccess value
is not valid for the stream.]
System.IO.Packaging.Package.ValidateModeAndAccess(Stream s, FileMode
mode, FileAccess access) +784533
System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode,
FileAccess packageAccess, Boolean streaming) +89
System.IO.Packaging.Package.Open(Stream stream, FileMode packageMode,
FileAccess packageAccess) +10
DocumentFormat.OpenXml.Packaging.OpenXmlPackage.CreateCore(Stream
stream) +192
DocumentFormat.OpenXml.Packaging.SpreadsheetDocument.Create(Stream
stream, SpreadsheetDocumentType type, Boolean autoSave) +215
DocumentFormat.OpenXml.Packaging.SpreadsheetDocument.Create(Stream
stream, SpreadsheetDocumentType type) +44
-------.GenerateSpreadSheetToStream(IDataReader dataReader, Stream outputStream) in
d:\Work\Epsilon\development\Web\trunk\Sources\Epsilon.DocumentGenerator\XlsXGenerator.cs:119
It seems to me, that this problem cannot be solved. On finalization of writing, OpenXmlWriter seeks, read and write in different positions of stream and without this actions xlsx file is broken.
I think, there is something wrong in design of OpenXml library.
The problem is a little deeper. Xlsx file is zip archive and OpenXml internally uses the ZipArchive class. Each file in archive has a header, it is placed before the data. ZipArchive writes data to the stream, then returns to the beginning of the file and writes file's header. It uses the Stream.Seek method and HttpResponse.OutputStream cannot work this way.
I am using jqgrid (standard) with EF 4 + MVC3. I'd like to implement excel export. Which method you would suggest me?
To generate excel, I'd like to use this library by Dr Stephen Walther, which has three types of output and allows to define headers too. Please tell me if you find it valid for my purpose.
I ask this question because I am still approaching to implement excel export and I found several techniques. Some suggest making a csv export, others indicate that it should return a JSON output and it is not clear to me whether this capability is present in the free version of jqgrid. In any case, I would like to pass the data to Walther's object.
About the jqgrid code, I found this interesting answer by Oleg, but I do not understand if could be applied to my needs.
Unfortunately, by now I only found parts of solutions for excel export with EF MVC, but no solution or complete examples...
About the MVC logic, I am going to implement and develop this code as kindly suggested by #Tommy.
Please sorry if the question could be silly, I am just a (enthusiast) beginner.
Thanks for your precious help!
Best Regards
As I wrote before (see here and here for example) the best way to export grid data to XML is the usage of Open XML SDK 2.0.
The post of Dr Stephen Walther shows how to create HTML file which can be read by Excel. It's not Excel file and have to be still converted to Excel format. The usage of CSV has even more problems. Depend on the content in the source table the automatic conversion to Excel data types can be absolutely wrong. In one project which I developed for a customer the grid contained information about software products: product name, version, and so on. The software version looks sometime as the date (1.3.1963 for example) and such cells will be wrong converted (in German one use '.' as the separator in the date). As the result one had really hard problems. The usage of CSV with texts having commas inside will be also frequently wrong imported. Even when one quotes the cells having commas (,) and escaped the texts having quotas the import still be wrong especially in the first column. I don't want to explain here the whole history of all attempts and errors, but after all I decide to give up with the usage of CSV and HTML and started to use Open XML SDK 2.0 which allows to create real Excel files with extension XLSX. The way seems me perfect because one don't need any Office
components installed on the server, no additional licenses.
The only restriction is that one should be able to use DocumentFormat.OpenXml.dll, so your server program should run on any Windows operation system. As it's well known, XLSX file is ZIP file which contains some XML files inside. If you still don't know that I recommend you to rename the XLSX file to ZIP file and extract it. The Open XML SDK 2.0 is the library which works with XLSX file like with XML files. So no additional Office components are required.
One can find a lot of information how to use Open XML SDK 2.0 (see here, here and here). Many helpful code examples one cam find directly on the MSDN (see here). Nevertheless the practical usage of Open XML SDK 2.0 is not so easy at least at the first time. So I created a demo from the parts of the code which I used myself.
You can download the demo project from here. The demo is an extension of the demos from the answer and this one.
To export data I use the DataForExcel helper class. It has constructor in the form
DataForExcel(string[] headers, DataType[] colunmTypes, List<string[]> data,
string sheetName)
or in a little simplified form
DataForExcel(string[] headers, List<string[]> data, string sheetName)
and the only public method
CreateXlsxAndFillData(Stream stream)
The usage of the class to create Excel file can be like the following
var excelData = new DataForExcel (
// column Header
new[]{"Col1", "Col2", "Col3"},
new[]{DataForExcel.DataType.String, DataForExcel.DataType.Integer,
DataForExcel.DataType.String},
new List<string[]> {
new[] {"a", "1", "c1"},
new[] {"a", "2", "c2"}
},
"Test Grid");
Stream stream = new FileStream ("Test.xlsx", FileMode.Create);
excelData.CreateXlsxAndFillData (stream);
stream.Close();
The usage in the demo from ASP.NET MVC is the following
static readonly string[] HeadersQuestions = {
"Id", "Votes", "Title"
};
static readonly DataForExcel.DataType[] ColunmTypesQuestions = {
DataForExcel.DataType.Integer,
DataForExcel.DataType.Integer,
DataForExcel.DataType.String
};
public ActionResult ExportAllQuestionsToExcel () {
var context = new HaackOverflowEntities ();
var questions = context.Questions;
questions.MergeOption = MergeOption.NoTracking; // we don't want to update the data
// to be able to use ToString() below which is NOT exist in the LINQ to Entity
// we should include in query only the properies which we will use below
var query = questions.ToList ();
if (query.Count == 0)
return new EmptyResult ();
var data = new List<string[]> (query.Count);
data.AddRange (query.Select (item => new[] {
item.Id.ToString(CultureInfo.InvariantCulture),
item.Votes.ToString(CultureInfo.InvariantCulture),
item.Title
}));
return new ExcelResult (HeadersQuestions, ColunmTypesQuestions, data,
"Questions.xlsx", "Questions");
}
where ExcelResult are defined as
public class ExcelResult : ActionResult {
private readonly DataForExcel _data;
private readonly string _fileName;
public ExcelResult (string[] headers, List<string[]> data, string fileName, string sheetName) {
_data = new DataForExcel (headers, data, sheetName);
_fileName = fileName;
}
public ExcelResult (string[] headers, DataForExcel.DataType[] colunmTypes, List<string[]> data, string fileName, string sheetName) {
_data = new DataForExcel (headers, colunmTypes, data, sheetName);
_fileName = fileName;
}
public override void ExecuteResult (ControllerContext context) {
var response = context.HttpContext.Response;
response.ClearContent();
response.ClearHeaders();
response.Cache.SetMaxAge (new TimeSpan (0));
using (var stream = new MemoryStream()) {
_data.CreateXlsxAndFillData (stream);
//Return it to the client - strFile has been updated, so return it.
response.AddHeader ("content-disposition", "attachment; filename=" + _fileName);
// see http://filext.com/faq/office_mime_types.php
response.ContentType = "application/vnd.openxmlformats-officedocument.spreadsheetml.sheet";
response.ContentEncoding = Encoding.UTF8;
stream.WriteTo (response.OutputStream);
}
response.Flush();
response.Close();
}
}
To make the code full I have to include the code of the class DataForExcel:
public class DataForExcel {
public enum DataType {
String,
Integer
}
private readonly string[] _headers;
private readonly DataType[] _colunmTypes;
private readonly List<string[]> _data;
private readonly string _sheetName = "Grid1";
private readonly SortedSet<string> _os = new SortedSet<string> ();
private string[] _sharedStrings;
private static string ConvertIntToColumnHeader(int index) {
var sb = new StringBuilder ();
while (index > 0) {
if (index <= 'Z' - 'A') // index=0 -> 'A', 25 -> 'Z'
break;
sb.Append (ConvertIntToColumnHeader (index / ('Z' - 'A' + 1) - 1));
index = index % ('Z' - 'A' + 1);
}
sb.Append ((char)('A' + index));
return sb.ToString ();
}
private static Row CreateRow(UInt32 index, IList<string> data) {
var r = new Row { RowIndex = index };
for (var i = 0; i < data.Count; i++)
r.Append (new OpenXmlElement[] { CreateTextCell (ConvertIntToColumnHeader (i), index, data[i]) });
return r;
}
private Row CreateRowWithSharedStrings(UInt32 index, IList<string> data) {
var r = new Row { RowIndex = index };
for (var i = 0; i < data.Count; i++)
r.Append (new OpenXmlElement[] { CreateSharedTextCell (ConvertIntToColumnHeader (i), index, data[i]) });
return r;
}
private Row CreateRowWithSharedStrings(UInt32 index, IList<string> data, IList<DataType> colunmTypes) {
var r = new Row { RowIndex = index };
for (var i = 0; i < data.Count; i++)
if (colunmTypes != null && i < colunmTypes.Count && colunmTypes[i] == DataType.Integer)
r.Append (new OpenXmlElement[] { CreateNumberCell (ConvertIntToColumnHeader (i), index, data[i]) });
else
r.Append (new OpenXmlElement[] { CreateSharedTextCell (ConvertIntToColumnHeader (i), index, data[i]) });
return r;
}
private static Cell CreateTextCell(string header, UInt32 index, string text) {
// create Cell with InlineString as a child, which has Text as a child
return new Cell (new InlineString (new Text { Text = text })) {
// Cell properties
DataType = CellValues.InlineString,
CellReference = header + index
};
}
private Cell CreateSharedTextCell(string header, UInt32 index, string text) {
for (var i=0; i<_sharedStrings.Length; i++) {
if (String.Compare (_sharedStrings[i], text, StringComparison.Ordinal) == 0) {
return new Cell (new CellValue { Text = i.ToString (CultureInfo.InvariantCulture) }) {
// Cell properties
DataType = CellValues.SharedString,
CellReference = header + index
};
}
}
// create Cell with InlineString as a child, which has Text as a child
throw new InstanceNotFoundException();
}
private static Cell CreateNumberCell(string header, UInt32 index, string numberAsString) {
// create Cell with CellValue as a child, which has Text as a child
return new Cell (new CellValue { Text = numberAsString }) {
// Cell properties
CellReference = header + index
};
}
private void FillSharedStringTable(IEnumerable<string> data) {
foreach (var item in data)
_os.Add (item);
}
private void FillSharedStringTable(IList<string> data, IList<DataType> colunmTypes) {
for (var i = 0; i < data.Count; i++)
if (colunmTypes == null || i >= colunmTypes.Count || colunmTypes[i] == DataType.String)
_os.Add (data[i]);
}
public DataForExcel(string[] headers, List<string[]> data, string sheetName) {
_headers = headers;
_data = data;
_sheetName = sheetName;
}
public DataForExcel(string[] headers, DataType[] colunmTypes, List<string[]> data, string sheetName) {
_headers = headers;
_colunmTypes = colunmTypes;
_data = data;
_sheetName = sheetName;
}
private void FillSpreadsheetDocument(SpreadsheetDocument spreadsheetDocument) {
// create and fill SheetData
var sheetData = new SheetData ();
// first row is the header
sheetData.AppendChild (CreateRow (1, _headers));
//const UInt32 iAutoFilter = 2;
// skip next row (number 2) for the AutoFilter
//var i = iAutoFilter + 1;
UInt32 i = 2;
// first of all collect all different strings in OrderedSet<string> _os
foreach (var dataRow in _data)
if (_colunmTypes != null)
FillSharedStringTable (dataRow, _colunmTypes);
else
FillSharedStringTable (dataRow);
_sharedStrings = _os.ToArray ();
foreach (var dataRow in _data)
sheetData.AppendChild (_colunmTypes != null
? CreateRowWithSharedStrings (i++, dataRow, _colunmTypes)
: CreateRowWithSharedStrings (i++, dataRow));
var sst = new SharedStringTable ();
foreach (var text in _os)
sst.AppendChild (new SharedStringItem (new Text (text)));
// add empty workbook and worksheet to the SpreadsheetDocument
var workbookPart = spreadsheetDocument.AddWorkbookPart ();
var worksheetPart = workbookPart.AddNewPart<WorksheetPart> ();
var shareStringPart = workbookPart.AddNewPart<SharedStringTablePart> ();
shareStringPart.SharedStringTable = sst;
shareStringPart.SharedStringTable.Save ();
// add sheet data to Worksheet
worksheetPart.Worksheet = new Worksheet (sheetData);
worksheetPart.Worksheet.Save ();
// fill workbook with the Worksheet
spreadsheetDocument.WorkbookPart.Workbook = new Workbook (
new FileVersion { ApplicationName = "Microsoft Office Excel" },
new Sheets (
new Sheet {
Name = _sheetName,
SheetId = (UInt32Value)1U,
// generate the id for sheet
Id = workbookPart.GetIdOfPart (worksheetPart)
}
)
);
spreadsheetDocument.WorkbookPart.Workbook.Save ();
spreadsheetDocument.Close ();
}
public void CreateXlsxAndFillData(Stream stream) {
// Create workbook document
using (var spreadsheetDocument = SpreadsheetDocument.Create (stream, SpreadsheetDocumentType.Workbook)) {
FillSpreadsheetDocument (spreadsheetDocument);
}
}
}
The above code create new XLSX file directly. You can extend the code to support more data types as String and Integer which I used in the code.
In more professional version of your application you can create some XLSX templates for exporting different tables. In the code you can place the data in the cells instead, so modify the spreadsheet instead of creating. In the way you can create perfect formatted XLSX files. The examples from the MSDN (see here) will help you to implement the way when it will be required.
UPDATED: The answer contains updated code which allows generate Excel documented with more cell formatting.
I looked at Stephen's post and it's old as hell, which btw doesn't make it wrong.
If you don't need custom formatting, headers and styles, then I think use CSV as it's very simple.
More importantly, don't think that excel export from MVC site that internally uses EF for data access is harder than, say, Ruby on Rails site that uses ActiveRecord. For me it's independent concerns, export shouldn't new anything about underlying technologies (at least not directly), just the structure of your data, that's all.
Search for codeplex libraries that allows to do Excel reading/writing and export, there are plenty of them these days, many really good solutions that's regularly maintained and tested by thousand of developers all over the globe. If I were you I won't use Stephen solution because it looks like he occasionally typed it in a notepad and then pasted to the post - no unit tests, no extensibility points + it's in VB so it even harder to understand, but may be that's just me.
Hope this help and good luck
I have console application in which I am doing sqldependency. My problem is when I set commandType as Text, it is working fine. But if I use commandType as StoredProcedure, onchange method is calling infinitely.
Please see the code below:
static DataSet myDataSet;
static SqlConnection connection;
static SqlCommand command;
static void Main(string[] args)
{
// Remove any existing dependency connection, then create a new one.
string connstr = "Data Source=XYZ;Initial Catalog=Dev;Integrated Security=True";
string ssql = #"[dbo].[SchedulerPendingControlRequestIDFetch]";
CanRequestNotifications();
SqlDependency.Stop(connstr);
SqlDependency.Start(connstr);
if (connection == null)
connection = new SqlConnection(connstr);
if (command == null)
command = new SqlCommand(ssql, connection);
command.CommandType = CommandType.StoredProcedure;
if (myDataSet == null)
myDataSet = new DataSet();
GetAdvtData();
System.Console.ReadKey();
connection.Close();
}
private static bool CanRequestNotifications()
{
SqlClientPermission permission =
new SqlClientPermission(
PermissionState.Unrestricted);
try
{
permission.Demand();
return true;
}
catch (System.Exception)
{
return false;
}
}
private static void GetAdvtData()
{
myDataSet.Clear();
// Ensure the command object does not have a notification object.
command.Notification = null;
// Create and bind the SqlDependency object to the command object.
SqlDependency dependency = new SqlDependency(command,null,100);
dependency.OnChange += new OnChangeEventHandler(dependency_OnChange);
using (SqlDataAdapter adapter = new SqlDataAdapter(command))
{
adapter.Fill(myDataSet, "ControlRequest");
}
}
private static void dependency_OnChange(object sender, SqlNotificationEventArgs e)
{
SqlDependency dependency =
(SqlDependency)sender;
dependency.OnChange -= dependency_OnChange;
Console.WriteLine(e.Info.ToString() + e.Source.ToString());
GetAdvtData();
}
My stored Procedure is:
IF OBJECT_ID('SchedulerSirasColcoDetailFetch') IS NOT NULL
DROP PROCEDURE SchedulerSirasColcoDetailFetch
Go
PRINT 'Creating stored procedure SchedulerSirasColcoDetailFetch'
Go
CREATE PROCEDURE [dbo].[SchedulerSirasColcoDetailFetch]
AS
BEGIN
SELECT Colco_Code AS 'CountryCode',Connection_String AS 'Url',Resend_Interval AS 'ResendInterval',
Default_Encoding AS 'Encoding' FROM dbo.SirasColcoDetail
END
If I copy the select statement inside stored procedure as my command text and set the commandType as Text, everything is working fine.
could you please let me know what the issue is????
Thanks a lot in advance.
Mahesh
You're supposed to check the values of the SqlNotificationEventArgs argument. Only if Type is Change and Source is Data where you notified for a data change.
You'll discover that you're not notified for data changes, but for incorrect settings or incorrect query. Your query and connection settings must comply with the requirements specified in Creating a Query for Notifications.