I need to read a variety of data file types, such as xlsx, csv, txt, and mdb, and I want to use an OleDB connection so that the process of reading the files is the same, just with a different connection string. However, OleDB is ignoring the delimiter in connection strings such as the following and only reads comma-delimited.
Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties='Text;HDR=Yes;Delimited(\t)';
Provider=Microsoft.Jet.OLEDB.4.0;Data Source={0};Extended Properties='Text;HDR=Yes;FMT=TabDelimited';
I would prefer to have the OleDB engine do the work rather than parse the tab-delimited files myself.
There are several StackOverflow questions concerning this, and the solution is usually to create an .ini file in the same directory, but sometimes my users do not have write access to the folder. Seeing as all of the StackOverflow questions similar to mine are at least a couple years old, does anybody have any updated information on this issue?
This is how I've used | delimiter to read |-delimited .csv or .txt files using OleDB, however, I was using ACE engine and constructing connection string from C#:
connectionString = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + Path.GetDirectoryName(catalogFile) + ";Extended Properties='text;HDR=YES;FMT=Delimited(" + (char)124 + ")'";
(char)124 stands for the ASCII code of |. Knowing that ASCII code of TAB is 9 you may try using this in your connection string:
...;Extended Properties='text;HDR=YES;FMT=Delimited(" + (char)9 + ")'";
Try the above code snippet and also try your code using MS Access Database Engine driver. Since it's newer, maybe it has better delimiter config handling.
Related
In my Rails app, I managed to stream large CSV files directly from Postgres based on solutions mentioned in this SO post. My working code looks somewhat like so:
query = <A Long SQL Query String>
response.headers["Cache-Control"] = "no-cache"
response.headers["Content-Type"] = "text/csv; charset=utf-8"
response.headers["Content-Disposition"] =
%(attachment; filename="#{csv_filename}")
response.headers["Last-Modified"] = Time.now.ctime.to_s
conn = ActiveRecord::Base.connection.raw_connection
conn.copy_data("COPY (#{query}) TO STDOUT WITH (FORMAT CSV, HEADER TRUE, FORCE_QUOTE *, ESCAPE E'\\\\');") do
while row = conn.get_copy_data
response.stream.write row
end
end
response.stream.close
end
Some of the columns (VARCHAR) being queried have values as either English or Chinese strings. The CSV file resulting from the above code doesn’t show the Chinese characters as is. Instead, I get something like this:
大大 文文
Am I supposed to change the way I’m using the copy_data function, or is there something I could do to the CSV file to solve this? I’ve tried saving the file as UTF-8 .txt file, as well as trying the convert_to function mentioned in the copy_data documentation, but to no avail.
This depends of the original encoding included in the CSV file.
Do this on Linux :
file -i you_file
Are you sure it's not UTF-16 or GB 18030 ?
And also in what kind of encoding is setup your database ?
do a \l in psql to see this.
So it boiled down to my MS Excel not being able to render the Chinese chars correctly. On MacOS, opening the same .csv file using the Numbers app (or even Atom, for that matter) resolved this issue for me.
I am scripting with DM and would like to read hdf5 file format.
I borrowed Tore Niermann's gms_HDF5_Plug-In (hdf5_GMS2X_amd64.dll) and his CMD_import_hdf5.s script. It use h5_read_dataset(filename, datapath) to read a image dataset.
I am trying to figure out the way to read a string info stored in the same file. I am particular interested to read the angle stored in string as shown in this figure.Demonstrated string to read. The h5_read_dataset(filename, datapath) function doesn't work for reading string.
There is a help file (hdf5_plugin.chm) with a list of functions but unfortunately I can't open them to see more info.
hdf5_plugin.chm showing the function list.
I suppose the right function to read strings should be something like h5_read_attr() or h5_info() but I didn't test them out. DM always says the two functions doesn't exist.
After reading out the angle by string, I will also need a bit help to convert the string to a double datatype.
Thank you.
Converting String to Number is done with the Val() command.
There is no integer/double/float concept for variables in DM-script, all are just number. ( This is different for images, where you can define the numeric type. Also: For file-inport/export a type differntiation can be made using the taggroup streaming commands in the other answer. )
Example script:
string numStr = "1.234e-2"
number num = val( numStr )
ClearResults()
Result( "\n As string:" + numStr )
Result( "\n As value:" + num )
Result( "\n As value, formatted:" + Format(num,"%3.2f") )
Potential answer regarding the .chm files: When you download (or email) .chm files in Windows, the OS classifies them as "potentially dagerouse" (because it could contain executable HTML code, I think). As a result, these files can not be shown by default. However, you can right-click these files and "unblock" them in the file properties.
Example:
I think this will be most likely a question specific to that plugin and not general DM scripting. So it might be better to contact the plugin-author directly.
The alternative (not good) solution would be to "rewrite" your own HDF5 file-reader, if you know the file-format. For this you would need the "Streaming" commands of the DM script language and browse through the (binary?) source file to the apropriate file location. The starting point for reading on this in the F1 help documentation would be here:
We are converting word to pdf using the openoffice(3.4.1 version) in java with JODConverter.
below is the code used.
OpenOfficeConnection connection =
new SocketOpenOfficeConnection(2100);
try
{
connection.connect();
DocumentConverter converter =
new OpenOfficeDocumentConverter(connection);
converter.convert(inputFile, outputFile);
connection.disconnect();
return "Sucess " + DestinationPath + DestinationFileName;
}
catch (Exception localException1) {
}
The problem is that after random no of days the converted PDF contains the garbled fonts.
like # # ! $ $ " % &
The only solution we have so far is to restart the server. System guys are saying the the problem is with Open Office.
We are using open office to convert the document since it converts the doc files exactly including all the formatting and table structure.
what could be the solution to this.
So OpenOffice can be a little temperamental when running on a server, especially as it isn't multi-threaded and you end up having to run a pool of OpenOffice processes - see How can I use OpenOffice in server mode as a multithreaded service?.
Added to that often the rendering is off when converting to PDF - see https://forum.openoffice.org/en/forum/viewtopic.php?f=7&t=68865 which is why you may want to consider using a conversion service to automate the conversion tasks for you ?
For complete transparency I work for Zamzar (an online file conversion service), we have recently released a developer API - https://developers.zamzar.com/ that allows you to convert between a multitude of file types, specifically applicable to you here in that we support both doc and docx to pdf with little or not loss in the way the PDF is rendered. It maybe worth a look to see if this is a better alternative to trying to run your own solution through OpenOffice on a server.
I want to export csv file that contains hebrew character in my ASP.net MVC application
I have tried many encoding but not work. Actually hebrew characters and not displaying as they are.
Can anybody have idea?
System.Text.UnicodeEncoding Enc = new UnicodeEncoding();
HttpContext.Current.Response.AddHeader("Content-Length", Enc.GetByteCount(strExport).ToString());
HttpContext.Current.Response.BinaryWrite(Enc.GetBytes(strExport));
HttpContext.Current.Response.ContentEncoding = System.Text.Encoding.GetEncoding("windows-1255");
//HttpContext.Current.Response.Charset = "iso-8859-8";
HttpContext.Current.Response.ContentType = "text/csv";
HttpContext.Current.Response.AddHeader("content-disposition", string.Format("attachment;inline; filename={0}.csv", fileName));
HttpContext.Current.Response.End();
Check this out and see if setting the encoding helps: http://msdn.microsoft.com/en-us/library/system.text.encoding.aspx
Once upon a time we had multiple clients, including Hebrew, text files for import into MySQL, Sql Server, etc. The company had standardized on UTF8 as the encoding for everything. That was a few years ago, so ymmv.
Might be easier to debug if you show us a code sample.
I am looking for a simple method of zipping and compressing with delphi. I have already looked at the components at torry delphi:http://www.torry.net/pages.php?s=99. They all seem as though they would accomplish what I want however a few disadvantages to using them is that none of them run in delphi 2009 and are very complex which makes it difficult for me to port them to delphi 2009. And besides, the documentation on them is scarce, well at least to me. I need basic zipping functionality without the overhead of using a bunch of DLLs. My quest lead me to FSCTL_SET_COMPRESSION which I thought would have settled the issue but unfortunately this too did not work. CREATEFILE looked promising, until I tried it yielded the same result as FSCTL_SET... I know that there are some limited native zipping capability on windows. For instance if one right clicks a file or folder and selects -> sendTo ->zipped folder, a zipped archive is smartly created. I think if I was able to access that capability from delphi it will be a solution. On a side issue, does linux have its own native zipping functions that can be used similar to this?
TurboPower's excellent Abbrevia can be downloaded for D2009 here, D2010 support is underway and already available in svn according to their forum.
Abbrevia used to be a commercial (for $$$) product, which means that the documentation is quite complete.
I use Zipforge. Why are there problems porting these to D2009? Is it because of the 64bit??
Here is some sample code
procedure ZipIt;
var
Archiver: TZipForge;
FileName: String;
begin
try
Archiver:= TZipForge.create(self);
with Archiver do begin
FileName := 'c:\temp\myzip.zip';
// Create a new archive file
OpenArchive(fmCreate);
// Set path to folder with some text files to BaseDir
BaseDir := 'c:\temp\';
// Add all files and directories from 'C:\SOURCE_FOLDER' to the archive
AddFiles('myfiletozip.txt');
// Close the archive
CloseArchive;
end;
finally
Archiver.Free;
end;
end;
If you can "do" COM from Delphi, then you can take advantage of the built-in zip capability of the Windows shell. It gives you good basic capability.
In VBScript it looks like this:
Sub CreateZip(pathToZipFile, dirToZip)
WScript.Echo "Creating zip (" & pathToZipFile & ") from folder (" & dirToZip & ")"
Dim fso
Set fso= Wscript.CreateObject("Scripting.FileSystemObject")
If fso.FileExists(pathToZipFile) Then
WScript.Echo "That zip file already exists - deleting it."
fso.DeleteFile pathToZipFile
End If
If Not fso.FolderExists(dirToZip) Then
WScript.Echo "The directory to zip does not exist."
Exit Sub
End If
NewZip pathToZipFile
dim sa
set sa = CreateObject("Shell.Application")
Dim zip
Set zip = sa.NameSpace(pathToZipFile)
WScript.Echo "opening dir (" & dirToZip & ")"
Dim d
Set d = sa.NameSpace(dirToZip)
For Each s In d.items
WScript.Echo s
Next
' http://msdn.microsoft.com/en-us/library/bb787866(VS.85).aspx
' ===============================================================
' 4 = do not display a progress box
' 16 = Respond with "Yes to All" for any dialog box that is displayed.
' 128 = Perform the operation on files only if a wildcard file name (*.*) is specified.
' 256 = Display a progress dialog box but do not show the file names.
' 2048 = Version 4.71. Do not copy the security attributes of the file.
' 4096 = Only operate in the local directory. Don't operate recursively into subdirectories.
WScript.Echo "copying files..."
zip.CopyHere d.items, 4
' wait until finished
sLoop = 0
Do Until d.Items.Count <= zip.Items.Count
Wscript.Sleep(1000)
Loop
End Sub
COM also allws you to use DotNetZip, which is a free download, that does password-encrypted zips, zip64, Self-extracting archives, unicode, spanned zips, and other things.
Personally I use VCL Zip which runs with D2009 and D2010 perfectly fine. it does cost $120 at the time of this post but is very simple, flexible and most of all FAST.
Have a look at VCLZIP and download the trail if your interested
code wise:
VCLZip1.ZipName := ‘myfiles.zip’;
VCLZip1.FilesList.add(‘c:\mydirectory\*.*’);
VCLZip1.Zip;
is all you need for a basic zip, you can of course set compression levels, directory structures, zip streams, unzip streams and much more.
Hope this is of some assistance.
RE
Take a look at this OpenSource SynZip unit. It's even faster for decompression than the default unit shipped with Delphi, and it will generate a smaller exe (crc tables are created at startup).
No external dll is needed. Works from Delphi 6 up to XE. No problem with Unicode version of Delphi. All in a single unit.
I just made some changes to handle Unicode file names inside Zip content, not only Win-Ansi charset but any Unicode chars. Feedback is welcome.