The underlying connection was closed: An unexpected error occurred on a receive (After file upload completed) - asp.net-mvc

In Asp.Net MVC 5
I use this code in a class to upload my file in FTP Server.
FtpWebRequest ftpRequest = (FtpWebRequest)WebRequest.Create(ftpURL);
ftpRequest.Method = WebRequestMethods.Ftp.UploadFile;
ftpRequest.Credentials = new NetworkCredential(UserName, Password);
ftpRequest.Timeout = -1;
Stream requestStream = ftpRequest.GetRequestStream();
long fileSize = 0;
Byte[] buffer = new Byte[buferSize];
int bytesRead = file.Read(buffer, 0, buferSize);
fileSize += bytesRead;
while (bytesRead > 0)
{
requestStream.Write(buffer, 0, bytesRead);
bytesRead = file.Read(buffer, 0, buferSize);
fileSize += bytesRead;
}
requestStream.Close();
return fileSize;
and I have this config in web.config because files are too big:
<location path="Programs/Upload">
<system.web>
<httpRuntime executionTimeout="86400" maxRequestLength="2097152" />
</system.web>
<system.webServer>
<security>
<requestFiltering>
<requestLimits maxAllowedContentLength="2147483648"/>
</requestFiltering>
</security>
</system.webServer>
When I try to upload small files (in release mode with debug = 'false') everything work well. but when file size become big I face with this error (after file completely uploaded):
**The underlying connection was closed: An unexpected error occurred on a receive**
The file is completely uploaded and is available in FTP Server!

Related

Xamarin - WCF Upload Large Files Report progress vis UIProgressView

I have create a WCF Service that allows uploading large files via BasicHttpBinding using streaming and it is working great! I would like to extended this to show a progress bar (UIProgressView) so that when a large file is being uploaded in 65k chunks, the user can see that it is actively working.
The client code calling the WCF Service is:
BasicHttpBinding binding = CreateBasicHttp ();
BTSMobileWcfClient _client = new BTSMobileWcfClient (binding, endPoint);
_client.UploadFileCompleted += ClientUploadFileCompleted;
byte[] b = File.ReadAllBytes (zipFileName);
using (new OperationContextScope(_client.InnerChannel)) {
OperationContext.Current.OutgoingMessageHeaders.Add(System.ServiceModel.Channels.MessageHeader.CreateHeader("SalvageId","",iBTSSalvageId.ToString()));
OperationContext.Current.OutgoingMessageHeaders.Add(System.ServiceModel.Channels.MessageHeader.CreateHeader("FileName","",Path.GetFileName(zipFileName)));
OperationContext.Current.OutgoingMessageHeaders.Add(System.ServiceModel.Channels.MessageHeader.CreateHeader("Length","",b.LongLength));
_client.UploadFileAsync(b);
}
On the server side, I read the file stream in 65k chuncks and do report back to the calling routine "bytes read", etc. A snippet of code for that is:
using (FileStream targetStream = new FileStream(filePath, FileMode.CreateNew,FileAccess.Write)) {
//read from the input stream in 65000 byte chunks
const int chunkSize = 65536;
byte[] buffer = new byte[chunkSize];
do {
// read bytes from input stream
int bytesRead = request.FileData.Read(buffer, 0, chunkSize);
if (bytesRead == 0) break;
// write bytes to output stream
targetStream.Write(buffer, 0, bytesRead);
} while (true);
targetStream.Close();
}
But I don't know how to hook into the callback on the Xamarin side to receive the "bytes read" versus "total bytes to send" so I can update the UIProgressView.
Has anyone tried this or is this even possible?
Thanks In Advance,
Bo

ASP.NET MVC2 and IIS 7.5 . Problems when read large data from Excel 2010

I have a problem when reading large amount of data from Excel 2010
I use the code below to read data:
public static DataTable GetExcelData(string filePath, string extension)
{
try
{
string conStr = "";
switch (extension)
{
case ".xls": //Excel 97-03
//conStr = ConfigurationManager.ConnectionStrings["Excel03ConString"].ConnectionString;
//conStr = "Provider=Microsoft.Jet.OLEDB.4.0;Data Source=" + filePath + ";Extended Properties='Excel 8.0;HDR={YES}'";
conStr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=Excel 12.0;";
break;
case ".xlsx": //Excel 07
//conStr = ConfigurationManager.ConnectionStrings["Excel07ConString"].ConnectionString;
conStr = "Provider=Microsoft.ACE.OLEDB.12.0;Data Source=" + filePath + ";Extended Properties=Excel 12.0;";
break;
}
//conStr = String.Format(conStr, filePath, true);
OleDbConnection connExcel = new OleDbConnection(conStr);
OleDbCommand cmdExcel = new OleDbCommand();
OleDbDataAdapter oda = new OleDbDataAdapter();
DataTable dt = new DataTable();
cmdExcel.Connection = connExcel;
//Get the name of First Sheet
log.Debug("Excel ConnectionString = " + conStr);
connExcel.Open();
log.Debug("Open Excel connection ok");
DataTable dtExcelSchema;
dtExcelSchema = connExcel.GetOleDbSchemaTable(OleDbSchemaGuid.Tables, null);
string SheetName = dtExcelSchema.Rows[0]["TABLE_NAME"].ToString();
connExcel.Close();
log.Debug("SHEET NAME = " + SheetName);
//Read Data from First Sheet
connExcel.Open();
cmdExcel.CommandText = "SELECT * From [" + SheetName + "]";
oda.SelectCommand = cmdExcel;
oda.Fill(dt);
connExcel.Close();
return dt;
}
catch(Exception ex)
{
log.Debug("GetExcelData Error:" + ex.ToString());
return null;
}
}
My ASP.NET MVC 2.0 project works correctly when running on Visual Studio 2010 with both a small Excel file and a large Excel file.
But, when I deploy to IIS 7.5 (windows 7 pro), my website only works correctly with small Excel file.
With a large excel file, it throw the following error:
GetExcelData Error:System.Data.OleDb.OleDbException (0x80004005): External table is not in the expected format.
at System.Data.OleDb.OleDbConnectionInternal..ctor(OleDbConnectionString constr, OleDbConnection connection)
at System.Data.OleDb.OleDbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningObject)
at System.Data.ProviderBase.DbConnectionFactory.CreateConnection(DbConnectionOptions options, DbConnectionPoolKey poolKey, Object poolGroupProviderInfo, DbConnectionPool pool, DbConnection owningConnection, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.CreateNonPooledConnection(DbConnection owningConnection, DbConnectionPoolGroup poolGroup, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionFactory.TryGetConnection(DbConnection owningConnection, TaskCompletionSource`1 retry, DbConnectionOptions userOptions, DbConnectionInternal& connection)
at System.Data.ProviderBase.DbConnectionClosed.TryOpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory, TaskCompletionSource`1 retry, DbConnectionOptions userOptions)
at System.Data.ProviderBase.DbConnectionInternal.OpenConnection(DbConnection outerConnection, DbConnectionFactory connectionFactory)
at System.Data.OleDb.OleDbConnection.Open()
(On IIS 7.5, my website uploads large file Excel success)
How can I fix this?
It might be a web.config change that is needed to allow larger files over 4MB. You can add the following to the web.config to allow this,
<configuration>
<system.web><!-- maxRequestLength (KB) -->
<httpRuntime maxRequestLength="10000" executionTimeout="110"/>
</system.web>
</configuration>
This may help,
Maximum request length exceeded
I use the following connection string where '{0}' is the file path, for .xls files,
strConnectionString = #"Provider=Microsoft.ACE.OLEDB.12.0;
Data Source={0};Extended Properties= ""Excel 12.0 Xml;HDR=YES;IMEX=1"""
and for .xlsx files I use,
strConnectionString = #"Provider=Microsoft.ACE.OLEDB.12.0;
Data Source={0};Extended Properties=""Excel 12.0 Xml;HDR=YES""";
It won't be the size of the files if you have set up the web.config up. I would try and save the file as an 97-2003 workbook .xls file and use the relevant connection string. If you are hosting this on a server, the server will need the Office drivers needed to process the file.
UPDATE
Set HDR to No and go through each row dismissing the header row. This will avoid a data type issue that occurs when the columns are checked for the data type.

is "HttpWebRequest" using of my site's Bandwidth?

I want to read mp3 file from external url and hide url path for client in my website.
I'm using this code:
//Create a stream for the file
Stream stream = null;
//This controls how many bytes to read at a time and send to the client
int bytesToRead = 10000;
// Buffer to read bytes in chunk size specified above
byte[] buffer = new Byte[bytesToRead];
// The number of bytes read
try
{
//Create a WebRequest to get the file
HttpWebRequest fileReq = (HttpWebRequest)HttpWebRequest.Create("http://www.alsacreations.fr/mp3/everywhere.mp3");
//Create a response for this request
HttpWebResponse fileResp = (HttpWebResponse)fileReq.GetResponse();
if (fileReq.ContentLength > 0)
fileResp.ContentLength = fileReq.ContentLength;
//Get the Stream returned from the response
stream = fileResp.GetResponseStream();
// prepare the response to the client. resp is the client Response
var resp = HttpContext.Current.Response;
//Indicate the type of data being sent
resp.ContentType = "application/octet-stream";
//Name the file
string fileName = "everywhere.mp3";
resp.AddHeader("Content-Disposition", "attachment; filename=\"" + fileName);
resp.AddHeader("Content-Length", fileResp.ContentLength.ToString());
int length;
do
{
// Verify that the client is connected.
if (resp.IsClientConnected)
{
// Read data into the buffer.
length = stream.Read(buffer, 0, bytesToRead);
// and write it out to the response's output stream
resp.OutputStream.Write(buffer, 0, length);
// Flush the data
resp.Flush();
//Clear the buffer
buffer = new Byte[bytesToRead];
}
else
{
// cancel the download if client has disconnected
length = -1;
}
} while (length > 0); //Repeat until no data is read
}
finally
{
if (stream != null)
{
//Close the input stream
stream.Close();
}
}
This code is work correctly, My question is for download this mp3 file(http://www.alsacreations.fr/mp3/everywhere.mp3) use My Site's BandWidth or not?

Video streaming to ipad does not work with Tapestry5

I want to stream a video to my IPad via the HTML5 video tag with tapestry5 (5.3.5) on the backend. Usually the serverside framework shouldn't even play a role in this but somehow it does.
Anyway, hopefully someone here can help me out. Please keep in mind that my project is very much a prototype and that what I describe is simplified / reduced to the relevant parts. I would very much appreciate it if people didn't respond with the obligatory "you want to do the wrong thing" or security/performance nitpicks that aren't relevant to the problem.
So here it goes:
Setup
I have a video taken from the Apple HTML5 showcase so I know that format isn't an issue. I have a simple tml page "Play" that just contains a "video" tag.
Problem
I started by implementing a RequestFilter that handles the request from the video control by opening the referenced video file and streaming it to client. That's basic "if path starts with 'file' then copy file inputstream to response outputstream". This works very well with Chrome but not with the Ipad. Fine, I though, must be some headers I'm missing so I looked at the Apple Showcase again and included the same headers and content type but no joy.
Next, I though, well, let's see what happens if I let t5 serve the file. I copied the video to the webapp context, disabled my request filter and put the simple filename in the video's src attribute. This works in Chrome AND IPad.
That surprised me and prompted me to look at how T5 handles static files / context request. Thus far I've only gotten so far as to feel like there are two different paths which I've confirmed by switching out the hardwired "video src" to an Asset with a #Path("context:"). This, again, works on Chrome but not on IPad.
So I'm really lost here. What's this secret juice in the "simple context" requests that allow it to work on the IPad? There is nothing special going on and yet it's the only way this works. Problem is, I can't really serve those vids from my webapp context ...
Solution
So, it turns out that there is this http header called "Range" and that the IPad, unlike Chrome uses it with video. The "secret sauce" then is that the servlet handler for static resource request know how to deal with range requests while T5's doesn't. Here is my custom implementation:
OutputStream os = response.getOutputStream("video/mp4");
InputStream is = new BufferedInputStream( new FileInputStream(f));
try {
String range = request.getHeader("Range");
if( range != null && !range.equals("bytes=0-")) {
logger.info("Range response _______________________");
String[] ranges = range.split("=")[1].split("-");
int from = Integer.parseInt(ranges[0]);
int to = Integer.parseInt(ranges[1]);
int len = to - from + 1 ;
response.setStatus(206);
response.setHeader("Accept-Ranges", "bytes");
String responseRange = String.format("bytes %d-%d/%d", from, to, f.length());
logger.info("Content-Range:" + responseRange);
response.setHeader("Connection", "close");
response.setHeader("Content-Range", responseRange);
response.setDateHeader("Last-Modified", new Date().getTime());
response.setContentLength(len);
logger.info("length:" + len);
byte[] buf = new byte[4096];
is.skip(from);
while( len != 0) {
int read = is.read(buf, 0, len >= buf.length ? buf.length : len);
if( read != -1) {
os.write(buf, 0, read);
len -= read;
}
}
} else {
response.setStatus(200);
IOUtils.copy(is, os);
}
} finally {
os.close();
is.close();
}
I want to post my refined solution from above. Hopefully this will be useful to someone.
So basically the problem seemed to be that I was disregarding the "Range" http request header which the IPad didn't like. In a nutshell this header means that the client only wants a certain part (in this case a byte range) of the response.
This is what an iPad html video request looks like::
[INFO] RequestLogger Accept:*/*
[INFO] RequestLogger Accept-Encoding:identity
[INFO] RequestLogger Connection:keep-alive
[INFO] RequestLogger Host:mars:8080
[INFO] RequestLogger If-Modified-Since:Wed, 10 Oct 2012 22:27:38 GMT
[INFO] RequestLogger Range:bytes=0-1
[INFO] RequestLogger User-Agent:AppleCoreMedia/1.0.0.9B176 (iPad; U; CPU OS 5_1 like Mac OS X; en_us)
[INFO] RequestLogger X-Playback-Session-Id:BC3B397D-D57D-411F-B596-931F5AD9879F
It means that the iPad only wants the first byte. If you disregard this header and simply send a 200 response with the full body then the video won't play. So, you need send a 206 response (partial response) and set the following response headers:
[INFO] RequestLogger Content-Range:bytes 0-1/357772702
[INFO] RequestLogger Content-Length:2
This means "I'm sending you byte 0 through 1 of 357772702 total bytes available".
When you actually start playing the video, the next request will look like this (everything except the range header ommited):
[INFO] RequestLogger Range:bytes=0-357772701
So my refined solution looks like this:
OutputStream os = response.getOutputStream("video/mp4");
try {
String range = request.getHeader("Range");
/** if there is no range requested we will just send everything **/
if( range == null) {
InputStream is = new BufferedInputStream( new FileInputStream(f));
try {
IOUtils.copy(is, os);
response.setStatus(200);
} finally {
is.close();
}
return true;
}
requestLogger.info("Range response _______________________");
String[] ranges = range.split("=")[1].split("-");
int from = Integer.parseInt(ranges[0]);
/**
* some clients, like chrome will send a range header but won't actually specify the upper bound.
* For them we want to send out our large video in chunks.
*/
int to = HTTP_DEFAULT_CHUNK_SIZE + from;
if( to >= f.length()) {
to = (int) (f.length() - 1);
}
if( ranges.length == 2) {
to = Integer.parseInt(ranges[1]);
}
int len = to - from + 1 ;
response.setStatus(206);
response.setHeader("Accept-Ranges", "bytes");
String responseRange = String.format("bytes %d-%d/%d", from, to, f.length());
response.setHeader("Content-Range", responseRange);
response.setDateHeader("Last-Modified", new Date().getTime());
response.setContentLength(len);
requestLogger.info("Content-Range:" + responseRange);
requestLogger.info("length:" + len);
long start = System.currentTimeMillis();
RandomAccessFile raf = new RandomAccessFile(f, "r");
raf.seek(from);
byte[] buf = new byte[IO_BUFFER_SIZE];
try {
while( len != 0) {
int read = raf.read(buf, 0, buf.length > len ? len : buf.length);
os.write(buf, 0, read);
len -= read;
}
} finally {
raf.close();
}
logger.info("r/w took:" + (System.currentTimeMillis() - start));
} finally {
os.close();
}
This solution is better then my first one because it handles all cases for "Range" requests which seems to be a prereq for clients like Chrome to be able to support skipping within the video ( at which point they'll issue a range request for that point in the video).
It's still not perfect though. Further improvments would be setting the "Last-Modified" header correctly and doing proper handling of clients requests an invalid range or a range of something else then bytes.
I suspect this is more about iPad than about Tapestry.
I might invoke Response.disableCompression() before writing the stream to the response; Tapestry may be trying to GZIP your stream, and the iPad may not be prepared for that, as video and image formats are usually already compressed.
Also, I don't see a content type header being set; again the iPad may simply be more sensitive to that than Chrome.

Can we compress a large file as a chunk data for GZIP in blackberry?

I saw the sample APIas below
public static byte[] compress( byte[] data )
{
try
{
ByteArrayOutputStream baos = new ByteArrayOutputStream();
GZIPOutputStream gzipStream = new GZIPOutputStream( baos, 6, GZIPOutputStream.MAX_LOG2_WINDOW_LENGTH );
gzipStream.write( data );
gzipStream.close();
}
catch(IOException ioe)
{
return null;
}
return baos.toByteArray();
}
But when I tried to compress with a large file with Curve 8900 OS 4.6, I got a "OutOfMemoryError" so I would like to know that how to compress as a chunk small data?
I already tried with this code as below but it doesn't work, compressed file cannot decompress...
file = (FileConnection)Connector.open(_fileOutputPath, Connector.READ_WRITE);
if (!file.exists()) {
file.create();
}
os = file.openOutputStream();
is = FileUtil.getInputStream(_fileInputPath, 0);
int tmpSize = 1024;
byte[] tmp = new byte[tmpSize];
int len = -1;
gzipStream = new GZIPOutputStream( os, 6, GZIPOutputStream.MAX_LOG2_WINDOW_LENGTH );
while((len = is.read(tmp, 0, tmpSize)) != -1) {
gzipStream.write(tmp, 0, len);
}
GZIPOutputStream does not produce a file suitable for use with the gzip command line tool. This is because it doesn't produce the necessary file headers. How did you test decompressing it? You should write a similar Java program that makes use of GZIPInputStream to test, as 'gunzip' is not going to recognize the input.
The problem of the first code sample is that the ByteArrayOutputStream is getting too big for the limited memory of a mobile device.
An option could be to first write to a file (for instance) on SD card.
The second code sample seems fine, but see Michael's answer.

Resources