I am trying to upload so many files via Azure Blob Storage .NET api and using with the current latest version 4.0.1. In ASP.NET MVC application i use async action method to upload via await blobFile.UploadFromStreamAsync but it really doesn't work and even i don't see an exception. It silently stops in that method without success.
But if i change action method to none-async and upload via blobFile.UploadFromStream method then everything to works well. I may uploaded via async way with 1% success rate that means very very low stability.
Do you experience same thing ? Is it bug in Storage Api implementation ?
Here is short example. One is async and the other one is none async action methods. There is no any problem if i upload small files but problem appears on large downloads. In this example UploadBlobSec method upload in short time but UploadBlob takes endless time.
public async Task UploadBlob()
{
var storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["AzureStorage"].ConnectionString);
var blobContainer = storageAccount.CreateCloudBlobClient().GetContainerReference("files");
var blobFile = blobContainer.GetBlockBlobReference("song.mp3");
using (var stream = new WebClient().OpenRead("http://apolyonstorage.blob.core.windows.net/files/e8b1a1fa-8791-44dc-92ce-1a67a62f7b0f.mp3"))
{
await blobFile.UploadFromStreamAsync(stream);
}
}
public void UploadBlobSec()
{
var storageAccount = CloudStorageAccount.Parse(ConfigurationManager.ConnectionStrings["AzureStorage"].ConnectionString);
var blobContainer = storageAccount.CreateCloudBlobClient().GetContainerReference("files");
var blobFile = blobContainer.GetBlockBlobReference("song.mp3");
using (var stream = new WebClient().OpenRead("http://apolyonstorage.blob.core.windows.net/files/e8b1a1fa-8791-44dc-92ce-1a67a62f7b0f.mp3"))
{
blobFile.UploadFromStream(stream);
}
}
Code snippet looks fine - although I am not sure what the application around it is doing. Have you turned on server logs to see what is happening server side? For large files you should see the async upload translated into a couple of PutBlocks and then a PutBlockList. If you don't see the PutBlockList them maybe something strange is happening in your application.
Then assuming you do see the operations in the server logs that obviously means the operations are occurring. At that point look at the E2ELatency numbers vs ServerLatency I think you will see a large difference with E2Latency being much higher as it incorporates the time spent client side - and it would be interesting to see if your client network could be contributing to the problem. For example on my connection the e2elatency on the first PutBlock was 1346 ms vs 137 for ServerLatency.
For more information on logging take a look here.
Jason
Related
I'm looking for a way to download large pdf files from an external server with a Flutter application for offline storage.
But downloading a large file (sometimes 100mb+) takes some time. I don't want the app being stuck in a wait function for it to download. What i'm looking for is a download function that has a callback with a progress report (Something like: 250000/500000 bytes done. Doesn't have to be exactly that. Just something that I can work with and make a progress bar out of).
Is this even possible to do in Flutter? The only things I came across were the HTTP library. But that does not seem to have a progress callback and just plainly reading the contents of a http call (Which also doesn't have a progress report). I hope someone has a method for me that I can use to make this happen.
Kind regards,
Kevin Walter
EDIT:
C# has the perfect example of what I mean
https://stackoverflow.com/a/9459441/2854656
https://docs.flutter.io/flutter/dart-io/HttpClient-class.html
https://docs.flutter.io/flutter/dart-io/HttpClientResponse-class.html
int fileSize;
int downloadProgress = 0;
new HttpClient().get('localhost', 80, '/file.txt')
.then((HttpClientRequest request) => request.close())
.then((HttpClientResponse response) {
fileSize ??= respone.contentLength;
response.transform(utf8.decoder).listen((contents) {
downloadProgres += contents.length;
// handle data
});
});
When I use the Download method of the FileReference class, everything works fine on Desktop and Android, but I get an error on iOS.
This is the code:
var req = new URLRequest(url);
var localRef:FileReference = new FileReference();
localRef.download(req);
On iOS I'm getting an Alert:
Download Error
File downloads not supported.
I already tried to NavigateToUrl() and it asks save the file in Dropbox or another App.
How can I fix this error?
You shouldn't use FileReference on mobile (or AIR, in general, though it does open the Load/Save dialog on desktop so there can be some use there). You should instead use File and FileStream, which give you far more control over the file system.
In this case, you could try to use File.download() and save it to File.applicationStorageDirectory, but I don't know if it will have any difference since it extends FileReference.
What I generally do is use URLStream instead of URLLoader. It gives you access to the raw bytes of the file you are downloading and then use File and FileStream
So something like (and this is untested off the top of my head, though I have used similar in the past):
var urlStream:URLStream = new URLStream();
urlStream.addEventListener(Event.COMPLETE, completeHandler);
urlStream.load(new URLLoader('url');
function completeHandler(e:Event):void {
var bytes:ByteArray = new ByteArray();
urlStream.readBytes(bytes);
var f:File = File.applicationStorageDirectory.resolvePath('filename');
var fs:FileStream = new FileStream();
fs.open(f, FileMode.WRITE);
fs.writeBytes(bytes);
fs.close();
}
Now, obviously, there is a lot more you want to account for (errors, progress, etc). That should be enough to point you in the right direction, however.
It's possible to create a full download manager using this method (something I did for an iOS project two years ago), since you can save as-you-go to the file system rather than waiting until Event.COMPLETE fires (using the ProgressEvent.PROGRESS event). That allows you to avoid having a 500MB file in memory, something most devices can't handle.
I'm currently converting a Chrome extension into a Firefox add-on and would appreciate to replicate the chrome.storage.sync feature.
However, I cannot manage to find whether the data stored by a Firefox add-on using simple-storage will be automatically synced whenever a user of the add-on is signed into Firefox Sync.
Given that all the pieces of data stored via the latter method can be found in the user profile, I presume it will... as long as the add-on is available at https://addons.mozilla.org ?
Any information on the topic would be greatly appreciated.
simple-storage is not synced. But you can sync it with little effort.
The trick it to store the storage object, it is serializable by definition, as a string preference and tell to the sync service to synchronize it.
Lets name that preference syncstorage and mark it as synchronizable.
var self = require("sdk/self");
var prefs = require("sdk/preferences/service");
prefs.set("services.sync.prefs.sync.extensions." + self.id + ".syncstorage", true);
When storing something to simple-storage reflect the change to syncstorage.
var sp = require("sdk/simple-prefs");
var ss = require("sdk/simple-storage");
sp.prefs["syncstorage"] = JSON.stringify(ss.storage);
For the opposite effect watch syncstorage for changes
sp.on("syncstorage", function(prefname){
ss.storage = JSON.parse(sp.prefs["syncstorage"]);
})
Last but not least, It would nice and perhaps mandatory to sync only with the explicit consent of the user.
the code below was developed for shared memory. when it is used on two services or two user app, it work pretty well. but when the memory is created in service, the application can not find the memory. what is wrong with this code?
in service:
mmf=MemoryMappedFile.CreateNew("ALFMap",10000);
bool mutexCreated;
Mutex mutex=new Mutex(true,"ALFMutex",out mutexCreated);
stream=mmf.CreateViewStream(0,1000);
BinaryWriter writer=new BinaryWriter(stream);
writer.Write("I am reza dadkhah");
mutex.ReleaseMutex();
in user app:
using (MemoryMappedFile mmf = MemoryMappedFile.OpenExisting("ALFMap",MemoryMappedFileRights.FullControl))
{
Mutex mutex=Mutex.OpenExisting("ALFMutex");
mutex.WaitOne();
using (MemoryMappedViewStream stream=mmf.CreateViewStream(0,1000))
{
BinaryReader reader = new BinaryReader(stream);
textBox1.Text=reader.ReadString();
}
mutex.ReleaseMutex();
}
Have you tried writing the content of the memory in a file using your application ? Try that first and confirm the value is in the memory or not.
Can anyone suggest how to handle a slow network when streaming video in a web view?
When the network strength is poor, a blank screen appears or video doesn't stream.
Is there a way to detect this condition so that we can alert the user? (Apart from using private API.)
Perhaps ifi_baudrate member of the if_data structure (declared in <net/if.h>) is what you need. If baudrate is less than some threshold value, then you can show an alert.
Please see the following answer to know how to obtain the if_data structure for a particular network interface:
https://stackoverflow.com/a/8014012/1310204
You can easily detect the state of the network connection via the HTML5 networking API
http://www.html5rocks.com/en/mobile/optimization-and-performance/#toc-network-detection
Also if you want to test the network speed, just set up some files on your server of a specific size, and do a ajax request for the file, while timing how long it takes to download.
You can use a simple:
var start = new Date();
$.get("someFile.jpg")
.done(function() {
var elapsed = (new Date() - start);
});
Or dig into the HTML5 performance API:
http://www.html5rocks.com/en/tutorials/webperformance/basics/
...if you not using javascript, the same applies. Just open a network connection with whatever is at your disposition, download a small file & do the math ;-)