Flutter: How to get upload / download progress for http requests - dart

I am writing an app that uploads an image to a server, and instead of just showing a spinner, I'd love to be able to get progress on the status of that upload.
Additionally, I want to do this without using Multipart form data. This is the code I'm currently using - but it appears to be stalling out with a broken pipe, and I have zero feedback as to whether data is being sent to the server:
Future<String> _uploadFile(File assetFile) async {
final url = <removed>;
final stream = await assetFile.openRead();
int length = assetFile.lengthSync();
final client = new HttpClient();
final request = await client.postUrl(Uri.parse(url));
request.headers.add(HttpHeaders.CONTENT_TYPE, "application/octet-stream");
request.contentLength = length;
await request.addStream(stream);
final response = await request.close();
// response prociessing.
}
Is it possible to send large data as a stream without reading it into memory, and can I get progress on that upload with current dart / flutter APIs?

Screenshot (Null Safe):
This solution
Downloads an image from server.
Shows downloading progress.
After download, the image is saved to device storage.
Code:
import 'package:http/http.dart' as http;
class _MyPageState extends State<MyPage> {
int _total = 0, _received = 0;
late http.StreamedResponse _response;
File? _image;
final List<int> _bytes = [];
Future<void> _downloadImage() async {
_response = await http.Client()
.send(http.Request('GET', Uri.parse('https://upload.wikimedia.org/wikipedia/commons/f/ff/Pizigani_1367_Chart_10MB.jpg')));
_total = _response.contentLength ?? 0;
_response.stream.listen((value) {
setState(() {
_bytes.addAll(value);
_received += value.length;
});
}).onDone(() async {
final file = File('${(await getApplicationDocumentsDirectory()).path}/image.png');
await file.writeAsBytes(_bytes);
setState(() {
_image = file;
});
});
}
#override
Widget build(BuildContext context) {
return Scaffold(
floatingActionButton: FloatingActionButton.extended(
label: Text('${_received ~/ 1024}/${_total ~/ 1024} KB'),
icon: Icon(Icons.file_download),
onPressed: _downloadImage,
),
body: Padding(
padding: const EdgeInsets.all(20.0),
child: Center(
child: SizedBox.fromSize(
size: Size(400, 300),
child: _image == null ? Placeholder() : Image.file(_image!, fit: BoxFit.fill),
),
),
),
);
}
}

The way that you are already using Stream means that you are not reading the whole file into memory. It's being read in as, probably, 64k chunks.
You could intercept the stream between the producer (File) and consumer (HttpClient) with a StreamTransformer, like this:
int byteCount = 0;
Stream<List<int>> stream2 = stream.transform(
new StreamTransformer.fromHandlers(
handleData: (data, sink) {
byteCount += data.length;
print(byteCount);
sink.add(data);
},
handleError: (error, stack, sink) {},
handleDone: (sink) {
sink.close();
},
),
);
....
await request.addStream(stream2);
You should see byteCount incrementing in 64k chunks.

Try dio library. The onSendProgress callback would be helpful.
example:
response = await dio.post(
"http://www.example.com",
data: data,
onSendProgress: (int sent, int total) {
print("$sent $total");
},
);
Reference: https://github.com/flutterchina/dio/issues/103

Update:
A day after posting this I realized that this is not actually measuring the upload progress, but only the progress of reading the bytes from the local JSON payload, which is almost instantaneous. If/when I figure out how to actually measure the upload progress, I'll update this answer.
Original Answer:
This works for me, without using Multipart:
import 'dart:convert';
import 'package:http/http.dart' as http;
// ...
final jsonPayload = {'base64File': 'abc123', 'something': 'else'};
// We are using a StreamedRequest so we can track the upload progress
final streamedRequest = http.StreamedRequest("POST", apiUri);
streamedRequest.headers['content-type'] = 'application/json';
// Length transferred (to calculate upload progress)
var transferredLength = 0;
// Upload progress (from 0.0 to 1.0)
var uploadProgress = 0.0;
// The stringified JSON payload
var stringEncodedPayload = jsonEncode(jsonPayload);
// Total length (to calculate upload progress)
var totalLength = stringEncodedPayload.length;
// Create a stream of the payload string
Stream.value(stringEncodedPayload)
// Transform the string-stream to a byte stream (List<int>)
.transform(utf8.encoder)
// Start reading the stream in chunks, submitting them to the streamedRequest for upload
.listen((chunk) {
transferredLength += chunk.length;
uploadProgress = transferredLength / totalLength;
print("Chunk: ${chunk.length}, transferred: $transferredLength, progress: $uploadProgress");
streamedRequest.sink.add(chunk);
}, onDone: () {
print("Done. Total: $totalLength, transferred: $transferredLength, progress: $uploadProgress");
streamedRequest.sink.close();
});
final result = await client.send(streamedRequest).then(http.Response.fromStream);
print("----------->");
print(result.statusCode);
print(result.body);
print("<-----------");
The output:
flutter: Chunk: 1024, transferred: 1024, progress: 0.0008807503580198599
flutter: Chunk: 1024, transferred: 2048, progress: 0.0017615007160397197
flutter: Chunk: 1024, transferred: 3072, progress: 0.0026422510740595796
...
flutter: Chunk: 1024, transferred: 1159168, progress: 0.9970094052784814
flutter: Chunk: 1024, transferred: 1160192, progress: 0.9978901556365013
flutter: Chunk: 1024, transferred: 1161216, progress: 0.9987709059945211
flutter: Chunk: 1024, transferred: 1162240, progress: 0.9996516563525409
flutter: Chunk: 405, transferred: 1162645, progress: 1.0
flutter: Done. Total: 1162645, transferred: 1162645, progress: 1.0

Related

Flutter download flie in stream for progress control for iOS devices

This code work well at Android and at Android Studio framework, in same time if code deployed to iOS devices i receive error
I reciving
OS (Exhausted heap space, trying to allocate 1073741840 bytes.
[VERBOSE-2:ui_dart_state.cc(198)] Unhandled Exception: Out of )
await response.stream.listen(
(List<int> newBytes) {
bytes.addAll(newBytes);
final downloadedLength = bytes.length;
_progress = (downloadedLength / contentLength!) / 2 * 100;
if(bytes.length>96607667)
{
print (bytes.length);
}
if(newBytes.length>96607667)
{
print (newBytes.length);
}
fileInfo.progress = _progress;
},
cancelOnError: true
).asFuture();
size of file is 96607667
i can't understand - what i do wrong?
I solved the problem in the following way
late Uint8List bytes;
int received = 0;
var completer = Completer<Uint8List>();
var sink = ByteConversionSink.withCallback(
(bytes) => completer.complete(Uint8List.fromList(bytes)));
response.stream.listen(
(List<int> newBytes) {
sink.add(newBytes);
received += newBytes.length;
fileInfo.progress = (received / contentLength) / 2 * 100;
},
onError: completer.completeError,
onDone: sink.close,
cancelOnError: true);

Assert all the items emitted by a Stream in order until it is canceled in Dart?

Is there any convenient way to assert all the items emitted by a Stream in order until it is canceled?
If I use:
expectLater(
stream,
emitsInOrder(<String>[
'item1',
'item2',
]),
);
and the Stream emits ['item1', 'item2', 'item3'] , the test won't fail.
The only way I've found so far is the following:
var count = 0;
final expected = ['item1', 'item2', 'item3'];
stream.listen(
expectAsync1(
(final result) {
expect(result, expected[count++]);
},
count: expected.length,
),
);
But it is a bit verbose and not very easy to read. Is there a simpler/more elegant way?
You can collect the items into a list, using toList, then compare it to your own expectation list:
await expectLater(stream.toList(), completion(expected));
This does not handle the case where the stream doesn't close at all (but then, nothing does, you just have to wait for a timeout).
It doesn't catch errors until all events have been emitted, the emitsInOrder approach is better for that. Not shorter, though.
emitsDone can be used if the Stream is closed at some point.
E.g:
test('Test', () async {
final controller = StreamController<String>();
final stream = controller.stream;
final matcher = expectLater(
stream,
emitsInOrder(<dynamic>[
'Item1',
'Item2',
emitsDone,
]),
);
controller
..add('Item1')
..add('Item2')
..add('Item3')
..close();
await matcher;
await controller.close();
});
The test fails with error:
Expected: should do the following in order:
• emit an event that 'Item1'
• emit an event that 'Item2'
• be done
Actual: <Instance of '_ControllerStream<String>'>
Which: emitted • Item1
• Item2
• Item3
x Stream closed.
which didn't be done
As #Irn suggest, a more compact alternative for Streams that complete at some point is using toList:
test('Test', () async {
final controller = StreamController<String>();
final stream = controller.stream;
final matcher = expectLater(stream.toList(), completion(<String>['Item1', 'Item2']));
controller
..add('Item1')
..add('Item2')
..add('Item3')
..close();
await matcher;
await controller.close();
});
If the Stream is never closed, you can add a timeout and check the items that have been emitted in that period:
test('Test3', () async {
final controller = StreamController<String>();
final stream = controller.stream.timeout(const Duration(milliseconds: 200));
final matcher = expectLater(
stream,
emitsInOrder(<dynamic>[
'Item1',
'Item2',
]),
);
controller
..add('Item1')
..add('Item2');
await matcher;
await controller.close();
});

How to await for the completion or error of an async download in Dart?

I'd like to write a multi-stage process linearly (as shown below) that starts with a file download with progress:
/// Processes the database, from download to prices processing.
Future<void> updateDatabase() async {
//final fontText = await File('./example/cosmic.flf').readAsString();
//print(art.renderFiglet('updateDatabase'));
// == 1) download
print('1 ======== DOWNLOADING ==========');
try {
await startDownloading();
} catch (err) {}
// == 2) Decompress: Whatever download was ok or not, we decompress the last downloaded zip file we have locally
print('2 ======== DECOMPRESSING ========');
try {
await startDecompressing();
} catch (err) {}
// == i) Stage i, etc.
But something does not work in my download stage as it starts stage 2) prior stage 1) completion.
My stage one (download) is like so:
/// Starts download procress
Future<void> startDownloading() async {
print("startDownloading…");
_state = DownloadState.downloading;
_progress = 0;
notifyListeners();
/// Database string url
final databaseUrlForInstantData = "https://XXX";
try {
final request = Request('GET', Uri.parse(databaseUrlForInstantData));
final StreamedResponse response = await Client().send(request);
final contentLength = response.contentLength;
// == Start from zero
_progress = 0;
notifyListeners();
/// The currently downloaded file as an array of bytes
List<int> bytes = [];
response.stream.listen(
/// = Closure listener for newly downloaded bytes
(List<int> newBytes) {
bytes.addAll(newBytes);
final downloadedLength = bytes.length;
if (contentLength == null) {
_progress = 0;
} else {
_progress = downloadedLength / contentLength;
}
notifyListeners();
print(
'Download in progress $_progress%: ${bytes.length} bytes so far');
},
/// = Download successfully completed
onDone: () async {
_state = DownloadState.downloaded;
notifyListeners();
/// The resulting local copy of the database
final file = await _getDownloadedZipFile();
// == Write to file
await file.writeAsBytes(bytes);
print('Download complete: ${bytes.length} bytes');
},
/// = Download error
onError: (e) {
_state = DownloadState.error;
_error = e.message;
print('Download error at $_progress%: $e');
},
cancelOnError: true,
);
}
// == Catches potential error
catch (e) {
_state = DownloadState.error;
_error = 'Could not download the databse: $e';
print('Download error at $_progress%: $e');
}
}
Your startDownloading function returns after it registers callbacks to listen to the Stream and does not wait for the Stream to complete.
To wait for the Stream to complete, you can save the StreamSubscription returned by .listen and then await the Future from StreamSubscription.asFuture:
var streamSubscription = response.stream.listen(...);
await streamSubscription.asFuture();

How to fix slow issue when using async and await for caching using Sqflite?

I store all API data to cache. some APIs have more than 10000 data. Postman response time is within one second. but in application very slow to navigate to next page. I used this code:
onPressed: () async {
...
}
else {
var token = Token(
id: 1,
token: tokens,
refreshToken: model.data.refreshToken,
);
await storeRegister(_url,tokens);
await storeEquipmentReg(_url,tokens);
await storeSyncLogin(_url,tokens);
await HelperDefCatMaster().deleteDefCatMaster();
await storeDefCatMaster(_url,tokens);
await HelperDefRegisterCat().deleteDefRegisterCat();
await storeDefRegisterCat(_url,tokens);
await HelperDefCatMaster().deleteDefCatRelation();
await storeDefCatRelation(_url,tokens);
await HelperDefCatMaster().deleteWoDescription();
await storeWoDescription(_url,tokens);
await HelperDefCatMaster().deleteCategoryDefect();
await storeCategoryDefect(_url,tokens);
await storeWorkSource(_url,tokens);
await storeWorkTypes(_url,tokens);
await storePriorities(_url,tokens);
await Helper().insert(token);
Navigator.push(
context,
MaterialPageRoute(
builder: (context) => ListPage(model.data.token)));
}
storePriorities function look like,
storePriorities(String url, String token) async {
final response = await http.get(
'${url}/v1.0/Priorities',
headers: {'Authorization': 'Bearer ${token}'},
);
final jsonResponse = json.decode(response.body);
Priorities model = Priorities.fromJson(jsonResponse);
int length = model.data.length;
for (int i = 0; i < length; i++) {
var data = DataPriorities(
i: model.data[i].i,
d: model.data[i].d,
);
await HelperDefCatMaster().insertPriorities(data);
}
}
I have given the first answer that suggests to use await only when it's needed.
Well if you are inserting too much data in SQLite I assume that you might be using something like this:
for (int i = 0; i <= 1000; i++) {
db.insert('table_name', dataObject.toMap());
}
Well this will do a lot many transactions at a time and it will consume a lot's of your time.
Change this to something like this and it will increase the speed of inserting data:
Batch batch = db.batch();
for (int i = 0; i <= 1000; i++) {
batch.insert('table_name', dataObject.toMap());
}
await batch.commit();
What we are doing here is that, in single transaction we are doing multiple inserts at a time.
I made this change in my demo project where I was inserting 1000 row at a time and results were great. db.insert when called 1000 times took 7 secs where as batch.insert took less than 1 sec for inserting same amount of data.
If you optimize your code with this solution and use await when needed you should not face any problem on UI. Let me know if this helps.
You are using await keyword to fetch data from SQLite.
And you are fetching a lots of data.
This will make data fetching synchronous, and will affect your UI.
If it is convenient for your use-case to fetch data asynchronously then you can use the following way:
Change :
await Helper().insert(token);
Navigator.push(
context,MaterialPageRoute(builder: (context) => ListPage(model.data.token)));
to :
Helper().insert(token).then((onValue) {
Navigator.push(context,MaterialPageRoute(
builder: (context) => ListPage(model.data.token),
),
);
}
Note: Make your insert method return Future<'token's return type'>
Now use this way for all other await calls.

Flutter image_picker post upload an image

I am using the Flutter Plugin Image_picker to choose images so that I want to upload image after selected the image
Future<File> _imageFile;
void _onImageButtonPressed(ImageSource source) async {
setState(() {
_imageFile = ImagePicker.pickImage(source: source);
});
}
I find this code in flutter documentation but its not work
var uri = Uri.parse("http://pub.dartlang.org/packages/create");
var request = new http.MultipartRequest("POST", url);
request.fields['user'] = 'nweiz#google.com';
request.files.add(new http.MultipartFile.fromFile(
'package',
new File('build/package.tar.gz'),
contentType: new MediaType('application', 'x-tar'));
request.send().then((response) {
if (response.statusCode == 200) print("Uploaded!");
});
Use MultipartRequest class
Upload(File imageFile) async {
var stream = new http.ByteStream(DelegatingStream.typed(imageFile.openRead()));
var length = await imageFile.length();
var uri = Uri.parse(uploadURL);
var request = new http.MultipartRequest("POST", uri);
var multipartFile = new http.MultipartFile('file', stream, length,
filename: basename(imageFile.path));
//contentType: new MediaType('image', 'png'));
request.files.add(multipartFile);
var response = await request.send();
print(response.statusCode);
response.stream.transform(utf8.decoder).listen((value) {
print(value);
});
}
Check this answer
This code works properly.
Used MultipartRequest class
void uploadImage() async {
File _image;
File pickedImage = await ImagePicker.pickImage(source: ImageSource.camera);
setState(() {
_image = pickedImage;
});
// open a byteStream
var stream = new http.ByteStream(DelegatingStream.typed(_image.openRead()));
// get file length
var length = await _image.length();
// string to uri
var uri = Uri.parse("enter here upload URL");
// create multipart request
var request = new http.MultipartRequest("POST", uri);
// if you need more parameters to parse, add those like this. i added "user_id". here this "user_id" is a key of the API request
request.fields["user_id"] = "text";
// multipart that takes file.. here this "image_file" is a key of the API request
var multipartFile = new http.MultipartFile('image_file', stream, length, filename: basename(_image.path));
// add file to multipart
request.files.add(multipartFile);
// send request to upload image
await request.send().then((response) async {
// listen for response
response.stream.transform(utf8.decoder).listen((value) {
print(value);
});
}).catchError((e) {
print(e);
});
}
name spaces:
import 'package:path/path.dart';
import 'package:async/async.dart';
import 'dart:io';
import 'package:http/http.dart' as http;
If you want the uploading function to return the server response, you can use toBytes() instead of transform(), in order to wait until data transmission is complete.
Future<String> upload() async {
String responseString = '';
// Pick image
final image = await ImagePicker().getImage(
source: ImageSource.gallery // or ImageSource.camera
imageQuality: 100,
maxWidth: 1000,
);
// Convert to File
final file = File(image.path);
// Set URI
final uri = Uri.parse('URL');
// Set the name of file parameter
final parameter = 'Name';
// Upload
final request = http.MultipartRequest('POST', uri)
..files.add(await http.MultipartFile.fromPath(parameter, file.path));
final response = await request.send();
if (response.statusCode == 200) {
responseString = String.fromCharCodes(await response.stream.toBytes());
}
return responseString;
}

Resources