Reusable Sync HttpClient Request - dart

I need a reusable function that makes an HTTP request and awaits for its completion before returning the response as a String.
Here's the main function:
main() async {
var json;
json = await makeRequest('https://...');
print(json);
print('*** request complete ***');
}
(First Case) This is the reusable function that makes the HTTP request:
makeRequest(String url) async {
var request = await new HttpClient().postUrl(Uri.parse(url));
// Includes the access token in the request headers.
request.headers.add(...);
// Waits until the request is complete.
var response = await request.close();
await for (var contents in response.transform(UTF8.decoder)) {
return contents;
}
}
This works as expected and the output is:
// Response contents as a String...
*** request complete ***
(Second Case) Then I tried to do this and it didn't work:
makeRequest(String url) async {
var request = await new HttpClient().postUrl(Uri.parse(url));
// Includes the access token in the request headers.
request.headers.add(...);
// Waits until the request is complete.
var response = await request.close();
var json = '';
await response.transform(UTF8.decoder).listen((contents) {
// At first I tried to return contents here, but then I added onDone().
json += contents;
}, onDone: () {
return json;
});
return json;
}
I've tried defining the function within listen with async and await, returning contents within listen without onDone(), but the output is the same:
// Empty line.
*** request complete ***
// Waits a few seconds doing nothing before terminating...
Does anyone know why the second case doesn't work?
EDIT:
After updating the code it does what it was supposed to do, but takes a few seconds before terminating execution:
Future<String> twitterRequest(String url) async {
var request = await new HttpClient().postUrl(Uri.parse(url));
// Includes the access token in the request headers.
request.headers.add(...);
// Waits until the request is complete.
var response = await request.close();
var json = '';
await for (var contents in response.transform(UTF8.decoder)) {
json += contents;
// Putting a break here produces the same output but terminates immediately (as wanted).
}
return json;
}
Output:
// Prints response contents...
*** request complete ***
// Takes a few seconds before execution terminates. With the break the code terminates immediately.
EDIT2:
After submitting this issue on GitHub, I found out that instances of the HttpClient have a connection pool and keep persistent connections by default, which keeps the Dart VM alive. Please consult the issue page to find out about the possible solutions.

It's probably caused by the await before the response.transform.
You might want something like
return response.transform(UTF8.decoder).join('');
The pause is not related to makeRequest(). The Dart VM seems to wait for something before it exits. Adding exit(0); as last line in main() makes the application exit immediately.
Update
According to the response on the Dart SDK issue
This is caused by the HttpClient instance having a connection pool
which can keep the Dart VM alive. There are two ways of avoiding this:
1) Close the HttpClient explicitly
2) Use non-persistent connections
import 'dart:async';
import 'dart:convert' show UTF8;
import 'dart:io';
Future main() async {
await makeRequest();
print('end of main');
}
Future makeRequest() async {
var client = new HttpClient();
var request = await client.postUrl(Uri.parse('https://example.com'));
var response = await request.close();
var contents = await response.transform(UTF8.decoder).join();
print(contents);
client.close(); // Close the client.
}
import 'dart:async';
import 'dart:convert' show UTF8;
import 'dart:io';
Future main() async {
await makeRequest();
print('end of main');
}
Future makeRequest() async {
var request = await new HttpClient().postUrl(Uri.parse('https://example.com'));
request.persistentConnection = false; // Use non-persistent connection.
var response = await request.close();
var contents = await response.transform(UTF8.decoder).join();
print(contents);
}

Related

How to log the response for a dart shelf request

I'm using the Dart Shelf package and I need to log the response it sends.
I've managed to log the request but the response technique is less clear:
final handler = const shelf.Pipeline()
.addMiddleware(corsHeaders())
.addMiddleware(shelf.logRequests(
logger: (message, isError) =>
_logRequest(message, isError: isError)))
.addHandler((req) async {
final res = await Router().call(req);
return res;
});
There two parts to the question.
how do I log the headers.
is it possible to log the body.
I know there is an issue in that the response body can only be read once.
As some of the responses are likely to be large I need to filter the requests for which the body is logged.
The answer is a bit of Dart-fu. You have an anonymous function returning an anonymous function.
var handler = const Pipeline()
.addMiddleware(
(handler) => (request) async {
final response = await handler(request);
print(response.headers);
// you could read the body here, but you'd also need to
// save the content and pipe it into a new response instance
return response;
},
)
.addHandler(syncHandler);

Wrong order of execution async handler

I have small app that execute requests to external service.
final app = Alfred();
app.post('/ctr', (req, res) async { // complex tender request
var data = await req.body;
await complexTendexAPIRequest(data as Map<String, dynamic>);
print('Hello World');
await res.json({'data': 'ok'});
});
Handler code:
complexTendexAPIRequest(Map<String, dynamic> data) async {
print('Request: $data');
try {
final response = await http.post(
Uri.parse(COMPLEX_URL),
headers: {'Content-Type': 'application/json', 'Authorization': 'bearer $ACCESS_TOKEN'},
body: json.encode(data)
);
if(response.statusCode == 200) {
var res = json.decode(response.body);
int latestId = res['id'];
String url = 'https://api.ru/v2/complex/status?id=$latestId';
stdout.write('Waiting for "complete" status from API: ');
Timer.periodic(Duration(seconds: 1), (timer) async {
final response = await http.get(
Uri.parse(url),
headers: {'Content-Type': 'application/json', 'Authorization': 'bearer $ACCESS_TOKEN'}
);
var data = json.decode(response.body);
if(data['status'] == 'completed') {
timer.cancel();
stdout.write('[DONE]');
stdout.write('\nFetching result: ');
String url = "https://api.ru/v2/complex/results?id=$latestId";
final response = await http.get(
Uri.parse(url),
headers: {'Content-Type': 'application/json', 'Authorization': 'bearer $ACCESS_TOKEN'}
);
stdout.write('[DONE]');
var data = prettyJson(json.decode(response.body));
await File('result.json').writeAsString(data.toString());
print("\nCreating dump of result: [DONE]");
}
});
}
else {
print('[ERROR] Wrong status code for complex request. StatusCode: ${response.statusCode}');
}
}
on SocketException catch(e) {
print('No Internet connection: $e');
} on TimeoutException catch(e) {
print('TenderAPI Timeout: $e');
} on Exception catch(e) {
print('Some unknown Exception: $e');
}
}
But output is very strange it's look like it's do not waiting complexTendexAPIRequest completion and go forward:
Waiting for "complete" status from API: Hello World
[DONE]
Fetching result: [DONE]
Creating dump of result: [DONE]
But should be:
Waiting for "complete" status from API: [DONE]
Fetching result: [DONE]
Creating dump of result: [DONE]
Hello World
I suppose that reason can be in Timer.periodic but how to fix it to get expected order and execution of:
print('Hello World');
await res.json({'data': 'ok'});
only after complexTendexAPIRequest completed.
upd: I rewrote code to while loop:
https://gist.github.com/bubnenkoff/fd6b4f0d7aeae7007680e7902fbdc1e9
it's seems that it's ok.
Alfred https://github.com/rknell/alfred
The problem is the Timer.periodic, as others have pointed out.
You do:
Timer.periodic(Duration(seconds: 1), (timer) async {
// do something ...
});
That sets up a timer, then immediately continues execution.
The timer triggers every second, calls the async callback (which returns a future that no-one ever waits for) and which does something which just might take longer than a second.
You can convert this to a normal loop, basically:
while (true) {
// do something ...
if (data['status'] == 'completed') {
// ...
break;
} else {
// You can choose your own delay here, doesn't have
// to be the same one every time.
await Future.delayed(const Duration(seconds: 1));
}
}
If you still want it to be timer driven, with fixed ticks, consider rewriting this as:
await for (var _ in Stream.periodic(const Duration(seconds: 1))) {
// do something ...
// Change `timer.cancel();` to a `break;` at the end of the block.
}
Here you create a stream which fires an event every second. Then you use an await for loop to wait for each steam event. If the thing you do inside the loop is asynchronous (does an await) then you are even ensured that the next stream event is delayed until the loop body is done, so you won't have two fetches running at the same time.
And if the code throws, the error will be caught by the surrounding try/catch, which I assume was intended, rather than being an uncaught error ending up in the Future that no-one listens to.
If you want to retain the Timer.periodic code, you can, but you need to do something extra to synchronize it with the async/await code around it (which only really understands futures and streams, not timers).
For example:
var timerDone = Completer();
Timer.periodic(const Duration(seconds: 1), (timer) async {
try {
// do something ...
// Add `timerDone.complete();` next to `timer.cancel()`.
} catch (e, s) {
timer.cancel();
timerDone.completeError(e, s);
}
});
await timerDone.future;
This code uses a Completer to complete a future and effectively bridge the gap between timers and futures that can be awaited (one of the listed uses of Completer in the documentation).
You may still risk the timer running concurrently if one step takes longer than a second.
Also, you can possibly use the retry package, if it's the same thing you want to retry until it works.

How to set up tests for a Dart HTTP server

I'm trying to build a Dart HTTP server and I want to test the API. I'm not able to set up the tests, though.
Here is what I have so far in my_server_test.dart:
import 'dart:io';
import 'package:my_server/my_server.dart';
import 'package:test/test.dart';
void main() {
HttpServer server;
setUp(() async {
final server = await createServer();
await handleRequests(server);
});
tearDown(() async {
await server.close(force: true);
server = null;
});
test('First try', () async {
final client = HttpClient();
final request = await client.get(InternetAddress.loopbackIPv4.host, 4040, '/');
final response = await request.close();
print(response);
});
}
And here is the server code in my_server.dart:
import 'dart:io';
import 'package:hundetgel_server/routes/handle_get.dart';
Future<HttpServer> createServer() async {
final address = InternetAddress.loopbackIPv4;
const port = 4040;
return await HttpServer.bind(address, port);
}
Future<void> handleRequests(HttpServer server) async {
await for (HttpRequest request in server) {
switch (request.method) {
case 'GET':
handleGet(request);
break;
default:
handleDefault(request);
}
}
}
void handleGet(HttpRequest request) {
request.response
..write('Hello')
..close();
}
void handleDefault(HttpRequest request) {
request.response
..statusCode = HttpStatus.methodNotAllowed
..write('Unsupported request: ${request.method}.')
..close();
}
When I run the test I just get a timeout:
TimeoutException after 0:00:30.000000: Test timed out after 30 seconds. See https://pub.dev/packages/test#timeouts
dart:isolate _RawReceivePortImpl._handleMessage
NoSuchMethodError: The method 'close' was called on null.
Receiver: null
Tried calling: close(force: true)
dart:core Object.noSuchMethod
2
main.<fn>
test/my_server_test.dart:15
===== asynchronous gap ===========================
dart:async _completeOnAsyncError
test/my_server_test.dart main.<fn>
test/my_server_test.dart:1
main.<fn>
test/my_server_test.dart:14
2
✖ First try
Exited (1)
How do I set up the server so I can start testing it?
It seems that you are await handleRequests in your setUp. Your handleRequests is a forever loop waiting for incoming requests. So it never halts. So you setup never finishes. That is the problem.
Thus, try to change
await handleRequests(server);
to
handleRequests(server); // NO await
Thus the handleRequests will run in the "background".
EDIT about the null exception of server variable:
change
final server = await createServer();
to
server = await createServer();
because the old code shadows the outside server variable - that variable is never assigned a value.

Server response with output from Future Object

i created a async/await function in another file thus its handler is returning a Future Object. Now i can't understand how to give response to client with content of that Future Object in Dart. I am using basic dart server with shelf package.Below is code where ht.handler('list') returns a Future Object and i want to send that string to client as response. But i am getting internal server error.
import 'dart:io';
import 'package:args/args.dart';
import 'package:shelf/shelf.dart' as shelf;
import 'package:shelf/shelf_io.dart' as io;
import 'HallTicket.dart' as ht;
// For Google Cloud Run, set _hostname to '0.0.0.0'.
const _hostname = 'localhost';
main(List<String> args) async {
var parser = ArgParser()..addOption('port', abbr: 'p');
var result = parser.parse(args);
// For Google Cloud Run, we respect the PORT environment variable
var portStr = result['port'] ?? Platform.environment['PORT'] ?? '8080';
var port = int.tryParse(portStr);
if (port == null) {
stdout.writeln('Could not parse port value "$portStr" into a number.');
// 64: command line usage error
exitCode = 64;
return;
}
var handler = const shelf.Pipeline()
.addMiddleware(shelf.logRequests())
.addHandler(_echoRequest);
var server = await io.serve(handler, _hostname, port);
print('Serving at http://${server.address.host}:${server.port}');
}
Future<shelf.Response> _echoRequest(shelf.Request request)async{
shelf.Response.ok('Request for "${request.url}"\n'+await ht.handler('list'));
}
The analyzer gives your the following warning for your _echoRequest method:
info: This function has a return type of 'Future', but
doesn't end with a return statement.
And if you check the requirement for addHandler you will see it expects a handler to be returned.
So you need to add the return which makes it work on my machine:
Future<shelf.Response> _echoRequest(shelf.Request request) async {
return shelf.Response.ok(
'Request for "${request.url}"\n' + await ht.handler('list2'),
headers: {'Content-Type': 'text/html'});
}

Dart: Processing an uploaded file on the server from client request

I am trying to process images uploaded from client on the server but am receiving the following errors. Am I processing the httpRequest wrong?
Unhandled exception:
Uncaught Error: HttpException: Connection closed while receiving data, uri = /api/upload
/// Client Code (dart file) (Works # sending request)
sendData(dynamic data) {
final req = new HttpRequest();
FormData fd = new FormData();
fd.append('uploadContent', data);
fd.appendBlob('uploadContent', data);
req.open("POST", "http://127.0.0.1:8080/api/upload", async: true);
req.send(fd);
req.onReadyStateChange.listen((Event e) {
if (req.readyState == HttpRequest.DONE &&
(req.status == 200 || req.status == 0)) {
window.alert("upload complete");
}
});
}
InputElement uploadInput = document.querySelector('#sideBar-profile-picture');
uploadInput.onChange.listen((Event event){
// read file content as dataURL
final files = uploadInput.files;
if (files.length == 1) {
File file = files[0];
FileReader reader = new FileReader();
reader.onLoad.listen((e) {
print('results: ${reader.result}');
sendData(reader.result);
});
reader.readAsArrayBuffer(file);
}
});
I have a small server listening for the request (/api/upload) and calling handleUploadRequest with the httpRequest being passed in as the param.
Server code (This is where I am stuck)
Future<Null> handleUploadRequest(final HttpRequest httpRequest) async {
httpRequest.fold(new BytesBuilder(), (b, d) => b..add(d)).then((builder) {
var data = builder.takeBytes();
print('bytes builder: ${data}');
});
}
I am trying to read the data so that I can store it on a cdn but never get a chance to since the connection always gets closed while receiving the data.
Any help on being able to complete this is appreciated. Been at this for the past couple days:/
It is hard to tell when/if you close the httpRequest. If you are doing it right after handleUploadRequest returns, it will indeed close the connection as you are not waiting for httpRequest.fold() to complete. Adding await as shown below and making sure to call httpRequest.close() after this function complete asynchronously should work
Future<Null> handleUploadRequest(final HttpRequest httpRequest) async {
await httpRequest.fold(new BytesBuilder(), (b, d) => b..add(d)).then((builder) {
var data = builder.takeBytes();
print('bytes builder: ${data}');
});
}
(FYI) I have a similar code that works when testing with curl and uploading a file

Resources