I made a simple web server but it crashes every time I am refreshing page many times in a short time. I just enter 127.0.0.1:8080 in my browser and then spam with F5. Here is the code to reproduce this issue:
void main()
{
HttpServer server = new HttpServer();
server.addRequestHandler((req) => true, handleGET);
server.listen('127.0.0.1', 8080);
}
void handleGET(HttpRequest req, HttpResponse res)
{
var requestedFile = ".${req.path}";
if(req.path == "/")
{
requestedFile = requestedFile.concat("index.html");
}
File file = new File(requestedFile);
file.exists().then((bool found) {
if(found)
{
file.openInputStream().pipe(res.outputStream);
}
else
{
res.statusCode = HttpStatus.NOT_FOUND;
res.outputStream.close();
}
});
}
The error I get is following:
Unhandled exception:
StreamException: Stream closed
#0 _SocketOutputStream._write (dart:io:6017:30)
#1 _HttpResponse._writeHeader (dart:io:5981:18)
#2 _HttpRequestResponseBase._ensureHeadersSent (dart:io:2696:19)
#3 _HttpResponse._streamClose (dart:io:2921:23)
#4 _HttpOutputStream.close (dart:io:3078:36)
#5 _pipe.<anonymous closure> (dart:io:6271:28)
#6 _BaseDataInputStream._checkScheduleCallbacks.issueCloseCallback (dart:io:6231:59)
#7 _Timer._createTimerHandler._handleTimeout (dart:io:6804:28)
#8 _Timer._createTimerHandler._handleTimeout (dart:io:6812:7)
#9 _Timer._createTimerHandler.<anonymous closure> (dart:io:6820:23)
#10 _ReceivePortImpl._handleMessage (dart:isolate-patch:37:92)
Often before this mayor exception I receive a bunch of warnings like WSASend failed: 10053 but those don't crash the server. I work on Windows if this problem is related to some specific implementation.
Because you are hitting reload very quickly, your code ends up trying to write to a socket that has already been closed. Hence, you should probably catch StreamException and just ignore it. It could be argued that the io library should help you out a bit more. I just filed this bug:
http://code.google.com/p/dart/issues/detail?id=7334&thanks=7334&ts=1355280746
Related
I have downloaded a Wikipedia dump and I am trying to read it line by line. But when doing the utf8-decode I get the following error
12633: FormatException: Unfinished UTF-8 octet sequence (at offset 65536)
Stacktrace :#0 _Utf8Decoder.convertSingle (dart:convert-patch/convert_patch.dart:1789:7)
#1 Utf8Decoder.convert (dart:convert/utf.dart:351:42)
#2 Utf8Codec.decode (dart:convert/utf.dart:63:20)
#3 _MapStream._handleData (dart:async/stream_pipe.dart:213:31)
#4 _ForwardingStreamSubscription._handleData (dart:async/stream_pipe.dart:153:13)
#5 _RootZone.runUnaryGuarded (dart:async/zone.dart:1618:10)
#6 _BufferingStreamSubscription._sendData (dart:async/stream_impl.dart:341:11)
#7 _BufferingStreamSubscription._add (dart:async/stream_impl.dart:271:7)
#8 _SyncStreamControllerDispatch._sendData (dart:async/stream_controller.dart:774:19)
#9 _StreamController._add (dart:async/stream_controller.dart:648:7)
#10 _StreamController.add (dart:async/stream_controller.dart:596:5)
#11 _FileStream._readBlock.<anonymous closure> (dart:io/file_impl.dart:98:19)
<asynchronous suspension>
That is this line
ar جزر_غالاباغوس 1 0
So I tried saving the file utf-8 encoded with this button
But that does not seem to work
This is my code
final filePath = p.join(
Directory.current.path,
'bin\\migrate_most_views\\data\\pageviews-20220416-170000',
);
final file = File(filePath);
logger.stderr('exporting pageviews...');
StreamSubscription? reader;
int lineNumer = 0;
reader = file.openRead().map(utf8.decode).transform(LineSplitter()).listen(
(line) {
final page = MostViewedPageDaily.fromLine(line);
db.collection('page_views').insert(page.toMap());
lineNumer++;
if (lineNumer % 1000 == 0) {
logger.stdout('inserting at line $lineNumer');
}
},
onDone: () {
logger.stdout('Reader read $lineNumer lines');
reader?.cancel();
exit(0);
},
onError: (error, stackTrace) {
final message = '$lineNumer: $error\n\nStacktrace :$stackTrace';
logger.stdout(logger.ansi.error(message));
exit(1);
},
cancelOnError: true,
);
What can I do?
I downloaded the file from here
https://dumps.wikimedia.org/other/pageviews/2022/2022-04/pageviews-20220417-010000.gz
You should use file.openRead().transform(utf8.decoder) instead of file.openRead().map(utf8.decode). (Also note the argument difference: utf8.decoder is a Utf8Decoder object, and utf8.decode is a method tear-off.)
The Stream.map documentation specifically discusses this:
Unlike transform, this method does not treat the stream as chunks of a single value. Instead each event is converted independently of the previous and following events, which may not always be correct. For example, UTF-8 encoding, or decoding, will give wrong results if a surrogate pair, or a multibyte UTF-8 encoding, is split into separate events, and those events are attempted encoded or decoded independently.
I am trying to perform a widget test, specifically navigation test. I am using bloc architecture, setting a stream on the bloc triggers a series of events inside the bloc, gets session info from the server call (which returns a future of session info object), on successful server call a login stream is set and the widget has a stream subscription to this stream and navigates to the next screen.
I am using mockito to mock the server call and stubbing the server call to return a future of success response. The problem is the when I am calling pumpAndSettle() it is getting timed out as it is not waiting for the future to complete and return the success response.
I apologize if I am not making it very clear, but here is the sample code:
login_bloc.dart
class LoginBloc {
LoginRepository _loginRepository;
final String searchKeyword = "special-keyword";
final _urlString = PublishSubject<String>();
final _isLoggedIn = BehaviorSubject<bool>(seedValue: false);
final _errorMessage = PublishSubject<String>();
Observable<bool> get isLoggedIn => _isLoggedIn.stream;
Observable<String> get isErrorState => _errorMessage.stream;
LoginBloc({LoginRepository loginRepository})
: _loginRepository = loginRepository ?? LoginRepository() {
// Listen on the _urlString stream to call the function which checks for the special keyword and if a match is found make a server call
_urlString.stream.listen((String url) {
_authorizationFullService(url);
});
}
// Search for special keyword and if a match is found call the server call function
void _authorizationFullService(String url) {
if (url.contains(searchKeyword)) {
int index = url.indexOf(searchKeyword);
String result = url.substring(index + searchKeyword.length);
result = result.trim();
String decodedUrl = Uri.decodeFull(result);
if (decodedUrl != null && decodedUrl.length > 0) {
_fullServiceServerCall(decodedUrl);
} else {
_isLoggedIn.sink.add(false);
}
}
}
// Call server call function from repository which returns a future of the Authorization object
void _fullServiceServerCall(String decodedUrl) {
_loginRepository
.getSession(decodedUrl)
.then(_handleSuccessAuthorization)
.catchError(_handleErrorState);
}
// Handle success response and set the login stream
void _handleSuccessAuthorization(Authorization authorization) {
if (authorization != null && authorization.idnumber != 0) {
_isLoggedIn.sink.add(true);
} else {
_isLoggedIn.sink.add(false);
}
}
// Handle error response and set the error stream
void _handleErrorState(dynamic error) {
_isLoggedIn.sink.add(false);
_errorMessage.sink.add(error.toString());
}
void dispose() {
_urlString.close();
_isLoggedIn.close();
_errorMessage.close();
}
}
widget_test.dart
group('Full Login Navigation test', () {
LoginRepository mockLoginRepository;
LoginBloc loginBloc;
NotificationBloc notificationBloc;
NavigatorObserver mockNavigatorObserver;
Authorization _auth;
String testUrl;
setUp(() {
mockLoginRepository = MockLoginRepository();
_auth = Authorization((auth) => auth
..param1 = "foo"
..param2 = "bar"
..param3 = "foobar"
..param4 = "barfoo");
loginBloc = LoginBloc(loginRepository: mockLoginRepository);
mockNavigatorObserver = MockNavigatorObserver();
testUrl = "http://test.test.com";
});
Future<Null> _buildFullLoginPage(LoginBloc loginBloc,
NotificationBloc notificationBloc, WidgetTester tester) async {
when(mockLoginRepository.getSession(testUrl))
.thenAnswer((_) => Future.value(_auth));
await tester.pumpWidget(LoginBlocProvider(
child: NotificationBlocProvider(
child: MaterialApp(
home: LoginFullService(),
onGenerateRoute: NavigationRoutes.routes,
navigatorObservers: [mockNavigatorObserver],
),
notificationBloc: notificationBloc,
),
loginBloc: loginBloc,
));
//TODO: Remove casting to dynamic after dart sdk bug fix: https://github.com/dart-lang/mockito/issues/163
verify(mockNavigatorObserver.didPush(any, any) as dynamic);
loginBloc.getAuthorization(
"http://testing.testing.com?search-keyword=http%3A%2F%2Ftest.test.com");
}
testWidgets('Navigate to landing page on correct login url',
(WidgetTester tester) async {
await _buildFullLoginPage(loginBloc, notificationBloc, tester);
await tester.pumpAndSettle();
expect(find.byKey(Key('webview_scaffold')), findsNothing);
//TODO: Remove casting to dynamic after dart sdk bug fix: https://github.com/dart-lang/mockito/issues/163
verify(mockNavigatorObserver.didPush(any, any) as dynamic);
});
});
On running the widget test the tester.pumpAndSettle() inside testWidgets times out before the future is completed. This is the error log:
══╡ EXCEPTION CAUGHT BY FLUTTER TEST FRAMEWORK ╞════════════════════════════════════════════════════
The following assertion was thrown running a test:
pumpAndSettle timed out
When the exception was thrown, this was the stack:
#0 WidgetTester.pumpAndSettle.<anonymous closure> (package:flutter_test/src/widget_tester.dart:299:11)
<asynchronous suspension>
#3 TestAsyncUtils.guard (package:flutter_test/src/test_async_utils.dart:69:41)
#4 WidgetTester.pumpAndSettle (package:flutter_test/src/widget_tester.dart:295:27)
#5 main.<anonymous closure>.<anonymous closure> (file:///Users/ssiddh/Documents/projects/mobile-flutter/test/ui/pages/login/login_full_test.dart:114:20)
<asynchronous suspension>
#6 testWidgets.<anonymous closure>.<anonymous closure> (package:flutter_test/src/widget_tester.dart:72:23)
#7 TestWidgetsFlutterBinding._runTestBody (package:flutter_test/src/binding.dart:555:19)
<asynchronous suspension>
#10 TestWidgetsFlutterBinding._runTest (package:flutter_test/src/binding.dart:539:14)
#11 AutomatedTestWidgetsFlutterBinding.runTest.<anonymous closure> (package:flutter_test/src/binding.dart:883:24)
#17 AutomatedTestWidgetsFlutterBinding.runTest (package:flutter_test/src/binding.dart:880:15)
#18 testWidgets.<anonymous closure> (package:flutter_test/src/widget_tester.dart:71:22)
#19 Declarer.test.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:168:27)
<asynchronous suspension>
#20 Invoker.waitForOutstandingCallbacks.<anonymous closure> (package:test/src/backend/invoker.dart:249:15)
<asynchronous suspension>
#25 Invoker.waitForOutstandingCallbacks (package:test/src/backend/invoker.dart:246:5)
#26 Declarer.test.<anonymous closure>.<anonymous closure> (package:test/src/backend/declarer.dart:166:33)
#31 Declarer.test.<anonymous closure> (package:test/src/backend/declarer.dart:165:13)
<asynchronous suspension>
#32 Invoker._onRun.<anonymous closure>.<anonymous closure>.<anonymous closure>.<anonymous closure> (package:test/src/backend/invoker.dart:403:25)
<asynchronous suspension>
#46 _Timer._runTimers (dart:isolate/runtime/libtimer_impl.dart:382:19)
#47 _Timer._handleMessage (dart:isolate/runtime/libtimer_impl.dart:416:5)
#48 _RawReceivePortImpl._handleMessage (dart:isolate/runtime/libisolate_patch.dart:169:12)
(elided 30 frames from class _FakeAsync, package dart:async, and package stack_trace)
I would really appreciate any kind of help or feedback.
Try wrapping your test with
testWidgets('Navigate to landing page on correct login url',
(WidgetTester tester) async {
await tester.runAsync(() async {
// test code here
});
});
I am getting this exception when I close the pool very soon after closing a query:
Uncaught Error: Bad state: Cannot write to socket, it is closed
Stack Trace:
#0 BufferedSocket.writeBufferPart (package:sqljocky/src/buffered_socket.dart:114:7)
#1 BufferedSocket.writeBuffer (package:sqljocky/src/buffered_socket.dart:108:27)
#2 _Connection._sendBufferPart (package:sqljocky/src/connection.dart:261:31)
#3 _Connection._sendBuffer (package:sqljocky/src/connection.dart:249:29)
#4 _Connection.processHandler (package:sqljocky/src/connection.dart:289:16)
#5 ConnectionPool._closeQuery.<anonymous closure> (package:sqljocky/src/connection_pool.dart:220:29)
#6 _rootRunUnary (dart:async/zone.dart:730)
#7 _RootZone.runUnary (dart:async/zone.dart:864)
#8 _Future._propagateToListeners.handleValueCallback (dart:async/future_impl.dart:488)
#9 _Future._propagateToListeners (dart:async/future_impl.dart:571)
#10 _Future._completeWithValue (dart:async/future_impl.dart:331)
#11 _Future._asyncComplete.<anonymous closure> (dart:async/future_impl.dart:393)
#12 _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)
#13 _asyncRunCallback (dart:async/schedule_microtask.dart:32)
#14 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:128)
Unhandled exception:
Bad state: Cannot write to socket, it is closed
#0 _rootHandleUncaughtError.<anonymous closure>.<anonymous closure> (dart:async/zone.dart:713)
#1 _asyncRunCallbackLoop (dart:async/schedule_microtask.dart:23)
#2 _asyncRunCallback (dart:async/schedule_microtask.dart:32)
#3 _asyncRunCallback (dart:async/schedule_microtask.dart:36)
#4 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:128)
the issue seems to be that the query close fires of a Future internally, so the close() function returns before the close is actually finished:
void _closeQuery(Query q, bool retain) {
_log.finest("Closing query: ${q.sql}");
for (var cnx in _pool) {
var preparedQuery = cnx.removePreparedQueryFromCache(q.sql);
if (preparedQuery != null) {
_waitUntilReady(cnx).then((_) {
_log.finest("Connection ready - closing query: ${q.sql}");
var handler = new _CloseStatementHandler(preparedQuery.statementHandlerId);
cnx.autoRelease = !retain;
cnx.processHandler(handler, noResponse: true);
});
}
}
}
The pool close happens immediately, it closes the socket right away. This means the query close (which is delayed till after the pool close due to the Future) fails, unable to send whatever information it needs to through the socket. I've opened a ticket to sqljocky at https://github.com/jamesots/sqljocky/issues/44 but I've received no replies, and I need a workaround if it's going to take a while to get a response.
This code has allowed me to replicate the issue 100% of the time:
Future _putMethod(RestRequest request) {
return new Future.sync(() {
mysql.ConnectionPool pool = getConnectionPool();
return pool.prepare("SELECT * FROM files").then((mysql.Query query) {
return query.execute().then((result) {
// Do something?
}).then((_) {
this._log.info("Closing");
query.close();
});
}).then((_) {
pool.close();
});
});
}
This is yet more a question than an answer but I can't put this code in a comment in a usable way.
You should ensure that you return the Future returned from every async invocation.
I don't know if the lines where I added the comment // added return are async invocations.
Can you please try and give feedback if this changes anything.
Future _putMethod(RestRequest request) {
return new Future.sync(() {
mysql.ConnectionPool pool = getConnectionPool();
return pool.prepare("SELECT * FROM files").then((mysql.Query query) {
return query.execute().then((result) {
// Do something? // also ensure that a Future of an async invocation is returned
}).then((_) {
this._log.info("Closing");
return query.close(); // added return
});
}).then((_) {
return pool.close(); // added return
});
});
}
I've modified little bit example from tutorial https://www.dartlang.org/docs/tutorials/streams/ by adding item after subscription:
import 'dart:async';
main() {
var data = new List<int>();
var stream = new Stream.fromIterable(data); // create the stream
// subscribe to the streams events
stream.listen((value) { //
print("Received: $value"); // onData handler
}); //
data.add(1);
}
And after running this program I've got:
Uncaught Error: Concurrent modification during iteration: _GrowableList len:1.
Stack Trace:
#0 ListIterator.moveNext (dart:_collection-dev/iterable.dart:315)
#1 _IterablePendingEvents.handleNext (dart:async/stream_impl.dart:532)
#2 _PendingEvents.schedule.<anonymous closure> (dart:async/stream_impl.dart:661)
#3 _asyncRunCallback (dart:async/schedule_microtask.dart:18)
#4 _createTimer.<anonymous closure> (dart:async-patch/timer_patch.dart:11)
#5 _Timer._createTimerHandler._handleTimeout (timer_impl.dart:151)
#6 _Timer._createTimerHandler.<anonymous closure> (timer_impl.dart:166)
#7 _RawReceivePortImpl._handleMessage (dart:isolate-patch/isolate_patch.dart:93)
Putting data.add(1) before adding listener works as expected.
I've checked documentation about Stream and didn't found what I am doing wrong. I was expecting that listener will be fired in best case and just not fired in worst case, but not exception.
Is it expected behavior? If yes, please describe why.
The exception comes from you trying to modify the list while it is iterated over. This is unspecified behaviour in Dart (*), and the used implementation simply chose to throw an exception. While it is obfuscated by the asynchronous stuff happening in Stream.fromIterable, it basically is the same as if you tried to do this:
var data = [1,2,3];
for(var d in data) {
print(d);
data.add(d+10);
}
If you wrapped your data.add in another async call, for example with Timer.run(() => data.add(2)), it would "work". By that, I mean it wouldn't throw an exception.
Received: 2 still would not be printed. The stream will only send the elements that where already in the list at the time new Stream.fromIterable was called. After that, the stream is closed (onDone will be called), and modifications to the original list will not be sent to your listener.
(*) Source: iterator.dart in SDK 1.1.3 -- "If the object iterated over is changed during the iteration, the behavior is unspecified." Why the text on api.dartlang.org is different is beyond me.
EDIT
To answer the question in the comment: One way would be to use a StreamController.
// or new StreamController<int>.broadcast(), if you want to listen to the stream more than once
StreamController s = new StreamController<int>();
// produce periodic errors
new Timer.periodic(new Duration(seconds: 5), (Timer t) {
s.isClosed ? t.cancel() : s.addError("I AM ERROR");
});
// add some elements before subscribing
s.add(6);
s.add(9);
// this will close the stream eventually
new Timer(new Duration(seconds: 20), () => s.close());
// start listening to the stream
s.stream.listen((v) => print(v),
onError: (err) => print("An error occured: $err"),
onDone: () => print("The stream was closed"));
// add another element before the next event loop iteration
Timer.run(() => s.add(4711));
// periodically add an element
new Timer.periodic(new Duration(seconds: 3), (Timer t) {
s.isClosed ? t.cancel() : s.add(0);
});
// one more (will be sent before 4711)
s.add(4);
The List can't be modified while it is iterated over.
You need an iterable that doesn't have this limitation (e.g. custom implementation) for your example.
I've got a technical problem while trying to consume a restful web service on the Stream server.
I use HTTPClient.openUrl to retrieve a JSON response from another remote server but once the connection is opened, I can no longer write response(connect.response.write) to my browser client.
The error is listed as following:
Unhandled exception:
Bad state: StreamSink is closed
#0 _FutureImpl._scheduleUnhandledError.<anonymous closure> (dart:async:325:9)
#1 Timer.run.<anonymous closure> (dart:async:2251:21)
#2 Timer.run.<anonymous closure> (dart:async:2259:13)
#3 Timer.Timer.<anonymous closure> (dart:async-patch:15:15)
#4 _Timer._createTimerHandler._handleTimeout (dart:io:6730:28)
#5 _Timer._createTimerHandler._handleTimeout (dart:io:6738:7)
#6 _Timer._createTimerHandler.<anonymous closure> (dart:io:6746:23)
#7 _ReceivePortImpl._handleMessage (dart:isolate-patch:81:92)
Any one knows the correct way of calling web services on the stream server?
The key is you have to return Future, since your task (openUrl) is asynchronous. In your sample code, you have to do:
return conn.then((HttpClientRequest request) {
//^-- notice: you must return a future to indicate when the serving is done
For more information, refer to Request Handling. To avoid this kind of mistake, I post a feature request here.
Here is a working sample:
library issues;
import "dart:io";
import "dart:uri";
import "package:rikulo_commons/io.dart" show IOUtil;
import "package:stream/stream.dart";
void main() {
new StreamServer(uriMapping: {
"/": (connect)
=> new HttpClient().getUrl(new Uri("http://google.com"))
.then((req) => req.close())
.then((res) => IOUtil.readAsString(res))
.then((result) {
connect.response.write(result);
})
}).start();
}
Here's my original code:
void startProcess(HttpConnect connect){
String method = "GET";
String url = "http://XXXXXX/activiti-rest/service/deployments";
Map authentication = {"username":"XXXXX", "password":"XXXXX"};
Map body = {"XXXXX":"XXXXX", "XXXXX":"XXXXX"};
processRequest(connect, method, url, authentication, body.toString());
}
void processRequest(HttpConnect connect, String method, String url, Map authentication, String body) {
HttpClient client = new HttpClient();
Uri requestUri = Uri.parse(url);
Future<HttpClientRequest> conn = client.openUrl(method, requestUri);
conn.then((HttpClientRequest request) {
request.headers.add(HttpHeaders.CONTENT_TYPE, Constants.CONTENT_TYPE_JSON);
// Add base64 authentication header, the class is contained in a separate dart file
String base64 = Base64String.encode('${authentication["username"]}:${authentication["password"]}');
base64 = 'Basic $base64';
request.headers.add("Authorization", base64);
switch( method ) {
case Constants.METHOD_GET: break;
case Constants.METHOD_POST:
body = body.replaceAllMapped(new RegExp(r'\b\w+\b'), (match) => '"${match.group(0)}"' );//no replacement for now
request.write(body);
break;
}
return request.close();
})
.then((HttpClientResponse response) {
return IOUtil.readAsString(response);
})
.then((String result) {
connect.response.write(result);
});
}