Since node-fetch was replaced by undici in #5117 some of us encountered the error
Node streams are no longer supported — use a ReadableStream instead
like in this post
It is not easy to reproduce, for me the error occured only in production.
This is a self-answered question in case you have the same problem.
The error comes from src/runtime/server/utils.js L46 and is thrown after checking the _readableState property and some type on the response body of the request.
For me the problem was that my endpoint.ts was returning the fetch directly.
export async function post({request}){
return fetch('...')
}
This used to work but not anymore since the fetch response is a complex object with the _readableState property. To fix this you have to consume the response and return a simpler object like
export async function post({request}){
try {
const res = await fetch('...')
const data = await res.json()
return {
status: 200,
body: JSON.stringify({...data}),
}
catch(error){
return { status: 500}
}
}
Related
We have an airtable database of over 24000 records. These records are websites, and many now have errors in them (missing "/", extra space...). We are trying to detect the websites that have these errors before manually fixing them.
What we have tried
So far, we have used the fetch method to call each URL and report back on the error status . This is the script we have used:
const inputConfig = input.config();
const url = inputConfig.url;
let status;
try {
const response = await fetch(url);
status = response.status; } catch (error) {
status = 'error'; }
output.set('status', status);
Issues we ran into
The script won't follow redirects, so it reports "error" back if there is a redirect even if the URL is working.
The output now is either "200" meaning the URL works, or "error". We don't get the actual response code of the error, which we ideally would like to get.
Any help would be appreciated! Thanks
There's some nuance to how fetch works. If you review Mozilla's documentation they say:
The Promise returned from fetch() won't reject on HTTP error status even if the response is an HTTP 404 or 500. Instead, as soon as the server responds with headers, the Promise will resolve normally (with the ok property of the response set to false if the response isn't in the range 200–299), and it will only reject on network failure or if anything prevented the request from completing.
So you have to do an extra check in your code to determine if the request was successful or not and throw your own error. In your case, you don't necessarily need to throw an error at all and can just rely on ok property of the response.
const config = input.config();
const url = config.url;
let status = null;
const response = await fetch(url);
if(response.ok) {
status = response.status
} else {
status = 'error'
}
output.set('status', status);
I'm using the Dart Shelf package and I need to log the response it sends.
I've managed to log the request but the response technique is less clear:
final handler = const shelf.Pipeline()
.addMiddleware(corsHeaders())
.addMiddleware(shelf.logRequests(
logger: (message, isError) =>
_logRequest(message, isError: isError)))
.addHandler((req) async {
final res = await Router().call(req);
return res;
});
There two parts to the question.
how do I log the headers.
is it possible to log the body.
I know there is an issue in that the response body can only be read once.
As some of the responses are likely to be large I need to filter the requests for which the body is logged.
The answer is a bit of Dart-fu. You have an anonymous function returning an anonymous function.
var handler = const Pipeline()
.addMiddleware(
(handler) => (request) async {
final response = await handler(request);
print(response.headers);
// you could read the body here, but you'd also need to
// save the content and pipe it into a new response instance
return response;
},
)
.addHandler(syncHandler);
I'm working on a cross platform library that makes HTTP requests. It's working fine on Android, but when I try to use it on iOS I'm getting an exception and I can't figure out how to fix it.
Here is my code:
// method from cross platform library
Task.Factory.StartNew(delegate
{
try
{
var client = new HttpClient();
// some other setup stuff
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.post, "http://myurl.com...");
var task = client.SendAsync(request);
task.Wait(); // Exception thrown on this line
var response = task.Result;
var responseString = response.Content.ReadAsStringAsync().Result;
}
catch (Exception e)
{
}
}
On task.Wait(); I get a System.AggregateException with an inner exception of System.InvalidOperationException that says Operation is invalid due to the current state of the object.
Trying to find some solutions, I found that the issue could be cause by calling this on the UI thread. But that's the whole point of wrapping this all in Task.Factory.StartNew.
I've tried everything I know to do and have yet to solve the issue. Any help would be very appreciated.
Edit:
I decided to try my solution on an iPhone simulator. It's an iPhone 6 simulator running iOS 10. My physical device is the same. It works on the simulator, but not the physical device for some reason... very strange.
Edit 2:
Thanks to #YuriS for finding a solution.
From: https://forums.xamarin.com/discussion/36713/issue-with-microsoft-http-net-library-operation-is-not-valid-due-to-the-current-state-of-the-objec
What you can do is:
1) Go to References of ios Project
2) Edit References
3) Check 'System.Net.Http'
Behaviour for android is the same.
There can be few problems described here:
https://forums.xamarin.com/discussion/36713/issue-with-microsoft-http-net-library-operation-is-not-valid-due-to-the-current-state-of-the-objec
https://bugzilla.xamarin.com/show_bug.cgi?id=17936
"Operation is not valid" error at Xamarin.iOS project with HttpClient
http://motzcod.es/post/78863496592/portable-class-libraries-httpclient-so-happy
Seems all post pointing on System.Net.Http
Regardless of the problem there is a better ways doing this.One of them:
public static async Task PostRequest()
{
try
{
HttpClient client = new HttpClient();
HttpRequestMessage request = new HttpRequestMessage(HttpMethod.Post, "https://myuri");
//request.Headers.Add("", "");
var response = await client.SendAsync(request);
var responseString = await response.Content.ReadAsStringAsync();
}
catch (Exception ex)
{
}
}
If you want to wait till function completes you call
await PostRequest();
If you don't need to wait then just omit "await" in the call or use
PostRequest.ContinueWith((t)=>
{
});
Also you need to handle an exception within the function, so probably returning just Task is not the best. I was just basing my answer on original function signature
I use my postgres database query to determine my next action. And I need to wait for the results before I can execute the next line of code. Now my conn.query returns a Future but I can't manage to get it async when I place my code in another function.
main() {
// get the database connection string from the settings.ini in the project root folder
db = getdb();
geturl().then((String url) => print(url));
}
Future geturl() {
connect(db).then((conn) {
conn.query("select trim(url) from crawler.crawls where content IS NULL").toList()
.then((result) { return result[0].toString(); })
.catchError((err) => print('Query error: $err'))
.whenComplete(() {
conn.close();
});
});
}
I just want geturl() to wait for the returned value but whatever I do; it fires immediately. Can anyone point me a of a piece of the docs that explains what I am missing here?
You're not actually returning a Future in geturl currently. You have to actually return the Futures that you use:
Future geturl() {
return connect(db).then((conn) {
return conn.query("select trim(url) from crawler.crawls where content IS NULL").toList()
.then((result) { return result[0].toString(); })
.catchError((err) => print('Query error: $err'))
.whenComplete(() {
conn.close();
});
});
}
To elaborate on John's comment, here's how you'd implement this using async/await. (The async/await feature was added in Dart 1.9)
main() async {
try {
var url = await getUrl();
print(url);
} on Exception catch (ex) {
print('Query error: $ex');
}
}
Future getUrl() async {
// get the database connection string from the settings.ini in the project root folder
db = getdb();
var conn = await connect(db);
try {
var sql = "select trim(url) from crawler.crawls where content IS NULL";
var result = await conn.query(sql).toList();
return result[0].toString();
} finally {
conn.close();
}
}
I prefer, in scenarios with multiple-chained futures (hopefully soon a thing of the past once await comes out), to use a Completer. It works like this:
Future geturl() {
final c = new Completer(); // declare a completer.
connect(db).then((conn) {
conn.query("select trim(url) from crawler.crawls where content IS NULL").toList()
.then((result) {
c.complete(result[0].toString()); // use the completer to return the result instead
})
.catchError((err) => print('Query error: $err'))
.whenComplete(() {
conn.close();
});
});
return c.future; // return the future to the completer instead
}
To answer your 'where are the docs' question: https://www.dartlang.org/docs/tutorials/futures/
You said that you were trying to get your geturl() function to 'wait for the returned value'. A function that returns a Future (as in the example in the previous answer) will execute and return immediately, it will not wait. In fact that is precisely what Futures are for, to avoid code doing nothing or 'blocking' while waiting for data to arrive or an external process to finish.
The key thing to understand is that when the interpreter gets to a call to then() or 'catchError()' on a Future, it does not execute the code inside, it puts it aside to be executed later when the future 'completes', and then just keeps right on executing any following code.
In other words, when using Futures in Dart you are setting up chunks of code that will be executed non-linearly.
I was trying to send data to a local server using HttpClient. However, the data is never added to the request, I'm using this code:
new HttpClient().put('127.0.0.1', 4040, '/employees/1').then((request) {
request.cookies.add(new Cookie('DARTSESSID',sessionId)..path = '/');
request.headers.add(HttpHeaders.ACCEPT_ENCODING, "");
request.headers.add(HttpHeaders.CONTENT_TYPE, "text/json");
request.write('{"id": 1, "name": "luis"}');
print(request.contentLength);
return request.close();
}).then(expectAsync((HttpClientResponse response) {
expect(response.statusCode, 200);
UTF8.decodeStream(response).then(expectAsync((body) {
expect(body, equals('"employee: 1"'));
}));
}));
but that always prints that the request.contentLenght is -1. I saw those links before without luck:
https://code.google.com/p/dart/issues/detail?id=13293
dart - HttpClientRequest failing on adding data
https://code.google.com/p/dart/issues/detail?id=10026
A ContentLength of -1 does not mean that there is no data, it means that the length of the content is unknown and that a streaming content mode is used - for HTTP 1.1, this will usually mean Chunked ContentEncoding.
I've tried to insert your code in a setup including a server, but without the unittest stuff:
import 'dart:convert';
import 'dart:io';
void main() {
HttpServer.bind('127.0.0.1', 4040).then((server) {
server.listen((request) {
UTF8.decodeStream(request).then((body) {
print(body);
request.response.close();
});
});
new HttpClient().put('127.0.0.1', 4040, '/employees/1').then((request) {
request.cookies.add(new Cookie('DARTSESSID', "1")..path = '/');
request.headers.add(HttpHeaders.ACCEPT_ENCODING, "");
request.headers.add(HttpHeaders.CONTENT_TYPE, "text/json");
request.write('{"id": 1, "name": "luis"}');
print(request.contentLength);
return request.close();
}).then((HttpClientResponse response) {
UTF8.decodeStream(response).then((body) {
print(body);
});
});
});
}
When I run the code, I get
-1
{"id": 1, "name": "luis"}
as expected. Perhaps the problem you are having are on the server?
Writing to the request is an asynchronous operation. Just because the contentLength says that it still is -1 doesn't mean that the data isn't added to the request before sending it to the server.
Also: the content-length is not supposed to update whenever you add new data. It is the value that is sent to the server. -1 means that you don't know the size yet.
I'm not sure, if the library automatically updates it, if it knows the size, but it doesn't need to.