there is similar question (how to process a HTTP stream with Dart) about processing streams in dart2js. I am focused on vm.
I can read from spec that:
The content of message can be: primitive values (null, num, bool, double, String), instances of SendPort, and lists and maps whose elements are any of these. List and maps are also allowed to be cyclic.
In the special circumstances when two isolates share the same code and are running in the same process (e.g. isolates created via spawn), it is also possible to send object instances (which would be copied in the process). This is currently only supported by the dartvm. For now, the dart2js compiler only supports the restricted messages described above.
I ve learnt I cannot send to isolates and back following objects: HttpRequest and HttpResponse objects, I cannot send streams.
Q: I cannot really understand how should I process big chunk of data in isolate and then send it back to main isolate and in turn it can be send back to client.
Normally if I want to read a file. I can obtain a stream, apply transforms and then pipe a stream to http response. what is best practice to do that using isolates?
thanks for help!
I have provided an example but Ill give a quick overview on how you can achieve this although it may not necessarily be the best practise - its simply the only way I personally know how.
We are given a method called Isolate.spawnUri(Uri uri, List<String> args, dynamic message)
The message parameter can hold any of the values the extract in your first post mentions. What we want to do is in the main thread create a new ReceivePort and listen for incoming data, then we want to spawn an isolate with the message as the ReceivePorts .sendPort.
The isolate should then create its own ReceivePort and use the message value to send back its own sendPort and listen on its receive port. What this essentially does is creates a 2 way communication. Allowing us to keep the isolate alive and send work back and forth.
Any response from your isolate will come through your original receive port and any data you want to send to the isolate will be through the send port the isolate just sent back.
You can then directly send the stream data to the isolate for it to process at will and it can send back data as its made (or you can just close the ports after isolate has sent its response - up to you how you want to go about it).
I have provided an example on GitHub:
https://github.com/TomCaserta/ExampleIsolate
Please note, if you want your isolates to work in dart2js its important that you run dart2js on each of your isolate files via the following command:
dart2js -o <filename>.dart.js <filename>.dart
Related
I'm facing a situation where I have multiple robots, most running full ROS stacks (complete with Master) and I'd like to selectively route some topics through another messaging framework to the other robots (some of which not running ROS).
The naive way to do this works, namely, to set up a node that subscribes to the ROS topics in question and sends that over the network, after which another node publishes it (if its ROS). Great, but it seems odd to have to do this much serializing. Right now the message goes from its message type to the ROS serialization, back to the message type, then to a different serialization format (currently Pickle), across the network, then back to the message type, then back to the ROS serialization, then back to the message type.
So the question is, can I simplify this? How can I operate on the ROS serialized data (ie subscribe without rospy automagically deserializing for me)? http://wiki.ros.org/rospy/Overview/Publishers%20and%20Subscribers suggests that I can access the connection information as dict of strings, which may be half of the solution, but how can the other end take the connection information and republish it without first deserializing and then immediately reserializing?
Edit: I just found https://gist.github.com/wkentaro/2cd56593107c158e2e02 , which seems to solve half of this. It uses AnyMsg to avoid deserializing on the ROS subscriber side, but then when it republishes it still deserializes and immediately reserializes the message. Is what I'm asking impossible?
Just to close the loop on this, it turns out you can publish AnyMsgs, it's just that the linked examples chose not to.
I want to see if the other side gave up and closed the sink of a StreamChannel, without actually reading the messages yet.
(I'm going to be handing the stream to someone else, so i can't listen() to it, since you're only allowed to listen once per stream.)
[posting for a friend, credit to them for asking the question]
In short, no.
There is no concept of "giving up". If you put events into a non-broadcast stream, they'll stay there until someone listens to the stream (which is why you shouldn't put data there until someone listens, you're just wasting memory).
That includes the done event, and you won't get to the done event without first reading all the preceding events. That's the core abstraction of a stream - a source of events accessed in order, it's not done until it's actually done.
What I think you are looking for is a "side channel" that can communicate information about the stream without going through the stream (that is, out-of-band).
Something like that can surely be built - in about one gazillion different ways, depending on what you want, but it's just not something that a Stream supports by default, nor does a StreamChannel, if I read it correctly (I have never used a StreamChannel myself).
A dart:html.WebSocket opens once and closes once. So I'd expect that the onOpen and onClose properties would be Futures, but in reality they are Streams.
Of course, I can use stream.onClose.first to get a Future. But the done property on the dart:io version of WebSocket is a Future as expected, so I'm wondering if I'm missing something.
Why use Streams and not Futures in dart:html?
In a word: transparency. The purpose of the dart:html package is to provide access to the DOM of the page, and so it mirrors the underlying DOM constructs as closely as possible. This is in contrast with dart:io where the goal is to provide a convenient server-side API rather than exposing some underlying layer.
Sure, as a consumer of the API you would expect open and close to be fired only once, whereas message would be fired multiple times, but at the root of things, open, close, message, and error are all just events. And in dart:html, DOM events are modeled as streams.
And actually, a WebSocket could very well fire multiple open events (or close events). The following is definitely a contrived example, but consider this snippet of javascript:
var socket = new WebSocket('ws://mysite.com');
socket.dispatchEvent(new Event('open'));
socket.dispatchEvent(new Event('open'));
socket.dispatchEvent(new Event('open'));
How would a Dart WebSocket object behave in a situation like this, if onOpen were a Future rather than a Stream? Of course I highly, highly doubt this would ever surface out there in the "real world". But the DOM allows for it, and dart:html should not be making judgment calls trying to determine which situations are likely and which are not. If it's possible according to the spec, dart:html should reflect that. Its role is simply to pass through the behavior - as transparently as possible - and let the consumer of the API decide what cases they need to handle and which they can ignore.
I want write a program in Delphi to watch the internet connection, and if a certain response received (in response to request from a program), send request again encoded to another server, get a new encoded response, decode it, and pass it as response to the program who sent the main request. But I don't now how to hook internet connection. I want to use this program to pass through filter my country governments made using a private program to avoid blocking it. Is there any idea?
Thanks for your answer.
Magenta Systems released a free set of Delphi components that let you see the network traffic on your computer and examine the content. If you see the response you are looking for, your monitoring program can send a request to another server.
Off the top of my head, I'm not sure if it will let you alter the content of the original packet. If not, then Marcus' suggestion of using a proxy might suit you better.
You can either try to hook stuff at the Winsock level (there's plenty of examples for that around), but I suggest you go one level deeper and use a Layered Service provider (LSP).
I have used Komodia's redirector from http://www.komodia.com. Commercial, but well worth it.
See also this post
Is it possible to intercept dns queries using LSP/SPI?
I really like the message passing primitives that D implements. I have only seen examples of message passing within a program though. Is there support for distributing the messages over e.g. a network?
The message passing functions are in std.concurrency, which only deals with threads. So, the type of message passing used to pass messages between threads is for threads only. There is no RMI or anything like that in Phobos. That's not to say that we'll never get something like that in Phobos (stuff is being added to Phobos all the time), but it doesn't exist right now.
There is, however, the std.socket module which deals with talking to sockets, which is obviously network-related. I haven't used it myself, but it looks like it sends and receives void[]. So, it's not as nice as sending immutable objects around like you do with std.concurrency, but it does allow you to do network communication via sockets and presumably in a much nicer manner than if you were using C calls directly.
Seems that this has been considered. From the Phobos documentation (found it through Jonathan M Davis answer)
This is a low-level messaging API upon
which more structured or restrictive
APIs may be built. The general idea is
that every messageable entity is
represented by a common handle type
(called a Cid in this implementation),
which allows messages to be sent to
in-process threads, on-host processes,
and foreign-host processes using the
same interface. This is an important
aspect of scalability because it
allows the components of a program to
be spread across available resources
with few to no changes to the actual
implementation.
Right now, only in-process threads are
supported and referenced by a more
specialized handle called a Tid. It is
effectively a subclass of Cid, with
additional features specific to
in-process messaging.