Is there a way of getting ahold of the ByteBuf allocator used in Reactor Netty in a request handler? Similar to how one can do final ByteBufAllocator byteBufAllocator = ctx.alloc(); in pure Netty?
HttpServer.create()
.host("0.0.0.0")
.port(8080)
.route(routes -> {
routes.ws("/ws", (in, out) ->
// how to get the allocator here?
});
})
You can access the ByteBufAllocator from the WebSocketOutbound or HttpServerResponse like this:
HttpServer.create()
.host("0.0.0.0")
.port(8080)
.route(routes -> routes
.ws("/ws", (in, out) -> {
ByteBufAllocator alloc = out.alloc();
// ...
})
.get("/path", (request, response) -> {
ByteBufAllocator alloc = response.alloc();
// ...
})
);
Related
I have a KMM app, and there is code:
fun getWeather(callback: (WeatherInfo) -> Unit) {
println("Start loading")
GlobalScope.launch(ApplicationDispatcher) {
while (true) {
val response = httpClient.get<String>(API_URL) {
url.parameters.apply {
set("q", "Moscow")
set("units", "metric")
set("appid", weatherApiKey())
}
println(url.build())
}
val result = Json {
ignoreUnknownKeys = true
}.decodeFromString<WeatherApiResponse>(response).main
callback(result)
// because ApplicationDispatcher on IOS do not support delay
withContext(Dispatchers.Default) { delay(DELAY_TIME) }
}
}
}
And if I replace withContext(Dispatchers.Default) { delay(DELAY_TIME) } with delay(DELAY_TIME) execution is never returned to while cycle and it will have only one iteration.
And ApplicationDispatcher for IOS looks like:
internal actual val ApplicationDispatcher: CoroutineDispatcher = NsQueueDispatcher(dispatch_get_main_queue())
internal class NsQueueDispatcher(
private val dispatchQueue: dispatch_queue_t
) : CoroutineDispatcher() {
override fun dispatch(context: CoroutineContext, block: Runnable) {
dispatch_async(dispatchQueue) {
block.run()
}
}
}
And from delay source code I can guess, that DefaultDelay should be returned and there is should be similar behaviour with/without withContext(Dispatchers.Default)
/** Returns [Delay] implementation of the given context */
internal val CoroutineContext.delay: Delay get() = get(ContinuationInterceptor) as? Delay ?: DefaultDelay
Thanks!
P.S. I got ApplicationDispatcher from ktor-samples.
Probably ApplicationDispatcher is some old stuff, you don't need to use it anymore:
CoroutineScope(Dispatchers.Default).launch {
}
or
MainScope().launch {
}
And don't forget to use -native-mt version of coroutines, more info in this issue
I am trying to extract messages (which are futures themselves) from an unbounded queue every N seconds and spawn them into the Tokio handler.
I’ve tried dozens of variations but I cannot seem to find the right approach. It looks like it should be possible, but I always hit a future type mismatch or end up with borrow issues.
This is the code that shows more or less what I want:
let fut = Interval::new_interval(Duration::from_secs(1))
.for_each(|num| vantage_dequeuer.into_future() )
.for_each(|message:VantageMessage |{
handle.spawn(message);
return Ok(());
})
.map_err(|e| panic!("delay errored; err={:?}", e));
core.run(fut);
Complete code:
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
extern crate tokio_core; // 0.1.17
use futures::future::ok;
use futures::sync::mpsc;
use futures::{Future, Stream};
use std::thread;
use std::time::Duration;
use tokio::timer::Interval;
use tokio_core::reactor::Core;
type VantageMessage = Box<Future<Item = (), Error = ()> + Send>;
fn main() {
let (enqueuer, dequeuer) = mpsc::unbounded();
let new_fut: VantageMessage = Box::new(ok(()).and_then(|_| {
println!("Message!");
return Ok(());
}));
enqueuer.unbounded_send(new_fut);
let joinHandle = worker(Some(dequeuer));
joinHandle.join();
}
/*
Every second extract one message from dequeuer (or wait if not available)
and spawn it in the core
*/
fn worker(
mut vantage_dequeuer: Option<mpsc::UnboundedReceiver<VantageMessage>>,
) -> thread::JoinHandle<()> {
let dequeuer = dequeuer.take().unwrap();
let joinHandle = thread::spawn(|| {
let mut core = Core::new().unwrap();
let handle = core.handle();
let fut = Interval::new_interval(Duration::from_secs(1))
.for_each(|num| vantage_dequeuer.into_future())
.for_each(|message: VantageMessage| {
handle.spawn(message);
return Ok(());
})
.map_err(|e| panic!("delay errored; err={:?}", e));
core.run(fut);
println!("Returned!");
});
return joinHandle;
}
Playground
error[E0425]: cannot find value `dequeuer` in this scope
--> src/main.rs:33:20
|
33 | let dequeuer = dequeuer.take().unwrap();
| ^^^^^^^^ not found in this scope
error[E0599]: no method named `into_future` found for type `std::option::Option<futures::sync::mpsc::UnboundedReceiver<std::boxed::Box<(dyn futures::Future<Item=(), Error=()> + std::marker::Send + 'static)>>>` in the current scope
--> src/main.rs:38:46
|
38 | .for_each(|num| vantage_dequeuer.into_future())
| ^^^^^^^^^^^
|
= note: the method `into_future` exists but the following trait bounds were not satisfied:
`&mut std::option::Option<futures::sync::mpsc::UnboundedReceiver<std::boxed::Box<(dyn futures::Future<Item=(), Error=()> + std::marker::Send + 'static)>>> : futures::Stream`
Interval and UnboundedReceiver are both streams, so I'd use Stream::zip to combine them:
The zipped stream waits for both streams to produce an item, and then returns that pair. If an error happens, then that error will be returned immediately. If either stream ends then the zipped stream will also end.
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
extern crate tokio_core; // 0.1.17
use futures::{
future::ok,
sync::mpsc,
{Future, Stream},
};
use std::{thread, time::Duration};
use tokio::timer::Interval;
use tokio_core::reactor::Core;
type VantageMessage = Box<Future<Item = (), Error = ()> + Send>;
pub fn main() {
let (tx, rx) = mpsc::unbounded();
let new_fut: VantageMessage = Box::new(ok(()).and_then(|_| {
println!("Message!");
Ok(())
}));
tx.unbounded_send(new_fut).expect("Unable to send");
drop(tx); // Close the sending side
worker(rx).join().expect("Thread had a panic");
}
fn worker(queue: mpsc::UnboundedReceiver<VantageMessage>) -> thread::JoinHandle<()> {
thread::spawn(|| {
let mut core = Core::new().unwrap();
let handle = core.handle();
core.run({
Interval::new_interval(Duration::from_secs(1))
.map_err(|e| panic!("delay errored; err={}", e))
.zip(queue)
.for_each(|(_, message)| {
handle.spawn(message);
Ok(())
})
})
.expect("Unable to run reactor");
println!("Returned!");
})
}
Note that this doesn't actually wait for any of the spawned futures to complete before the reactor shuts down. If you want that, I'd switch to tokio::run and tokio::spawn:
fn worker(queue: mpsc::UnboundedReceiver<VantageMessage>) -> thread::JoinHandle<()> {
thread::spawn(|| {
tokio::run({
Interval::new_interval(Duration::from_secs(1))
.map_err(|e| panic!("delay errored; err={}", e))
.zip(queue)
.for_each(|(_, message)| {
tokio::spawn(message);
Ok(())
})
});
println!("Returned!");
})
}
I am writing a low-level network app that deals with TCP sockets where I often need to process binary data streams. When some data is available, I read it into u8 array, then wrap into std::io::Cursor<&[u8]> and then pass it to handlers. In a handler, I often need to know if there is some more data in the Cursor or not.
Imagine that the handle function receives data and then processes it in chunks using the handle_chunk function. For simplicity, assume that chunk size is fixed at 10 bytes; if the data size is not divisible by 10, it's an error. This simple logic can be implemented in the following way:
fn handle(mut data: Cursor<&[u8]>) {
while !data.empty() {
if let Err(err) = handle_chunk(&mut data) {
eprintln!("Error while handling data: {}", err);
}
}
}
fn handle_chunk(data: &mut Cursor<&[u8]>) -> Result<(), String> {
// Returns Err("unexpected EOF".to_string()) if chunk is incomplete
// ...
}
However, Cursor does not have an empty() method or any other method capable of telling if there is more data to process. The working solution that I could come up with is:
fn handle(data: Cursor<&[u8]>) {
let data = data.into_inner();
let len = data.len();
let mut data = Cursor::new(data);
while (data.position() as usize) < len - 1 {
if let Err(err) = handle_chunk(&mut data) {
eprintln!("Error while handling data: {}", err);
}
}
}
This looks hacky and inelegant though. Is there a better solution? Maybe there is a different tool in the Rust standard library that fits here better than Cursor?
Your code can be simplified by using Cursor::get_ref to avoid breaking up the input and putting it back together:
fn handle(mut data: Cursor<&[u8]>) {
let len = data.get_ref().len();
while (data.position() as usize) < len - 1 {
if let Err(err) = handle_chunk(&mut data) {
eprintln!("Error while handling data: {}", err);
}
}
}
Now, you haven't shown any code that requires a Cursor. Many times, people think it's needed to convert a &[u8] to something that implements Read, but it's not. Read is implemented for &'a [u8]:
use std::io::Read;
fn handle(mut data: &[u8]) {
while !data.is_empty() {
if let Err(err) = handle_chunk(&mut data) {
eprintln!("Error while handling data: {}", err);
}
}
}
fn handle_chunk<R: Read>(mut data: R) -> Result<(), String> {
let mut b = [0; 10];
data.read_exact(&mut b).unwrap();
println!("Chunk: {:?}", b);
Ok(())
}
fn main() {
let d: Vec<u8> = (0..20).collect();
handle(&d)
}
By having mut data: &[u8] and using &mut data, the code will update the slice variable in place to advance it forward. We can't easily go backward though.
an empty() method
Rust style indicates that an empty method would be a verb — this would remove data (if it were possible). The method you want should be called is_empty, as seen on slices.
I try to put g_timeout_add () on hold in if it meets one condition without being removed with g_source_remove ().
Basically pause/resume thread.
From documentation g_timeout_add () I do not realize how this could be achieved
Now can do this with:
void
schedule_popup (gpointer data)
{
unschedule_popup (data);
timeout_id = g_timeout_add_seconds (POPUP_TIMEOUT, (GSourceFunc) hide_popup_timeout_cb, data);
g_source_set_name_by_id (timeout_id, "[myapp] hide_popup_timeout_cb");
}
void
unschedule_popup (gpointer data)
{
if (timeout_id > 0)
g_source_remove (timeout_id);
timeout_id = 0;
}
(I'm using the new lib v2 version of dart:io.)
I'd like to register multiple handlers for an HttpServer, specifically a WebSocketTransformer and other arbitrary handlers. Something like this:
// pseudo-code
var server = HttpServer;
server.register('/foo', someHandlerFunction); // 1
server.register('/bar', someOtherHandlerFunction); // 2
server.register('/ws', webSocketHandler); // 3
If #1 matches, then #2 isn't tested, and so on. So, it's greedy.
I've seen samples with just one handler. How do I register many handlers? Thanks in advance!
New answer: Use the route package: http://pub.dartlang.org/packages/route
Here's your example using route's serve() method:
HttpServer.bind('127.0.0.1', 8889).then((server) {
var router = new Router(server)
..serve('/ws').transform(new WebSocketTransformer()).listen(handleWebSocket)
..serve('/foo').listen((req) {
req.response..addString('foo')..close();
});
});
Router automatically catches unhandled requests and sends a 404, though soon you'll be able to override that with a defaultStream you can listen to.
Router also supports filters, useful for logging, auth, compression, etc.:
HttpServer.bind('127.0.0.1', 8889).then((server) {
var router = new Router(server)
..filter(new RegExp(r'/.*'), (req) {
//log all requests
_logger.info("request: $req");
return new Future.immediate(true); // keep processing request
})
..filter(new Regexp(r'/secure/.*'), (req) {
// check authentication asynchronously
return getUserFromRequest(req).then((user) {
if (user == null) {
sendRedirect('/login'); // sendRedirect coming soon
return false; // stop processing request
} else {
return true; // keep processing
}
});
})
..serve(/* ... */ );
});
Here's how the API docs recommend to register a WebSocket handler:
server
.where((request) => request.uri.path == "/ws")
.transform(new WebSocketTransformer()).listen((webSocket) => ...);
However, the server is a single-subscription stream. Once a listen is attached, you can't attach other listeners.
What I really want is for something to look at an event, decide if it can handle it, and if so then route it to another stream. Otherwise, pass it along. This way, the event (in this case an HttpRequest object) is passed along a chain until it's handled.
I built a TakeAndRoute class that extends StreamEventTransformer. The TakeAndRoute uses a function to determine if it should grab the event and route it to another stream, or simply forward it along.
Here's what I came up with:
import 'dart:io';
import 'dart:async';
handleWebSocket(WebSocket webSocket) {
webSocket.listen((event) {
if (event is MessageEvent) {
/* Handle message. */
} else if (event is CloseEvent) {
/* Handle closed. */
}
});
}
typedef bool ShouldTake(e);
typedef void RouteTo(Stream stream);
typedef void HandleEvent(e);
class TakeAndRoute<S, T> extends StreamEventTransformer<S, T> {
ShouldTake shouldTake;
RouteTo routeTo;
StreamController controller = new StreamController();
HandleEvent handler;
TakeAndRoute(this.shouldTake, {this.routeTo, this.handler}) {
if (routeTo != null) routeTo(controller.stream);
}
handleData(event, StreamSink sink) {
print("handling");
if (shouldTake(event)) {
if (routeTo != null) {
controller.add(event);
}
if (handler != null) {
handler(event);
}
} else {
sink.add(event);
}
}
}
main() {
HttpServer.bind('127.0.0.1', 8888)
.then((HttpServer server) {
server
.transform(new TakeAndRoute<HttpRequest, HttpRequest>(
(req) => req.uri.path == '/ws',
routeTo: (stream) => stream.transform(new WebSocketTransformer()).listen(handleWebSocket)))
.transform(new TakeAndRoute<HttpRequest, HttpRequest>(
(req) => req.uri.path == '/foo',
handler: (req) {
print('got foo');
req.response.addString("foo");
req.response.close();
}))
.listen((req) {
print("got 404 for ${req.uri}");
req.response.statusCode = 404;
req.response.close();
});
});
}
Admittedly, this might be overkill.
Here is a more manual, but shorter way to do it:
HttpServer.bind('127.0.0.1', 8889)
.then((HttpServer server) {
var sc = new StreamController();
sc.stream.transform(new WebSocketTransformer()).listen(handleWebSocket);
server.listen((HttpRequest request) {
print("new connection from ${request.uri.scheme} ${request.uri}");
// See https://code.google.com/p/dart/issues/detail?id=8825
//if (request.uri.scheme == 'ws') {
if (request.uri.path == '/ws') {
sc.add(request);
} else if (request.uri.path == '/foo') {
request.response.addString('foo');
request.response.close();
} else {
print("got 404 for ${request.uri}");
request.response.statusCode = 404;
request.response.close();
}
});
});
Notice how I had to create a StreamController so I could pump events to WebSocketTransformer