How to cancel an infinite stream from within the stream itself? - stream

I'm trying to cancel an interval (interval_timer) after emptying a queue but not sure what is the right strategy.
let mut some_vars = vec![1, 2, 3, 4, 5, 6, 7, 8];
let interval_timer = tokio_timer::Timer::default();
let timer = interval_timer
.interval(Duration::from_millis(1000))
.map_err(|_| {
println!("Errored out");
});
let s = timer.for_each(move |_| {
println!("Woke up");
let item = some_vars.pop().unwrap();
let f = futures::future::ok(item).map(|x| {
println!("{:?}", x);
});
tokio::spawn(f)
});
tokio::run(s);
I tried drop as suggested in gitter but that ended up with an error:
let mut some_vars = vec![1, 2, 3, 4, 5, 6, 7, 8];
let mut interval_timer = tokio_timer::Timer::default();
let timer = interval_timer
.interval(Duration::from_millis(1000))
.map_err(|_| {
println!("Errored out");
});
let s = timer.for_each(move |_| {
println!("Woke up");
if some_vars.len() == 1 {
drop(interval_timer);
}
let item = some_vars.pop().unwrap();
let f = futures::future::ok(item).map(|x| {
println!("{:?}", x);
});
tokio::spawn(f)
});
tokio::run(s);
The error:
error[E0507]: cannot move out of captured outer variable in an `FnMut` closure
--> src/main.rs:72:22
|
60 | let mut interval_timer = tokio_timer::Timer::default();
| ------------------ captured outer variable
...
72 | drop(interval_timer);
| ^^^^^^^^^^^^^^ cannot move out of captured outer variable in an `FnMut` closure

For cases where you want to cancel a stream from outside of the stream, see stream-cancel.
For your specific case, it's easiest to convert your collection into a stream and zip it together with the interval timer. This way, the resulting stream naturally stops when the collection is empty:
use futures::{future, stream, Stream}; // 0.1.29
use std::time::Duration;
use tokio; // 0.1.22
use tokio_timer::Interval; // 0.2.11
fn main() {
tokio::run({
let some_vars = vec![1, 2, 3, 4, 5, 6, 7, 8];
let timer =
Interval::new_interval(Duration::from_millis(100)).map_err(|e| panic!("Error: {}", e));
let some_vars = stream::iter_ok(some_vars.into_iter().rev());
let combined = timer.zip(some_vars);
combined.for_each(move |(_, item)| {
eprintln!("Woke up");
tokio::spawn(future::lazy(move || {
println!("{:?}", item);
Ok(())
}));
Ok(())
})
});
}
Otherwise, you can stop the stream by using and_then to both remove the value from the collection and control if the stream should continue:
use futures::{future, Stream}; // 0.1.29
use std::time::Duration;
use tokio; // 0.1.22
use tokio_timer::Interval; // 0.2.11
fn main() {
tokio::run({
let mut some_vars = vec![1, 2, 3, 4, 5, 6, 7, 8];
let timer =
Interval::new_interval(Duration::from_millis(100)).map_err(|e| panic!("Error: {}", e));
let limited = timer.and_then(move |_| {
if some_vars.len() <= 4 {
Err(())
} else {
some_vars.pop().ok_or(())
}
});
limited.for_each(move |item| {
eprintln!("Woke up");
tokio::spawn(future::lazy(move || {
println!("{:?}", item);
Ok(())
}));
Ok(())
})
});
}

I created a copy of Tokio's Interval struct, adding a reference to a method of my application to indicate when to interrupt early.
In my case, I want to interrupt the Interval to shutdown.
My Interval poll method looks like this:
fn poll(&mut self) -> Poll<Option<Self::Item>, Self::Error> {
if self.session.read().unwrap().shutdown {
return Ok(Async::Ready(Some(Instant::now())));
}
// Wait for the delay to be done
let _ = match self.delay.poll() {
Then you need to keep a handle on the task (call task = futures::task::current() when running inside the timeout task).
At any point you can then call task.notify() to kick the interval into action and hit your break out code, interrupting the Interval early.
Inside Interval there is a Delay struct that can be modified, you could create an Interval that you can interrupt and change the timeout, this way you could interrupt once and then continue.

tokio_timer::Interval implements futures::Stream, so try to use the take_while method:
let s = timer
.take_while(|()|
future::ok(is_net_completed()))
.for_each(move |_| {
println!("Woke up");
// ...
})

Related

Cancelling an iOS/UIKit Async Task with a global flag

Environment: Swift 5, Xcode 14, iOS 15, UIKit (NOT SwiftUI)
I have a long-running async task which I execute in a Task block:
Task { () -> () in
do {
for z in arrayData{
if killTask { // an external property
try Task.cancel() // **Swift Errors here**
}
let x1 = try await self.longTask1(z.var1)
let x2 = try await self.longTask2(z.var2)
etc.
}
} catch { print("Bad") }
} // end task
This fails with Swift syntax errors that Success and Failure in the Task can not be inferred. The Task produces no Result type. Can someone please point me in the correct direction by which I can cancel a task (with no input/output/Result types) by an external semaphore condition?
Rather than introducing a killTask property, you can save the Task which you can later cancel.
So you would have a property:
var task: Task<Void, Error>?
and
task = Task {
for z in arrayData {
try Task.checkCancellation()
let x1 = try await self.longTask1(z.var1)
let x2 = try await self.longTask2(z.var2)
...
}
}
And later
task?.cancel()

How do I extract messages from an unbounded queue every N seconds and spawn them on to a Tokio handler?

I am trying to extract messages (which are futures themselves) from an unbounded queue every N seconds and spawn them into the Tokio handler.
I’ve tried dozens of variations but I cannot seem to find the right approach. It looks like it should be possible, but I always hit a future type mismatch or end up with borrow issues.
This is the code that shows more or less what I want:
let fut = Interval::new_interval(Duration::from_secs(1))
.for_each(|num| vantage_dequeuer.into_future() )
.for_each(|message:VantageMessage |{
handle.spawn(message);
return Ok(());
})
.map_err(|e| panic!("delay errored; err={:?}", e));
core.run(fut);
Complete code:
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
extern crate tokio_core; // 0.1.17
use futures::future::ok;
use futures::sync::mpsc;
use futures::{Future, Stream};
use std::thread;
use std::time::Duration;
use tokio::timer::Interval;
use tokio_core::reactor::Core;
type VantageMessage = Box<Future<Item = (), Error = ()> + Send>;
fn main() {
let (enqueuer, dequeuer) = mpsc::unbounded();
let new_fut: VantageMessage = Box::new(ok(()).and_then(|_| {
println!("Message!");
return Ok(());
}));
enqueuer.unbounded_send(new_fut);
let joinHandle = worker(Some(dequeuer));
joinHandle.join();
}
/*
Every second extract one message from dequeuer (or wait if not available)
and spawn it in the core
*/
fn worker(
mut vantage_dequeuer: Option<mpsc::UnboundedReceiver<VantageMessage>>,
) -> thread::JoinHandle<()> {
let dequeuer = dequeuer.take().unwrap();
let joinHandle = thread::spawn(|| {
let mut core = Core::new().unwrap();
let handle = core.handle();
let fut = Interval::new_interval(Duration::from_secs(1))
.for_each(|num| vantage_dequeuer.into_future())
.for_each(|message: VantageMessage| {
handle.spawn(message);
return Ok(());
})
.map_err(|e| panic!("delay errored; err={:?}", e));
core.run(fut);
println!("Returned!");
});
return joinHandle;
}
Playground
error[E0425]: cannot find value `dequeuer` in this scope
--> src/main.rs:33:20
|
33 | let dequeuer = dequeuer.take().unwrap();
| ^^^^^^^^ not found in this scope
error[E0599]: no method named `into_future` found for type `std::option::Option<futures::sync::mpsc::UnboundedReceiver<std::boxed::Box<(dyn futures::Future<Item=(), Error=()> + std::marker::Send + 'static)>>>` in the current scope
--> src/main.rs:38:46
|
38 | .for_each(|num| vantage_dequeuer.into_future())
| ^^^^^^^^^^^
|
= note: the method `into_future` exists but the following trait bounds were not satisfied:
`&mut std::option::Option<futures::sync::mpsc::UnboundedReceiver<std::boxed::Box<(dyn futures::Future<Item=(), Error=()> + std::marker::Send + 'static)>>> : futures::Stream`
Interval and UnboundedReceiver are both streams, so I'd use Stream::zip to combine them:
The zipped stream waits for both streams to produce an item, and then returns that pair. If an error happens, then that error will be returned immediately. If either stream ends then the zipped stream will also end.
extern crate futures; // 0.1.24
extern crate tokio; // 0.1.8
extern crate tokio_core; // 0.1.17
use futures::{
future::ok,
sync::mpsc,
{Future, Stream},
};
use std::{thread, time::Duration};
use tokio::timer::Interval;
use tokio_core::reactor::Core;
type VantageMessage = Box<Future<Item = (), Error = ()> + Send>;
pub fn main() {
let (tx, rx) = mpsc::unbounded();
let new_fut: VantageMessage = Box::new(ok(()).and_then(|_| {
println!("Message!");
Ok(())
}));
tx.unbounded_send(new_fut).expect("Unable to send");
drop(tx); // Close the sending side
worker(rx).join().expect("Thread had a panic");
}
fn worker(queue: mpsc::UnboundedReceiver<VantageMessage>) -> thread::JoinHandle<()> {
thread::spawn(|| {
let mut core = Core::new().unwrap();
let handle = core.handle();
core.run({
Interval::new_interval(Duration::from_secs(1))
.map_err(|e| panic!("delay errored; err={}", e))
.zip(queue)
.for_each(|(_, message)| {
handle.spawn(message);
Ok(())
})
})
.expect("Unable to run reactor");
println!("Returned!");
})
}
Note that this doesn't actually wait for any of the spawned futures to complete before the reactor shuts down. If you want that, I'd switch to tokio::run and tokio::spawn:
fn worker(queue: mpsc::UnboundedReceiver<VantageMessage>) -> thread::JoinHandle<()> {
thread::spawn(|| {
tokio::run({
Interval::new_interval(Duration::from_secs(1))
.map_err(|e| panic!("delay errored; err={}", e))
.zip(queue)
.for_each(|(_, message)| {
tokio::spawn(message);
Ok(())
})
});
println!("Returned!");
})
}

How to cancel a Stream when using Stream.periodic?

I'm having trouble canceling a stream that is created using the Stream.periodic constructor. Below is my attempt at canceling the stream. However, I'm having a hard time extracting out the 'count' variable from the internal scope. Therefore, I can't cancel the subscription.
import 'dart:async';
void main() {
int count = 0;
final Stream newsStream = new Stream.periodic(Duration(seconds: 2), (_) {
return _;
});
StreamSubscription mySubscribedStream = newsStream.map((e) {
count = e;
print(count);
return 'stuff $e';
}).listen((e) {
print(e);
});
// count = 0 here because count is scoped inside mySubscribedStream
// How do I extract out 'count', so I can cancel the stream?
if (count > 5) {
mySubscribedStream.cancel();
mySubscribedStream = null;
}
}
I'd rather use take(5) instead of checking > 5 and then cancel
final Stream newsStream = new Stream.periodic(Duration(seconds: 2), (_) => count++);
newsStream.map((e) {
count = e;
print(count);
return 'stuff $e';
}).take(5).forEach((e) {
print(e);
});

Handling streaming iterator as normal iterator by using PhantomData and unsafe

I know the code below is hacky, but could it be called safe and idiomatic Rust? Is there better way for this?
// needs to do 'rustup default nightly' to run under valgrind
// #![feature(alloc_system, global_allocator, allocator_api)]
// extern crate alloc_system;
// use alloc_system::System;
// #[global_allocator]
// static A: System = System;
struct Foo<'a> {
v: Vec<u8>,
pos: usize,
phantom: std::marker::PhantomData<&'a u8>,
}
impl<'a> Iterator for Foo<'a> {
type Item = &'a mut u8;
fn next(&mut self) -> Option<&'a mut u8> {
let r = self.v.get_mut(self.pos);
if r.is_some() {
self.pos += 1;
unsafe { Some(&mut *(r.unwrap() as *mut u8)) }
} else {
None
}
}
}
impl<'a> Foo<'a> {
fn reset(&mut self) {
self.pos = 0;
}
}
fn main() {
let mut x = Foo {
v: (1..10).collect(),
pos: 0,
phantom: std::marker::PhantomData,
};
let vp = x.v.as_ptr();
{
for i in &mut x {
println!("{}", i);
}
}
{
x.reset();
}
{
for i in &mut x {
*i *= *i;
}
}
{
x.reset();
}
{
for i in &mut x {
println!("{}", i);
}
}
assert!(vp == x.v.as_ptr());
}
Write a little bit in the comment, Valgrind told me no leak and the result is as expected under Rust 1.26.0-nightly and 1.25.0.
Related:
How do I write an iterator that returns references to itself?
Iterator returning items by reference, lifetime issue
This code is not safe. The user of the type may choose any lifetime, including 'static:
fn constructor() -> Foo<'static> {
Foo {
v: vec![42; 10],
pos: 0,
phantom: std::marker::PhantomData,
}
}
fn example() -> &'static u8 {
let mut f = constructor();
f.next().unwrap()
}
fn main() {
println!("example: {}", example());
}
Here, example returns a reference to a variable that is no longer in scope, accessing invalid memory and subverting the restrictions you must uphold.
There's an example of how you could write this code with no unsafe whatsoever in another Q&A.

Delaying a Tokio Stream

Given a Stream, I want to create a new Stream where elements are yielded with a time delay between them.
I tried to write code that does that using tokio_core::reactor::Timeout and the and_then combinator for Streams, but the delay doesn't work: I get all the elements immediately, without a delay.
Here is a self contained example (playground):
extern crate tokio_core;
extern crate futures;
use std::time::Duration;
use futures::{Future, Stream, stream, Sink};
use self::futures::sync::{mpsc};
use tokio_core::reactor;
const NUM_ITEMS: u32 = 8;
fn main() {
let mut core = reactor::Core::new().unwrap();
let handle = core.handle();
let chandle = handle.clone();
let (sink, stream) = mpsc::channel::<u32>(0);
let send_stream = stream::iter_ok(0 .. NUM_ITEMS)
.and_then(move |i: u32| {
let cchandle = chandle.clone();
println!("Creating a timeout object...");
reactor::Timeout::new(Duration::new(1,0), &cchandle)
.map_err(|_| ())
.and_then(|_| Ok(i))
});
let sink = sink.sink_map_err(|_| ());
handle.spawn(sink.send_all(send_stream).and_then(|_| Ok(())));
let mut incoming_items = Vec::new();
{
let keep_messages = stream.for_each(|item| {
incoming_items.push(item);
println!("item = {}", item);
Ok(())
});
core.run(keep_messages).unwrap();
}
assert_eq!(incoming_items, (0 .. NUM_ITEMS).collect::<Vec<u32>>());
}
For completeness, this is the output I get:
Creating a timeout object...
Creating a timeout object...
item = 0
Creating a timeout object...
item = 1
Creating a timeout object...
item = 2
Creating a timeout object...
item = 3
Creating a timeout object...
item = 4
Creating a timeout object...
item = 5
Creating a timeout object...
item = 6
item = 7
I suspect that the problem is somewhere in these lines:
reactor::Timeout::new(Duration::new(1,0), &cchandle)
.map_err(|_| ())
.and_then(|_| Ok(i))
It is possible that I don't really wait on the returned Timeout object, though I'm not sure how to solve it.
As I suspected, the problem was was the manipulation (using and_then) of the newly created Timeout. We either need to first unwrap the result from the call to reactor::Timeout::new, which could become messy if done manually, or use into_future, to convert the result into a Future, and then work with it using Future combinators.
Code for solving the problem:
extern crate tokio_core;
extern crate futures;
use std::time::Duration;
use futures::{Future, Stream, stream, Sink, IntoFuture};
use self::futures::sync::{mpsc};
use tokio_core::reactor;
const NUM_ITEMS: u32 = 8;
fn main() {
let mut core = reactor::Core::new().unwrap();
let handle = core.handle();
let chandle = handle.clone();
let (sink, stream) = mpsc::channel::<u32>(0);
let send_stream = stream::iter_ok(0 .. NUM_ITEMS)
.and_then(move |i: u32| {
let cchandle = chandle.clone();
println!("Creating a timeout object...");
reactor::Timeout::new(Duration::new(1,0), &cchandle)
.into_future()
.and_then(move |timeout| timeout.and_then(move |_| Ok(i)))
.map_err(|_| ())
});
let sink = sink.sink_map_err(|_| ());
handle.spawn(sink.send_all(send_stream).and_then(|_| Ok(())));
let mut incoming_items = Vec::new();
{
let keep_messages = stream.for_each(|item| {
incoming_items.push(item);
println!("item = {}", item);
Ok(())
});
core.run(keep_messages).unwrap();
}
assert_eq!(incoming_items, (0 .. NUM_ITEMS).collect::<Vec<u32>>());
}
Note that two and_then are being used. The first one unwraps the Result obtained from calling reactor::Timeout::new. The second one actually waits for the Timeout to fire.

Resources