I'm new to reactive programming using rxjava and after going through the simpler examples I'm now trying to figure out how to work with continuous streams. The problem I have with the example below is that the program doesn't terminate after I've taken the 3 elements. My assumption is that I somehow need to unsubscribe to my observable but I don't fully grasp how to terminate the while loop and make the program exit.
I've come across the following post RxJava -- Terminating Infinite Streams but I still can't figure out what I'm missing.
class MyTwitterDataProvider {
/*
This example is written in Groovy
Instance variables and constructor omitted
*/
public Observable<String> getTweets() {
BufferedReader reader = new BufferedReader(new InputStreamReader(getTwitterStream()))
Observable.create({ observer ->
executor.execute(new Runnable() {
def void run() {
String newLine
while ((newLine = reader.readLine()) != null) {
System.out.println("printing tweet: $newLine")
observer.onNext(newLine)
}
observer.onCompleted()
}
})
})
}
def InputStream getTwitterStream() {
// code omitted
}
public static void main (String [] args) {
MyTwitterDataProvider provider = new MyTwitterDataProvider()
Observable<String> myTweetsObservable = provider.getTweets().take(3)
Subscription myTweetSubscription = myTweetsObservable.subscribe({tweet-> println("client prints: $tweet")})
// myTweetSubscription.unsubscribe()
}
}
You must add a check in your loop to see if the observer is still subscribed:
while ((newLine = reader.readLine()) != null && !observer.isUnsubsribed()) {
System.out.println("printing tweet: $newLine")
observer.onNext(newLine)
}
Related
RxJava2 has a doAfterNext operator that emits items downstream, and then invokes the consumer. It doesn't seem like Project Reactor has such an operator so I'd like to get some pointers on the best way to create my own to achieve the same thing.
The use case is freeing memory after the subscriber has received the item
Not sure if leavering doOnEach is a valid solution:
public class ByteBufferSafeReleaseConsumer implements Consumer<Signal<ByteBuffer<?>>> {
private final List<ByteBuffer<?>> elements = new ArrayList<>();
#Override
public void accept(Signal<ByteBuffer<?>> signal) {
if (signal.isOnNext()) {
ByteBuffer<?> next = signal.get();
if (next != null) {
elements.add(next);
}
}
if (signal.isOnComplete() || signal.isOnError()) {
for (ByteBuffer<?> buffer : elements) {
ByteBufferUtils.safeRelease(buffer);
}
}
}
}
ByteBufferSafeReleaseConsumer consumer = new ByteBufferSafeReleaseConsumer()
Flux.from(byteBufferPublisher).doOnEach(consumer)
I am creating a batch data streamer in apache ignite, and need to control what happening after data receive.
My batch has a structure:
public class Batch implements Binarylizable, Serializable {
private String eventKey;
private byte[] bytes;
etc..
Then i trying to stream my data:
try (IgniteDataStreamer<Integer, Batch> streamer = serviceGrid.getIgnite().dataStreamer(cacheName);
StreamBatcher batcher = StreamBatcherFactory.create(event) ){
streamer.receiver(StreamTransformer.from(new BatchDataProcessor(event)));
streamer.autoFlushFrequency(1000);
streamer.allowOverwrite(true);
statusService.updateStatus(event.getKey(), StatusType.EXECUTING);
int counter = 0;
Batch batch = null;
IgniteFuture<?> future = null;
while ((batch = batcher.batch()) != null) {
future = streamer.addData(counter++, batch);
}
Object getted = future.get();
Just for test use lets get only the last future, and try to analyze this object. In the code above I'm using BatchDataProcessor, that look like this:
public class BatchDataProcessor implements CacheEntryProcessor<Integer, Batch, Object> {
private final Event event;
private final String eventKey;
public BatchDataProcessor(Event event) {
this.event = event;
this.eventKey = event.getKey();
}
#Override
public Object process(MutableEntry<Integer, Batch> mutableEntry, Object... objects) throws EntryProcessorException {
Node node = NodeIgniter.node(Ignition.localIgnite().cluster().localNode().id());
ServiceGridContainer container = (ServiceGridContainer) node.getEnvironmentContainer().getContainerObject(ServiceGridContainer.class);
ProcessMarshaller marshaller = (ProcessMarshaller) container.getService(ProcessMarshaller.class);
LocalProcess localProcess = marshaller.intoProccessing(event.getLambdaExecutionKey());
try {
localProcess.addBatch(mutableEntry);
} catch (IOException e) {
e.printStackTrace();
} finally {
return new String("111");
}
}
}
So after localProcess.addBatch(mutableEntry) I want to send back an information about the status of this particular batch, so I think that I should do this in IgniteFuture object, but I don't find any information how to control the future object that's received in addData function.
Can anybody help with understanding, where can I control future that receives in addData function or some other way to realize a callback to streamed batch?
When you do StreamTransformer.from(), you forfeit the result of your BatchDataProcessor, because
for (Map.Entry<K, V> entry : entries)
cache.invoke(entry.getKey(), this, entry.getValue());
// ^ result of cache.invoke() is discarded here
DataStreamer is for one-directional streaming of data. It is not supposed to return values as far as I know.
If you depend on the result of cache.invoke(), I recommend calling it directly instead of relying on DataStreamer.
BTW, be careful with fut.get(). You should do dataStreamer.flush() first, or DataStreamer's futures will wait indefinitely.
I am developing in a Grails application. What I want to do is to lock the request/response, create a promise, and let someone else resolve it, that is somewhere else in the code, and then flush the response.
What I find really strange is that the Promise promise = task {} interface has no method that resembles resolve or similar.
I need to lock the response until someone resolves the promise, which is a global/static property set in development mode.
Promise interface:
http://grails.org/doc/latest/api/grails/async/Promise.html
I have looked at the GPars doc and can't find anything there that resembles a resolve method.
How can I create a promise, that locks the response or request, and then flushes the response when someone resolves it?
You can call get() on the promise which will block until whatever the task is doing completes, but I imagine what that is not what you want. What you want seems to be equivalent to a GPars DataflowVariable:
http://gpars.org/1.0.0/javadoc/groovyx/gpars/dataflow/DataflowVariable.html
Which allows using the left shift operator to resolve the value from another thread. Currently there is no way to use the left shift operator via Grails directly, but since Grails' promise API is just a layer over GPars this can probably be accomplished by using the GPars API directly with something like:
import org.grails.async.factory.gpars.*
import groovyx.gpars.dataflow.*
import static grails.async.Promise.*
def myAction() {
def dataflowVar = new DataflowVariable()
task {
// do some calculation and resolve data flow variable
def expensiveData = ...
dataflowVar << expensiveData
}
return new GParsPromise(dataflowVar)
}
It took me quite some time to get around this and have a working answer.
I must say that it appears as if Grails is quite a long way of making this work properly.
task { }
will always execute immediatly, so the call is not put on hold until dispatch() or whatever is invoked which is a problem.
Try this to see:
public def test() {
def dataflowVar = new groovyx.gpars.dataflow.DataflowVariable()
task {
// do some calculation and resolve data flow variable
println '1111111111111111111111111111111111111111111111111111'
//dataflowVar << expensiveData
}
return new org.grails.async.factory.gpars.GparsPromise(dataflowVar);
}
If you are wondering what this is for, it is to make the lesscss refresh automatically in grails, which is a problem when you are using import statements in less. When the file is touched, the lesscss compiler will trigger a recompilation, and only when it is done should it respond to the client.
On the client side I have some javascript that keeps replacing the last using the refresh action here:
In my controller:
/**
* Refreshes link resources. refresh?uri=/resource/in/web-app/such/as/empty.less
*/
public def refresh() {
return LessRefresh.stackRequest(request, params.uri);
}
A class written for this:
import grails.util.Environment
import grails.util.Holders
import javax.servlet.AsyncContext
import javax.servlet.AsyncEvent
import javax.servlet.AsyncListener
import javax.servlet.http.HttpServletRequest
/**
* #Author SecretService
*/
class LessRefresh {
static final Map<String, LessRefresh> FILES = new LinkedHashMap<String, LessRefresh>();
String file;
Boolean touched
List<AsyncContext> asyncContexts = new ArrayList<AsyncContext>();
String text;
public LessRefresh(String file) {
this.file = file;
}
/** Each request will be put on hold in a stack until dispatchAll below is called when the recompilation of the less file finished **/
public static AsyncContext stackRequest(HttpServletRequest request, String file) {
if ( !LessRefresh.FILES[file] ) {
LessRefresh.FILES[file] = new LessRefresh(file);
}
return LessRefresh.FILES[file].handleRequest(request);
}
public AsyncContext handleRequest(HttpServletRequest request) {
if ( Environment.current == Environment.DEVELOPMENT ) {
// We only touch it once since we are still waiting for the less compiler to finish from previous edits and recompilation
if ( !touched ) {
touched = true
touchFile(file);
}
AsyncContext asyncContext = request.startAsync();
asyncContext.setTimeout(10000)
asyncContexts.add (asyncContext);
asyncContext.addListener(new AsyncListener() {
#Override
void onComplete(AsyncEvent event) throws IOException {
event.getSuppliedResponse().writer << text;
}
#Override
void onTimeout(AsyncEvent event) throws IOException {
}
#Override
void onError(AsyncEvent event) throws IOException {
}
#Override
void onStartAsync(AsyncEvent event) throws IOException {
}
});
return asyncContext;
}
return null;
}
/** When recompilation is done, dispatchAll is called from LesscssResourceMapper.groovy **/
public void dispatchAll(String text) {
this.text = text;
if ( asyncContexts ) {
// Process all
while ( asyncContexts.size() ) {
AsyncContext asyncContext = asyncContexts.remove(0);
asyncContext.dispatch();
}
}
touched = false;
}
/** A touch of the lessfile will trigger a recompilation **/
int count = 0;
void touchFile(String uri) {
if ( Environment.current == Environment.DEVELOPMENT ) {
File file = getWebappFile(uri);
if (file && file.exists() ) {
++count;
if ( count < 5000 ) {
file << ' ';
}
else {
count = 0
file.write( file.getText().trim() )
}
}
}
}
static File getWebappFile(String uri) {
new File( Holders.getServletContext().getRealPath( uri ) )
}
}
In LesscssResourceMapper.groovy of the lesscsss-recources plugin:
...
try {
lessCompiler.compile input, target
// Update mapping entry
// We need to reference the new css file from now on
resource.processedFile = target
// Not sure if i really need these
resource.sourceUrlExtension = 'css'
resource.contentType = 'text/css'
resource.tagAttributes?.rel = 'stylesheet'
resource.updateActualUrlFromProcessedFile()
// ==========================================
// Call made here!
// ==========================================
LessRefresh.FILES[resource.sourceUrl.toString()]?.dispatchAll( target.getText() );
} catch (LessException e) {
log.error("error compiling less file: ${originalFile}", e)
}
...
In the index.gsp file:
<g:set var="uri" value="${"${App.files.root}App/styles/empty.less"}"/>
<link media="screen, projection" rel="stylesheet" type="text/css" href="${r.resource(uri:uri)}" refresh="${g.createLink(controller:'home', action:'refresh', params:[uri:uri])}" resource="true">
JavaScript method refreshResources to replace the previous link href=...
/**
* Should only be used in development mode
*/
function refreshResources(o) {
o || (o = {});
var timeoutBegin = o.timeoutBegin || 1000;
var intervalRefresh = o.intervalRefresh || 1000;
var timeoutBlinkAvoid = o.timeoutBlinkAvoid || 400 ;
var maxErrors = o.maxErrors || 200 ;
var xpath = 'link[resource][type="text/css"]';
// Find all link[resource]
$(xpath).each(function(i, element) {
refresh( $(element) );
});
function refresh(element) {
var parent = element.parent();
var next = element.next();
var outer = element.clone().attr('href', '').wrap('<p>').parent().html();
var uri = element.attr('refresh');
var errorCount = 0;
function replaceLink() {
var link = $(outer);
link.load(function () {
// The link has been successfully added! Now remove the other ones, then do again
errorCount = 0;
// setTimeout needed to avoid blinking, we allow duplicates for a few milliseconds
setTimeout(function() {
var links = parent.find(xpath + '[refresh="'+uri+'"]');
var i = 0;
// Remove all but this one
while ( i < links.length - 1 ) {
links[i++].remove();
}
replaceLinkTimeout();
}, timeoutBlinkAvoid );
});
link.error(function(event, handler) {
console.log('Error refreshing: ' + outer );
++errorCount;
if ( errorCount < maxErrors ) {
// Load error, it happens. Remove this & redo!
link.remove();
replaceLink();
}
else {
console.log('Refresh: Aborting!')
}
});
link.attr('href', urlRandom(uri)).get(0);
link.insertBefore(next); // Insert just after
}
function urlRandom(uri) {
return uri + "&rand=" + Math.random();
}
function replaceLinkTimeout() {
setTimeout(function() {
replaceLink();
}, intervalRefresh ) ;
}
// Waith 1s before triggering the interval
setTimeout(function() {
replaceLinkTimeout();
}, timeoutBegin);
}
};
Comments
I am unsure why Javascript style promises have not been added to the Grails stack.
You can not render or stuff like that in the onComplete. render, redirect and what not are not available.
Something tells me that Grails and Promises/Futures are not there yet. The design of the GPars libraries seems not take into account of the core features which is to resolve later. At least it is not simple to do so.
It would be great if the dispatch() method actually could be invoked with some paramaters to pass from the resolving context. I am able to go around this using static properties.
I might continue to write my own solution and possibly contribute with a more fitting solutions around the AsyncContext class, but for now, this is enough for me.
I just wanted to refresh my less resources automatically.
Phew...
EDIT:
I made it to support several number of files. It is complete now!
Hi friends i am trying to read incoming sms but getting warning like this . Invocation of questionable method: java.lang.String.(String) found in: mypackage.MyApp$ListeningThread.run()
Here is my code is
public class MyApp extends UiApplication {
//private ListeningThread listener;
public static void main(String[] args) {
MyApp theApp = new MyApp();
theApp.enterEventDispatcher();
}
public MyApp() {
invokeAndWait(new Runnable() {
public void run() {
ListeningThread listener = new ListeningThread();
listener.start();
}
});
pushScreen(new MyScreen());
}
private static class ListeningThread extends Thread {
private boolean _stop = false;
private DatagramConnection _dc;
public synchronized void stop() {
_stop = true;
try {
_dc.close(); // Close the connection so the thread returns.
} catch (IOException e) {
System.err.println(e.toString());
}
}
public void run() {
try {
_dc = (DatagramConnection) Connector.open("sms://");
for (;;) {
if (_stop) {
return;
}
Datagram d = _dc.newDatagram(_dc.getMaximumLength());
_dc.receive(d);
String address = new String(d.getAddress());
String msg = new String(d.getData());
if(msg.startsWith("START")){
Dialog.alert("hello");
}
System.out.println("Message received: " + msg);
System.out.println("From: " + address);
System.exit(0);
}
} catch (IOException e) {
System.err.println(e.toString());
}
}
}
}
Please correct me where i am wrong.Is possible give me some code to read incoming sms content in blackberry.
A few points about your code:
That invokeAndWait call to launch a thread makes no sense. It doesn't harm, but is kind of waste. Use that method only to perform UI related operations.
You should try using "sms://:0" as param for Connector.open. According to the docs, a parameter with the form {protocol}://[{host}]:[{port}] will open the connection in client mode (which makes sense, since you are on the receiving part), whereas not including the host part will open it in server mode.
Finally, if you can't get it working, you could use instead the third method specified in this tutorial, which you probably have already read.
The error you quoted is complaining about the use of the String constructor that takes a string argument. Since strings are immutable in Java-ME, this is just a waste. You can use the argument string directly:
Invocation of questionable method: java.lang.String.(String) found in: mypackage.MyApp$ListeningThread.run()
//String address = new String(d.getAddress());
String address = d.getAddress();
// getData() returns a byte[], so this is a different constructor
// However, this leaves the character encoding unspecified, so it
// will default to cp1252, which may not be what you want
String msg = new String(d.getData());
`hi
I am doing a simple synchronous socket programming,in which i employed twothreads
one for accepting the client and put the socket object into a collection,other thread will
loop through the collection and send message to each client through the socket object.
the problem is
1.i connect to clients to the server and start send messages
2.now i want to connect a new client,while doing this i cant update the collection and add
a new client to my hashtable.it raises an exception "collection modified .Enumeration operation may not execute"
how to add a NEW value without having problems in a hashtable.
private void Listen()
{
try
{
//lblStatus.Text = "Server Started Listening";
while (true)
{
Socket ReceiveSock = ServerSock.Accept();
//keys.Clear();
ConnectedClients = new ListViewItem();
ConnectedClients.Text = ReceiveSock.RemoteEndPoint.ToString();
ConnectedClients.SubItems.Add("Connected");
ConnectedList.Items.Add(ConnectedClients);
ClientTable.Add(ReceiveSock.RemoteEndPoint.ToString(), ReceiveSock);
//foreach (System.Collections.DictionaryEntry de in ClientTable)
//{
// keys.Add(de.Key.ToString());
//}
//ClientTab.Add(
//keys.Add(
}
//lblStatus.Text = "Client Connected Successfully.";
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
private void btn_receive_Click(object sender, EventArgs e)
{
Thread receiveThread = new Thread(new ThreadStart(Receive));
receiveThread.IsBackground = true;
receiveThread.Start();
}
private void Receive()
{
while (true)
{
//lblMsg.Text = "";
byte[] Byt = new byte[2048];
//ReceiveSock.Receive(Byt);
lblMsg.Text = Encoding.ASCII.GetString(Byt);
}
}
private void btn_Send_Click(object sender, EventArgs e)
{
Thread SendThread = new Thread(new ThreadStart(SendMsg));
SendThread.IsBackground = true;
SendThread.Start();
}
private void btnlist_Click(object sender, EventArgs e)
{
//Thread ListThread = new Thread(new ThreadStart(Configure));
//ListThread.IsBackground = true;
//ListThread.Start();
}
private void SendMsg()
{
while (true)
{
try
{
foreach (object SockObj in ClientTable.Keys)
{
byte[] Tosend = new byte[2048];
Socket s = (Socket)ClientTable[SockObj];
Tosend = Encoding.ASCII.GetBytes("FirstValue&" + GenerateRandom.Next(6, 10).ToString());
s.Send(Tosend);
//ReceiveSock.Send(Tosend);
Thread.Sleep(300);
}
}
catch (Exception ex)
{
MessageBox.Show(ex.Message);
}
}
}
You simply can't modify a Hashtable, Dictionary, List or anything similar while you're iterating over it - whether in the same thread or a different one. There are concurrent collections in .NET 4 which allow this, but I'm assuming you're not using .NET 4. (Out of interest, why are you still using Hashtable rather than a generic Dictionary?)
You also shouldn't be modifying a Hashtable from one thread while reading from it in another thread without any synchronization.
The simplest way to fix this is:
Create a new readonly variable used for locking
Obtain the lock before you add to the Hashtable:
lock (tableLock)
{
ClientTable.Add(ReceiveSock.RemoteEndPoint.ToString(), ReceiveSock);
}
When you want to iterate, create a new copy of the data in the Hashtable within a lock
Iterate over the copy instead of the original table
Do you definitely even need a Hashtable here? It looks to me like a simple List<T> or ArrayList would be okay, where each entry was either the socket or possibly a custom type containing the socket and whatever other information you need. You don't appear to be doing arbitrary lookups on the table.
Yes. Don't do that.
The bigger problem here is unsafe multi-threading.
The most basic "answer" is just to say: use a synchronization lock on the shared object. However this hides a number of important aspects (like understanding what is happening) and isn't a real solution to this problem in my mind.