RxJava cache last item for future subscribers - caching

I have implemented simple RxEventBus which starts emitting events, even if there is no subscribers. I want to cache last emitted event, so that if first/next subscriber subscribes, it receive only one (last) item.
I created test class which describes my problem:
public class RxBus {
ApplicationsRxEventBus applicationsRxEventBus;
public RxBus() {
applicationsRxEventBus = new ApplicationsRxEventBus();
}
public static void main(String[] args) {
RxBus rxBus = new RxBus();
rxBus.start();
}
private void start() {
ExecutorService executorService = Executors.newScheduledThreadPool(2);
Runnable runnable0 = () -> {
while (true) {
long currentTime = System.currentTimeMillis();
System.out.println("emiting: " + currentTime);
applicationsRxEventBus.emit(new ApplicationsEvent(currentTime));
try {
Thread.sleep(500);
} catch (InterruptedException e) {
e.printStackTrace();
}
}
};
Runnable runnable1 = () -> applicationsRxEventBus
.getBus()
.subscribe(new Subscriber<ApplicationsEvent>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable throwable) {
}
#Override
public void onNext(ApplicationsEvent applicationsEvent) {
System.out.println("runnable 1: " + applicationsEvent.number);
}
});
Runnable runnable2 = () -> applicationsRxEventBus
.getBus()
.subscribe(new Subscriber<ApplicationsEvent>() {
#Override
public void onCompleted() {
}
#Override
public void onError(Throwable throwable) {
}
#Override
public void onNext(ApplicationsEvent applicationsEvent) {
System.out.println("runnable 2: " + applicationsEvent.number);
}
});
executorService.execute(runnable0);
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
executorService.execute(runnable1);
try {
Thread.sleep(3000);
} catch (InterruptedException e) {
e.printStackTrace();
}
executorService.execute(runnable2);
}
private class ApplicationsRxEventBus {
private final Subject<ApplicationsEvent, ApplicationsEvent> mRxBus;
private final Observable<ApplicationsEvent> mBusObservable;
public ApplicationsRxEventBus() {
mRxBus = new SerializedSubject<>(BehaviorSubject.<ApplicationsEvent>create());
mBusObservable = mRxBus.cache();
}
public void emit(ApplicationsEvent event) {
mRxBus.onNext(event);
}
public Observable<ApplicationsEvent> getBus() {
return mBusObservable;
}
}
private class ApplicationsEvent {
long number;
public ApplicationsEvent(long number) {
this.number = number;
}
}
}
runnable0 is emitting events even if there is no subscribers. runnable1 subscribes after 3 sec, and receives last item (and this is ok). But runnable2 subscribes after 3 sec after runnable1, and receives all items, which runnable1 received. I only need last item to be received for runnable2. I have tried cache events in RxBus:
private class ApplicationsRxEventBus {
private final Subject<ApplicationsEvent, ApplicationsEvent> mRxBus;
private final Observable<ApplicationsEvent> mBusObservable;
private ApplicationsEvent event;
public ApplicationsRxEventBus() {
mRxBus = new SerializedSubject<>(BehaviorSubject.<ApplicationsEvent>create());
mBusObservable = mRxBus;
}
public void emit(ApplicationsEvent event) {
this.event = event;
mRxBus.onNext(event);
}
public Observable<ApplicationsEvent> getBus() {
return mBusObservable.doOnSubscribe(() -> emit(event));
}
}
But problem is, that when runnable2 subscribes, runnable1 receives event twice:
emiting: 1447183225122
runnable 1: 1447183225122
runnable 1: 1447183225122
runnable 2: 1447183225122
emiting: 1447183225627
runnable 1: 1447183225627
runnable 2: 1447183225627
I am sure, that there is RxJava operator for this. How to achieve this?

Your ApplicationsRxEventBus does extra work by reemitting a stored event whenever one Subscribes in addition to all the cached events.
You only need a single BehaviorSubject + toSerialized as it will hold onto the very last event and re-emit it to Subscribers by itself.

You are using the wrong interface. When you susbscribe to a cold Observable you get all of its events. You need to turn it into hot Observable first. This is done by creating a ConnectableObservable from your Observable using its publish method. Your Observers then call connect to start receiving events.
You can also read more about in the Hot and Cold observables section of the tutorial.

Related

Wiring an Observable to a SingleInterop in Vert.x

I'm having trouble figuring out how to correctly wire in an Observables onNext() call with a SingleInterop. I'm also not sure how to return the expected result without blocking the main thread. My rough attempt is shown below. Any help would be appreciated.
public class SomeClass {
private ExecutorService executor = Executors.newSingleThreadExecutor;
private ObservableOnSubscribe<MyCustomObj> disHandler;
public SomeClass() {
init();
}
/** Create an observable to listen to a data input stream **/
private void init() {
disHandler = emitter ->
executor.submit(() -> {
try {
while (true) {
MyCustomObj mco = readFromDataInputStream();
emitter.onNext(mco);
}
// Should never complete since always listening...
emitter.onComplete();
}
catch(Exception e) {
emitter.onError(e);
}
});
}
public Single<String> invokeSomethingAndGetResponseFromHandler(Object someObj) {
// Eventually the disHandler will send an onNext() as
// a result of doSomethingToBackend() call.
Observable<String> observer = Observable.fromCallable(doSomethingToBackend(someObj));
observer.subscribe(item -> item);
return ???.to(SingleInterop.get());
}
}
public class GraphQLVertx {
public VertxDataFetcher<Single<String>> dataFetcherTest() {
return new VertxDataFetcher<>((env, future) -> {
try {
future.complete(someClass.invokeSomethingAndGetResponseFromHandler("Blah");
}
catch(Exception e) {
future.fail();
}
});
}
}
The observable has a helper method that returns the first emitted element as a single, or an error, so if you already have an Observable<String> you can transform it to Single<String> like this:
public Single<String> invokeSomethingAndGetResponseFromHandler(Object someObj) {
return Observable.fromCallable(doSomethingToBackend(someObj))
.singleOrError();
}

Subscribers onnext does not contain complete item

We are working with project reactor and having a huge problem right now. This is how we produce (publish our data):
public Flux<String> getAllFlux() {
return Flux.<String>create(sink -> {
new Thread(){
public void run(){
Iterator<Cache.Entry<String, MyObject>> iterator = getAllIterator();
ObjectMapper mapper = new ObjectMapper();
while(iterator.hasNext()) {
try {
sink.next(mapper.writeValueAsString(iterator.next().getValue()));
} catch (IOException e) {
e.printStackTrace();
}
}
sink.complete();
}
} .start();
});
}
As you can see we are taking data from an iterator and are publishing each item in that iterator as a json string. Our subscriber does the following:
flux.subscribe(new Subscriber<String>() {
private Subscription s;
int amount = 1; // the amount of received flux payload at a time
int onNextAmount;
String completeItem="";
ObjectMapper mapper = new ObjectMapper();
#Override
public void onSubscribe(Subscription s) {
System.out.println("subscribe");
this.s = s;
this.s.request(amount);
}
#Override
public void onNext(String item) {
MyObject myObject = null;
try {
System.out.println(item);
myObject = mapper.readValue(completeItem, MyObject.class);
System.out.println(myObject.toString());
} catch (IOException e) {
System.out.println(item);
System.out.println("failed: " + e.getLocalizedMessage());
}
onNextAmount++;
if (onNextAmount % amount == 0) {
this.s.request(amount);
}
}
#Override
public void onError(Throwable t) {
System.out.println(t.getLocalizedMessage())
}
#Override
public void onComplete() {
System.out.println("completed");
});
}
As you can see we are simply printing the String item which we receive and parsing it into an object using jackson wrapper. The problem we got now is that for most of our items everything works fine:
{"itemId": "someId", "itemDesc", "some description"}
But for some items the String is cut off like this for example:
{"itemId": "some"
And the next item after that would be
"Id", "itemDesc", "some description"}
There is no pattern for those cuts. It is completely random and it is different everytime we run that code. Ofcourse our jackson is gettin an error Unexpected end of Input with that behaviour.
So what is causing such a behaviour and how can we solve it?
Solution:
Send the Object inside the flux instead of the String:
public Flux<ItemIgnite> getAllFlux() {
return Flux.create(sink -> {
new Thread(){
public void run(){
Iterator<Cache.Entry<String, ItemIgnite>> iterator = getAllIterator();
while(iterator.hasNext()) {
sink.next(iterator.next().getValue());
}
}
} .start();
});
}
and use the following produces type:
#RequestMapping(value="/allFlux", method=RequestMethod.GET, produces="application/stream+json")
The key here is to use stream+json and not only json.

Storm SQS messages not getting acked

I have a topology with 1 spout reading from 2 SQS queues and 5 bolts. After processing when i try to ack from second bolt it is not getting acked.
I'm running it in reliable mode and trying to ack in the last bolt. I get this message as if the messages are getting acked. But it is not getting deleted from the queue and the overwritten ack() methods are not getting called. It looks like it calls the default ack method in backtype.storm.task.OutputCollector instead of the overridden method in my spout.
8240 [Thread-24-conversionBolt] INFO backtype.storm.daemon.task - Emitting: conversionBolt__ack_ack [-7578372739434961741 -8189877254603774958]
I have anchored message ID to the tuple in my SQS queue spout and emitting to first bolt.
collector.emit(getStreamId(message), new Values(jsonObj.toString()), message.getReceiptHandle());
I have ack() and fail() methods overridden in my queue spout.Default Visibility Timeout has been set to 30 seconds
Code snippet from my topology:
TopologyBuilder builder = new TopologyBuilder();
builder.setSpout("firstQueueSpout",
new SqsQueueSpout(StormConfigurations.getQueueURL()
+ StormConfigurations.getFirstQueueName(), true),
StormConfigurations.getAwsQueueSpoutThreads());
builder.setSpout("secondQueueSpout",
new SqsQueueSpout(StormConfigurations.getQueueURL()
+ StormConfigurations.getSecondQueueName(),
true), StormConfigurations.getAwsQueueSpoutThreads());
builder.setBolt("transformerBolt", new TransformerBolt(),
StormConfigurations.getTranformerBoltThreads())
.shuffleGrouping("firstQueueSpout")
.shuffleGrouping("secondQueueSpout");
builder.setBolt("conversionBolt", new ConversionBolt(),
StormConfigurations.getTranformerBoltThreads())
.shuffleGrouping("transformerBolt");
// To dispatch it to the corresponding bolts based on packet type
builder.setBolt("dispatchBolt", new DispatcherBolt(),
StormConfigurations.getDispatcherBoltThreads())
.shuffleGrouping("conversionBolt");
Code snippet from SQSQueueSpout(extends BaseRichSpout):
#Override
public void nextTuple()
{
if (queue.isEmpty()) {
ReceiveMessageResult receiveMessageResult = sqs.receiveMessage(
new ReceiveMessageRequest(queueUrl).withMaxNumberOfMessages(10));
queue.addAll(receiveMessageResult.getMessages());
}
Message message = queue.poll();
if (message != null)
{
try
{
JSONParser parser = new JSONParser();
JSONObject jsonObj = (JSONObject) parser.parse(message.getBody());
// ack(message.getReceiptHandle());
if (reliable) {
collector.emit(getStreamId(message), new Values(jsonObj.toString()), message.getReceiptHandle());
} else {
// Delete it right away
sqs.deleteMessageAsync(new DeleteMessageRequest(queueUrl, message.getReceiptHandle()));
collector.emit(getStreamId(message), new Values(jsonObj.toString()));
}
}
catch (ParseException e)
{
LOG.error("SqsQueueSpout SQLException in SqsQueueSpout.nextTuple(): ", e);
}
} else {
// Still empty, go to sleep.
Utils.sleep(sleepTime);
}
}
public String getStreamId(Message message) {
return Utils.DEFAULT_STREAM_ID;
}
public int getSleepTime() {
return sleepTime;
}
public void setSleepTime(int sleepTime)
{
this.sleepTime = sleepTime;
}
#Override
public void ack(Object msgId) {
System.out.println("......Inside ack in sqsQueueSpout..............."+msgId);
// Only called in reliable mode.
try {
sqs.deleteMessageAsync(new DeleteMessageRequest(queueUrl, (String) msgId));
} catch (AmazonClientException ace) { }
}
#Override
public void fail(Object msgId) {
// Only called in reliable mode.
try {
sqs.changeMessageVisibilityAsync(
new ChangeMessageVisibilityRequest(queueUrl, (String) msgId, 0));
} catch (AmazonClientException ace) { }
}
#Override
public void close() {
sqs.shutdown();
((AmazonSQSAsyncClient) sqs).getExecutorService().shutdownNow();
}
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("message"));
}
Code snipped from my first Bolt(extends BaseRichBolt):
public class TransformerBolt extends BaseRichBolt
{
private static final long serialVersionUID = 1L;
public static final Logger LOG = LoggerFactory.getLogger(TransformerBolt.class);
private OutputCollector collector;
#Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
}
#Override
public void execute(Tuple input) {
String eventStr = input.getString(0);
//some code here to convert the json string to map
//Map datamap, long packetId being sent to next bolt
this.collector.emit(input, new Values(dataMap,packetId));
}
catch (Exception e) {
LOG.warn("Exception while converting AWS SQS to HashMap :{}", e);
}
}
#Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
declarer.declare(new Fields("dataMap", "packetId"));
}
}
Code snippet from second Bolt:
public class ConversionBolt extends BaseRichBolt
{
private static final long serialVersionUID = 1L;
private OutputCollector collector;
#Override
public void prepare(Map stormConf, TopologyContext context,
OutputCollector collector) {
this.collector = collector;
}
#Override
public void execute(Tuple input)
{
try{
Map dataMap = (Map)input.getValue(0);
Long packetId = (Long)input.getValue(1);
//this ack is not working
this.collector.ack(input);
}catch(Exception e){
this.collector.fail(input);
}
}
#Override
public void declareOutputFields(OutputFieldsDeclarer declarer) {
}
Kindly let me know if you need more information. Somebody shed some light on why the overridden ack in my spout is not getting called(from my second bolt)...
You must ack all incoming tuples in all bolts, ie, add collector.ack(input) to TransformerBolt.execute(Tuple input).
The log message you see is correct: your code calls collector.ack(...) and this call gets logged. A call to ack in your topology is not a call to Spout.ack(...): Each time a Spout emits a tuple with a message ID, this ID gets registered by the running ackers of your topology. Those ackers will get a message on each ack of a Bolt, collect those and notify the Spout if all acks of a tuple got received. If a Spout receives this message from an acker, it calls it's own ack(Object messageID) method.
See here for more details: https://storm.apache.org/documentation/Guaranteeing-message-processing.html

how to make and AsyncCallback deliver data before next method is called

I have a method that calls 2 services that make AsyncCallBacks
centroService.buscarCentroPorNombre(nombreCentroSeleccionado, new AsyncCallback<Centro>() {
#Override
public void onSuccess(Centro centro) {
cArticuloCentro.setIdCentro(centro.getIdCentro());
cArticuloCentro.setPrecio(Double.parseDouble(precioTextBox.getText()));
}
#Override
public void onFailure(Throwable caught) {
//do something
}
});
articuloService.buscarArticuloPorNombre(nombreArticuloSeleccionado, new AsyncCallback<Articulo>() { //se llama al sevivio para q busque el la base de datos la Entity por nombre
public void onSuccess(Articulo articulo) {
cArticuloCentro.setIdArticulo(articulo.getCod());
}
#Override
public void onFailure(Throwable caught) {
//do something
}
});
the problem comes when the next method is called
becouse these serviceCalls are asynchronous method activates before the calls are made, does not getting desired data. next method is
save(){
articuloCentroService.saveArticuloCentro(cArticuloCentro, new AsyncCallback<String>() {
#Override
public void onFailure(Throwable caught) {
//do something
}
#Override
public void onSuccess(String result) {
Window.alert("saved");
}
});
}
please can you tell me a way to make save() method execute when the asyncCallbacks have finished
thank you
In pure java you would need to synchronize threads, but in GWT, there's only one thread running at all times, so you can do a simply sync logic using an array:
final int[] sync = new int[1];
centroService.buscarCentroPorNombre(nombreCentroSeleccionado, new AsyncCallback<Centro>() {
#Override
public void onSuccess(Centro centro){
//...
if (++sync[0] == 2){
save();
}
}
#Override
public void onFailure(Throwable caught) {
//do something
}
});
articuloService.buscarArticuloPorNombre(nombreArticuloSeleccionado, new AsyncCallback<Articulo>() {
public void onSuccess(Articulo articulo) {
//...
if (++sync[0] == 2){
save();
}
}
#Override
public void onFailure(Throwable caught) {
//do something
}
});
Explanation: since there's only one thread running, the sync array will only be updated by one thread at a time. You can't control which method will finish first, but when they do, only one callback will be executed at a time. The sync array is just a counter you can use to sync any number of async invocations.

change android UI according to a background thread results

I'm developing an android app that requires to make UI changes according to a background thread processing results, I tried the following code at first:
Thread run_time = new Thread (){
public void run(){
ConnectToServer connect = new ConnectToServer(null);
while(true){
String server_response = connect.getServerResponse();
if(!server_response.equals(null)){
setResponse(server_response);
response_received();
}
}
}
};
run_time.start();
but my App crashes because i tried to make a UI changes from that background thread, then I tried that way:
runOnUiThread(new Runnable() {
public void run(){
ConnectToServer connect = new ConnectToServer(null);
while(true){
String server_response = connect.getServerResponse();
if(!server_response.equals(null)){
setResponse(server_response);
response_received();
}
}
}
});
but i got that exception:
01-29 16:42:17.045: ERROR/AndroidRuntime(605): android.os.NetworkOnMainThreadException
01-29 16:42:17.045: ERROR/AndroidRuntime(605): at android.os.StrictMode$AndroidBlockGuardPolicy.onNetwork(StrictMode.java:1084)
01-29 16:42:17.045: ERROR/AndroidRuntime(605): at libcore.io.BlockGuardOs.recvfrom(BlockGuardOs.java:151)
01-29 16:42:17.045: ERROR/AndroidRuntime(605): at libcore.io.IoBridge.recvfrom(IoBridge.java:503)
01-29 16:42:17.045: ERROR/AndroidRuntime(605): at java.net.PlainSocketImpl.read(PlainSocketImpl.java:488)
01-29 16:42:17.045: ERROR/AndroidRuntime(605): at java.net.PlainSocketImpl.access$000(PlainSocketImpl.java:46)
and after search i found that I must run the code as AsyncTask to avoid these problems, but when attempting to use it i found that it's must be used with small tasks only not like a thread that runs in the background all the run_time.
So, what's the best day to run a thread or a task in the background in whole the run_time and also reflect it's changes to the UI.
EDIT:
For Long running network work you have a few options.
First and formost check the android docs on this topic:
http://developer.android.com/training/basics/network-ops/index.html
Next, I generally use Services for this type of thing:
https://developer.android.com/guide/components/services.html
I will point you at the vogella tutorial for this as well:
http://www.vogella.com/tutorials/AndroidServices/article.html
For communication from threads/asynctasks/services to the UI use Handlers:
Use Handlers:
static public class MyThread extends Thread {
#Override
public void run() {
try {
// Simulate a slow network
try {
new Thread().sleep(5000);
} catch (InterruptedException e) {
e.printStackTrace();
}
downloadBitmap = downloadBitmap("http://www.devoxx.com/download/attachments/4751369/DV11");
// Updates the user interface
handler.sendEmptyMessage(0);
} catch (IOException e) {
e.printStackTrace();
} finally {
}
}
}
handler = new Handler() {
#Override
public void handleMessage(Message msg) {
// cal uiMethods here...
imageView.setImageBitmap(downloadBitmap);
// dialog.dismiss();
}
};
Taken from this tutorial:
http://www.vogella.com/tutorials/AndroidBackgroundProcessing/article.html
You can make this more interesting by defining constant_codes which corespond to the desired action:
private int DO_THIS = 0x0;
private int DO_THAT = 0x1;
// in your UI:
public void handleMessage(Message msg) {
// cal uiMethods here...
switch(msg.what()){
case(DO_THIS):
// do stuff
break;
case(DO_THAT):
// do other stuff
break;
}
}
// in your thread:
Message m = handler.obtainMessage(DO_THIS);
handler.sendMessage(m);
If the thread code (asynch task, service etc...) is separate from the UI you can use Broadcasts to pass the data between the two and then use Handlers from there to act on the UI thread.
you need to use handlers
here is documntation: https://developer.android.com/training/multiple-threads/communicate-ui.html
Use this code - it may contain compile time error you have to do it correct
public class MainActivity extends Activity {
#Override
public void onCreate(Bundle savedInstanceState) {
super.onCreate(savedInstanceState);
setContentView(R.layout.activity_main);
Connect connect = new Connect();
connect.execute();
}
class Connect extends AsyncTask<Void, String, Void>
{
#Override
protected Void doInBackground(Void... params)
{
ConnectToServer connect = new ConnectToServer(null);
while(true)
{
String server_response = connect.getServerResponse();
if(!server_response.equals(null))
{
publishProgress(server_response);
//setResponse(server_response);
response_received();
}
}
return null;
}
#Override
protected void onProgressUpdate(String... values) {
super.onProgressUpdate(values);
setResponse(values[0]);
}
#Override
protected void onPostExecute(Void result) {
super.onPostExecute(result);
}
}
}
You need "handlers" along with "loopers" for optimization
Example:
public void myMethod(){
Thread background = new Thread(new Runnable(){
#Override
public void run(){
Looper.prepare();
//Do your server process here
Runnable r=new Runnable() {
#Override
public void run() {
//update your UI from here
}
};
handler.post(r);
Looper.loop();
}
});
background.start();
}
And of course this is without using AsyncTask

Resources