I'm having trouble figuring out how to accomplish what seems to be a pretty standard pattern when doing a RANGE_ADD mutation.
Say on page load I pull in and render a connection chatmessages with first: 10 pagination. I now do a AddMessageMutation which does a prepend to the that same connection. Since the connection is paginated by first: 10 the last item of the connection is now gone to give room for my new edge and is thus removed from rendering. I can of course add +1 to first on the onSuccess of the mutation, but this often leaves a weird flickering effect of removing and reinserting the edge at the end.
This problem seem to get even more difficult if I want to do an optimistic update to the connection since there is no onOptimistic callback.
Since this seems like a pretty common pattern I figured I'd ask if I'm approaching this the wrong way.
Referenced in issue:
https://github.com/facebook/relay/issues/384
I think that the problem is that you're incrementing the count in the onSuccess handler (ie. after the server has responded) when what you want to do is to increment it in tandem with the optimistic mutation (ie. right away).
Try this:
_handleMessageCreated() {
Relay.Store.update(new AddMessageMutation(
{/* ... */},
{onFailure: () => this._handleMessageCreationRollback()}
);
// Optimistically increment the count
this.props.relay.setVariables({
numMessagesToShow: this.props.relay.variables.numMessagesToShow + 1,
});
}
_handleMessageCreationRollback() {
this.props.relay.setVariables({
numMessagesToShow: this.props.relay.variables.numMessagesToShow - 1,
});
}
See also: https://github.com/facebook/relay/issues/135#issuecomment-134400856
Related
I have the following epic I use in my application to handle api requests:
action$ => {
return action$.ofType(actions.requestType)
.do(() => console.log('handled epic ' + actions.requestType))
.switchMap((action) => (
Observable.create((obs) => {
obs.next({ type: type, value: action.value, form: action.form });
})
.debounceTime(250)
.switchMap((iea) => (
Observable.ajax(ajaxPost(url(iea.value), body ? body(iea.value) : action.form))
.mergeMap(payload => {
return Observable.merge(
Observable.of(actions.success(payload)),
/* some other stuff */
);
})
.catch(payload => {
return [actions.failure(payload)];
})
))
))
.takeUntil(action$.filter((a) => (a.type === masterCancelAction))
.repeat();
};
Basically, any time I perform an api request, I dispatch a request action. If I dispatch another request quickly, the previous one is ignored using debounceTime. Additionally, the request can be cancelled using the masterCancelAction and when cancelled repeat() restarts the epic. This epic works as intended in all cases expect one.
The failure case occurs when a user uses the browser back during a request. In this case I fire the masterCancelAction to the request. However, on the same execution context as a result from the masterCancelAction, another request action dispatches to perform a new request on the same epic, but the api request does not occur (the console.log does occur though) as if there was no repeat(). In other cases where cancels occur, the next request is not invoked from the same execution context and it works fine, so it seems in this case my code does not give repeat a chance to restart the epic?
A dirty workaround I found was to use setTimeout(dispatch(action), 0) on the request that dispatches after the cancellation. This seems to allow repeat() to execute. I tried passing different schedulers into repeat, but that didn't seem to help. Also, attaching takeUntil and repeat into my inner switchMap solves the problem, but then other cases where my next request does not execute in the same call stack fail.
Is there a way I can solve this problem without using setTimeout? Maybe it is not a repeat related problem, but it seems to be the case.
Using rxjs 5.0.3 and redux-observable 0.14.1.
The issue is not 100% clear without something like a jsbin to see what you mean, but I do see some general issues that might help:
Anonymous Observable never completes
When creating a custom anonymous Observable it's important to call observer.complete() if you do indeed want it to complete. In most cases, not doing so will cause the subscription to be a memory leak and might also other strange behaviors
Observable.create((observer) => {
observer.next({ type: type, value: action.value, form: action.form });
observer.complete();
})
Observable.of would have been equivalent:
Observable.of({ type: type, value: action.value, form: action.form })
However, it's not clear why this was done as the values it emits are in captured in scope.
debounceTime in this case does not debounce, it delays
Since the anonymous observable it's applied to only ever emits a single item, debounceTime will act just as a regular .delay(250). I'm betting you intended instead to debounce actions.requestType actions, in which case you'd need to apply your debouncing outside the switchMap, after the action$.ofType(actions.requestType).
Observable.of accepts any number of arguments to emit
This is more of a "did you know?" rather than an issue, but I noticed you're merging your of and /* some other actions */ I assume would be other of observables merged in. Instead, you can just return a single of and pass the actions as arguments.
Observable.of(
actions.success(payload),
/* some other actions */
actions.someOtherOne(),
actions.etc()
);
Also, when you find yourself emitting multiple actions synchronously like this, consider whether your reducers should be listening for the same, single action instead of having two or more. Sometimes this wouldn't make sense as you want them to have completely unrelated actions, just something to keep in mind that people often forget--that all reducers receive all actions and so multiple reducers can change their state from the same action.
.takeUntil will stop the epic from listening for future actions
Placing the takeUntil on the top-level observable chain causes the epic to stop listening for action$.ofType(actions.requestType), which is why you added the .repeat() after. This might work in some cases, but it's inefficient and can cause other hard to realize bugs. Epics should be thought of instead as sort of like sidecar processes that usually "start up" with the app and then continue listening for a particular action until the app "shuts down" aka the user leaves the app. They aren't actually processes, it's just helpful to conceptually think of them this way as an abstraction.
So each time it matches its particular action it then most often will switchMap, mergeMap, concatMap, or exhaustMap into some side effect, like an ajax call. That inner observable chain is what you want to make cancellable. So you'd place your .takeUntil on it, at the appropriate place in the chain.
Summary
As mentioned, it's not clear what you intended to do and what the issue is, without a more complete example like a jsbin. But strictly based on the code provided, this is my guesstimate:
const someRequestEpic = action$ => {
return action$.ofType(actions.requestType)
.debounceTime(250)
.do(() => console.log('handled epic ' + actions.requestType))
.switchMap((action) =>
Observable.ajax(ajaxPost(url(action.value), body ? body(action.value) : action.form))
.takeUntil(action$.ofType(masterCancelAction))
.mergeMap(payload => {
return Observable.of(
actions.success(payload),
/* some other actions */
...etc
);
})
.catch(payload => Observable.of(
actions.failure(payload)
))
);
};
Check out the Cancellation page in the redux-observable docs.
If this is a bit confusing, I'd recommend digging a bit deeper into what Observables are and what an "operator" is and does so that it doesn't feel magical and where you should place an operator makes more sense.
Ben's post on Learning Observable by Building Observable is a good start.
I'm working on something that is recording data coming from a queue. It was easy enough to process the queue into an Observable so that I can have multiple endpoints in my code receiving the information in the queue.
Furthermore, I can be sure that the information arrives in order. That bit works nicely as well since the Observables ensure that. But, one tricky bit is that I don't want the Observer to be notified of the next thing until it has completed processing the previous thing. But the processing done by the Observer is asynchronous.
As a more concrete example that is probably simple enough to follow. Imagine my queue contains URLs. I'm exposing those as an Observable in my code. The I subscribe an Observer whose job is to fetch the URLs and write the content to disk (this is a contrived example, so don't take issue with these specifics). The important point is that fetching and saving are async. My problem is that I don't want the observer to be given the "next" URL from the Observable until they have completed the previous processing.
But the call to next on the Observer interface returns void. So there is no way for the Observer to communicate back to me that has actually completed the async task.
Any suggestions? I suspect there is probably some kind of operator that could be coded up that would basically withhold future values (queue them up in memory?) until it somehow knew the Observer was ready for it. But I was hoping something like that already existed following some established pattern.
similar use case i ran into before
window.document.onkeydown=(e)=>{
return false
}
let count=0;
let asyncTask=(name,time)=>{
time=time || 2000
return Rx.Observable.create(function(obs) {
setTimeout(function() {
count++
obs.next('task:'+name+count);
console.log('Task:',count ,' ', time, 'task complete')
obs.complete();
}, time);
});
}
let subject=new Rx.Subject()
let queueExec$=new Rx.Subject()
Rx.Observable.fromEvent(btnA, 'click').subscribe(()=>{
queueExec$.next(asyncTask('A',4000))
})
Rx.Observable.fromEvent(btnB, 'click').subscribe(()=>{
queueExec$.next(asyncTask('B',4000))
})
Rx.Observable.fromEvent(btnC, 'click').subscribe(()=>{
queueExec$.next(asyncTask('C',4000))
})
queueExec$.concatMap(value=>value)
.subscribe(function(data) {
console.log('onNext', data);
},
function(error) {
console.log('onError', error);
},function(){
console.log('completed')
});
What you describe sounds like "backpressure". You can read about it in RxJS 4 documentation https://github.com/Reactive-Extensions/RxJS/blob/master/doc/gettingstarted/backpressure.md. However this is mentioning operators that don't exist in RxJS 5. For example have a look at "Controlled Observables" that should refer to what you need.
I think you could achieve the same with concatMap and an instance of Subject:
const asyncOperationEnd = new Subject();
source.concatMap(val => asyncOperationEnd
.mapTo(void 0)
.startWith(val)
.take(2) // that's `val` and the `void 0` that ends this inner Observable
)
.filter(Boolean) // Always ignore `void 0`
.subscribe(val => {
// do some async operation...
// call `asyncOperationEnd.next()` and let `concatMap` process another value
});
Fro your description it actually seems like the "observer" you're mentioning works like Subject so it would make maybe more sense to make a custom Subject class that you could use in any Observable chain.
Isn't this just concatMap?
// Requests are coming in a stream, with small intervals or without any.
const requests=Rx.Observable.of(2,1,16,8,16)
.concatMap(v=>Rx.Observable.timer(1000).mapTo(v));
// Fetch, it takes some time.
function fetch(query){
return Rx.Observable.timer(100*query)
.mapTo('!'+query).startWith('?'+query);
}
requests.concatMap(q=>fetch(q));
https://rxviz.com/v/Mog1rmGJ
If you want to allow multiple fetches simultaneously, use mergeMap with concurrency parameter.
In my frontend I have an input-field that sends an ajax request on every character typed in (using vue.js) to get realtime-filtering (can't use vue filter because of pagination).
Everything works smooth in my test environment, but could this lead to performance issues on (a bigger amount of) real data and if so, what can I do to prevent this?
Is it problematic?
Yes.
The client will send a lot of requests. Depending on the network connection and browser, this could lead to a perceptible feeling of lag by the client.
The server will receive a lot of requests, potentially leading to degraded performance for all clients, and extra usage of resources on the server side.
Responses to requests have a higher chance of arriving out of order. If you send requests very fast, it has increased chances of being apparent (e.g. displaying autocomplete for "ab" when the user has already typed "abc")
Overall, it's bad practice mostly because it's not necessary to do that many requests.
How to fix it?
As J. B. mentioned in his answer, debouncing is the way to go.
The debounce function (copied below) ensures that a certain function doesn't get called more than once every X milliseconds. Concretely, it allows you to send a request as soon as the user hasn't typed anything for, say, 200ms.
Here's a complete example (try typing text very fast in the input):
function debounce(func, wait, immediate) {
var timeout;
return function() {
var context = this, args = arguments;
var later = function() {
timeout = null;
if (!immediate) func.apply(context, args);
};
var callNow = immediate && !timeout;
clearTimeout(timeout);
timeout = setTimeout(later, wait);
if (callNow) func.apply(context, args);
};
}
var sendAjaxRequest = function(inputText) {
// do your ajax request here
console.log("sent via ajax: " + inputText);
};
var sendAjaxRequestDebounced = debounce(sendAjaxRequest, 200, false); // 200ms
var el = document.getElementById("my-input");
el.onkeyup = function(evt) {
// user pressed a key
console.log("typed: " + this.value)
sendAjaxRequestDebounced(this.value);
}
<input type="text" id="my-input">
For more details on how the debounce function works, see this question
I actually discuss this exact scenario in my Vue.js training course. In short, you may want to wait until a user clicks a button or something of that nature to trigger sending the request. Another approach to consider is to use the lazy modifier, which will delay the event until the change event is fired.
It's hard to know the correct approach without knowing more about the goals of the app. Still, the options listed above are two options to consider.
I hope this helps.
The mechanism I was searching for is called debouncing.
I used this approach in the application.
I'm using the Select2 select boxes in my Django project. The ajax calls it makes can be fairly time-consuming if you've only entered a character or two in the query box, but go quicker if you've entered several characters. So what I'm seeing is you'll start typing a query, and it will make 4 or 5 ajax calls, but the final one returns and the results display. It looks fine on the screen, but meanwhile, the server is still churning away on the earlier queries. I've increased the "delay" parameter to 500 ms, but it's still a bit of a problem.
Is there a way to have the AJAX handler on the server detect that this is a new request from the same client as one that is currently processing, and tell the older one to exit immediately? It appears from reading other answers here that merely calling .abort() on the client side doesn't stop the query running on the server side.
If they are DB queries that are taking up time, then basically nothing will stop them besides stopping the database server, which is of course not tangible. If it is computation in nested loops for example, then you could use cache to detect whether another request has been submitted from the same user. Basically:
from django.core.cache import cache
def view(request):
start_time = timestamp # timezone.now() etc.
cache.set(request.session.session_key + 'some_identifier', start_time)
for q in werty:
# Very expensive computation with millions of loops
if start_time != cache.get(request.session.session_key + 'some_identifier'):
break
else:
# Continue the nasty computations
else:
cache.delete(request.session.session_key + 'some_identifier')
But the Django part aside - what I would do: in JS add a condition that when the search word is less than 3 chars, then it waits 0.5s (or less, whatever you like) before searching. And if another char is added then search right away.
I.e.
var timeout;
function srch(param) {
timeout = false;
if (param.length < 3) {
timeout = true;
setTimeout(function () {
if (timeout) {
$.ajax({blah: blah});
}
}, 500);
} else {
$.ajax({blah: blah});
}
}
I'm testing out CouchDB to see how it could handle logging some search results. What I'd like to do is produce a view where I can produce the top queries from the results. At the moment I have something like this:
Example document portion
{
"query": "+dangerous +dogs",
"hits": "123"
}
Map function
(Not exactly what I need/want but it's good enough for testing)
function(doc) {
if (doc.query) {
var split = doc.query.split(" ");
for (var i in split) {
emit(split[i], 1);
}
}
}
Reduce Function
function (key, values, rereduce) {
return sum(values);
}
Now this will get me results in a format where a query term is the key and the count for that term on the right, which is great. But I'd like it ordered by the value, not the key. From the sounds of it, this is not yet possible with CouchDB.
So does anyone have any ideas of how I can get a view where I have an ordered version of the query terms & their related counts? I'm very new to CouchDB and I just can't think of how I'd write the functions needed.
It is true that there is no dead-simple answer. There are several patterns however.
http://wiki.apache.org/couchdb/View_Snippets#Retrieve_the_top_N_tags. I do not personally like this because they acknowledge that it is a brittle solution, and the code is not relaxing-looking.
Avi's answer, which is to sort in-memory in your application.
couchdb-lucene which it seems everybody finds themselves needing eventually!
What I like is what Chris said in Avi's quote. Relax. In CouchDB, databases are lightweight and excel at giving you a unique perspective of your data. These days, the buzz is all about filtered replication which is all about slicing out subsets of your data to put in a separate DB.
Anyway, the basics are simple. You take your .rows from the view output and you insert it into a separate DB which simply emits keyed on the count. An additional trick is to write a very simple _list function. Lists "render" the raw couch output into different formats. Your _list function should output
{ "docs":
[ {..view row1...},
{..view row2...},
{..etc...}
]
}
What that will do is format the view output exactly the way the _bulk_docs API requires it. Now you can pipe curl directly into another curl:
curl host:5984/db/_design/myapp/_list/bulkdocs_formatter/query_popularity \
| curl -X POST host:5984/popularity_sorter/_design/myapp/_view/by_count
In fact, if your list function can handle all the docs, you may just have it sort them itself and return them to the client sorted.
This came up on the CouchDB-user mailing list, and Chris Anderson, one of the primary developers, wrote:
This is a common request, but not supported directly by CouchDB's
views -- to do this you'll need to copy the group-reduce query to
another database, and build a view to sort by value.
This is a tradeoff we make in favor of dynamic range queries and
incremental indexes.
I needed to do this recently as well, and I ended up doing it in my app tier. This is easy to do in JavaScript:
db.view('mydesigndoc', 'myview', {'group':true}, function(err, data) {
if (err) throw new Error(JSON.stringify(err));
data.rows.sort(function(a, b) {
return a.value - b.value;
});
data.rows.reverse(); // optional, depending on your needs
// do something with the data…
});
This example runs in Node.js and uses node-couchdb, but it could easily be adapted to run in a browser or another JavaScript environment. And of course the concept is portable to any programming language/environment.
HTH!
This is an old question but I feel it still deserves a decent answer (I spent at least 20 minutes on searching for the correct answer...)
I disapprove of the other suggestions in the answers here and feel that they are unsatisfactory. Especially I don't like the suggestion to sort the rows in the applicative layer, as it doesn't scale well and doesn't deal with a case where you need to limit the result set in the DB.
The better approach that I came across is suggested in this thread and it posits that if you need to sort the values in the query you should add them into the key set and then query the key using a range - specifying a desired key and loosening the value range. For example if your key is composed of country, state and city:
emit([doc.address.country,doc.address.state, doc.address.city], doc);
Then you query just the country and get free sorting on the rest of the key components:
startkey=["US"]&endkey=["US",{}]
In case you also need to reverse the order - note that simple defining descending: true will not suffice. You actually need to reverse the start and end key order, i.e.:
startkey=["US",{}]&endkey=["US"]
See more reference at this great source.
I'm unsure about the 1 you have as your returned result, but I'm positive this should do the trick:
emit([doc.hits, split[i]], 1);
The rules of sorting are defined in the docs.
Based on Avi's answer, I came up with this Couchdb list function that worked for my needs, which is simply a report of most-popular events (key=event name, value=attendees).
ddoc.lists.eventPopularity = function(req, res) {
start({ headers : { "Content-type" : "text/plain" } });
var data = []
while(row = getRow()) {
data.push(row);
}
data.sort(function(a, b){
return a.value - b.value;
}).reverse();
for(i in data) {
send(data[i].value + ': ' + data[i].key + "\n");
}
}
For reference, here's the corresponding view function:
ddoc.views.eventPopularity = {
map : function(doc) {
if(doc.type == 'user') {
for(i in doc.events) {
emit(doc.events[i].event_name, 1);
}
}
},
reduce : '_count'
}
And the output of the list function (snipped):
165: Design-Driven Innovation: How Designers Facilitate the Dialog
165: Are Your Customers a Crowd or a Community?
164: Social Media Mythbusters
163: Don't Be Afraid Of Creativity! Anything Can Happen
159: Do Agencies Need to Think Like Software Companies?
158: Customer Experience: Future Trends & Insights
156: The Accidental Writer: Great Web Copy for Everyone
155: Why Everything is Amazing But Nobody is Happy
Every solution above will break couchdb performance I think. I am very new to this database. As I know couchdb views prepare results before it's being queried. It seems we need to prepare results manually. For example each search term will reside in database with hit counts. And when somebody searches, its search terms will be looked up and increments hit count. When we want to see search term popularity, it will emit (hitcount, searchterm) pair.
The Link Retrieve_the_top_N_tags seems to be broken, but I found another solution here.
Quoting the dev who wrote that solution:
rather than returning the results keyed by the tag in the map step, I would emit every occurrence of every tag instead. Then in the reduce step, I would calculate the aggregation values grouped by tag using a hash, transform it into an array, sort it, and choose the top 3.
As stated in the comments, the only problem would be in case of a long tail:
Problem is that you have to be careful with the number of tags you obtain; if the result is bigger than 500 bytes, you'll have couchdb complaining about it, since "reduce has to effectively reduce". 3 or 6 or even 20 tags shouldn't be a problem, though.
It worked perfectly for me, check the link to see the code !