angular - how costly are pipes for more complex transformations - performance

I have an array of tasks and "taskstore" which looks like this
tasks:Task[]=[...]
interface Task{
title:string,
deadline:Date
}
taskStore$ = new BehaviourSubject<Task[]>(this.tasks)
Now I want to display the tasklist with multiple filter options.
Sorting after the deadline is quite straightforward e.g.
<div *ngFor="let task in tasks | sortyByDay='ascending'"></<div>
In the pipe using underscore.js (and ignoring the order option here)
transform(tasks:Task[]): any {
return _.sortBy(tasks[])
}
Now I want to transform the simple tasklist into something like
taskDaylist:Taskdaylist=[...]
interface TaskdayList{
day:Date;
tasks:Task[]
}
In order to be able to display lists with list header which sums up the tasks of a day (like number of tasks, or total duration like in toggle)
This would involve multiple sorting and mappings inside the pipe.
I would like to know how it will affect the performance if I'd do this inside a pipe, instead of e.g. a service and then .next() the result.
The pipe is obviously more reusable more easy to implement than a service.

Related

How to keep derived state up to date with Apollo/GraphQL?

My situation is this: I have multiple components in my view that ultimately depend on the same data, but in some cases the view state is derived from the data. How do I make sure my whole view stays in sync when the underlying data changes? I'll illustrate with an example using everyone's favorite Star Wars API.
First, I show a list of all the films, with a query like this:
# ALL_FILMS
query {
allFilms {
id
title
releaseDate
}
}
Next, I want a separate component in the UI to highlight the most recent film. There's no query for that, so I'll implement it with a client-side resolver. The query would be:
# MOST_RECENT_FILM
query {
mostRecentFilm #client {
id
title
}
}
And the resolver:
function mostRecentFilmResolver(parent, variables, context) {
return context.client.query({ query: ALL_FILMS }).then(result => {
// Omitting the implementation here since it's not relevant
return deriveMostRecentFilm(result.data);
})
}
Now, where it gets interesting is when SWAPI gets around to adding The Last Jedi and The Rise of Skywalker to its film list. We can suppose I'm polling on the list so that it gets periodically refetched. That's great, now my list UI is up to date. But my "most recent film" UI isn't aware that anything has changed — it's still stuck in 2015 showing The Force Awakens, even though the user can clearly see there are newer films.
Maybe I'm spoiled; I come from the world of MobX where stuff like this Just Works™. But this doesn't feel like an uncommon problem. Is there a best practice in the realm of Apollo/GraphQL for keeping things in sync? Am I approaching this problem in entirely the wrong way?
A few ideas I've had:
My "most recent film" query could also poll periodically. But you don't want to poll too often; after all, Star Wars films only come out every other year or so. (Thanks, Disney!) And depending on how the polling intervals overlap there will still be a big window where things are out of sync.
Instead putting the deriveMostRecentFilm logic in a resolver, just put it in the component and share the ALL_FILMS query between components. That would work, but that's basically answering "How do I get this to work in Apollo?" with "Don't use Apollo."
Some complicated system of keeping track of the dependencies between queries and chaining refreshes based on that. (I'm not keen to invent this if I can avoid it!)
In Apollo observables are (in components) over queried values (cached data 'slots') but your mostRecentFilm is not an observable, is not based on cached values (they are cached) but on one time fired query result (updated on demand).
You're only missing an 'updating connection', f.e. like this:
# ALL_FILMS
query {
allFilms {
id
title
releaseDate
isMostRecentFilm #client
}
}
Use isMostRecentFilm local resolver to update mostRecentFilm value in cache.
Any query (useQuery) related to mostRecentFilm #client will be updated automatically. All without additional queries, polling etc. - Just Works? (not tested, it should work) ;)

Subquery, for lack of a better term, when using an API written in GraphQL

I'm relatively new to GraphQL so please bear with me ...
That said, I'm writing an app in node.js to push/pull data from two disparate systems, one of which has an API written in GraphQL.
For the Graph system, I have, something like, the following types defined for me:
Time {
TimeId: Int
TaskId: Int
ProjectId: Int
Project: [Project]
TimeInSeconds: Int
Timestamp: Date
}
and
Task {
TaskId: Int
TaskName: String
TaskDescription: String
}
Where Project is another type whose definition isn't important, only that it is included in the type definition as a field...
What I would like to know is if there is a way to write a query for Time in such a way that I can include the Task type's values in my results in a similar way as the values for the Project type are included in the definition?
I am using someone else's API and do not have the ability to define my own custom types. I can write my own limited queries, but I don't know if the limits are set by the devs that wrote the API or my limited ability with GraphQL.
My suspicion is that I cannot and that I will have to query both separately and combine them after the fact, but I wanted to check here just in case.
Unfortunately, unless the Time type exposes some kind of field to fetch the relevant Task, you won't be able to query for it within the same request. You can include multiple queries within a single GraphQL request; however, they are ran in parallel, which means you won't be able to use the TaskId value returned by one query as a variable used in another query.
This sort of problem is best solved by modifying the schema, but if that's not an option then unfortunately the only other option is to make each request sequentially and then combine the results client-side.

graphql- same query with different arguments

Can the below be achieved with graph ql:
we have getusers() / getusers(id=3) / getusers(name='John). Can we use same query to accept different parameters (arguments)?
I assume you mean something like:
type Query {
getusers: [User]!
getusers(id: ID!): User
getusers(name: String!): User
}
IMHO the first thing to do is try. You should get an error saying that Query.getusers can only be defined once, which would answer your question right away.
Here's the actual spec saying that such a thing is not valid: http://facebook.github.io/graphql/June2018/#example-5e409
Quote:
Each named operation definition must be unique within a document when
referred to by its name.
Solution
From what I've seen, the most GraphQL'y way to create such an API is to define a filter input type, something like this:
input UserFilter {
ids: [ID]
names: [String]
}
and then:
type Query {
users(filter: UserFilter)
}
The resolver would check what filters were passed (if any) and query the data accordingly.
This is very simple and yet really powerful as it allows the client to query for an arbitrary number of users using an arbitrary filter. As a back-end developer you may add more options to UserFilter later on, including some pagination options and other cool things, while keeping the old API intact. And, of course, it is up to you how flexible you want this API to be.
But why is it like that?
Warning! I am assuming some things here and there, and might be wrong.
GraphQL is only a logical API layer, which is supposed to be server-agnostic. However, I believe that the original implementation was in JavaScript (citation needed). If you then consider the technical aspects of implementing a GraphQL API in JS, you might get an idea about why it is the way it is.
Each query points to a resolver function. In JS resolvers are simple functions stored inside plain objects at paths specified by the query/mutation/subscription name. As you may know, JS objects can't have more than one path with the same name. This means that you could only define a single resolver for a given query name, thus all three getusers would map to the same function Query.getusers(obj, args, ctx, info) anyway.
So even if GraphQL allowed for fields with the same name, the resolver would have to explicitly check for whatever arguments were passed, i.e. if (args.id) { ... } else if (args.name) { ... }, etc., thus partially defeating the point of having separate endpoints. On the other hand, there is an overall better (particularly from the client's perspective) way to define such an API, as demonstrated above.
Final note
GraphQL is conceptually different from REST, so it doesn't make sense to think in terms of three endpoints (/users, /users/:id and /users/:name), which is what I guess you were doing. A paradigm shift is required in order to unveil the full potential of the language.
a request of the type works:
Query {
first:getusers(),
second:getusers(id=3)
third:getusers(name='John)
}

Reading a large batch of objects taking way too long

I’m experimenting with scripting a batch of OmniFocus tasks in JXA and running into some big speed issues. I don't think the problem is specific to OmniFocus or JXA; rather I think this is a more general misunderstanding of how getting objects works - I'm expecting it to work like a single SQL query that loads all objects in memory but instead it seems to do each operation on demand.
Here’s a simple example - let’s get the names of all uncompleted tasks (which are stored in a SQLite DB on the backend):
var tasks = Application('OmniFocus').defaultDocument.flattenedTasks.whose({completed: false})
var totalTasks = tasks.length
for (var i = 0; i < totalTasks; i++) {
tasks[i].name()
}
[Finished in 46.68s]
Actually getting the list of 900 tasks takes ~7 seconds - already slow - but then looping and reading basic properties takes another 40 seconds, presumably because it's hitting the DB for each one. (Also, tasks doesn't behave like an array - it seems to be recomputed every time it's accessed.)
Is there any way to do this quickly - to read a batch of objects and all their properties into memory at once?
Introduction
With AppleEvents, the IPC technology that JavaScript for Automation (JXA) is built upon, the way you request information from another application is by sending it an "object specifier," which works a little bit like dot notation for accessing object properties, and a little bit like a SQL or GraphQL query.
The receiving application evaluates the object specifier and determines which objects, if any, it refers to. It then returns a value representing the referred-to objects. The returned value may be a list of values, if the referred-to object was a collection of objects. The object specifier may also refer to properties of objects. The values returned may be strings, or numbers, or even new object specifiers.
Object specifiers
An example of a fully-qualified object specifier written in AppleScript is:
a reference to the name of the first window of application "Safari"
In JXA, that same object specifier would be expressed:
Application("Safari").windows[0].name
To send an IPC request to Safari to ask it to evaluate this object specifier and respond with a value, you can invoke the .get() function on an object specifier:
Application("Safari").windows[0].name.get()
As a shorthand for the .get() function, you can invoke the object specifier directly:
Application("Safari").windows[0].name()
A single request is sent to Safari, and a single value (a string in this case) is returned.
In this way, object specifiers work a little bit like dot notation for accessing object properties. But object specifiers are much more powerful than that.
Collections
You can effectively perform maps or comprehensions over collections. In AppleScript this looks like:
get the name of every window of Application "Safari"
In JXA it looks like:
Application("Safari").windows.name.get()
Even though this requests multiple values, it requires only a single request to be sent to Safari, which then iterates over its own windows, collecting the name of each one, and then sends back a single list value containing all of the name strings. No matter how many windows Safari has open, this statement only results in a single request/response.
For-loop anti-pattern
Contrast that approach to the for-loop anti-pattern:
var nameOfEveryWindow = []
var everyWindowSpecifier = Application("Safari").windows
var numberOfWindows = everyWindowSpecifier.length
for (var i = 0; i < numberOfWindows; i++) {
var windowNameSpecifier = everyWindowSpecifier[i].name
var windowName = windowNameSpecifier.get()
nameOfEveryWindow.push(windowName)
}
This approach may take much longer, as it requires length+1 number of requests to get the collection of names.
(Note that the length property of collection object specifiers is handled specially, because collection object specifiers in JXA attempt to behave like native JavaScript Arrays. No .get() invocation is needed (or allowed) on the length property.)
Filtering, and why your code example is slow
The really interesting part of AppleEvents is the so-called "whose clause". This allows you provide criteria with which to filter the objects from which the values will be returned from.
In the code you included in your question, tasks is an object specifier that refers to a collection of objects that have been filtered to only include uncompleted tasks using a whose clause. Note that this is still just reference at this point; until you call .get() on the object specifier, it's just a pointer to something, not the thing itself.
The code you included then implements the for-loop anti-pattern, which is probably why your observed performance is so slow. You are sending length+1 requests to OmniFocus. Each invocation of .name() results in another AppleEvent.
Furthermore, you're asking OmniFocus to re-filter the collection of tasks every time, because the object specifier you're sending each time contains a whose clause.
Try this instead:
var taskNames = Application('OmniFocus').defaultDocument.flattenedTasks.whose({completed: false}).name.get()
This should send a single request to OmniFocus, and return an array of the names of each uncompleted task.
Another approach to try would be to ask OmniFocus to evaluate the "whose clause" once, and return an array of object specifiers:
var taskSpecifiers = Application('OmniFocus').defaultDocument.flattenedTasks.whose({completed: false})()
Iterating over the returned array of object specifies and invoking .name.get() on each one would likely be faster than your original approach.
Answer
While JXA can get arrays of single properties of collections of objects, it appears that due to an oversight on the part of the authors, JXA doesn't support getting all of the properties of all of the objects in a collection.
So, to answer you actual question, with JXA, there is not a way to read a batch of objects and all their properties into memory at once.
That said, AppleScript does support it:
tell app "OmniFocus" to get the properties of every flattened task of default document whose completed is false
With JXA, you have to fall back to the for-loop anti-pattern if you really want all of the properties of the objects, but we can avoid evaluating the whose clause more than once by pulling its evaluation outside of the for loop:
var tasks = []
var taskSpecifiers = Application('OmniFocus').defaultDocument.flattenedTasks.whose({completed: false})()
var totalTasks = taskSpecifiers.length
for (var i = 0; i < totalTasks; i++) {
tasks[i] = taskSpecifiers[i].properties()
}
Finally, it should be noted that AppleScript also lets you request specific sets of properties:
get the {name, zoomable} of every window of application "Safari"
But there is no way with JXA to send a single request for multiple properties of an object, or collection of objects.
Try something like:
tell app "OmniFocus"
tell default document
get name of every flattened task whose completed is false
end tell
end tell
Apple event IPC is not OOP, it’s RPC + simple first-class relational queries. AppleScript obfuscates this, and JXA not only obfuscates it even worse but cripples it too; but once you learn to see through the faux-OO syntactic nonsense it makes a lot more sense. This and this may give a bit more insight.
[ETA: Omni recently implemented its own embedded JavaScriptCore-based scripting support in its apps; if JS is your thing you might find that a better bet.]

Split node.js event stream based on unique id in event body

I have a node.js-based location service that produces events as users move about, and I want to use RxJS to process these events and look for arbitrarily complex patterns like a user enters a region and visits 2 points of interest within 1 hour.
To begin with I need to split the stream of events base on unique user ids (from event body), but I am not finding any stock RxJS functions that will do this.
filter() would require that all uuids be known beforehand which is not desirable.
groupBy() looks like it would need to process the entire sequence prior to returning the grouped observables, which is not possible.
I'm thinking that maybe I need to build a custom observable that maintains a map of uuids to observables, and instantiate new observables as required. Each of these observables would then need to undergo identical processing in search of the pattern match, and ultimately trigger some action when a user's movements match the pattern. One of the obvious challenges here is I have a dynamically growing map of observables being produced as user enter the system and move about.
Any ideas how something like this could be achieved with RxJS?
I think you are misunderstanding how groupBy() works. It will generate a new Observable every time a new key is generated, and if the key already exists, it will just be pushed to the existing Observable.
So for your problem it should look something like this:
var source = getLocationEvents();
var disposable = new Rx.CompositeDisposable();
disposable.add(
source.groupBy(function(x) { return x.userid; })
.map(function(x) {
return x.map(function(ev) { /*Process the the event*/ });
})
.subscribe(function(group) {
disposable.add(group.subscribe(/*Do something with the event*/));
});

Resources