I'm using Tornado web server and want to take advantage of static caching for an asynchronous query result. Python makes it easy to wrap a function with a cache of some sort, for example using a decorator:
#cache.wrap(ttl=60)
def get_data(arg):
return do_query(arg)
However, it quickly gets complex using continuation passing:
def wrap_static_result(key, result, callback, ttl):
cache.set(key, result, ttl)
callback(result)
def get_data(arg, callback):
cached = cache.get(arg)
if cached:
callback(cached)
else:
callback2 = lambda result: wrap_static_result(arg, result, callback, ttl=60)
do_async_query(arg, callback2)
The most elegant solution I can think of requires making assumptions about call signatures, which isn't always practical. Can anyone think of a nicer way?
Use Deferreds. (The lack of such an abstraction is one of the reasons that Tornado is dramatically inferior to Twisted. You may want to check out Cyclone while you're at it.)
Related
I have a Flux and I want to convert it to List. How can I do that?
Flux<Object> getInstances(String serviceId); // Current one
List<Object> getInstances(String serviceId); // Demanded one
Java 8 or reactive components have a prepared method to map or convert it to List ??
I should use .map()
final List<ServiceInstance> sis = convertedStringList.parallelStream()
.map( this.reactiveDiscoveryClient::getInstances )
// It should be converted to List<Object>
1. Make sure you want this
A fair warning before diving into anything else: Converting a Flux to a List/Stream makes the whole thing not reactive in the strict sense of the concept because you are leaving the push domain and trading it with a pull domain. You may or may not want this (usually you don't) depending on the use-case. Just wanted to leave the note.
2. Converting a Flux to a List
According to the Flux documentation, the collectList method will return a Mono<List<T>>. It will return immediately, but it's not the resulting list itself, but a lazy structure, the Mono, that promises the result will eventually be there when the sequence is completed.
According to the Mono documentation, the block method will return the contents of the Mono when it completes. Keep in mind that block may return null.
Combining both, you could use someFlux.collectList().block(). Provided that someFlux is a Flux<Object>, the result would be a List<Object>.
The block method won't return anything if the Flux is infinite. As an example, the following will return a list with two words:
Flux.fromArray(new String[]{"foo", "bar"}).collectList().block()
But the following will never return:
Flux.interval(Duration.ofMillis(1000)).collectList().block()
To prevent blocking indefinitely or for too long, you may pass a Duration argument to block, but that will timeout with an exception when the subscription does not complete on time.
3. Converting a Flux to a Stream
According to the Flux documentation, the toStream method converts a Flux<T> into a Stream<T>. This is more friendly to operators such as flatMap. Mind this simple example, for the sake of demonstration:
Stream.of("f")
.flatMap(letter ->
Flux.fromArray(new String[]{"foo", "bar"})
.filter(word -> word.startsWith(letter)).toStream())
.collect(Collectors.toList())
One could simply use .collectList().block().stream(), but not only it's less readable, but it could also result in NPE if block returned null. This approach does not finish for an infinite Flux as well, but because this is a stream of unknown size, you can still use some operations on it before it's complete, without blocking.
Coming from AngularJS I'm struggling trying to solve the next problem. I need a function that returns an object (lets call it A, but this object cannot be returned till all the requests that are contained in that function are resolved. The process should be like:
The object A is downloaded from a remote server
Using A, we do operations over another object (B)
B is downloaded from the server
B is patched using some attributes from A
Using A and the result of B we do operations over a third object, C
C is downloaded from the server
C is patched using some attributes from A and B
After B and C are processed, the function must return A
I'd like to understand how to do something like this using rxjs, but with Angular 6 most of the examples around the internet seem to be deprecated, and the tutorials out there are not really helping me. And I cannot modify the backend to make this a bit more elegant. Thanks a lot.
Consider the following Observables:
const sourceA = httpClient.get(/*...*/);
const sourceB = httpClient.get(/*...*/);
const sourceC = httpClient.get(/*...*/);
Where httpClient is Angular's HTTPClient.
The sequence of the operations you described may look as follows:
const A = sourceA.pipe(
switchMap(a => sourceB.pipe(
map(b => {
// do some operation using a and b.
// Return both a and b in an array, but you can
// also return them in an object if you wish.
return [a,b];
})
)),
switchMap(ab => sourceC.pipe(
map(c => {
// do some operations using a, b, and/or c.
return a;
})
))
);
Now you just need to subscribe to A:
A.subscribe(a => console.log(a));
You can read about RxJs operators here.
Well, first of all, it appears to me that this function-call, as described, would be somehow expected to block the calling process until all of the specified events have occurred – which of course is unreasonable in JavaScript.
Therefore, first of all, I believe that your function should require, as its perhaps-only parameter, a callback that will be invoked when everything has finally taken place.
Now – as to "how to handle steps 1, 2, and 3 elegantly" ... what immediately comes to mind is the notion of a finite-state machine (FSM) algorithm.
Let's say that your function-call causes a new "request" to be placed on some request-table queue, and, if necessary, a timer-request (set to go off in 1 millisecond) to service that queue. (This entry will contain, among other things, a reference to your callback.) Let's assume also that the request is given a random-string "nonce" that will serve to uniquely identify it: this will be passed to the various external requests and must be included in their corresponding replies.
The FSM idea is that the request will have a state, (attribute), such as: DOWNLOADING_FROM_B, B_DOWNLOADS_COMPLETE, DOWNLOADING_FROM_C, C_REQUESTS_COMPLETE, and so on. Such that each and every callback that will play a part in this fully-asynchronous process will (1) be able to locate a request-entry by its nonce, and then (2) unambiguously "know what to do next," and "what new-state (if any) to assign," based solely upon examination of the entry's state.
For instance, when the state reaches C_REQUESTS_COMPLETE, it would be time to invoke the callback that you originally provided, and to delete the request-table entry.
You can easily map-out all of the "state transitions" that might occur in an arbitrarily-complex scenario (what states can lead to what states, and what to do when they do), whether or not you actually create a data-structure to represent that so-called "state table," although sometimes it is even-more elegant(!) when you do. (Possibly-messy decision logic is simply pushed to a simple table-lookup.)
This is, of course, a classic algorithm that is applicable to – and, has been used in – "every programming language under the sun." (Lots of hardware devices use it, too.)
Suppose I want to be able to write a test like this:
lambda {
do_something_involving_web_requests
}.should make(1).http_requests
It seems to me there would be several possible ways to implement this kind of functionality; however, it also seems that:
Someone might have already done so (in which case I want to look into their solution); or
Someone on StackOverflow might have an idea I haven't thought of.
So, has this been done already? And/or what are your ideas?
If you're interested in more granular testing of how your app responds to specific HTTP responses, rather than simply counting requests, you can use mocks. Here's how to use RSpec's mocks to test http requests:
#mock_http = mock("http")
Net::HTTP.stub!(:start).and_yield #mock_http
#mock_http.should_receive(:get).with("/")
One library I use is Fakeweb. Fakeweb does what #Joe mentions: it hooks into Net::HTTP and can be configured to return a canned response from a given URL. Many other HTTP libs depend on Net::HTTP so this technique has broad compatibility. Fakeweb example from its docs:
FakeWeb.register_uri(:get, "http://example.com/test1", :body => "Hello World!")
Net::HTTP.get(URI.parse("http://example.com/test1"))
=> "Hello World!"
Neither of these methods have a simple access count though, if you want that you can use rspec-mocks which has the following method count functionality (these can be used on stubs or test doubles):
double.should_receive(:msg).once
double.should_receive(:msg).twice
double.should_receive(:msg).exactly(n).times
double.should_receive(:msg).at_least(:once)
double.should_receive(:msg).at_least(:twice)
double.should_receive(:msg).at_least(n).times
double.should_receive(:msg).at_most(:once)
double.should_receive(:msg).at_most(:twice)
double.should_receive(:msg).at_most(n).times
double.should_receive(:msg).any_number_of_times
A ZVAL is typically created with emalloc so it is destroyed at the end of a page request. Is there a way to take an existing ZVAL and make it persist in the SAPI (equivalent of pemalloc)? What about creating a ZVAL with pemalloc?
Ideally what I'd like to do (in PHP code) is this:
class Object
{
public $foo;
}
if(!($object = persist("object")))
{
$object = persist("object", new Object());
}
$object->foo[] = "bar";
print count($object->foo);
On each request count would return +1 (assuming the same PHP "worker" is used every time - I'm using PHP-FPM).
You're basically describing http://lxr.php.net/opengrok/xref/PHP_5_3/ext/sysvshm/sysvshm.c#242
The closest you can get without duplicating functionality which is already in shm is https://github.com/flavius/php-persist. The catch: in a prefork/multiprocess SAPI (like apache's), different requests from the same client may end up in different processes, and as such, you'll see different data (try it on Linux + Firefox with a hard refresh every time in the browser).
Note: It's a work-in-progress, currently it persists only an integer. Adding an array should be trivial. Patches welcome. It still needs the deserialization part, and to actually use persist()'s first parameter.
zend_object can't be persisent therefore you can't do this. extensions like APC serializes the object into memory.
I'm trying to use iChemLabs cloud services from a html5 web worker. Normally the cloudservices requires jQuery but I can't import that into a web worker so I'm using Pollen instead with a ChemDoodle Web Components library with which I have stripped out the document-related things.
jQuery.Hive.Pollen provides a nice ajax function very similar to jQuery, but I can't seem to get it to work at all. I know this problem will be tricky to solve considering that Access-control-headers need to be set to allow any of you to actually find the solution. However, I'm a beginning javascript programmer and I was wondering if my two weeks of frustration is actually a small difference. I am trying to invoke the following function:
var cloudmolecule;
ChemDoodle.iChemLabs.readSMILES('N1(C)C(=O)N(C)C(C(=C1N1)N(C=1)C)=O', function(mol){
cloudmolecule = mol;
});
Here is a link to the library code I am using, see the 'q.ajax' call and substitute jQuery = q for p = q (p is for pollen) in that block of code.
Right now I'm just trying to get the ajax call to work in an ordinary block of javascript with the plan to migrate to a web worker later.
If anybody could point out the problem to me I would be extremely grateful.
solved! turns out iChemLabs rejects these two extra headers that pollen creates:
_xhr.setRequestHeader("X-Requested-With", "Worker-XMLHttpRequest");
_xhr.setRequestHeader("X-Worker-Hive", "Pollen-JS" );
Simply comment them out
Also, Pollen ajax seems to return a JSON object containing the data in JSON format AND as a string, so do
o = JSON.parse(data.string)//data is the parameter to the callback function
The reduced ChemDoodle library (without document-related methods) will work like a charm with pollen ajax.