What would be the best way to incorporate something similar to the RabbitMQ channel.waitForConfirmsOrDie() method, while utilizing the Bunny gem for a publish confirmation?
Right now I am using:
if !#channel.using_publisher_confirmations?
#channel.confirm_select
end
#channel.default_exchange.publish(args)
was_successful = #channel.wait_for_confirms()
But ideally, for the scenario I need, I would like to have a much shorter timeout on waiting for the confirmations. Right now, it seems as though there is a default timeout of roughly 15 seconds, but that is far too long to block the thread. If I don't receive confirmation within, say, three seconds, what I'd like to have happen is raise an exception/return false.
I saw there was a waitForConfirmsOrDie() in the RabbitMQ documentation, but Bunny does not have this as a method available.
Am I considering rewriting some methods for similar functionality. Has anyone come across something similar and found a good way to implement this?
Don't wait for confirms synchronously. You should use a technique similar to this to keep track of outstanding confirms and handle them.
NOTE: the RabbitMQ team monitors the rabbitmq-users mailing list and only sometimes answers questions on StackOverflow.
Related
This is more of a theorical question.
Well, imagine that I have two programas that work simultaneously, the main one only do something when he receives a flag marked with true from a secondary program. So, this main program has a function that will keep asking to the secondary for the value of the flag, and when it gets true, it will do something.
What I learned at college is that the polling is the simplest way of doing that. But when I started working as an developer, coworkers told me that this method generate some overhead or it's waste of computation, by asking every certain amount of time for a value.
I tried to come up with some ideas for doing this in a different way, searched on the internet for something like this, but didn't found a useful way about how to do this.
I read about interruptions and passive ways that can cause the main program to get that data only if was informed by the secondary program. But how this happen? The main program will need a function to check for interruption right? So it will not end the same way as before?
What could I do differently?
There is no magic...
no program will guess when it has new information to be read, what you can do is decide between two approaches,
A -> asks -> B
A <- is informed <- B
whenever use each? it depends in many other factors like:
1- how fast you need the data be delivered from the moment it is generated? as far as possible? or keep a while and acumulate
2- how fast the data is generated?
3- how many simoultaneuos clients are requesting data at same server
4- what type of data you deal with? persistent? fast-changing?
If you are building something like a stocks analyzer where you need to ask the price of stocks everysecond (and it will change also everysecond) the approach you mentioned may be the best
if you are writing a chat based app like whatsapp where you need to check if there is some new message to the client and most of time wont... publish subscribe may be the best
but all of this is a very superficial look into a high impact architecture decision, it is not possible to get the best by just looking one factor
what i want to show is that
coworkers told me that this method generate some overhead or it's
waste of computation
it is not a right statement, it may be in some particular scenario but overhead will always exist in distributed systems
The typical way to prevent polling is by using the Publish/Subscribe pattern.
Your client program will subscribe to the server program and when an event occurs, the server program will publish to all its subscribers for them to handle however they need to.
If you flip the order of the requests you end up with something more similar to a standard web API. Your main program (left in your example) would be a server listening for requests. The secondary program would be a client hitting an endpoint on the server to trigger an event.
There's many ways to accomplish this in every language and it doesn't have to be tied to tcp/ip requests.
I'll add a few links for you shortly.
Well, in most of languages you won't implement such a low level. But theorically speaking, there are different waiting strategies, you are talking about active waiting. Doing this you can easily eat all your memory.
Most of languages implements libraries to allow you to start a process as a service which is at passive waiting and it is triggered when a request comes.
I'm creating a raffle site as a small side project. It will handle multiple raffles each with an end time. At the end of each raffle a single winner is chosen.
Are Laravel Jobs the best way to go with this? Do I just create a single forever-repeating job to check if any raffles have ended and need a winner?
If not, what would be the best way to go?
I don't think that forever-repeating scripts are generally a good idea.
I just create a single forever-repeating job
This is almost never a good idea. It has its applications in legacy code bases but websockets and events are best considered for this job. Also, you have the benefit of using a really good framework like Laravel, so take advantage of it
Websockets
If you want people to be notified in real time in the browser.
If you have all your users subscribe to a websocket channel when they load the page, you can easily send a message to a websocket server to all subscribed clients (ie browsers) to let them know who the winner is.
Then, in your client side code (Javascript), you can parse that message to determine who the winner is and render a pop up that let's the user know.
Events
If you don't mind a bit of a delay, most definitely use events for this.
At the end of every action that might potentially end a raffle (ie, a name is chosen at random by a computer - function chooseName()). Fire an event that notifies all participants in the raffle.
https://laravel.com/docs/5.2/events
NB: I've listed the above two as separate issues, but actually, the could be used together. For example, in the event that a name is chosen at random, determine if the raffle is over and notify clients via a websocket connection.
Why I wouldn't use delayed Jobs
The crux of the reason - maintainability
Imagine a scenario where something extends the time of your raffle by a week. This could've happened because a raffle was cheated on or whatever (can't really think of all the use cases in that area).
Now, your job has a set delay in place - is it really a good programming principle to have to change two things when only one scenario changed? Nope. Having something like an event in place - onRaffleEnd - explicitly looks for the occurrence of an event. Laravel doesn't care when that event happens.
Using delayed Jobs can work - it's just not a good programming use case in your scenario and limits what you're able to do in the longer run. It will force you to make more considerations when unforeseen circumstances come along as well as when you want to change things. This also decentralizes the logic related to your raffle. Whilst decoupling code is good practice, having logic sit in completely different places makes maintenance a nightmare.
Is typhoeus safe to use with activerecord? resque? I've poked around in the source and googling here and there and I can't make heads or tails of it. I guess what I really want to know is, are the response callbacks run one at a time or in parallel? Because if the callbacks are atomic or sequential or whatever then it's safe to do anything in them. I think.
From my answer to the github question:
I've used it extensively within Resque workers. Typhoeus::Hydra
returns (fires #on_complete) in the order that the requests are
returned. Rather than doing any database work within the #on_complete
block, I'd suggest storing a collection of response objects and
performing database work after you've done any validation you might
need to do.
I've got a need to retain the data and keep a Ruby program waiting for a response for anything up to a couple of days. I'm thinking about implementing this using threads (there may be a number of concurrent requests across a network). My question; is it reasonable to leave a thread running for anything up to a couple of days awaiting a response?
In general there is no problem with that. Check out the Queue class, it might facilitate the "job polling":
http://www.ruby-doc.org/stdlib-1.8.7/libdoc/thread/rdoc/Queue.html
We have an application that uses the window message queue to pass data from a socket to consumer HWNDs (at a rate of ~2100Hz). This application has worked for >2 years. Recently our application has started exhibiting problems where WM_TIMER is not being fired/executed by our application. I think this is due to the data being pumped into the message queue.
My question is there a way to determine how many pending messages are in the message queue for a given thread/HWND?
This is answered in really great detail by Raymond Chen in his post "but then we ran into problems when we started posting 10,000 messages per second".
The research team asked to meet with
the user interface team to help work
out their problems under load. They
outlined their design and explained
that it worked well at low data rates,
"but then we ran onto problems when we
started posting 10,000 messages per
second."
At that point, the heads of all the
user interface people just sat there
and boggled for a few seconds.
"That's like saying your Toyota Camry
has stability problems once you get
over 500 miles per hour."
There isn't a good way to do this. One thing you could do is aggressively empty the message que and put them in your own queue. But, this will not solve your problem.
I hate telling you this, but you should really find a way to process your socket data. I think you will find some other mechanism scales better, performs better, and is easier to debug than using the windows message queue for this.
Foredecker