I have a site that contains 100 available books.
But at the same moment of time I have 101 request that want to reserve this book. How to handle this situation?
Robert Hanmer's book Patterns for Fault Tolerant Software has a pattern called Queue for resources (46):
Store requests for service that cannot be handled immediately in a queue [...]. Give the queue a finite length to improve the likelihood that the request is still important when it reaches the head of the line.
When the requests are computer generated and must be processed in order, a First In First Out (FIFO) queue should be used. When people are generating the requests, the queue should use a Last in First Out (LIFO, a.k.a. a stack) strategy (as in FRESH WORK BEFORE STALE (55)) to govern insertion and removal. This will help people receive good service. The request that was placed on the queue last will think that they received excellent service, and the person that placed the longest ago request on the queue probably gave up already.
Allocation of resources under the guidance of EQUITABLE RESOURCE ALLOCATION (45) should recognize both the requests that have been queued and those that are fresh and have never been queued.
You should have a waitlist table which holds waitlisted/reserved requests with subscriber number, book id and time requested. Then when one of books is returned - trigger a procedure which allots the freed book to the oldest waitlisted request and notifies the subscriber who has raised this request.
Instead of a trigger you can also have a scheduled jobs which runs daily/twice a day and checks whether any of the waitlisted books are available now. If available it allocates the book to the subscriber who has the oldest waitlisted request for that book.
Use trigger if your system load is not heavy, else use scheduled job and time it for non-peak hours.
Related
I have an array of objects that i need to send to an endpoint. I am currently looping through the array and sending the requests one by one. The issue is that i now have over 35,000 requests to be made, and i need to update the database with the response.In my limited knowledge of springboot , i am not aware of any method i can use to send the 35,000 requests at once (without looping through one by one).
Is the best method to use still employing looping but utilize asynchronous calls, or is there a method that i can use to send the 35,000 http requests at once?..i just need a pointer because i am not aware how threads can be used, since this is already an array and each element needs to be sent.
Thank you
Well, first off 35,000 at a time of, well, anything, is a bad idea.
However, if you look in to the Java ExecutorService, this gives you the ability to fill a queue with tasks, and then each task will be performed by a thread taken from a thread pool. As the threads complete, the service pulls another request from the queue and handles that. So, you simply provide a Runnable that performs your web requests, create an Adequately Sized Thread Pool (which is basically sized through experimentation to give the best throughput), and then let the threads crunch away on the queue of tasks.
You will need a queue large enough to absorb all of your tasks, or you can look at something like the NotifyingBlockingThreadPoolExecutor. This will allow you to just gorge a queue and block when the queue gets to full, until all of your tasks are complete.
Addenda:
I don't know enough about Spring Boot to comment about whether a "batch job" would do what you want or not.
However, on that note, an alternative to creating 35,000 individual entries for the ExecutorService, you could, indeed, send a subset. For example 3,500 entries representing 10 items each, or 350 with 100 each. The idea there is to leverage any potential gains from reusing HTTP connections and what not, so there's less stand up and tear down for each request. Standing up 350 connections if far cheaper than standing up 35,000.
we are currently working in a message driven Microservice environment and some of our messages/events are event sourced (using Apache Kafka). Now we are struggling with implementing more complex business requirements, were we have to take multiple events into account to create new events and side effects.
In the current situation we are working with devices that can produce errors and we already process them and have a single topic which contains ERROR_OCCURRED and ERROR_RESOLVED events (so they are in order). We also make sure, that all messages regarding a specific device always go onto the same partition. And both messages share an ID that identifies that specific error incident. We already have a projection that consumes those events and provides an API for our customers, s.t. they can see all occurred errors and their current state.
Now we have to deal with the following requirement:
Reporting Errors
We need a push system that reports errors of devices to our external partners, but only after 15 minutes and if they have not been resolved in that timeframe. Our first approach was to consume all ERROR_RESOLVED events, store the IDs and have another consumer that is handling the ERROR_OCCURRED events in a delayed fashion (e.g. by only consuming the next ERROR_OCCURRED event on the topic if its timestamp is at least 15 minutes old). We would then be able to know if that particular error has already been resolved and does not need to be reported (since they share a common ID with the corresponding ERROR_RESOLVED event). Otherwise we send an HTTP request to our external partner and create an ERROR_REPORTED event on a new topic. Is there any better approach for delayed and conditional message processing?
We also have to take the following special use cases into account:
Service restarts: currently we are planning to keep the list of resolved errors in memory, so if a service restarts, that list has to be created from scratch. We could just replay the ERROR_RESOLVED messages, but that may take some time and in that time no ERROR_OCCURRED events should be processed because that may result in reporting errors that have been resolved in less then 15 minutes, but we are just not aware of it. Are there any good practices regarding replay vs. "normal" processing?
Scaling: we may increase or decrease the number of instances of our service at any time, so the partition assignment may change during runtime. That should not be a problem if we create a consumer group for each service instance when consuming the ERROR_RESOLVED events, s.t. every instance knows all resolved errors while still only handling the ERROR_OCCURRED events of its assigned partitions (in another consumer group which is shared by all instances). Is there a better approach for handling partition reassignment and internal state?
Thanks in advance!
For side effects, I would record all "side" actions in the event store. In your particular example, when it is time to send a notification, I would call SEND_NOTIFICATION command that emit NOTIFICATION_SENT event. These events would be processed by some worker process that does actual HTTP request.
Actually I would elaborate this even furter, since notifications could fail, so I would have, say, two events NOTIFICATION_REQUIRED, and NORIFICATION_SENT, so we can retry failed notifications.
And finally your logic would be "if error was not resolved in 15 minutes and notification was not sent - send a notification (or just discard if it missed its timeframe)"
I have an endpoint in my api that supports writes. The resource in question is collaborative, so it is reasonable to expect that there will be parallel write requests arriving concurrently.
If the number of writes is small, then this is relatively straight forward to do with a simple lambda - read the current state, compute the new state, compare and swap, spin until the swap succeeds or until we give up. In either case, we compute the appropriate http response and return it to the caller.
If the API is successful, then eventually the waste of conflicting writes becomes expensive enough to address.
It looks as though the natural response is to copy the requests into a queue, with a function that consumes batches; within each batch, we process the requests in sequence, storing the new write, and computing the appropriate response to the request.
What are the options for getting those computed responses copied into the http responses, and what are the trade offs to be be considered?
My sense is that in handling the http request, after (synchronously) enqueue the message, I need to block/poll on something that will eventually be populated with the response to the request.
I'm not sure if this will count an an answer, but I do not agree that the natural response is to copy/queue/block; that feels like you're just trading optimistic concurrency control for a kind of pessimistic one (and you'd probably have an easier time just implementing a lock using e.g. Redis - not to mention there are other issues with Lambda itself that would make the approach you describe even more difficult).
Users probably do not want an API like this as it would have high latency.
In my opinion an API that is well designed for collaborate modification of some shared state has higher order constructs that make the API successful: thinking of a conversation as an example, you would decompose the chat in to individual messages, where each message is in reply to some other message; the concurrent modification to the conversation is append-only for the most part (you might allow a user to edit an individual message but that's not a point of resource contention) and you might do things like count the number of messages within the conversation asynchronously such that it is eventually consistent.
You can look at the domain of your API and see if there's a way to expose modification to it in such a way that reduces contention by making modifications target sub-entities (even if the API represents this as a single resource, the storage engine does not have to).
Another option is looking in to a model like event sourcing, where the changes themselves are literally appended and you derive the state from some snapshot plus recent changes.
I have a question regarding the dequeue mechanism during discrete event simulation.
Most of the implementations use some kind of priority queue which can be used to quickly retrieve the event with the earliest timestamp. What happens when such an event cannot be scheduled because, say, it needs a resource to be able to run.
There may be another event in the queue whose timestamp is greater than the timestamp of the event that is blocked on a resource.
For example, let us assume we are modelling a grocery-store with separate checkout lines and a cashier per line. A shopper entering a checkout line is an event. We enqueue this event based on the time the shopper entered the checkout line. However, the order in which our simulation should execute two such events in not necessarily the time order in which they entered the checkout line because the cashiers might free up in a different order.
In such a scenario how does using a priority queue solely based on timestamp --- and independent of resource availability --- work out?
You need a queue for each cashier, or at least a count of waiting customers if customer identity is not important in your simulation ( e.g. I would join a queue of three people with one item each over a queue with one person with a full trolley, so just a queue length may not capture the information needed to incorporate that heuristic ).
When a customer joins the queue, the number of queuing customers is incremented or the customer is pushed onto the cashier's queue.
When the cashier is ready to serve, the first customer is popped of the cashier's queue. So the customer service event is dependent not on the time the customer arrives, but when the cashier is ready.
These queues or counters are independent of the scheduling mechanism for events - the events scheduled manipulate these queues, they aren't dependent on them for scheduling.
As Pete Kirkham pointed out, it's important to be aware that the lines (queues) that customers wait in are completely separate things from the priority queue that's used to determine event ordering.
In discrete-event simulation an event is a point in time at which the system state changes. When an event occurs you figure out what to do next based on the state. Joining the line of customers is an event, but so is becoming eligible for service. Once a customer becomes eligible for service, the logic of that event has to check whether service is possible or not. If so, schedule a new event for when the service will end. If there are resource constraints, then nothing gets scheduled and that customer is on hold. However, at some point in the future the required resource will become available. That's an event too, and that event's logic should check to see if there are customers on hold due to lack of the resource. If not, there's no need to schedule anything, but if so, you can now schedule the actual service for the customer. You can see that customer delays in the queue will increase with resource constraints.
For a much fuller explanation of how discrete-event simulations work, please look at this introductory tutorial paper.
I'm about to create an application that will spawn tasks of about 100,000 requests expecting responses. I'm wondering whether to use a static reply queue or temporary queues. There is only one client requesting and only one server replying. The use case for the client will be to spawn a task about once a day.
I'm thinking I want to use temporary queues for the responses but I'm wondering if there is a reasonable limit to the amount of temporary queues or how long I would want to keep them open.
Some replies make take days to come back or never come back so I would time out the temporary queues after about 3 days.
My immediate thoughts are that 3 days stretches the definition of temporary. In that time you want to survive both requester (producer, who also consumes the response) and broker outages. Temporary queues are a contract between the subscriber and the broker - if one of them goes down, the temporary queue disappears and the responder will get an error when they attempt to reply on that queue.
I would use static queues in this instance - you will need to implement a layer for correlating responses back to requests in your requester, but you would need to do that anyway if you want to survive the outage of that process (probably by storing additional state in a database).