We have an application that pulls in about 50 records from a database with about 13 data points for each record. Each record needs to be reviewed for accuracy, sometimes edited, and then 'approved' or 'rejected'.
It seems as if the process of running an approval or rejection takes some time before another approval or rejection can occur (yes this backend could be optimized).
Im looking for techniques or suggestions to make the front end of the application much quicker while the back end continues to process the previous approval or rejection. This would help speed our team of record reviewers go through each record.
Would a messaging service like RabbitMQ apply here?
All help, links, feedback is appreciated.
-=Vin
Tabbed browsing.
Related
We are using a cloud based system that after we have signed out found out that there are massive downtime windows a few times per year to the amount of 30+ hours of downtime to do an upgrade. We are a 24 hour service and cannot be down for that long.
The front end UI is also horrible functionally, visually and accessibility wise which led us to think of building our won front end UI. The idea being we would submit the requests to the system from a custom front end via API calls.
If the backend system goes down for an upgrade, I am trying to think of how we could sync the requests that come in throughout the outage into the front end. I could add something to sync all that haven't been sent, or have the backend platform look at the front end and create any objects that haven't been created.
Is there anything else that could be down to keep the systems in sync?
Any thoughts or ideas are appreciated.
I am experiencing a very frustrating issue with SagePay Direct when a card payment initiates a 3DSecure challenge.
Customers are reporting either a hanging iFrame, or payment declined response. Whats worse is that in some instances, Sage takes the payment but the user is unaware of this and tries to buy again
Looking at my logs my code is working as expected and is loading the iFrame with the returned ACSURL as the src.
After searching the web, it appears it is a known issue with a timeout occurring on the secure merchant issuer that i hand off to.
The trouble i have is that i have no control of the response(or lack of) from the issuer as its in an iFrame.
Sage have not been very helpful with this problem only going as far as to say "we have heard of customers who experience this issue"
Does anyone have any experience of this problem and know how to resolve it? I guess the bottom line is to turn off the 3DSecure checks but this seems counter productive to the new EU ruling coming into force at some point.
Worth pointing out that this is only affecting a small percentage of my customer base and a lot of transactions are processing successfully (even with the password challenge) but the customers who experience problems are rightly shouting loudly.
anyone any ideas?
Thanks
We process up to 1000-2000 transactions daily via SagePay, using the Direct protocol. They are very cheap but their service is in all honesty fairly terrible. We have a single digit quantity of transactions every day that fail in this way. We've also got another provider and don't experience the same issues.
We have a routine job that asks the SagePay Reporting API about transactions that failed, to see what the current status is (did SagePay get the transaction? was it successfully authorised? etc). This API is utterly, utterly terrible and was a nightmare to integrate with, but it's useful as at least we can refund customers without having to log into the SagePay dashboard.
One thing that we discovered (that isn't documented anywhere on the SagePay site as far as I can tell) is that you're limited to one transaction at a time, or around 20-30 transactions per minute by default. If you go over this (a temporary peak or whatever) your transactions queue up and are delayed. If it gets really busy it completely falls over, and takes a while to recover. We had to switch SagePay off entirely for a few hours due to this (we've got backups in place).
Anyway, so it turns out our transactions were all being processed on one TID (short for Terminal ID). This is akin to a physical card terminal in a shop which can only process one transaction at a time. We asked SagePay support for more and we now have 10-15.
I hope this helps you. I'd recommend implementing a fallback payment supplier in case SagePay fails. A year or two ago they had a 3 day(!!!!) outage which was fairly devastating for us. We now take this seriously!
We've recently had an increase in what I believe may be the same thing. Basically the customer would be sent off to the 3ds page, then returned to the callback page, but for reasons I can't explain the PHP session wouldn't reestablish. The POST response to the callback page was enough to identify the order and complete it (as we'd taken payment), but the customer would then subsequently be prompted to log in again - they'd then see their basket as still having products in and place a second order (that would go through successfully).
After many hours debugging and making changes I managed to replicate this on a development server whilst using mobile emulation...
Long story short, what I have done is to add:
session_regenerate_id();
When I perform the initial vsp register CURL (this is the CURL where you get given the ACSURL). So far, this seems to be enough to ensure that the session gets reestablished when the customer returns to the callback page.
Across all browsers/devices, I find random different pages, at random times, are very slow to load/don't load. The browser is stuck on 'Waiting for website.com'. I will wait 20 seconds and nothing will happen until I manually refresh the page. As I realise this is very vague, can you suggest a) most likely issues to look for first or b) some diagnostic tools that I could use to try and de-bug the issue as a starting point, so that my hosts/developers can solve the issue. Here are some results of recent speed tests.
One thing to also add is that, it seems it more often gets stuck on particular pages. Namely the pages where users take practice tests. After each time the user clicks 'Next', their selected answer is inserted into the database. My speculation is that potentially it's an issue with the DB itself, or the process which inserts into the database. It's when clicking 'Next', that the whole website sometimes just dies as described above.
Results from Google Speed Test
Waterfall image
A wait time of 20secs at random times and random pages could possibly be due to stop-the-world garbage collection. So GC logs are probably a good starting point.
A thread sampler such as Djigger a colleague of mine wrote might probably also help you figuring out what the machine is doing during the 20 seconds.
If that doesn't help I suggest to use a Profiler or better an APM tool to monitor whats going on on your system. Those tools give a you a broader insight of the internals.
You need to run a few page speed tests and look at the waterfall images.
It is very common on shared servers for the server to be too busy to get to your request. 20 seconds would indicate a serious issue with the server.
Another common reason is the page has a link to a third party resource and that resource is too often unavailable.
In your case the culprit is website.com and I assume that is your site.
Use something like webpagetest.org to run the tests.
In the waterfall image below
Dark Green is DNS lookup time.
Orange is the time for Browser to connect to server.
Green is the wait time for server to put image in output buffer.
Blue is the time for the server to transmit to the Browser.
The problem with the sample waterfall page is the index page took 4 seconds to be generated or retrieved. Most likely this is a Word Press site with plugins.
I suspect yours may be 20 seconds. But due to the randomness, is is also a good possibility it is a page resource that is stalled.
If it is the index page, then you likely have a poor ISP and or one of the other users of the server is hogging the CPU.
Keep running the tests until you see the problem occur.
It will be very obvious where the problem is located.
You can post the waterfall image and send me a message if you have any questions.
Waterfall from webpagetest.org
Use a StackOverflow Q&A thread as an example - when you vote up, vote down, or favorite a question, you can see the UI quickly respond to that action with changes in the # of up-votes on the side.
How can we achieve that effect? If send every of such action to back-end for processing and use the returned response to update UI, you will see a slow update and feel the glitches. But if put some of the logic on the front-end, you will also need to take care of the fraud/abuse etc before reflecting the action on UI, i.e - before changing the # of up-votes, don't you need to make sure that's a valid click by an valid user first?
You make sure that a valid user is using the app before a user clicks on anything. This is done through authentication, and it must include various protection mechanisms against malicious users.
When a user clicks, a call is made to a server. In a properly architected app this call is lightweight, and the server responds very quickly. I don't know why you believe that "you will see a slow update and feel the glitches". Adding an upvote to the database should take a few hundred milliseconds at most (including the roundtrip from the client), especially if the commit is asynchronous or a memcache is used.
If a database update results in a need to do some complex operations, typically these operations are not done right away. For example, a cron job may run periodically to compute new rankings, etc., precisely because you do not want every user to wait. Alternatively, a task is created and put in a task queue to be executed when resources are available - again to make sure that a user does not wait.
In some apps a UI is updated immediately after the call to the server is made, before any response from a server arrives. You can do it when the consequences of a failed call are negligible. For example, if an upvote fails to be saved in the database, it's not a disaster, especially if it happens once in a million tries. Again, in a properly architected app calls fail extremely rarely.
This is a decision that an app developer needs to make. I would not update a UI before a server response if such an update may lead a user to believe that some other action is now possible. For example, if a user uploads a new photo, I would not show icons to edit or share this photo until I know that the photo is safely saved.
I'm trying to create a real-time webbrowser game in ASP.Net MVC3, but I'm unsure about what's the best approach to processing 'real-time' events on the server side.
Imagine that the client wants to upgrade a building, upgrading a building takes time. A record gets inserted in the database that holds the end-time and on the client a ajax timer start's running. I was thinking about having a windows service running all the time. Every second the service checks the table and does the real processing when the time passed the end-time. However I could imagine that when you have a huge amount of data to process this can get problematic.
What would be the best way of doing this?
Cheers.
I've seen games like that before. You do not need to count down yourself, you just need to store the datetime for whenever the building will be ready.
When a page is requested you send a timestamp of when the building will be completed, and use javascript to count down. For example, like this.