I implementing RESTful API service and i have a question about saving related records.
For example i have users table and related user_emails table. User emails should be unique.
On client side i have a form with user data fields and a number of user_email fields (user can add any number of fields independently). When the user saves the form i must first make query to create record in users table to get her ID, and only then i can make query to save user emails (because in now i have id of record which come with response after saving user data). But if user enters not unique email in any field then the request will fail. So I create a record in the users table but not create record in user_emails table.
What are the approaches to implement validation of all this data before saving?
This is nor related restful api but transactional processing on the backend. If you are using Java, with JPA you can persist both element in the same transaction then you can notice if there is a problem and rollback the entire transaction returning a response.
I would condense it down to a single request, if you could. Just for performance's sake, if nothing else. Use the user_email as your key, and have the request return some sort of status result: if the user_email is unique, it'll respond with a success message. Otherwise, it'd indicate failure.
It's much better to implement that check solely on the server side and not both with the ID value unless you need to. It'll offer better performance to do that, and it'll let you change your implementation later more easily.
As for the actual code you use, since I'm not one hundred percent on what you're actually asking, you could use a MERGE if you're using SQL Server. That'd make it a bit easier to import the user's email and let the database worry about duplicates.
Related
I am working on a function that allows a user to check if their existing device contacts are using our platform, based on phone numbers.
For privacy and security, we are hashing the user's contact's phone numbers on device (salted with the user's id) before sending to our server.
Server side, we then need to hash our entire contacts table (using the user's id as a salt), which is currently being done in a for loop.
We then check this list against the request list, and return the details for any matches.
However, I'm sure there is a more efficient way of doing this, something like computing the hash in a calculated field then including the $request->hashes in a "whereIn" clause.
Could someone give me a pointer on the best approach to be taking here?
The question is, what privacy and security are you achieving by sending hashed value of contact number?
You are hasing the contact in client side(device), that means you are using a key and salt that is available in clinet side already. How can that be a security feature?
If you want to search hashed value in database then it's better to save hashed contract number in a column in the first place. So you can directly run where query in database table.
Ideally, if you really concern about user's contact number you should:
Encrypt the user's contacts in backend/databse not in frontend.
If you need to query for a field in database then you should make a hash valued column that can be matched easily. I mean searchable fields should be hashed so you can run direct query.
Nothing to worry about user's contact security in frontend if you are already passing it trhough Secure HTTP(HTTPS).
Even it a common practice in the industry, to pass a submitted plain password via HTTPS when a user submit it in frontend. It shouln't be a concern of privacy or security.
I have been working on a project. I always followed this idea. Don't send all the data in one call.
Here is an example,
Suppose there is an API to return all the list of students that can be added to test they need to finish.
So, on UI side every student have one button "add" which will show a pop up if the student is already assigned to take the test. Or it will show a pop up he has already finished the test.
I could join many table and send all the data in one api call while fetchig students. Or
I could send the send the students and then on "add" there is another API to make sure the above mentioned conditioned met.
Which approach is better?
Because If I send all the data in one api call, there might be only few students be assigned the test.
Checking if a student is already assigned or not should happen in the backend, not frontend, and also atomically so as to prevent duplicates - either using a database transaction or a unique constraint.
When the Add button is clicked then in any case a backend call will need to be made (to perform the actual Add). If the add failed, the backend can interpret the "unique constraint violation" database error and return a "student is already assigned" message.
For the rest of the question, the rule is simply: don't fetch more data than is required by the UI.
If the Add button is always shown regardless of whether or not the student is already added, there is no need to retrieve this information beforehand.
But it might be useful to give a visual indication of which students are already added, in that case obviously there's no choice but to retrieve and return this information to the UI.
Fortunately GraphQL is precisely the tool for this job - it makes it possible for the UI to request exactly what information is needed for a given page, without having to code each and every possible query by hand.
My app uses AppSync resolvers to fetch data from DDB and return it to our front-end. One table we have is for Notifications. A Notification can be either pending or default (non-pending). The table itself has a primary key of notification_id and we have a GSI called userIndex to grab the notifications for a user, with a sort key of timestamp.
In the app, I show all notifications in a list, pending first and then default. Given that a user may have many notifications, I'd like to implement pagination to fetch a batch at a time. The only way I've been able to do this is to
change the query to include a isPending parameter, which I use as a filter expression for the query to only return notifications that are isPending or isNotPending.
Store two "nextTokens", one for each isPending and isNotPending, along with corresponding lists.
Make separate queries for pending/non-pending, and use the filter to return to the appropriate list.
This is obviously inefficient and I am re-reading data from DynamoDB. My question is, given my DynamoDB table/requirements, is there a way I can paginate so that I can get all the pending notifications first (sorted by timestamp) and then all the default notifications next (sorted by timestamp) by using one query and one nextToken
I've seen the use of #model and #key, but I haven't been able to make it work in my app.
Thanks!
No, not really. There is a hard limit on returns for a Dynamodb query - and that cannot be bypassed. the only way to make use of nextToken is another query.
However, it is also worth noting that the FilterExpression happens after the data has already been retrieved and is filtered client side. It does not reduce the documents pulled from the query - only whats displayed. So the next token is still going to be (relatively) the same for each query. You can instead filter it yourself after the call before the next pagination query and save yourself a little bit in terms of multiple calls.
I'm building off of a previous discussion I had with Jon Skeet.
The gist of my scenario is as follows:
Client application has the ability to create new 'PlaylistItem' objects which need to be persisted in a database.
Use case requires the PlaylistItem to be created in such a way that the client does not have to wait on a response from the server before displaying the PlaylistItem.
Client generates a UUID for PlaylistItem, shows the PlaylistItem in the client and then issue a save command to the server.
At this point, I understand that it would be bad practice to use the UUID generated by the client as the object's PK in my database. The reason for this is that a malicious user could modify the generated UUID and force PK collisions on my DB.
To mitigate any damages which would be incurred from forcing a PK collision on PlaylistItem, I chose to define the PK as a composite of two IDs - the client-generated UUID and a server-generated GUID. The server-generated GUID is the PlaylistItem's Playlist's ID.
Now, I have been using this solution for a while, but I don't understand why/believe my solution is any better than simply trusting the client ID. If the user is able to force a PK collison with another user's PlaylistItem objects then I think I should assume they could also provide that user's PlaylistId. They could still force collisons.
So... yeah. What's the proper way of doing something like this? Allow the client to create a UUID, server gives a thumbs up/down when successfully saved. If a collision is found, revert the client changes and notify of collison detected?
You can trust a client generated UUID or similar global unique identifier on the server. Just do it sensibly.
Most of your tables/collections will also hold a userId or be able to associate themselves with a userId through a FK.
If you're doing an insert and a malicious user uses an existing key then the insert will fail because the record/document already exists.
If you're doing an update then you should validate that the logged in user owns that record or is authorized (e.g. admin user) to update it. If pure ownership is being enforced (i.e. no admin user scenario) then your where clause in locating the record/document would include both the Id and the userId. Now technically the userId is redundant in the where clause because the Id will uniquely find one record/document. However adding the userId makes sure the record belongs to the user that's doing the update and not the malicious user.
I'm assuming that there's an encrypted token or session of some sort that the server is decrypting to ascertain the userId and that this is not supplied by the client otherwise that's obviously not safe.
A nice solution would be the following: To quote Sam Newman's "Building Microservices":
The calling system would POST a BatchRequest, perhaps passing in a
location where a file can be placed with all the data. The Customer
service would return a HTTP 202 response code, indicating that the
request was accepted, but has not yet been processed. The calling
system could then poll the resource waiting until it retrieves a 201
Created indicating that the request has been fulfilled
So in your case, you could POST to server but immediately get a response like "I will save the PlaylistItem and I promise its Id will be this one". Client (and user) can then continue while the server (maybe not even the API, but some background processor that got a message from the API) takes its time to process, validate and do other, possibly heavy logic until it saves the entity. As previously stated, API can provide a GET endpoint for the status of that request, and the client can poll it and act accordingly in case of an error.
I am writing a magento module for a gateway that authorizes loans as a payment system.
Since there is a relatively high probability the customer will get declined, I chose to implement this using getCheckoutRedirectURL() (placing the loan steps before 'place order' in the checkout flow) so that in the failure case, I can easily return the customer to the payment choice page.
I then do the gateway API call in my redirectAction in my controller.
As a result, I get a URL to open in a lightbox to take the customer through the loan process, as well as some id's from the loan gateway.
I would like to store these additional id's as part of the quote and later copy them to the order when I convert the quote to an order (I did the conversion similar to google checkout - based on a callback from the loan gateway).
However, I cannot figure out how to persist data on the quote.
The obvious way:
$quote->setCustId($custId);
$quote->save();
doesn't work; the additional data does not get stored in the database and hence is not available in the postback handler to convert quote to order.
The same happens for
$quote->setData('custId', $custId);
$quote->save();
(and I assume this is just a more explicit form of the first in the sense that it doesn't use the magic setter/getter)
I've seen references to setAdditionalData (for example, here) but that looks to be available only on payment objects, which I don't think I have yet on the quote (although I could be wrong ?)
Is there any way to store some fields on an order in the database without having to actually add database fields for them ?
You will need to add actual columns to sales_flat_quote table to store extra variables in quote object.
Since the stuff you are making is related to payment method instead I'd suggest you to store those variables in additional_information dataset that is stored to sales_flat_quote_payment table as serialised php array and payment methods can set data to this variable as follows:
$quote->getPayment()->getInfoInstance()->setAdditionalInformation($key, $value);
$order->getPayment()->getInfoInstance()->setAdditionalInformation($key, $value);
see more from app/code/core/Mage/Payment/Model/Info.php
the only downside for this is that it is stored as serialised array at the end which means no direct sql access later if you somehow need to depend on this data, offer filters and such.