We are trying to enable collaborative chats in an educational application. The general idea is that the users (students) will start an exercise, this will make them join an specific waiting room. And somehow the system should be able to decide it has enough students to create a group of n students (depending on the exercise) according to a given strategy and will send a message to those students to join a chatroom with a generated ID so that they can collaborate.
Right now we are totally blank and cannot decide on how to make the server decide wether to try and create groups or to wait for more students. Our stack is in Spring Boot, Redis and Postgress. Our initial idea was to add the students into a waiting room in Redis and launch a Spring event every time a student joined the waitlist. However, we understand that approach might generate many race conditions, which should be avoided.
Create an exercise_students table which has a SERIAL column on it, call it arrival_order or something. Another column for group_id. As students sign in for the exercise, insert them into this table. By nature SERIAL is atomically auto-incremented, so you avoid race conditions. Regularly query the table for students with no group_id (I assume you have a exercise_group table of some sort that defines how many students are part of a group). When the count reaches n, update them with the group_id and create a new group in the exercise_group table for the next group.
Relational databases are pretty good at this sort of thing. Atomic updating of state is pretty straightforward stuff.
Related
I have a (skills) table and (users) table, and I have created (skill_user) table as a pivot,
but my problem is how can I let the user insert any skills he wants, and at the same time it is many to many relationships, which means the user should select the skills which already exist? i cannot understand how can I fix this issue?
There are 3 methods to work with many-to-many relationships:
attach
detach
sync.
I would say sync is more suitable for your case: user selects a list of skills, and you save only these. This also allows to remove skills if user unselected one.
This is kind of an odd situation and I am pretty new to RLS so please forgive me if what I am asking about here might seem a little silly. I am trying to create Row Level Security for a School District. I have a table that has the different schools codes, employee IDs and their position. I have another table that has the employee ID for teachers and their Teacher ID along with the ID if the students they have for the current year with a separate row for each bell period.
I have tried to create a bridge table that contains the Employee ID, Teacher ID as well as the School Codes and connected it with the other two tables.
For testing purposes, I am trying to connect it to the students basic information and set security to see how to give teachers access. I feel like I am almost there but I might be missing something out in here.
Can you please tell me how to go forward from here. Thank you
Suppose we have two microservices, Customers and Orders, with no dependencies between them, i.e. they don't call each other, they don't know each other. Every order, though, has a reference to a customer by means of a customer id. In other words one customer may have zero or more orders, and one order belongs to exactly one customer.
For the sake of the example, it's totally fine to delete a customer unless there are orders belonging to that customer. What is the most appropriate way to implement a constraint on Customers that prevents a customer from being deleted if one or more orders have a reference to that customer? Analogous to referential integrity in a relational database.
These are the approaches I can think of:
Let Customers ask Orders if a given customer has any orders, e.g. via API call.
Let Customers keep track of which orders are assigned to every customer, e.g. by having each customer record maintain a list of order ids.
Merge Customers and Orders into a single microservice and manage it internally.
I'm not sure which approach is the best for the given example in a microservices context. I can see pros and cons in all three approaches but I won't list them here because I'd like to hear other people's thoughts on the problem, including approaches not listed above. Thanks.
Probably the second approach would help if you're going to decouple through events, either tracking a list of ids or a counter just telling how many orders are stored for such a Customer.
On the Order microservice you will emit an event when there is a creation/deletion that will be captured by the Customer (or any other microservice interested) who will take care of updating the list of order ids (or increment/reduce the counter).
If customer order counter is 0 then you may delete the customer.
Let's start with your third approach: This will not work in a Microservice world, because you will always have those constraints between some Services. And if you want to solve all of them this way, you'll end up with a Monolith - and that's the end of your Microservice story.
The first and second approach have both the same "problem": These are async operations, that may return false positive (or false negative) results: It's possible to make api requests for delete customer and create order (or delete order) at the same time.
Though this can happen:
For your first approach: Customer Service asks Order Service if there are any Orders for this Customer. Order Service returns 0. And at the same time Order Service creates a new Order for that Customer in another thread. So you end up with a deleted Customer and still created an Order.
The same applies for your second approach: The messaging between those services is async. Though it's possible that Customer Service knows of 0 Orders, and permits the delete. But at the same time the Order Service creates a new Order for this Customer. And the OrderCreated message will hit the Customer Service after the Customer has already been deleted.
If you try to do it the other way around, you'll end up with the same situation: Your Order Service could listen to CustomerDeleted messages, and then disallow creating new Orders for this Customer. But again: This message can arrive while there are still Orders in the database for this Customer.
Of course this is very unlikely to happen, but it still is possible and you cannot prevent it in an async Microservice world without transactions (which of course you want to avoid).
You should better ask yourself: How should the system handle Orders where the corresponding Customer has been deleted?
The answer to this question is most likely dependent on your business rules. For example: If the Order Service receives a CustomerDeleted message, it may be okay to simply delete all Orders from this Customer. Or maybe the behavior depends on the Order's state property: It's okay to delete all Orders with state = draft, but every other Order from this Customer should still be processed and shipped as usual.
I have two scenarios that I want to support but I don’t know the best way to design relations in the elasticsearch. I read the entire elasticsearch documentation but I couldn’t find the best way to design types for my scenarios.
Multiple one to many.
Let’s assume that I have the following tables in my relational database that I want to transfer to the elasticsearch:
Transaction table Id User1Id User2Id ….
User table Id Name
Transaction contains two references to User. As far as I know I cannot use the parent->child relation specifying two parents? I need to store transaction and user in separate types because they can be changed separately. I need to be able to search transaction through user details and return users connected with transactions. Any idea how to design such structure in the elastic search?
Many to many
Let’s assume that we have the following tables:
Order Id …
OrderLine OrderId UserId Amount …
User Id Name
Order line is always saved with the order so I thought that I can store order with order lines as a nested object relation but the user must be in the separate table. Is there any way how can I connected multiple users from order line with user type? I assume that I can use application side join but I need to retrieve order and order line always together and be able to search order by user data.
I can use grandparent and grandchildren relations but then I need to make joins in the application. Any idea how to design it in the best way?
We use CRM 4.0 at our institution and have no plans to upgrade presently as we've spend the last year and a half customising and extending the CRM to work with our processes.
A tiny part of model is a simply hierarchy, we have a group of learning rooms that has a one-to-many relationship with another entity that describes the courses available for that learning room.
Another entity has a list of all potential and enrolled students who have expressed an interest in whichever course.
That bit's all straightforward and works pretty well and is modelled into 3 custom entities.
Now, we've got an Admin application that reads the rooms and then wants to show the courses for that room, but only where there are enrolled students.
In SQL this is simplified to:
SELECT DISTINCT r.CourseName, r.OtherInformation
FROM Rooms r
INNER JOIN Students S
ON S.CourseId = r.CourseId
WHERE r.RoomId = #RoomId
And this indeed is very close to the eventual SQL that CRM generates.
We use a Crm QueryEntity, a Filter and a LinkEntity to represent this same structure.
The problem now is that the CRM normalizes the a customize entity into a Base Table which has the standard CRM entity data that all share, and then an ExtensionBase Table which has our customisations. To Give a flattened access to this, it creates a view that merges both tables.
This view is what is used by the Generated SQL.
Now the base tables have indices but the view doesn't.
The problem we have is that all we want to do is return Courses where the inner join is satisfied, it's enough to prove there are entries and CRM makes it SELECT DISTINCT, so we only get one item back for Room.
At first this worked perfectly well, but now we have thousands of queries, it takes well over 30 seconds and of course causes a timeout in anything but SMS.
I'm given to believe that we can create and alter indices on tables in CRM and that's not considered to be an unsupported modification; but what about Views ?
I know that if we alter an entity then its views are recreated, which would of course make us redo our indices when this happens.
Is there any way to hint to CRM4.0 that we want a specific index in place ?
Another source recommends that where you get problems like this, then it's best to bring data closer together, but this isn't something I'd feel comfortable in trying to engineer into our solution.
I had considered putting a new entity in that only has RoomId, CourseId and Enrolment Count in to it, but that smacks of being incredibly hacky too; After all, an index would resolve the need to duplicate this data and have some kind of trigger that updates the data after every student operation.
Lastly, whilst I know we're stuck on CRM4 at the moment, is this the kind of thing that we could expect to have resolved in CRM2011 ? It would certainly add more weight to the upgrading this 5 year old product argument.
Since views are "dynamic" (conceptually, their contents are generated on-the-fly from the base tables every time they are used), they typically can't be indexed. However, SQL Server does support something called an "indexed view". You need to create a unique clustered index on the view, and the query analyzer should be able to use it to speed up your join.
Someone asked a similar question here and I see no conclusive answer. The cited concerns from Microsoft are Referential Integrity (a non-issue here) and Upgrade complications. You mention the unsupported option of adding the view and managing it over upgrades and entity changes. That is an option, as unsupported and hackish as it is, it should work.
FetchXml does have aggregation but the query execution plans still uses the views: here is the SQL generated from a simple select count from incident:
'select
top 5000 COUNT(*) as "rowcount"
, MAX("__AggLimitExceededFlag__") as "__AggregateLimitExceeded__" from (select top 50001 case when ROW_NUMBER() over(order by (SELECT 1)) > 50000 then 1 else 0 end as "__AggLimitExceededFlag__" from Incident as "incident0" ...
I dont see a supported solution for your problem.
If you are building an outside admin app and you are hosting CRM 4 on-premise you could go directly to the database for your query bypassing the CRM API. Not supported but would allow you to solve the problem.
I'm going to add this as a potential answer although I don't believe its a sustainable or indeed valid long-term solution.
After analysing the indexes that CRM had defined automatically, I realised that selecting more information in my query would be enough to fulfil the column requirements of an Index and now the query runs in less then a second.