I'm new to database design and faced a problem while trying to design one. Here's my system:
There are 3 different softwares (SW_A, SW_B, SW_C) and 500-600 users. Each user can use any of the softwares and do some work. Each work is sent to the database approx. 6 times per minute. (Randomly for users and software)
I want a database to keep all user actions related to the softwares for archive and also want to keep track instant actions for all users related to these softwares. (Only the latest action for distinct users for each software)
So I designed sth. like that:
Table: SW_X_DATA_ARCHIVE
Columns :
UserID
Data_Date
SW_A_ActionID
Table: SW_X_INSTANT_DATA
Columns:
UserID
Data_Date
SW_X_ActionID
When a new data comes, it updates INSTANT_DATA table's related row if that UserID exists, inserts new row with UserID and data if not. Then inserts the same data to DATA_ARCHIVE table.
The question is: Is there a better way for this kind of work? Views maybe?
(I've read sth. about materialized views but not sure how to use it here.)
Thanks
Only the latest action for distinct users for each software
Your INSTANT table should have a unique composite key on (USERID,SOFTWAREID) so that it is not possible to have more than one row relating to a user's use of a software.
The ARCHIVE table does not require such a unique key or constraint.
You would insert into INSTANT if a row for (USERID, SOFTWARE) does not exist, or update the row if it already exists.
Typically, an insert/update trigger would be placed on the INSTANT table, to insert a row into ARCHIVE whenever a change happens to INSTANT.
EDIT: alternatively, you could maintain a single table, and have a view that selected only the latest action per user per software. You'd need to timestamp every action and place a composite index (non-unique) on (USERID,SOFTWAREID). The decision on which approach to take might be based on relative performance. I would tend to favor the trigger if the Latest Action information was needed frequently.
Related
I have an ASP.NET Core application that uses EF6 for dealing with a third-party application's database.
Everything is working as expected, but I'm unable to insert rows into a joining table.
I have two tables, Users and Groups, and a joining table GroupUser that identifies which users are members of which groups. Users has a PK of UserId, and Groups has a PK of GroupId.
GroupUser has only 3 columns: GroupId, UserId and another column (which is irrelevant for this post). The two foreign keys in this table identify a unique record.
Every time I try to insert into GroupUser, I get the inner exception
The table/view does not have a primary key defined. The entity is read-only
The error is correct. There is no PK, but both of the FKs are marked as keys in the model. Shouldn't VS be able to use those as a PK somehow?
The inserts used to work as some point, but required some manual modification of the .edmx file as XML in order to work. Unfortunately, our version control records containing this modification have been lost (and I wasn't the one originally working on this).
I've looked at and tried about a dozen articles around this, but they generally have to do with views instead of tables, so don't seem applicable to my case. The ones that did seem applicable didn't solve the issue.
The only other clue I have for a solution is this comment I found in the code:
// Important note: If you have updated the edmx file in the [redacted]
// project and suddenly start having problems, the edmx file may need to be
// edited as an xml file so that you can make changes necessary to make
// VS believe that the GroupUser table has a primary key. See revision #[redacted]
I'm able to insert into User and Group tables just fine, and as I've said, I don't have access to the revision log mentioned.
Edit: The database is for a third-party application, and unfortunately, it's not as simple as just modifying the table to add a PK. I wish it was. Problem would be solved. But I've been advised by the vendor not to make this change, as it may have unexpected consequences, and would void our support.
How can I 'trick' EF into thinking the table has a key? I'm also open to other workarounds. Modifying the DB structure is currently out of the question.
Background
Each Dataverse table contains a primary name column. When displayed in a subgrid, clicking on the primary name column will navigate to the form so that the user can edit that row. Most subgrids in my application work this way.
The Problem
I have a Course form with a list of participants displayed in a subgrid. The subgrid displays each student's name (as a link) and the grade received in the course. There is no appropriate primary name column for this Participant table. To edit the participant record, the user must select the row in the subgrid, then click the subgrid's Edit button. As a result, this UI is different from all other subgrids in the application and I know that user's will click the student name to try to edit the participant record and be confused when they are presented with the student record.
Am I missing something? Is there a better way to handle this?
It's a common problem I face quite often. Here is usually what I would do.
Make sure the Primary Name Column always contains relevant information to the user to be able to quickly identify a record. Sometimes it requires copying information from one or multiple other columns into the primary column.
In your case that would probably means concatenating the student's name and grade.
How to do that?
Common to all solutions below
Use one of the following solution to copy the content of one or several fields into the primary column.
Make sure the solution you select also updates the content of the primary name column when one of the copied field is updated.
Remove or hide the primary column from the form, the name of the record will be displayed at the top of the form anyway and you probably don't want users to play with it.
Display the primary name column in every subgrid.
I would recommend not adding the fields copied into the primary column in the subgrids to avoid confusion.
Solution 1 - Classic Workflow
Create a classic workflow that runs when a record is created / updated
Pros:
Very quick to put in place
Runs synchronously (users will see the name updated in real-time)
Cons:
Not very practical if you need to add business logic (using different fields as source depending on a certain condition for example)
Solution 2 - Power Automate
Create a Flow that runs when a record is created / updated
Pros:
You can implement complex business logic in your Flow
Cons:
Runs asynchronously (users will have to refresh the page after the creation of a record to see the record's name)
According to Power Automate licensing that flow would certainly be considered as an "enterprise flow" and you are supposed to pay 100$ / month. That specific point must be taken with a grain of salt. I had several discussions with Microsoft about it and they haven't given me a clear answer about what would be considered an enterprise flow.
Solution 3 - Plugin
Create a plugin that executes when a record is created / updated
Pros:
You can implement very complex business logic in your Flow
It can run synchronously
Cons:
Pro-code (I put it as a con since Model-Driven App is a low-code / no-code approach but there is nothing wrong about pro-code per say)
Developing a new plugin for each entity where you need this logic is kind of overkill in my opinion. I would consider developing something very generic that would only require some sort of configuration when the logic needs to be applied to a new table.
Using Java and Oracle.
We need to update changes in Email, UserID of employee to third party.
Actual table is Employee and intermediate table we keep which we will use for comparison of changes before sending to third party.
Following are database designs coming in mind for intermediate table:
Only Single table:
EmployeeiD|Value|Type|UpdateDate
Value is userid or email, type will be 'email' or 'userid'. Update date is kept so to figure out that which of email or userid was different and update to third party.
Multiple Table:
Employee_EmailID
EmpId|EmailID|Updatedate
Employee_UserID
EmpId|UserID|Updatedate
Java flow will be:
Pick employee from actual table.
Pick employee from above intermediate table.
Compare differences. Update difference to third party.
Update above table with updated value and last update date.
Which one is consider as best way, single table approach or multiple table or is there any standard way to implement the same? There are 10,000 Employees in system.
Intermediate table is just storing Delta records i.e Records transferred to third party so that it can be compared next day.
Good database design has separate tables for different concepts. Using the same database column to hold different types of data will lead to code which is harder to understand, prone to data corruption and less performative.
You may think it's only two tables and a few tens of thousands of rows, so does it matter? But that is only your current requirement. What you choose now will set the template for what happens when (say) you need to add telephone numbers to the process.
Now in future if we get 5 more entities to update
Do you mean "entities", like say Customers rather than Employees? Or do you really mean "attributes" as in my example of Employee Telephone Number?
Generally speaking we have a separate table for distinct entities, and all the attributes of that entity are grouped at the same cardinality. To take your example, I would expect an Employee to have one UserID and one Email Address so I would design the table like this:
Employee_audit
EmpId|UserID|EmailID|Updatedate
That is, I have one record which stores the complete state of the Employee record at the Updatedate.
If we add a new entity, Customers then we have a new table. Simple. But a new attribute like Employee Phone Number offers a choice, because an employee can have more than one: work landline, mobile, fax, home, etc. So we could represent this in three ways: a child table with a type column, multiple child tables for each type, or as distinct columns on the Employee record.
For the main Employee table I would choose the separate table (or tables, depending on whether I'm shooting for 6NF). But for an audit table I would choose one record per Employee and pivot the phone numbers like this:
Employee_audit
EmpId|UserID|EmailID|Landline|Mobile|Fax|Home|Updatedate
The one thing I would never do is have a single table with type and value columns. It seems attractive because it means we could track additional entities without any further DDL. But in fact it becomes harder to re-assemble the complete state of an Employee at any given time with each attribute we add. Also it means the auditing process itself is more complicated (because it needs to determine which attributes have changed and whether it needs to audit the change) and more expensive (because changing three attributes on the same record entails inserting three audit records).
I am creating an informatica workflow which can extract data from two tables of DB2 database and load into a temporary table. Suppose the two source tables name are Account (Parent) and Activities (Child). They have 1:M relationship. Means an Account can have many Activities (Account.PK = Activities.FK). Activities table has two columns- first 'Type' whose value could be 'Paid', 'Will-Pay' or 'Not-Paid'.And second column is 'Created_Date' datetime datatype, whenever you create new activity record, date and time would get stamp in this field. Now, condition to load data in temporary table is - "For an Account record, it would 1st check in Activities table for today's Paid activities (Type = Paid). If it finds more than one paid activities, then it would pick the Latest created one (Created_Date column) out of them. If there is no Paid activity record for the Account, then it would pick latest created 'Will-Pay' activity." Means, it should pick latest Paid activity for today (Sysdate) for an Account, if it is not present then only It will pick latest Will-pay activity for today. Please help me to understand how I can implement this logic in Informatica workflow and which transformations I should use and how? Thanks alot. Kindly help.
Best way to do it on SQL cause realize business logic on ETL it's not good. But if you insist it can be created by many ways. As example:
With SQL override
You can create 3 lookup transformation for Activities table with overrided SQL (and columns too) and one expression transformation for condition.
Lookup to find more than one 'paid' activities accounts
Lookup to find last 'paid' activity per account
Lookup to find last 'will pay' activity per account
Expression to return correct Activities key based from 1-3 lookup results
Without SQL override you need to recreate similar logic with filter, aggregator, joiner transformations.
Like the title says, I've been asked to come up with an estimate for retrofitting an existing asp application.
The current security mechanism controls access to different parts of the application (page-level restrictions), but has no mechanism for flagging individual records as restricted. Assigning rights to a user (using the existing, custom access management code) is no problem, but enforcing the rights is a different matter - each asp page has embedded sql - there's no use of stored procs, objects, etc.
Is the only solution to modify each table and query, or is there a better way? Any pointers, suggestions or prayers would be welcome.
This is classic asp, running on IIS6, against an oracle database.
Update: Here's a user scenario.
We have users, managers, directors, and VPs. The managers can see data created by users who report to them, but not users who report to other managers. Users can't see data created by any managers. Same thing with directors - they can see down, but their reports can't see up.
This sounds like an ideal time to implement row-level security. Oracle has a package DBMS_RLS that allows you to define arbitrary access policies that can be applied to one or more tables that limit what rows a particular user is allowed to see. Conceptually, when a user issues a query with no filters on a protected table, i.e.
SELECT *
FROM my_table
Oracle automatically and transparently inserts a WHERE clause defined by your security policy that limits the result set. You shouldn't need to make any changes to the SQL your application is executing.
Assuming you need maximum granularity, the ability to "grant" each and any row to any of very many users, then you have a many-to-many relation, yes?
So apply the following pattern:
Add a tables of users.
Then, for each restricted table, so the following:
Rename it tablename + "_base".
create a many-to-many table that
associates that table's id with a
user id, called tablename +
"allowed_user".
create a view with the name table
name that joins tablename_base to
table_name_allowed_user, with a
select* from tablename_base and
user_id from tablename_allowed_user.
This view should meet Oracle's
requirements rto be "inherently
updatable."
Now comes the hard part. You need to add "and user_id = $user_id" to every query. Find the various functions you're using to make queries. Wrap those function(s) in ones that gets the user id from the session and add that predicate.
One passable way to do this is to read select string, find the all "where"s (for subqueries there may be more that one), and replace it with "where (user = $user) and ". For queries that don't have a where, you'll need to insert this before any "group by" or "order by". This is fragile, so obviously you'll test that this works for all pages (you have an automated test for all pages, right?), and add hacks to cover special cases.
"update" statements won't have to change; "inserts" will presumably insert both to the view and then do a separate insert to the table's "allow_user" table with the id of the inserting user, to automatically grant teh inserting user acces to what he inserted.
If your number of users is more limited, or you're restricting types of users, you can go with a strategy of multiple views named for the user or type; then you'd replace tables names in the queries with the appropriate views.