I think I can ask this question without any example data:
Let's say I have a large table, maybe 100 columns and a million rows. On each row, there are only a few valid combinations of data, but that only applies to five of the columns. The other 95 columns can really be anything.
I have another validation table of just those five relevant columns, where each row is a valid combination of values. Can I use this to create a "Valid/Invalid" flag on the main table?
I was thinking that we wouldn't be making a relationship between these tables, although, every validation column will also exist in the main table, so I suppose you could? In my real-world case there are no good primary keys, but if the solution is a many-to-many relationship I'll edit the question and title here.
Edit: the above was answered but here's a more difficult variation:
What if rows in the main table don't all include all five validation values? The validation table shows valid combinations, but nulls are allowed in the main table?
Related
I don't know if I even worded the question correctly, but I'm trying to create a measure that depends on what is showing in the pivot table (using PowerPivot). In the image I posted, "DealMonth" is an expression in the PowerQuery table itself that simply takes the start date of the employee and subtracts it from the month a deal was closed in. That will show how long it took for that salesperson to close the deal. "TenureMonths" is also an expression in the PowerQuery table that calculates the tenure of the person. The values populating this screenshot are coming from a total headcount measure created. What I'm trying to do is create a separate measure that will show when the "TenureMonths" is less than the "DealMonth." So if the TenureMonths is 5, then after DealMonth of 5, the value would be 0. Is this possible?
Screenshot
I should add the following information.
"DealMonth" - Comes from the FactData table
"TenureMonths" - Comes from the DimSalesStart table
These two tables are joined by name. I feel like I'm so close because I can see what I want. The second image below is a copy/paste of the pivot table result but with my edits to show what I'd want to have shown. Basically, if(TenureMonths >= DealMonth,1,0). The trouble seems to be that since they're in two different tables, I can't make it work. The rows in the fact table are transactions, but the rows in the dim table are just the people with their start and end dates.
Desired Result
This is possible with some IF([measure1]<[measure2],blank(),[measure1]), however without seeing more of the data it will be hard to guide you specifically.
However you need to create two separate measures, one for TenureMonths and one for DealMonth, depending on the data this can be done with an aggregator forumla such as sum, min, max, etc (depends if there will be more than one value).
Then reference those two measures in the formula pattern I mentioned above, and that should give you want you want.
I figured out a solution. I added a dimension table for DealMonth itself and joined to my fact table. That allowed me to do the formulas that I needed.
I have Table 1. It is filled with dates a inspection is going on. Plumbing or Garden inspections for example.
Table 2 links to these appointments and has additional columns with details such as a Person assigned to the inspection, and what property the inspection is at. I need these two tables to be separate as described, and they are linked by a simple ID column.
Is it possible at all to add a column to Table 2 called 'Last Date of Plumbing Inspection'. The idea is for any given Property in Table 2, there can be multiple inspection entries in Table 1 for it. The point of this column is that it should look in Table 1, find the matching ID, find the latest inspection date out of all the Plumbing-related inspections, and then set the column value to that.
The problem I am having with this is it seems like calculated columns can ONLY implement logic using the columns of the table the calculated column was created in. In Table 2, I can't create a calculated column that interacts with Table 1 at all. I could create a look up column, but I can't combine calculated columns with look up columns. Is there a way to build this latest inspection date column without too much complexity?
Actually you can create a Rollup field and put a MAX aggregate function for achieving your requirement from related table. Read more
I have one table that has many fields,
Is it possible to count how many fields that filled?
I tried count() but seems doesnt work correctly, it only count the rows of the table
The purpose is i want to take the percentage of each student's completed task
This is the view from the page
This is the table
I think your table structure is messy.
why not, setup a one to many relation instead of doing that.
https://laravel.com/docs/8.x/eloquent-relationships#one-to-many-inverse
I would also suggest you use increment, every new task completed you increment the value. Please refer to this https://laravel.com/docs/8.x/queries.
I have designed my database in such a way that One of my table contains 52 columns. All the attributes are tightly associated with the primary key attribute, So there is no scope of further Normalization.
Please let me know if same kind of situation arises and you don't want to keep so many columns in a single table, what is the other option to do that.
It is not odd in any way to have 50 columns. ERP systems often have 100+ columns in some tables.
One thing you could look into is to ensure most columns got valid default values (null, today etc). That will simplify inserts.
Also ensure your code always specifies the columns (i.e no "select *"). Any kind of future optimization will include indexes with a subset of the columns.
One approach we used once, is that you split your table into two tables. Both of these tables get the primary key of the original table. In the first table, you put your most frequently used columns and in the second table you put the lesser used columns. Generally the first one should be smaller. You now can speed up things in the first table with various indices. In our design, we even had the first table running on memory engine (RAM), since we only had reading queries. If you need to get the combination of columns from table1 and table2 you need to join both tables with the primary key.
A table with fifty-two columns is not necessarily wrong. As others have pointed out many databases have such beasts. However I would not consider ERP systems as exemplars of good data design: in my experience they tend to be rather the opposite.
Anyway, moving on!
You say this:
"All the attributes are tightly associated with the primary key
attribute"
Which means that your table is in third normal form (or perhaps BCNF). That being the case it's not true that no further normalisation is possible. Perhaps you can go to fifth normal form?
Fifth normal form is about removing join dependencies. All your columns are dependent on the primary key but there may also be dependencies between columns: e.g, there are multiple values of COL42 associated with each value of COL23. Join dependencies means that when we add a new value of COL23 we end up inserting several records, one for each value of COL42. The Wikipedia article on 5NF has a good worked example.
I admit not many people go as far as 5NF. And it might well be that even with fifty-two columns you table is already in 5NF. But it's worth checking. Because if you can break out one or two subsidiary tables you'll have improved your data model and made your main table easier to work with.
Another option is the "item-result pair" (IRP) design over the "multi-column table" MCT design, especially if you'll be adding more columns from time to time.
MCT_TABLE
---------
KEY_col(s)
Col1
Col2
Col3
...
IRP_TABLE
---------
KEY_col(s)
ITEM
VALUE
select * from IRP_TABLE;
KEY_COL ITEM VALUE
------- ---- -----
1 NAME Joe
1 AGE 44
1 WGT 202
...
IRP is a bit harder to use, but much more flexible.
I've built very large systems using the IRP design and it can perform well even for massive data. In fact it kind of behaves like a column organized DB as you only pull in the rows you need (i.e. less I/O) rather that an entire wide row when you only need a few columns (i.e. more I/O).
I have an assignment where I have two tables. Both of these two tables have multiple records that can be grouped by a certain ID creating record sets within those two tables
Those record sets can have various number of records. The trick is I have to compare those two tables and compare them by those record sets. If one record set ordered by update date (one of the record fields) doesn't find an identical record set in another table, I have to output that record set
What is the best way to do it? How do I compare two different tables by record groups/record sets/record blocks?
Should I use sub-query factoring? Should I temporary tables? Should I use something else?
Thank you very much for your generous responses and please let me know if I made my question unclear
i guess you just need a minus query to show the differences.
If you use Toad there is a specific function. Or you can use the minus operator or read this other post link