I have a dataset which has files of hotel reviews. Each file contains multiple reviews for a single hotel. Here are my two relations in BCNF:
Hotel(hotelID, OverallRating, AveragePrice, URL)
Review(hotelID, Author, Content, Date, No. Reader, No. Helpful,
Overall, Value, Rooms, Location, Cleanliness, Checkin / front desk,
Service, Business Service)
I am trying to write the following query in relational algebra:
Find all the reviews by the same user (i.e., given a user ID, return the list of all their
reviews).
By User ID, the question is referring to the Author attribute found in my second relation. The way I understand the question, it must take a user ID as an argument. Maybe you see it differently?
Here is what I have so far:
(Selection) Author = $1 (Review)
Replace selection with the sigma symbol used to represent selection in relational algebra, I was having trouble inserting it into my question. $1 represents where it would take the user ID argument, this is just to show my thinking, I do not think its correct.
Thanks for your time
Query will be:
σ(Author="Your_User Id") ( Hotel Join(X)(Hotel.hotelID=Review.hotelID) Review )
Where
σ = Selection Operator
X= Join Operator
(-----) = Condition
Hope it helps. For More detail Refer My notes for DBMS: Relational Algebra
Search "Relational Algebra" Term in site to find your exact information fast.
Related
I've looked at many threads regarding COUNT and COUNTA, but I can't seem to figure out how to use it correctly.
I am new to DAX and am learning my way around. I have attempted to look this up and have gotten a little ways to where I need to be but not exactly. I think I am confused about how to apply a filter.
Here's the situation:
Four separate queries used to generate the data in the report; but only need to use two for the DAX function (Products and Display).
I have three columns I need to filter by, as follows:
Customer (Display or Products query; can do either)
Brand (Products query)
Location (Display query)
I want to count the columns based on if the data is unique.
Here's an example:
Customer: Big Box Buy;
Item: Lego Big Blocks;
Brand: Lego;
Location: Toys;
BREAK
Customer: Big Box Buy;
Item: Lego Star Wars;
Brand: Lego;
Location: Toys;
BREAK
Customer: Big Box Buy;
Item: Surface Pro;
Brand: Microsoft;
Location: Electronics;
BREAK
Customer: Little Shop on the Corner;
Item: Red Bicycle;
Brand: Trek;
Location: Racks;
In this example, no matter the fact that the items are different, we want to look at just the customer, the brand, and the location. We see in the first two records, the customer is "Big Box Buy" and the brand is "Lego" and the location is "Toys". This appears twice, but I want to count it distinct as "1". The next "Big Box Buy" store has the brand "Microsoft" and the location is "Electronics". It appears once and only once, and thus the distinct count is "1" anyway. This means that there are two separate entries for "Big Box Buy", both with a count of 1. And lastly there is "Little Shop on the Corner" which appears just once and is counted just once.
The "skeleton" of the code I have is basically just to see if I can get a count to work at all, which I can. It's the FILTER that I think is the problem (not used in the below example) judging by other threads I've read.
TotalDisplays = CALCULATE(COUNTA(products[Brand]))
Obviously I can't just count the amount of times a brand appears as that would give me duplicates. I need it unique based on if the following conditions are met:
Customer must be the same
Brand must be the same
Location must be the same
If so, we distinctly count it as one.
I know I ranted a bit and may seem to have gone in circles, but I was trying to figure out how to explain it. Please let me know if I need to edit this post or post clarification.
Many thanks in advance as I go through my journey with DAX!
I believe I have the answer. I used a NATURALINNERJOIN in DAX to create a new, merged table since I needed to reference all values in the same query (couldn't figure out how to do it otherwise). I also created an "unique identity" calculated column that combined data from multiple rows, but was hidden behind the scenes (not actually displayed on the report) so I could then take a measure of the unique values that way.
TotalDisplays = COUNTROWS(DISTINCT('GD-DP-Merge'[DisplayCountCalcCol]))
My calculated column is as follows:
DisplayCountCalcCol = 'GD-DP-Merge'[CustID] & 'GD-DP-Merge'[Brand] & 'GD-DP-Merge'[Location] & 'GD-DP-Merge'[Order#]
So the measure TotalDisplays now reports back the distinct count of rows based on the unique value of the customer ID, the brand, and the location of the item. I also threw in an order number just in case.
Thanks!
I am semi new to DAX and was struggling with Count and CountA formula, you post has helped me with answers. I would like to add the solution which i got for my query: Wanted count for Right Time start Achieved hence if anyone is looking for this kind of answer use below, filter will be selecting the table and adding string which you want to
RTSA:=calculate(COUNTA([RTS]),VEO_Daily_Services[RTS]="RTSA")
It is hard to describe my question in the subject line. Here is an example.
I want Tableau to run query to show only Account ID that has both 2 products i selected in Product A quick filter.In this example only the second Account ID should qualify . Is this possible?
Thanks for your help in advance!
Hmm, good question. It is not possible in the way you want (at least I can't think of a way to do that), with quick filters.
I can solve your specific problem (filtering customers that have at least 2 specific products in their history), but expanding for variable n products can be really troublesome.
So first thing, create 2 parameters. Product1 and Product2. Each is a string, and you can get a list from the [Product A] field. You will use this 2 parameters to specify the 2 products you want.
Now create a calculated field, [Product flag]:
IF [Product A] = [Product1] OR [Product A] = [Product2]
THEN 1
END
Now drag [Account ID] to the filters shelf. Open the filter options and go to condition. Now select By field, [Product flag], Sum, = 2
That will work if there are not duplicated [Product A] under the same [Account ID]. If that can happen, you need a little bit more sophisticated approach. [Product Flag] becomes:
IF [Product A] = [Product1]
THEN 1
ELSEIF [Product A] = [Product2]
THEN 2
END
And the condition should be Count (Distinct) = 2
In both cases it will keep only the Account IDs that have both the products you selected under them. They can have other products under them.
EDIT: For the N product problem, I believe you're going to use a solution outside Tableau. One possibility is to use the JS API, so you can select the products you need in a JS interface, and pass a parameter to Tableau.
In JS you could have a list you could select as many items you want, and a script to pass a parameter to Tableau based on the selection. Could be something like: product1,product2,product3...
Then you could use CONTAINS() to see if that product is in that list (and raise a flag), and make a count of ',' to see how many products were selected.
Unfortunately I have very limited knowledge on JS API, but I strong encourage you to take a look
Really interesting question. It's surprisingly trickier to list the accounts that reference every product in a list than it is to list the accounts that reference any product in a list.
If you are willing to start with a less convenient user interface (suitable for ad-hoc analysis but not published dashboards) then try the following:
Create a filter based on Account Id, select Use all on the General tab, and By formula on the Condition tab. Enter the formula
Count(if [Product A] = "Business Office Consolidation" then 1 end) > 0 and Count(if [Product A] = "Cabled Barcode Scanner" then 1 end) > 0
This will only filter to only include Account IDs that reference both products. You can extend this to a list of any number of required products. For relational data sources, it is implemented using a HAVING clause.
Of course, it can be tedious to revise this formula by hand, but it is one way to accomplish your analysis goal, and it can be instructive to understand how filter conditions work. Similar formulas are useful for many conditions.
You can create one or more dynamic sets using the same approach and then use them in calculated fields, any shelf in Tableau and combine them to create new sets. You can also move the formula to a calculated field for convenience.
Note, the 1 in the formula is not significant, any non-null value would work. Since there is no else clause, the formula evaluates to null for rows that fail the if test. And the Count() function just counts the number of rows that have non-null values for the expression.
To come up with an approach that lets you easily select products from a list without editing a formula, will probably take some combination of more advanced features. I don't have an answer for you right now, but the features that are worth learning about that may or may not be part of the solution include filter actions, context filters, top filters, count distinct, custom SQL, computed sets, table calculations, LOD expressions and the Javascript API. This would also be a good questions to pose, with an example workbook, on the Tableau online forums at http://www.tableau.com under the Support menu.
Suppose I have a large (300-500k) collection of text documents stored in the relational database. Each document can belong to one or more (up to six) categories. I need users to be able to randomly select documents in a specific category so that a single entity is never repeated, much like how StumbleUpon works.
I don't really see a way I could implement this using slow NOT IN queries with large amount of users and documents, so I figured I might need to implement some custom data structure for this purpose. Perhaps there is already a paper describing some algorithm that might be adapted to my needs?
Currently I'm considering the following approach:
Read all the entries from the database
Create a linked list based index for each category from the IDs of documents belonging to the this category. Shuffle it
Create a Bloom Filter containing all of the entries viewed by a particular user
Traverse the index using the iterator, randomly select items using Bloom Filter to pick not viewed items.
If you track via a table what entries that the user has seen... try this. And I'm going to use mysql because that's the quickest example I can think of but the gist should be clear.
On a link being 'used'...
insert into viewed (userid, url_id) values ("jj", 123)
On looking for a link...
select p.url_id
from pages p left join viewed v on v.url_id = p.url_id
where v.url_id is null
order by rand()
limit 1
This causes the database to go ahead and do a 1 for 1 join, and your limiting your query to return only one entry that the user has not seen yet.
Just a suggestion.
Edit: It is possible to make this one operation but there's no guarantee that the url will be passed successfully to the user.
It depend on how users get it's random entries.
Option 1:
A user is paging some entities and stop after couple of them. for example the user see the current random entity and then moving to the next one, read it and continue it couple of times and that's it.
in the next time this user (or another) get an entity from this category the entities that already viewed is clear and you can return an already viewed entity.
in that option I would recommend save a (hash) set of already viewed entities id and every time user ask for a random entity- randomally choose it from the DB and check if not already in the set.
because the set is so small and your data is so big, the chance that you get an already viewed id is so small, that it will take O(1) most of the time.
Option 2:
A user is paging in the entities and the viewed entities are saving between all users and every time user visit your page.
in that case you probably use all the entities in each category and saving all the viewed entites + check whether a entity is viewed will take some time.
In that option I would get all the ids for this topic- shuffle them and store it in a linked list. when you want to get a random not viewed entity- just get the head of the list and delete it (O(1)).
I assume that for any given <user, category> pair, the number of documents viewed is pretty small relative to the total number of documents available in that category.
So can you just store indexed triples <user, category, document> indicating which documents have been viewed, and then just take an optimistic approach with respect to randomly selected documents? In the vast majority of cases, the randomly selected document will be unread by the user. And you can check quickly because the triples are indexed.
I would opt for a pseudorandom approach:
1.) Determine number of elements in category to be viewed (SELECT COUNT(*) WHERE ...)
2.) Pick a random number in range 1 ... count.
3.) Select a single document (SELECT * FROM ... WHERE [same as when counting] ORDER BY [generate stable order]. Depending on the SQL dialect in use, there are different clauses that can be used to retrieve only the part of the result set you want (MySQL LIMIT clause, SQLServer TOP clause etc.)
If the number of documents is large the chance serving the same user the same document twice is neglibly small. Using the scheme described above you don't have to store any state information at all.
You may want to consider a nosql solution like Apache Cassandra. These seem to be ideally suited to your needs. There are many ways to design the algorithm you need in an environment where you can easily add new columns to a table (column family) on the fly, with excellent support for a very sparsely populated table.
edit: one of many possible solutions below:
create a CF(column family ie table) for each category (creating these on-the-fly is quite easy).
Add a row to each category CF for each document belonging to the category.
Whenever a user hits a document, you add a column with named and set it to true to the row. Obviously this table will be huge with millions of columns and probably quite sparsely populated, but no problem, reading this is still constant time.
Now finding a new document for a user in a category is simply a matter of selecting any result from select * where == null.
You should get constant time writes and reads, amazing scalability, etc if you can accept Cassandra's "eventually consistent" model (ie, it is not mission critical that a user never get a duplicate document)
I've solved similar in the past by indexing the relational database into a document oriented form using Apache Lucene. This was before the recent rise of NoSQL servers and is basically the same thing, but it's still a valid alternative approach.
You would create a Lucene Document for each of your texts with a textId (relational database id) field and multi valued categoryId and userId fields. Populate the categoryId field appropriately. When a user reads a text, add their id to the userId field. A simple query will return the set of documents with a given categoryId and without a given userId - pick one randomly and display it.
Store a users past X selections in a cookie or something.
Return the last selections to the server with the users new criteria
Randomly choose one of the texts satisfying the criteria until it is not a member of the last X selections of the user.
Return this choice of text and update the list of last X selections.
I would experiment to find the best value of X but I have in mind something like an X of say 16?
Let's say I have list of persons in my datastore. Each person there may have the following fields:
last name (*)
first name
middle name
id (*)
driving licence id (*)
another id (*)
date of birth
region
place of birth
At least one of the fields marked with (*) must exist.
Now user provides me with the same list of fields (and again at least one of the fields marked with (*) must be provided). I should search for the person user provided. But not all fields should be matched. I should display to the user somehow how I am sure in the results of search. Something like:
if person matched by id and last name (and user provided just these 2 fields for the search), then I am sure that result is correct (100%);
if person matched by id and last name (and user provided other fields, which were found in the database, but were not matched), then I am sure that result is almost correct by 60%;
etc.
(numbers are provided just as example)
How can I organize such search? Is there any standard algorithm? I also would like to minimize number of requests to the database.
P.S. I can not provide user with the actual field values from the database.
It sounds like your logic for determining the quality of a match will be too complex to handle at the database layer. I think you'll get the best performance by retrieving all of the records that match at least one of the mandatory keys, calculating the match score for each of them in memory, and returning the best score. For example, if the user provides you with an id, last name and place of birth, your query would look something like:
SELECT * FROM users WHERE id = `the_id` OR last_name = `the_last_name`;
This could be a performance problem if you have a VERY large dataset with lots of common last names but otherwise I would expect not to see too many collisions. You can check this on your own dataset outside of GAE. You could also get better performance if all mandatory fields MUST match by changing the OR to an AND.
I'm a Seam newbie in an already established project, so a lot of code I use is borrowed and I'm not always fully sure how things work. My problem is that I am using a query object extended from EntityQuery to back a list page with search and sort capabilities that needs to search across a many-to-many relationship and a separate many-to-one relationship which must also be used to sort. Because the many-to-many relationship has to be joined in to allow for the search capability, the query returns duplicate records for each assignment. That's not a big deal because I just added "distinct" to the ejbql and that worked fine. However, when I try to order by the other many-to-one relationship, Oracle throws an error. It appears that Oracle will not accept an order by column that is not in the select clause when using the distinct keyword http://ora-01791.ora-code.com/, and http://oraclequirks.blogspot.com/2009/04/ora-01791-not-selected-expression.html.
Here are the relationships as they are defined in the entities: [Subject m:m JobFunction] (obviously through an assignment table [Subject o:m Subject_JobFunction m:o JobFunction]), and [Subject m:o Type]. Because I need to search Subject by JobFunction, it is joined in in the ejbql which requires the distinct keyword to only return distinct Subjects to the list page. When I try to order by the Type.name (through the many-to-one relationship), the resulting query makes Oracle angry and throws the "ORA-01791: not a SELECTed expression" error. SubjectQuery code:
#Override
public String getEjbql() {
return "select subject from Subject subject left outer join subject.jobFunctions as jobFunction";
}
#Override
#SuppressWarnings("rawtypes")
public List<ValueExpression> getRestrictions() {
ValueExpression[] RESTRICTIONS = {
createValueExpression("lower(subject.name) like #{subjectQuery.prepRestriction(subjectQuery.subject.name)}"),
createValueExpression("subject.active = #{subjectQuery.active}"),
createValueExpression("subject.type.name = #{subjectQuery.typeName}"),
createValueExpression("jobFunction.name = #{subjectQuery.jobFunctionName}")
};
return Arrays.asList(RESTRICTIONS);
}
When I set the query order when a user sorts by the Type name through the front end:
"#{subjectQuery.order=='UPPER(subject.type.name) asc'}"
I get the Oracle error. If I take the distinct out of the ejbql, the sort works fine, but I get duplicate Subject records. When I add the distinct keyword the list works fine without duplicate records, but the sort throws an error. Does anyone have any suggestions about how I can restructure the ejbql to return distinct records without the distinct keyword to make the sort happy, or how to do the sort without making Oracle angry that the sort column referenced in the query is not in the select clause? I have read several places that my answer might be in the the Hibernate Criteria API, but I have no idea how to leverage it in the context of an extended EntityQuery class with what I am trying to accomplish. Please Help!
If you are adding a DISTINCT, then something is broken.
"Because the many-to-many relationship has to be joined in to allow for the search capability, the query returns duplicate records for each assignment. "
Consider the case that a person can work on many projects and a project can have many persons. There is a uniqueness of a 'person/project'. If you want a list of people that work in either project A or B (or both) then you may get
FRED/PROJ_A
BILL/PROJ_A
FRED/PROJ_B
TOM/PROJ_B
BILL/PROJ_C
If you only show the names (not the projects), you can still order by project, but you will see
FRED
BILL
FRED
TOM
BILL
If you do a DISTINCT, you can no longer order by project, because you don't know whether the FRED is the one from PROJ_A or PROJ_B or whether BILL comes before TOM (based on PROJ_A) or after TOM (based on PROJ_C).
So remove the DISTINCT and always show the column on which you are ordering (because then you'll see why the duplicates aren't actually duplicates).
I'm not sure how the generated query(ies) is(are) different, but I found an answer. I wasn't aware of the fetch command for hibernate, which fixes the need for the distinct keyword (again, not sure exactly how, maybe by subquerying?). After changing the ejbql to:
#Override
public String getEjbql() {
return "select subject from Subject subject left join fetch subject.jobFunctions jobFunction";
}
the distinct is no longer needed and therefore, Oracle does not complain about the order by column not being in the select clause. The list works as expected and the sort column works! Yay!
Predictably, I found the answer here on stackoverflow. The question was not exactly the same, but the hql syntax worked for me: HQL order by within a collection