I’m using Apache Kylin, version: apache-kylin-2.6.2-bin-hadoop3/
In Insight query, I find the derived dimension column always returns an empty result. But using the same SQL in Hive shows that the column has data.
Here is the SQL I'm trying:
SELECT trans.user_id, user.tag1
FROM trans
INNER JOIN user ON trans.user_id = user.id;
...where user.tag1 is the derived dimension, which is empty.
If I only change dimension type from derived to normal and rebuild the cube, there's no such problem.
What's even more strange is, if I try another lookup table, the result shows that some rows have a value and some don't. I cannot find the difference between these correlations, and everything is fine in Hive query.
What might the problem be?
Related
I have a BigQuery table with point registers along a whole country, and I need to assign a "censal zone" to each one of them, which polygons are contained in another table. I've been trying to do so using a query like this one:
SELECT id_point, code_censal_zone
FROM `points_table`
JOIN `zones_table`
ON ST_CONTAINS(zone_polygon, point_geo)
The first table is quite large, so the query performes very inefficiently as it is comparing each possible pairs of (point, censal zone). However, both tables have a column identifier for the municipality in which they are in, so the question is, can rewrite my query in some way that ST_CONTAINS(*) is performed for each (point, censal zone) pair that belongs to the same municipality, hence not comparing all posible censal zones within the country for each point? Can I do this without having to read points_table multiple times?
SELECT id_point, code_censal_zone
FROM `points_table`
JOIN `zones_table`
ON 1.municipality = 2.municipality
AND ST_CONTAINS(zone_geo, point_geo)
I'm quite new to BigQuery so I don't really know if a query like this would actually do what I'am expecting, as I couldn't find anything in the documentation.
Thanks!
SELECT id_point, code_censal_zone
FROM `points_table`
JOIN `zones_table`
ON 1.municipality = 2.municipality
AND ST_CONTAINS(zone_geo, point_geo)
For the past couple of days, I have been trying to overcome a weird issue within Power Query in Power BI Desktop.
I have two tables, one is called the "Mappings" table, and the purpose of this table is to give me a column called "FMReference" after a bunch of transformations.
Mappings Screenshot
I have my main table called "Entry", which DOESN'T HAVE FMReference, and thus, needs to be merged with the Mappings table. To make the merge simpler, I have created a simple column on both tables named "CombinedForMerge". Entry Screenshot
After the setup for the columns are complete, I proceed to create a Left Outer Merge with the Entry and Mappings tables, using the columns I have just made. Left Outer Merge Screenshot
As you can see on the screenshot, the FMReference column says A4.28, which is what I am expecting on my results. However, once I expand that merge to give me the FMReference, I am getting a different value, A3.2! Wrong Value Screenshot
Power BI clearly cached the wrong value on PowerQuery, as I have filtered the Mappings table to JUST one row, to get JUST the one value. Filtered Mappings Screenshot
I have tried clearing the cache from the Power BI Options, but this didn't change anything.
I've also tried rebuilding the Power BI report from a clean, new PBIX, just copying the transformations up to this point, but I am still getting the same issue.
I've tried copying the results of the Mappings table onto a static table, and used it for the merge instead. This worked, which clearly means it is something to do caching on the data transformations on Power BI.
I've refreshed the Preview, as well as Refreshed ALL a few times, but it still returns the wrong result.
I would appreciate any help at this point, the data is very sensitive so I cannot share a lot, but I am happy to give more details if needed.
Thanks in advance!
I ran into similar inexplicably odd behavior when using fuzzy merge, set for an exact match. I wound up swapping the tables and using a Right Outer merge instead. For some reason, that worked. I have no idea what caused the problem or why the Right Outer merge approach worked though. It was quite odd.
So what I'm saying is: Instead of starting your merge from XXXXEntry and matching to XXXXMappings with a Left Outer merge, you might try starting from XXXXMappings and matching to XXXXEntry with a Right Outer merge.
One way to do that, if you still want the merge in the same place in the same query, is to just edit your existing query's code in the formula bar. To do that, click on the Applied Step that is associated with the merge, most likely called Merged Queries; then in the formula bar, swap the references to the tables and change JoinKind.LeftOuter to JoinKind.RightOuter. (One table reference will likely be the name of the Applied Step before your Merged Queries step, instead of XXXXEntry. For me, in the before and after examples that follow, it was #"Changed Type".)
Before:
= Table.NestedJoin(#"Changed Type", {"CombinedForMerge"}, XXXXMappings, {"CombinedForMerge"}, "Changed Type", JoinKind.LeftOuter)
After:
= Table.NestedJoin(XXXXMappings, {"CombinedForMerge"}, #"Changed Type", {"CombinedForMerge"}, "XXXXMappings", JoinKind.RightOuter)
Good luck.
I have implemented laravel dataTable as a service.
The initial two columns are actual id and names so, I am able to sort it asc/desc after the table renders.
But the next few columns renders after performing few calculations, i.e. these values are not fetched directly from any column rather it is processed.
I am unable to sort these columns where calculations were performed, and I get this error. And I know it is looking for that particular column for eg outstanding_amount which I don't have in the DB, rather it is a calculated amount from two or more columns that are in some other tables.
Any Suggestions on how to overcome this issue?
It looks like you're trying to sort by values that aren't columns, but calculated values.
So the main issue here is to give Eloquent/MySql the data it needs to provide the sorting.
// You might need to do some joins first
->addSelect(DB::raw('your_calc as outstanding_amount'))
->orderBy('outstanding_amount') // asc can be omitted as this is the default
// Anternative: you don't need the value sorted by
// Don't forget any joins you might need
->orderByRaw('your_calc_for_outstanding_amount ASC')
For SQL functions it'll work as follow
->addSelect(DB::raw('COUNT(products.id) as product_count'));
->orderByRaw(DB::raw('COUNT(products.id)'),'DESC');
Say I have an entity MyEntity, and it has a formula-based property fmlaProp. Now say I create a criteria:
s.createCriteria(MyEntity.class)
.setProjection(
Projections.distinct(
Projections.property("fmlaProp")))
.addOrder(Order.asc("fmlaProp"));
in this case I get the following SQL:
SELECT DISTINCT fmlaProp-sql FROM MY_ENTITY_TABLE ORDER BY fmlaProp-sql
Which gives an error on Oracle saying that order-by expression is non-selected. Then I tried the following criteria:
s.createCriteria(MyEntity.class)
.setProjection(
Projections.distinct(
Projections.alias(
Projections.property("fmlaProp"),
"alias1"))
.addOrder(Order.asc("alias1"));
Which generates "order by alias1" which works fine. But it is kind of ugly -- the code must "know" of those formula properties, which violates "write once" principle. Any thoughts or suggestions on that? Thank you in advance.
This is expected behavior from Hibernate. It doesn't have to do with the formula property specifically, but that you want to do ordering with a projected value. From the Hibernate Docs:
An alias can be assigned to a projection so that the projected value can be referred to in restrictions or orderings. Here are two different ways to do this...
As far as alternatives, you could try making the formula property a virtual column (in versions of Oracle 11 and above) or wrapping the table in a view with this column computed. That way, Oracle will know fmlaprop directly, which can be used just like a "normal" column.
I am looking for a way to have a query that returns a tuple first sorted by a column, then grouped by another (in that order). Simply .sort_by().group_by() didn't appear to work. Now I tried the following, which made the return value go wrong (I just got the orm object, not the initial tuple), but read for yourself in detail:
Base scenario:
There is a query which queries for test orm objects linked from the test3 table through foreign keys.
This query also returns a column named linked that either contains true or false. It is originally ungrouped.
my_query = session.query(test_orm_object)
... lots of stuff like joining various things ...
add_column(..condition that either puts 'true' or 'false' into the column..)
So the original return value is a tuple (the orm object, and additionally the true/false column).
Now this query should be grouped for the test orm objects (so the test.id column), but before that, sorted by the linked column so entries with true are preferred during the grouping.
Assuming the current unsorted, ungrouped query is stored in my_query, my approach to achieve this was this:
# Get a sorted subquery
tmpquery = my_query.order_by(desc('linked')).subquery()
# Read the column out of the sub query
my_query = session.query(tmpquery).add_columns(getattr(tmpquery.c,'linked').label('linked'))
my_query = my_query.group_by(getattr(tmpquery.c, 'id')) # Group objects
The resulting SQL query when running this is (it looks fine to me btw - the subquery 'anon_1' is inside itself properly sorted, then fetched and its id aswell as the 'linked' column is extracted (amongst a few other columns SQLAlchemy wants to have apparently), and the result is properly grouped):
SELECT anon_1.id AS anon_1_id, anon_1.name AS anon_1_name, anon_1.fk_test3 AS anon_1_fk_test3, anon_1.linked AS anon_1_linked, anon_1.linked AS linked
FROM (
SELECT test.id AS id, test.name AS name, test.fk_test3 AS fk_test3, CASE WHEN (anon_2.id = 87799534) THEN 'true' ELSE 'false' END AS linked
FROM test LEFT OUTER JOIN (SELECT test3.id AS id, test3.fk_testvalue AS fk_testvalue
FROM test3)
AS anon_2 ON anon_2.fk_testvalue = test.id ORDER BY linked DESC
)
AS anon_1 GROUP BY anon_1.id
I tested it in phpmyadmin, where it gave me, as expected, the id column (for the orm object id), then the additional columns SQL_Alchemy seems to want there, and the linked column. So far, so good.
Now my expected return values would be, as they were from the original unsorted, ungrouped query:
A tuple: 'test' orm object (anon_1.id column), 'true'/'false' value (linked column)
The actual return value of the new sorted/grouped query is however (the original query DOES indeed return a touple before the code above is applied):
'test' orm object only
Why is that so and how can I fix it?
Excuse me if that approach turns out to be somewhat flawed.
What I actually want is, have the original query simply sorted, then grouped without touching the return values. As you can see above, my attempt was to 'restore' the additional return value again, but that didn't work. What should I do instead, if this approach is fundamentally wrong?
Explanation for the subquery use:
The point of the whole subquery is to force SQLAlchemy to execute this query separately as a first step.
I want to order the results first, and then group the ordered results. That seems to be hard to do properly in one step (when trying manually with SQL I had issues combining order and group by in one step as I wanted).
Therefore I don't simply order, group, but I order first, then subquery it to enforce that the order step is actually completed first, and then I group it.
Judging from manual PHPMyAdmin tests with the generated SQL, this seems to work fine. The actual problem is that the original query (which is now wrapped as the subquery you were confused about) had an added column, and now by wrapping it up as a subquery, that column is gone from the overall result. And my attempt to readd it to the outer wrapping failed.
It would be much better if you provided examples. I don't know if these columns are in separate tables or what not. Just looking at your first paragraph, I would do something like this:
a = session.query(Table1, Table2.column).\
join(Table2, Table1.foreign_key == Table2.id).\
filter(...).group_by(Table2.id).order_by(Table1.property.desc()).all()
I don't know exactly what you're trying to do since I need to look at your actual model, but it should look something like this with maybe the tables/objs flipped around or more filters.