How can i dynamically transform an SQL-Query?
I know there is a Select.getSelect(), but how can i add fields in the select-query?
Use-case: for a Rest-Query i have a lot of paginated resources and i have an abstraction to create the paginated-query. It takes the SelectConditionStep and adds the rest, depending on additional parameters. It works really well for simple queries, but for queries containing joins a little bit of transformation of the query would required. (Mainly because i can't naively limit the number results, since the join can be a one to many relationship)
The easiest way is to keep a List<Field<?>> where you add the fields for your select() clause, and then create the Select statement only when you actually execute it, instead of passing a Select object around. Example:
List<Field<?>> fields = new ArrayList<>();
// Just some examples:
fields.addAll(getDefaultFields());
fields.addAll(getFieldsFromUI());
fields.addAll(getCalculatedFields());
// Much later on, you finally create the statement:
DSL.using(configuration)
.select(fields)
.from(...)
.fetch();
Related
I am trying to create an SAP data query using SQVI (or SQ01) to display all entries that meet certain criteria. I can use the 'Selection Fields' tab to have a user specify any of these parameters, but I want to be able to query the data with more complex conditions such as a nested 'AND'/'OR'. I have researched the question for a couple hours and have yet to find a solution that works. Here is an example simplified query that I would like to do, written in SQL form:
SELECT t0.name, t0.birthYear, t1.grade, t1.county
FROM t0
INNER JOIN t1 on t0.personID = t1.personID
WHERE t0.name = 'Bob'
AND t0.birthyear = 2000
AND (t1.grade = 12
OR t1.county <> 'Cook');
Now the tricky part is figuring out how to do a nested 'AND' and 'OR' in SQVI. At first, I pulled all the data without these conditions, exported it to Excel, and then performed this logic to get the correct entries that meet these criteria. However, I do not want to do this every time, as it is highly repetitive and there HAS TO be some solution within the SAP environment. Ideally, I would be able to create a query that I can share with co-workers to execute once a week, where they don't need to enter any values to test against 'name', 'birthyear', 'grade', or 'county'. They should be able to type in the code for this query and hit execute, and it should spit out all of the entries that meet all the criteria. I want to be able to hard-code the testing parameters in this instance.
Let me know if this is even possible! If it's not possible using SQVI, what would I need access to in order to do a complex conditional query like this? I do not have write-access on the data, so I am not authorized to use 'DBACOCKPIT' to write the query as SQL (which would be so much simpler).
I have a somewhat complicated #Query in a JpaRepository.
I need to get the results of this query in two forms (but not at the same time!):
First, the client asks for a count of the number of results: SELECT COUNT(x.*) FROM my_table x ...
Then later (maybe), they want to see the actual data: SELECT x.* FROM my_table x ...
What follows (the ...) is identical for both queries. Is there any way to combine these so that I don't repeat myself?
I know I could just use the second method, and count the number of elements in the resulting List. However, this adds the overhead of actually fetching all those elements from the database.
I could put the ... in a String constant somewhere, but that kind of separates it from its context (I'd lose IntelliJ's syntax highlighting/error checking)
I can't convert it to a Criteria or Example query, because I need to use PostGIS's geography type. (And these are less readable anyway...)
Any other ideas?
If your worries is about some developer change the COUNT query and forgot to change the SELECT query too, you can create a repository integration test to guarantee the expected result between the two queries.
Another alternative is create a unit test to read the annotation content and verify if the final of these two queries are equal.
I'm having a problem joining two LINQ queries.
Currently, my (original) code looks like this
s.AnimalTypes.Sort((x, y) => string.Compare(x.Type, y.Type));
What I'm needing to do is change this based on a date, then select all data past that date, so I have
s.AnimalTypes.Select(t=>t.DateChanged > dateIn).ToList()
s.AnimalTypes.Sort((…
This doesn't look right as it's not sorting the data selected, rather sorting everything in s.AnimalTypes.
Is there a way to concatenate the two LINQ lines? I've tried
s.AnimalTypes.Select(t=>t.DateChanged > dateIn).ToList().Sort((…
but this gives me an error on the Sort section.
Is there a simple way to do this? I've looked around and Grouo and OrderBy keep cropping up, but I'm not sure these are what I need here.
Thanks
From your description, I believe you want something like:
var results = s.AnimalTypes.Where(t => t.DateChanged > dateIn).OrderBy(t => t.Type);
You can call ToList() to convert to a List<T> at the end if required.
There are a couple of fundamental concepts I believe you are missing here -
First, unlike List<T>.Sort, the LINQ extension methods don't change the original collections, but rather return a new IEnumerable<T> with the filtered or sorted results. This means you always need to assign something to the return value (hence my var results = above).
Second, Select performs a mapping operation - transforming the data from one form to another. For example, you could use it to extract out the DateChanged (Select(t => t.DateChanged)), but this would give you an enumeration of dates, not the original animal types. In order to filter or restrict the list with a predicate (criteria), you'd use Where instead.
Finally, you can use OrderBy to reorder the resulting enumerable.
You are using Select when you actually want to use Where.
Select is a projection from one a collection of one type into another type - you won't increase or reduce the number of items in a collection using Select, but you can instead select each object's name or some other property.
Where is what you would use to filter a collection based on a boolean predicate.
I am looking for a way to have a query that returns a tuple first sorted by a column, then grouped by another (in that order). Simply .sort_by().group_by() didn't appear to work. Now I tried the following, which made the return value go wrong (I just got the orm object, not the initial tuple), but read for yourself in detail:
Base scenario:
There is a query which queries for test orm objects linked from the test3 table through foreign keys.
This query also returns a column named linked that either contains true or false. It is originally ungrouped.
my_query = session.query(test_orm_object)
... lots of stuff like joining various things ...
add_column(..condition that either puts 'true' or 'false' into the column..)
So the original return value is a tuple (the orm object, and additionally the true/false column).
Now this query should be grouped for the test orm objects (so the test.id column), but before that, sorted by the linked column so entries with true are preferred during the grouping.
Assuming the current unsorted, ungrouped query is stored in my_query, my approach to achieve this was this:
# Get a sorted subquery
tmpquery = my_query.order_by(desc('linked')).subquery()
# Read the column out of the sub query
my_query = session.query(tmpquery).add_columns(getattr(tmpquery.c,'linked').label('linked'))
my_query = my_query.group_by(getattr(tmpquery.c, 'id')) # Group objects
The resulting SQL query when running this is (it looks fine to me btw - the subquery 'anon_1' is inside itself properly sorted, then fetched and its id aswell as the 'linked' column is extracted (amongst a few other columns SQLAlchemy wants to have apparently), and the result is properly grouped):
SELECT anon_1.id AS anon_1_id, anon_1.name AS anon_1_name, anon_1.fk_test3 AS anon_1_fk_test3, anon_1.linked AS anon_1_linked, anon_1.linked AS linked
FROM (
SELECT test.id AS id, test.name AS name, test.fk_test3 AS fk_test3, CASE WHEN (anon_2.id = 87799534) THEN 'true' ELSE 'false' END AS linked
FROM test LEFT OUTER JOIN (SELECT test3.id AS id, test3.fk_testvalue AS fk_testvalue
FROM test3)
AS anon_2 ON anon_2.fk_testvalue = test.id ORDER BY linked DESC
)
AS anon_1 GROUP BY anon_1.id
I tested it in phpmyadmin, where it gave me, as expected, the id column (for the orm object id), then the additional columns SQL_Alchemy seems to want there, and the linked column. So far, so good.
Now my expected return values would be, as they were from the original unsorted, ungrouped query:
A tuple: 'test' orm object (anon_1.id column), 'true'/'false' value (linked column)
The actual return value of the new sorted/grouped query is however (the original query DOES indeed return a touple before the code above is applied):
'test' orm object only
Why is that so and how can I fix it?
Excuse me if that approach turns out to be somewhat flawed.
What I actually want is, have the original query simply sorted, then grouped without touching the return values. As you can see above, my attempt was to 'restore' the additional return value again, but that didn't work. What should I do instead, if this approach is fundamentally wrong?
Explanation for the subquery use:
The point of the whole subquery is to force SQLAlchemy to execute this query separately as a first step.
I want to order the results first, and then group the ordered results. That seems to be hard to do properly in one step (when trying manually with SQL I had issues combining order and group by in one step as I wanted).
Therefore I don't simply order, group, but I order first, then subquery it to enforce that the order step is actually completed first, and then I group it.
Judging from manual PHPMyAdmin tests with the generated SQL, this seems to work fine. The actual problem is that the original query (which is now wrapped as the subquery you were confused about) had an added column, and now by wrapping it up as a subquery, that column is gone from the overall result. And my attempt to readd it to the outer wrapping failed.
It would be much better if you provided examples. I don't know if these columns are in separate tables or what not. Just looking at your first paragraph, I would do something like this:
a = session.query(Table1, Table2.column).\
join(Table2, Table1.foreign_key == Table2.id).\
filter(...).group_by(Table2.id).order_by(Table1.property.desc()).all()
I don't know exactly what you're trying to do since I need to look at your actual model, but it should look something like this with maybe the tables/objs flipped around or more filters.
I've got a file filled with records like this:
NCNSCF1124557200811UPPY19871230
The codes are all fixed-length, and some of them link to other flat files (sort of like a relational database). What's the best way of querying this data using LINQ?
This is what I came up with intuitively, but I was wondering if there's a more elegant way:
var records = File.ReadAllLines("data.txt");
var table = from record in records
select new { FirstCode = record.Substring(0, 2),
OtherCode = record.Substring(18, 4) };
For one thing I wouldn't read it all into memory to start with. It's very easy to write a LineReader class which iterates over a file a line at a time. I've got a version in MiscUtil which you can use.
Unless you only want to read the results once, however, you might want to call ToList() at the end to avoid reading the file multiple times. (This is still nicer than reading all the lines and keeping that in memory - you only want to do the splitting once.)
Once you've basically got in-memory collections of all the tables, you can use normal LINQ to Objects to join them together etc. You might want to go to a more sophisticated data model to get indexes though.
I don't think there's a better way out of the box.
One could define a Flat-File Linq Provider which could make the whole thing much simpler, but as far as I know, no one has yet.