I'm trying to join query between my Cars regular Table and my Makers Virtual Table for FTS
This is the makers Virtual Table
let markersTable = VirtualTable("makers")
let idColumn = Expression<String>("id")
let name = Expression<String>("name")
let text = Expression<String>("text")
let code = Expression<String>("code")
let config = FTS5Config()
.column(idColumn)
.column(name, [.unindexed])
.column(code)
.column(text)
and this is the cars regular table
let carsTable = Table("cars")
let idColumn = Expression<String>("id")
let name = Expression<String>("name")
let description = Expression<String>("description")
let makersCode = Expression<String>("makersCode")
The tables can be joined by
let query = carsTable
.limit(1)
.join(markersTable, on: makersCode == markersTable[code])
This works but takes a long time and I'm wondering what is the best way to make this join be faster?
I did some tests before using the FTS Virtual Table and if both tables are regular tables, the join is a lot faster.
Any help is appreciated.
Related
Hi I am using a Jqwidgets Grid to display my data. It has a build in possibility to use filters but if you filter your records on the server side you have to build your own query. As I am working with Linq I thought to use the Dynamic Linq Library for Asp net core. Problem is there are not many examples or explanations how to do this. But I am busy for days now and not getting very far.The way I am setup; I have a normal Linq query:
var Mut = from M in _DB.Mutations
join S in _DB.Shifts on M.ShiftId equals S.ShiftId
join U in _DB.RoosterUsers on M.UserId equals U.RoosterUserId
join D in deps on M.UserId equals D.UserId
join DD in _DB.Departements on D.DepartementID equals DD.DepartementId
select new MutationModel
{
MutId=M.MutationId,
Naam=U.FirstName + " " + U.LastName,
UserId=M.UserId,
Departement= DD.DepartementName,
MutationType = S.publicName,
MutationGroup = S.ShiftType.ToString(),
DateTot =M.DateTill,
TijdVan=M.DateStartOn,
TijdTot=M.DateTill,
Status=CreateStatus(M.Tentative, M.ApprovedOn, M.Processed, M.CancelRefId, M.Deleted)
};
This query is running OK and gives me all the data I need for the Grid.
Then for the filter I would like to add a dynamic Linq Query using the System.Linq.Dynamic.Core library
But this is as far as I get things working until now:
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").Select("Status");
My questions now :
1. In the where clause If I make the fieldname variable I get an error. how to do this??
2. In the Select Clause, how to add multiple Columns? (actually I just like to output all columns.)
Best would be to see an example. has somebody used Dynamic Linq to build a dynamic linq query for the JQWidgets Grid?
Thank you very much.
In what way you are trying to use fieldname variable in where clause ?
If you want to output all columns you can use ToList()
like
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").ToList();
If you want to get some specific columns you can use Select clause like this
var outQuery = Mut.Where("Status = #0 and UserId = #1", "Nieuw", "KLM22940").Select("new(Status,UserId )");
This Select clause creates data class which contains Status and UserId properties and returns a sequence of instances of that data class.
I'm creating new data model in oracle fusion cloud. I have a different tables, and I want to link all 3 of them. What should I do?
Below are the tables i used:
doo_headers_all dha,
ra_terms rt,
ra_customer_trx_all rcta,
hz_cust_site_uses_all bill_hcsua,
hz_cust_acct_sites_all bill_hcasa,
hz_cust_accounts bill_hca,
hz_party_sites bill_hps,
hz_parties bill_hp,
hz_party_site_uses ship_hpsu,
hz_party_sites ship_hps,
hz_parties ship_hp,
hz_addtnl_party_names bill_hapn,
hz_addtnl_party_names ship_hapn
I link all of them except for DOO_HEADERS_ALL, I tried to search which foreign keys can I use so that I can link it all but I don't see any answer.
rcta.term_id = rt.term_id(+)
and rcta.bill_to_site_use_id = bill_hcsua.site_use_id(+)
and bill_hcsua.cust_acct_site_id = bill_hcasa.cust_acct_site_id(+)
and bill_hcasa.cust_account_id = bill_hca.cust_account_id(+)
and bill_hcasa.party_site_id = bill_hps.party_site_id(+)
and bill_hps.party_id = bill_hp.party_id(+)
and rcta.ship_to_party_site_use_id = ship_hpsu.party_site_use_id(+)
and ship_hpsu.party_site_id = ship_hps.party_site_id(+)
and ship_hps.party_id = ship_hp.party_id(+)
and bill_hp.party_id = bill_hapn.party_id(+) and bill_hapn.party_name_type (+) = 'PHONETIC'
and ship_hp.party_id = ship_hapn.party_id(+) and ship_hapn.party_name_type (+) = 'PHONETIC'
this is I used to link other tables except doo_headers_all
dha.ORDER_NUMBER = rcta.CT_REFERENCE(+)
or
dha.SOLD_TO_CUSTOMER_ID = bill_hca.CUST_ACCOUNT_ID (+)
This is highly dependent on your setup though.
It's also not really clear what you mean by 'different tables, and I want to link all 3 of them', as your list is a lot longer than 3 tables.
I'm looking to factor out some common queries over several tables. In a very simple example all tables have a DataDate column, so I have queries like this:
let dtexp1 = query { for x in table1 do maxBy x.Datadate }
let dtexp2 = query { for x in table2 do maxBy x.Datadate }
Based on a previous question I can do the following:
let mkQuery t q =
query { for rows in t do maxBy ((%q) rows) }
let getMaxDt1 = mkQuery table1 (<# fun q -> q.Datadate #>)
let getMaxDt2 = mkQuery table2 (<# fun q -> q.Datadate #>)
I would be interested if there are any other solutions not using quotations. The reason being is that for more complicated queries the quotations and the splicing become difficult to read.
This for example won't work, obviously, as we don't know that x has property DataDate.
let getMaxDt t = query { for x in t do maxBy x.Datadate }
Unless I can abstract over the type of table1, table2, etc. which are generated by SqlProvider.
The answer very much depends on what kind of queries you need to construct and how static or dynamic they are. Generally speaking:
LINQ is great if they are mostly static and if you can easily list all the templates for all queries you'll need - the main nice thing is that it statically type checks the queries
LINQ is not so great when your query structure is very dynamic, because then you end up composing lots of quotations and the type checking sometimes gets into the way.
If your queries are very dynamic (including selecting the source dynamically), but are not too complex (e.g. no fancy groupings no fancy joins), then it might be easier to write code to generate SQL query from an F# domain model.
For your simple example, the query is really just a table name and aggregation:
type Column = string
type Table = string
type QueryAggregate =
| MaxBy of Column
type Query =
{ Table : Table
Aggregate : QueryAggregate }
You can then create your two queries using:
let q1 = { Table = "table1"; Aggregate = MaxBy "Datadate" }
let q2 = { Table = "table2"; Aggregate = MaxBy "Datadate" }
Translating those queries to SQL is quite simple:
let translateAgg = function
| MaxBy col -> sprintf "MAX(%s)" col
let translateQuery q =
sprintf "SELECT %s FROM %s" (translateAgg q.Aggregate) q.Table
Depending on how rich your queries can be, the translation can get very complicated, but if the structure is fairly simple then this might just be an easier alternative than constructing the query using LINQ. As I said, it's hard to say what will be better without knowing the exact use case!
I have a query generated by entity framework running against oracle that's too slow. It runs in about 4 seconds.
This is the main portion of my query
var query = from x in db.BUILDINGs
join pro_co in db.PROFILE_COMMUNITY on x.COMMUNITY_ID equals pro_co.COMMUNITY_ID
join co in db.COMMUNITies on x.COMMUNITY_ID equals co.COMMUNITY_ID
join st in db.STATE_PROFILE on co.STATE_CD equals st.STATE_CD
where pro_co.PROFILE_NM == authorizedUser.ProfileName
select new
{
COMMUNITY_ID = x.COMMUNITY_ID,
COUNTY_ID = x.COUNTY_ID,
REALTOR_GROUP_NM = x.REALTOR_GROUP_NM,
BUILDING_NAME_TX = x.BUILDING_NAME_TX,
ACTIVE_FL = x.ACTIVE_FL,
CONSTR_SQFT_AVAIL_NB = x.CONSTR_SQFT_AVAIL_NB,
TRANS_RAIL_FL = x.TRANS_RAIL_FL,
LAST_UPDATED_DT = x.LAST_UPDATED_DT,
CREATED_DATE = x.CREATED_DATE,
BUILDING_ADDRESS_TX = x.BUILDING_ADDRESS_TX,
BUILDING_ID = x.BUILDING_ID,
COMMUNITY_NM = co.COMMUNITY_NM,
IMAGECOUNT = x.BUILDING_IMAGE2.Count(),
StateCode = st.STATE_NM,
BuildingTypeItems = x.BUILDING_TYPE_ITEM,
BuildingZoningItems = x.BUILDING_ZONING_ITEM,
BuildingSpecFeatures = x.BUILDING_SPEC_FEATURE_ITEM,
buildingHide = x.BUILDING_HIDE,
buildinghideSort = x.BUILDING_HIDE.Count(y => y.PROFILE_NM == ProfileName) > 0 ? 1 : 0,
BUILDING_CITY_TX = x.BUILDING_CITY_TX,
BUILDING_ZIP_TX = x.BUILDING_ZIP_TX,
LPF_GENERAL_DS = x.LPF_GENERAL_DS,
CONSTR_SQFT_TOTAL_NB = x.CONSTR_SQFT_TOTAL_NB,
CONSTR_STORIES_NB = x.CONSTR_STORIES_NB,
CONSTR_CEILING_CENTER_NB = x.CONSTR_CEILING_CENTER_NB,
CONSTR_CEILING_EAVES_NB = x.CONSTR_CEILING_EAVES_NB,
DESCR_EXPANDABLE_FL = x.DESCR_EXPANDABLE_FL,
CONSTR_MATERIAL_TYPE_TX = x.CONSTR_MATERIAL_TYPE_TX,
SITE_ACRES_SALE_NB = x.SITE_ACRES_SALE_NB,
DESCR_PREVIOUS_USE_TX = x.DESCR_PREVIOUS_USE_TX,
CONSTR_YEAR_BUILT_TX = x.CONSTR_YEAR_BUILT_TX,
DESCR_SUBDIVIDE_FL = x.DESCR_SUBDIVIDE_FL,
LOCATION_CITY_LIMITS_FL = x.LOCATION_CITY_LIMITS_FL,
TRANS_INTERSTATE_NEAREST_TX = x.TRANS_INTERSTATE_NEAREST_TX,
TRANS_INTERSTATE_MILES_NB = x.TRANS_INTERSTATE_MILES_NB,
TRANS_HIGHWAY_NAME_TX = x.TRANS_HIGHWAY_NAME_TX,
TRANS_HIGHWAY_MILES_NB = x.TRANS_HIGHWAY_MILES_NB,
TRANS_AIRPORT_COM_NAME_TX = x.TRANS_AIRPORT_COM_NAME_TX,
TRANS_AIRPORT_COM_MILES_NB = x.TRANS_AIRPORT_COM_MILES_NB,
UTIL_ELEC_SUPPLIER_TX = x.UTIL_ELEC_SUPPLIER_TX,
UTIL_GAS_SUPPLIER_TX = x.UTIL_GAS_SUPPLIER_TX,
UTIL_WATER_SUPPLIER_TX = x.UTIL_WATER_SUPPLIER_TX,
UTIL_SEWER_SUPPLIER_TX = x.UTIL_SEWER_SUPPLIER_TX,
UTIL_PHONE_SVC_PVD_TX = x.UTIL_PHONE_SVC_PVD_TX,
CONTACT_ORGANIZATION_TX = x.CONTACT_ORGANIZATION_TX,
CONTACT_PHONE_TX = x.CONTACT_PHONE_TX,
CONTACT_EMAIL_TX = x.CONTACT_EMAIL_TX,
TERMS_SALE_PRICE_TX = x.TERMS_SALE_PRICE_TX,
TERMS_LEASE_SQFT_NB = x.TERMS_LEASE_SQFT_NB
};
There is a section of code that tacks on dynamic where and sort clauses to the query but I've left those out. The query takes about 4 seconds to run no matter what is in the where and sort.
I dropped the generated SQL in Oracle and an explain plan didn't appear to show anything that screamed fix me. Cost is 1554
If this isn't allowed I apologize but I can't seem to find a good way to share this information. I've uploaded the explain plan generated by Sql Developer here: http://www.123server.org/files/explainPlanzip-e1d291efcd.html
Table Layout
Building
--------------------
- BuildingID
- CommunityId
- Lots of other columns
Profile_Community
-----------------------
- CommunityId
- ProfileNM
- lots of other columns
state_profile
---------------------
- StateCD
- ProfileNm
- lots of other columns
Profile
---------------------
- Profile-NM
- a few other columns
All of the tables with allot of columns have 120-150 columns each. It seems like entity is generating a select statement that pulls every column from every table instead of just the ones I want.
The thing that's bugging me and I think might be my issue is that in my LINQ I've selected 50 items, but the generated sql is returning 677 columns. I think returning so many columns is the source of my slowness possibly.
Any ideas why I am getting so many columns returned in SQL or how to speed my query?
I have a suspicion some of the performance is being impacted by your object creation. Try running the query without just a basic "select x" and see if it's the SQL query taking time or the object creation.
Also if the query being generated is too complicated you could try separating it out into smaller sub-queries which gradually enrich your object rather than trying to query everything at once.
I ended up creating a view and having the view only select the columns I wanted and joining on things that needed to be left-joined in linq.
It's pretty annoying that EF selects every column from every table you're trying to join across. But I guess I only noticed this because I am joining a bunch of tables with 150+ columns in them.
This is my query:
Dim vendorId = 1, categoryId = 1
Dim styles = From style In My.Context.Styles.Include("Vendor") _
Where style.Vendor.VendorId = vendorId _
AndAlso (From si In style.StyleItems _
Where si.Item.Group.Category.CategoryId = _
categoryId).Count > 0 _
Distinct
I have the feeling that I can improve the performance, cuz the above query is (correct me if I am wrong) performs 2 round-trips to the server; 1 time by the Count and then when it's executed.
I want to send this Count thing to the DB so it should be only one round trip to the server.
Even it's not the exact thing, this is actually what I need:
SELECT DISTINCT Style.*
FROM Style INNER JOIN
Vendor ON Style.VendorId = Vendor.VendorId INNER JOIN
StyleItem ON Style.StyleId = StyleItem.StyleId INNER JOIN
Item ON StyleItem.ItemId = Item.ItemId INNER JOIN
[Group] ON Item.GroupId = [Group].GroupId INNER JOIN
Category ON [Group].CategoryId = Category.CategoryId
WHERE (Style.VendorId = #vendorid) AND (Category.CategoryId = #CategoryId)
I wish I could use this SPROC (i.e. function import etc.), but I need to Include("Vendor"), which constraints me to do it with Linq.
Any kind of suggestion will be really welcommed!
It is probably not doing two trips to the database. It will get optimized before it is executed, and nothing gets executed until you try the read the data.
Normally I check the SQL that is created using SQL Profiler. I have also found LinqPad to be very usefull.