My site generate a lot of queries to database, for each user generates 6 queries. I tried to fine the source of that but my knowlegde was not enough to find. If anyone could help me how to fine the source of that queries?
I used:
Joomla 2.5.8
Main Components: CB, Kunena, SH404SEF, K2, Komento, UddeIM PMS
Main Modules: Gavick News PRO4
Block spam IP
Block bots
The queries which are generate for each user:
SELECT *
FROM `_users`
WHERE `id` = 15
SELECT `g`.`id`,`g`.`title`
FROM `_usergroups` AS g
INNER JOIN `_user_usergroup_map` AS m ON m.group_id = g.id
WHERE `m`.`user_id` = 15
SELECT b.id
FROM _user_usergroup_map AS map
LEFT JOIN _usergroups AS a ON a.id = map.group_id
LEFT JOIN _usergroups AS b ON b.lft <= a.lft
AND b.rgt >= a.rgt
WHERE map.user_id = 15
SELECT a.rules
FROM _assets AS a
WHERE (a.id = 1)
GROUP BY a.id, a.rules, a.lft
SELECT id
FROM _assets
WHERE parent_id = 0
SELECT b.rules
FROM _assets AS a
LEFT JOIN _assets AS b ON b.lft <= a.lft
AND b.rgt >= a.rgt
WHERE (a.id = 1 OR a.parent_id = 0)
GROUP BY b.id, b.rules, b.lft
ORDER BY b.lft
well of course there are going to be a fair amount of queries per use, as you are using extensions such as CB and Kunena, that include queries for each user. Unless you get a message from your host saying too much memory is being used or there is too much traffic, you should be fine.
Joomla is a CMS and therefore these sort of things need to be expected when there are a fair amount of users.
Actually we just fixed a bug in the rules field that was generating an excessive number of queries. It will be fixed in 3.0.4 which is due out next week and whenever another 2.5 release comes out. In the meantime you can fix it yourself.
https://github.com/joomla/joomla-platform/pull/1792
But that's not what you are asking about. The number of queries isn't really the issue (it's totally reasonable) the question is how fast are they.
Related
I have the following LINQ query (using EF Core 6 and MS SQL Server):
var resultSet = dbContext.Systems
.Include(system => system.Project)
.Include(system => system.Template.Type)
.Select(system => new
{
System = system,
TemplateText = system.Template.TemplateTexts.FirstOrDefault(templateText => templateText.Language == locale.LanguageIdentifier),
TypeText = system.Template.Type.TypeTexts.FirstOrDefault(typeText => typeText.Language == locale.LanguageIdentifier)
})
.FirstOrDefault(x => x.System.Id == request.Id);
The requirement is to retrieve the system matching the requested ID and load its project, template and template's type info. The template has multiple TemplateTexts (one for each translated language) but I only want to load the one matching the requested locale, same deal with the TypeTexts elements of the template's type.
The LINQ query above does that in one query and it gets converted to the following SQL query (I edited the SELECT statements to use * instead of the long list of columns generated):
SELECT [t1].*, [t2].*, [t5].*
FROM (
SELECT TOP(1) [p].*, [t].*, [t0].*
FROM [ParkerSystems] AS [p]
LEFT JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
LEFT JOIN [Types] AS [t0] ON [t].[TypeId] = [t0].[Id]
LEFT JOIN [Projects] AS [p0] ON [p].[Project_ProjectId] = [p0].[ProjectId]
WHERE [p].[SystemId] = #__request_Id_1
) AS [t1]
LEFT JOIN (
SELECT [t3].*
FROM (
SELECT [t4].*, ROW_NUMBER() OVER(PARTITION BY [t4].[ReferenceId] ORDER BY [t4].[Id]) AS [row]
FROM [TemplateTexts] AS [t4]
WHERE [t4].[Language] = #__locale_LanguageIdentifier_0
) AS [t3]
WHERE [t3].[row] <= 1
) AS [t2] ON [t1].[Id] = [t2].[ReferenceId]
LEFT JOIN (
SELECT [t6].*
FROM (
SELECT [t7].*, ROW_NUMBER() OVER(PARTITION BY [t7].[ReferenceId] ORDER BY [t7].[Id]) AS [row]
FROM [TypeTexts] AS [t7]
WHERE [t7].[Language] = #__locale_LanguageIdentifier_0
) AS [t6]
WHERE [t6].[row] <= 1
) AS [t5] ON [t1].[Id0] = [t5].[ReferenceId]
which is not bad, it's not a super complicated query, but I feel like my requirement can be solved with a much simpler SQL query:
SELECT *
FROM [Systems] AS [p]
JOIN [Templates] AS [t] ON [p].[TemplateId] = [t].[Id]
JOIN [TemplateTexts] AS [tt] ON [p].[TemplateId] = [tt].[ReferenceId]
JOIN [Types] AS [ty] ON [t].[TypeId] = [ty].[Id]
JOIN [TemplateTexts] AS [tyt] ON [ty].[Id] = [tyt].[ReferenceId]
WHERE [p].[SystemId] = #systemId and tt.[Language] = 2 and tyt.[Language] = 2
My question is: is there a different/simpler LINQ expression (either in Method syntax or Query syntax) that produces the same result (get all info in one go) because ideally I'd like to not have to have an anonymous object where the filtered sub-collections are aggregated. For even more brownie points, it'd be great if the generated SQL would be simpler/closer to what I think would be a simple query.
Is there a different/simpler LINQ expression (...) that produces the same result
Yes (maybe) and no.
No, because you're querying dbContext.Systems, therefore EF will return all systems that match your filter, also when they don't have TemplateTexts etc. That's why it has to generate outer joins. EF is not aware of your apparent intention to skip systems without these nested data or of any guarantee that these systems don't occur in the database. (Which you seem to assume, seeing the second query).
That accounts for the left joins to subqueries.
These subqueries are generated because of FirstOrDefault. In SQL it always requires some sort of subquery to get "first" records of one-to-many relationships. This ROW_NUMBER() OVER construction is actually quite efficient. Your second query doesn't have any notion of "first" records. It'll probably return different data.
Yes (maybe) because you also Include data. I'm not sure why. Some people seem to think Include is necessary to make subsequent projections (.Select) work, but it isn't. If that's your reason to use Includes then you can remove them and thus remove the first couple of joins.
OTOH you also Include system.Project which is not in the projection, so you seem to have added the Includes deliberately. And in this case they have effect, because the entire entity system is in the projection, otherwise EF would ignore them.
If you need the Includes then again, EF has to generate outer joins for the reason mentioned above.
EF decides to handle the Includes and projections separately, while hand-crafted SQL, aided by prior knowledge of the data could do that more efficiently. There's no way to affect that behavior though.
This LINQ query is close to your SQL, but I'm afraid of correctness of the result:
var resultSet =
(from system in dbContext.Systems
from templateText in system.Template.TemplateTexts
where templateText.Language == locale.LanguageIdentifier
from typeText in system.Template.Type.TypeTexts
where typeText.Language == locale.LanguageIdentifier
select new
{
System = system,
TemplateText = templateText
TypeText = typeText
})
.FirstOrDefault(x => x.System.Id == request.Id);
I have to make a query in sccm to get all Clients with a specific Software installed. But i have no idea how to get this Job done.
I've tried this on but, it's not accurate enough.
select SMS_G_System_SYSTEM.Name from SMS_R_System inner join SMS_G_System_SYSTEM on SMS_G_System_SYSTEM.ResourceID = SMS_R_System.ResourceId inner join SMS_G_System_SoftwareFile on SMS_G_System_SoftwareFile.ResourceID = SMS_R_System.ResourceId where SMS_G_System_SoftwareFile.FileName = "program.exe" and SMS_G_System_SoftwareFile.FileVersion > "version" and SMS_R_System.Client = 1
I would like to check the programs in the control Panel. I have this tried with this query:
select SMS_R_SYSTEM.ResourceID,SMS_R_SYSTEM.ResourceType,SMS_R_SYSTEM.Name,SMS_R_SYSTEM.SMSUniqueIdentifier,SMS_R_SYSTEM.ResourceDomainORWorkgroup,SMS_R_SYSTEM.Client from SMS_R_System inner join SMS_G_System_COMPUTER_SYSTEM on SMS_G_System_COMPUTER_SYSTEM.ResourceID = SMS_R_System.ResourceId where SMS_G_System_COMPUTER_SYSTEM.Name not in (select distinct SMS_G_System_COMPUTER_SYSTEM.Name from SMS_R_System inner join SMS_G_System_COMPUTER_SYSTEM on SMS_G_System_COMPUTER_SYSTEM.ResourceID = SMS_R_System.ResourceId inner join SMS_G_System_ADD_REMOVE_PROGRAMS on SMS_G_System_ADD_REMOVE_PROGRAMS.ResourceID = SMS_R_System.ResourceId where SMS_G_System_ADD_REMOVE_PROGRAMS.DisplayName like "%program%")
But this one gives me no result.
Thanks for the help!
SCCM's built in reporting has a super easy way to see all systems with a particular application installed on them. Would that not work for you?
If you need a query for it, here's an example based on Google Chrome
select SCCM.Name0
,CAT.[ProductName0]
,CAT.[ProductVersion0]
,CAT.[Publisher0]
,CAT.[NormalizedName]</code>
FROM v_GS_INSTALLED_SOFTWARE_CATEGORIZED as CAT
join v_R_System as SCCM on CAT.ResourceID = SCCM.ResourceID
where CAT.[ProductName0] like '%Google Chrome%'
This will return the name of the system, and then a bunch of info about the application.
At my job our main application was written long ago before n-tier was really a thing, ergo - it has tons and tons of business logic begin handled in stored procs and such.
So we have finally decided to bite the bullet and make it not suck so bad. I have been tasked with converting a 900+ line sql script to a .NET exe, which I am doing in C#/Linq. Problem is...for the last 5-6 years at another job, I had been doing Linq exclusively, so my SQL has gotten somewhat rusty, and some of thing I am converting I have never tried to do before in Linq, so I'm hitting some roadblocks.
Anyway, enough whining.
I'm having trouble with the following sql statement, I think due to the fact that he is joining on a temp table and a derived table. Here's the SQL:
insert into #processedBatchesPurgeList
select d.pricebatchdetailid
from pricebatchheader h (nolock)
join pricebatchstatus pbs (nolock) on h.pricebatchstatusid = pbs.pricebatchstatusid
join pricebatchdetail d (nolock) on h.pricebatchheaderid = d.pricebatchheaderid
join
( -- Grab most recent REG.
select
item_key
,store_no
,pricebatchdetailid = max(pricebatchdetailid)
from pricebatchdetail _pbd (nolock)
join pricechgtype pct (nolock) on _pbd.pricechgtypeid = pct.pricechgtypeid
where
lower(rtrim(ltrim(pct.pricechgtypedesc))) = 'reg'
and expired = 0
group by item_key, store_no
) dreg
on d.item_key = dreg.item_key
and d.store_no = dreg.store_no
where
d.pricebatchdetailid < dreg.pricebatchdetailid -- Make sure PBD is not most recent REG.
and h.processeddate < #processedBatchesPurgeDateLimit
and lower(rtrim(ltrim(pbs.pricebatchstatusdesc))) = 'processed' -- Pushed/processed batches only.
So that's raising an overall question first: how to handle temp tables in Linq? This script uses about 10 of them. I currently have them as List. The problem is, if I try to .Join() on one in a query, I get the "Local sequence cannot be used in LINQ to SQL implementations of query operators except the Contains operator." error.
I was able to get the join to the derived table to work using 2 queries, just so a single one wouldn't get nightmarishly long:
var dreg = (from _pbd in db.PriceBatchDetails.Where(pbd => pbd.Expired == false && pbd.PriceChgType.PriceChgTypeDesc.ToLower().Trim() == "reg")
group _pbd by new { _pbd.Item_Key, _pbd.Store_No } into _pbds
select new
{
Item_Key = _pbds.Key.Item_Key,
Store_No = _pbds.Key.Store_No,
PriceBatchDetailID = _pbds.Max(pbdet => pbdet.PriceBatchDetailID)
});
var query = (from h in db.PriceBatchHeaders.Where(pbh => pbh.ProcessedDate < processedBatchesPurgeDateLimit)
join pbs in db.PriceBatchStatus on h.PriceBatchStatusID equals pbs.PriceBatchStatusID
join d in db.PriceBatchDetails on h.PriceBatchHeaderID equals d.PriceBatchHeaderID
join dr in dreg on new { d.Item_Key, d.Store_No } equals new { dr.Item_Key, dr.Store_No }
where d.PriceBatchDetailID < dr.PriceBatchDetailID
&& pbs.PriceBatchStatusDesc.ToLower().Trim() == "processed"
select d.PriceBatchDetailID);
So that query gives the expected results, which I am holding in a List, but then I need to join the results of that query to another one selected from the database, which is leading me back to the aforementioned "Local sequence cannot be used..." error.
That query is this:
insert into #pbhArchiveFullListSaved
select h.pricebatchheaderid
from pricebatchheader h (nolock)
join pricebatchdetail d (nolock)
on h.pricebatchheaderid = d.pricebatchheaderid
join #processedBatchesPurgeList dlist
on d.pricebatchdetailid = dlist.pricebatchdetailid -- PBH list is restricted to PBD purge list rows that have PBH references.
group by h.pricebatchheaderid
The join there on #processedBatchesPurgeList is the problem I am running into.
So uh...help? I have never written SQL like this, and certainly never tried to convert it to Linq.
As pointed out by the comments above, this is no longer being rewritten as Linq.
Was hoping to get a performance improvement along with achieving better SOX compliance, which was the whole reason for the rewrite in the first place.
I'm happy with just satisfying the SOX compliance issues.
Thanks, everyone.
I have customers with contract at various sites which must be charged.
There are two types of billing address for the sites: sites charged at the customer's address and sites charged to another address.
The kind of site is handled by the field 'status': if the site is charged at the customer's address, the field is equal to 'C'. If it is charged to a different site's address, it is equal to 'A' and the field 'bill_site_id' is filled with the 'site_id' used for billing. I want to retrieve, in one query, the address where I have to send the bill...
Let's say a customer have two sites (the A refers to the B for billing); the request obviously shows the two sites, but I only want the one used for billing (the B).
How to retrieve this one only?
Here is my query (anonymized, so please be gentle :-) )
SELECT CASE wp.status
WHEN 'A'
THEN wp2.name
ELSE c.NAME
END AS NAME,
CASE wp.status
WHEN 'A'
THEN wp2.adress
ELSE c.street
END AS street,
CASE wp.status
WHEN 'A'
THEN wp2.city
ELSE c.city
END AS city,
CASE wp.status
WHEN 'A'
THEN wp2.postcode
ELSE c.postcode
END AS postcode
FROM customer c, site wp, site wp2, customer_site cwp
WHERE c.idcompany = cwp.idcompany
AND cwp.site_id = wp.site_id
AND wp2.site_id(+) = wp.bill_site_id
AND c.idcompany = :someId
GROUP BY wp.status,
wp2.name,
wp2.adress,
wp2.city,
wp2.postcode
I am pretty sure I know what you want but am not 100% sure what 'ch.status' refers to in the query. Also i am not sure that from the information given you can achieve what you want ...
Hopefully the data structures are under your control so you can set it up how you want.
If I read your specification correctly then I think you want/need only 1 instance of the site table in the query, but it is then not clear how you identify the customer's address.
One way round this is for the site table to have an main_address flag. (I think this is the confusion around ch.status)
If there is a flag on the site table then ...
WHERE c.idcompany = cwp.idcompany
AND cwp.site_id = wp.site_id
AND (
(C.STATUS = 'C' AND WP.MAIN_ADDRESS = 1) OR
(C.STATUS = 'A' AND WP.SITE_ID = C.BILL_SITE_ID)
)
Another way would be to have the Main Adress and Billing addresses as foreign keys on the customer table, and to forget about the status flag so :
WHERE WP.SITE_ID = DECODE(C.BILLING_ADDRESS, NULL, C.MAIN_ADDRESS, C.BILLING_ADDRESS)
If this makes no sense then perhaps you can post a DESCRIBE of the tables involved.
Technically a Union is two queries, but you can run it in a single command and it should do what you want so I'm throwing it out as an option. :)
SELECT c.Name AS NAME, c.street AS STREET, c.city AS CITY, c.postcode AS postcode
FROM customer c
INNER JOIN customer_site cwp ON c.idcompany = cwp.idcompany
INNER JOIN site wp ON cwp.site_id = wp.site_id
WHERE c.idcompany = :someid AND wp.status = 'C'
UNION
SELECT wp2.name AS NAME, wp2.adress AS STREET, wp2.city AS CITY, wp2.postcode AS postcode
FROM customer c
INNER JOIN customer_site cwp ON c.idcompany = cwp.idcompany
INNER JOIN site wp ON cwp.site_id = wp.site_id
INNER JOIN site wp2 ON wp.bill_site_id = wp2.site_id
WHERE c.idcompany = :someid AND wp.status = 'A'
I can't see your table structure so it's probably possible to optimize more, but I think that'll help point you in a working direction. This may very well not be any faster then your existing query, but it's a fair bit easier to understand.
This is my query:
Dim vendorId = 1, categoryId = 1
Dim styles = From style In My.Context.Styles.Include("Vendor") _
Where style.Vendor.VendorId = vendorId _
AndAlso (From si In style.StyleItems _
Where si.Item.Group.Category.CategoryId = _
categoryId).Count > 0 _
Distinct
I have the feeling that I can improve the performance, cuz the above query is (correct me if I am wrong) performs 2 round-trips to the server; 1 time by the Count and then when it's executed.
I want to send this Count thing to the DB so it should be only one round trip to the server.
Even it's not the exact thing, this is actually what I need:
SELECT DISTINCT Style.*
FROM Style INNER JOIN
Vendor ON Style.VendorId = Vendor.VendorId INNER JOIN
StyleItem ON Style.StyleId = StyleItem.StyleId INNER JOIN
Item ON StyleItem.ItemId = Item.ItemId INNER JOIN
[Group] ON Item.GroupId = [Group].GroupId INNER JOIN
Category ON [Group].CategoryId = Category.CategoryId
WHERE (Style.VendorId = #vendorid) AND (Category.CategoryId = #CategoryId)
I wish I could use this SPROC (i.e. function import etc.), but I need to Include("Vendor"), which constraints me to do it with Linq.
Any kind of suggestion will be really welcommed!
It is probably not doing two trips to the database. It will get optimized before it is executed, and nothing gets executed until you try the read the data.
Normally I check the SQL that is created using SQL Profiler. I have also found LinqPad to be very usefull.