I am aware I should not use the database directly from SonarQube, but this is a one-shot complex thing, and saves me days if I can do it directly from the database.
I need to know the amount of classes per project, where the amount of lines is less than 200. So far no problem creating this in SQL.
Only problem I have: for 2 projects, this information isn't stored at all in the database! In the GUI from SonarQube I can see this measure for each file, but than in the database these files have only 1 measures stored (technical debt).
My guess is that for some strange reasons these measures are calculated on the fly for this project? Could that be? And is there a way to force SonarQube to store proejct measures for each file in the database? I tried with the sonar.analysis.mode=analysis parameter but that didn't work?
Thanks a lot and regards,
Pieter
it was due to the query. this is the right query
SELECT metrics.description, root.name, COUNT(project_measures.metric_id) AS AmountOfFiles,
SUM(CASE WHEN project_measures.value < 200 THEN 1 ELSE 0 END) Less200,
SUM(CASE WHEN ((project_measures.value >= 200) AND (project_measures.value <= 1000)) THEN 1 ELSE 0 END) Between200And1000,
SUM(CASE WHEN project_measures.value > 1000 THEN 1 ELSE 0 END) More1000
FROM project_measures
INNER JOIN metrics on project_measures.metric_id = metrics.id
INNER JOIN snapshots on project_measures.snapshot_id = snapshots.id
INNER JOIN projects root on snapshots.root_project_id = root.id
WHERE metrics.id = '1' /*Line: code lines + comments + blanc lines*/
and root.scope = 'PRJ' /*projects*/
and snapshots.scope = 'FIL'
and root.name like '%' /* '%' to show all projects*/
GROUP BY root.name
I am a little bit confused by your question. Are you saying that, when using your SQL queries, you have the results correct for all but 2 projects, whose metrics appear in the SonarQube dashboard?
I suppose the problem cannot come from your SQL queries?
Have you been running these 2 projects recently?
Try clearing your browser cache?
Regards.
Related
I am trying to run a simple query against any of the tables contained in SNOWFLAKE.ACCOUNT_USAGE schema but for some reason it is taking up a long time to run even if I try to limit it to show only the first row, like the following example:
SELECT * FROM "SNOWFLAKE"."ACCOUNT_USAGE"."ACCESS_HISTORY" limit 1;
Is that a normal behavior? If not, can someone help me to figure out why this is happening?
Its always a good practice to add a WHERE condition so the optimizer can make use of query pruning.
If you know your objects were accessed within the past say 24 hours, can you add a date filter and see if that helps?
SELECT * FROM "SNOWFLAKE"."ACCOUNT_USAGE"."ACCESS_HISTORY"
WHERE QUERY_START_TIME > CURRENT_DATE() - 1
limit 1;
More info on mirco-partitions and query pruning: https://docs.snowflake.com/en/user-guide/tables-clustering-micropartitions.html#query-pruning
We are facing performance issue in production. Mv refersh program is running for long, almost 13 to 14 hours.
In the MV refersh program is trying to refersh 5 MV. Among that one of the MV is running for long.
Below is the MV script which is running for long.
SELECT rcvt.transaction_id,
rsh.shipment_num,
rsh.shipped_date,
rsh.expected_receipt_date,
(select rcvt1.transaction_date from rcv_transactions rcvt1
where rcvt1.po_line_id = rcvt.po_line_id
AND rcvt1.transaction_type = 'RETURN TO VENDOR'
and rcvt1.parent_transaction_id=rcvt.transaction_id
)transaction_date
FROM rcv_transactions rcvt,
rcv_shipment_headers rsh,
rcv_shipment_lines rsl
WHERE 1 =1
AND rcvt.shipment_header_id =rsl.shipment_header_id
AND rcvt.shipment_line_id =rsl.shipment_line_id
AND rsl.shipment_header_id =rsh.shipment_header_id
AND rcvt.transaction_type = 'RECEIVE';
Shipment table contains millions of records and above query is trying to extract almost 60 to 70% of the data. We are suspecting data load is the reason.
We are trying to improve the performance for the above script.So we added date filter to restrict the data.
SELECT rcvt.transaction_id,
rsh.shipment_num,
rsh.shipped_date,
rsh.expected_receipt_date,
(select rcvt1.transaction_date from rcv_transactions rcvt1
where rcvt1.po_line_id = rcvt.po_line_id
AND rcvt1.transaction_type = 'RETURN TO VENDOR'
and rcvt1.parent_transaction_id=rcvt.transaction_id
)transaction_date
FROM rcv_transactions rcvt,
rcv_shipment_headers rsh,
rcv_shipment_lines rsl
WHERE 1 =1
AND rcvt.shipment_header_id =rsl.shipment_header_id
AND rcvt.shipment_line_id =rsl.shipment_line_id
AND rsl.shipment_header_id =rsh.shipment_header_id
AND rcvt.transaction_type = 'RECEIVE'
AND TRUNC(rsh.creation_date) >= NVL(TRUNC((sysdate - profile_value),'MM'),TRUNC(rsh.creation_date) );
For 1 year profile, it shows some improvement but if we give for 2 years range its more worse than previous query.
Any suggestions to improve the performance.
Pls help
I'd pull out that scalar subquery into a regular outer join.
Costing for scalar subqueries can be poor and you are forcing it to do a lot of single record lookups (presumably via index) rather than giving it other options.
"The main query then has a scalar subquery in the select list.
Oracle therefore shows two independent plans in the plan table. One for the driving query – which has a cost of two, and one for the scalar subquery, which has a cost of 2083 each time it executes.
But Oracle does not “know” how many times the scalar subquery will run (even though in many cases it could predict a worst-case scenario), and does not make any cost allowance whatsoever for its execution in the total cost of the query."
This query returns results that aren't grouped in the way the query expects it to. This started only happening recently and has been occurring at least twice a week.
SELECT
UserN, Program, CPair, PosCcy1, OpenPosOpt, CrossR,
OptPrem, DailyOptValue, OptPandL
FROM (
(SELECT
UserN, Program, CPair,PosCcy1,OpenPosOpt,RevalC1,
RevalC2, CrossR, OptPrem,DailyOptValue,OptValue, OptPandL
FROM
(SELECT
UserN, Program, CPair,
ROUND(SUM(Ccy1Amount),4) AS PosCcy1,
ROUND(SUM(OpenOptPos),4) AS OpenPosOpt,
MAX(RevalC1) as RevalC1,
MAX(RevalC2) as RevalC2,
case when MAX(RevalC2) = 0 and MAX(RevalC1) = 0 then 0
else MAX(RevalC2)/MAX(RevalC1) end as CrossR,
ROUND(SUM(OptPrem),2) AS OptPrem,
ROUND(SUM(DailyOp),4) AS DailyOptValue,
ROUND(SUM(OptValue),4) AS OptValue,
ROUND(SUM(OpPandL),4) AS OptPandL,
sum(Prem) as Prem
FROM
AReport rr
GROUP BY
UserN, Program, CPair) a
WHERE
not(PosCcy1= 0 and OpenPosOpt= 0 and Prem = 0)
)) results
The 'AReport' is a view.
As could be seen I expect the query to return the data grouped by User, Program and CPair. Almost always I get that result, but all of a sudden at times the results aren't grouped like that.
When this happens SQL Server returns the un-grouped results the whole day. I've run parts of the query on SSMS during then and the results I get are grouped as expected for the two inner nested selects. The moment I highlight the outer most select to be included the results set is un-grouped.
I've looked for errors in the error log and the event viewer which doesn't seem to suggest anything. This behaviour is automatically rectified by the following day. Could someone tell me if there's anything in the way the query is written that's causing this behaviour.
Read this
http://msdn.microsoft.com/en-gb/library/ms189822.aspx
and review your query
Finally found that the inconsistency was the data displayed was almost always ordered in the desired way . See Why does SQL Server 2008 order when using a GROUP BY and no order has been specified?
Adding an order by to the query fixed the problem.
Running this line of code
Activity.uniq.select(:ip).where("created_at > NOW() - INTERVAL 1 DAY").count
Returns the number of all lines (without the uniquness filter) although the SQL is correct (with the DISTINCT keyword)
If I run this command in 2 steps:
c = Activity.uniq.select(:ip).where("created_at > NOW() - INTERVAL 1 DAY")
c.size
Then I get the right count.
How can it be?
Thank you
Generally DISTINCT will remove duplicate results from a result set but it won't do any grouping. Since you have one row in a COUNT result, you won't see any effect. What you really want is this:
SELECT COUNT(DISTINCT activities.ip) FROM activities WHERE ...
Note that you want a count of the distinct activities, not a distinct count.
I'm not sure how you'd manage to emit this SQL using ActiveRecord. You may need to resort to using the ActiveRecord connection directly and use select_value for such an unusual sort of inquiry unless you can coach select to do it for you.
The reports that I use (and update) are taking a long time (some take hours). I felt this is far too long and asked previously about this. After taking a long look at various web sites that discuss SQL performance, they all take the stance of being DBA's. However I'm not, and neither are my colleagues (I guess if we had a DBA then we wouldn't have this problem).
What I want is a simple way of returning the top 10 or so most run and worst performing scripts. I would of hoped there is a nice SET METRICS ON switch, but I guess if that was the case then the sites wouldn't go on about recording profiles.
The last thing I want to do is to cause performance to drop even further and recording a profile sounds like a performance killer.
You have at least following options.
look at the plan of a bad performing query in SQL Analyzer and try to optimize it, query by query from there.
or use a script (see below) to give you advice by analyzing SQLServer's statistics on what indexes you could create.
or use the Database Engine Tuning Advisor to suggest and/or create indexes for you to speed up your queries
or use a tool like redgate's SQL Response to give you more information than you can digest
In the end, automated tools will get you a long way. It may even be enough in your case but keep in mind that there is no automated tool that will be able to outperform a skilled DBA for the mear fact that automated tools can not rewrite your queries.
SET CONCAT_NULL_YIELDS_NULL OFF
--Joining the views gives a nice picture of what indexes
--would help and how much they would help
SELECT
'CREATE INDEX IX_' + UPPER(REPLACE(REPLACE(COALESCE(equality_columns, inequality_columns), '[', ''), ']', ''))
+ ' ON ' + d.statement + '(' + COALESCE(equality_columns, inequality_columns)
+ CASE WHEN equality_columns IS NOT NULL THEN
CASE WHEN inequality_columns IS NOT NULL THEN ', ' + inequality_columns
END END
+ ')' + CASE WHEN included_columns IS NOT NULL THEN ' INCLUDE (' + included_columns + ')' END
, object_name(object_id)
, d.*
, s.*
FROM sys.dm_db_missing_index_details d
LEFT OUTER JOIN sys.dm_db_missing_index_groups g ON d.index_handle = g.index_handle
LEFT OUTER JOIN sys.dm_db_missing_index_group_stats s ON g.index_group_handle = s.group_handle
WHERE database_id = db_id()
ORDER BY avg_total_user_cost DESC
You should be able to go thru the sys.dm_exec_query_stats table, which keeps information on all queries against a database.
SELECT creation_time
,last_execution_time
,total_physical_reads
,total_logical_reads
,total_logical_writes
, execution_count
, total_worker_time
, total_elapsed_time
, total_elapsed_time / execution_count avg_elapsed_time
,SUBSTRING(st.text, (qs.statement_start_offset/2) + 1,
((CASE statement_end_offset
WHEN -1 THEN DATALENGTH(st.text)
ELSE qs.statement_end_offset END
- qs.statement_start_offset)/2) + 1) AS statement_text
FROM sys.dm_exec_query_stats AS qs
CROSS APPLY sys.dm_exec_sql_text(qs.sql_handle) st
ORDER BY last_execution_time,total_elapsed_time / execution_count DESC;
Gives you basic timing information of how long, historically, queries took.