I'm getting "Read timed out" when running a query on a 1,3b row db.
It is not a particular advanced query that groups together hashtags in tweets:
SELECT case when match(hashtag,
'[Cc]orona.*|COVID.*|[Cc]ovid.*|[Cc]oVID_19.*|[Cc]orvid19.*|COVD19.*|CORONA.*|KILLTHEVI.*|SARSCoV.*|ChineseVi.*|WuhanVir.*|ChinaVir.*|[Vv]irus.*|
[Qq]uarantine|[Pp]andemic.*|[Cc]linical[Tt]rial.*|FlattenTheCurve.*|SocialDistancing.*|StayHome.*|StayTheFHome.*|StayAtHome.*|stopthespread.*|
SafeHands.*|WashYourHands.*|SelfIsolation.*') then 'COVID19'
when match(hashtag, '[Jj]anta[Cc]urfew.*|[Jj]anata[Cc]urfew.*') then 'JantaCurfew'
when match(hashtag, 'Bhula.*') then 'Bhula'
when match(hashtag, '[Ss]t[Pp]atrick.*|HappyStPatrick') then 'StPatricks day'
when match(hashtag, '[Cc]hina.*') then 'China'
when match(hashtag, '[Ii]taly.*') then 'Italy'
when match(hashtag, '[Ii]ran.*') then 'Iran'
when match(hashtag, '[Ii]ndia.*') then 'India'
when match(hashtag, '[Hh]appy[Mm]others[Dd]ay.*|[Mm]others[Dd]ay.*') then 'MothersDay'
else hashtag END
as Hashtag,
SUM(CASE WHEN created >= '2020-05-14 00:00:00' AND created <= '2020-03-14 23:59:59' THEN 1 END) "May 14th'20",
SUM(CASE WHEN created >= '2020-05-13 00:00:00' AND created <= '2020-03-13 23:59:59' THEN 1 END) "May 13th'20",
SUM(CASE WHEN created >= '2020-05-12 00:00:00' AND created <= '2020-03-12 23:59:59' THEN 1 END) "May 12th'20"
FROM twitterDBhashtags
group by Hashtag
order by "May 12th'20" DESC limit 20;
Clickhouse is running on a striped hdd and accessed through GB network.
How can the timeout, if that is the challenge, be changed to allow for more time?
I would very much want to be able to run multi minutes queries without getting the "Read timed out" message, if possible.
CH jdbc driver has a socket_timeout = 30000 (30s) by default
Under the Advanced tab, you can configure advanced connections settings, > e.g., Character Coding.
Connection / Advanced properties / New property -> socket_timeout = 300000
Related
I need a query to print some 4-5 errors in the if condition with id. need to check every error with its occurrence when it occurs 5 times then need to check id. if the same id for the error then need to print the error and associated id.
select id from table1 where
data like '% System is down%'
and date>= '05-SEP-15 07.55.00.000000000 PM' and date<= '05-SEP-15 07.59.00.000000000 PM'
having count(id)>=2 group by id;
I need to write this in if statement. If "system down" occurs 5 time with same id "1" then it needs to print as system down with id:1.
else communication error occurs with id 1 the it needs to print as "Communication error with id : 1" like this.
Confused with your requirements but you can adjust the values to get your required results
SELECT DISTINCT id,
CASE
WHEN count(*) over(PARTITION BY id) >=5 THEN 'System Down'
WHEN count(*) over(PARTITION BY id) < 5 THEN 'Communication Error'
END AS status
FROM table1
WHERE DATA LIKE '% System is down%'
AND date>= '05-SEP-15 07.55.00.000000000 PM'
AND date<= '05-SEP-15 07.59.00.000000000 PM'
or
SELECT id,
CASE
WHEN cnt>=5 THEN 'System Down'
ELSE
WHEN cnt < 5 THEN 'Communication error'
END status
FROM
(SELECT id,
count(id) cnt
FROM table1
WHERE DATA LIKE '% System is down%'
AND date>= '05-SEP-15 07.55.00.000000000 PM'
AND date<= '05-SEP-15 07.59.00.000000000 PM'
HAVING count(id)>=2
GROUP BY id)
P.S:-Not sure about 1 times but if you want exactly one occurrence need to comment out having clause(in second method) and set second in case clause to =1
Edit :- you can remove second when clause just use case when >=5 then 'system down' else 'communication error'
How I can achieve below,
I have to check last 30 days from the latest statement date w.r. to account using following formula
if ((sum of credits for 30 days from the latest statement date w.r. to account/
sum debits for 30 days from the latest statement date w.r. to account)
-sum debits for 30 days from the latest statement date w.r. to account)>0 then YES else NO
below are the two columns from two different tables based on which i need to derived formula for each account. And accounts present in both tables A and B.
STATEMENT_DATE_LATEST; --from TableA
LATEST_BAL_IN_USD; -- from TableB
Note: "/" is Divided by and "-" is Minus sign in above statement
If I understood it correctly, would something like this help?
with temp as
(select a.account,
sum(case when b.transaction_amount >= 0 then b.transaction_amount end) sum_credits,
sum(case when b.transaction_amount < 0 then b.transaction_amount end) sum_debits
from table_a a join table_b b on a.account = b.account
where b.statement_date > a.statement_date_latest + 30
group by s.account
)
select t.account,
case when (t.sum_credits / t.sum_debits) - t.sum_debits > 0 then 'YES'
else 'NO'
end result
from temp t;
select RP.COUNTRYID,RP.PRDCODE,
RP.REPID,
RP.CHANNELID,
RP.CUSTOMERID,
RP.DIVISION,
RP.WWCOGS_GAUSS,
RP.WWCOGS_SAP,
RP.WWCOGS_BusLine,
RP.CURRID ,
ADD_MONTHS(to_date('01-01-'||rp.year,'DD-MM-YYYY'),level-1) KFDATE
from
(SELECT CP.COUNTRYID,
CP.PRDCODE,
CP.REPID,
CP.CHANNELID,
CP.CUSTOMERID,
IP.DIVISION,
CP.WWCOGS_GAUSS,
CP.WWCOGS_SAP,
CP.WWCOGS_BusLine,
CP.year,
CP.CURRID from
(select distinct IBC.COUNTRYID ,
decode(ls.dmdunit,null,mwa.ProductName,ls.dmdunit) PRDCODE,
ReportingUnit REPID,
'99' CHANNELID,
IBC.COUNTRYID CUSTOMERID ,
(
CASE
WHEN MWA.COGSSourceCF LIKE '%Gauss%'
THEN MWA.COGSPriceCF * MWA.ExchRate
ELSE NULL
END) WWCOGS_GAUSS,
(
CASE
WHEN MWA.COGSSourceCF LIKE '%SAP%'
THEN MWA.COGSPriceCF * MWA.ExchRate
ELSE NULL
END) WWCOGS_SAP,
(
CASE
WHEN MWA.COGSSourceCF NOT LIKE '%Gauss%'
AND MWA.COGSSourceCF NOT LIKE '%SAP%'
THEN MWA.COGSPriceCF * MWA.ExchRate
ELSE NULL
END) WWCOGS_BusLine,
mwa.year,
IRC.CURRID
from BAM.M_WWCOGS_AREA MWA,
MICSTAG.M_IBP_REPUNIT_CURRENCY IRC,
micstag.M_IBP_BDREPORTINGCOUNTRY IBC
,MICSTAG.M_LOCALPRODUCT_STAG LS
WHERE MWA.ReportingUnit=IRC.REPID
--and mwa.productname='FR21030390085'
--and MWA.GaussCountry='BE BELGIUM'
AND IBC.COUNTRYPLANNINGGROUP =MWA.GaussCountry
and IBC.businessdivision=31
and MWA.COGSPriceCF <>0
and ls.REPORTINGUNITID(+)=mwa.ReportingUnit
and ls.IBPLOCALPRDID(+)= mwa.ProductName ) CP, micstag.M_IBP_PRODUCT IP
where CP.PRDCODE=IP.PRDID) RP
CONNECT BY level <= 12 ;
the above query is getting unwanted duplicate, if i use distinct the query is running forever.
req. duplicate the records based on year in result set of rp
consider value of year is 2019 than 12 records should came from 1-jan-2019 to 1-dec-2019.
more than one value of year are possible
The CONNECT BY LEVEL <= 12 trick works nicely with dual because that table returns one row. Things are messier when they base result set returns more than one row, because the CONNECT BY generates a product. This is why you're getting duplicates.
What you need to do is specify some additional criteria for the connection. Ideally you will have a primary key in the projection - I'm going to assume it's REPID, so if it's something different you'll need to tweak this. Anyway, you'll need something like this:
) RP
CONNECT BY level <= 12
and rp.repid = prior rp.repid
and prior sys_guid() is not null
The prior sys_guid() bit prevents ORA-01436: CONNECT BY loop in user data.
I have connections into a system and in the database I have the username, starttime_of_connection, and endtime_of_connection. I want to be able to check by minute, how many connections were active at each minute. This is using Oracle DB. Any suggestions or help?
Assuming that starttime_of_connection and endtime_of_connection are date or timestamp columns, you can do something like this. I'm dynamically generating dates with every minute today in the each_minute common table expression (CTE). You can extend this to generate dates across multiple days or to substitute in some other table that you already have that gives you the data for each minute.
WITH each_minute AS (
SELECT trunc(sysdate) + numtodsinterval( level, 'minute' ) dt
FROM dual
CONNECT BY level <= 24*60
)
SELECT em.dt, count(*) num_active_connections
FROM each_minute em
LEFT OUTER JOIN your_table yt
ON( em.dt BETWEEN yt.starttime_of_connection AND yt.endtime_of_connection )
I'm also assuming that endtime_of_connection is always populated-- you may use a NULL endtime_of_connection to indicate a connection that has not yet ended in which case your join would be something like
ON( em.dt BETWEEN yt.starttime_of_connection AND yt.endtime_of_connection
OR (em.dt >= yt.starttime_of_connection AND yt.endtime_of_connection IS NULL) )
This query worked for me:
with period as (select date '2015-04-01' d1,
date '2015-04-30' + 1 - interval '1' minute d2 from dual),
tmp as (
select d1, d2, trunc(starttime_of_connection, 'mi') soc,
trunc(nvl(endtime_of_connection, greatest(starttime_of_connection, d2)), 'mi') eoc
from connections, period),
cn as (select greatest(soc, d1) cd1, least(eoc, d2) cd2
from tmp where soc<=d2 and eoc>=d1),
mt as (select d1 + (level-1)/(24*60) m
from period connect by d1 + (level-1)/(24*60) <= d2)
select to_char(m, 'mm-dd hh24:mi') minute, count(cd1) cnt
from mt /*left*/ join cn on m between cd1 and cd2
group by m order by m
SQLFiddle demo
In first line please change dates for analyzed period. If you want to show minutes with no connections, for instance to make graph then change join at bottom to left join.
For big number of connections and long period query may execute very long. If so you can divide period to weekends/days and union results.
Subquery tmp truncates dates to minutes and fills endtime of connections for null values.
cn filters previous results to interesting period. mt is hierarchical minute generator.
Main query joins connections cn and minutes mt, groups them and orders by time.
If there was connection which started at hour 23:34:07 and ended at 23:34:45 it is counted for minute 23:34.
I am running this query currently for one scenario but I have two scenarios: If SYSDATE = Monday, then run "SYSDATE - 2", otherwise run "SYSDATE - 1". I'm connecting to the database via an OLE connection from Excel so I'm not sure I can use a stored procedure. Is there a way to write the query to accomplish both scenarios? Thanks for all help.
SELECT
DISTINCT VERSION_NAME VERSION, MIN(RECONCILE_START_DT) DATES
FROM
SDE.GDBM_RECONCILE_HISTORY
WHERE
RECONCILE_RESULT = 'Conflicts'
AND
RECONCILE_START_DT > SYSDATE -1
GROUP BY VERSION_NAME
ORDER BY 2 ASC NULLS LAST
You may use a CASE statement in your WHERE condition to subtract either 2 for Mondays or 1 for the rest of the week. TO_CHAR(DATE, 'D') delivers the day of week beginning with Sundays = 1. Therefore Mondays are 2.
Try this:
SELECT
VERSION_NAME AS VERSION,
MIN(RECONCILE_START_DT) AS DATES
FROM
SDE.GDBM_RECONCILE_HISTORY
WHERE
RECONCILE_RESULT = 'Conflicts'
AND
RECONCILE_START_DT > SYSDATE -
CASE TO_CHAR(SYSDATE, 'D')
WHEN '2' THEN 2
ELSE 1 END
GROUP BY VERSION_NAME
ORDER BY 2 ASC NULLS LAST
Also you don't need the DISTINCT keyword as you're already use GROUP BY.