I'd like to create a table like the following columns:
server - event count - event count last period
what do I mean by that?
I have to do a count by of the events, and this is trivial.
base query | stats count as events by source
now i have a selector that let me select the period (the classic splunk time selector).
what i need is the following:
if the selector is "last week" i need to count in the first column the events of the last week and in the second column the events of the week before that
if the selector is "last mount" i need to count in the first column the events of the last mount and in the second column the events of the mount before that
etc...
Id like to do that without messing with html, xml or any other language. I'd like, if possible, a plain splunk search.
Thanks a lot.
Andrea
This is a partial solution because involves changing variables to change the time span.
BASE SEARCH earliest=-14d latest=now
| eval when=if(_time>relative_time(now(), "-7d#d"), "Current_Week", "Prev_Week")
| stats count as events by source when
| chart sum(events) by source, when
| eval perc = (Current_Week-Prev_Week)/Prev_Week
| eval trend = case(perc < -0.3, "low", (perc >= -0.3 and perc <= 0.3 ), "madium", perc > 0.3, "high")
| table source, Current_Week, Prev_Week, perc, trend
Related
In my cube, I have several measures at the day grain that I'd like to sum at the day grain but average (or take latest) at the month grain or year grain.
Example:
We have a Fact table with Date and number of active subscribers in that day (aka PMC). This is snapshotted per day.
dt
SubscriberCnt
1/1/22
50
1/2/22
55
This works great at the day level. At the month level, we don't want to sum these two values (count = 105) because it doesn't make sense and not accurate.
when someone is looking at month grain, it should look like this - take the latest for the month. (we may change this to do an average instead, management is still deciding)
option 1 - Take latest
Month-Dt
Subscribers
Jan-2022
55
Feb-2022
-
option 2 - Take aveage
Month-Dt
Subscribers
Jan-2022
52
Feb-2022
-
I've not been able to find the right search terms for this but this seems like a common problem.
I added some sample data at the end of a month for testing:
dt
SubscriberCnt
12/30/21
46
12/31/21
48
This formula uses LASTNONBLANKVALUE, which sorts by the first column and provides the latest value that is not blank:
Monthly Subscriber Count = LASTNONBLANKVALUE( 'Table'[dt], SUM('Table'[SubscriberCnt]) )
If you do an AVERAGE, a simple AVERAGE formula will work. If you want an average just for the current month, then try this:
Current Subscriber Count =
VAR _EOM = CLOSINGBALANCEMONTH( SUM('Table'[SubscriberCnt]), DateDim[Date] )
RETURN IF(_EOM <> 0, _EOM, AVERAGE('Table'[SubscriberCnt]) )
But the total row will be misleading, so I would add this so the total row is the latest number:
Current Subscriber Count =
VAR _EOM = CLOSINGBALANCEMONTH( SUM('Table'[SubscriberCnt]), DateDim[Date] ) //Get the number on the last day of the month
VAR _TOT = NOT HASONEVALUE(DateDim[MonthNo]) // Check if this is a total row (more than one month value)
RETURN IF(_TOT, [Monthly Subscriber Count], // For total rows, use the latest nonblank value
IF(_EOM <> 0, _EOM, AVERAGE('Table'[SubscriberCnt]) ) // For month rows, use final day if available, else use the average
)
I have created a running total measure in PBI using Dax, however, when the total does not filter by column filter when it is in the table. The running total should sum the balance but then break out into the individual maturity buckets within the table.
Here is the Measure and the resulting table
I have tried adding extra filters using the FILTER/ALL command to break out the maturity buckets and gotten either the same result or errors. Not sure what else I can do?
Here is a fake sample of data. in the comments i have include the language of the measure.
|Date |Tenor|Balance |
|-------------|-----|-----------------|
|December 2022|18m |0.196072326627487|
|December 2022|2y |0.149643186475954|
|December 2022|3y |0.180522608363889|
|December 2022|4y |0.780540306321475|
|December 2022|5y |0.156029893270158|
|January 2022|18m |0.512496934496972|
|January 2023|2y |0.068123785829084|
|January 2023|3y |0.349971677118287|
Here is my solution! I am sorry if I kept you waiting too much!
First, I need to say that your design is not so efficient! You need a full date table with full date values(not only month &year parts)
Here is your calendar table:
Here is the DAX Code you need to write to obtain correct value:
Total_Correct =
CALCULATE (
SUM('Callable'[Balance]),
FILTER ( ALL ( 'Callable'[Date] ), 'Callable'[Date] <= MAX('Callable'[Date])),
ALL('Calendar')
)
I'm currently evaluating a use case in Azure Application Insights but I'm open to use any other framework of infrastructure that would fit best.
So basically I have a desktop application who logs some events or traces (I don't exactly know which one it should be). Examples of events (or traces?)
| timestamp | state | user |
------------------------------------------
| yyyy-mm-dd 12:00 | is_at_home | John |
| yyyy-mm-dd 15:00 | is_at_work | John |
| yyyy-mm-dd 18:00 | is_outside | John |
Users are considered to be in the last state received until new event comes.
I need to extract data to answer questions like this:
I want to see if the total duration John is at home is growing or going down.
I want to get in which states the users pass most time.
I want the average duration of the state "is_at_work". And if it's going down or up over time.
So, Can the application insights output this kind of analysis? If not, which architecture/platform should I use? I'm I using the right keywords to describe what I want?
Thank you
the ai/log analytics query language (kql) supports all kinds of things like that. the trick you'll have is getting your queries exactly right, here you'll have to figure out exactly what you need to do so that you calculate the times between rows as "state" changes.
here's my first attempt:
let fakeevents = datatable (timestamp: datetime, state: string, user: string ) [
datetime(2021-08-02 12:00), "is_at_home" , "John" ,
datetime(2021-08-02 15:00), "is_at_work" , "John",
datetime(2021-08-02 18:00), "is_outside" , "John",
datetime(2021-08-02 11:00), "is_at_home" , "Jim" ,
datetime(2021-08-02 12:00), "is_at_work" , "Jim",
datetime(2021-08-02 13:00), "is_outside" , "Jim",
];
fakeevents | partition by user (
order by user, timestamp desc |
extend duration = prev(timestamp, 1, now()) - timestamp
)
gets me:
timestamp
state
user
duration
2021-08-02T18:00:00Z
is_outside
John
06:20:23.1748874
2021-08-02T15:00:00Z
is_at_work
John
03:00:00
2021-08-02T12:00:00Z
is_at_home
John
03:00:00
2021-08-02T13:00:00Z
is_outside
Jim
11:25:14.6912472
2021-08-02T12:00:00Z
is_at_work
Jim
01:00:00
2021-08-02T11:00:00Z
is_at_home
Jim
01:00:00
before you send any data real data, you can create "fake" data by using the datatable operator to make a fake table full of data.
you can then apply things like summarize to calculate things like which had the max, etc. note the use of partition by user to make sure each user is treated separately. in my assumption i use now() if there's no value to end the duration of an event, you'll want to do something there otherwise you'll have blank cells.
I have a list of products and would like to get a 50 day simple moving average of its volume using Power Query (M).
The table is sorted by product name and date. I add a custom column and applied the code below.
if [date] >= #date(2018,1,29)
then List.Average(List.Range(Source[Volume],[Volume]-1,-50))
else ""
Since it is already sorted by date and name, an if statement was applied with a date as criteria/filter. However, an error occurs that says
'Volume' column not found in the table.
I expect to have an added column in the power query with volume 50 day moving average per product. the calculation to be done if date is greater than or equal Jan 29, 2018.
We don't know what your columns are, but assuming you have [product], [date] and [volume] in Source, this would average the last 50 days of [volume] for the identical [product] based on each [date], and place in a new column
AvgAmountAdded = Table.AddColumn(Source, "AverageAmount", (i) => List.Average(Table.SelectRows(Source, each ([product] = i[product] and [date]<=i[date] and [date]>=Date.AddDays(i[date],-50)))[volume]), type number)
Finally! found a solution.
First, apply Index by product see this post for further details
Then index again without criteria (index all rows)
Then, apply below code
= Table.AddColumn(#"Previous Step", "Volume SMA(50)", each if [Index_byProduct] >= 50 then List.Average(List.Range(#"Previous Step"[Volume], ([Index_All]-50),50)) else 0),
For large dataset, Table.Buffer function is recommended after index-expand step to improve PQ calculation speed
I've searched quite a bit for this and can't find a good solution anywhere to what seems to me like a normal problem for this product.
I've got a data table (in memory) that is from a rollup table(call it 'Ranges'). Basically like so:
id | name | f1 | f2 | totals
0 | Channel1 | 450 | 680 | 51
1 | Channel2 | 890 | 990 | 220
...and so on
Which creates a bar chart with Name on the X and Totals on the Y.
I have another table that is an external link to a large (500M+ rows) table. That table (call it 'Actuals') has a column ('Fc') that can fit inside the F1 and F2 values of Ranges.
I need a way for Spotfire Analyst (v7.x) to use the selection of the the bar chart for Ranges to trigger this select statement:
SELECT * FROM Actuals WHERE Actuals.Fc between [Ranges].[F1] AND [Ranges].[F2]
But there aren't any relationships (Foreign keys) between the two data sources, one is in memory (Ranges) and the other is dynamic loaded.
TLDR: How do I use the selected rows from one visualization as a filter expression for another visualization's data?
My choice for the workaround:
Add a button which says 'Load Selected Data'
This will run the following code, which will store the values of F1 and F2 in a Document Property, which you can then use to filter your Dynamically Loaded table and trigger a refresh (either with the refresh code or by setting it to load automatically).
rowIndexSet=Document.ActiveMarkingSelectionReference.GetSelection(Document.Data.Tables["IL_Ranges"]).AsIndexSet()
if rowIndexSet.IsEmpty != True:
Document.Properties["udF1"] = Document.Data.Tables["IL_Ranges"].Columns["F1"].RowValues.GetFormattedValue(rowIndexSet.First)
Document.Properties["udF2"] = Document.Data.Tables["IL_Ranges"].Columns["F2"].RowValues.GetFormattedValue(rowIndexSet.First)
if Document.Data.Tables.Contains("IL_Actuals")==True:
myTable=Document.Data.Tables["IL_Actuals"]
if myTable.IsRefreshable and myTable.NeedsRefresh:
myTable.Refresh()
This is currently operating on the assumption that you will not allow your user to view multiple ranges at a time, and simply shows the first one selected.
If you DO want to allow them to view multiple ranges, you can run a cursor through your IL_Ranges table to either get the Min and Max for each value, and limit the Actuals between the min and max, or you can create a string that will essentially say 'Fc between 450 and 680 or Fc between 890 and 990', pass that through to a stored procedure as a string, which will execute the quasi-dynamic statement, and grab the resulting dataset.