more problems with the LAG function is SAS - algorithm

The following bit of SAS code is supposed to read from a dataset which contains a numeric variable called 'Radvalue'. Radvalue is the temperature of a radiator, and if a radiator is switched off but then its temperature increases by 2 or more it's a sign that it has come on, and if it is on but its temperature decreases by 2 or more it's a sign that it's gone off.
Radstate is a new variable in the dataset which indicates for every observation whether the radiator is on or off, and it's this I'm trying to fill in automatically for the whole dataset.
So I'm trying to use the LAG function, trying to initialise the first row, which doesn't have a dif_radvalue, and then trying to apply the algorithm I just described to row 2 onwards.
Any idea why the columns Radstate and l_radstate come out completely blank?
Thanks everso much!! Let me know if I haven't explained the problem clearly.
Data work.heating_algorithm_b;
Input ID Radvalue;
Datalines;
1 15.38
2 15.38
3 20.79
4 33.47
5 37.03
6 40.45
7 40.45
8 40.96
9 39.44
10 31.41
11 26.49
12 23.06
13 21.75
14 20.16
15 19.23
;
DATA temp.heating_algorithm_c;
SET temp.heating_algorithm_b;
DIF_Radvalue = Radvalue - lag(Radvalue);
l_Radstate = lag(Radstate);
if missing(dif_radvalue) then
do;
dif_radvalue = 0;
radstate = "off";
end;
else if l_Radstate = "off" & DIF_Radvalue > 2 then Radstate = "on";
else if l_Radstate = "on" & DIF_Radvalue < -2 then Radstate = "off";
else Radstate = l_Radstate;
run;

You were trying to perform the LAG function on a variable only existing in the output data set (RADSTATE). I replaced the LAG on RADSTATE with a RETAIN. Also, you were right to keep the LAG function outside any conditional logic...Try the below code.
Data work.heating_algorithm_b;
Input ID Radvalue;
Datalines;
1 15.38
2 15.38
3 20.79
4 33.47
5 37.03
6 40.45
7 40.45
8 40.96
9 39.44
10 31.41
11 26.49
12 23.06
13 21.75
14 20.16
15 19.23
;
DATA work.heating_algorithm_c;
length radstate $3;
retain radstate;
SET work.heating_algorithm_b;
old_radvalue=lag(radvalue);
if _n_=1 then do;
dif_radvalue=0;
radstate="off";
end;
else do;
DIF_Radvalue = Radvalue-Old_Radvalue;
if Radstate = "off" & DIF_Radvalue > 2 then Radstate = "on";
else if Radstate = "on" & DIF_Radvalue < -2 then Radstate = "off";
/* Else Radstate stays the same */
end;
run;

I have no SAS experience, but maybe you need a missing(l_Radstate) check to cover the first time through, maybe something like this:
if missing(l_Radstate) then
do; radstate = "off"; end;
I think that would only be needed if the Radvalue - lag(Radvalue) did not force DIF_Radvalue to be missing. If it does, I am not sure what would help...

Related

Comparing several table columns

I have two tables with a different number of rows and three columns. If, for a certain row, the values in column one and two are the same as in the other table, I want to select the values from the third column. Below is a sample code that works, but since in reality, I have over 2 million rows in both tables, it takes a very long time to run. Is there a way to speed up the code by getting rid of the loops?
MOOSE2=table(['2010/03/30 00:30:00'; '2010/03/22 18:00:00' ; '2010/04/21 18:30:00'; '2010/02/20 02:20:00'; '2010/03/10 02:30:00'],[5 ;8 ;4; 9 ;7],[10; 11 ;12 ;13 ;14]);
Lion2=table(['2010/03/30 00:30:00'; '2010/04/21 18:30:00'; '2010/03/20 22:00:00'; '2010/03/10 02:00:00'],[5;4;6;7],[17;12;11;14]);
[sMOOSE,~]=size(MOOSE2);
[sLion,~]=size(Lion2);
dmoose=[];
dlion=[];
for i=1:sLion
for j=1:sMOOSE
if (MOOSE2.(1)(j,:)==Lion2.(1)(i,:))&(MOOSE2.(2)(j,:)==Lion2.(2)(i,:))
dmoose=[dmoose;MOOSE2.(3)(j,:)];
dlion=[dlion;Lion2.(3)(i,:)];
end
end
end
Which gives me the correct output of
dlion =
17
12
dmoose =
10
12
Perfect scenario for intersect:
MOOSE2=table(['2010/03/30 00:30:00'; '2010/03/22 18:00:00' ; '2010/04/21 18:30:00'; '2010/02/20 02:20:00'; '2010/03/10 02:30:00'],[5 ;8 ;4; 9 ;7],[10; 11 ;12 ;13 ;14]);
Lion2=table(['2010/03/30 00:30:00'; '2010/04/21 18:30:00'; '2010/03/20 22:00:00'; '2010/03/10 02:00:00'],[5;4;6;7],[17;12;11;14]);
[~,moose_index,lion_index] = intersect(MOOSE2(:,1:2),Lion2(:,1:2),'rows');
dlion = Lion2.Var3(lion_index)
dmoose = MOOSE2.Var3(moose_index)

netcool omnibus probe rules

I have following probable values for $6 coming to netcool omnibus rules.
I would like to extract the InstanceName from $6
Eg: SQLSERVER1
SQL2012TESTRTM
SQL2012TESTSTD1
SQL2014STD
MSSQLSERVER
Below are $6 values
6 = "Microsoft.SQLServer.DBEngine:TM-B33F-FAD4.cap.dev.net;SQLSERVER1:1"
6 = "Microsoft.SQLServer.2012.Agent:TM-B33F-FAD4.cap.dev.net;SQL2012TESTRTM;SQLAgent$SQL2012TESTRTM:1"
6 = "Microsoft.SQLServer.2012.Agent:TM-B33F-FAD4.cap.dev.net;SQL2012TESTRTM;SQLAgent$SQL2012TESTRTM:1"
6 = "Microsoft.SQLServer.2012.Agent:TM-B33F-FAD4.cap.dev.net;SQL2012TESTSTD1;SQLAgent$SQL2012TESTSTD1:1"
6 = "Microsoft.SQLServer.Database:TM-B33F-FAD4.cap.dev.net;SQL2012TESTSTD1;DB2:1"
6 = "Microsoft.SQLServer.2012.Agent:TM-B33F-FAD4.cap.dev.net;SQL2012TESTRTM;SQLAgent$SQL2012TESTRTM:1"
6 = "Microsoft.SQLServer.2014.Agent:TM-B33F-FAD4.cap.dev.net;SQL2014STD;SQLAgent$SQL2014STD:1"
6 = "Microsoft.SQLServer.Database:TM-B33F-FAD4.cap.dev.net;SQL2012TESTSTD1;DB2:1"
6 = "Microsoft.SQLServer.2014.DBEngine:TM-B33F-FAD4.cap.dev.net;SQL2014STD:1"
6 = "Microsoft.SQLServer.Database:TM-B33F-FAD4.cap.dev.net;SQL2012TESTSTD1;DB2:1"
6 = "Microsoft.SQLServer.2014.Agent:TM-B33F-FAD4.cap.dev.net;SQL2014STD;SQLAgent$SQL2014STD:1"
6 = "Microsoft.SQLServer.Database:TM-B33F-FAD4.cap.dev.net;SQL2012TURKSTD1;DB1:1"
6 = "Microsoft.SQLServer.2014.DBFile:CTNTV01;MSSQLSERVER;SPOT;1;35:1"
6 = "Microsoft.SQLServer.Library.EventLogCollectionTarget:TM-B33F-FAD4.cap.dev.net:1"
I have tried below code to extract, it works for most of them above.
#temp = extract($6, ";([^\:]+)\:")
if (regmatch(#temp, "[\;]"))
{
#temp = extract(#temp, "([^\:]+)\;")
}
But it does not work for
Microsoft.SQLServer.2014.DBFile:CTNTV01;MSSQLSERVER;SPOT;1;35:1
I believe the second extract inside if statement needs to be corrected little more.
It extracts until MSSQLSERVER;SPOT;1, however I only want MSSQLSERVER from it.
Can you please help in correcting this.
Try with below.
#temp = extract($6, ";([^\:]+)\:")
if (regmatch(#temp, "[\;]"))
{
#temp = extract(#temp, "([^\;]+)\;")
}

VBA written in Excel for Windows not working on Mac

I have a set of macros to hide and unhide columns based on the contents of a specific row. They were all written in Excel 2013 for Windows (running in parallels on my MBA, if that's relevant) and work fine there. But when I open the worksheet in Excel 2011 for Mac, the macros give odd results. The "unhide all columns" macro works fine; the other functions get as far as hiding all columns but not as far as unhiding the ones I want to see.
I can only assume Excel for Mac is having a problem with what's in the FOR EACH loop, but I can't figure out what! I'd appreciate any guidance: I need to get this system working on both Windows and Mac.
Code below.
This function works:
Sub GANTT_Filter_Show_All()
Dim rngDates As Range
Set rngDates = Range("GANTT_Dates")
rngDates.EntireColumn.Hidden = False
End Sub
But this one only hides all the columns:
Sub GANTT_Filter_This_Quarter()
Dim intCurrentMonth As Integer, intCurrentYear As Integer, rngDates As Range, cell As Range
Dim intCurrentQuarterMonths(3) As Integer
Set rngDates = Range("GANTT_Dates")
intCurrentMonth = DatePart("m", Date)
intCurrentYear = DatePart("yyyy", Date)
'loading months of current quarter into an array intCurrentMonth
Select Case intCurrentMonth
Case 1 To 3
intCurrentQuarterMonths(0) = 1
intCurrentQuarterMonths(1) = 2
intCurrentQuarterMonths(2) = 3
Case 4 To 6
intCurrentQuarterMonths(0) = 4
intCurrentQuarterMonths(1) = 5
intCurrentQuarterMonths(2) = 6
Case 7 To 9
intCurrentQuarterMonths(0) = 7
intCurrentQuarterMonths(1) = 8
intCurrentQuarterMonths(2) = 9
Case 10 To 12
intCurrentQuarterMonths(0) = 10
intCurrentQuarterMonths(1) = 11
intCurrentQuarterMonths(2) = 12
End Select
'hiding all columns
rngDates.EntireColumn.Hidden = True
'comparing each column to array of months in current quarter and hiding if false
For Each cell In rngDates
For Each v In intCurrentQuarterMonths
If v = DatePart("m", cell.Value) And DatePart("yyyy", cell.Value) = intCurrentYear Then cell.EntireColumn.Hidden = False
Next v
Next cell
Application.Goto Reference:=Range("a1"), Scroll:=True
End Sub
I'm with #Steven on this one, nothing obviously wrong with the code. I'm not a Mac user, but it's entirely possible that there's some weirdness around the date functions, particularly those that require formatting to resolve.
I would try replacing the calls to DatePart() with calls to Month() and Year() in situations like this - even for non-Mac users. It doesn't rely on parsing the strings for formatting, so it's much more efficient (and easy to read):
Sub Benchmarks()
Dim starting As Double, test As Date, i As Long
test = Now
starting = Timer
For i = 1 To 1000000
Year test
Next i
Debug.Print "Elapsed: " & (Timer - starting)
starting = Timer
For i = 1 To 1000000
DatePart "yyyy", test
Next i
Debug.Print "Elapsed: " & (Timer - starting)
End Sub
Since you likely can't run the benchmark...
Elapsed for Year(): 0.109375
Elapsed for DatePart(): 0.515625
Also note that in addition to this, the dates in the column you're searching are coming through as Variants, it may help to explicitly cast them to dates:
If v = Month(CDate(cell.Value)) And intCurrentYear = Year(CDate(cell.Value)) Then
cell.EntireColumn.Hidden = False
End If

Excel Macro, vlookup function works slow, ways to speed up macro

Hello stackexchange community.
I've built a simple tables converter, the main function of which is to convert the table from
1a Value
1b Value
1c Value
1d Value
to
a b c d
1 Value Value Value Value
Unfortunately, the macro runs pretty slow (~ 3 lines per second for one column).
I'd really appreciate if someone could take a look at my piece of code and suggest the way to speed it up.
Here's the piece of code:
Dim LastFinalList As Integer: LastFinalList = Sheet1.Range("O1000").End(xlUp).Row
For Col = 16 To 19
For c = 2 To LastFinalList
searchrange = Sheet1.Range("J:L")
lookfor = Sheet1.Cells(c, 15) & Sheet1.Cells(1, Col)
CountFor = Application.VLookup(lookfor, searchrange, 3, False)
If IsError(CountFor) Then
Sheet1.Cells(c, Col).Value = "0"
Else
Sheet1.Cells(c, Col).Value = CountFor
End If
Next c
Next Col
Thanks in advance and best regards!
UPD:
The Data in unconverted table looks like this (e.g):
Updated by Macro
Value Number Type Key Count Average Value
10 1 a 1a 2 20
30 1 a 1a 2 20
40 1 b 1b 1 40
50 1 c 1c 1 50
So it is also required to calculate averages of repeating types, create a unique list of Numbers (which is LastFinalList in my case) and finally convert it to this:
Number a b c
1 20 40 50
application.vlookupseraches by Number&Type Key, which is also assigned in the unconverted table by macro. The same time those Keys are counted, in order to calculate average for the repeating ones.
Everything works in a blink of an eye till it comes to 'to update final table part.
Full Code:
Sub ConvertToTable()
Dim LastMeter As Integer: LastMeter = Sheet1.Range("I1000").End(xlUp).Row
Sheet1.Range(Cells(2, 9), Cells(LastMeter, 9)).AdvancedFilter Action:=xlFilterCopy, CopyToRange:=Sheet1.Range("O2"), Unique:=True
Sheet1.Range("O1").Value = "The List"
Sheet1.Range("O2").Delete Shift:=xlUp
' to assign keys
For i = 2 To LastMeter
Set CountOpt = Sheet1.Cells(i, 10)
Sheet1.Cells(i, 10).FormulaR1C1 = "=r[0]c[-1]&r[0]c[-2]"
Sheet1.Cells(i, 11).FormulaR1C1 = "=COUNTIF(c10:c10, r[0]c10)"
Next i
'to calculate averages
For x = 2 To LastMeter
If Sheet1.Cells(x, 11).Value = 1 Then
Sheet1.Cells(x, 12).FormulaR1C1 = "=rc7"
ElseIf Sheet1.Cells(x, 11).Value > 1 Then
If Sheet1.Cells(x, 10).Value <> Sheet1.Cells(x - 1, 10).Value Then
Sheet1.Cells(x, 12).FormulaR1C1 = "=ROUND((SUM(rc7:r[" & Sheet1.Cells(x, 11).Value - 1 & "]c7)/" & Sheet1.Cells(x, 11).Value & "),4)"
Else
Sheet1.Cells(x, 12).FormulaR1C1 = "=r[-1]c12"
End If
End If
Next x
'to update final table
Application.ScreenUpdating = False
Application.Calculation = xlCalculationManual
Dim LastFinalList As Integer: LastFinalList = Sheet1.Cells(Rows.Count, 15).End(xlUp).Row
For Col = 16 To 19
For c = 2 To LastFinalList
searchrange = Sheet1.Range("J:L")
lookfor = Sheet1.Cells(c, 15) & Sheet1.Cells(1, Col)
CountFor = Application.VLookup(lookfor, searchrange, 3, False)
If IsError(CountFor) Then
Sheet1.Cells(c, Col).Value = "0"
Else
Sheet1.Cells(c, Col).Value = CountFor
End If
Next c
Next Col
Application.Calculation = xlCalculationAutomatic
Application.ScreenUpdating = True
Sheet1.Range("O1").Select
End Sub
Also, initially i had a SUMIF formula instead of application.vlookup to be input in each cell in the converted table. But the code was working as slow as now an was bit bulky, that's why i've decide to switch to VLOOKUP.
The thing is, if it actually the way application.vlookup works (with 0.3sec delay for each row), then i guess there's nothing that can be done, and i'm ok to accept that. Although, if that's not the case, i'd really appreciate if someone could help me out and speed up the process.
Thanks!
You can redefine your LastFinalList variable something like
LastFinalList = Sheets("Sheet1").UsedRange.Rows.Count
OR
LastFinalList = Sheets("Sheet1").Cells(Rows.Count, 2).End(xlUp).Row
instead of explicitly defining used range.
Also use following line of code before your code
Application.ScreenUpdating = False
(Turn screen updating off to speed up macro code. User won't be able to see what the macro is doing, but it will run faster.)
After the whole code runs you can(optional) turn screen updating on with
Application.ScreenUpdating = True
It appears that application.vlookup in my particular case was indeed working very slow (no idea why, though). I've managed to improve macro by replacing vlookup with SUMIF formula in each cell, so now the converted table is updated instantly. Thanks everyone who participated and provided their suggestions!

Very simple set value of array cell, program very slow when he writes on specify column

I am using a continuous and old professional program. My program builds several simple data arrays and writes the array to an excel cell like this:
Sheets("toto").Cells(4,i) = "blabla"
But for one value of i, the write time is very long and I don't understand why.
Here is my code :
...
For No_Bug = 0 To Indtab - 1
If mesComments(No_Bug) <> "" Then
Sheets(feuille_LBT).Cells(Ligne_Bug, 1) = Ligne_Bug - 5
Sheets(feuille_LBT).Cells(Ligne_Bug, 2) = mesID_Test(No_Bug)
Sheets(feuille_LBT).Cells(Ligne_Bug, 3) = mesResultats(No_Bug)
Sheets(feuille_LBT).Cells(Ligne_Bug, 4) = mesComments(No_Bug)
Sheets(feuille_LBT).Cells(Ligne_Bug, 5).FormulaLocal = mesScreens(No_Bug)
Sheets(feuille_LBT).Cells(Ligne_Bug, 6) = 2 'If I comment only this line, the programm is fast, ifnot the programm is very slow (~1, 2 secondes per loop), What the hell ??? xD
Sheets(feuille_LBT).Cells(Ligne_Bug, 7) = 1
End If
...
Is this cell referenced from other cells? Check if any complicated computations related with this cell.

Resources