Why can't I call load() on a variable? - empirical-lang

I can call load() in the REPL.
>>> load("trades.csv")
symbol timestamp price size
AAPL 2019-05-01 09:30:00.578802 210.5200 780
AAPL 2019-05-01 09:30:00.580485 210.8100 390
BAC 2019-05-01 09:30:00.629205 30.2500 510
CVX 2019-05-01 09:30:00.944122 117.8000 5860
AAPL 2019-05-01 09:30:01.002405 211.1300 320
AAPL 2019-05-01 09:30:01.066917 211.1186 310
AAPL 2019-05-01 09:30:01.118968 211.0000 730
BAC 2019-05-01 09:30:01.186416 30.2450 380
... ... ... ...
But I can't load a command-line argument in a script:
let trades = load(argv[1])
I get an error:
Error: macro parameter filename requires a comptime literal
Error: unable to determine type for trades

Empirical is statically typed. load() will infer a Dataframe's type if the file path can be determined at compile time. Variables and arbitrary expressions are permitted. For example:
let filename = "trades.csv"
let trades = load("./" + filename)
The filename above is a constant (immutable) because it was declared with let. But argv is mutable (not constant) because it is implicitly declared with var.
Empirical must know the type of the Dataframe, which we will have to pass explicitly in this case. Fortunately, we can get a hint in the REPL.
>>> columns(load("trades.csv"))
symbol: String
timestamp: Timestamp
price: Float64
size: Int64
We will also have to call csv_load() instead, since that takes a template. Here is the script as it should be:
data Trade:
symbol: String,
timestamp: Timestamp,
price: Float64,
size: Int64
end
let trades = csv_load{Trade}(argv[1])
This compiles as expected. Calling print(trades) in the script will display the Dataframe.

Related

Pine-Script - Input actual time as Default

i´m trying to fill a Date input with a default value, the default value should be the actual date.
The original code:
end_time = input.time(defval=timestamp('01 Aug 2021 00:00 +0000'), title='End Time')
I would like to have the actual date as defval.
i tried several things, like timestamp(timenow) but i get always the same error-message:
An argument of 'series int' type was used but a 'const string' is expected
I understand that i bring a int to a string, but how can i convert the actual date/time to fit in the code above?
Thanx for your brains,
The input.*() function defval= argument requires a value known at the compilation time and could not be dynamic.
As a workaround, you can use the interactive feature (confirm= argument) to select the last bar on the chart with a mouse-click:
//#version=5
indicator("My script")
end_time = input.time(defval=timestamp('01 Aug 2021 00:00 +0000'), title='End Time', confirm = true)
bgcolor(time == end_time ? color.red : na)

Convert REG_BINARY value (64 bit Hex Value - Little Endian) to date and time

I would like to convert a Windows NT Shutdown Time to Date and Time:
88 33 9d cb 38 36 d0 01 --> 88339dcb3836d001 --> 22.01.2015 11:44:35 (UTC)
The input data is this 64 bit hex value (8 bytes). You find this value in your windows registry under HKLM\SYSTEM\ControlSet001\Control\Windows --> Shutdown Time. This value contains time and date of last shutdown. The value is of type FILETIME - A 64-bit time value (Hex Value, Little Endian) representing the number of 100-nanosecond intervals since midnight UTC, 1 Jan 1601.
Is this possible via shell (bash)? I would like to implement it in a shell script.
As written in the question, a Windows NT shutdown time is a count of 100ns intervals from 1st January 1601. The only catch is that the timestamp is encoded as little-endian. Decoding the timestamp and adding it to the epoch is a bit tricky in pure Bash, but becomes rather easy if you can spare a Python installation.
Here is a script doing the conversion:
#! /usr/bin/python3
from __future__ import division
import struct
import sys
from binascii import unhexlify
from datetime import datetime, timedelta
nt_timestamp = struct.unpack("<Q", unhexlify(sys.argv[1]))[0]
epoch = datetime(1601, 1, 1, 0, 0, 0)
nt_datetime = epoch + timedelta(microseconds=nt_timestamp / 10)
print(nt_datetime.strftime("%c"))
It accepts the 64bit value as an argument and prints the locale representation of the date encoded:
$ ./nttime.py 88339dcb3836d001
Thu Jan 22 11:44:35 2015
If you need another format, then edit the last line using the standard strftime() format specifiers.
To share more knowledge :-)
$regKey = Get-ItemProperty -Path
Registry::HKEY_LOCAL_MACHINE\System\CurrentControlSet\Control\Windows
$shutDown = $regKey.ShutdownTime
$Int64Value = [System.BitConverter]::ToInt64($shutDown, 0)
$date = [DateTime]::FromFileTime($Int64Value)
$date
Got this one here

How to display managed objects with certain value in one of the fields in WinDbg using SOS (or SOSEX)?

My problem is this:
0:000> !DumpHeap -type Microsoft.Internal.ReadLock -stat
------------------------------
Heap 0
total 0 objects
------------------------------
Heap 1
total 0 objects
------------------------------
Heap 2
total 0 objects
------------------------------
Heap 3
total 0 objects
------------------------------
total 0 objects
Statistics:
MT Count TotalSize Class Name
000007fef3d14088 74247 2375904 Microsoft.Internal.ReadLock
Total 74247 objects
The way I read this output is that I have 74,247 Microsoft.Internal.ReadLock instances on my heap. However, some of them are probably pending collection.
I want to display only those which are not pending collection.
For example, 0000000080f88e90 is the address of one of these objects and it is garbage. I know it, because:
0:000> !mroot 0000000080f88e90
No root paths were found.
0:000> !refs 0000000080f88e90 -target
Objects referencing 0000000080f88e90 (Microsoft.Internal.ReadLock):
NONE
0:000> !do 0000000080f88e90
Name: Microsoft.Internal.ReadLock
MethodTable: 000007fef3d14088
EEClass: 000007fef3c63410
Size: 32(0x20) bytes
File: C:\Windows\Microsoft.Net\assembly\GAC_MSIL\System.ComponentModel.Composition\v4.0_4.0.0.0__b77a5c561934e089\System.ComponentModel.Composition.dll
Fields:
MT Field Offset Type VT Attr Value Name
000007fef3d13fb0 400001e 8 ...oft.Internal.Lock 0 instance 0000000080001010 _lock
000007fef0a8c7d8 400001f 10 System.Int32 1 instance 1 _isDisposed
As one can see, both sosex.mroot and sosex.refs indicate no one references it, plus dumping its fields reveals that it was disposed through IDisposable, so it makes sense that the object is garbage (I know that being disposed does not imply the object is garbage, but it is in this case).
Now I want to display all those instances which are not garbage. I guess I am to use the .foreach command. Something like this:
.foreach(entry {!dumpheap -type Microsoft.Internal.ReadLock -short}){.if (???) {.printf "%p\n", entry} }
My problem is that I have no idea what goes into the .if condition.
I am able to inspect the _isDisposed field like this:
0:000> dd 0000000080f88e90+10 L1
00000000`80f88ea0 00000001
But .if expects an expression and all I have is a command output. If I knew how to extract information from the command output and arrange it as an expression then I could use it as the .if condition and be good.
So, my question is this - is there a way to get the field value as an expression suitable for .if? Alternatively, is it possible to parse the command output in a way suitable for using the result as the .if condition?
I didn't have an example which uses ReadLock objects, but I tried with Strings and this is my result:
.foreach (entry {!dumpheap -short -type Microsoft.Internal.ReadLock})
{
.if (poi(${entry}+10) == 1)
{
.printf "%p\n", ${entry}
}
}
I'm using poi() to get pointer size data from the address. Also note I'm using ${entry} not entry in both, poi() and .printf. You might also like !do ${entry} inside the .if.
In one line for copy/paste:
.foreach (entry {!dumpheap -short -type Microsoft.Internal.ReadLock}) {.if (poi(${entry}+10) == 1) {.printf "%p\n", ${entry}}}

Meaning of mxlc in Oracle Trace file

I am seeing the following in my trace file:
Bind#3 oacdty=01 mxl=128(35) mxlc=36 mal=00 scl=00 pre=00
oacflg=03 fl2=1000010 frm=01 csi=31 siz=0 off=168
kxsbbbfp=ffffffff79f139a8 bln=128 avl=35 flg=01 value="1234 W
1234 West, West Groves City"
I am wondering what the mxlc value is?
I quote
Bind #n
oacdty - Datatype code
mxl - Maximum length of the bind variable value (private maximum length in parentheses)
mxlc - Unknown :(
mal - array length
scl - Scale
pre - Precision
oacflg - Special flag indicating bind options
fl2 - second part of oacflg
frm - Unknown :(
csi - Unknown :(
siz - Amount of memory to be allocated for this chunk
off - Offset into this chunk for this bind buffer
kxsbbbfp- Bind address
bln - Bind buffer length
avl - actual value length
flg - bind status flag
value - Value of the bind variable
Source (& snippet of the book)
The book also quotes-
There is currently no information on three parameters.
Which are mxlc,frm, and csi.
Summary
mxlc appears to be the maximum number of characters for the bind variable, but only if the variable uses character length semantics.
Method
I searched My Oracle Support for mxlc. Almost every article has mxlc=00, the only exceptions involve an NVARCHAR or NCHAR. The code below is based on the code from Document ID 552262.1. I changed the variable sizes (99 and 123 char) around, and each time mxlc was set to the variable size if character length semantics was used.
Code
create table t1(ncol1 nvarchar2(100), col1 varchar2(100));
alter session set timed_statistics = true;
alter session set statistics_level=all;
alter session set max_dump_file_size = unlimited;
alter session set events '10046 trace name context forever,level 4';
VAR nvar1 NVARCHAR2(99)
VAR var1 VARCHAR2(123 char)
EXEC :nvar1 := 'nvarchar'
EXEC :var1 := 'varchar'
SELECT * FROM T1 WHERE ncol1 = :nvar1 and col1 = :var1;
ALTER SESSION SET EVENTS '10046 trace name context off';
Results:
Bind#0
oacdty=01 mxl=2000(198) mxlc=99 mal=00 scl=00 pre=00
oacflg=03 fl2=1000010 frm=02 csi=2000 siz=4000 off=0
kxsbbbfp=0e702edc bln=2000 avl=16 flg=05
value=0 6e 0 76 0 61 0 72 0 63 0 68 0 61 0 72
Bind#1
oacdty=01 mxl=2000(369) mxlc=123 mal=00 scl=00 pre=00
oacflg=03 fl2=1000010 frm=01 csi=873 siz=0 off=2000
kxsbbbfp=0e7036ac bln=2000 avl=07 flg=01
value="varchar"
More Questions
Normally the relationship between mxl and mxlc makes sense. For a NVARCHAR, UTF16 on my system, there will be 2 bytes per character, thus 198 and 99. My database is UTF8, a character could take up to 4 bytes. Maybe Oracle guesses the average size will be 3 bytes, thus 123 and 369. Obviously it could be more than 369, perhaps that's just the initial memory allocated, and it can grow later?
But your numbers, 36 and 35, don't make sense to me. Surely the number of bytes can never be LESS than the number of characters? Is Oracle making a bad guess, or is some client program sending in bad data?

What are the numbers on the end of lines in a stacktrace

What do the the numbers after the "+" at the end of the lines in a stack trace represent?
Function Source
ntdll!KiFastSystemCallRet
ntdll!ZwRemoveIoCompletion+c
kernel32!GetQueuedCompletionStatus+29
w3tp!THREAD_POOL_DATA::ThreadPoolThread+33
w3tp!THREAD_POOL_DATA::ThreadPoolThread+24
w3tp!THREAD_MANAGER::ThreadManagerThread+39
kernel32!BaseThreadStart+34
here they are
+c
+29
+33
+24
+39 +34
They are offsets, in hexadecimal, from the start of the named subroutine. For example
kernel32!BaseThreadStart+34
is 52 (34 hex) bytes into the routine BaseThreadStart in the kernel32 module.
Offset inside the function. Eg. on frame 3 the return address is: the address of the kernel32!GetQueuedCompletionStatus symbol + 29 bytes.

Resources