ANTLR3: how to map the instructions in AST to line number in its java source file - antlr3
How to find the line numbers(of source file) of instructions from AST.
example:
for the following code
24> void foo(){
25> System.out.println(" hi ");
26> }
the ast corresponding to print statement is
METHOD_CALL
.
.
System
out
println
ARGUMENT_LIST
EXPR
" hi "
I want to retrieve the line number of "System" from the generated Tree. The answer for "System" should be 25(line number in the source code).
If your Tree for the System token is in fact a CommonTree, then you can use the CommonTree.getToken() method to get the Token for Symbol. You can then call Token.getLine() to get the line number.
Related
JMeter Beanshell Postprocessor : Reading a file
I am using the below script in a Beanshell Postprocessor
import java.io.*;
File f =new File ("C:\Users\xxxxx\Desktop\testresults.csv");
FileWriter fw=new FileWriter(f,true);
BufferedWriter bw=new BufferedWriter(fw);
var r=prev.getResponseCode();
if (r.equals("200"))
{
bw.write("Test Passed");
}
else
{
bw.write("Test failed");
}
bw.close();
fw.close();
But I am getting the below error
BeanShellInterpreter: Error invoking bsh method: eval Sourced file: inline evaluation of: ``import java.io.*; File f =new File ("C:\Users\xxxxx\Desktop\testresults.csv") . . . '' Token Parsing Error: Lexical error at line 2, column 23. Encountered: "U" (85), after : ""C:\".
What could cause the above error.
You need to escape a backslash with a backslash like: C:\\Users\\xxxxx\\Desktop\\testresults.csv or use a forward slash instead: C:/Users/xxxxx/Desktop/testresults.csv A couple more hints: Since JMeter 3.1 you should be using JSR223 Test Elements and Groovy language for scripting If you run your test with 2 or more concurrent threads they will be writing into the same file resulting in data corruption due to a race condition so maybe it worth considering switching to Flexible File Writer instead
Change to JSR223 Post Processor and write as one line (groovy default)
new File("C:\\Users\\xxxx\\Desktop\\\testresults.csv") << (prev.getResponseCode().equals("200") ? "Test Passed" : "Test failed")
Serialize and deserialize protobufs through CLI?
I am trying to deserialize a file saved as a protobuf through the CLI (seems like the easiest thing to do). I would prefer not to use protoc to compile, import it into a programming language and then read the result. My use case: A TensorFlow lite tool has output some data in a protobuf format. I've found the protobuf message definition in the TensorFlow repo too. I just want to read the output quickly. Specifically, I am getting back a tflite::evaluation::EvaluationStageMetrics message from the inference_diff tool.
I assume that the tool outputs a protobuf message in binary format. protoc can decode the message and output in text format. See this option: --decode=MESSAGE_TYPE Read a binary message of the given type from standard input and write it in text format to standard output. The message type must be defined in PROTO_FILES or their imports.
While Timo Stamms answer was instrumental, I still struggled with the paths to get protoc to work in a complex repo (e.g. TensorFlow).
In the end, this worked for me:
cat inference_diff.txt | \
protoc --proto_path="/Users/ben/butter/repos/tensorflow/" \
--decode tflite.evaluation.EvaluationStageMetrics \
$(pwd)/evaluation_config.proto
Here I pipe the binary contents of the file containing protobuf (inference_diff.txt in my case, generated by following this guide), and specify the fully qualified protobuf message (which I got by combining the package tflite.evaluation; and the message name, EvaluationStageMetrics), the absolute path of the project for the proto_path (which is the project root/ TensorFlow repo), and also the absolute path for the file which actually contains the message. proto_path is just used for resolving imports, where as the PROTO_FILE (in this case, evaluation_config.proto), is used to decode the file.
Example Output
num_runs: 50
process_metrics {
inference_profiler_metrics {
reference_latency {
last_us: 455818
max_us: 577312
min_us: 453121
sum_us: 72573828
avg_us: 483825.52
std_deviation_us: 37940
}
test_latency {
last_us: 59503
max_us: 66746
min_us: 57828
sum_us: 8992747
avg_us: 59951.646666666667
std_deviation_us: 1284
}
output_errors {
max_value: 122.371696
min_value: 83.0335922
avg_value: 100.17548828125
std_deviation: 8.16124535
}
}
}
If you just want to get the numbers in a rush and can't be bothered to fix the paths, you can do
cat inference_diff.txt | protoc --decode_raw
Example output
1: 50
2 {
5 {
1 {
1: 455818
2: 577312
3: 453121
4: 72573828
5: 0x411d87c6147ae148
6: 37940
}
2 {
1: 59503
2: 66746
3: 57828
4: 8992747
5: 0x40ed45f4b17e4b18
6: 1284
}
3 {
1: 0x42f4be4f
2: 0x42a61133
3: 0x40590b3b33333333
4: 0x41029476
}
}
}
How do I find a line of code with pyelftools/libdwarf
I have a function name and an offset from the top of that function. I know I can find the line of code from looking at the assembly listing file and compute the offset for the line of code and get the line number that way. What I'm trying to do is use the .o file to get that same information. I can see the DWARF information for the ELF file and can find the DIE for function in the DWARF data, but how do I actually see the info for the instructions of that function and map that to a line of code. I've been using pyelftools so I would hopefully like to be able to use that but I am open to other options if I can't use pyelftools.
There's a sample in pyelftools that does that: https://github.com/eliben/pyelftools/blob/master/examples/dwarf_decode_address.py Specifically, finding the line for the address goes like this: def decode_file_line(dwarfinfo, address): # Go over all the line programs in the DWARF information, looking for # one that describes the given address. for CU in dwarfinfo.iter_CUs(): # First, look at line programs to find the file/line for the address lineprog = dwarfinfo.line_program_for_CU(CU) prevstate = None for entry in lineprog.get_entries(): # We're interested in those entries where a new state is assigned if entry.state is None: continue if entry.state.end_sequence: # if the line number sequence ends, clear prevstate. prevstate = None continue # Looking for a range of addresses in two consecutive states that # contain the required address. if prevstate and prevstate.address <= address < entry.state.address: filename = lineprog['file_entry'][prevstate.file - 1].name line = prevstate.line return filename, line prevstate = entry.state return None, None
How write BeanShellPostProcessor script for comparing two values
I want compare two values and pick which one is equal to second variable.
I have written code like below in BeanShellPostProcessor
HitID = vars.get("AddPrpc139");
b=139
if(HitID.equals(b))
{
log.info("......value=");
}else
{
log.info("......value=");
}
But i am getting below error
2018-11-27 14:48:53,504 ERROR o.a.j.u.BeanShellInterpreter: Error invoking bsh method: eval In file: inline evaluation of: `` HitID = vars.get("AddPrpc139"); b=139 if(HitID.equals(b)) { log.info("......val . . . '' Encountered "if" at line 4, column 1.
2018-11-27 14:48:53,504 WARN o.a.j.e.BeanShellPostProcessor: Problem in BeanShell script: org.apache.jorphan.util.JMeterException: Error invoking bsh method: eval In file: inline evaluation of: `` HitID = vars.get("AddPrpc139"); b=139 if(HitID.equals(b)) { log.info("......val . . . '' Encountered "if" at line 4, column 1.
Java/Beanshell expect ; at end of the line. Also b can be inlined
HitID = vars.get("AddPrpc139");
if("139".equals(HitIDb))
{
Also consider moving to JSR223 PostProcessor
You need to follow Java syntax rules, to wit you need to add a semicolon after b=139 line.
You should also surround this 139 with quotation marks otherwise JMeter will be comparing a String with an Integer and you will always get into else branch even if the values will be the same
Amended code:
HitID = vars.get("AddPrpc139");
b = "139";
if (HitID.equals(b)) {
log.info("Values are equal, expected: " + b + ", got: " + HitID);
} else {
log.info("Values are NOT equal, expected: " + b + ", got: " + HitID);
}'
Demo:
Be aware that according to JMeter Best Practices you should be using JSR223 PostProcessor with Groovy language starting from JMeter 3.1. Groovy is more modern language, it is compatible with latest Java features and it has much better performance. Check out Apache Groovy - Why and How You Should Use It article for more details.
Forcing a package's function to use user-provided function
I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions Namespaces are sealed once they are loaded. Sealing means that imports and exports cannot be changed and that internal variable bindings cannot be changed. Sealing allows a simpler implementation strategy for the namespace mechanism. Sealing also allows code analysis and compilation tools to accurately identify the definition corresponding to a global variable reference in a function body. The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path.