Semgrep: Is it possible to match a function based on its body, instead of its name? - static-analysis
The question is in the title.
As an example, say I am interested in catching all calls of functions that use a certain global variable (foo here).
I tried the following:
rules:
- id: dont-call-functions-using-foo
message: "don't call functions using `foo`"
languages: [ javascript ]
severity: ERROR
patterns:
- pattern: $FUNCTION(...)
- patterns:
- pattern-inside: |
$FUNCTION(...) {
...
}
- pattern: bar
On the following JavaScript file:
const foo = "GLOBAL";
function bar() { console.log(foo); }
function baz() { console.log("hello"); }
// I would like this to be reported
bar()
// ... but not this
baz()
Sadly, this does not work:
$ semgrep -c test.yaml test.js
Running 1 rules...
ran 1 rules on 1 files: 0 findings
I think I "understand" why my rule is not correct:
It looks for a pattern that is both a function definition and a function call; And it does not "correlate" two distinct code locations.
I came to believe that what I am trying to achieve is impossible without the join mode, but I am quite a beginner with semgrep, and would be happy to be proven wrong.
Related
Unable to Call Function in Go debugger
I am following the "Little Go Book" by Karl Seguin, in order to learn Go.
My working environment is Visual Studio Code.
Upon debugging, when I try to call a function from the debug console, i get the following error:
"function calls not allowed without using 'call'", if I try using "call fib(10)", i get "Unable to eval expression: "1:6: expected 'EOF', found fib".
This is the function I am trying to evaluate:
//Fibonnaci
func fib(n int) int64 {
if n == 0 {
return 0
} else if n == 1 {
return 1
} else {
return fib(n-1) + fib(n-2)
}
}
If i try to call the function from the code itself ( from the main() for instance, it works perfectly).
However, if I set a breakpoint and try to call the same function from the debugger console, I get the below error:
Eval error: function calls not allowed without using 'call'
call fib(10)
Unable to eval expression: "1:6: expected 'EOF', found fib"
Failed to eval expression: {
"Expr": "call fib(10)",
"Scope": {
"goroutineID": 1,
"frame": 0
},
"Cfg": {
"followPointers": true,
"maxVariableRecurse": 1,
"maxStringLen": 64,
"maxArrayValues": 64,
"maxStructFields": -1
}
}
Looks like "Function calls via delve 'call' are not supported" yet github issue in microsoft/vscode-go repo :(
The issue vscode-go issue 100 "debug: support function calls via delve 'call'" just got closed with PR 101 and commit 5a7752c / CL 249377 Delve supports function calls. Even though it is still experimental and can be applied only to a limited set of functions, this is a useful feature, many vscode-go users long for. Unlike other javascript/typescript debuggers, delve treats function calls specially and requires different call paths than usual expression evaluation. That is because Go is a compiled, runtime-managed GC language, calling a function safely from debugger is complex. DAP and VS Code UI does not distinguish function calls and other expression evaluation either, so we have to implement this in the same evaluateRequest context. We use a heuristic to guess which route (call or expression evaluation) we need to take based on evaluateRequest's request. This is part of the 0.17.0 milestone, yet to be released, and available for now in the nightly build.
Unreachable command line option in MAIN
zef search includes :$update as a named argument:
multi MAIN('search', Int :$wrap = False, :$update, *#terms ($, *#))
However, it's not recognized as such:
% zef search --update
Usage:
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [--wrap=<Int>] search [<terms> ...] -- Get a list of possible distribution candidates for the given terms
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [--version] -- Detailed version information
/home/jmerelo/.rakudobrew/bin/../moar-2019.03.1/install/share/perl6/site/bin/zef [-h|--help]
What am I missing here? How could I assign a value to this $update?
If I run this MAIN candidate by itself, it works:
$ perl6 -e 'multi MAIN("search", Int :$wrap = False, :$update, *#terms ($, *#)) { say "foo" }' search --update
foo
so it looks to me there is more than one candidate that matches, and that causes the USAGE feedback message to appear.
Named parameters are used as tie-breakers only, unless they are made mandatory (which makes them effectively a part of the dispatch process). So maybe the fix is to make two candidates:
multi MAIN('search', Int :$wrap = False, :$update!, *#terms ($, *#)) { ... }
multi MAIN('search', Int :$wrap = False, *#terms ($, *#)) { ... }
?
YML syntax error
I want to have a multi-line bit of markdown java in a yam file. I tried many things but I guess I don't quite get the quoting rules of Yaml.
{
title: Museum,
body: |
"```java
code code code
java2",
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
}
Produces:
/Users/pitosalas/.rbenv/versions/2.3.1/lib/ruby/2.3.0/psych.rb:377:in `parse': (generator/test.yml): found character that cannot start any token while scanning for the next token at line 3 column 9 (Psych::SyntaxError)
YAML has two styles: the JSON like flow style and the much better human readable block style.
Roughly speaking you can have nested structures each style nested within itself and can have flow style nested within block style, but block style nested within flow style is not allowed.
Your to level { and } are flow style but you try to introduce, with |, a literal block style scalar within that flow style. Replace the flow style with block style upwards from that scalar:
title: Museum
body: |
"```java
code code code
java2"
answers: [
"`museum`",
"`museum.getFloor(3)`",
"`museum.getFloor(3).getExhibit(5)`",
"`museum.getFloor(3).getExhibit(5).getCurator()`",
"`museum.getFloor(3).getExhibit(5).getCurator().name`",
"`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
]
and your YAML is fine. Note that the double quotes "around" the value for the key body are not going to be stripped when loading, maybe that is not what you intended.
You should IMO not leave out the trailing , after the last value in the (flow style) sequence that is the value for answers. This will certainly lead to errors when you extend the list and forget to put in the trailing comma on the line above.
I would personally go for block style all the way:
title: Museum
body: |
"```java
code code code
java2"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
When dealing with YAML file generation that is convoluted or complex, or when it's not working as I expect, I revert to letting Ruby show me the way:
require 'yaml'
body = <<EOT
"```java
code code code
java2
"
EOT
answers = %w(
`museum`
`museum.getFloor(3)`
`museum.getFloor(3).getExhibit(5)`
`museum.getFloor(3).getExhibit(5).getCurator()`
`museum.getFloor(3).getExhibit(5).getCurator().name`
`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`
)
obj = {
"title" => "Museum",
"body" => body,
"answers" => answers
}
puts obj.to_yaml
Which, in this case, outputs:
---
title: Museum
body: |
"```java
code code code
java2
"
answers:
- "`museum`"
- "`museum.getFloor(3)`"
- "`museum.getFloor(3).getExhibit(5)`"
- "`museum.getFloor(3).getExhibit(5).getCurator()`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name`"
- "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"
If you then pass that YAML back into the parser, you should get the original data structure back:
YAML.load(obj.to_yaml)
# => {"title"=>"Museum",
# "body"=>"\"```java\n" +
# "code code code\n" +
# "java2\n" +
# "\"\n",
# "answers"=>
# ["`museum`",
# "`museum.getFloor(3)`",
# "`museum.getFloor(3).getExhibit(5)`",
# "`museum.getFloor(3).getExhibit(5).getCurator()`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name`",
# "`museum.getFloor(3).getExhibit(5).getCurator().name.toUpper()`"]}
How can I run a sub function in Bash
How can I run a 'sub function' from a script in command line? Example:
#script_1.sh
main_function() {
sub_function() {
echo "hello world"
}
}
I tried to source this file and call the function from another script:
#script_2.sh
source script_1.sh
sub_function
But I get
script_2.sh: line 3: sub_function: command not found
while I expected to just get hello world.
Thus defined the sub_function will be defined after function is called.
So:
#script_1.sh
function() {
sub_function() {
#cmd
}
}
#script_2.sh
source script_1.sh
function
sub_function
... should work ... except you should rename function, as it's a reserved word
The step missing in your question is to invoke function first - its action is to define sub_function. Note that sub_function doesn't 'belong' to function in any way - its definition is just a side-effect of running function. P.S. I assume you aren't really trying to call it function - that's a reserved word in bash.
Forcing a package's function to use user-provided function
I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions Namespaces are sealed once they are loaded. Sealing means that imports and exports cannot be changed and that internal variable bindings cannot be changed. Sealing allows a simpler implementation strategy for the namespace mechanism. Sealing also allows code analysis and compilation tools to accurately identify the definition corresponding to a global variable reference in a function body. The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path.