I'm getting Checkmarx issue:
The resource Parse allocated by FUNCTION_X in the file FILE at line LINE is prone to resource exhaustion when used by FUNCTION_Y in the file FILE at line LINE.
I can see that data from Parse result is processed in a loop, so I added a length check before the loop, sth like this:
if len(query) > 100 {
return nil, fmt.Errorf("too many query params [%d]", len(query))
}
for k := range query {
// ...
}
But Checkmarx is still complaining. Does anyone know how to fix this?
Thank you.
Having looked into it, it’s not about url.parse, it is about passing user inputs to a loop controlled by len().
Your code is fine, and Checkmarx SAST needs to understand that len() is the size/length function for Go.
You should mark this as Not Exploitable and report it as False Positive to Checkmarx.
Related
All log in this area is not printed (showing inside it's not running). However the last line is execute anyway. I'm so frustrated and sad, totally had no idea. Many thanks if there's any idea on it.
if !reflect.DeepEqual(MachineNow.TCP_machine.Two_D_Dta_Old, twoD_new) {
//the situation should not be executed
log.Println("new to old updated") //all log is not printing (O)
fmt.Println("new", twoD_new[0][0])
fmt.Println("old", MachineNow.TCP_machine.Two_D_Dta_Old[0][0])
MachineNow.TCP_machine.Two_D_Dta_Old = twoD_new //this line is doing anyway (X)
}
Have you checked your “log” output and level?
Maybe you set the output level to be higher than “Println”, so it’s simply ignored.
Try to use debug to know if this runs into this code block or just your assumption.
One more thing to check is: that if you run it by unit test, you need to add the “-v” flag to show output.
Solution: I trasform the object into JSON format and assign to the object. I don't know why but this is the only way to avoid execute that line no matter what. I thought it was just a mistake, now seems that line was indeed executed for unknown reason. it was running windows x64.
new_json, _ := json.Marshal(twoD_new)
_ = json.Unmarshal([]byte(new_json), &MachineNow.TCP_machine.Two_D_Dta_Old)
Suppose we are debugging some Go code, and somewhere in an external dependency we encounter this line:
return json.Marshal(foo)
We want to set a breakpoint and use IntelliJ's "Evaluate Expression" to inspect the JSON being produced. However, this doesn't work:
If we evaluate the expression json.Marshal(foo), we only get to see the byte array.
Evaluating string(json.Marshal(foo)) doesn't work because json.Marshal returns two values, the byte array and an error.
There is no way in Go to access one of the return values directly.
So how can I use "Evaluate Expression" to achieve my goal of just printing the produced JSON string when I'm not able to change the underlying source code?
you can print the returned bytes as a string
bytes, err := json.Marshal(foo)
// check error here
fmt.Println(string(bytes))
update based on comments
You can't change the byte slice in the debugger to a string without changing the source code.
Unfortunately golang error documentation in the standard library is next to non-existent. e.g. just opening a file, the return values for the error are not documented except that you can print a string. But this is not always the right way to handle it.
Is there a way to determine what the real error code might be through trial and error rather than just printing out the text? It seems silly to match the whole text for the specific error.
e.g. given I want to ultimately achieve something like this (assuming it's right)
if fd, err := io.Open("filename"); err != nil {
if err != io.ErrFileNotFound {
log.Fatalf("Error opening file: %s", err)
}
}
As far as I can tell anything that implements the error interface will be able to be used as an error. But determining what the error is is what I'm struggling with. The error may be a struct that has other fields in it like a number field to tell me what type of error it is aside from the text itself.
But how would I know what other data the error contains short of looking through many source files and sometimes tens of function calls.
Does this make sense?
As a more practical example. I am using a yaml library to load a config file.
If the config file doesn't exist I want to carry on (it'll use defaults). But if there is a permissions error I want the error to be treated as fatal. The problem is, it's not entirely clear what the error will look like ahead of time.
Use os.IsNotExist to check for file not found errors:
f, err := os.Open("filename")
if os.IsNotExist(err) {
// handle missing file
} else if err != nil {
// handle other errors
}
The functions os.IsExist, os.IsPermission and is.Timeout check for other common types of errors.
The os, io and other packages declare variables for specific errors.
Always check godoc if you are not cleared about a library.
.below is the godoc io URL and read more
https://godoc.org/io
In my journey with go discovered that there are no stacktraces. so whenever something breaks, all we get an simple string error message without any information where is this is coming from. This is in stark contrast with other languages where I am used to seing detailed stacktraces
For example, below is the error message from apex
$ cat event.json | apex invoke --logs webhook
⨯ error parsing response: json: cannot unmarshal array into Go value of type map[string]interface {}
here its telling me that unmarshal to a map ins't working because event.json is an array. We have unmarshal to interface{} to support both arrays & maps.However, it doesn't tell me which file/line is causing this error.
Questions:
What is way to quickly find which file/line this error coming from?
In General, Are there tips/tricks which gophers use to get to the source of problem quickly from this string error message?
is this how stack traces are for most go projects or there are any best practices that should be followed?
What is way to quickly find which file/line this error coming from?
There are no default stacks printed unless it's an unrecovered panic.
In General, Are there tips/tricks which gophers use to get to the source of problem quickly from this string error message? is this how stack traces are for most go projects or there are any best practices that should be followed?
In General, you need to check error returns from most of the function calls. There are more than one way to do that.
I usually use standard library package log to print out error logs with file and line numbers for easy debugging in simple programs. For example:
package main
import "log"
import "errors"
func init() { log.SetFlags(log.Lshortfile | log.LstdFlags) }
func callFunc() error {
return errors.New("error")
}
func main() {
if err := callFunc(); err != nil {
log.Println(err)
}
}
http://play.golang.org/p/0iytNw7eZ7
output:
2009/11/10 23:00:00 main.go:14: error
Also, there are functions for you to print or retrieve current stacks in standard library runtime/debug, e.g. https://golang.org/pkg/runtime/debug/#PrintStack
There are many community efforts on bringing error handling easier, you can search error in GoDoc: https://godoc.org/?q=error
Your attempted solution: Finding the piece of code that produces the error to fix the code.
Your actual problem: The content of event.json.
This is called the X-Y-Problem
Invoke expects a json object, you are passing a json array. Fix that and your problem is gone!
$ echo -n '{ "value": "Tobi the ferret" }' | apex invoke uppercase
Relevant part of the documentation: Invoking Functions
And that's the piece of code that produces the error: Github
And yes, Go does have stack traces! Read Dave Cheneys blog post on errors and exceptions.
Go does produce stack traces when a panic happens, crashing the program. This will happen if the code calls panic() directly, typically in cases like:
if err != nil {
panic("it broke")
}
or, when a runtime error happens:
a := []int{1, 2, 3}
b := a[12] // index out of range
Here's a minimal example:
package main
func main() {
panic("wtf?!")
}
Output:
panic: wtf?!
goroutine 1 [running]:
panic(0x94e60, 0x1030a040)
/usr/local/go/src/runtime/panic.go:464 +0x700
main.main()
/tmp/sandbox366642315/main.go:4 +0x80
Note the main.go:4 indicating the filename and line number.
In your example, the program did not panic, instead opting to call (I'm guessing) os.Exit(1) or log.Fatal("error message") (which calls os.Exit(1)). Or, the panic was simply recovered from in the calling function. Unfortunately, there's nothing you can do about this if you aren't the author of the code.
I would recommend reading Defer, Panic, and Recover on the Golang blog for more about this.
Setting log.SetFlags(log.LstdFlags | log.Lshortfile) in main() should do it.
For example, log.Printf("Deadline!") would print:
03/11/2020 23:59:59 liberty_test.go:42: Deadline!
I have the following line in a long loop
page = Nokogiri::HTML(open(topic[:url].first)).xpath('//ul[#class = "pages"]//li').first
Sometimes my Ruby application crashes raising the "End of file reached " exception in this line.
How can I resolve this problem? Just a begin;raise;end block?
Is a script that performs a forum backup, so is important that doesn't skip any thread.
Thanks in advance.
In addition to #Phrogz's excellent advice (in particular about at_css with the simpler expression), I would pull the raw xml [content] separately:
page = if (content = open(topic[:url].first)).strip.length > 0
Nokogiri::HTML(content).xpath('//ul[#class = "pages"]//li').first
end
I would suggest that you should first to fix the underlying issue so that you do not get this error.
Does the same URL always cause the problem? (Output it in your log files.) If so, perhaps you need to URI encode the URL.
Is it random, and therefor likely related to a connection hiccup or server problem? If so, you should rescue the specific error and then retry one or more times to get the crucial data.
Secondarily, you should know that the CSS syntax for that query is far simpler:
page = Nokogiri.HTML(...).at_css('ul.pages li')
Not only is this less than half the bytes, it allows for cases like <ul class="foo pages"> that the XPath would miss.
Using at_css (or at_xpath) is the same as .css(...).first, but is faster and simpler.