Parallel table - driven testing in go fails miserably - go

Question

I have the following test function
func TestIntegrationAppsWithProductionSelf(t *testing.T) {
// here is where the apps array that will act as my test suite is being populated
myapps, err := RetrieveApps(fs)
for _, v := range apps {
v := v
t.Run("", func(t *testing.T) {
t.Parallel()
expectedOutput = `=` + v + `
`
cmpOpts.SingleApp = v
t.Logf("\t\tTesting %s\n", v)
buf, err := VarsCmp(output, cmpOpts)
if err != nil {
t.Fatalf("ERROR executing var comparison for %s: %s\n", v, err)
}
assert.Equal(t, expectedOutput, buf.String())
})
}
}
The test fails, despite the fact that when I remove t.Parallel() (even keeping the sub-testing structure) it succeeds.
The failure (happens as said before only when t.Parallel() is incorporated) has to do with the fact that the values to be compared passed to assert are out of sync, i.e. the assert method compares values that it shouldn't)
Why is that?
I also perform this cryptic re-assignment of the test suite variable (v := v) which I do not understand)
edit: Wandering if it was the usage of the assert method from this package, I made the following substitution, nonetheless the end result is the same,
//assert.Equal(t, expectedOutput, buf.String())
if expectedOutput != buf.String() {
t.Errorf("Failed! Expected %s - Actual: %s\n", expectedOutput, buf.String())
}

Answer 1

Let's dissect the case.
First, let's refer to the docs on testing.T.Run:
Run runs f as a subtest of t called name.
It runs f in a separate goroutine <…>
(Emphasis mine.)
So, when you call t.Run("some_name", someFn), that SomeFn is being run by the test suite as if you would manually do something like
go someFn(t)
Next, let's notice that you do not pass a named function into your call to t.Run, but rather you pass it a so-called function literal; let's cite the language spec on them:
Function literals are closures: they may refer to variables defined in a surrounding function. Those variables are then shared between the surrounding function and the function literal, and they survive as long as they are accessible.
In your case, it means when the compiler compiles the body of your function literal, it makes the function "close over" any variable its body mentions, and which is not one of the formal function parameters; in your case, the only function parameter is t *testing.T, hence every other accessed variable is captured by the created closure.
In Go, when a function literal closes over a variable, it does so by retaining a reference to that variable — which is explicitly mentioned in the spec as («Those variables are then shared between the surrounding function and the function literal <…>», again, emphasis mine.)
Now notice that loops in Go reuse iteration variables on each iteration; that is, when you write
for _, v := range apps {
that variable v is created once in the "outer" scope of the loop and then gets reassigned on each iteration of the loop. To recap: the same variable, whose storage is located at some fixed point in memory, gets assigned a new value on each iteration.
Now, since a function literal closes over external variables by keeping references to them — as opposed to copying their values at the "time" of its definition into itself, — without that funky-looking v := v "trick" each function literal created at each call to t.Run in your loop would reference exactly the same iteration variable v of the loop.
The v := v construct declares another variable named v which is local to the loop's body and at the same time assigns it the value of the loop iteration variable v. Since the local v "shadows" loop iterator's v, the function literal declared afterwards would close over that local variable, and hence each function literal created on each iteration will close over a distinct, separate variable v.
Why is this needed, you may ask?
This is needed because of a subtle problem with the interaction of loop iteration variable and goroutines, which is detailed on the Go wiki:
when one does something like
for _, v := range apps {
go func() {
// use v
}()
}
A function literal closing over v is created, and then it's run with the go statement—in parallel both with the goroutine which runs the loop and with all the other goroutines started on the len(apps)-1 other iterations.
These goroutines running our function literals all refer to the same v and so they all have a data race over that variable: the goroutine running the looop writes to it, and the goroutines running function literals read from it—concurrently and without any synchronization.
I hope, by now you should see the puzzle's pieces coming together: in the code
for _, v := range apps {
v := v
t.Run("", func(t *testing.T) {
expectedOutput = `=` + v + `
// ...
the function literal passed to t.Run closes over v, expectedOutput,
cmpOpts.SingleApp (and may be something else),
and then t.Run() makes that function literal run in a separate goroutine, as documented,—producing the classic data race on expectedOutput and cmpOpts.SingleApp, and whatever else which is not v (a fresh variable on each iteration) or t (passed to the call of the function literal).
You might run go test -race -run=TestIntegrationAppsWithProductionSelf ./... to see the engaged race detector crashing your test case's code.

Answer 2

I am going to post what actually worked, but (unless the question is closed) I will accept the answer that actually elaborates on it.
The problem was that the variable used to store the expectedOutput was declared with a declaration inside the TestIntegrationAppsWithProductionSelf function but outside the for loop (this is now reflected in the code snippet of the initial question).
What worked was to remove the var expectedOutput string statement and do within the for loop
for _, v := range apps {
v := v
expectedOutput := `=` + v + `
`
t.Run("", func(t *testing.T) {
t.Parallel()
cmpOpts.SingleApp = v
t.Logf("\t\tTesting %s\n", v)
buf, err := VarsCmp(output, cmpOpts)
if err != nil {
t.Fatalf("ERROR executing var comparison for %s: %s\n", v, err)
}
//assert.Equal(t, expectedOutput, buf.String())
if expectedOutput != buf.String() {
t.Errorf("Failed! Expected %s - Actual: %s\n", expectedOutput, buf.String())
}
})
}