vim rand() is not deterministic. Is this expected? - random

Question

According to :help rand(),
rand([{expr}])
Return a pseudo-random Number generated with an xoshiro128**
algorithm using seed {expr}. The returned number is 32 bits,
also on 64 bits systems, for consistency.
{expr} can be initialized by srand() and will be updated by
rand(). If {expr} is omitted, an internal seed value is used
and updated.
Examples:
:echo rand()
:let seed = srand()
:echo rand(seed)
:echo rand(seed) % 16 " random number 0 - 15
It doesn't explain how a seed is changed every time rand() is called, but I expected it to be deterministically altered because
C++'s std::rand() does so,
and Wikipedia says
A pseudorandom number generator (PRNG), also known as a deterministic random bit generator (DRBG), is an algorithm...
However, in the code below, the value of a is deterministic but the values of b are not deterministic; they take different values when you restart the script.
let seed = srand(0)
let a = rand(seed) "deterministic
let b = rand() "not deterministic (why?)
echo [a, b]
let seed = [0, 1, 2, 3]
let a = rand(seed) "deterministic
let b = rand() "not deterministic (why?)
echo [a, b]
Is this an expected behavior? I think the behavior contradicts the documentation.
Environments:
~ $ vi --version
VIM - Vi IMproved 8.2 (2019 Dec 12, compiled Apr 30 2020 13:32:36)
Included patches: 1-664

Answer 1

An algorithm used in Vim is fully deterministic. What creates a confusion is the fact that calling rand(seed) updates the seed "in place", but does not update any internal value(s). Therefore any subsequent rand() uses another (more or less random - quality depends on platform) internal seed value. So if you want to produce fully deterministic sequence, you must consequently invoke rand(seed) with the same variable.
This behaviour is easy to deduce from Vim's source code. Also :h rand() says that:
Return a pseudo-random Number generated with an xoshiro128**
algorithm using seed {expr}. The returned number is 32 bits,
also on 64 bits systems, for consistency.
{expr} can be initialized by srand() and will be updated by
rand(). If {expr} is omitted, an internal seed value is used
and updated.
If you find the wording misleading you can open an issue on github.

Answer 2

The documentation is badly written but the behavior is actually the expected one from the source code's perspective.
Analysis
rand() is defined as f_rand() in src/evalfunc.c. From the snippet at the end of this answer, we know some things:
f_rand() has only two sets of static variables: gx, ..., gw and initialized.
gx, ..., gw are the internal seeds. Their values are touched and referenced only when f_rand() is called with no argument (i.e. when argvars[0].v_type == VAR_UNKNOWN).
initialized remembers if f_rand() has ever been called with no argument and it is also touched and referenced only when f_rand() is called with no argument.
When f_rand() is called with a seed,
The value of the seed is used once and that is not saved as a static variable. In other words, the sentence "{expr} can be initialized by srand() and will be updated by rand()" in the documentation is nothing but a "lie"; {expr} is not remembered and thus not updated by the subsequent f_rand().
The value of the seed is updated in place via the pointers lx, ..., lw.
Conclusion
The sentence
{expr} can be initialized by srand() and will be updated by rand()
shall be modified to
{expr} can be initialized by srand() and will be updated by rand({expr}). You may want to store a seed into a variable and pass it to rand() since {expr} is not remembered in the function.
If you need the deterministic rand(), do this:
let seed = srand(0)
let a = rand(seed) "The value of `seed` is changed in place.
let b = rand(seed) "ditto
echo [a, b]
The Source Code of rand()
#define ROTL(x, k) ((x << k) | (x >> (32 - k)))
#define SPLITMIX32(x, z) ( \
z = (x += 0x9e3779b9), \
z = (z ^ (z >> 16)) * 0x85ebca6b, \
z = (z ^ (z >> 13)) * 0xc2b2ae35, \
z ^ (z >> 16) \
)
#define SHUFFLE_XOSHIRO128STARSTAR(x, y, z, w) \
result = ROTL(y * 5, 7) * 9; \
t = y << 9; \
z ^= x; \
w ^= y; \
y ^= z, x ^= w; \
z ^= t; \
w = ROTL(w, 11);
/*
* "rand()" function
*/
static void
f_rand(typval_T *argvars, typval_T *rettv)
{
list_T *l = NULL;
static UINT32_T gx, gy, gz, gw;
static int initialized = FALSE;
listitem_T *lx, *ly, *lz, *lw;
UINT32_T x, y, z, w, t, result;
if (argvars[0].v_type == VAR_UNKNOWN)
{
// When no argument is given use the global seed list.
if (initialized == FALSE)
{
// Initialize the global seed list.
init_srand(&x);
gx = SPLITMIX32(x, z);
gy = SPLITMIX32(x, z);
gz = SPLITMIX32(x, z);
gw = SPLITMIX32(x, z);
initialized = TRUE;
}
SHUFFLE_XOSHIRO128STARSTAR(gx, gy, gz, gw);
}
else if (argvars[0].v_type == VAR_LIST)
{
l = argvars[0].vval.v_list;
if (l == NULL || list_len(l) != 4)
goto theend;
lx = list_find(l, 0L);
ly = list_find(l, 1L);
lz = list_find(l, 2L);
lw = list_find(l, 3L);
if (lx->li_tv.v_type != VAR_NUMBER) goto theend;
if (ly->li_tv.v_type != VAR_NUMBER) goto theend;
if (lz->li_tv.v_type != VAR_NUMBER) goto theend;
if (lw->li_tv.v_type != VAR_NUMBER) goto theend;
x = (UINT32_T)lx->li_tv.vval.v_number;
y = (UINT32_T)ly->li_tv.vval.v_number;
z = (UINT32_T)lz->li_tv.vval.v_number;
w = (UINT32_T)lw->li_tv.vval.v_number;
SHUFFLE_XOSHIRO128STARSTAR(x, y, z, w);
lx->li_tv.vval.v_number = (varnumber_T)x;
ly->li_tv.vval.v_number = (varnumber_T)y;
lz->li_tv.vval.v_number = (varnumber_T)z;
lw->li_tv.vval.v_number = (varnumber_T)w;
}
else
goto theend;
rettv->v_type = VAR_NUMBER;
rettv->vval.v_number = (varnumber_T)result;
return;
theend:
semsg(_(e_invarg2), tv_get_string(&argvars[0]));
rettv->v_type = VAR_NUMBER;
rettv->vval.v_number = -1;
}