The docs say rand draws iid uniformly distributed values in the half-open unit interval [0,1) which includes zero but is rand ever throwing a zero?
Yes, it is possible for rand() to return zero and rand() does not return one.
Software should be designed to work whether or not iszero(rand()). And it is
good practice to provide an appropriate test case. How often does a computer that is continuously generating rand() yield zero? Using very rough estimates: about every six weeks.
Sometimes it may be more appropriate to sample from (0.0, 1.0], omitting zero
while permitting one. Here is one way to do that.
zero_as_one(x::T) where {T<:AbstractFloat} =
ifelse(iszero(x), one(T), x)
rand1(::Type{T}) where {T<:AbstractFloat} =
zero_as_one(rand(T))
function rand1(::Type{T}, n::I) where {T<:AbstractFloat, I<:Integer}
n < 1 && throw(ArgumentError("n must be >= 1"))
n === one(I) && return rand1(T)
return map(zero_as_one, rand(T, n))
end
Here is a way to sample from (0.0, 1.0), omitting both zero and one.
openrand() = openrand(Float64)
openrand(::Type{T}) where {T<:AbstractFloat} =
let result = rand(T)
while iszero(result)
result = openrand(T)
end
result
end
function openrand(::Type{T}, n::I) where {T<:AbstractFloat, I<:Integer}
n < 1 && throw(ArgumentError("n must be >= 1"))
n === one(I) && return openrand(T)
result = Vector{T}(undef, n)
for i in 1:n
result[i] = openrand(T)
end
return result
end
In addition to Julia, C, C++, Fortran, Perl, Python, Ruby, and Spreadsheets
also use the interval [0.0, 1.0) for uniform random sampling. Matlab and R use (0.0, 1.0). Mathematica uses [0.0, 1.0].
the original thread on discourse
Related
There is a portion of my f90 program that is taking up a significant amount of compute time. I am basically looping through three matrices (of the same size, with dimensions as large as 250-by-250), and trying to make sure values stay bounded within the interval [-1.0, 1.0]. I know that it is best practice to avoid conditionals in loops, but I am having trouble figuring out how to re-write this block of code for optimal performance. Is there a way to "unravel" the loop or use a built-in function of some sort to "vectorize" the conditional statements?
do ind2 = 1, size(u_mat,2)
do ind1 = 1,size(u_mat,1)
! Dot product 1 must be bounded between [-1,1]
if (b1_dotProd(ind1,ind2) .GT. 1.0_dp) then
b1_dotProd(ind1,ind2) = 1.0_dp
else if (b1_dotProd(ind1,ind2) .LT. -1.0_dp) then
b1_dotProd(ind1,ind2) = -1.0_dp
end if
! Dot product 2 must be bounded between [-1,1]
if (b2_dotProd(ind1,ind2) .GT. 1.0_dp) then
b2_dotProd(ind1,ind2) = 1.0_dp
else if (b2_dotProd(ind1,ind2) .LT. -1.0_dp) then
b2_dotProd(ind1,ind2) = -1.0_dp
end if
! Dot product 3 must be bounded between [-1,1]
if (b3_dotProd(ind1,ind2) .GT. 1.0_dp) then
b3_dotProd(ind1,ind2) = 1.0_dp
else if (b3_dotProd(ind1,ind2) .LT. -1.0_dp) then
b3_dotProd(ind1,ind2) = -1.0_dp
end if
end do
end do
For what it's worth, I am compiling with ifort.
You can use the intrinsic min and max functions for this.
As they are both elemental, you can use them on the whole array, as
b1_dotProd = max(-1.0_dp, min(b1_dotProd, 1.0_dp))
While there are processor instructions which allow min and max to be implemented without branches, it will depend on the compiler implementation of min and max as to whether or not this is actually done and if this is actually any faster, but it is at least a lot more concise.
The answer by #veryreverie is definitely correct, but there
are two things to consider.
A where statement is another sensible choice. Since it still is a conditional choice the same caveat of
whether or not this actually avoids branches and if it's actually any faster, but it is at least a lot more concise
still applies.
One example is:
pure function clamp(X) result(res)
real, intent(in) :: X(:)
real :: res(size(X))
where (X < -1.0)
res = -1.0
else where (X > 1.0)
res = 1.0
else
res = X
end where
end function
If you want to normalize to strictly 1 or -1, I would actually think about changing the datatype to integer. Then you can actually use a == 1 etc. without thinking about floating point equality problems. Depending on your code I would also think about cases where the dot product gets close to zero. Of course this point only applies, if you are only interested in the sign.
pure function get_sign(X) result(res)
real, intent(in) :: X(:)
integer :: res(size(X))
! Or use another appropiate choice to test for near_zero
where (abs(X) < epsilon(X) * 10.)
res = 0
else where (X < 0.0)
res = -1
else where (X > 0.0)
res = +1
end where
end function
What's the best way to generate pseudo-random numbers in the closed interval [0,1] instead of the usual [0,1)? One idea I've came up with is to reject values in (1/2,1), then double the number. I wonder if there is a better method.
real x
do
call random_number(x)
if (x <= 0.5) exit
end do
x = 2*x
print *, x
end
The most important requirement is that the algorithm should not make a worse distribution (in terms of uniformity and correlation) than that generated by random_number(). Also I'd favour simplicity. A wrapper around random_number() would be perfectly good, I'm not looking to implement a whole new generator.
As #francescalus points out in the comments, with the algorithm above lots of numbers in [0,1] will have zero probability of appearing. The following code implements a slightly different approach: the interval is enlarged a bit, then values in excess of 1 are cut out. It should behave better in that aspect.
real x
do
call random_number(x)
x = x*(1 + 1e-6)
if (x <= 1.) exit
end do
print *, x
end
What about swapping x and 1-x? Sorry, my Fortran is rusty
real function RNG()
real :: x
logical, save :: swap = .TRUE.
call random_number(x)
if (swap .EQV. .TRUE.)
RNG = x
swap = .FALSE.
else
RNG = 1.0 - x
swap = .TRUE.
end if
end
And if you want to use Box-Muller, use 1-U everywhere and it should work
z0 = sqrt(-2.0*log(1.0-U1))*sin(TWOPI*U2)
z1 = sqrt(-2.0*log(1.0-U1))*cos(TWOPI*U2)
same for rejection version of Box-Muller
I am trying to compute the IEEE-754 32-bit Floating Point Square Root of various inputs but for one particular input the below algorithm based upon the Newton-Raphson method won't converge, I am wondering what I can do to fix the problem? For the platform I am designing I have a 32-bit floating point adder/subtracter, multiplier, and divider.
For input 0x7F7FFFFF (3.4028234663852886E38)., the algorithm won't converge to the correct answer of 18446743523953729536.000000 This algorithm's answer gives 18446743523953737728.000000.
I am using MATLAB to implement my code before I implement this in hardware. I can only use single precision floating point values, (SO NO DOUBLES).
clc; clear; close all;
% Input
R = typecast(uint32(hex2dec(num2str(dec2hex(((hex2dec('7F7FFFFF'))))))),'single')
% Initial estimate
OneOverRoot2 = single(1/sqrt(2));
Root2 = single(sqrt(2));
% Get low and high bits of input R
hexdata_high = bitand(bitshift(hex2dec(num2hex(single(R))),-16),hex2dec('ffff'));
hexdata_low = bitand(hex2dec(num2hex(single(R))),hex2dec('ffff'));
% Change exponent of input to -1 to get Mantissa
temp = bitand(hexdata_high,hex2dec('807F'));
Expo = bitshift(bitand(hexdata_high,hex2dec('7F80')),-7);
hexdata_high = bitor(temp,hex2dec('3F00'));
b = typecast(uint32(hex2dec(num2str(dec2hex(((bitshift(hexdata_high,16)+ hexdata_low)))))),'single');
% If exponent is odd ...
if (bitand(Expo,1))
% Pretend the mantissa [0.5 ... 1.0) is multiplied by 2 as Expo is odd,
% so it now has the value [1.0 ... 2.0)
% Estimate the sqrt(mantissa) as [1.0 ... sqrt(2))
% IOW: linearly map (0.5 ... 1.0) to (1.0 ... sqrt(2))
Mantissa = (Root2 - 1.0)/(1.0 - 0.5)*(b - 0.5) + 1.0;
else
% The mantissa is in range [0.5 ... 1.0)
% Estimate the sqrt(mantissa) as [1/sqrt(2) ... 1.0)
% IOW: linearly map (0.5 ... 1.0) to (1/sqrt(2) ... 1.0)
Mantissa = (1.0 - OneOverRoot2)/(1.0 - 0.5)*(b - 0.5) + OneOverRoot2;
end
newS = Mantissa*2^(bitshift(Expo-127,-1));
S=newS
% S = (S + R/S)/2 method
for j = 1:6
fprintf('S %u %f %f\n', j, S, (S-sqrt(R)));
S = single((single(S) + single(single(R)/single(S))))/2;
S = single(S);
end
goodaccuracy = (abs((single(S)-single(sqrt(single(R)))))) < 2^-23
difference = (abs((single(S)-single(sqrt(single(R))))))
% Get hexadecimal output
hexdata_high = (bitand(bitshift(hex2dec(num2hex(single(S))),-16),hex2dec('ffff')));
hexdata_low = (bitand(hex2dec(num2hex(single(S))),hex2dec('ffff')));
fprintf('FLOAT: T Input: %e\t\tCorrect: %e\t\tMy answer: %e\n', R, sqrt(R), S);
fprintf('output hex = 0x%04X%04X\n',hexdata_high,hexdata_low);
out = hex2dec(num2hex(single(S)));
I took a whack at this. Here's what I came up with:
float mysqrtf(float f) {
if (f < 0) return 0.0f/0.0f;
if (f == 1.0f / 0.0f) return f;
if (f != f) return f;
// half-ass an initial guess of 1.0.
int expo;
float foo = frexpf(f, &expo);
float s = 1.0;
if (expo & 1) foo *= 2, expo--;
// this is the only case for which what's below fails.
if (foo == 0x0.ffffffp+0) return ldexpf(0x0.ffffffp+0, expo/2);
// do four newton iterations.
for (int i = 0; i < 4; i++) {
float diff = s*s-foo;
diff /= s;
s -= diff/2;
}
// do one last newton iteration, computing s*s-foo exactly.
float scal = s >= 1 ? 4096 : 2048;
float shi = (s + scal) - scal; // high 12 bits of significand
float slo = s - shi; // rest of significand
float diff = shi * shi - foo; // subtraction exact by sterbenz's theorem
diff += 2 * shi * slo; // opposite signs; exact by sterbenz's theorem
diff += slo * slo;
diff /= s; // diff == fma(s, s, -foo) / s.
s -= diff/2;
return ldexpf(s, expo/2);
}
The first thing to analyse is the formula (s*s-foo)/s in floating-point arithmetic. If s is a sufficiently good approximation to sqrt(foo), Sterbenz's theorem tells us that the numerator is within an ulp(foo) of the right answer --- all of that error is approximation error from computing s*s. Then we divide by s; this gives us at worst another half-ulp of approximation error. So, even without a fused multiply-add, diff is within 1.5 ulp of what it should be. And we divide it by two.
Notice that the initial guess doesn't in and of itself matter as long as you follow it up with enough Newton iterations.
Measure the error of an approximation s to sqrt(foo) by abs(s - foo/s). The error of my initial guess of 1 is at most 1. A Newton iteration in exact arithmetic squares the error and divides it by 4. A Newton iteration in floating-point arithmetic --- the kind I do four times --- squares the error, divides it by 4, and kicks in another 0.75 ulp of error. You do this four times and you find you have a relative error at most 0x0.000000C4018384, which is about 0.77 ulp. This means that four Newton iterations yield a faithfully-rounded result.
I do a fifth Newton step to get a correctly-rounded square root. The reason why it works is a little more intricate.
shi holds the "top half" of s while slo holds the "bottom half." The last 12 bits in each significand will be zero. This means, in particular, that shi * shi and shi * slo and slo * slo are exactly representable as floats.
s*s is within two ulps of foo. shi*shi is within 2047 ulps of s*s. Thus shi * shi - foo is within 2049 ulps of zero; in particular, it's exactly representable and less than 2-10.
You can check that you can add 2 * shi * slo and get an exactly-representable result that's within 2-22 of zero and then add slo*slo and get an exactly representable result --- s*s-foo computed exactly.
When you divide by s, you kick in an additional half-ulp of error, which is at most 2-48 here since our error was already so small.
Now we do a Newton step. We've computed the current error correctly to within 2-46. Adding half of it to s gives us the square root to within 3*2-48.
To turn this into a guarantee of correct rounding, we need to prove that there are no floats between 1/2 and 2, other than the one I special-cased, whose square roots are within 3*2-48 of a midpoint between two consecutive floats. You can do some error analysis, get a Diophantine equation, find all of the solutions of that Diophantine equation, find which inputs they correspond to, and work out what the algorithm does on those. (If you do this, there is one "physical" solution and a bunch of "unphysical" solutions. The one real solution is the only thing I special-cased.) There may be a cleaner way, however.
Consider the following algorithm.
function Rand():
return a uniformly random real between 0.0 and 1.0
function Sieve(n):
assert(n >= 2)
for i = 2 to n
X[i] = true
for i = 2 to n
if (X[i])
for j = i+1 to n
if (Rand() < 1/i)
X[j] = false
return X[n]
What is the probability that Sieve(k) returns true as a function of k ?
Let's define a series of random variables recursively:
Let Xk,r denote the indicator variable, taking value 1 iff X[k] == true by the end of the iteration in which the variable i took value r.
In order to have fewer symbols and since it makes more intuitive sense with the code, we'll just write Xk,i which is valid although would have been confusing in the definition since i taking value i is confusing when the first refers to the variable in the loop and the latter to the value of the variable.
Now we note that:
P(Xk,i ~ 0) = P(Xk,i-1 ~ 0) + P(Xk,i-1 ~ 1) * P(Xk-1,i-1 ~ 1) * 1/i
(~ is used in place of = just to make it understandable, since = would otherwise take two separate meanings and looks confusing).
This equality holds by virtue of the fact that either X[k] was false at the end of the i iteration either because it was false at the end of the i-1, or it was true at that point, but in that last iteration X[k-1] was true and so we entered the loop and changed X[k] with probability of 1/i. The events are mutually exclusive, so there is no intersection.
The base of the recursion is simply the fact that P(Xk,1 ~ 1) = 1 and P(X2,i ~ 1) = 1.
Lastly, we note simply that P(X[k] == true) = P(Xk,k-1 ~ 1).
This can be programmed rather easily. Here's a javascript implementation that employs memoisation (you can benchmark if using nested indices is better than string concatenation for the dictionary index, you could also redesign the calculation to maintain the same runtime complexity but not run out of stack size by building bottom-up and not top-down). Naturally the implementation will have a runtime complexity of O(k^2) so it's not practical for arbitrarily large numbers:
function P(k) {
if (k<2 || k!==Math.round(k)) return -1;
var _ = {};
function _P(n,i) {
if(n===2) return 1;
if(i===1) return 1;
var $ = n+'_'+i;
if($ in _) return _[$];
return _[$] = 1-(1-_P(n,i-1) + _P(n,i-1)*_P(n-1,i-1)*1/i);
}
return _P(k,k-1);
}
P(1000); // 0.12274162882390949
More interesting would be how the 1/i probability changes things. I.e. whether or not the probability converges to 0 or to some other value, and if so, how changing the 1/i affects that.
Of course if you ask on mathSE you might get a better answer - this answer is pretty simplistic, I'm sure there is a way to manipulate it to acquire a direct formula.
I'm about to optimize a problem that is defined by n (n>=1, typically n=4) non-negative variables. This is not a n-dimensional problem since the sum of all the variables needs to be 1.
The most straightforward approach would be for each x_i to scan the entire range 0<=x_i<1, and then normalizing all the values to the sum of all the x's. However, this approach introduces redundancy, which is a problem for many optimization algorithms that rely on stochastic sampling of the solution space (genetic algorithm, taboo search and others). Is there any alternative algorithm that can perform this task?
What do I mean by redundancy?
Take two dimensional case as an example. Without the constrains, this would be a two-dimensional problem which would require optimizing two variables. However, due to the requirement that X1 + X2 == 0, one only needs to optimize one variable, since X2 is determined by X1 and vice versa. Had one decided to scan X1 and X2 independently and normalizing them to the sum of 1, then many solution candidates would have been identical vis-a-vis the problem. For example (X1==0.1, X2==0.1) is identical to (X1==0.5, X2==0.5).
If you are dealing with real valued variables then arriving with 2 samples that become identical is quite unlikely. However you do have the problem that your samples would not be uniform. You are much more likely to choose (0.5, 0.5) than (1.0, 0). Oneway of fixing this is subsampling. Basically what you do is that when you are shrinking space along a certain point, you shrink the probability of choosing it.
So basically what you are doing is mapping all the points that are inside the unit cube that satisfy that are in the same direction, map to a single points. These points in the same direction form a line. The longer the line, the larger the probability that you will choose the projected point. Hence you want to bias the probability of choosing a point by the inverse of the length of that line.
Here is the code that can do it(Assuming you are looking for x_is to sum up to 1):
while(true) {
maximum = 0;
norm = 0;
sum = 0;
for (i = 0; i < N; i++) {
x[i] = random(0,1);
maximum = max(x[i], max);
sum += x[i];
norm += x[i] * x[i];
}
norm = sqrt(norm);
length_of_line = norm/maximum;
sample_probability = 1/length_of_line;
if (sum == 0 || random(0,1) > sample_probability) {
continue;
} else {
for (i = 0; i < N; i++) {
x[i] = x[i] /sum;
}
return x;
}
Here is the same function provided earlier by Amit Prakash, translated to python
import numpy as np
def f(N):
while(True):
count += 1
x = np.random.rand(N)
mxm = np.max(x)
theSum = np.sum(x)
nrm = np.sqrt(np.sum(x * x))
length_of_line = nrm / mxm
sample_probability = 1 / length_of_line
if theSum == 0 or rand() > sample_probability:
continue
else:
x = x / theSum
return x