Related
I have unsigned 64bit number, representing mantissa, or fraction (which represent range from [0..1), where 0.0 maps to 0 and 0xffffff.. maps to a number "just before 1.0")
Now i want to split this range into equal buckets - and to answer - given random number key, to which part of the range it will fall to?
Its easier to get from following code:
func BucketIndex(key, buckets uint64) uint64 {
return uint64(float64(key) / ((math.Pow(2, 64) / float64(buckets)))
}
My attempt to "hack this over" - was to split 2^64 to two, like if I will reduce range to 32bit, and operate in 64bit in order to conduct math:
// ~=key / ((1 << 64) / buckets)
return ((key >> 32) * buckets) >> 32
but ranges stopped to be equal..
eg one third (buckets==3) will be at 0x5555555600000000, instead of being at 0x5555555555555556
thats sad story, so im asking do you know of a better methods of finding (1 << 64) / buckets?
If buckets is (compile-time) constant, you may use constant expression to calculate bucket size: constants are of arbitrary size. Else you may use big.Int to calculate it at runtime, and store the result (so you don't have to use big.Int calculations all the time).
Using a constant expression, at compile-time
To achieve an integer division rounding up, add divisor - 1 to the dividend:
const (
max = math.MaxUint64 + 1
buckets = 3
bucketSize = uint64((max + buckets - 1) / buckets)
)
Using big.Int, at runtime
We can use the above same logic with big.Int too. An alternative would be to use Int.DivMod() (instead of adding buckets -1), and if mod is greater than zero, increment the result by 1.
func calcBucketSize(max, buckets *big.Int) uint64 {
max = max.Add(max, buckets)
max = max.Add(max, big.NewInt(-1))
return max.Div(max, buckets).Uint64()
}
var bucketSize = calcBucketSize(new(big.Int).SetUint64(math.MaxUint64), big.NewInt(3))
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 10 months ago.
Improve this question
I am trying concurrency in Go with simple problem example which is the nth prime palindrome numbers, eg: 1st to 9th palindrome sequence is 2, 3, 5, 7, 11, 101, 131, 151. I am stuck and have no idea what to do with it.
My current code is like this:
n := 9999999
count := 0
primePalindrome := 0
for i := 0; count < n; i++ {
go PrimeNumber(i)
go PalindromeNumber(i)
if PrimeNumber(i) && PalindromeNumber(i) {
primePalindrome = i
count++
}
}
How do I know that function PrimeNumber(i) and PalindromeNumber(i) already executed and done by go routine so that I can give IF CONDITION to get the number?
Here's my solution: https://go.dev/play/p/KShMctUK9yg
The question is, what if I want to find the 9999999th palindrome with faster runtime using Go concurrency, how to apply concurrency in PrimeNumber() and PalindromeNumber() function?
Similar to Torek, I add concurrency by creating multiple workers that can each check a number, thereby checking several numbers in parallel.
Algorithmically, the program works like this. One goroutine generates possible prime palindromes. A pool of multiple goroutines all check candidates. The main goroutine collects results. When we have at least enough results to provide the nth prime palindrome, we sort the list and then return the answer.
Look closely at the use of channels and a wait group to communicate between the goroutines:
multiple worker goroutines. Their job is to run palindrome and then prime checks for candidate numbers. They listen on the candidates channel; when the channel is closed, they end, communicating that they're done to the wg sync.WaitGroup.
a single candidate generator. This goroutine sends each candidate to one of the workers by sending to the candidates channel. If it finds the done channel to be closed, it ends.
the main goroutine also functions as the collector of results. It reads from the resc (results) channel and adds the results to a list. When the list is at least the required length, it closes done, signaling the generator to stop generating candidates.
The done channel may seem redundant, but it's important because the main results collector knows when we are done generating candidates, but isn't the one sending to candidates. IF we closed candidates in main, there's a good chance the generator would attempt to write to it and that would crash the program. The generator is the one writing candidates; it is the only goroutine that "knows" no more candidates will be written.
Note that this implementation generates at least n prime palindromes. Since it generates the prime palindromes in parallel, there's no guarantee that we have them in order. We might generate up to prime palindrome n+m where m is the number of workers minus one, in the worst case.
There's a lot of room for improvement here. I'm pretty sure the generator and collector roles could be combined in one select loop on candidate and result channels. The program also seems to have a very hard time if n is as big as 9999999 when I run it on my windows machine - see if your results vary.
Edit: Performance enhancements
If you're looking to improve performance, here's a few things I found and noticed last night.
No need to check even numbers. for i := start; ; i += 1 + (i % 2) skips to the next odd number, then adds 2 every other time to keep on the odd numbers.
all palindromes of even numbered decimal length are divisble by 11. So whole sets of numbers of even length can be skipped. I do this by adding math.Pow10(len(str)) to even numbered decimal representations to add another digit. This is what caused the program to stop outputting numbers for large amounts of time - every even numbered set of numbers cannot produce prime palindromes.
if the number's decimal notation starts with an even number it can't be a prime palindrome unless it's only one digit long. Same is true of 5. In the code below I add math.Pow10(len(str)-1) to the number to move to the next odd numbered sequence. If the number starts with 5, I double that to move to the next odd numbered sequence.
These tricks make the code a lot more performant, but it's still a brute force at the end of the day and I still haven't gotten even close to 9999999.
// send candidates
go func() {
// send candidates out
for i := start; ; i += 1 + (i % 2) {
str := strconv.FormatInt(i, 10)
if len(str) % 2 == 0 && i != 11 {
newi := int64(math.Pow10(len(str)))+1
log.Printf("%d => %d", i, newi)
i = newi
continue
}
if first := (str[0]-'0'); first % 2 == 0 || first == 5 {
if i < 10 {
continue
}
nexti := int64(math.Pow10(len(str)-1))
if first == 5 {
nexti *= 2
}
newi := i + nexti
log.Printf("%d -> %d", i, newi)
i = newi-2
continue
}
select {
case _, ok := <-done:
if !ok {
close(candidates)
return
}
case candidates <- i:
continue
}
}
}()
There are multiple issues to solve here:
we want to spin off "is prime" and "is palindrome" tests
we want to sequentially order the numbers that pass the tests
and of course, we have to express the "spin off" and "wait for result" parts of the problem in our programming language (in this case, Go).
(Besides this, we might want to optimize our primality testing, perhaps with a Sieve of Eratothenese algorithm or similar, which may also involve parallelism.)
The middle problem here is perhaps the hardest one. There is a fairly obvious way to do it, though: we can observe that if we assign an ascending-order number to each number tested, the answers that come back (n is/isnot suitable), even if they come back in the wrong order, are easily re-shuffled into order.
Since your overall loop increments by 1 (which is kind of a mistake1), the numbers tested are themselves suitable for this purpose. So we should create a Go channel whose type is a pair of results: Here is the number I tested, and here is my answer:
type result struct {
tested int // the number tested
passed bool // pass/fail result
}
testC := make(chan int)
resultC := make(chan result)
Next, we'll use a typical "pool of workers". Then we run our loop of things-to-test. Here is your existing loop:
for i := 0; count < n; i++ {
go PrimeNumber(i)
go PalindromeNumber(i)
if PrimeNumber(i) && PalindromeNumber(i) {
primePalindrome = i
count++
}
}
We'll restructure this as:
count := 0
busy := 0
results := []result{}
for toTest := 0;; toTest += 2 {
// if all workers are busy, wait for one result
if busy >= numWorkers {
result := <-resultC // get one result
busy--
results := addResult(results, result)
if result.passed {
count++ // passed: increment count
if count >= n {
break
}
}
}
// still working, so test this number
testC <- toTest
busy++
}
close(testC) // tell workers to stop working
// collect remaining results
for result := range resultC {
results := addResult(results, result)
}
(The "busy" test is a bit klunky; you could use a select to send or receive, whichever you can do first, instead, but if you do that, the optimizations outlined below get a little more complicated.)
This does mean our standard worker pool pattern needs to close the result channel resultC, which means we'll add a sync.WaitGroup when we spawn off the numWorkers workers:
var wg sync.WaitGroup
wg.Add(numWorkers)
for i := 0; i < numWorkers; i++ {
go worker(&wg, testC, resultC)
}
go func() {
wg.Wait()
close(resultC)
}()
This makes our for result := range resultC loop work; the workers all stop (and return and call wg.Done() via their defers, which are not shown here) when we close testC so resultC is closed once the last worker exits.
Now we have one more problem, which is: the results come back in semi-random order. That's why we have a slice of results. The addResult function needs to expand the slice and insert the result in the proper position, using the value-tested. When the main loop reaches the break statement, the number in toTest is at least the n'th palindromic prime, but it may be great than the n'th. So we need to collect the remaining results and look backwards to see if some earlier number was in fact the n'th.
There are a number of optimizations to make at this point: in particular, if we've tested numbers through k and they're all known to have passed or failed and k + numWorkers < n, we no longer need any of these results (whether they passed or failed) so we can shrink the results slice. Or, if we're interested in building a table of palindromic primes, we can do that, or anything else we might choose. The above is not meant to be the best solution, just a solution.
Note again that we "overshoot": whatever numWorkers is, we may test up to numWorkers-1 values that we didn't need to test at all. That, too, might be optimizable by having each worker quit early (using some sort of quit indicator, whether that's a "done" channel or just a sync/atomic variable) if they're working on a number that's higher than the now-known-to-be-at-least-n'th value.
1We can cut the problem in half by starting with answers pre-loaded with 2, or 1 and 2 if you choose to consider 1 prime—see also http://ncatlab.org/nlab/show/too+simple+to+be+simple. Then we run our loop from 3 upwards by 2 each time, so that we don't even bother testing even numbers, since 2 is the only even prime number.
The answer depends of many aspects of your problem
For instance, if each step is independent, you can use the example below. If the next step depends on the previous you need to find another solution.
For instance: a palindrome number does not depends of the previous number. You can paralelize the palindrome detection. While the prime is a sequential code.
Perhaps check if a palindrome is prime by checking the divisors until the square root of that number. You need to benchmark it
numbers := make(chan int, 1000) // add some buffer
var wg sync.WaitGroup
for … { // number of consumers
wg.Add(1)
go consume(&wg, numbers)
}
for i … {
numbers <- i
}
close(numbers)
wg.Wait()
// here all consumers ended properly
…
func consume(wg *sync.WaitGroup, numbers chan int){
defer wg.Done()
for i := range numbers {
// use i
}
}
Somehow, I happened to look at source code for Go on how it implements Random function when passed a length of array.
Here's the calling code
func randomFormat() string {
formats := []string{
"Hi, %v. Welcome!",
"Great to see you, %v!",
"Hail, %v! Well met!",
}
return formats[rand.Intn(len(formats))]
}
Go Source code: main part
func (r *Rand) Intn(n int) int {
if n <= 0 {
panic("invalid argument to Intn")
}
if n <= 1<<31-1 {
return int(r.Int31n(int32(n)))
}
return int(r.Int63n(int64(n)))
}
Go Source code: reference part - Most of devs have this already on their machines or go repo.
// Int31n returns, as an int32, a non-negative pseudo-random number in [0,n).
// It panics if n <= 0.
func (r *Rand) Int31n(n int32) int32 {
if n <= 0 {
panic("invalid argument to Int31n")
}
if n&(n-1) == 0 { // n is power of two, can mask
return r.Int31() & (n - 1)
}
max := int32((1 << 31) - 1 - (1<<31)%uint32(n))
v := r.Int31()
for v > max {
v = r.Int31()
}
return v % n
}
// It panics if n <= 0.
func (r *Rand) Int63n(n int64) int64 {
if n <= 0 {
panic("invalid argument to Int63n")
}
if n&(n-1) == 0 { // n is power of two, can mask
return r.Int63() & (n - 1)
}
max := int64((1 << 63) - 1 - (1<<63)%uint64(n))
v := r.Int63()
for v > max {
v = r.Int63()
}
return v % n
}
func (r *Rand) Int31() int32 { return int32(r.Int63() >> 32) }
func (r *Rand) Int63() int64 { return r.src.Int63() }
type Source interface {
Int63() int64
Seed(seed int64)
}
I want to understand how the random function works encapsulating all inner functions. I am overwhelmed by the code and if someone has to plan the steps out in plain English what would those be?
For example, I don't get the logic for doing minus 1 in
if n <= 1<<31-1
Then, I don't get any of the head or toe of Int31n function
if n&(n-1) == 0 { // n is power of two, can mask
return r.Int31() & (n - 1)
}
max := int32((1 << 31) - 1 - (1<<31)%uint32(n))
v := r.Int31()
for v > max {
v = r.Int31()
}
return v % n
This is more of a question about algorithms than it is about Go, but there are some Go parts. In any case I'll start with the algorithm issues.
Shrinking the range of a uniform random number generator
Suppose that we have a uniform-distribution random number generator that returns a number between, say, 0 and 7 inclusive. That is, it will, over time, return about the same number of 0s, 1s, 2s, ..., 7s, but with no apparent pattern between them.
Now, if we want a uniformly distributed random number between 0 and 7, this thing is perfect. That's what it returns. We just use it. But what if we want a uniformly distributed random number between 0 and 6 instead?
We could write:
func randMod7() int {
return generate() % 7
}
so that if generate() returns 7 (which it has a 1 out of 8 chance of doing), we convert that value to zero. But then we'll get zero back 2 out of 8 times, instead of 1 out of 8 times. We'll get 1, 2, 3, 4, 5, and 6 back 1 out of 8 times, and zero 2 out of 8 times, on average: once for each actual zero, and once for each 7.
What we need to do, then, is throw away any occurrences of 7:
func randMod7() int {
for {
if i := generate() < 7 {
return i
}
// oops, got 7, try again
}
}
Now, if we had a uniform-random-number-generator named generate() that returned a value between 0 and (say) 11 (12 possible values) and we wanted a value between 0 and 3 (four possible values), we could just use generate() % 4, because the 12 possible results would fall into 3 groups of four with equal probability. If we wanted a value between 0 and 5 inclusive, we could use generate() % 6, because the 12 possible results would fall into two groups of 6 with equal probability. In fact, all we need to do is examine the prime factorization of the range of our uniform number generator to see what moduli work. The factors of 12 are 2, 2, 3; so 2, 3, 4, and 6 all work here. Any other modulus, such as generate() % 10, produce a biased result: 0 and 1 occur 2 out of 12 times, but 2 through 9 occur 1 out of 12 times. (Note: generate() % 12 also works, but is kind of pointless.)
In our particular case, we have two different uniform random number generators available. One, Int31(), produces values between 0 and 0x7fffffff (2147483647 decimal, or 231 - 1, or 1<<31 - 1) inclusive. The other, Int63(), produces values between 0 and 0x7fffffffffffffff (9223372036854775807, or 263 - 1, or 1<<63 - 1). These are ranges that hold 231 and 263 values respectively, and hence their prime factorization is 31 2s, or 63 2s.
What this means is that we can compute Int31() mod 2k, for any integer k in zero to 31 inclusive, without messing up our uniformity. With Int63(), we can do the same with k ranging all the way up to 63.
Introducing the computer
Now, mathematically-and-computer-ly speaking, given any nonnegative integer n in [0..0x7ffffff] or [0..0x7fffffffffffffff], and a non-negative integer k in the right range (no more than 31 or 63 respectively), computing that integer n mod 2k produces the same result as computing that integer and doing a bit-mask operation with k bits set. To get that number of set bits, we want to take 1<<k and subtract 1. If k is, say, 4, we get 1<<4 or 16. Subtracting 1, we get 15, or 0xf, which has four 1 bits in it.
So:
n % (1 << k)
and:
n & (1<<k - 1)
produce the same result. Concretely, when k==4, this is n%16 or n&0xf. When k==5 this is n%32 or n&0x1f. Try it for k==0 and k==63.
Introducing Go-the-language
We're now ready to consider doing all of this in Go. We note that int (plain, unadorned int) is guaranteed to be able to hold values between -2147483648 and +2147483647 (-0x80000000 through +0x7fffffff) respectively. It may extend all the way to -0x8000000000000000 through +0x7ffffffffffffff.
Meanwhile, int32 always handles the smaller range and int64 always handles the larger range. The plain int is a different type from these other two, but implements the same range as one of the two. We just don't know which one.
Our Int31 implementation returns a uniformly distributed random number in the 0..0x7ffffff range. (It does this by returning the upper 32 bits of r.Int63(), though this is an implementation detail.) Our Int63 implementation returns a uniformly distributed random number in the 0..0x7ffffffffffffff range.
The Intn function you show here:
func (r *Rand) Intn(n int) int {
if n <= 0 {
panic("invalid argument to Intn")
}
if n <= 1<<31-1 {
return int(r.Int31n(int32(n)))
}
return int(r.Int63n(int64(n)))
}
just picks one of the two functions, based on the value of n: if it's less than or equal to 0x7fffffff (1<<31 - 1), the result fits in int32, so it uses int32(n) to convert n to int32, calls r.Int31n, and converts the result back to int. Otherwise, the value of n exceeds 0x7fffffff, implying that int has the larger range and we must use the larger-range generator, r.Int63n. The rest is the same except for types.
The code could just do:
return int(r.Int63n(int64(n)))
every time, but on 32-bit machines, where 64-bit arithmetic may be slow, this might be slow. (There's a lot of may and might here and if you were writing this yourself today, you should start by profiling / benchmarking the code. The Go authors did do this, though this was many years ago; at that time it was worth doing this fancy stuff.)
More bit-manipulation
The insides of both functions Int31n and Int63n are quite similar; the main difference is the types involved, and then in a few places, the maximum values. Again, the reason for this is at least partly historical: on some (mostly old now) computers, the Int63n variant is significantly slower than the Int32n variant. (In some non-Go language, we might write these as generics and then have the compiler generate a type-specific version automatically.) So let's just look at the Int63 variant:
func (r *Rand) Int63n(n int64) int64 {
if n <= 0 {
panic("invalid argument to Int63n")
}
if n&(n-1) == 0 { // n is power of two, can mask
return r.Int63() & (n - 1)
}
max := int64((1 << 63) - 1 - (1<<63)%uint64(n))
v := r.Int63()
for v > max {
v = r.Int63()
}
return v % n
}
The argument n has type int64, so that its value will not exceed 263-1 or 0x7fffffffffffffff or 9223372036854775807. But it could be negative, and negative values won't work right, so the first thing we do is test for that and panic if so. We also panic if the input is zero (this is something of a choice, but it's useful to note it now).
Next we have the n&(n-1) == 0 test. This is a test for powers of two, with one slight flaw, and it works in many languages (those that have bit-masking):
A power of two is always represented as a single set bit, in the binary representation of a number. For instance, 2 itself is 000000012, 4 is 000000102, 8 is 000001002, and so on, through 128 being 100000002. (Since I only "drew" eight bits this series maxes out at 128.)
Subtracting 1 from that number causes a borrow: that bit goes to zero, and all the lesser bits become 1. For instance, 100000002 - 1 is 011111112.
AND-ing these two together produces zero if there was just the single bit set initially. If not—for instance, if we have the value 130 or 100000102 initially, subtracting 1 produces 100000012—there's no borrow out of the top bit, so the top bit is set in both inputs and therefore is set in the AND-ed result.
The slight flaw is that if the initial value is zero, then we have 0-1, which produces all-1s; 0&0xffffffffffffffff is zero too, but zero is not an integer power of two. (20 is 1, not 0.) This minor flaw is not important for our purpose here, because we already made sure to panic for this case: it just doesn't happen.
Now we have the most complicated line of all:
max := int64((1 << 63) - 1 - (1<<63)%uint64(n))
The recurring 63s here are because we have a value range going from zero to 263-1. 1<<63 - 1 is (still, again, always) 9223372036854775807 or 0x7fffffffffffffff. Meanwhile, 1<<63, without 1 subtracted from it, is 9223372036854775808 or 0x8000000000000000. This value does not fit into int64 but it does fit into uint64. So if we turn n into a uint64, we can compute uint64(9223372036854775808) % uint64(n), which is what the % expression does. By using uint64 for this calculation, we ensure that it doesn't overflow.
But: what is this calculation all about? Well, go back to our example with a generate() that produces values in [0..7]. When we wanted a number in [0..5], we had to discard both 6 and 7. That's what we're going for here: we want to find the value above which we should discard values.
If we were to take 8%6, we'd get 2. 8 is one bigger than the maximum that our 3-bit generate() would generate. 8%6 == 2 is the number of "high values" that we have to discard: 8-2 = 6 and we want to discard values that are 6 or more. Subtract 1 from this, and we get 7-2 = 5; we can accept numbers in this input range, from 0 to 5 inclusive.
So, this somewhat fancy calculation for setting max is just a way to find out what the maximum value we like is. Values that are greater than max need to be tossed out.
This particular calculation works nicely even if n is much less than our generator returns. For instance, suppose we had a four-bit generator, returning values in the [0..15] range, and we wanted a number in [0..2]. Our n is therefore 3 (to indicate that we want a number in [0..2]). We compute 16%3 to get 1. We then take 15 (one less than our maximum output value) - 1 to get 14 as our maximum acceptable value. That is, we would allow numbers in [0..14], but exclude 15.
With a 63-bit generator returning values in [0..9223372036854775807], and n==3, we would set max to 9223372036854775805. That's what we want: it throws out the two biasing values, 9223372036854775806 and 9223372036854775807.
The remainder of the code simply does that:
v := r.Int63()
for v > max {
v = r.Int63()
}
return v % n
We pick one Int63-range number. If it exceeds max, we pick another one and check again, until we pick one that is in the [0..max] range, inclusive of max.
Once we get a number that is in range, we use % n to shrink the range if needed. For instance, if the range is [0..2], we use v % 3. If v is (say) 14, 14%3 is 2. Our actual max is, again, 9223372036854775805, and whatever v is, between 0 and that, v%3 is between 0 and 2 and remains uniformly distributed, with no slight bias to 0 and 1 (9223372036854775806 would give us that one extra 0, and 9223372036854775807 would give us that one extra 1).
(Now repeat the above for int32 and 32 and 1<<32, for the Int31 function.)
I want to create a loop from i := 1 to n which takes a random number from the procedure and assigns its value to the variables named x1,x2,x3,x4...xn
So it could look like this:
for i:=1 to 100 do begin
GenerateRandomNumber(x); // this procedure gives the updated value of x
xi := x;
end
I want to solve it by creating an array[1..n] of string and then assign this random number to the ith element of the array. But to be honest, I don't like this solution. Is there any faster way? I was thinking about concatenating a string with a number, or converting "x" (the string) to a number and then somehow merging it with the consecutive values of i. What do you think?
I want to get a random number in pascal from between a range. Basically something like this:
r = random(100,200);
The above code would then have a random number between 100 and 200.
Any ideas?
The built in pascal function only lets you get a number from between 0-your range, while i need to specify the minimum number to return
Just get a random number with the correct range (ie 100 to 200 would be range 100) then add the starting value to it
So: random(100) + 100 for your example
As already pointed out, you should use
myrandomnumber := random(span) + basenumber;
However, to get better quality random numbers, you should call
randomize();
once, on start of your application, to initialize the random number generator.
Couldn't you just declare a starting variable and an end variable and pass random those? e.g.
var
varMyRandomNumber, x, y := extended;
begin
x := 100;
y := 200;
varMyRandomNumber := random(x,y);
ShowMessage(IntToStr(varMyRandomNumber));
end;
?
There's a good example here of using a for loop to set starting and end values : http://www.freepascal.org/docs-html/rtl/system/random.html
Use RandomRange or RandomFrom:
function RandomRange(const aFrom: Integer; const aTo: Integer): Integer;
RandomFrom returns a random element from the array AValues. The return value has the same type as the type of the array elements.
first of all, i recommend you to use Randomize at the beginning of the program (it changes the algorithm of selecting the number).
To get a random number between some two numbers you need this:
Result:=Min+random(10000)mod max + 1;
I don't remember the maximum value for random, so you can change it (it don't changes anything).
By using 'mod' you get module from division Random and max. +1 is needed, because you never get the number that = max, only the number that =max-1, so you need to write +1.
Good luck!
You can make it like
Int:=Random(100);
it give's 100 random numbers.
then when you display it or use it just add 101 to that integer so its between 100 and 200