I want to write a self checking testbench in VHDL for my design. I'm confused on which values to test. Should I check all possible values and see if they are correct? My design also contains a generic parameter, so I should also test different bit sizes. Testing all possible values becomes impossible for larger bit sizes. For example: how would people test a 64 bit floating point adder? I'm sure that it is physically impossible to test all values then?
Maybe you can start with checking the outcome for some specific inputs.
If you can find out the expected values for random numbers in your testbench, you can use
seeds
depending on the timestamp.
You should have some check-function which compares the output with the expected value.
I hope my suggestions are helpful.
Related
I'm looking to use a specific set of seeds for the intrinsic function RANDOM_NUMBER (a PRNG). What I've read so far is that the seed value can be set via calling RANDOM_SEED, specifically RANDOM_SEED(GET = array). My confusion is how to (if it's possible) set a specific value for the algorithm, for instance in the RAND, RANDU, or RANDM algorithms one can specify their own seed directly. I'm not sure how to set the seed, as the get function seems to take an array. If it takes an array, does it always pull the seed value from a specific index in the array?
Basically, is there a way to set a specific single seed value? If so, would someone be able to write-it out?
As a side note - I'm attempting to set my seed because allegedly one of the other PRNGs I mentioned only works well with "large odd numbers" according to my professor, so I decided that I may as well control this when comparing the PRNG's.
First, the RANDOM_SEED is only used for controlimg the seed of RANDOM_NUMBER(). If you use any other random number generator subroutine or function, it will not affect them at all or if yes then in some compiler specific way which you must find in the manual. But most probably it does not affect them.
Second, you should not care at all whether the seed array contains 1, 4 or 42 integers, it doesn't matter because the PRNG uses the bits from the whole array in some unspecified custom way. For example you may be generating 64 bit real numbers, but the seed array is made of 32 bit integers. You cannot simply say which integer from the seed array does what. You can view the whole seed array as one big integer number that is cut into smaller pieces if you want.
Regarding your professors advice, who knows what he meant, but the seed is probably set by some procedure of that particular generator, and not by the standard RANDOM_SEED, so you must read the documentation for that generator.
And how to use a specific seed in RANDOM_SEED? It was described on this site several times,jkust search for RANDOM_SEED in the top right search field, really. But it is simple, once you know the size of the array, size it to any non-trivial numbers (you need enough non-zero bits) you want and use put=. That's really all, just don't think about individual values in the array, the whole array is one piece of data together.
I have seen two different kinds of CRC algorithms. The one kind is called "direct" the other kind is called "non-direct" or "indirect". The code for both is a bit different. Both are able to calculate the same checksum if direct type is supplied with a converted initial value.
I can successfully run both algorithms and I know how to convert the initial value. So this is no problem.
What I couldn't find out: Why do these two algorithms exist? Is there something that one can do what the other can't? Are they redundant from the user's point of view?
UPDATE You can find a testable online implementation (and C implementations of both aglorithms) here. However these terms (or one of them) are mentioned in some more places. Like here ("direct table algorithm"), in a microcontroller reference document, in forums etc.
The "direct" is referring to how to avoid processing n zero bits at the end for an n-bit CRC.
The mathematical definition of the CRC is a division of the message with n zero bits appended to it. You can avoid the extra operations by exclusive-oring the message with the CRC before operating on it instead of after. This requires processing the initial value of the register in the normal version through the CRC, and having that be the new initial value.
Since it is not necessary, you will never see a real-world CRC algorithm doing the extra operations.
See the section "10. A Slightly Mangled Table-Driven Implementation" in the document you link for a more detailed explanation.
I am confused between these three functions and I was wondering for some explanation. If I set the range how do I make the range exclusive or inclusive? Are the ranges inclusive or exclusive if I don't specify the range?
In addition to the answer from #dave_59, there are other important differences:
i) $random returns a signed 32-bit integer; $urandom and $urandom_range return unsigned 32-bit integers.
ii) The random number generator for $random is specified in IEEE Std 1800-2012. With the same seed you will get exactly the same sequence of random numbers in any SystemVerilog simulator. That is not the case for $urandom and $urandom_range, where the design of the random number generator is up to the EDA vendor.
iii) Each thread has its own random number generator for $urandom and $urandom_range, whereas there is only one random number generator for $random shared between all threads (ie only one for the entire simulation). This is really important, because having separate random number generators for each thread helps you simulation improve a property called random stability. Suppose you are using a random number generator to generate random stimulus. Suppose you find a bug and fix it. This could easily change the order in which threads (ie initial and always blocks) are executed. If that change changed the order in which random numbers were generated then you would never know whether the bug had gone away because you'd fixed it or because the stimulus has changed. If you have a random number generator for each thread then your testbench is far less vulnerable to such an effect - you can be far more sure that the bug has disappeared because you fixed it. That property is called random stability.
So, As #dave_59 says, you should only be using $urandom and $urandom_range.
You should only be using $urandom and $urandom_range. These two functions provide better quality random numbers and better seed initialization and stability than $random. The range specified by $urandom_range is always inclusive.
Although $random generates the exact same sequence of random numbers for each call, it is extremely difficult to keep the same call ordering as soon as any change is made to the design or testbench. Even more difficult when multiple threads are concurrently generating random numbers.
I have an assignment to create a Test Bench for a N-bit multiplier. The code is odd to me. It is a black box n-bit but for his std_logic_vectors he does not specify any size. Im guessing this is done with the test bench. I have not seem this before and was hoping someone could explain how this works
VHDL supports unconstrained std_logic_vectors. This means that you can design a block that doesn't specify the length of the inputs and outputs. There can be a number of pitfalls to doing this (see this article), but in the case of something like a multiplier it can help with code reuse. You get to define what the input and output widths of the block are by connecting them to a std_logic_vector of the desired width.
Since you indicated that the block is a multiplier, I would check the documentation or interface to see if there are generics associated with the port widths. That is a common way of creating a generic block interface with I/O specific logic inside. That way you can specify how "wide" the logic needs to be, without having to have a separate block for every possible width.
I have some code which delivers things based on weighted random. Things with more weight are more likely to be randomly chosen. Now being a good rubyist I of couse want to cover all this code with tests. And I want to test that things are getting fetched according the correct probabilities.
So how do I test this? Creating tests for something that should be random make it very hard to compare actual vs expected. A few ideas I have, and why they wont work great:
Stub Kernel.rand in my tests to return fixed values. This is cool, but rand() gets called multiple times and I'm not sure I can rig this with enough control to test what I need to.
Fetch a random item a HUGE number of times and compare the actual ratio vs the expected ratio. But unless I can run it an infinite number of times, this will never be perfect and could intermittently fail if I get some bad luck in the RNG.
Use a consistent random seed. This makes the RNG repeatable but it still doesn't give me any verification that item A will happen 80% of the time (for example).
So what kind of approach can I use to write test coverage for random probabilities?
I think you should separate your goals. One is to stub Kernel.rand as you mention. With rspec for example, you can do something like this:
test_values = [1, 2, 3]
Kernel.stub!(:rand).and_return( *test_values )
Note that this stub won't work unless you call rand with Kernel as the receiver. If you just call "rand" then the current "self" will receive the message, and you'll actually get a random number instead of the test_values.
The second goal is to do something like a field test where you actually generate random numbers. You'd then use some kind of tolerance to ensure you get close to the desired percentage. This is never going to be perfect though, and will probably need a human to evaluate the results. But it still is useful to do because you might realize that another random number generator might be better, like reading from /dev/random. Also, it's good to have this kind of test because let's say you decide to migrate to a new kind of platform whose system libraries aren't as good at generating randomness, or there's some bug in a certain version. The test could be a warning sign.
It really depends on your goals. Do you only want to test your weighting algorithm, or also the randomness?
It's best to stub Kernel.rand to return fixed values.
Kernel.rand is not your code. You should assume it works, rather than trying to write tests that test it rather than your code. And using a fixed set of values that you've chosen and explicitly coded in is better than adding a dependency on what rand produces for a specific seed.
If you wanna go down the consistent seed route, look at Kernel#srand:
http://www.ruby-doc.org/core/classes/Kernel.html#M001387
To quote the docs (emphasis added):
Seeds the pseudorandom number
generator to the value of number. If
number is omitted or zero, seeds the
generator using a combination of the
time, the process id, and a sequence
number. (This is also the behavior if
Kernel::rand is called without
previously calling srand, but without
the sequence.) By setting the seed
to a known value, scripts can be made
deterministic during testing. The
previous seed value is returned. Also
see Kernel::rand.
For testing, stub Kernel.rand with the following simple but perfectly reasonable LCPRNG:
##q = 0
def r
##q = 1_103_515_245 * ##q + 12_345 & 0xffff_ffff
(##q >> 2) / 0x3fff_ffff.to_f
end
You might want to skip the division and use the integer result directly if your code is compatible, as all bits of the result would then be repeatable instead of just "most of them". This isolates your test from "improvements" to Kernel.rand and should allow you to test your distribution curve.
My suggestion: Combine #2 and #3. Set a random seed, then run your tests a very large number of times.
I do not like #1, because it means your test is super-tightly coupled to your implementation. If you change how you are using the output of rand(), the test will break, even if the result is correct. The point of a unit test is that you can refactor the method and rely on the test to verify that it still works.
Option #3, by itself, has the same problem as #1. If you change how you use rand(), you will get different results.
Option #2 is the only way to have a true black box solution that does not rely on knowing your internals. If you run it a sufficiently high number of times, the chance of random failure is negligible. (You can dig up a stats teacher to help you calculate "sufficiently high," or you can just pick a really big number.)
But if you're hyper-picky and "negligible" isn't good enough, a combination of #2 and #3 will ensure that once the test starts passing, it will keep passing. Even that negligible risk of failure only crops up when you touch the code under test; as long as you leave the code alone, you are guaranteed that the test will always work correctly.
Pretty often when I need predictable results from something that is derived from a random number I usually want control of the RNG, which means that the easiest is to make it injectable. Although overriding/stubbing rand can be done, Ruby provides a fine way to pass your code a RNG that is seeded with some value:
def compute_random_based_value(input_value, random: Random.new)
# ....
end
and then inject a Random object I make on the spot in the test, with a known seed:
rng = Random.new(782199) # Scientific dice roll
compute_random_based_value(your_input, random: rng)