I use time command to measure execution time of the code A and B. I got results, and now, I want to get ratio between A and B.
Concretely, I have execution time as below.
A : 0m0.819s
B : 0m2.119s
I need x in B = x * A. So, I did the division 2.119 / 0.819, and the result is 2.5873015873. Now, I want to know the significant figure of 2.5873015873. Can I treat 2.5873015873 as the answer or must I decide the significant figure? And if I must decide the significant figure, is it 4 by A and B's execution time?
Related
my function has to work with very big numbers, so in order to do that I used parts in my code such as big() . Unfortunately this resulted in giving me a result that is too precise (in other words its slowing the entire code down).
This is how the result looks like.
ΔE = 0.08298347005140644564908076516066986088852555871299296314640532293721884964540988
If possible I would like to limit the result to 4 digits
ΔE = 0.0829
If performance is a concern, probably the best way to do this is with https://github.com/dzhang314/MultiFloats.jl, e.g.
using MultiFloats
x = Float64x4(2.0)
# Calculations performed on x will have Float64x4 precision subsequently...
MultiFloats.jl appears to be the fastest package around at present for such calculations, and will let you choose from precision levels between Float64x2 and Float64x8. In any event, this will be dramatically faster than the BigFloats used in the example above.
I'm trying to calculate the Kolmogorov-Smirnov statistic in R. I have the following sample, which clearly comes from a random variable that follows a long-tailed distribution.
Download link
https://drive.google.com/file/d/1hIgqikX7p343zdyc-Goq34THUpsZA63n/view?usp=sharing
As you may know, the Kolmogorov-Smirnov statistic requires the calculation of the empirical cumulative distribution function and the presumed cumulative distribution function. For both calculations I take the following approach: first, I create a vector with the same length as the length of the sample, and then I modify each of the components of the vector so as for it to contain the empirical cdf (or presumed cdf) of the corresponding observation of the sample.
For the sake of illustration, I'll show you the code I wrote in order to calculate the empirical cdf.
I'm assuming that the data has been read and stored in a dataframe called data.
ecdf = vector("numeric", length(data$logueos))for (i in 1:length(data$logueos)) {ecdf[i] = sum (data$logueos <= data$logueos[i])/length(data$logueos)}
The code I wrote for the calculation of the presumed cdf is analogous to the preceding one; the only difference is that I set each component of the pcdf vector equal to the formula $P(X<=t)$ —where t is the corresponding observation of the sample— according to the distribution that I'm assuming.
The problem is that this 'for' loop never ends. If I force it to end by clicking RStudio's stop button it works: it makes the vector store what I want it to store. But, if I press Ctrl+Shift+k in order to render my notebook and preview it, the load gets stuck when trying to execute the first chunk encountered that contains one of those loops.
First of all, your loop is not endless. It will finish, eventually.
You start initializing a vector with as much elements as the number of observations (1.245.888, which is a lot of iterations). This vector is FULL OF ZEROS.
What your loop does is iterate while changing each zero with the calculus sum (data$logueos <= data$logueos[i])/length(data$logueos). Check that when you stop the execution, the first values of your vector will be values between 0 and 1 while the last values is going to be 0s (because the loop hasn't arrived there yet).
So, you will have to wait more time.
In order to make the execution faster, you could consider loop parallelization (because standard loops go sequentially, one by one, and if it's too much wait, parallelization makes it faster. For example, executing 4 by 4, depending of your computer capacities). Here you'll find some information about it: https://nceas.github.io/oss-lessons/parallel-computing-in-r/parallel-computing-in-r.html
Then, my proposal to you:
if(!require(foreach)){install.packages("foreach")}; require(foreach)
registerDoParallel(detectCores() - 1)
ecdf = vector("numeric", length(data$logueos))
foreach (i=1:length(data$logueos)) %do% {
print(i)
ecdf[i] = sum (data$logueos <= data$logueos[i])/length(data$logueos)
}
The first line will download and load foreach library, that you
need for parallelization.
detectCores() - 1 is going to use all the
processors that your computer has except one (to avoid freezing your
machine) for computing this loop. You'll see that is going to be
faster!
registerDoParallel function is what tells to foreach how many cores use.
Could this be an example ?
x = randomNumber()
If x < 100 add x to the list and go to step 1.
Print list
Wikipedia says: "An output-sensitive algorithm is an algorithm whose running time depends on the size of the output." So, that's pretty easy:
Choose a random number in some range, say 10-1000.
Output that many dots.
The more dots, the longer it takes for your program to run. To make it more dramatic, you can pause for a second between dots.
Your example works too, since the longer the program runs (when the numbers happen to be small), the larger the output will be.
I am running simulations in Anylogic and I'm trying to calibrate the following distribution:
Jump = normal(coef1, coef2, -1, 1);
However, I keep getting the following message as soon as I start the calibration (experimentation):
Random number generation requires too many iterations (> 10000)
I tried to replace -1 and 1 by other values and keep getting the same thing.
I also tried to change the bounds of coef1 and coef2 and put things like [0,1], but I still get the same error.
I don't get it.
Any ideas?
The four parameter normal method is not deprecated and is not a "calibration where coef1 and coef2 are the coefficicents to be solved for". Where did you get that understanding from? Or are you saying that you're using your AnyLogic Experiment (possibly a multi-run or optimisation experiment) to 'calibrate' that distribution, in which case you need to explain what you mean by 'calibrate' here---what is your desired outcome?
If you look in the API reference (AnyLogic classes and functions --> API Reference --> com.xj.anylogic.engine --> Utilities), you'll see that it's a method to use a truncated normal distribution.
public double normal(double min,
double max,
double shift,
double stretch)
The first 2 parameters are the min and max (where it will sample repeatedly and ignore values outside the [min,max] range); the second two are effectively the mean and standard deviation. So you will get the error you mentioned if min or max means it will sample too many times to get a value in range.
API reference details below:
Generates a sample of truncated Normal distribution. Distribution
normal(1, 0) is stretched by stretch coefficient, then shifted to the
right by shift, after that it is truncated to fit in [min, max]
interval. Truncation is performed by discarding every sample outside
this interval and taking subsequent try. For more details see
normal(double, double)
Parameters:
min - the minimum value that this function will return. The distribution is truncated to return values above this. If the sample
(stretched and shifted) is below this value it will be discarded and
another sample will be drawn. Use -infinity for "No limit".
max - the maximum value that this function will return. The distribution is truncated to return values below this. If the sample
(stretched and shifted) is bigger than this value it will be discarded
and another sample will be drawn. Use +infinity for "No limit".
shift - the shift parameter that indicates how much the (stretched) distribution will shifted to the right = mean value
stretch - the stretch parameter that indicates how much the distribution will be stretched = standard deviation Returns:
the generated sample
According to AnyLogic's documentation, there is no version of normal which takes 4 arguments. Also note that if you specify a mean and standard deviation, the order is unusual (to probabilists/statisticians) by putting the standard deviation before the mean.
I currently have an application which can contain 100s of user defined formulae. Currently, I use reverse polish notation to perform the calculations (pushing values and variables on to a stack, then popping them off the stack and evaluating). What would be the best way to start parallelizing this process? Should I be looking at a functional language?
The calculations are performed on arrays of numbers so for example a simple A+B could actually mean 100s of additions. I'm currently using Delphi, but this is not a requirement going forward. I'll use the tool most suited to the job. Formulae may also be dependent on each other So we may have one formula C=A+B and a second one D=C+A for example.
Let's assume your formulae (equations) are not cyclic, as otherwise you cannot "just" evaluate them. If you have vectorized equations like A = B + C where A, B and C are arrays, let's conceptually split them into equations on the components, so that if the array size is 5, this equation is split into
a1 = b1 + c1
a2 = b2 + c2
...
a5 = b5 + c5
Now assuming this, you have a large set of equations on simple quantities (whether integer, rational or something else).
If you have two equations E and F, let's say that F depends_on E if the right-hand side of F mentions the left-hand side of E, for example
E: a = b + c
F: q = 2*a + y
Now to get towards how to calculate this, you could always use randomized iteration to solve this (this is just an intermediate step in the explanation), following this algorithm:
1 while (there is at least one equation which has not been computed yet)
2 select one such pending equation E so that:
3 for every equation D such that E depends_on D:
4 D has been already computed
5 calculate the left-hand side of E
This process terminates with the correct answer regardless on how you make your selections on line // 2. Now the cool thing is that it also parallelizes easily. You can run it in an arbitrary number of threads! What you need is a concurrency-safe queue which holds those equations whose prerequisites (those the equations depend on) have been computed but which have not been computed themselves yet. Every thread pops out (thread-safely) one equation from this queue at a time, calculates the answer, and then checks if there are now new equations so that all their prerequisites have been computed, and then adds those equations (thread-safely) to the work queue. Done.
Without knowing more, I would suggest taking a SIMD style approach if possible. That is, create threads to compute all formulas for a single data set. Trying to divide the computation of formulas to parallelise them wouldn't yield much speed improvement as the logic required to be able to split up the computations into discrete units suitable for threading would be hard to write and harder to get right, the overhead would cancel out any speed gains. It would also suffer quickly from diminishing returns.
Now, if you've got a set of formulas that are applied to many sets of data then the parallelisation becomes easier and would scale better. Each thread does all computations for one set of data. Create one thread per CPU core and set its affinity to each core. Each thread instantiates one instance of the formula evaluation code. Create a supervisor which loads a single data set and passes it an idle thread. If no threads are idle, wait for the first thread to finish processing its data. When all data sets are processed and all threads have finished, then exit. Using this method, there's no advantage to having more threads than there are cores on the CPU as thread switching is slow and will have a negative effect on overall speed.
If you've only got one data set then it is not a trivial task. It would require parsing the evaluation tree for branches without dependencies on other branches and farming those branches to separate threads running on each core and waiting for the results. You then get problems synchronizing the data and ensuring data coherency.