The minimize function in lmfit allows the specification of a function that is called every iteration with the keyword iter_cb. This function is called every function evaluation (so not every iteration in the least squares process). What I want to do is to call a function every iteration of the least squares process, after the parameters have been updated. For example, with 3 parameters, the residuals are evaluated 4 times to get 3 derivatives (4 function evaluations). After the parameters are updated I need to call a function before the next iteration (again 4 function evaluations). Is that possible?
Well the answer is sort of "no" in that the underlying compiled code (scipy.optimize.leastsq uses MINPACK lmdif) makes no distinctions between calling the objective function to estimate the finite-difference jacobian. In fact, it doesn't really count "iterations" of the estimated solution, but it does count number of function evaluations.
Also, it's not true that it will always do Nvars+1 function evaluations between updates of the values. That is, there will often be a Nvars+1 evaluations, but it seems like it is sometimes Nvars+2 evaluations (or, rather 1 evaluation with rather unique values).
You could check "iter" (the count of the current function evaluation) that is passed into the iteration callback, and return quickly if the value isn't what you think it should be. Or, you could check that the values have changed significantly since the last time the iteration-callback actually ran completely.
Related
I have a Julia program which runs some montecarlo simulations. To do this, I set a seed before calling the function that generates my random values. This function can be called from different parts of my program, and the problem is that I am not generating same random values if I call the function from different parts of my program.
My question is: In order to obtain the same reproducible results, do I have to call this function always from the same part of the program? Maybe this questions is more computer science related than language programming itself.
I thought of seeding inside the function with an increasing index but I still do not get same results.
it is easier to have randomness according to which are your needs if you pass to your function a RNG (random number generator) object as parameter.
For example:
using Random
function foo(a=3;rng=MersenneTwister(123))
return rand(rng,a)
end
Now, you can call the function getting the same results at each function call:
foo() # the same result on any call
However, this is rarely what you want. More often you want to guarantee the replicability of your overall experiment.
In this case, set an RNG at the beginning of your script and call the function with it. This will cause the output of the function to be different at each call, but the same between one run of the whole script and another run.
myGlobalProgramRNG = MersenneTwister() # at the beginning of your script...
foo(rng=myGlobalProgramRNG)
Two further notes: attention to multi-thread. If you want to guarantee replicability in case of multithreading, and in particular independently from the fact of your script running with 1, 2, 4,... threads, look for example how I dealt it in aML library using a generateParallelRngs() function.
The second notice is that even if you create your own RNG at the beginning of your script, or seed the default random seed, different Julia versions may (and indeed they do) change the RNG stream, so the replicability is guaranteed only within the same Julia version.
To overcome this problem you can use a package like StableRNG.jl that guarantee the stability of the RNG stream (at the cost of losing a bit of performance)...
This is a general question about if programming languages remember/store the output of function calls.
Suppose I need to calculate a quantity X which depends on some number of simpler calculations. Let's say
X=sin(t)+cos(t)+(cos(t)-sin(t))^2.
Naively, I could compute X as above, calling sin(t) twice, and cos(t) twice.
Or I could call sin(t) and cos(t) once:
a=sin(t)
b=cos(t)
and do
X=a+b+(b-a)^2
Intuitively, the second method should be twice as fast right? Is this the case in all programming languages?
I ask because, doing such a calculation in Julia, I noticed that computing the simpler quantities once vs calling them at each point they appear in the expression for X does not change the runtime.
It depends on how clever your compiler is, and on properties of the function.
First your compiler would need to figure out for example that you are calling sin(t) twice. That's not too difficult.
Second it needs to convince itself that t has the same value for each call. For example, if t was a static variable, and you didn't call sin(t) but some other function, that function call could modify t, so the second call sin(t) would have a different argument and sin(t) would have to be called twice.
Third it needs to convince itself that it doesn't matter whether sin(t) is called once or twice. (Such a function is called idempotent). For example, if you called a function that writes a message to a logfile, then the compiler would have to call it twice, or only one message is written to the logfile instead of two.
I'm trying to calculate the Kolmogorov-Smirnov statistic in R. I have the following sample, which clearly comes from a random variable that follows a long-tailed distribution.
Download link
https://drive.google.com/file/d/1hIgqikX7p343zdyc-Goq34THUpsZA63n/view?usp=sharing
As you may know, the Kolmogorov-Smirnov statistic requires the calculation of the empirical cumulative distribution function and the presumed cumulative distribution function. For both calculations I take the following approach: first, I create a vector with the same length as the length of the sample, and then I modify each of the components of the vector so as for it to contain the empirical cdf (or presumed cdf) of the corresponding observation of the sample.
For the sake of illustration, I'll show you the code I wrote in order to calculate the empirical cdf.
I'm assuming that the data has been read and stored in a dataframe called data.
ecdf = vector("numeric", length(data$logueos))for (i in 1:length(data$logueos)) {ecdf[i] = sum (data$logueos <= data$logueos[i])/length(data$logueos)}
The code I wrote for the calculation of the presumed cdf is analogous to the preceding one; the only difference is that I set each component of the pcdf vector equal to the formula $P(X<=t)$ —where t is the corresponding observation of the sample— according to the distribution that I'm assuming.
The problem is that this 'for' loop never ends. If I force it to end by clicking RStudio's stop button it works: it makes the vector store what I want it to store. But, if I press Ctrl+Shift+k in order to render my notebook and preview it, the load gets stuck when trying to execute the first chunk encountered that contains one of those loops.
First of all, your loop is not endless. It will finish, eventually.
You start initializing a vector with as much elements as the number of observations (1.245.888, which is a lot of iterations). This vector is FULL OF ZEROS.
What your loop does is iterate while changing each zero with the calculus sum (data$logueos <= data$logueos[i])/length(data$logueos). Check that when you stop the execution, the first values of your vector will be values between 0 and 1 while the last values is going to be 0s (because the loop hasn't arrived there yet).
So, you will have to wait more time.
In order to make the execution faster, you could consider loop parallelization (because standard loops go sequentially, one by one, and if it's too much wait, parallelization makes it faster. For example, executing 4 by 4, depending of your computer capacities). Here you'll find some information about it: https://nceas.github.io/oss-lessons/parallel-computing-in-r/parallel-computing-in-r.html
Then, my proposal to you:
if(!require(foreach)){install.packages("foreach")}; require(foreach)
registerDoParallel(detectCores() - 1)
ecdf = vector("numeric", length(data$logueos))
foreach (i=1:length(data$logueos)) %do% {
print(i)
ecdf[i] = sum (data$logueos <= data$logueos[i])/length(data$logueos)
}
The first line will download and load foreach library, that you
need for parallelization.
detectCores() - 1 is going to use all the
processors that your computer has except one (to avoid freezing your
machine) for computing this loop. You'll see that is going to be
faster!
registerDoParallel function is what tells to foreach how many cores use.
I have an equation of motion function file which I feed into ode45. Necessarily, the output variables of the function file is ydot.
Within my equation of motion function file, I calculate many objects from the state vector, y, to prescribe forces.
After ode45 is finished, I would like access to these objects at every time step so that I can calculate an energy.
Instead of recalculating them over every time step, it would be faster to just pull them from the Runge-Kutta process when they are calculated as intermediate steps anyway.
Is it possible to do this?
There is no guarantee that the ODE function for the right side is even called at the output points as they are usually interpolated from the points computed by the adaptive step size algorithm.
One trick I have often seen but would need to search for references is to have the function return all the values you will need and cut the return list down to the derivative in the ODE45 call. Modulo appropriate syntax
function [ydot, extra] = odefunc(t,y,params)
and then use
sol = ode45(#(t,y): odefunc(t,y,params)(1),...)
and then run odefunc on the points in sol to extract the extra information.
Perhaps that idea of selecting the output only works in python. Then define an explicit wrapper
function ydot = odewrapper(t,y)
[ydot,~] = odefunc(t,y,params)
end
that you then normally call in ode45.
I've been trying to learn what recursion in programming is, and I need someone to confirm whether I have thruly understood what it is.
The way I'm trying to think about it is through collision-detection between objects.
Let's say we have a function. The function is called when it's certain that a collision has occured, and it's called with a list of objects to determine which object collided, and with what object it collided with. It does this by first confirming whether the first object in the list collided with any of the other objects. If true, the function returns the objects in the list that collided. If false, the function calls itself with a shortened list that excludes the first object, and then repeats the proccess to determine whether it was the next object in the list that collided.
This is a finite recursive function because if the desired conditions aren't met, it calls itself with a shorter and shorter list to until it deductively meets the desired conditions. This is in contrast to a potentially infinite recursive function, where, for example, the list it calls itself with is not shortened, but the order of the list is randomized.
So... is this correct? Or is this just another example of iteration?
Thanks!
Edit: I was fortunate enough to get three VERY good answers by #rici, #Evan and #Jack. They all gave me valuable insight on this, in both technical and practical terms from different perspectives. Thank you!
Any iteration can be expressed recursively. (And, with auxiliary data structures, vice versa, but not so easily.)
I would say that you are thinking iteratively. That's not a bad thing; I don't say it to criticise. Simply, your explanation is of the form "Do this and then do that and continue until you reach the end".
Recursion is a slightly different way of thinking. I have some problem, and it's not obvious how to solve it. But I observe that if I knew the answer to a simpler problem, I could easily solve the problem at hand. And, moreover, there are some very simple problems which I can solve directly.
The recursive solution is based on using a simpler (smaller, fewer, whatever) problem to solve the problem at hand. How do I find out which pairs of objects in a set of objects collide?
If the set has fewer than 2 elements, there are no pairs. That's the simplest problem, and it has an obvious solution: the empty set.
Otherwise, I select some object. All colliding pairs either include this object, or they don't. So that gives me two subproblems.
The set of collisions which don't involve the selected object is obviously the same problem which I started with, but with a smaller set. So I've replaced this part of the problem with a smaller problem. That's one recursion.
But I also need the set of objects which the selected object collides with (which might be an empty set). That's a simpler problem, because now one element of each pair is known. I can solve that problem recursively as well:
I need the set of pairs which include the object X and a set S of objects.
If the set is empty, there are no pairs. Simple.
Otherwise, I choose some element from the set. Then I find all the collisions between X and the rest of the set (a simpler but otherwise identical problem).
If there is a collision between X and the selected element, I add that to the set I just found.
Then I return the set.
Technically speaking, you have the right mindset of how recursion works.
Practically speaking, you would not want to use recursion for an instance such as the one you described above. Reasons being is that every recursive call adds to the stack (which is finite in size), and recursive calls are expensive on the processor, with enough objects you are going to run into some serious bottle-necking on a large application. With enough recursive calls, you would result with a stack overflow, which is exactly what you would get in "infinite recursion". You never want something to infinitely recurse; it goes against the fundamental principal of recursion.
Recursion works on two defining characteristics:
A base case can be defined: It is possible to eventually reach 0 or 1 depending on your necessity
A general case can be defined: The general case is continually called, reducing the problem set until your base case is reached.
Once you have defined both cases, you can define a recursive solution.
The point of recursion is to take a very large and difficult-to-solve problem and continually break it down until it's easy to work with.
Once our base case is reached, the methods "recurse-out". This means they bounce backwards, back into the function that called it, bringing all the data from the functions below it!
It is at this point that our operations actually occur.
Once the original function is reached, we have our final result.
For example, let's say you want the summation of the first 3 integers. The first recursive call is passed the number 3.
public factorial(num) {
//Base case
if (num == 1) {
return 1;
}
//General case
return num + factorial(num-1);
}
Walking through the function calls:
factorial(3); //Initial function call
//Becomes..
factorial(1) + factorial(2) + factorial(3) = returned value
This gives us a result of 6!
Your scenario seems to me like iterative programming, but your function is simply calling itself as a way of continuing its comparisons. That is simply re-tasking your function to be able to call itself with a smaller list.
In my experience, a recursive function has more potential to branch out into multiple 'threads' (so to speak), and is used to process information the same way the hierarchy in a company works for delegation; The boss hands a contract down to the managers, who divide up the work and hand it to their respective staff, the staff get it done, and had it back to the managers, who report back to the boss.
The best example of a recursive function is one that iterates through all files on a file system. ( I will do this in pseudo code because it works in all languages).
function find_all_files (directory_name)
{
- Check the given directory name for sub-directories within it
- for each sub-directory
find_all_files(directory_name + subdirectory_name)
- Check the given directory for files
- Do your processing of the filename; it is located at directory_name + filename
}
You use the function by calling it with a directory path as the parameter. The first thing it does is, for each subdirectory, it generates a value of the actual path to the subdirectory and uses it as a value to call find_all_files() with. As long as there are sub-directories in the given directory, it will keep calling itself.
Now, when the function reaches a directory that contains only files, it is allowed to proceed to the part where it process the files. Once done that, it exits, and returns to the previous instance of itself that is iterating through directories.
It continues to process directories and files until it has completed all iterations and returns to the main program flow where you called the original instance of find_all_files in the first place.
One additional note: Sometimes global variables can be handy with recursive functions. If your function is merely searching for the first occurrence of something, you can set an "exit" variable as a flag to "stop what you are doing now!". You simply add checks for the flag's status during any iterations you have going on inside the function (as in the example, the iteration through all the sub-directories). Then, when the flag is set, your function just exits. Since the flag is global, all generations of the function will exit and return to the main flow.