I hear the term algorithm use often and have been confused by the context I am seeing it in on this site sometimes so I thought I would try and clear up my understanding.
To me an algorithm is some for of mathematical process such as this
uint UPDC16( unsigned char a, uint crc )
{
uint b,p;
a^=crc; crc=(crc>>8)|(a<<8); p=a^(a>>4); p^=(p>>2); b=a; a>>=1;
if( (p^(p>>1))&1 ) { crc^=0x0001; a|=0x80; }
if( b & 1 ) crc^=0x0040; b=a; a^=(crc>>8);
if( a & 1 ) crc^=0x0080; a>>=1;
if( b & 0x80 ) a|=0x80;
crc = (crc&0x00ff)|(a<<8);
return crc;
}
Where as I thought that as this performed an action (rotating image) through nester if statments and not a mathmatical function it was not an algorithm but a function.
for (int block_x = 0; block_x < 2048; block_x+=8)
{
for (int block_y = 0; blocky_y < 2048; block_y+=8)
{
// this is the inner-loop that processes a block
// of 8x8 pixels.
for (int x= 0; x<8; x++)
for (int y=0; y<8; y++)
dest[x+block_x][y+block_y] = src[y+block_y][x+block_x]
}
}
I have googled it but I am looking for an experienced coders explanation. can anyone help explain algorithms to me ?
The other thing that is bothering me is that I have seen the term "script it" several times and do not understand. I have heard there are scripting languages like lua (may be wrong).
Do they mean to used these languages or is a "script" a special method of coding ?
I mostly use c/c++ if this makes any difference.
For your first question : for me an algorithm can be an idea such as "to compute the sum of all the elements of the array you need to.....", a function (there is an input and un output and some steps in between) or a serie of mathematical operation.
So an algorithm would be a serie of steps that allow to go from somewhere to somewhere else (going from your home to your work using the subway is also an algorithm).
For your second question : there are two big types (I'm simplifying) of programming languages, the "compiled" ones and the "interprated" ones and among the latters you have the interactive ones or scripting languages. Also, generally speaking, scripting languages are considered high-levels ones : you can do powerful things in a few lines that together are forming a script.
Of course some scripting language can also be compiled....
Related
What I need
I need an algorithm that produces a bijective output. I have a 31-bit input and need a pseudo-random 31-bit output.
What I have considered
CRCs are bijective within their bit-width.
I have looked on Google and can find the polynomials for this, but not the tables or algorithm.
Could anyone point me in the right direction?
I need a CRC-31 algorithm using polynomial say 0x737e312b, or any bijective function that will do what I need.
NOTE
I found the following code, but I unfortunately don't have the tools to compile and run it.
For any hash function hash, you can do:
function bijectiveHash31(int val) {
val &= 0x7FFFFFFF; //make sure it's 31 bits
for (int i=0; i<5; ++i) {
// the high bits affect the low bits
val ^= hash(val>>15) & 32767;
// rotate bits
val = ((val&32767)<<16) | ((val>>15)&65535);
}
return val;
}
This is a Feistel structure, which forms the basis of many ciphers: https://en.wikipedia.org/wiki/Feistel_cipher
If you need it to be fast and you don't need it to be super good, then this works fine:
function bijectiveHash31(int val) {
val = ((val*RANDOM_ODD_NUMBER) + RANDOM_NUMBER) & 0x7FFFFFFF;
val ^= (val>>15);
val ^= (val>>8);
return val;
}
In both of these cases, it's not too difficult to figure out how you could undo each elementary operation, which shows that the whole hash is bijective. If you need help establishing that for the multiplication, see https://en.wikipedia.org/wiki/Modular_multiplicative_inverse
I am writing a function in RcppEigen for weighted covariances. In one of the steps I want to take column i and column j of a matrix, X, and compute the cwiseProduct, which should return some kind of vector. The output of cwiseProduct will go into an intermediate variable which can be reused many times. From the docs it seems cwiseProduct returns a CwiseBinaryOp, which itself takes two types. My cwiseProduct operates on two column vectors, so I thought the correct return type should be Eigen::CwiseBinaryOp<Eigen::ColXpr, Eigen::ColXpr>, but I get the error no member named ColXpr in namespace Eigen
#include <RcppEigen.h>
// [[Rcpp::depends(RcppEigen)]]
Rcpp::List Crossprod_sparse(Eigen::MappedSparseMatrix<double> X, Eigen::Map<Eigen::MatrixXd> W) {
int K = W.cols();
int p = X.cols();
Rcpp::List crossprods(W.cols());
for (int i = 0; i < p; i++) {
for (int j = i; j < p; j++) {
Eigen::CwiseBinaryOp<Eigen::ColXpr, Eigen::ColXpr> prod = X.col(i).cwiseProduct(X.col(j));
for (int k = 0; k < K; k++) {
//double out = prod.dot(W.col(k));
}
}
}
return crossprods;
}
I have also tried saving into a SparseVector
Eigen::SparseVector<double> prod = X.col(i).cwiseProduct(X.col(j));
as well as computing, but not saving at all
X.col(i).cwiseProduct(X.col(j));
If I don't save the product at all, the functions returns very quickly, hinting that cwiseProduct is not an expensive function. When I save it into a SparseVector, the function is extremely slow, making me think that SparseVector is not the right return type and Eigen is doing extra work to get it into that type.
Recall that Eigen relies on expression templates, so if you don't assign an expression then this expression is essentially a no-op. In your case, assigning it to a SparseVector is the right thing to do. Regarding speed, make sure to compile with compiler optimizations ON (like -O3).
Nonetheless, I believe there is a faster way to write your overall computations. For instance, are you sure that all X.col(i).cwiseProduct(X.col(j)) are non empty? If not, then the second loop should be rewritten to iterate over the sparse set of overlapping columns only. Loops could also be interchanged to leverage efficient matrix products.
I started to learn Halide from last month.
And finally encounterd big problem for me.
I'm trying to implement function like following C-like code in Halide.
for( int y = 0; y < 3; ++y ){
for( int x = 0; x < 3; ++x ){
out(x, y) = out(x-1, y-1) + 1;
}
}
so assuming initial image is below.
0 0 0
0 0 0
0 0 0
output image will be …(0 out of bound)
1 1 1
1 2 2
1 2 3
so I thought two possible solutions.
・Solution1
Define above algorithm like this recursive function.
Func algorithm1(Func input, int n)
{
Func src, clamped, dst;
Var x, y;
if(n == 1){
src = input;
}else{
src = algorithm1(input, n-1);
src.compute_root();
}
clamped = BoundaryConditions::constant_exterior(src, 0, 0, SIZE, 0, SIZE);
dst(x, y) = clamped(x-1, y-1) + 1;
return dst;
}
And use above function like following code.
Func input, output;
input(x, y) = src(x, y);
output = algorithm1(input, SIZE);
output.realize(src);
This implementation barely works. But obviously rebundunt.
Because most of the computation result of the each stage(Func) are not match the final result although each Func computes across over entire image.
And I need to handle more large(normal) images.
So I thought another possible solution.
・Solution2
At first of second solution.
Declare a function defines relationship between one column and another one.
Func algorithm2(Func src)
{
Func clamped, dst;
Var x;
clamped = BoundaryConditions::constant_exterior(src, 0, 0, SIZE);
dst(x) = clamped(x-1) + 1;
return dst;
}
Then, let's combine this.
Func output[3];
output[0](x) = cast<uint32_t>(0);
for(int i = 1; i < SIZE; ++i){
output[i] = algorithm2(output[i-1]);
}
Alright... Here's the problem. How can I combine this array of Funcs as a Func?
Of cource, I can get an Image if I realize this array of Funcs at each func to an pointer of the column's head. But What if I want to pass it to the next Func?
I looked around entire Halide examples(test, apps) these days. But I think there's no similar example.
And you might already noticed my discomfort of English, actually I'm a japanese. So if there are useful example for this problem, I'm so sorry in advance. If so, please tell me where it is. If there's another good implementation idea, please teach me. Anyway I need someone's help!
I appreciate for your reading.
[edit 2]
edit 1 is my foolish question. I can schedule it compute_root().
I've decided to left them on here, really embarrassing though.
I hope this will be helpful to another foolish man.
[edit 1]
I'm appreciate to your fast and detailed response from bottom of my heart!
I'm sorry for late response, I wanted to reply to you after succeeding to implement my algorithm. However, my Halide code still doesn't work what I wanna do and got some things to confirm.
First off, I would like to tell you I realized my misunderstanding of Halide thanks to you. At first of my algorithm's implementation step, I wrote definition using only pure 'Var's.
So I got following error.
All of a functions recursive references to itself must contain the same pure variables in the same places as on the left-hand-side.
I thought this error occured because of scheduling flexibility. If such definition is allowed and schedule it to split, It means that scheduling changes algorithm. This comprehension is correct? From such comprehension, although I already read reduction part of Tutorials and blur example, I misunderstood that I cannot access neighbor pixels in all of Func definitions. I don't know why though.
And reduction domain couldn't be split because of same reason. I think I got it now.
Here's another question to your code. Thanks to your Halide implementation example, I've almost succeeded to implement what I wanna do with no consideration. However, this implementation is desperately slow although I'm handling 20x20 cropped image for ease of debugging.
I'm considering this slowness is caused by reduction domain. In your example, for example when calculating the value g(10, 10), Halide calculation is scheduled from f(0, 0) to f(0, 0) and finally get there value. In the other hand, C implementation just loads the value at g(9, 9) and just increment it though. We can confirm such calculation from printing loop nest.
produce g:
for y:
for x:
produce f:
for y:
for x:
f(...) = ...
for range:
for range:
f(...) = ...
consume f:
g(...) = ...
I would like to confirm that Avoiding this recomputation is impossible? and so you suggested it?
And I would like to ask you another simple question. If there is reverse-dependency like this,
for( int y = 2; y > 0; --y ){
for( int x = 2; x > 0; --x ){
out(x, y) = out(x+1, y+1) + 1;
}
}
Is Halide able to express this code?
The algorithm1 and algorithm2 parts here are not very clear to me. I understand the initial problem statement and the English seems fine so I will endeavor to provide some help answering the question I think you are asking. I'll do this by illustrating a few Halide mechanisms you may not know about or that aren't obvious for use here. Hopefully this will be helpful.
First off, to map a dimension of a Halide Func to different expressions, you pretty much have to use a select statement:
Var x, y, n;
Func f_0, f_1, f_both;
f_0(x, y) = ...;
f_1(x, y) = ...;
f_both(x, y, n) = select(n == 0, f_zero, f_one);
This can be expanded to more cases via adding arguments to the select. This is more useful for piecewise computations than for recursive structures but seems the most direct answer to the question in the title.
The second mechanism is Tuple. This allows a Func to have more than one value, which can be indexed with compile time constants. I don't think this is the answer you are looking for, but i tis convered in tutorial/lesson_13_tuples.cpp .
Finally, Halide supports reductions, which are designed to handle the case in the first code example. This looks like so:
Var x, y;
Func f, g;
RDom range(0, 3, 0, 3); // Form is min/extent, not start/end
f(x, y) = 0; // Initial condition
f(range.x, range.y) = f(range.x - 1, range.y - 1) + 1;
g(x, y) = f(x, y);
Buffer<int32t> result = g.realize(3, 3);
This should produce the output from your first example. Reductions, or "update definitions" are covered in tutorial/lesson_09_update_definitions.cpp .
I'm using GA, so I took example from this page (http://www.ai-junkie.com/ga/intro/gat3.html) and tried to do on my own.
The problem is, it doesn't work. For example, maximum fitness does not always grow in the next generation, but becomes smallest. Also, after some number of generations, it just stops getting better. For example, in first 100 generations, it found the largest circle with radius 104. And in next 900 largest radius is 107. And after drawing it, I see that it can grow much more.
Here is my code connected with GA. I leave out generating random circles, decoding and drawing.
private Genome ChooseParent(Genome[] population, Random r)
{
double sumFitness = 0;
double maxFitness = 0;
for (int i = 0; i < population.Length; i++)
{
sumFitness += population[i].fitness;
if (i == 0 || maxFitness < population[i].fitness)
{
maxFitness = population[i].fitness;
}
}
sumFitness = population.Length * maxFitness - sumFitness;
double randNum = r.NextDouble() *sumFitness;
double acumulatedSum = 0;
for(int i=0;i<population.Length;i++)
{
acumulatedSum += population[i].fitness;
if(randNum<acumulatedSum)
{
return population[i];
}
}
return population[0];
}
private void Crossover(Genome parent1, Genome parent2, Genome child1, Genome child2, Random r)
{
double d=r.NextDouble();
if(d>this.crossoverRate || child1.Equals(child2))
{
for (int i = 0; i < parent1.bitNum; i++)
{
child1.bit[i] = parent1.bit[i];
child2.bit[i] = parent2.bit[i];
}
}
else
{
int cp = r.Next(parent1.bitNum - 1);
for (int i = 0; i < cp; i++)
{
child1.bit[i] = parent1.bit[i];
child2.bit[i] = parent2.bit[i];
}
for (int i = cp; i < parent1.bitNum; i++)
{
child1.bit[i] = parent2.bit[i];
child2.bit[i] = parent1.bit[i];
}
}
}
private void Mutation(Genome child, Random r)
{
for(int i=0;i<child.bitNum;i++)
{
if(r.NextDouble()<=this.mutationRate)
{
child.bit[i] = (byte)(1 - child.bit[i]);
}
}
}
public void Run()
{
for(int generation=0;generation<1000;generation++)
{
CalculateFitness(population);
System.Diagnostics.Debug.WriteLine(maxFitness);
population = population.OrderByDescending(x => x).ToArray();
//ELITIZM
Copy(population[0], newpopulation[0]);
Copy(population[1], newpopulation[1]);
for(int i=1;i<this.populationSize/2;i++)
{
Genome parent1 = ChooseParent(population, r);
Genome parent2 = ChooseParent(population, r);
Genome child1 = newpopulation[2 * i];
Genome child2 = newpopulation[2 * i + 1];
Crossover(parent1, parent2, child1, child2, r);
Mutation(child1, r);
Mutation(child2, r);
}
Genome[] tmp = population;
population = newpopulation;
newpopulation = tmp;
DekodePopulation(population); //decoding and fitness calculation for each member of population
}
}
If someone can point on potential problem that caused such behaviour and ways to fix it, I'll be grateful.
Welcome to the world of genetic algorithms!
I'll go through your issues and suggest a potential problem. Here we go:
maximum fitness does not always grow in the next generation, but becomes smallest - You probably meant smaller. This is weird since you employed elitism, so each generation's best individual should be at least as good as in the previous one. I suggest you check your code for mistakes because this really should not happen. However, the fitness does not need to always grow. It is impossible to achieve this in GA - it's a stochastic algorithm, working with randomness - suppose that, by chance, no mutation nor crossover happens in a generation - then the fitness cannot improve to the next generation since there is no change.
after some number of generations, it just stops getting better. For example, in first 100 generations, it found the largest circle with radius 104. And in next 900 largest radius is 107. And after drawing it, I see that it can grow much more. - this is (probably) a sign of a phenomenon called premature convergence and it's, unfortunately, a "normal" thing in genetic algorithm. Premature convergence is a situation when the whole population converges to a single solution or to a set of solutions which are near each other and which is/are sub-optimal (i.e. it is not the best possible soluion). When this happens, the GA has a very hard time escaping this local optimum. You can try to tweak the parameters, especially the mutation probability, to force more exploration.
Also, another very important thing that can cause problems is the encoding, i.e. how is the bit string mapped to the circle. If the encoding is much too indirect, it can lead to poor performance of the GA. GAs work when there are some building blocks in the genotype which can be exchanged between among the population. If there are no such blocks, the performance of a GA is usually going to be poor.
I have implemented this exercise and achieved good results. Here is the link:
https://github.com/ManhTruongDang/ai-junkie
Hope this can be of use to you.
Consider you have some expression like
i = j = 0
supposing this is well-defined in your language of choice. Would it generally be better to split this up into two expressions like
i = 0
j = 0
I see this sometimes in library code. It doesn't seem buy you much in terms of brevity and shouldn't perform any better than the two statements (though that may be compiler dependant). So, is there a reason to use one over the other? Or is it just personal preference? I know this sounds like a silly question but it's bugging me for a long time now :-).
Once upon a time there was a performance difference, which is one of the reason that this kind of assignment was used. The compilers would turn i = 0; j = 0; into:
load 0
store [i]
load 0
store [j]
So you could save an instruction by using i = j = 0 as the compiler would turn this into:
load 0
store [j]
store [i]
Nowadays compilers can do this type of optimisations by themselves. Also, as the current CPUs run several instructions at once, performance can no longer simply be measured in number of instructions. Instructions where one action doesn't rely on the result of another can run in parallel, so the version that uses a separate value for each variable might actually be faster.
Regarding programming style, you should use the way that best expresses the intention of the code.
You can for example chain the assignments when you simply want to clear some variables, and make it separate assignments when the value has a specific meaning. Especially if the meaning of setting one variable to the value is different from setting the other variable to the same value.
The two forms reflects different points of view on the assignment.
The first case treats assignment (at least the inner one) as an operator (a function returning a value).
The second case treats assignment as a statement (a command to do something).
There is some cases where the assignment as an operator has it's point, mostly for brevity, or to use in contexts that expect a result. However I feel it confusing. For several reasons:
Assignment operator are basically side effect operators, and nowadays it's a problem to optimize them for compilers. In languages like C and C++ they lead to many Undefined Behavior cases, or unoptimized code.
It is unclear what it should return. Should assignment operator return the value that as been assigned, or should it return the address of the place it has been stored. One or the other could be useful, depending on the context.
With composite assignments like +=, it's even worse. It is unclear if the operator should return the initial value, the combined result, or even the place it was stored to.
The assignment as a statement lead sometimes to intermediate variables, but that's the only drawback I see. It is clear and compilers know how to optimize efficiently successive such statements.
Basically, I would avoid assignment as operator whenever possible. The presented case is very simple and not really confusing, but as a general rule I would still prefer.
i = 0
j = 0
or
i, j = 0, 0
for languages that supports, parallel assignment.
It depends on the language. In highly-object-oriented languages, double assignment results in the same object being assigned to multiple variables, so changes in one variable are reflected in the other.
$ python -c 'a = b = [] ; a.append(1) ; print b'
[1]
Firstly, at a semantic level, it depends whether you want to say that i and j are the same value, or just happen to both have the same value.
For example, if i and j are the indexes into a 2D array, they both start at zero. j = i = 0 says i starts at zero, and j starts where i started. If you wanted to start at the second row, you wouldn't necessarily want to start at the second column, so I wouldn't initialise them both in the same statement - the indices for rows and columns independently happen to both start at zero.
Also, in languages where i and j represent complicated objects rather than integral variables, or where assignment may cause an implicit conversion, they are not equivalent:
#include <iostream>
class ComplicatedObject
{
public:
const ComplicatedObject& operator= ( const ComplicatedObject& other ) {
std::cout << " ComplicatedObject::operator= ( const ComplicatedObject& )\n";
return *this;
}
const ComplicatedObject& operator= ( int value ) {
std::cout << " ComplicatedObject::operator= ( int )\n";
return *this;
}
};
int main ()
{
{
// the functions called are not the same
ComplicatedObject i;
ComplicatedObject j;
std::cout << "j = i = 0:\n";
j = i = 0;
std::cout << "i = 0; j = 0:\n";
i = 0;
j = 0;
}
{
// the result of the first assignment is
// effected by implicit conversion
double j;
int i;
std::cout << "j = i = 0.1:\n";
j = i = 0.1;
std::cout << " i == " << i << '\n'
<< " j == " << j << '\n'
;
std::cout << "i = 0.1; j = 0.1:\n";
i = 0.1;
j = 0.1;
std::cout << " i == " << i << '\n'
<< " j == " << j << '\n'
;
}
}
Most of the people will find both possibilities equally readable. Some of these people will have a personal preference for either way. But there are people who might, at first glance, get confused by the "double assignment". I personally like the separate approach, bacause
It is 100% readable
It is not really verbose compared to the double variant
It allows me forget the rules of associativity for = operator
The second way is more readable and clear, I prefer it.
However I try to avoid "double" declaration:
int i, j;
instead of
int i;
int j;
if they're going consecutively. Especially in case of MyVeryLong.AndComplexType