I've been looking for the specific algorithm statement of the Naive Bayes but I can't find any. All I found are pseudocodes and implementations in various languages and documentations discussing this algorithm, but what I need is the step-by-step process itself. Maybe I can't find one because Naive Bayes is a general process (by general, I mean there are algorithms that are specified under the Naive Bayes category or something like that). I have to make an enhancement of it for e-mail classification for my studies. Can you please help me?
Related
I'm learning about Genetic algorithms and procedural content generation and in some papers people talk about Search-based algorithms to create content such as levels.
Anyway, I'm reading some papers about the topic and in some of them talk about Search-based algorithms and basically they explain what it is, and for me looks basically like a genetic algorithm, but I'm not really sure if they're the same, or a particular case of genetic algorithms.
Is the other way around. Genetic Algorithms are a kind of search-based algorithm. In GA you are searching inside the universe of possible solutions, getting better at doing the job by combining parts of the successful solutions (and a bit of chance: mutation).
https://en.wikipedia.org/wiki/Search-based_software_engineering
I'm currently working on a Machine Learning project for my Artificial Intelligence exam. The goal is to correctly choose two classification algorithms to compare using WEKA, bearing in mind that these two algorithms must be different enough to give the comparison a reason to be made. Besides, the algorithms must handle both nominal and numeric data (I suppose this is mandatory to let the comparison be made).
My professor suggested to choose a statistical classifier and a decision tree classifier, for example, or to delve into a comparison between a bottom-up classifier and a top-down one.
Since I have very little experience in the Machine Learning field, I am doing some research on the various algorithms WEKA offers, and I stepped on kNN, that is, k-nearest neighbors algorithm.
Is it statistical? And could it be compared with a Decision Stump algorithm, for example?
Or else, can you suggest a couple of algorithms that match these requirements I have pointed out above?
P. S.: Handled data must be both numerical and nominal. On WEKA there are numerical/nominal features and numerical/nominal classes. Do I have to choose algorithms with both numerical/nominal features AND classes or just one of them?
I would really appreciate any help guys, thanks for your patience!
Based on your professor's description, I would not consider k-Nearest Neighbors (kNN) a statistical classifier. In most contexts, a statistical classifier is one that generalizes via statistics of the training data (either by using statistics directly or by transforming them). An example of this is the Naïve Bayes Classifier.
By contrast, kNN is an example of Instance-Based Learning. It doesn't use statistics of the training data; rather, it compares new observations directly to the training instances to perform classification.
With regard to comparison, yes you can compare performance of kNN with a Decision Stump (or any other classifier). Since any two supervised classifiers will yield a classification accuracies with respect to your training/testing data, you can compare their performance.
Please can someone tell me what the effectiveness of an algorithm relates to? I understand what the efficiency component entails
thanks
Effectiveness relates to the ability to produce the desired result.
Some tasks inherently do not have strict definitions - for example, machine translation between two human languages. Different algorithms exist to translate, say, from English to Spanish; their effectiveness is a measure of how good are the results that these algorithms produce. Their efficiency , on the other hand, measure how fast they are at producing the results, how much memory they use, how much disk space they need, etc.
This question suggests that you have read something which refers to the effectiveness of algorithms and have not understood the author's explanation of the term -- if the author has provided one. I don't think that there is a generally accepted interpretation of the term, I think it is one of those terms which falls under the Humpty-Dumpty rule 'a word means what I say it means'.
It might refer to an aspect of some algorithms which return only approximate solutions to problems. For example, we all know that the travelling salesman problem has NP time complexity, an actual algorithm which 'solves' the TSP might provide some bounds on the difference between the solutions it can find and an optimal solution which might take too long to find.
What properties should the problem have so that I can decide which method to use dynamic programming or greedy method?
Dynamic programming problems exhibit optimal substructure. This means that the solution to the problem can be expressed as a function of solutions to subproblems that are strictly smaller.
One example of such a problem is matrix chain multiplication.
Greedy algorithms can be used only when a locally optimal choice leads to a totally optimal solution. This can be harder to see right away, but generally easier to implement because you only have one thing to consider (the greedy choice) instead of multiple (the solutions to all smaller subproblems).
One famous greedy algorithm is Kruskal's algorithm for finding a minimum spanning tree.
The second edition of Cormen, Leiserson, Rivest and Stein's Algorithms book has a section (16.4) titled "Theoretical foundations for greedy methods" that discusses when the greedy methods yields an optimum solution. It covers many cases of practical interest, but not all greedy algorithms that yield optimum results can be understood in terms of this theory.
I also came across a paper titled "From Dynamic Programming To Greedy Algorithms" linked here that talks about certain greedy algorithms can be seen as refinements of dynamic programming. From a quick scan, it may be of interest to you.
There's really strict rule to know it. As someone already said, there are some things that should turn the red light on, but at the end, only experience will be able to tell you.
We apply greedy method when a decision can be made on the local information available at each stage.We are sure that following the set of decisions at each stage,we will find the optimal solution.
However, in dynamic approach we may not be sure about making a decision at one stage, so we carry a set of probable decisions , one of the probable elements may take to a solution.
does anyone know about some good sources about counting complexity of recursive algorithms?
somehow recurrent equation isn't really popular title for web page or what, I just couldn't google out anything reasonable...
This is a complex topic that is not so well-documented on free licterature on internet.
I just did a similar exam and I can point you to the handbook written by my teacher: PDF Handbook
This handbook covers mostly another tool called generating functions that are useful to solve any kind of recurrence without bothering too much on the kind of recurrence.
There is a good book about Analysis of Algorithms that is An introduction to the Analysis of Algorithms (amazon link) by Sedgewick and Philippe Flajolet but you won't find it online (I had to scan parts of it).
By the way I've searched over internet a lot but I haven't found any complete reference with examples useful to learn the techniques.
I think you would have had more luck with recurrence equation.
You can also check out the Master theorem.
In the analysis of algorithms, the
master theorem, which is a specific
case of the Akra-Bazzi theorem,
provides a cookbook solution in
asymptotic terms for recurrence
relations of types that occur in
practice. It was popularized by the
canonical algorithms textbook
Introduction to Algorithms by Cormen,
Leiserson, Rivest, and Stein, which
introduces and proves it in sections
4.3 and 4.4, respectively. Nevertheless, not all recurrence
relations can be solved with the use
of the master theorem.