Related
I'm currently developing a multi-level SSRS report, and I'm struggling with the algorithm. I've developed a recursion class which looks like below, but the level numbers are incorrect. I want the parent record (represented by a, b, and c) to show the child records so that the child records' level = (parentRecLevel+1). Right now, the level values just increment by 1. Anyone have any advice?
protected BOMLevel getBomLevelItem(str itemId, int numLevel, boolean firstRec)
while select tmpBOM
{
bomLevel = this.getBomLevelItem(bomLevel.ItemId, bomLevel.Level, false);
}
Current Outcome (where b1, c1, and c2 are children of b and c respectively):
1 a
2 b
2 b1
3 c
4 c1
5 c2
Wanted Outcome:
1 a
2 b
3 b1
2 c
3 c1
3 c2
TLDR: Do not reinvent the wheel, use existing algorithms and frameworks.
I'm assuming your question is not for a training exercise, but a real world problem. If it is an exercise, try to get a good grasp of recursion in an easy to use language of your choice with a big community before coming back to x++.
Your recursion method looks incomplete, because in each recursion, you iterate through all records of tmpBom, which (unless you modify the records in that table somewhere else) does not make sense and will never terminate. I also don't see how this method could produce the outcome you describe. I suggest you take a look at some recursion algorithm training material to learn about the fundamental parts of a recursion.
You tagged the question x++ and the syntax also looks very much like that. Unfortunately you did not add the information which version of microsoft-dynamics you are using, but I will assume dynamics-ax-2012 as it is currently the most common version in use.
In this version, there is already an out-of-the-box SSRS report that will show you the structure of a bill of material. You can call the report at Inventory management > Reports > Bills of materials > Lines. It should be fairly easy to modify this report so that it also shows the level if the report does not already fulfill your requirements.
If you still need to implement your own solution, take a look at class BOMSearch and its children. It is used in several places (check the cross references) and can also used to expand/explode a bill of material.
Also note that there are a lot of articles out there that try to explain how to expand or explode a bill of material in x++ code, but as with all things on the internet, be careful: Most of them are incomplete or plain wrong.
I'm working on an intermediate language and a virtual machine to run a functional language with a couple of "problematic" properties:
Lexical namespaces (closures)
Dynamically growing call stack
A slow integer type (bignums)
The intermediate language is stack based, with a simple hash-table for the current namespace. Just so you get an idea of what it looks like, here's the McCarthy91 function:
# McCarthy 91: M(n) = n - 10 if n > 100 else M(M(n + 11))
.sub M
args
sto n
rcl n
float 100
gt
.if
.sub
rcl n
float 10
sub
.end
.sub
rcl n
float 11
add
list 1
rcl M
call-fast
list 1
rcl M
tail
.end
call-fast
.end
The "big loop" is straightforward:
fetch an instruction
increment the instruction pointer (or program counter)
evaluate the instruction
Along with sto, rcl and a whole lot more, there are three instructions for function calls:
call copies the namespace (deep copy) and pushes the instruction pointer onto the call stack
call-fast is the same, but only creates a shallow copy
tail is basically a 'goto'
The implementation is really straightforward. To give you a better idea, here's just a random snippet from the middle of the "big loop" (updated, see below)
} else if inst == 2 /* STO */ {
local[data] = stack[len(stack) - 1]
if code[ip + 1][0] != 3 {
stack = stack[:len(stack) - 1]
} else {
ip++
}
} else if inst == 3 /* RCL */ {
stack = append(stack, local[data])
} else if inst == 12 /* .END */ {
outer = outer[:len(outer) - 1]
ip = calls[len(calls) - 1]
calls = calls[:len(calls) - 1]
} else if inst == 20 /* CALL */ {
calls = append(calls, ip)
cp := make(Local, len(local))
copy(cp, local)
outer = append(outer, &cp)
x := stack[len(stack) - 1]
stack = stack[:len(stack) - 1]
ip = x.(int)
} else if inst == 21 /* TAIL */ {
x := stack[len(stack) - 1]
stack = stack[:len(stack) - 1]
ip = x.(int)
The problem is this: Calling McCarthy91 16 times with a value of -10000 takes, near as makes no difference, 3 seconds (after optimizing away the deep-copy, which adds nearly a second).
My question is: What are some common techniques for optimizing interpretation of this kind of language? Is there any low-hanging fruit?
I used slices for my lists (arguments, the various stacks, slice of maps for the namespaces, ...), so I do this sort of thing all over the place: call_stack[:len(call_stack) - 1]. Right now, I really don't have a clue what pieces of code make this program slow. Any tips will be appreciated, though I'm primarily looking for general optimization strategies.
Aside:
I can reduce execution time quite a bit by circumventing my calling conventions. The list <n> instruction fetches n arguments of the stack and pushes a list of them back onto the stack, the args instruction pops off that list and pushes each item back onto the stack. This is firstly to check that functions are called with the correct number of arguments and secondly to be able to call functions with variable argument-lists (i.e. (defun f x:xs)). Removing that, and also adding an instruction sto* <x>, which replaces sto <x>; rcl <x>, I can get it down to 2 seconds. Still not brilliant, and I have to have this list/args business anyway. :)
Another aside (this is a long question I know, sorry):
Profiling the program with pprof told me very little (I'm new to Go in case that's not obvious) :-). These are the top 3 items as reported by pprof:
16 6.1% 6.1% 16 6.1% sweep pkg/runtime/mgc0.c:745
9 3.4% 9.5% 9 3.4% fmt.(*fmt).fmt_qc pkg/fmt/format.go:323
4 1.5% 13.0% 4 1.5% fmt.(*fmt).integer pkg/fmt/format.go:248
These are the changes I've made so far:
I removed the hash table. Instead I'm now passing just pointers to arrays, and I only efficiently copy the local scope when I have to.
I replaced the instruction names with integer opcodes. Before, I've wasted quite a bit of time comparing strings.
The call-fast instruction is gone (the speedup wasn't measurable anymore after the other changes)
Instead of having "int", "float" and "str" instructions, I just have one eval and I evaluate the constants at compile time (compilation of the bytecode that is). Then eval just pushes a reference to them.
After changing the semantics of .if, I could get rid of these pseudo-functions. it's now .if, .else and .endif, with implicit gotos ànd block-semantics similar to .sub. (some example code)
After implementing the lexer, parser, and bytecode compiler, the speed went down a little bit, but not terribly so. Calculating MC(-10000) 16 times makes it evaluate 4.2 million bytecode instructions in 1.2 seconds. Here's a sample of the code it generates (from this).
The whole thing is on github
There are decades of research on things you can optimize:
Implementing functional languages: a tutorial, Simon Peyton Jones and David Lester. Published by Prentice Hall, 1992.
Practical Foundations for Programming Languages, Robert Harper
You should have efficient algorithmic representations for the various concepts of your interpreter. Doing deep copies on a hashtable looks like a terrible idea, but I see that you have already removed that.
(Yes, your stack-popping operation using array slices look suspect. You should make sure they really have the expected algorithmic complexity, or else use a dedicated data structure (... a stack). I'm generally wary of using all-purposes data structure such as Python lists or PHP hashtables for this usage, because they are not necessarily designed to handle this particular use case well, but it may be that slices do guarantee an O(1) pushing and popping cost under all circumstances.)
The best way to handle environments, as long as they don't need to be reified, is to use numeric indices instead of variables (de Bruijn indices (0 for the variable bound last), or de Bruijn levels (0 for the variable bound first). This way you can only keep a dynamically resized array for the environment and accessing it is very fast. If you have first-class closures you will also need to capture the environment, which will be more costly: you have to copy the part of it in a dedicated structure, or use a non-mutable structure for the whole environment. Only experiment will tell, but my experience is that going for a fast mutable environment structure and paying a higher cost for closure construction is better than having an immutable structure with more bookkeeping all the time; of course you should make an usage analysis to capture only the necessary variables in your closures.
Finally, once you have rooted out the inefficiency sources related to your algorithmic choices, the hot area will be:
garbage collection (definitely a hard topic; if you don't want to become a GC expert, you should seriously consider reusing an existing runtime); you may be using the GC of your host language (heap-allocations in your interpreted language are translated into heap-allocations in your implementation language, with the same lifetime), it's not clear in the code snippet you've shown; this strategy is excellent to get something reasonably efficient in a simple way
numerics implementation; there are all kind of hacks to be efficient when the integers you manipulate are in fact small. Your best bet is to reuse the work of people that have invested tons of effort on this, so I strongly recommend you reuse for example the GMP library. Then again, you may also reuse your host language support for bignum if it has some, in your case Go's math/big package.
the low-level design of your interpreter loop. In a language with "simple bytecode" such as yours (each bytecode instruction is translated in a small number of native instructions, as opposed to complex bytecodes having high-level semantics such as the Parrot bytecode), the actual "looping and dispatching on bytecodes" code can be a bottleneck. There has been quite a lot of research on what's the best way to write such bytecode dispatch loops, to avoid the cascade of if/then/else (jump tables), benefit from the host processor branch prediction, simplify the control flow, etc. This is called threaded code and there are a lot of (rather simple) different techniques : direct threading, indirect threading... If you want to look into some of the research, there is for example work by Anton Ertl, The Structure and Performance of Efficient Interpreters in 2003, and later Context threading: A flexible and efficient dispatch technique for virtual machine interpreters. The benefits of those techniques tend to be fairly processor-sensitive, so you should test the various possibilities yourself.
While the STG work is interesting (and Peyton-Jones book on programming language implementation is excellent), it is somewhat oriented towards lazy evaluation. Regarding the design of efficient bytecode for strict functional languages, my reference is Xavier Leroy's 1990 work on the ZINC machine: The ZINC experiment: An Economical Implementation of the ML Language, which was ground-breaking work for the implementation of ML languages, and is still in use in the implementation of the OCaml language: there are both a bytecode and a native compiler, but the bytecode still uses a glorified ZINC machine.
I started doing Project Euler and got to problem number 9. Since I was using Project Euler to learn Haskell, I decided to use list comprehensions (as shown in Learn You A Haskell). I do that and GHCI takes awhile to figure out the triplet, which I figured is normal because of the calculations involved. Now, at work yesterday (I don't work as a programmer professionally, yet) I was talking to a friend who knows VBA and he wanted to try to find the answers in VBA. I thought it would be a fun challenge as well, and I churn out some basic for loops and if statements, but what got me was that it was much faster than Haskell was.
My question is: are Haskell's list comprehension incredibly inefficient? At first I thought it was just because I was in GHC's interactive mode, but then I realized VBA is interpreted too.
Please note, I didn't post my code because of it being an answer to project euler. If it will answer my question (as in I'm doing something wrong) then I will gladly post the code.
[edit]
Here is my Haskell list comprehension:
[(a,b,c) | c <- [1..1000], b <- [1..c], a <- [1..b], a+b+c=1000, a^2+b^2=c^2]
I guess I could've lowered the range on c but is that what is really slowing it down?
There are two things you could be doing with this problem that could make your code slow. One is how you are trying values for a, b and c. If you loop through all possible values for a, b, c from 1 to 1000, you'll be spending a long time. To give a hint, you can make use of a+b+c=1000 if you rearrange it for c. The other is that if you only use a list comprehension, it will process every possible value for a, b and c. The problem tells you that there is only one unique set of numbers that satisfies the problem, so if you change your answer from this:
[ a * b * c | .... ]
to:
head [ a * b * c | ... ]
then Haskell's lazy evaluation means that it will stop after finding the first answer. This is the Haskell equivalent of breaking out of your VBA loop when you find the first answer. When I used both these tips, I had an answer that completed very quickly (under a second) in ghci.
Addendum: I missed at first the condition a < b < c. You can also make use of this in your list comprehensions; it is valid to say things along the lines of:
[(a, b) | b <- [1..100], a <- [1..b-1]]
Consider this simplified version of your list comprehension:
[(a,b,c) | a <- [1..1000], b <- [1..1000], c <- [1..1000]]
This will give all possible combinations of a, b, and c. It's kind of like saying, "how many ways can three one-thousand-sided dice land?" The answer is 1000*1000*1000 = 1,000,000,000 different combinations. If it took 0.001 seconds to generate each combination, it would therefore take 1,000,000 seconds (~11.5 days) to finish all combinations. (OK, 0.001 seconds is actually pretty slow for a computer, but you get the idea)
When you add predicates to your list comprehension, it still takes the same amount of time to compute; in fact, it takes longer since it needs to check the predicate for each of the 1 billion combinations it computes.
Now consider your comprehension. It looks like it should be much faster, right?
[(a,b,c) | c <- [1..1000], b <- [1..c], a <- [1..b], a+b+c=1000, a^2+b^2=c^2]
There are 1000 choices for c. How many are there for b and a? Well, the average choice for c is 500. For all choices of c, then, there are an average of 500 choices for b (since b can range from 1 to c). Likewise, for all choices of c and b, there are an average of 250 choices for a. That's very hand-wavy, but I'm fairly sure it's accurate. So 1000 choices for c * 1000/2 choices for b * 1000/4 choices for a = 1 billion / 8 ~= 100 million. It's 8x faster, but if you paid attention, you'll notice it's actually the same big-Oh complexity as the simplified version above. If we compared "simplified" vs "improved" versions of the same problem, but from [1..100000] instead of [1..1000], the "improved" would still only be 8x faster than the "simplified".
Don't get me wrong, 8x is a wonderful constant-factor speedup. But unless you want to wait a couple hours to get the solution, you'll need to get a better big-Oh.
As Neil noted, the way to reduce the complexity of this problem is, for a given b and c, choose the a that satisfies a+b+c=1000. That way, you're not trying a bunch of as that will fail. This will drop the big-Oh complexity; you'll only be considering approximately 1000 * 500 * 1 = 500,000 combinations, instead of ~100,000,000.
Once you get the solution to the problem you can check out other peoples versions of Haskell solutions on the Project Euler site to get an idea of how other people have solved the problem. Incidentally, here is a link to the referenced problem: http://projecteuler.net/index.php?section=problems&id=9
In addition to what everyone else has said about generating fewer elements in the generators, you can also switch to using Int instead of Integer as the type of the numbers. The default is Integer, but your numbers are small enough to fit in an Int.
(Also, to nitpick, Haskell list comprehensions have no speed. Haskell is a language definition with very little operational semantics. A particular Haskell implementation might have slow list comprehensions, though.)
Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 4 years ago.
Improve this question
I need to translate some python and java routines into pseudo code for my master thesis but have trouble coming up with a syntax/style that is:
consistent
easy to understand
not too verbose
not too close to natural language
not too close to some concrete programming language.
How do you write pseudo code? Are there any standard recommendations?
I recommend looking at the "Introduction to Algorithms" book (by Cormen, Leiserson and Rivest). I've always found its pseudo-code description of algorithms very clear and consistent.
An example:
DIJKSTRA(G, w, s)
1 INITIALIZE-SINGLE-SOURCE(G, s)
2 S ← Ø
3 Q ← V[G]
4 while Q ≠ Ø
5 do u ← EXTRACT-MIN(Q)
6 S ← S ∪{u}
7 for each vertex v ∈ Adj[u]
8 do RELAX(u, v, w)
Answering my own question, I just wanted to draw attention to the TeX FAQ entry Typesetting pseudocode in LaTeX. It describes a number of different styles, listing advantages and drawbacks. Incidentally, there happen to exist two stylesheets for writing pseudo code in the manner used in "Introductin to Algorithms" by Cormen, as recommended above: newalg and clrscode. The latter was written by Cormen himself.
I suggest you take a look at the Fortress Programming Language.
This is an actual programming language, and not pseudocode, but it was designed to be as close to executable pseudocode as possible. In particular, for designing the syntax, they read and analyzed hundreds of CS and math papers, courses, books and journals to find common usage patterns for pseudocode and other computational/mathematical notations.
You can leverage all that research by just looking at Fortress source code and abstracting out the things you don't need, since your target audience is human, whereas Fortress's is a compiler.
Here is an actual example of running Fortress code from the NAS (NASA Advanced Supercomputing) Conjugate Gradient Parallel Benchmark. For a fun experience, compare the specification of the benchmark with the implementation in Fortress and notice how there is almost a 1:1 correspondence. Also compare the implementation in a couple of other languages, like C or Fortran, and notice how they have absolutely nothing to do with the specification (and are also often an order of magnitude longer than the spec).
I must stress: this is not pseudocode, this is actual working Fortress code! From https://umbilicus.wordpress.com/2009/10/16/fortress-parallel-by-default/
Note that Fortress is written in ASCII characters; the special characters are rendered with a formatter.
If the code is procedural, normal pseudo-code is probably easy (Wikipedia has some examples).
Object-oriented pseudo-code might be more difficult. Consider:
using UML class diagrams to depict the classes/inheritence
using UML sequence diagrams to depict the sequence of code
I don't understand your requirement of "not too close to some concrete programming language".
Python is generally considered as a good candidate for writing pseudo-code. Perhaps a slightly simplified version of python would work for you.
Pascal has always been traditionally the most similar to pseudocode, when it comes to mathematical and technical fields. I don't know why, it was just always so.
I have some (oh, I don't know, 10 maybe books on a shelf, which concrete this theory).
Python as suggested, can be nice code, but it can be so unreadable as well, that it's a wonder by itself. Older languages are harder to make unreadable - them being "simpler" (take with caution) than today's ones. They'll maybe be harder to understand what's going on, but easier to read (less syntax/language features is needed for to understand what the program does).
This post is old, but hopefully this will help others.
"Introduction to Algorithms" book (by Cormen, Leiserson and Rivest) is a good book to read about algorithms, but the "pseudo-code" is terrible. Things like Q[1...n] is nonsense when one needs to understand what Q[1...n] is suppose to mean. Which will have to be noted outside of the "pseudo-code." Moreover, books like "Introduction to Algorithms" like to use a mathematical syntax, which is violating one purpose of pseudo-code.
Pseudo-code should do two things. Abstract away from syntax and be easy to read. If actual code is more descriptive than the pseudo-code, and actual code is more descriptive, then it is not pseudo-code.
Say you were writing a simple program.
Screen design:
Welcome to the Consumer Discount Program!
Please enter the customers subtotal: 9999.99
The customer receives a 10 percent discount
The customer receives a 20 percent discount
The customer does not receive a discount
The customer's total is: 9999.99
Variable List:
TOTAL: double
SUB_TOTAL: double
DISCOUNT: double
Pseudo-code:
DISCOUNT_PROGRAM
Print "Welcome to the Consumer Discount Program!"
Print "Please enter the customers subtotal:"
Input SUB_TOTAL
Select the case for SUB_TOTAL
SUB_TOTAL > 10000 AND SUB_TOTAL <= 50000
DISCOUNT = 0.1
Print "The customer receives a 10 percent discount"
SUB_TOTAL > 50000
DISCOUNT = 0.2
Print "The customer receives a 20 percent discount"
Otherwise
DISCOUNT = 0
Print "The customer does not a receive a discount"
TOTAL = SUB_TOTAL - (SUB_TOTAL * DISCOUNT)
Print "The customer's total is:", TOTAL
Notice that this is very easy to read and does not reference any syntax. This supports all three of Bohm and Jacopini's control structures.
Sequence:
Print "Some stuff"
VALUE = 2 + 1
SOME_FUNCTION(SOME_VARIABLE)
Selection:
if condition
Do one extra thing
if condition
do one extra thing
else
do one extra thing
if condition
do one extra thing
else if condition
do one extra thing
else
do one extra thing
Select the case for SYSTEM_NAME
condition 1
statement 1
condition 2
statement 2
condition 3
statement 3
otherwise
statement 4
Repetition:
while condition
do stuff
for SOME_VALUE TO ANOTHER_VALUE
do stuff
compare that to this N-Queens "pseudo-code" (https://en.wikipedia.org/wiki/Eight_queens_puzzle):
PlaceQueens(Q[1 .. n],r)
if r = n + 1
print Q
else
for j ← 1 to n
legal ← True
for i ← 1 to r − 1
if (Q[i] = j) or (Q[i] = j + r − i) or (Q[i] = j − r + i)
legal ← False
if legal
Q[r] ← j
PlaceQueens(Q[1 .. n],r + 1)
If you can't explain it simply, you don't understand it well enough.
- Albert Einstein
I've got a classification system, which I will unfortunately need to be vague about for work reasons. Say we have 5 features to consider, it is basically a set of rules:
A B C D E Result
1 2 b 5 3 X
1 2 c 5 4 X
1 2 e 5 2 X
We take a subject and get its values for A-E, then try matching the rules in sequence. If one matches we return the first result.
C is a discrete value, which could be any of a-e. The rest are just integers.
The ruleset has been automatically generated from our old system and has an extremely large number of rules (~25 million). The old rules were if statements, e.g.
result("X") if $A >= 1 && $A <= 10 && $C eq 'A';
As you can see, the old rules often do not even use some features, or accept ranges. Some are more annoying:
result("Y") if ($A == 1 && $B == 2) || ($A == 2 && $B == 4);
The ruleset needs to be much smaller as it has to be human maintained, so I'd like to shrink rule sets so that the first example would become:
A B C D E Result
1 2 bce 5 2-4 X
The upshot is that we can split the ruleset by the Result column and shrink each independently. However, I cannot think of an easy way to identify and shrink down the ruleset. I've tried clustering algorithms but they choke because some of the data is discrete, and treating it as continuous is imperfect. Another example:
A B C Result
1 2 a X
1 2 b X
(repeat a few hundred times)
2 4 a X
2 4 b X
(ditto)
In an ideal world, this would be two rules:
A B C Result
1 2 * X
2 4 * X
That is: not only would the algorithm identify the relationship between A and B, but would also deduce that C is noise (not important for the rule)
Does anyone have an idea of how to go about this problem? Any language or library is fair game, as I expect this to be a mostly one-off process. Thanks in advance.
Check out the Weka machine learning lib for Java. The API is a little bit crufty but it's very useful. Overall, what you seem to want is an off-the-shelf machine learning algorithm, which is exactly what Weka contains. You're apparently looking for something relatively easy to interpret (you mention that you want it to deduce the relationship between A and B and to tell you that C is just noise.) You could try a decision tree, such as J48, as these are usually easy to visualize/interpret.
Twenty-five million rules? How many features? How many values per feature? Is it possible to iterate through all combinations in practical time? If you can, you could begin by separating the rules into groups by result.
Then, for each result, do the following. Considering each feature as a dimension, and the allowed values for a feature as the metric along that dimension, construct a huge Karnaugh map representing the entire rule set.
The map has two uses. One: research automated methods for the Quine-McCluskey algorithm. A lot of work has been done in this area. There are even a few programs available, although probably none of them will deal with a Karnaugh map of the size you're going to make.
Two: when you have created your final reduced rule set, iterate over all combinations of all values for all features again, and construct another Karnaugh map using the reduced rule set. If the maps match, your rule sets are equivalent.
-Al.
You could try a neural network approach, trained via backpropagation, assuming you have or can randomly generate (based on the old ruleset) a large set of data that hit all your classes. Using a hidden layer of appropriate size will allow you to approximate arbitrary discriminant functions in your feature space. This is more or less the same idea as clustering, but due to the training paradigm should have no issue with your discrete inputs.
This may, however, be a little too "black box" for your case, particularly if you have zero tolerance for false positives and negatives (although, it being a one-off process, you get an arbitrary degree of confidence by checking a gargantuan validation set).