Related
A year ago I programmed a chess AI using the Alphabeta prunning algorithm. This was relatively straight forward to do in c++. One of the main issues I considered while doing this was making my code efficient. I did this by using having a data type I called a "game" that I passed around through the search tree made by the algorithm. To increase efficiency I didn't ever copy the "game" data type but rather mutated it while keeping the nessisary information needed to return it to its previous states.
Recently I have been reading about functional programming and the concept of purely using functions that do not change the state of the parameters they are passed appeals to me. I am wondering how I would using the paradigm of functional programming while still taking efficiency of the program into account.
In OOP the solution seems quite straight forward (which is what I implemented) while in functional programming it seems that copying data types is nessisary which decreases efficiency. Is it possible to use functional programming without this loss of efficiency?
In functional programming, data structures are not always copied completely. In many cases, only the part that changes needs to be copied, while the old parts can be referenced (since no mutation is allowed, this is safe).
The article on persistant data structures describes this in more detail.
Jephron's answer points out the important fact that only small parts of a persistent data structure need to get updated, thus the bigger part is shared between the old state and the new state.
To be honest, this would still be slower than a mutation in most cases.
But immutable, persistent data structures have other advantages. Let's assume you have already completed the playing engine. And now, you want to implement a history (for example to allow the player to undo earlier moves). This is dead simple: Just remember all states in a list. You'll find that you need to touch only a few functions to take a list of states instead of just the last state, and you're done. You don't need to worry about compromising your game engine --- there is no global variable or something you could destroy.
Another thing is taking advantage of the many CPU cores you probably have by employing parallelism. Needless to say that you can't let many tasks, threads, fibers or whatever operate on a single mutable data structure. This would just become a synchronization nightmare, and your code would probably go slower even. However, there simply are no synchronization problems on immutable data, as they are read only for all threads.
This could very well speed up your code in such a way that it dwarfs the C++ solution, even if "doing a move" on a functional data structure is much slower than on mutable data.
Here is an example for changing a board game (TTT) from single threaded to parallel: https://dierk.gitbooks.io/fregegoodness/content/src/docs/asciidoc/incremental_episode4.html
I currently work with PHP and Ruby on Rails as a web developer. My question is why would I need to know algorithms and data structures? Do I need to learn C, C++ or Java first? What are the practical benefits of knowing algorithms and data structures? What are algorithms and data structures in layman’s terms? (As you can tell unfortunately I have not done a CS course.)
Please provide as much information as possible and thank you in advance ;-)
Data structures are ways of storing stuff, just like you can put stuff in stacks, queues, heaps and buckets - you can do the same thing with data.
Algorithms are recipes or instructions, the quick start manual for your coffee maker is an algorithm to make coffee.
Algorithms are, quite simply, the steps by which you do something. For instance the Coffee Maker Algorithm would run something like
Turn on Coffee Maker
Grind Coffee Beans
Put in filter and place coffee in filter
Add Water
Start brewing process
Drink coffee
A data structure is a means by which we store information in a organized fashion. For further info, check out the Wikipedia Article.
An algorithm is a list of instructions and data structures are ways to represent information. If you're writing computer programs then you're already using algorithms and data structures even if you don't know what the words mean.
I think the biggest advantages in knowing standard algorithms and data structures are:
You can communicate with other programmers using a common language.
Other people will be able to understand your code once you've left.
You will also learn better methods for solving common problems. You could probably solve these problems eventually anyway even without knowing the standard way to do it, but you will spend a lot of time reinventing the wheel and it's unlikely your solutions will be as good as those that thousands of experts have worked on and improved over the years.
An algorithm is a sequence of well defined steps leading to the solution of a type of problem.
A data structure is a way to store and organize data to facilitate access and modifications.
The benefit of knowing standard algorithms and data structures is they are mostly better than you yourself could develop. They are the result of months or even years of work by people who are far more intelligent than the majority of programmers. Knowing a range of data structures and algorithms allows you to fit a problem roughly to a data structure or/and algorithm and tweak as required.
In the classic "cooking/baking equivalent", algorithms are recipes and data structures are your measuring cups, your baking sheets, your cookie cutters, mixing bowls and essentially any other tool you would be using (your cooker is your compiler/interpreter, though).
(source: mit.edu)
This book is the bible on algorithms. In general, data structures relate to how to organize your data to access it in memory, and algorithms are methods / small programs to resolve problems (ex: sorting a list).
The reason you should care is first to understand what can go wrong in your code; poorly implemented algorithms can perform very badly compared to "proven" ones. Knowing classic algorithms and what performance to expect from them helps in knowing how good your code can be, and whether you can/should improve it.
Then there is no need to reinvent the wheel, and rewrite a buggy or sub-optimal implementation of a well-known structure or algorithm.
An algorithm is a representation of the process involved in a computation.
If you wanted to add two numbers then the algorithm might go:
Get first number;
Get second number;
Add first number to second number;
Return result.
At its simplest, an algorithm is just a structured list of things to do - its use in computing is that it allows people to see the intent behind the code and makes logical (as opposed to syntactical) errors easier to spot.
e.g. if step three above said multiply instead of add then someone would be able to point out the error in the logic without having to debug code.
A data structure is a representation of how a system's data should be referenced. It might match a table structure exactly or may be de-normalised to make data access easier. At its simplest it should show how the entities in a system are related.
It is too large a topic to go into in detail but there are plenty of resources on the web.
Data structures are critical the second your software has more than a handful of users. Algorithms is a broad topic, and you'll want to study it if a good knowledge of data structures doesn't fix your performance problems.
You probably don't need a new programming language to benefit from data structures knowledge, though PHP (and other high level languages) will make a lot of it invisible to you, unless you know where to look. Java is my personal favorite learning language for stuff like this, but that's pretty subjective.
My question is why would I need to know algorithms and data structures?
If you are doing any non-trivial programming, it is a good idea to understand the class data structures and algorithms and their uses in order to avoid reinventing the wheel. For example, if you need to put an array of things in order, you need to understand the various ways of sorting, so that you can choose the most appropriate one for the task in hand. If you choose the wrong approach, you can end up with a program that is grossly inefficient in some circumstances.
Do I need to learn C, C++ or Java first?
You need to know how to program in some language in order to understand what the algorithms and data structures do.
What are the practical benefits of knowing algorithms and data structures?
The main practical benefits are:
to avoid having to reinvent the wheel all of the time,
to avoid the problem of square wheels.
Are data structures like linked lists something that are purely academic for real programming or do you really use them? Are they things that are covered by generics so that you don't need to build them (assuming your language has generics)? I'm not debating the importance of understanding what they are, just the usage of them outside of academia. I ask from a front end web, backend database perspective. I'm sure someone somewhere builds these. I'm asking from my context.
Thank you.
EDIT: Are Generics so that you don't have to build linked lists and the like?
It will depend on the language and frameworks you're using. Most modern languages and frameworks won't make you reinvent these wheels. Instead, they'll provide things like List<T> or HashTable.
EDIT:
We probably use linked lists all the time, but don't realize it. We don't have to write implementations of linked lists on our own, because the frameworks we use have already written them for us.
You may also be getting confused about "generics". You may be referring to generic list classes like List<T>. This is just the same as the non-generic class List, but where the element is always of type T. It is probably implemented as a linked list, but we don't have to care about that.
We also don't have to worry about allocation of physical memory, or how interrupts work, or how to create a file system. We have operating systems to do that for us. But we may be taught that information in school just the same.
Certainly. Many "List" implementations in modern languages are actually linked lists, sometimes in combination with arrays or hash tables for direct access (by index as opposed to iteration).
Linked lists (especially doubly linked lists) are very commonly used in "real-world" data structures.
I would dare to say every common language has a pre-built implementation of linked list, either as a language primitive, native template library (e.g. C++), native library (e.g. Java) or some 3rd party implementation (probably open-source).
That being said, several times in the past I wrote a linked list implementation from scratch myself when creating infrastructure code for complex data structures. Sometimes it's a good idea to have full control over the implementation, and sometimes you need to add a "twist" to the classic implementation for it to satisfy your specific requirement. There's no right or wrong when it comes to whether to code your own implementation, as long as you understand the alternatives and trade-offs. In most cases, and certainly in very modern languages like C# I would avoid it.
Another point is when you should use lists versus array/vectors or hash tables. From your question I understand you are aware of the trade-offs here so I won't go too much into it, but basically, if your main usage is traversing lists by-order, and the list size may vary significantly, a list may be a viable option. Another consideration is the type of insertion. If a common use case is "inserting in the middle", than lists have a significant advantage over arrays/vectors. I can go on but this information is in the classic CS books :)
Clarification: My answer is language agnostic and does not relate specifically to Generics which to my understanding have a linked list implementation.
A singly-linked list is the only way to have a memory efficient immutable list which can be composed to "mutate" it. Look at how Erlang does it. It may be slightly slower than an array-backed list but it has very useful properties in multithreaded and purely-functional implementations.
Yes, there are real world application that use linked list, I sometimes have to maintain a huge application that makes very have use of linked lists.
And yes, linked lists are included in just about any class library from C++/STL to .net.
And I wish it used arrays instead.
In the real world linked lists are SLOW because of things like paging and CPU cache size (linked lists tend to spread you data and that makes it more likely that you will need to access data from different areas of memory and that is much slower on todays computers than using arrays that store all the data in one sequence).
Google "locality of reference" for more info.
Never used hand-made lists except for homeworks at university.
Depending on usage a linked list could be the best option. Deletes from the front of the list are much faster with a linked list than an array list.
In a Java program that I maintain profiling showed that I could increase performance by moving from an ArrayList to a LinkedList for a List that had lots of deletes at the beginning.
I've been developing line of business applications (.NET) for years and I can only think of one instance where I've used linked list and even then I did not have to create the object.
This has just been my experience.
I would say it depends on the usage, in some cases they are quicker than typical random access containers.
Also I think they are used by some libraries as an underlying collection type, so what may look like a non-linked list might actually be one underneath.
In a C/C++ application I developed at my last company we used doubly linked lists all the time. They were essential to what we were doing, which was real-time 3D graphics.
Yes all sorts of data-structures are very useful in daily software development. In most languages that I know (C/C++/Python/Objective-C) there are frameworks that implement those data-structures so that you don't have to reinvent the wheel.
And yes, data-structures are not only for academics, they are very useful and you would not be able to write software without them (depends on what you do).
You use data-structures in message queues, data maps, hash tables, keeping data ordered, fast access/removal/insertion and so on depends what needs to be done.
Yes, I do. It all depends on the situation. If I won't be storing a lot of data in them, or if the specific application needs a FIFO structure, I'll use them without a second thought because they are fast to implement.
However, in applications for other developers I know, there are times that a linked list would fit perfectly except that poor locality causes a lot of cache misses.
I cannot imagine many programs that doesn't deal with lists.
The minute you need to deal with more than 1 thing of something, lists in all forms and shapes becomes needed, as you need somewhere to store these things. That list might be a singly/doubly linked list, an array, a set, a hashtable if you need to index your things based on a key, a priority queue if you need to sort it etc.
Typically you'd store these lists in a database system, but somewhere you need to fetch them from the db, store them in your application and manipulate them, even if it's as simple to retrieve a little list of things you populate into a drop-down combobox.
These days, in languages such as C#,Python,Java and many more, you're usually abstracted away from having to implement your own lists. These languages come with a great deal of abstractions of containers you can store stuff in. Either via standard libraries or built into the language.
You're still at an advantage of learning these topics, e.g. if you're working with C# you'd want to know how an ArrayList works, and wheter you'd choose ArrayList or something else depending on your need to add/insert/search/random index such a list.
The "queue", or FIFO, is one of the most common data structures, and have native implementations in many languages and frameworks. However, there seems to be little consensus as to how fundamental queue operations should be named. A survey of several popular languages show:
Python: put / get
C#, Qt : enqueue /dequeue
Ruby, C++ STD: push / pop
Java: add / remove
If one need to implement a queue (say, in some embedded platform that does not have a native queue implementation already), what naming convention would be best?
Enqueue/dequeue seem to be the most explicit, but is wordy; put/get is succinct but does not provide any hint as to the FIFO nature of the operations; push/pop seems to be suggest stack operations instead of queue operations.
I'm kind of a pedant, so I'd go with enqueue/dequeue.
Though add/next has a certain appeal.
Just to cloud the issue a little more, in Perl it's push/shift. :)
push/pop is plain wrong for a fifo as these are stack (first in last out) operations.
queue can refer to the object as well as an operation so is a bit overloaded and dequeue can cause confusion because it was commonly used to refer to a double ended queue.
put/get - short, obvious and generic (doesn't assume an implementation and can be used for all sorts of queues/lists/collections) - what's not to like?
I'll probably name it as push_back and pop_front.
Add/Remove sounds like the most logical one to use, especially if you are intending for it to possibly be read by person unfamiliar with the structure or the language (easier to understand).
Push/Pop would be next in my rankings because of personal preferences.
Put/Get comes next.
Enqueue/Dequeue is very last because I really hate the letter Q.
Add/remove has the advantage that you can easily change from a queue to another data structure.
For example, storing states in a queue vs. a stack makes the difference between breadth-first and depth-first search.
I like enqueue and dequeue, but typing them sucks. So in my Queue structures (both C++ and Java), I named the functions enQ and deQ :)
Pop / push sounds wrong as it suggests a stack data structure instead of a queue.
To add something new to the suggestions: my teachers always used in and out on the blackboard.
I like entail and behead. Not for everyone though. Or "in with the new", "out with the old". And then for us Southwesterners there are berattle and defang. But my favorite is graphical. Right Arrow for in and then Right Arrow for out.
Closed. This question needs to be more focused. It is not currently accepting answers.
Want to improve this question? Update the question so it focuses on one problem only by editing this post.
Closed 8 years ago.
Improve this question
I have a Delphi 2009 program that handles a lot of data and needs to be as fast as possible and not use too much memory.
What small simple changes have you made to your Delphi code that had the biggest impact on the performance of your program by noticeably reducing execution time or memory use?
Thanks everyone for all your answers. Many great tips.
For completeness, I'll post a few important articles on Delphi optimization that I found.
Before you start optimizing Delphi code at About.com
Speed and Size: Top 10 Tricks also at About.com
Code Optimization Fundamentals and Delphi Optimization Guidelines at High Performance Delphi, relating to Delphi 7 but still very pertinent.
.BeginUpdate;
.EndUpdate;
;)
Use a Delphi Profiling tool (Some here or here) and discover your own bottle necks. Optimizing the wrong bottlenecks is a waste of time. In other words, if you apply all of these suggestions here, but ignore the fact someone put a sleep(1000) (or similar) in some very important code is a waste of your time. Fix your actual bottlenecks first.
Stop using TStringList for everything.
TStringList is not a general purpose datastructure for effective storage and handling of everything from simple to complex types. Look for alternatives. I use Delphi Container and Algorithm Library (DeCAL, formerly known as SDL). Julians EZDSL should also be a good alternative.
Pre-allocating lists and arrays, rather than growing them with each iteration.
This has probably had the biggest impact for me in terms of speed.
If you need to use Application.processmesssages (or similar) in a loop, try calling it only every Nth iteration.
Similarly, if updating a progressbar, don't update it every iteration. Instead, increment it by x units every x iterations, or scale the updates according to time or as a percentage of overall task length.
FastMM
FastCode (lib)
Use high performance data structures, like hash table (etc). Many places it is faster to make one loop which makes lookup hash table for your data. Uses quite lot of memory but it surely is fast. (this maybe is most important one, but 2 first are dead simple and need very little of effort to do)
Reduce disk operations. If there's enough memory, load the file entirely to RAM and do all operations in memory.
Consider the careful use of threads. If you are not using threads now, then consider adding a couple. If you are, make sure you are not using too many. If you are running on a Dual or Quad core computer (which most are any more) then proper thread tuning is very important.
You could look at OmniThread Library by Gabr, but there are a number of thread libraries in development for Delphi. You could easily implement your own parallel for using anonymous types.
Before you do anything, identify slow parts. Do not touch working code which performs fast enough.
The biggest improvement came when I started using AsyncCalls to convert single-threaded applications that used to freeze up the UI, into (sort of) multi-threaded apps.
Although AsyncCalls can do a lot more, I've found it useful for this very simple purpose. Let's say you have a subroutine blocked like this: Disable Button, Do Work, Enable Button.
You move the 'Do Work' part to a local function (call it AsyncDoWork), and add four lines of code:
var a: IAsyncCall;
a := LocalAsyncCall(#AsyncDoWork);
while (NOT a.Finished) do
application.ProcessMessages;
a.Sync;
What this does for you is run AsyncDoWork in a separate thread, while your main thread remains available to respond to the UI (like dragging the window or clicking Abort.) When AsyncDoWork is finished the code continues. Because I moved it to a local function, all local vars are available, an the code does not need to be changed.
This is a very limited type of 'multi-threading'. Specifically, it's dual threading. You must ensure that your Async function and the UI do not both access the same VCL components or data structures. (I disable all controls except the stop button.)
I don't use this to write new programs. It's just a really quick & easy way to make old programs more responsive.
When working with a tstringlist (or similar), set "sorted := false" until needed (if at all). Seems like a no-brainer...
Create unit tests
Verify tests all pass
Profile your application
Refactor looking for bottlenecks and memory
Repeat from Step 2 (comparing to previous pass)
Make intelligent use of SetLength() for strings and arrays. Optimise initialisation with FillChar or ZeroMemory.
Local variables created on stack (e.g. record types) are faster than heap allocated (objects and New()) variables.
Reuse objects rather than Destroy then create. But make sure management code for this is faster than memory manager!
Check heavily-used loops for calculations that could be (at least partially) pre-calculated or handled with a lookup table. Trig functions are a classic for this, but it applies to many others.
If you have a list, use a dynamic array of anything, even a record as follows:
This needs no classes, no freeing and access to it is very fast. Even if it needs to grow you can do this - see below. Only use TList or TStringList if you need lots of size changing flexibility.
type
TMyRec = record
SomeString : string;
SomeValue : double;
end;
var
Data : array of TMyRec;
I : integer;
..begin
SetLength( Data, 100 ); // defines the length and CLEARS ALL DATA
Data[32].SomeString := 'Hello';
ShowMessage( Data[32] );
// Grow the list by 1 item.
I := Length( Data );
SetLength( Data, I+1 );
..end;
Separating the program logic from user interface, refactoring, then optimizing the most-used, most resource-intensive elements independently.
Turn debugging OFF
Turn optimizations ON
Remove all references to units that
you don't actually use
Look for memory leaks
Use a lot of assertions to debug, then turn them off in shipping code.
Turn off range and overflow checking after you have tested extensively.
If you really, really, really need to be light weight then you can shed the VCL. Take a look at the KOL & MCK. Granted if you do that then you are trading features for reduced footprint.
Use the full FastMM and study the documentation and source and see if you can tweak it to your specifications.
For an old BDE development when I first started Delphi, I was using lots of TQuery components. Someone told me to use TTable master-detail after I explained him what I was doing, and that made the program run much faster.
Calling DisableControls can omit unnecessary UI updates.
When identifying records, use integers if at all possible for record comparison. While a primary key of "company name" might seem logical, the time spent generating and storing a hash of this will greatly improve overall search times.
You might consider using runtime packages. This could reduce your memory foot print if there are more then one program running that is written using the same packages.
If you use threads, set their processor affinity. If you don't use threads yet, consider using them, or look into asynchronous I/O (completion ports) if your application does lots of I/O.
Consider if a DBMS database is really the perfect choice. If you are only reading data and never changing it, then a flat fixed record file could work faster, especially if the path to the data can be easily mapped (ie, one index). A trivial binary search on a fixed record file is still extremely fast.
BeginUpdate ... EndUpdate
ShortString vs. String
Use arrays instead of TStrings and TList
But the sad answer is that tuning and optimization will give you maybe 10% improvement (and it's dangerous); re-design can give you 90%. Once you really understand the goal, you often can restate the problem (and therefore the solution) in much better terms.
Cheers
Examine all loops, and look for ways to short circuit. If your looking for something specific and find it in a loop, then use the BREAK command to immediately bail...no sense looping thru the rest. If you know that you don't have a match, then use a CONTINUE as quickly as possible.
Take advantage of some of the FastCode project code. Parts of it were incorporated into VCL/RTL proper (like FastMM was), but there is more out there you can use!
Note, they have a new site they are moving too, but it seems to be a bit inactive.
Consider hardware issues. If you really need performance then consider the type of hard drive(s) your program and your databases are running on. There are a lot of variables especially if you are running a database. RAID is not always the best answer either.