I am learning Go at the moment and got quite frustrated understanding the different between Concurrency vs Parallelism vs Sequential.
Let's say we have a process that scraps a slice of 5 URLs and paste the content in a text file. The process takes 2 seconds per URL.
Sequentially -> it takes 10 seconds since it does one after the other
Parallel -> takes less than 10 seconds since it does them simultaneously but using multiple threads or processors.
Concurrently -> takes less than 10 seconds but it does not require multiple threads or processors.
Not sure if I am right until here. My questions are:
I read that parallelism does things simultaneously (running and listening to music for example) and concurrency handles things at the same time (getting breakfast done while ironing shirts for example).
But if that's the case, why is concurrency not taking 10 seconds to complete since at the end of the day you are not doing everything at the same time but just doing bits of everything until you complete it all?
Here's an analogy to explain.
You need to fry 5 eggs, sunny side up. To cook an egg you crack it onto the griddle, wait for a few minutes, then take it off.
The sequential approach is to fry the first egg to completion, then fry the second egg to completion, and so on, until you have 5 fried eggs.
The parallel approach is to hire 5 cooks, tell each of them to fry an egg, and wait until they are all finished.
The concurrent approach is that you cook all 5 eggs yourself the way you would actually do it. That is, you quickly crack each egg onto the pan, then take each one off when it's ready.
The reason you're able to save time without having to hire 5 cooks is because the number of cooks wasn't what was limiting you from going faster. It takes a couple minutes to cook an egg, but it only occupies your attention and your hands for a few seconds at the beginning and end.
The Go runtime and modern OS runtimes are similarly smart. They know that while your thread is waiting to receive a network response, the processor can look for other things to occupy it's attention.
The larger picture of concurrency is concerned not primarily with the number of processors, but with resource contention in general. The execution of tasks demands resources, and we cannot use more resources than are available. Processors are one resource, but there is also memory storage, memory bandwidth, network bandwidth, file handles, and the list goes on.
If in a Near contract I write to a key in storage, then delete that same key all in one transaction, is the gas cost less than if I were to write to the key and delete it in separate transactions?
The reasoning behind it possibly being cheaper is, on the level of the runtime, no change to storage needs to be made if the state is the same before and after the transaction (the intermediate state could be in memory internally)
Currently, the cost is the same. The general rule for smart contract fees is to deduct the gas immediately before the operation occurs, because otherwise if we have delayed cost of a certain operation than someone can create a contract that attempts 10,000 (or some other large number) of operations while it has attached gas for only one. Then once fees are attempted to be deducted the contract will fail, however it will incur 10,000 more CPU cost than what it actually paid for, which can be used to "grind" the blockchain by issuing cheap transactions that congest all the blocks. Even worse, it will break the invariant that the burnt gas reflects the execution time of the block. In this case our blocks will start taking far more than 1 sec to execute creating complex network effects in the consensus.
Storage operations are special compared to other contract operations, because the bulk of CPU computation (writing state into Rocksdb and recomputing merkle hashes) occurs not at the moment when contract call the host function, but when the chunk is finalized. This means that we can delay deduction of the storage costs until the very end of the contract execution, and then if the writing of the key-value was reverted by deletion we can avoid deducting a large fee. There are however caveats:
The operation of writing key-value still need to cost something, because just calling a host function and "scheduling" a write of a key-value still incurs some CPU costs, even if they are small. Therefore we need to split key-value fees into two categories: immediate and delayed. This will require a protocol upgrade and will make runtime more complex;
DevX can become more unpredictable because we are breaking the common-sense invariant. Currently, the contract can requests how much gas it has burnt with env::used_gas() and developers know that this number can be only increasing. If we make key-value costs lazy, this number will either be sometimes decreasing (when key-value is deleted) or it won't reflect the gas that is about to be burnt;
So protocol upgrade can be considered to make key-value write cost delayed (as long as it follows https://gov.near.org/t/quality-control-for-protocol-api-changes/1941) but there needs to be a proper justification (e.g. EVM will be much cheaper) since there is going to be a trade-off: we will be improving performance at the cost of making runtime more complex (and thus potentially impeding our velocity) and making DevX more confusing.
How many concurrent players that are close to each other can a very well written MMO handle? Let's assume basic actions such as:
Point and click movement
Casting only targetable spells (no aoe spells on the ground)
Any other basic point and click interactions
Can it easily propagate such simple interactions to 1000? 10000? or perhaps more concurrent players?
Where is the bottleneck that comes in? I used to play a game Lineage 2 back in 20xx, and often it's the client that was using a single CPU that was the problem. Are there more significant issues at the server level, or with today's technology, for simple interactions it is possible to build a single instanced game, on a single machine for 10000 players? What are the other possible bottlenecks?
I read that FPGA is suited for algorithms which are parallel or which can be pipelined. What is by definition an algorithm that can be pipelined?
It means you can split a task T into multiple steps T1, T2,...,Tn and each of these steps is more or less independent. Now the data is first injected into a processor P1 that performs task T1 on it, after a time step the results of P1 are transferred to P2 where task T2 is executed. The point is that at that time, processor P1 is again available, so you load the next chunk of data that needs to be processed into processor P1. You can compare it to an assembly line where each worker (processor) does his/her part in a large process. A process that can be pipelined is efficient because the time it takes to process n chunks of data scales with n but still requires the same amount of hardware as if you would process only one chunk of data (evidently there will be some overhead in order to organize this).
Note that with processor, I do not mean a physical processor (like an 80x86), I simply mean a device that can do a certain job. Whether it requires an instruction set, memory, clock cycles, etc. is irrelevant.
Not all algorithms can be pipelined because sometimes there are dependencies between data which makes it hard/impossible to let the data be processed in chunks: you need all data available at once, or you cannot process it (or at least not efficiently).
As #Paebbels says (see comment below)
Such processors or processing elements (PEs) or processing units (PUs) can be implemented in FPGAs. It's possible to map a network of PEs onto the FPGA area especially when many bit operation or non power of 2 data types are required. FPGAs perform mostly bad if floating point operations or fast DRAM access is required. Then GPUs or standard CPUs might be faster. Note: FPGAs are mounted onto PCIe cards so even a x100 faster algorithm can be slower compared to CPU algorithms, because latency or PCIe transfer rates will eat all benefits.
The point is nevertheless to achieve speedup without a substantial increase of hardware cost.
Analogy
In my course text of digital electronics, they made an analogy with a launderette. Say you want to do your laundry. Now evidently you cannot put all these clothes in the washing machine and dryer at once: you need to split it into ten parts.
Now say you have a machine that acts as both a washing machine and a dryer. And it requires two time steps to do the washing and drying. Then it would require twenty time steps to do your laundry, and you use a single machine.
A solution is to hire ten washing machines and ten dryers. Put all the clothes in the washing machines, then when it is done put all the clothes in the dryers and you are done in two steps. The downside is that you need to hire ten washing machines and dryers.
A solution using pipelining is that you hire one washing machine, and one dryer. Now you put the first batch of clothes in the washing machine. When done you put the washed clothes in the dryer, but in the meantime you put a next batch of clothes into the washing machine. Thus the washing machine and the dryer (processors) work in parallel, but at a different chuck of clothes (data). At each time step, you remove the clothes from the dryer, put clothes from the washing machine in the dryer, and put a new batch of clothes into the washing machine. As a result you will have eleven time steps, but only have to hire one washing machine and one dryer. When it comes to costs and time, pipelining can be efficient.
Well looks too simple a question to be asked but i asked after going through few ppts on both.
Both methods increase instruction throughput. And Superscaling almost always makes use of pipelining as well. Superscaling has more than one execution unit and so does pipelining or am I wrong here?
Superscalar design involves the processor being able to issue multiple instructions in a single clock, with redundant facilities to execute an instruction. We're talking about within a single core, mind you -- multicore processing is different.
Pipelining divides an instruction into steps, and since each step is executed in a different part of the processor, multiple instructions can be in different "phases" each clock.
They're almost always used together. This image from Wikipedia shows both concepts in use, as these concepts are best explained graphically:
Here, two instructions are being executed at a time in a five-stage pipeline.
To break it down further, given your recent edit:
In the example above, an instruction goes through 5 stages to be "performed". These are IF (instruction fetch), ID (instruction decode), EX (execute), MEM (update memory), WB (writeback to cache).
In a very simple processor design, every clock a different stage would be completed so we'd have:
IF
ID
EX
MEM
WB
Which would do one instruction in five clocks. If we then add a redundant execution unit and introduce superscalar design, we'd have this, for two instructions A and B:
IF(A) IF(B)
ID(A) ID(B)
EX(A) EX(B)
MEM(A) MEM(B)
WB(A) WB(B)
Two instructions in five clocks -- a theoretical maximum gain of 100%.
Pipelining allows the parts to be executed simultaneously, so we would end up with something like (for ten instructions A through J):
IF(A) IF(B)
ID(A) ID(B) IF(C) IF(D)
EX(A) EX(B) ID(C) ID(D) IF(E) IF(F)
MEM(A) MEM(B) EX(C) EX(D) ID(E) ID(F) IF(G) IF(H)
WB(A) WB(B) MEM(C) MEM(D) EX(E) EX(F) ID(G) ID(H) IF(I) IF(J)
WB(C) WB(D) MEM(E) MEM(F) EX(G) EX(H) ID(I) ID(J)
WB(E) WB(F) MEM(G) MEM(H) EX(I) EX(J)
WB(G) WB(H) MEM(I) MEM(J)
WB(I) WB(J)
In nine clocks, we've executed ten instructions -- you can see where pipelining really moves things along. And that is an explanation of the example graphic, not how it's actually implemented in the field (that's black magic).
The Wikipedia articles for Superscalar and Instruction pipeline are pretty good.
A long time ago, CPUs executed only one machine instruction at a time. Only when it was completely finished did the CPU fetch the next instruction from memory (or, later, the instruction cache).
Eventually, someone noticed that this meant that most of a CPU did nothing most of the time, since there were several execution subunits (such as the instruction decoder, the integer arithmetic unit, and FP arithmetic unit, etc.) and executing an instruction kept only one of them busy at a time.
Thus, "simple" pipelining was born: once one instruction was done decoding and went on towards the next execution subunit, why not already fetch and decode the next instruction? If you had 10 such "stages", then by having each stage process a different instruction you could theoretically increase the instruction throughput tenfold without increasing the CPU clock at all! Of course, this only works flawlessly when there are no conditional jumps in the code (this led to a lot of extra effort to handle conditional jumps specially).
Later, with Moore's law continuing to be correct for longer than expected, CPU makers found themselves with ever more transistors to make use of and thought "why have only one of each execution subunit?". Thus, superscalar CPUs with multiple execution subunits able to do the same thing in parallel were born, and CPU designs became much, much more complex to distribute instructions across these fully parallel units while ensuring the results were the same as if the instructions had been executed sequentially.
An Analogy: Washing Clothes
Imagine a dry cleaning store with the following facilities: a rack for hanging dirty or clean clothes, a washer and a dryer (each of which can wash one garment at a time), a folding table, and an ironing board.
The attendant who does all of the actual washing and drying is rather dim-witted so the store owner, who takes the dry cleaning orders, takes special care to write out each instruction very carefully and explicitly.
On a typical day these instructions may be something along the lines of:
take the shirt from the rack
wash the shirt
dry the shirt
iron the shirt
fold the shirt
put the shirt back on the rack
take the pants from the rack
wash the pants
dry the pants
fold the pants
put the pants back on the rack
take the coat from the rack
wash the coat
dry the coat
iron the coat
put the coat back on the rack
The attendant follows these instructions to the tee, being very careful not to ever do anything out of order. As you can imagine, it takes a long time to get the day's laundry done because it takes a long time to fully wash, dry, and fold each piece of laundry, and it must all be done one at a time.
However, one day the attendant quits and a new, smarter, attendant is hired who notices that most of the equipment is laying idle at any given time during the day. While the pants were drying neither the ironing board nor the washer were in use. So he decided to make better use of his time. Thus, instead of the above series of steps, he would do this:
take the shirt from the rack
wash the shirt, take the pants from the rack
dry the shirt, wash the pants
iron the shirt, dry the pants
fold the shirt, (take the coat from the rack)
put the shirt back on the rack, fold the pants, (wash the coat)
put the pants back on the rack, (dry the coat)
(iron the coat)
(put the coat back on the rack)
This is pipelining. Sequencing unrelated activities such that they use different components at the same time. By keeping as much of the different components active at once you maximize efficiency and speed up execution time, in this case reducing 16 "cycles" to 9, a speedup of over 40%.
Now, the little dry cleaning shop started to make more money because they could work so much faster, so the owner bought an extra washer, dryer, ironing board, folding station, and even hired another attendant. Now things are even faster, instead of the above, you have:
take the shirt from the rack, take the pants from the rack
wash the shirt, wash the pants, (take the coat from the rack)
dry the shirt, dry the pants, (wash the coat)
iron the shirt, fold the pants, (dry the coat)
fold the shirt, put the pants back on the rack, (iron the coat)
put the shirt back on the rack, (put the coat back on the rack)
This is superscalar design. Multiple sub-components capable of doing the same task simultaneously, but with the processor deciding how to do it. In this case it resulted in a nearly 50% speed boost (in 18 "cycles" the new architecture could run through 3 iterations of this "program" while the previous architecture could only run through 2).
Older processors, such as the 386 or 486, are simple scalar processors, they execute one instruction at a time in exactly the order in which it was received. Modern consumer processors since the PowerPC/Pentium are pipelined and superscalar. A Core2 CPU is capable of running the same code that was compiled for a 486 while still taking advantage of instruction level parallelism because it contains its own internal logic that analyzes machine code and determines how to reorder and run it (what can be run in parallel, what can't, etc.) This is the essence of superscalar design and why it's so practical.
In contrast a vector parallel processor performs operations on several pieces of data at once (a vector). Thus, instead of just adding x and y a vector processor would add, say, x0,x1,x2 to y0,y1,y2 (resulting in z0,z1,z2). The problem with this design is that it is tightly coupled to the specific degree of parallelism of the processor. If you run scalar code on a vector processor (assuming you could) you would see no advantage of the vector parallelization because it needs to be explicitly used, similarly if you wanted to take advantage of a newer vector processor with more parallel processing units (e.g. capable of adding vectors of 12 numbers instead of just 3) you would need to recompile your code. Vector processor designs were popular in the oldest generation of super computers because they were easy to design and there are large classes of problems in science and engineering with a great deal of natural parallelism.
Superscalar processors can also have the ability to perform speculative execution. Rather than leaving processing units idle and waiting for a code path to finish executing before branching a processor can make a best guess and start executing code past the branch before prior code has finished processing. When execution of the prior code catches up to the branch point the processor can then compare the actual branch with the branch guess and either continue on if the guess was correct (already well ahead of where it would have been by just waiting) or it can invalidate the results of the speculative execution and run the code for the correct branch.
Pipelining is what a car company does in the manufacturing of their cars. They break down the process of putting together a car into stages and perform the different stages at different points along an assembly line done by different people. The net result is that the car is manufactured at exactly the speed of the slowest stage alone.
In CPUs the pipelining process is exactly the same. An "instruction" is broken down into various stages of execution, usually something like 1. fetch instruction, 2. fetch operands (registers or memory values that are read), 2. perform computation, 3. write results (to memory or registers). The slowest of this might be the computation part, in which case the overall throughput speed of the instructions through this pipeline is just the speed of the computation part (as if the other parts were "free".)
Super-scalar in microprocessors refers to the ability to run several instructions from a single execution stream at once in parallel. So if a car company ran two assembly lines then obviously they could produce twice as many cars. But if the process of putting a serial number on the car was at the last stage and had to be done by a single person, then they would have to alternate between the two pipelines and guarantee that they could get each done in half the time of the slowest stage in order to avoid becoming the slowest stage themselves.
Super-scalar in microprocessors is similar but usually has far more restrictions. So the instruction fetch stage will typically produce more than one instruction during its stage -- this is what makes super-scalar in microprocessors possible. There would then be two fetch stages, two execution stages, and two write back stages. This obviously generalizes to more than just two pipelines.
This is all fine and dandy but from the perspective of sound execution both techniques could lead to problems if done blindly. For correct execution of a program, it is assumed that the instructions are executed completely one after another in order. If two sequential instructions have inter-dependent calculations or use the same registers then there can be a problem, The later instruction needs to wait for the write back of the previous instruction to complete before it can perform the operand fetch stage. Thus you need to stall the second instruction by two stages before it is executed, which defeats the purpose of what was gained by these techniques in the first place.
There are many techniques use to reduce the problem of needing to stall that are a bit complicated to describe but I will list them: 1. register forwarding, (also store to load forwarding) 2. register renaming, 3. score-boarding, 4. out-of-order execution. 5. Speculative execution with rollback (and retirement) All modern CPUs use pretty much all these techniques to implement super-scalar and pipelining. However, these techniques tend to have diminishing returns with respect to the number of pipelines in a processor before stalls become inevitable. In practice no CPU manufacturer makes more than 4 pipelines in a single core.
Multi-core has nothing to do with any of these techniques. This is basically ramming two micro-processors together to implement symmetric multiprocessing on a single chip and sharing only those components which make sense to share (typically L3 cache, and I/O). However a technique that Intel calls "hyperthreading" is a method of trying to virtually implement the semantics of multi-core within the super-scalar framework of a single core. So a single micro-architecture contains the registers of two (or more) virtual cores and fetches instructions from two (or more) different execution streams, but executing from a common super-scalar system. The idea is that because the registers cannot interfere with each other, there will tend to be more parallelism leading to fewer stalls. So rather than simply executing two virtual core execution streams at half the speed, it is better due to the overall reduction in stalls. This would seem to suggest that Intel could increase the number of pipelines. However this technique has been found to be somewhat lacking in practical implementations. As it is integral to super-scalar techniques, though, I have mentioned it anyway.
Pipelining is simultaneous execution of different stages of multiple instructions at the same cycle. It is based on splitting instruction processing into stages and having specialized units for each stage and registers for storing intermediate results.
Superscaling is dispatching multiple instructions (or microinstructions) to multiple executing units existing in CPU. It is based thus on redundant units in CPU.
Of course, this approaches can complement each other.