Firstly, I'd like to mention I wasn't sure where this question should go. It may be better of in a sister site but I didn't know which one therefore if it is in the wrong category, would someone be able to migrate it?
I'm conducting some research into how heterogeneous architectures work in terms of the short term scheduling queue. I know its a collection of different processors such as CPU's and GPU's.
I also found out some information regarding asymmetric scheduling. I know homogeneous uses symmetrical, but struggling to understand how heterogeneous works?
Related
I am looking to implement an automated way of allocating processes to a variety of servers available. There are many types of servers (characterized by things like location, cpu, network card, etc..) and there are various types of processes (more than there are servers) with different priorities and location/hardware requirements. I can think of pretty much greedy algorithms that are simplistic in nature but was wondering what other references and approaches exist for this type of problem (which I feel is pretty standard). I am also interested in solving a related problem - in which say we remove one of the servers after things have been allocated and we need to reshuffle with minimal interference. This latter one I also feel is standard but I'm not sure what some good references to look at are. Any suggestions on where to start?
Your question is pretty vague. Normally problems like this are handled either by modeling them as a set of linear equations and optimizing an objective function given the linear constraints, or the problem is modeled as a knapsack problem.
I have been asked to compare the programming models used by two different OSs for wireless sensor networks, TinyOS (which uses an event-based model) and Contiki (which uses events internally, but offers a protothread model for application programmers). I have developed the same application in both systems, and I can present a qualitative analysis of the pros and cons of both models, and give my subjective impression.
However, I have been asked to put forward metrics for comparing them. Apart from the time spent to write the programs (which is roughly equal), I'm not sure what other metrics are applicable. Can you suggest some?
Time to understand these programs? Number of questions ask on net about deadlocks (normalized by userbase)
I ended up using lines of code and cyclomatic complexity to show how different models impact code organization. I also estimated the difficulty of understanding the two programs by asking another programmer to read them.
Are there standard rules engine/algorithms around AI that would predict the user taste on a particular kind of product like clothes.
I know it's one thing all e-commerce website will kill for. But I am looking out for theoretical patterns defined out there which would help make that prediction in a better way, if not accurately.
Two books that cover recommender systems:
Programming Collective Intelligence: Python, does a good job explaining the algorithm, but doesn't provide enough help IMO in terms of understanding how to scale.
Algorithms of the Intelligent Web: Java, harder to follow, but also covers using persistence, in this case MySQL, to facilitate scaling and identifiers areas in example code that will not scale as-is.
Basically two ways of approaching the problem, user or item based. Netflix appears to use the former, while Amazon the latter. Typically user based requires more time and/or processing power to generate recommendations because you tend to have more users than items to consider.
Not sure how to answer this, as this question is overly broad. What you are describing is a Machine Learning kind of task, and thus would fall under that (very broad) umbrella. There are a number of different algorithms that can be used for something like this, but most texts would tell you that the definition of the problem is the important part.
What parts of fashion are important? What parts are not? How are you going to gather the data? How noisy is the data? All of these are important considerations to the problem space. Pandora does a similar type of thing with music, with their big benefit being that their users tell them initially what they like and don't like.
To categorize their music, they actually have trained musicians listening to the music to identify all sorts of stuff. See the article on Ars Technica here for more information about that. Based on what I know about fashion tastes, I would say that it is a similar problem space, and would probably require experts to "codify" the information before you could attempt to draw parallels.
Sorry for the vague answer - if you want more specifics, I would recommend asking a more specific question, about specific algorithms or data sets, etc.
I'm looking for information on things like ehcache and other alternatives to memcached for a project that will likely involve 3-4 webservers and something like 2-10 million distributed objects that need to be available to all servers.
Specifically, I'm trying to understand how other systems distribute data, whether or not memcached is unique in distributing data among multiple caches, or other caches perform similarly (that is, the property that a given key may exist on any of N servers, and the clients don't care, as opposed to updates on a single server propagating to other caches that essentially act as copies).
For example, in looking at documentation for things like ehcache it's not clear to me if by "distributed" they mean a strategy similar to memcached or something more like "replicated/synchronized".
Edit: although the refs on distributed computing are useful, I'm more interested in how specific implementations behave. e.g. will I be paying for synchronization overhead in some systems?
You are not extremely precise in your question, although I might see where you want to go this is a pretty large field in itself.
You might want to start here: http://www.metabrew.com/article/anti-rdbms-a-list-of-distributed-key-value-stores/
Also having a look at Dynamo, BigTable, and all the theoritical questions associated with this (CAP theorem and the presentation by Werner Vogels on this that you can find on infoq).
You have more and more information about this thanks to the multiple videos found about the NoSQL meetups.
Hope it helps,
Edit: about the synchronization overheads, it really depends on the system. every system has specific requirements, Dynamo for example aims at a high availability system that might not be always fully consistent (eventual consistency), so it is meant (by design and because of its requirements) to be a distributed systems in which every write must be accepted and fast. Other systems might behave differently,
I suspect you are after a discussion on consistency across "distributed data". This topic is vast but a good reference on the trade-offs is available here.
In other words, it pretty much depends on your requirements (which aren't very detailed here). If I have misunderstood your question, you can safely disregard my contribution ;-)
The feature or property you are probably looking for is a "shared nothing" architecture. Memcached is an example, e. g. there is no single point of failure, no synchronization or any other traffic between nodes, nodes don't even know each other.
So if this is what you want and you're evaluating a product/project, look for the "shared nothing" term. If it is not mentioned on the first screen, it probably is not a shared nothing architecture ;)
I've read Lamport's paper on Paxos. I've also heard that it isn't used much in practice, for reasons of performance. What algorithms are commonly used for consensus in distributed systems?
Not sure if this is helpful (since this is not from actual production information), but in our "distributed systems" course we've studied, along with Paxos, the Chandra-Toueg and Mostefaoui-Raynal algorithms (of the latter our professor was especially fond).
Check out the Raft algorithm for a consensus algorithm that is optimized for ease of understanding and clarity of implementation. Oh... it is pretty fast as well.
https://ramcloud.stanford.edu/wiki/display/logcabin/LogCabin
https://ramcloud.stanford.edu/wiki/download/attachments/11370504/raft.pdf
If performance is an issue, consider whether you need all of the strong consistency guarantees Paxos gives you. See e.g. http://queue.acm.org/detail.cfm?id=1466448 and http://incubator.apache.org/cassandra/. Searching on Paxos optimised gets me hits, but I suspect that relaxing some of the requirements will buy you more than tuning the protocol.
The Paxos system I run (which supports really, really big web sites) is halfway in-between Basic-Paxos Multi-paxos. I plan on moving it to a full Multi-Paxos implementation.
Paxos isn't that great as a high-throughput data storage system, but it excels in supporting those systems by providing leader election. For example, say you have a replicated data store where you want a single master for performance reasons. Your data store nodes will use the Paxos system to choose the master.
Like Google Chubby, my system is run as a service and can also store data as configuration container. (I use configuration loosely; I hear Google uses Chubby for DNS.) This data doesn't change as often as user input so it doesn't need high throughput write SLAs. Reading, on the other hand, is extremely quick because it is fully replicated and you can read from any node.
Update
Since writing this, I have upgraded my Paxos system. I am now using a chain-consensus protocol as the primary consensus system. The chain system still utilizes Basic-Paxos for re-configuration—including notifying chain nodes when the chain membership changes.
Paxos is optimal in terms of performance of consensus protocols, at least in terms of the number of network delays (which is often the dominating factor). It's clearly not possible to reliably achieve consensus while tolerating up to f failures without a single round-trip communication to at least (f-1) other nodes in between a client request and the corresponding confirmation, and Paxos achieves this lower bound. This gives a hard bound on the latency of each request to a consensus-based protocol regardless of implementation. In particular, Raft, Zab, Viewstamped Replication and all other variants on consensus protocols all have the same performance constraint.
One thing that can be improved from standard Paxos (also Raft, Zab, ...) is that there is a distinguished leader which ends up doing more than its fair share of the work and may therefore end up being a bit of a bottleneck. There is a protocol known as Egalitarian Paxos which spreads the load out across multiple leaders, although it's mindbendingly complicated IMO, is only applicable to certain domains, and still must obey the lower bound on the number of round-trips within each request. See the paper "There Is More Consensus in Egalitarian Parliaments" by Moraru et al for more details.
When you hear that Paxos is rarely used due to its poor performance, it is frequently meant that consensus itself is rarely used due to poor performance, and this is a fair criticism: it is possible to achieve much higher performance if you can avoid the need for consensus-based coordination between nodes as much as possible, because this allows for horizontal scalability.
Snarkily, it's also possible to achieve better performance by claiming to be using a proper consensus protocol but actually doing something that fails in some cases. Aphyr's blog is littered with examples of these failures not being as rare as you might like, where database implementations have either introduced bugs into good consensus-like protocols by way of "optimisation", or else developed custom consensus-like protocols that fail to be fully correct in some subtle fashion. This stuff is hard.
You should check the Apache Zookeeper project. It is used in production by Yahoo! and Facebook among others.
http://hadoop.apache.org/zookeeper/
If you look for academic papers describing it, it is described in a paper at usenix ATC'10. The consensus protocol (a variant of Paxos) is described in a paper at DSN'11.
Google documented how they did fast paxos for their megastore in the following paper: Link.
With Multi-Paxos when the leader is galloping it can respond to the client write when it has heard that the majority of nodes have written the value to disk. This is as good and efficient as you can get to maintain the consistency guarantees that Paxos makes.
Typically though people use something paxos-like such as zookeeper as an external service (dedicated cluster) to keep critical information consistent (who has locked what, who is leader, who is in a cluster, what's the configuration of the cluster) then run a less strict algorithm with less consistency guarantees which relies upon application specifics (eg vector clocks and merged siblings). The short ebook distributed systems for fun and profit as a good overview of the alternatives.
Note that lots of databases compete on speed by using risky defaults which risk consistency and can loose data under network partitions. The Aphry blog series on Jepson shows whether well know opensouce systems loose data. One cannot cheat the CAP Theorem; if you configure systems for safety then they end up doing about the same messaging and same disk writes as paxos. So really you cannot say Paxos is slow you have to say "a part of a system which needs consistency under network partitions requires a minimum number of messages and disk flushes per operation and that is slow".
There are two general blockchain consensus systems:
Those that produce unambiguous 100% finality given a defined set of
validators
Those which do not provide 100% finality but instead
rely on high probability of finality.
The first generation blockchain consensus algorithms (Proof of Work, Proof of Stake, and BitShares’ Delegated Proof of Stake) only offer high probability of finality that grows with time. In theory someone could pay enough money to mine an alternative “longer” Bitcoin blockchain that goes all the way back to genesis.
More recent consensus algorithms, whether HashGraph, Casper, Tendermint, or DPOS BFT all adopt long-established principles of Paxos and related consensus algorithms. Under these models it is possible to reach unambiguous finality under all network conditions so long as more than ⅔ of participants are honest.
Objective and unambiguous 100% finality is a critical property for all blockchains that wish to support inter-blockchain communication. Absent 100% finality, a reversion on one chain could have irreconcilable ripple effects across all interconnected chains.
The abstract protocol for these more recent protocols involves:
Propose block
All participants acknowledge block (pre-commitment)
All participants acknowledge when ⅔+ have sent them pre-commitments
(commitment)
A block is final once a node has received ⅔+ commitments
Unanimous agreement on finality is guaranteed unless ⅓+
are bad and evidence of bad behavior is available to all
It is the technical differences in the protocols that give rise to real-world impact on user experience. This includes things such as latency until finality, degrees of finality, bandwidth, and proof generation / validation overhead.
Look for more details on delegated proof of stake by eos here
Raft is more understandable, and faster alternative of Paxos. One of the most popular distributed systems which uses Raft is Etcd. Etcd is the distributed store used in Kubernetes.
It's equivalent to Paxos in fault-tolerance.