Pyro vs Pymc? What are the difference between these Probabilistic Programming frameworks? - pymc

I used 'Anglican' which is based on Clojure, and I think that is not good for me. Bad documents and a too small community to find help. Also, I still can't get familiar with the Scheme-based languages. So I want to change the language to something based on Python.
Maybe Pyro or PyMC could be the case, but I totally have no idea about both of those.
What are the difference between the two frameworks?
Can they be used for the same problems?
Are there examples, where one shines in comparison?

(Updated for 2022)
Pyro is built on PyTorch. It has full MCMC, HMC and NUTS support. It has excellent documentation and few if any drawbacks that I'm aware of.
PyMC was built on Theano which is now a largely dead framework, but has been revived by a project called Aesara. PyMC3 is now simply called PyMC, and it still exists and is actively maintained. Its reliance on an obscure tensor library besides PyTorch/Tensorflow likely make it less appealing for widescale adoption--but as I note below, probabilistic programming is not really a widescale thing so this matters much, much less in the context of this question than it would for a deep learning framework.
There still is something called Tensorflow Probability, with the same great documentation we've all come to expect from Tensorflow (yes that's a joke). My personal opinion as a nerd on the internet is that Tensorflow is a beast of a library that was built predicated on the very Googley assumption that it would be both possible and cost-effective to employ multiple full teams to support this code in production, which isn't realistic for most organizations let alone individual researchers.
That said, they're all pretty much the same thing, so try them all, try whatever the guy next to you uses, or just flip a coin. The best library is generally the one you actually use to make working code, not the one that someone on StackOverflow says is the best. As for which one is more popular, probabilistic programming itself is very specialized so you're not going to find a lot of support with anything.

From here
Pyro is a deep probabilistic programming language that focuses on
variational inference, supports composable inference algorithms.
Pyro aims to be more dynamic (by using PyTorch) and universal
(allowing recursion).
Pyro embraces deep neural nets and currently focuses on variational inference. Pyro doesn't do Markov chain Monte Carlo (unlike PyMC and Edward) yet.
Pyro is built on pytorch whereas PyMC3 on theano. So you get PyTorch’s dynamic programming and it was recently announced that Theano will not be maintained after an year. However, I found that PyMC has excellent documentation and wonderful resources. Another alternative is Edward built on top of Tensorflow which is more mature and feature rich than pyro atm. Authors of Edward claim it's faster than PyMC3.
I guess the decision boils down to the features, documentation and programming style you are looking for.

Related

Encryption benchmark test

I am working on a project where I am going to performance compare encryption algorithms. How should I accomplish this with software? I have found Crypto++ but I am not familiar with C++. Crypto++ also feels way too advanced for me. Is there any simplier software to use since im only a high school student?
This can be quite challenging if you don't understand how the different algorithms are put together, and so what configuration of each algorithm is appropriate. On many modern systems it will also tend to bias towards AES, since some chips have AES primitives built in (particularly if you're using an optimized library). On the other hand, a naïve C implementation of the algorithm may be unrepresentative of real-world, optimized implementations that most people use in real life. On the other hand, a good implementation may be slower than a poor implementation because it's maintaining constant-time (and/or constant-power) performance, which is important to security.
Now that I've told you some of the problems, it's still not a bad high school project and you can learn a lot doing it (just make sure that when you publish your results, you are clear that you don't really know what you're doing, so no one should take these as definitive benchmarks).
The best toolkit for this IMO is BouncyCastle. It's in Java, which is relatively easy to learn (compared to some of the fancy C++ in Crypto++). It has a fairly approachable API. It has a lot of different algorithms available. And it's a real-world library that professional developers use every day.
Learning your way around BC, even if you use it incorrectly (which you will; crypto is very hard to do correctly, even with a library), is likely to be a good learning experience, so go for it.

What language would be more efficient to implement a graph library?

Brief Description
I have a college work where I have to implement a graph library (I'll have to give a presentation about this work later)
The basic idea is to write all the code of the data structures and their algorithms from scratch, using the tools provided by some programming language, like C/C++, Java, Python, doesn't really matter which one of them I'll pick at first.
But I should not use any built-in graph libraries in the language: the goal of the work is to make the students learn how these algorithms work. There are some test cases which my program will be later submitted to.
It is not really necessary but, if you wanna take a look, here is the homework assignment: http://pastebin.com/GdtvMTMR (I used Control-C Control-V plus google translate from a LaTeX text, this is why the formatting is poor).
The Question
So, my question is: which programming language would be more time efficient to implement this library?
It doesn't really matter if the language is functional, structured or object oriented. My priority is time efficiency and performance.
The better language is the one you know more.
But if you're looking for some performance, take a look at compiled languages with optimisations. Keep in mind that the code you write is the major component responsible in final performance, the language itself cant do miracles.
A more low level language give to you controls but requires deeply knowledge of the language and the machine you're running your code, so it's a tradeoff.
By a personal choose I would recommend C/C++ to implement a graph library. I've already done this in the past and I used vanilla ANSI C and the performance was awesome.
The one you feel more passionate about and feel more comfortable coding with.
This way you will rock your project.
Myself would pick Java.

is Boost.Geometry mature enough?

I've been recently hired by a GIS company to rewrite their old geomatics libraries. So I'm currently looking for a good computational geometry library. I've seen CGAL, which is amazing, but my boss wants something free.
So I'm now checking Boost.Geometry. This library seems great, but it also seems that it is changing pretty fast. A lot of things are not implemented yet, and a lot of issues are discussed on the mailing list.
Hence my question : is Boost.Geometry mature enough so I can build something on it ? Or is the design still going to evolve ?
Thanks
Depending on your needs, you could look at GEOS, which is used by PostGIS for its geometry operations, among others, and is more mature than boost::geometry. However, it is hard to extend, which you will have to do yourself if you want support for more than Simple Features.
Where boost::geometry shines, in my opinion, is its ability to adapt to existing (legacy) libraries, which means that you should be able to keep some of your employer's data structures (or any other existing geometric library), and build upon them with boost::geometry.
Boost::geometry is gaining maturity and extended features pretty fast, but I don't think the design has changed much since last year (when it was accepted into boost). I would ask on the mailing list -- the authors are very active and helpful.
It depends on your needs.
Boost is one of the most efficient, performant and mature c++ libraries, but when it comes to geometries geos is the best, i tried both of them in a 3D data optimization project.
Boost-Geometry Pros :
-Performant
-Well designed API
Geos Pros :
-Accurate
-Mature

How can this linear solver be linked within Mathematica?

Here is a good linear solver named GotoBLAS. It is available for download and runs on most computing platforms. My question is, is there an easy way to link this solver with the Mathematica kernel, so that we can call it like LinearSolve? One thing most of you may agree on for sure is that if we have a very large Linear system then we better get it solved by some industry standard Linear solver. The inbuilt solver is not meant for really large problems.
Now that Mathematica 8 has come up with better compilation and library link capabilities we can expect to use some of those solvers from within Mathematica. The question is does that require little tuning of the source code, or you need to be an advanced wizard to do it. Here in this forum we may start linking some excellent open source programs like GotoBLAS with Mathematica and exchange our views. Less experienced people can get some insight from the pro users and at the end we get a much stronger Mathematica. It will be an open project for the ever increasing Mathematica community and a platform where these newly introduced capabilities of Mathematica 8 could be transparently documented for future users.
I hope some of you here will give solid ideas on how we can get GotoBLAS running from within Mathematica. As the newer compilation and library link capabilities are usually not very well documented, they are not used by the common users very often. This question can act as a toy example to document these new capabilities of Mathematica. Help in this direction by the experienced forum members will really lift the motivation of new users like me as well as it will teach us a very useful thing to extend Mathematica's number crunching arsenal.
The short answer, I think, is that this is not something you really want to do.
GotoBLAS, as I understand it, is a specific implementation of BLAS, which stands for Basic Linear Algebra Subroutines. "Basic" really means quite basic here - multiply a matrix times a vector, for example. Thus, BLAS is not a solver that a function like LinearSolve would call. LinearSolve would (depending on the exact form of the arguments) call a LAPACK command, which is a higher level package built on top of BLAS. Thus, to really link GotoBLAS (or any BLAS) into Mathematica, one would really need to recompile the whole kernel.
Of course, one could write a C/Fortran program that was compiled against GotoBLAS and then link that into Mathematica. The resulting program would only use GotoBLAS when running whatever specific commands you've linked into Mathematica, however, which rather misses the whole point of BLAS.
The Wolfram Kernel (Mathematica) is already linked to the highly-optimized Intel Math Kernel Library, and is distributed with Mathematica. The MKL is multithreaded and vectorized, so I'm not sure what GotoBLAS would improve upon.

Priority of learning programming craft and other suggestions [closed]

Closed. This question is opinion-based. It is not currently accepting answers.
Want to improve this question? Update the question so it can be answered with facts and citations by editing this post.
Closed 9 years ago.
Improve this question
As I am in my starting career year in software development (C++ & C#) I now see my flaws and what I miss in this sphere. Because of that I came into some conclusions and made myself a plan to fill those gaps and increase my knowledge in software development. But the question I stumbled upon after making a tasks which I need to do has not quite obvious answer to me. What is the priority of those tasks? Here are these tasks and my priority by numbering:
Learning:
Functional programming (Scala)
Data structures & Algorithms (Cormen book to the rescue + TopCoder/ProjectEuler/etc)
Design patterns (GOF or Head First)
Do you agree with this tasks and priorities? Or do I miss something here? Any suggestions are welcome!
I think you have it backwards. Start with design patterns, which will help you reduce the amount messy code you produce, and understand better code made by other people (particularly libraries written with design patterns in mind).
In addition to the book of four, there are many other design pattern books -- Patterns of Enterprise Application Architecture, for example. It might be worth looking at them after you get a good grounding. But I also highly recommend Domain Driven Design, which I think gives you a way of thinking about how to structure your program, instead of just identifying pieces here and there.
Next you can go with algorithms. I prefer Skiena's The Algorithm Design Manual, whose emphasis is more on getting people to know how to select and use algorithms, as well as building them from well known "parts" than on getting people to know to make proofs about algorithms. It is also available for Kindle, which was useful to me.
Also, get a good data structures book -- people often neglect that. I like the Handbook of Data Structures and Applications, though I'm also looking into Advanced Data Structures.
However, I cannot recommend either TopCoder or Euler for this task. TopCoder is, imho, mostly about writing code fast. Nothing bad about it, but it's hardly likely to make a difference on day-to-day stuff. If you like it, by all means do it. Also, it's excellent preparation for job interviews with the more technically minded companies.
Project Euler, on the other hand, is much more targeted at scientific computing, computer science and functional programming. It will be an excellent training ground when learning functional programming.
There's something that has a bit of design patterns, algorithms and functional programming, which is Elements of Programming. It uses C++ for its examples, which is a plus for you.
As for functional programming, I think it is less urgent than the other two. However, I indicate either Clojure or Haskell instead of Scala.
Learning functional programming in Scala is like learning Spanish in a latino neighborhood, while learning functional programming in Clojure is like learning Spanish in Madrid, and learning functional programming in Haskell is like learning Spanish in an isolated monastery in Spain. :-)
Mind you, I prefer Scala as a programming language, but I already knew FP when I came to it.
When you do get to functional programming, get Chris Okasaki's Purely Functional Data Structures, for a good grounding on algorithms and data structures for functional programming.
Beyond that, try to learn a new language every year. Even if not for the language itself, you are more likely to keep up to date with what people are doing nowadays.
Data structures and algorithms will help you no matter what language you use. I'd work on it first. Then design patterns (any OOP language will benefit from them). Functional programming is nice, but not necessarily a top priority.
Depends entirely on what you're doing.
I'd tailor which one you learn first to what would help you the most with your current job.
Write lots of code. Try to do it better every time. Occasionally work with more senior people, who can provide guidance praise and gentle correction.
I think that in general the topics that you have picked are very important, and my give you the chance to do something more than the usual boring stuff. However, I believe that the order should be something like this:
Data structures & Algorithms
Functional programming
Software Design
Specific technologies you need
My opinion is that Algorithms and data structures should be first. It is very hard to study algorithms if you have a lot of other things in you head (good coding practices, lots of programming paradigms, etc.). Also with time, people tend to become more lazy, and lose the patience to get into the ideas of this complex matter. On the other hand, missing some fundamental understanding about how things can be represented or operate, may lead to serious flaws in understanding anything more sophisticated. So, assuming that you have some ideas about imperative programming (the usual stuff tаught in the introductory courses) you should enhance your knowledge with algorithms and data structures.
It is important to have at least basic understanding of other paradigms. Functional programming is a good example. You may also consider getting familiar with logic programming. Having basic understanding of Algorithms and Data Structures will help you a lot in understanding how such languages work. I don't know whether Scala is the best language for that purpose, but will probably do. Alternatively, you can pick something more classic like Lisp, or Scheme. Haskell is also an interesting language.
About the Design Patterns... knowing design patterns will help you in doing object oriented design, but you should be aware, that design patterns are just a set of solutions to popular problems. Knowing Design Patterns is by no means that same as knowing how to design software. In order to improve you software design skills you should study other materials too. A good example from where you can get understanding about these concepts is the book Code Complete, or the MIT course 6.170 (its materials are publicly available).
At some point you will need to get into the details of a specific framework (or frameworks) that you will need for what you do. Keep in mind, that such frameworks change, and you should be able to adapt, and learn new technologies. For instance, knowing ASP.NET MVC now, may be worthless 5 years from now (or may not be, who knows?).
Finally, keep in mind, that no matter what you read, you need to practice a lot, which means solving problems, writing code, designing software, etc. Most of these concepts can not be easily explained, or even expressed with words, so you will need to reach most of them by yourself, (that is, you will need to reinvent the wheel many times).
Good luck with your career!
If would think Functional Programming would be low in priority since the languages you use are OO in nature, I would think spending some time in Design Patterns and on the specifics of the language itself would be more useful.
I read both GOF and HeadFirst, HeadFirst is probably the easier and more fun of the 2 but much thicker. You should probably look at Enterprise Design Patterns, like Martin Fowler's page http://martinfowler.com/eaaCatalog/
What field do you think you will work in? Games ? Web? That will probably decide how important the Algo part would be for.
I would say that you first need to understand (even if not remember) the base algorithms and data structures. (use Knuth and Cormen), then get to learn architecture (design patterns are here.)..
Functional programming is just one type of programming and is mandatory. There are many great programmers that are not using functional programming, but I assume that for all kinds you must first know the basics- algorithms and data structures.
I'd say #2 goes first, especially if you are planning to use C++/C# at work, having a good command of data structures and algorithms will give you some edge. I see #1 and #3 as somewhat parallel paths, but I do have a couple of suggestions: start with the Head First book for patterns, the GOF is more like a reference book and also the notation and language may get quite abstruse. As for functional programming, may I suggest Clojure instead of Scala? I'm convinced that a "functional-first" language (like F# or Clojure) will force you to think functional (a good thing) instead of just patching your O-O/imperative skills.

Resources