I recently stumbled upon this 2002 conference paper (Getting Erlang to talk to the outside world by Joe Armstrong) and I was wondering whether this is a standard that was ignored or if there was any adoption?
Should I focus on Apache Thrift for inter-platform communication? (whatever solution I choose will involve at least Erlang)
You might find the following information helpful about UBF (i.e. UBF User's Guide).
http://ubf.github.com/ubf/ubf-user-guide.en.html
This might help answer some of your questions about UBF. There is also an Erlang server and client implementation of Thrift using the UBF framework (https://github.com/ubf/ubf-thrift).
Joe N.
p.s. We have been using UBF, EBF, and JSF in a production environment for approximately ~5 years. The UBF contract checker is very helpful for developing, debugging, and deploying a system.
UBF never really caught on. Here's what Armstrong had to say in 2008. As you look into Thrift you might also look into Protocol Buffers. You can find an Erlang implementation here.
Hope this helps.
I don't know if anyone's using Erlang's binary serialization format per se, outside of Erlang programs; but BERT (Binary ERlang Term) is an Erlang-compatible binary data interchange format and RPC protocol specified and open-sourced by the GitHub guys. It's based on and fully binary-compatible with the Erlang external term format.
As you can see on the BERT website, there are implementations in a number of programming languages, including C++, JavaScript, Ruby, Python, Scala, Haskell, Go, Factor, Scheme, Clojure, and Common Lisp. The mailing list currently has 85+ members subscribed to it, so quite a few people are indeed using BERT, and hence indirectly the Erlang binary format.
In my own use, mostly in Ruby and Common Lisp applications, BERT has proved useful as essentially a binary form of S-expressions.
I think the Hibari guys are using it.
https://github.com/ubf/ubf/
Related
I am looking from a security perspective and are there any frameworks available to build and use JSON?
I am interested because of the performance which C++ can offer. Currently, Python and Node.js are also available option for me. How can I decide which language+framework should I use ?
Appreciate your support.
Thanks !
PS. - Currently, I am using Java Spring to implement restful apis.
There's plenty of them out there. The absolute simplest one I found, and use, is this. https://github.com/eidheim/Simple-Web-Server .
There are clearly more sophisticated ones out there, just "ask the google". I don't know of any exhaustive comparisons between these frameworks and the ones you specified. However, you could write your own simple benchmarks around the domain you're most interested in. That's what really matters, right?
For json libs there's rapidjson and spirit json. Don't waste your time with boost::property_tree, it's not fully fleshed out wrt JSON.
As for speed. It's compiled so.... its a good possibility that a C++ framework will outperform one based on an interpreted language. So lets say it's faster, you have a heck of a learning curve to climb (assuming you don't know C++ already) but in the end, in my humble opinion it's worth it. I've done these before in Python and Ruby. I really enjoy having the compiler check types. My code is more robust, it does what I tell it to do, and I'm not forced to use exceptions.
Tip: get a code completion plugin like YouCompleteMe
So as I have asked in a previous post, I want to be able to make programs or functions written in different languages to communicate between them.
I have come across zeromq recently and I'm trying to figure out whether or not this is something that could help me since it provides some sort of sockets. Can zeromq for example exchange data (or pass arguments) between a program written in python with a program or a function written in C++ or is its function for something completely different?
A: Oh Yes, exactly that is the power of ZeroMQ or nanomsg frameworks
Both of these are not sockets but rather BEHAVIOUR created within a context of a Zero-* -- a set of courageous maxims the Scaleable Formal Communication Pattern Framework was designed, developed and finetuned to meet.
That will enable you to assemble your own fast & smart messaging layer(s).
Q: What is the best next step?
In spite of your first impression, simly do forget anything you know about sockets and multithreaded synchronisation tricks.
Yes, rather forget and build your new understanding on "green field".
Take Pieter HINTJENS' book "Code Connected, Volume 1" (accessible in PDF ) and spend a few weeks on understanding both the motivation and the typical errors Pieter has hammered into this must-read bible of ZeroMQ.
Code-snippets are dangerous in case you did not get or completely missed the full-context of the bigger picture.
Believe me. I could not give you better advice. You may check my other posts on ZeroMQ & nanomsg, to see the difference.
You will definitely benefit from this book and ZeroMQ will give you many powers you would never ( and believe me never ) would be ready to program from scratch on your own. The power is so immense ( if well re-used ).
nota bene
For real-world inter-process communications, there is one minor issue to be aware of. Various ZeroMQ versions' inter-operability. Yes, the power of ZeroMQ is immense, nevertheless, it is necessary to keep the version control built in your messaging layer so as to solve situations, where some platforms do not have an update-path to "newer" releases available. Went into this issue with re-integration of a trading system with a component, where as old as zmq.__version__ == 2.1.11 was necessary, while recent are versions well above 14.x.y, so as to be assured to be 100% end-to-end backward-compatible.
Still, the overall potential is so immense, it makes sense to persevere and get the job done. G/L on that.
ZeroMQ is an abstraction of sockets. It is cross platform and have lots of language binding: I personally don't know any language that doesn't have ZeroMQ bindings.
So yes, you can use ZeroMQ to communicate between a program written in Python and program written in C++.
I recommend going through the zguide as it contains a lot of very useful information about ZeroMQ.
PyZMQ can be used as Python binding, and zmqpp for your C++ code. Note that for the C++ code you could also use cppzmq or the zmq C API directly. I would recommend using zmqpp as its higher level and (imho) easier to use.
Is Lisp good for web programming/applications (interactive), the way ruby and php are?
Things to take into consideration would be:
Ease of use
Deployability
Difficulty (especially for a beginner to programming)
(Edit)
I was Particularly referring to Common Lisp, after reading Paul Graham's essay. Would be my first programming language. In this regard. Is it suitable to do so ?
I hear that Clojure's macro functionality is not as powerful as Common Lisp's, and that's why I'm trying to learn Clojure. It teaches programming and is very powerful.
Lisp is a language family, not a single language. To somewhat answer your question, yes, web frameworks exist for the various Lisp dialects, like UnCommon Web for Common Lisp and PLT Racket (a Scheme dialect) has a tutorial on their web site too:
http://docs.racket-lang.org/continue/
But one Lisp dialect where you'll see quite a lot of web development going on at the moment is Clojure, since it can leverage existing Java libraries and infrastructure. Here's a list of projects, and that's not comprehensive:
Hiccup - represent HTML in Clojure
Ring - a HTTP abstraction similar to Rack in Ruby
Enlive - selector based templating and more
ClojureQL - a SQL integration library
CongoMongo - a wrapper for MongoDB
There'd be many more, but I think you get the idea. As for Clojure web apps in the wild, the Clojure learning site 4Clojure would be an example and you can check out the source code on GitHub (I occasionally contribute to this):
https://www.4clojure.com/
https://github.com/dbyrne/4clojure
As you can see there's quite a lot going on in terms of Clojure and web development. It may not always be smooth sailing, but people are working fast at making the experience better. Making use of existing Java infrastructure does have its benefits, like the possibility to deploy to Google App Engine etc.:
http://www.infoq.com/articles/deadline-clojure-appengine
Hope that somewhat answers your question...
I have almost no experience with PHP or Ruby. However, I can say that it is quite easy to write a web application using Hunchentoot, CL-WHO, and one of the many database backends (Postmodern, CL-SQLite, CLSQL, or even a simple serialization library like USerial or Rucksack).
There are also frameworks like Uncommon Web or Weblocks, but I have not tried them yet.
There's a funny quote from Kent Pitman
"...Please don't assume Lisp is only useful for Animation and Graphics, AI, Bioinformatics, B2B and E-Commerce, Data Mining, EDA/Semiconductor applications, Expert Systems, Finance, Intelligent Agents, Knowledge Management, Mechanical CAD, Modeling and Simulation, Natural Language, Optimization, Research, Risk Analysis, Scheduling, Telecom, and Web Authoring just because these are the only things they happened to list."
And, look, "web authoring" even made it on the list!
Lisp is good for web programming because it is so flexible and the s-expressions that make up Lisp forms have some nice similarities to HTML. To see what I mean, check out Lisp for the Web. Notice how he writes a macro to basically create a domain-specific language for creating web pages -- instead of defining functions, he's now defining pages! This DSL concept is also apparent in the CL-WHO library he's using, which lets you write Lisp that turns into HTML.
Paul Graham had success rolling his own solution while creating what later became Yahoo! Store in Common Lisp (and even more success writing about it afterwards), and since then many packages have sprung up.
Weblocks is the Common Lisp web framework that seems to me to have the most momentum at the moment. There's a video of it being discussed at the Twin Cities Lisp user group. UCW is very similar, but I don't see much action on it now (please excuse me if I'm just looking in the wrong places). The blog post on why the creator made Weblocks is pretty enlightening: he basically wrote enough Lisp so that he'd never have to deal with HTML, AJAX, and JavaScript again. That might be an overstatement, but the fact that he could even do some of that should answer your question.
This is opinionated but so is the nature of the question, so here's my take.
I would not recommend a beginner to programming to start with a functional language, especially a Lisp.
T concepts on functional languages and especially Clojure are more advanced and the power it gives you, you propably won't need in your first steps into web development.
So since you are a beginner, then my answer would be no, Lisps are not as easy to learn and use as Ruby and PHP are.
For web application, framework is important consideration than language. If language is very powerful but don't have good web framework, developing web application immediately become hard and face lot of known complexity.
Think about ruby without rail framework, people start to like it because of rail framework. This is same as java, using only Servlet without Jsp or other framework developing web application will be horrible.
Now come to your point, easy of use always relative- I could not find any end to end framework in LISP family like ruby or python. But I found Clojure reduce lot of complexity in server side. Deployment- it depends on server, not language like java server need war or ear file. Learning- it is also relative but tools some times help like debugging or ide. But most of the lisp family don't have good debugging option.
I'm sitting here writing a function that I'm positive has been written before, somewhere on earth. It's just too common to have not been attempted, and I'm wondering why I can't just go to a website and search for a function that I can then copy and paste into my project in 2 seconds, instead of wasting my day reinventing the wheel.
Sure there are certain libraries you can use, but where do you find these libraries and when they are absent, is there a site like I'm describing?
Possibly a wiki of some type that contains free code that anybody can edit and improve?
Edit: I can code things fine, I just don't know HOW to do them. So for example, right now, I'm trying to localize a robot/car/point in space. I KNOW there is a way to do it, just based off of range and distance. Triangulation and Trilateration. How to code that is a different story. A site that could have psuedo code, step by step how to do that would be ridiculously helpful. It would also ensure the optimal solution since everybody can edit it. I'm also writing in Matlab, which I hate because it's quirky, adding to my desire for creating a website like I describe.
StackOverflow.com. No, I'm not joking.
At its best, people come here saying "hasn't some library done X already", and very often the Collective Wisdom answers "yes". But the biggest obstacle is lack of a description language: even here, a big problem for many posters is describing the problem clearly enough for others to recognize it as something they've seen before.
And if people can't understand what you're trying to do, no search engine will.
Firstly, two caveats:
Copy and pasting code you don't understand is a bad idea. Make sure you understand exactly what the code does before you use it.
Make sure you respect the license of the code you are copying. This is important!
Those caveats aside, it's often language dependent. Languages with an open development ethos (not just an open source implementation, think Python as compared to Java) tend to have official archives of open source libraries. For example:
Perl (which probably started this trend) has CPAN
Python has PyPI and Python Cookbook
PHP has PEAR
C++ has boost
Ruby has gems
R has CRAN.
Haskell has Hoogle and Hackage
Furthermore, don't forget to look in your languages standard library. Some modern languages have massive standard libraries, which have often contained the functionality I am looking for:
Java has its API documentation
C# and VB.NET have the massive MSDN
Non-openly developed languages often have non-official community archives. For example:
C# tends to have a lot of code at CodePlex and CodeProject
MATLAB has the Matlab Central File Exchange
A third category of sites are language agnostic. They are often best search through POG (plain old-fashioned Google). For example:
Stack Overflow
SourceForge
The confusingly language agnostic Java2s
Planet source code
Github
Finally, a fourth category of sites that I find increasingly useful are source-code search engines:
Google Codesearch
Koders
You may also be able to find useful source code, or at least get help writing something, through various pastebins.
Pastebin is language-agnostic
HPaste is mostly Haskell, but has a little in other languages.
Often, at the end of the day it is easiest just to google it, though.
There is a wiki that contains free code that anybody can edit and improve:
Rosetta Code.
As a means of an overview there is the "Solutions by Programming Task" page.
From the former page:
"Rosetta Code is a programming chrestomathy site. The
idea is to present solutions to the same task in as
many different languages as possible, to demonstrate
how languages are similar and different, and to aid a
person with a grounding in one approach to a problem
in learning another."
Cutting and pasting code you find on the Internet into production code would be like chewing gum found in the street. - Mike Johnson
With that in mind, try sites that host opensource projects like GitHub, CodePlex, code.google.com, etc.
I'm not sure this question is language agnostic, but I use GitHub this way ;) Other languages may have places where this is possible.
Safari Bookshelf from O'Reilly has many, many books that contain many implementations from which to choose.
http://my.safaribooksonline.com/
I was a subscriber for a few years before coming to my current job, where we have a corporate account! It's one of the best perks, and one of the best resources I have available. I haven't bought a computer book in years.
Aside from sites like this (Stack Overflow) I don't think there's many, maybe CodePlex, but I almost marked you -1 for assuming that code found on the Internet is yours to copy.
I'd suggest reading about software licencing, I hope you'd at least comment where you got it from.
My language of choice is Ruby, but I know because of twitter that Ruby can't handle a lot of requests. It is a good idea using it for socket development? or Should I use a functional language like erlang or haskell or scala like twitter developers did?
The company I work for uses Ruby for our web site. We have so far handled a little over 34,000,000,000 hits. We have no problem handling around 10,000,000 hits per day. Peak hits have exceeded 40,000,000 hits per day.
Scalability depends on a lot of factors. Our databases do a disproportionately high percentage of writes compared to reads, for example. While most websites do about 90% reads to 10% writes, we are closer to 50%-50%. My point is that scalability is affected by a lot of factors. If you are database-limited, as is often the case for web apps, it won't matter what language you use, you'll be waiting on your database.
There's a lot to think about if you are looking at handling large scales. Sharding databases, memcached, etc. etc. etc. etc. The language you use for your application is just one aspect, and often, though not always, a small aspect of scalability.
Ruby may be a good option for you, but there's a lot to like in other languages. Erlang tries hard to make it easier to recover from errors, for example.
I'm not sure that any "lessons" that the Twitter team has learned about Ruby (more specifically, Rails) and scaling would apply to your project. They're looking at WAY more traffic than most people can reasonably expect to see.
As far as sockets and Ruby go, check out I like Unicorn because it's Unix. It's quite an interesting read about doing sockets in Ruby.
I'd like to provide a bit of context first. I'm pretty active with the Scala community, and I would choose Scala over Ruby for any project.
So, having said that, keep with Ruby unless you actually hit barrier. If Ruby is your language of choice, it might just be that you'll never be happy with the choices you mention, particularly the statically typed ones.
It might be good to learn a new language, to have something to fall back on if you need an alternative. In your case, I'd recommend Clojure or Erlang. Scala is a good statically typed, OO language with functional programming perks. It might be easier to learn than the others, but people who really like dynamic typing don't convert to static typing easy.
As for Haskell, it's one of the most awesome languages out there (and much more well support and popular than the equally awesome alternatives), and can open your mind like nothing else. It's also tough to master.
If ruby is your favorite language, yes it is a good idea. It is always better to use what you know and what you like
Whereas you may get better performance from a functional language such as Erlang the suitability of Ruby will really depend on what you are trying to achieve. For example how many requests are you going to be handling is probably the first question, if the performance benefits of using Erlang don't make much difference use something you are comfortable with, why learn a new language if you don't have to?
You at least have the option of staying in your favorite high level language if you use a fast, concurrent language like Haskell, Erlang or Scala. With Ruby, performance bottlenecks will mean switching to compiled C (or Haskell, or ...) for speed anyway.
Ruby has the advantage of good frontend frameworks.
I have also used Ruby for many projects though I've recently moved to Scala and like it quite a bit. One thing that I've heard good things about (but never tried myself) for network stuff in Ruby is EventMachine. It uses the Reactor Pattern just like twisted and it seems quite solid.
The key is to have a low level library in C/C++ that does the socket multiplexing for you. Socket multiplexing is what makes a TCP server process truly multi-user. such libraries in C (which is what you want) could be libevent/libev... and in c++ boost::asio. Python has twisted that does it behind the scenes.
If you get such a library and use it in ruby you should be able to implement most socket programs fairly well. This is especially true on UNIX oses which favour multi-process to multi-threading.
Having recently written (actually still doing so now), a project using sockets with Ruby and Java I would say no. The ruby socket implementation is poorly documented unless you plan on writing a basic blocking chat server. I found writing in C or Java simpler, Ruby wraps up native sockets and your kinda left wondering how the hell to use it now. I have previously written plenty of socket code on windows, Linux and other platforms in C, with less stress.
My Ruby code now is very small and works well, getting to that point was a real pain.