On a site like www.codewars.com, one can run ruby in a sort of sandbox, almost identical to IRB.
How does this actually work?
If the submitted code is eval()d, what's preventing me from submitting a system("rm -rf *") or redefining basic functions so that 50% of the time Array.sort actually runs Array.shuffle?
The simplest and safest solution is to run the Ruby code on separate computer, which you wipe and re-install after every run. This is, however, also a pretty heavyweight solution.
More lightweight, but (almost) as safe, would be using a virtual machine or a container instead of a whole separate computer, and e.g. using a read-only filesystem with a ramfs overlay, which you umount after every run. (Or just throw away and recreate the container.)
You could also use JRuby together with the JVM's security features (or IronRuby with the CLI's). The JVM has sandboxing features for JVM programs, and after all, JRuby is just a Java program like any other.
Lastly, you could write your own Ruby implementation with sandboxing in mind, or modify an existing one. The three options above are fairly simple, this one is hard, because most Ruby implementations aren't designed for sandboxing. TryRuby.Com worked this way, for example, and it took a significant amount of time to update it for Ruby 1.9, because it was originally based on a modified version of MRI, but MRI doesn't support Ruby 1.9. So, the implementation had to be switched to YARV, and a lot of the modifications to make it sandboxing-safe had to re-implemented from scratch. (The JRuby/IronRuby option above is similar to this, but you push off the work of making the implementation sandbox-safe to someone else, e.g. Oracle or Microsoft.)
A not-so-safe but also simple solution would be to run the interpreter under a restricted user account.
Of course, you can combine multiple approaches for defense-in-depth, for example, running a sandboxed interpreter under a restricted user account on a separate VM.
What does not work is to statically analyze the code before running it. The pesky Halting Problem bites us here.
Its my understanding that when I make a gem I'm compiling my ruby code into some form of executable, right? Does this mean that unless someone used introspection techniques (which is an acceptable risk to me), my source code is secure?
A gem is not a compiled executable. It's not compiled at all. Ruby is interpreted. Creating a gem just bundles the necessary files together, much like a zip file or tar archive.
If you want your gem secure you should keep it out of rubygems.org. You can set up your own private gem server or you can just include your gem in projects that need it.
While it is possible to compile Ruby code into an executable or shared library using Ruby's C API, that has nothing to do with gems.
A gem is just a collection of Ruby code (which could be regular scripts or compiled libraries) in a nice package for use with the rubygems package manager. It makes no effort to hide/protect the code.
I think
gem unpack
can extract your code. Never tried to do it and see if it's "human-readable" but you can try it before publishing your gem ;)
Its my understanding that when I make a gem I'm compiling my ruby code into some form of executable, right?
No. It's just a zipfile with some metadata. The contents of the zipfile are exactly what you put into them.
Does this mean that unless someone used introspection techniques (which is an acceptable risk to me), my source code is secure?
This depends on what you mean by "secure", but is completely orthogonal to RubyGems.
If you mean that "it can't be stolen", then that is already guaranteed for you by copyright law. Unless you live in a really weird country, software is protected by copyright automatically from the moment you write it.
If you mean "cannot be reverse engineered", then that is impossible. If you want people to be able to run your program, then you must give it to them in a format that can be understood by the CPU. Humans are much smarter than computers, so, if the program can be understood by the CPU, then it can also be understood by a human.
There are two common ways around this, which I will call the "Nintendo way" and the "Google way".
The Nintendo way is to give the user the CPU as well as the program, therefore, the user's CPU doesn't have to understand it. However, that model is still flawed. As long as you give the user something, he can figure it out. In the end, it's all just maths and physics, which can be understood. And users are pretty clever. Note that, for example, most game consoles were not cracked by evil crime syndicates trying to steal the code or pirating games, no, they were cracked by students wanting to run Linux or BSD on their hardware.
The Google way is to give the user nothing. You type something in the search box, Google sends you back the results, but at no time does the software leave Google's datacenter.
I am looking for a way to automate the testing, web page data filling, and also wanted to extract web page data and get them stored into our database permanent basis. Is there any way to fulfil such requirement using Ruby? If so, please guide me to what Ruby modules can help me.
Yes you can do all this tasks using Ruby and some gems.
I recommend you to take a look at Nokogiri gem for data extraction:
https://github.com/sparklemotion/nokogiri
And Capybara gem for testing and automation of forms and stuff:
https://github.com/jnicklas/capybara
P.S.: Capybara gem does much more than just this, but it can be applied to your case too.
Since some Webpages may not be valid XML, you are also able to use Regular Expressions to fetch the data you want from a webpage. Sometimes a XMLReader-approach just fails.
Sample:
require 'open-uri'
page_content = open("http://your_page.com").read
page_body = page_content.scan(/<body>(.*)<\/body>/i).first
# do whatever you want with it
As VBSlover said, capybara is useful to deal with browsing related stuff.
Doing this in an automated way every n minutes or the like is also possible with the whenever gem.
For handling Database-Storing there are plenty of very good gems out there.
Final answer: there is nothing you can't do with Ruby nowadays. Okay, maybe except writing some really (!) high-performance code / 3D-Engines.
Edit:
if you can tell what you exactly want to do i may suggest you some matching gems.
Usually "There is a gem for it" is a good saying. you can browse rubygems.org for some keywords you need, or look at https://www.ruby-toolbox.com/ for some categorized/ranked suggestions for your problem. :)
EDIT 2:
have a look at http://watir.com/
maybe just play around with it in some little painless scripts to get a feeling for it and if it is the solution for you.
Watir drives browsers the same way people do. It clicks links, fills
in forms, presses buttons. Watir also checks results, such as whether
expected text appears on the page.
Once you have it clicked everything for you, just scrape the results (or whatever you need) from the webpage, using some XML-Parser (nokogiri would be a good choice) or some regexp's.
Then stuff your data in your database. Activerecord comes to mind for this, but it may or may not be overkill. depending on your database, choose whatever adapter/connection gem you like (again: there are MANY).
If you want to do this every hour or the like, just use the whenever gem (manages a cronjob for you) or simply write a infinite loop with sleep(x) in it if you want. There is more than one way to do it. :)
First of all, you need a proper operation system, either use Linux or BSD or MacOS.
Windows will fit for some people, but not for you as ruby developer, too much libraries need c extensions with are pain in the ass to compile under cygwin.
I recommend, install a Ruby version manager, so you can try out different ruby versions, I prefer RVM, the Ruby Version Manager.
Install Ruby 1.9.3 it is the standard nowadays.
Trough rubygems install the gem mechanize, with does pretty all automation for websites you will need. It is a successor of LWP::Mechanize from Perl.
Nokogiri would be also useful, for parsing XML data like (X)HTML, but remember you should have prior libxml libs installed on your system.
Ah, according to your question:
Yes, you can read websites using ruby, for example read this webpage:
http = HTTPClient.new
http.get "http://stackoverflow.com/questions/14235393/can-i-read-webpage-data-using-ruby"
Done
I'm looking for a simple Ruby framework for a non-web project. I'm very familiar with Rails, so when I write pure Ruby I miss:
The different environments (development, test, production)
The console rails c
And many other utilities provided by rake or rails
I know I can require ActiveSupport and friends, but it's not what I need. It's mostly the development framework that I miss.
One thing I investigated is to do my project as a gem (even if I don't need a gem at the end). Using jeweler for instance provides versioning. I'm sure there are better ways but I can't find any. What would you use ?
Alright, that is somewhat of a three-part question. A threstion, or triesti... actually that doesn't work. Threstion it is then.
One thing to understand about Rails is that it isn't magical. It's code, written by mortal humans who have faults (such as oversized-egos) but who are also incredibly damn smart. Therefore, we can understand it after perhaps a little bit of studying their work.
1. Rails environments are lovely
The code behind the Rails environment management is actually quite simple. You have a default environment which is set to development by this line in railties/lib/rails/commands/server.rb. You could do much the same thing in your application. Run a bit of initialization code that defines YourAwesomeThing.environment = ENV["AWESOME_ENV"] || "development". Then it's a matter of requiring config/environments/#{YourAwesomeThing.environment}.rb where you need it. Then it's a matter of having configuration in those files that modify how your library works depending on the environment.
I couldn't find the line that does that in 3-1-stable (it's gone walkies), so here's one that I think does it in the 3-0-stable branch..
As for the lovely methods such as Rails.env.production?, they are provided by the ActiveSupport::StringInquirer class.
2. Rails console is the Second Coming
I think rails console is amazing. I don't want to create a new file just to manually test some code like I would have done back in the Black Days of PHP. I can jump into the console and try it out there and then type exit when I'm done. Amazing. I think Rails developers don't appreciate the console nearly enough.
Anyway, this is just an irb session that has the Rails environment loaded before it! I'll restate it again: Rails is not magical. If you typed irb and then inside that irb session typed require 'config/environment' you would get (at least as far as I am aware...) an identical thing to the Rails console!
This is due to the magic that goes on in the initialization part of Rails which is almost magic, but not and I've explained it in the Initialization Guide Which Will Be Finished And Updated For Rails 3.1 Real Soon I Promise Cross My Heart And Hope to Die If Only I Had The Time!
You could probably learn a lot by reading that text seemingly of a similar length to a book written by George R. R. Martin or Robert Jordan. I definitely learned a lot writing it!
3. The Others
This is where I falter. What other utilities do you need from Rake / Rails? It's all there, you just need to pick out the parts you need which is actually quite simple... but I can't tell you how to do that unless you go ahead and paint those targets.
Anyway, this is the kind of question that is exciting for me to answer. Thanks for giving me that bit of entertainment :)
Edit
I just noticed the BONUS fourth question at the end of your question. What would be the point of versioning your project? Why not use a version control system such as Git to do this and just git tag versions as you see fit? I think that would make the most sense.
First of all, Rails goal as a web framework is to let you mostly forget about the low-level details of a web based application (interacting with a web server, a database, etc.) and provide you with high-level abstractions which allow you to concentrate yourself on the actual job: Getting your application done.
There is such a huge diversity of non-web applications, that it is simply impossible to have a framework that "makes them all".
It is useful to understand what a framework is, and why and when you should use it.
Let's say you would have to build a dog house. You would buy some wood, nails and a hammer to build it the way you want from scratch, or you could go to Walmart and buy a DIY doghouse kit.
You will get a manual with detailed instructions on how to build your dog's "dream doghouse", and you will get it faster done than when making it from scratch, but you will be limited to the wood that comes with it and may be forced to use their nails or screws, and in some cases you would also need to buy a special screw driver.
The DIY kit is your framework. Some smart guys already provided you with the basic tools of making your doghouse, you only had to figure out (or not, if you had a manual) how to do it given their "framework".
Now, let's say you would build a skyscraper. The materials, tools and conventions (the framework) used would probably not be the same as they would when building a doghouse, and it would be over-kill to use a "skyscraper framework" to build a doghouse.
You wouldn't write Twitter client with the same set of tools and conventions as you would when writing a banking application with a billion concurrent users.
Rails is just one framework which provides you with a set of many, many good (and useful) conventions to write a web application, but that doesn't necessarily mean that it's the ultimate way to go. You can choose another framework, and maybe you will like the conventions for writing your application even more than the ones provided by Rails; or you could write all from scratch which would not limit you to what is provided by a specific framework and maybe make it even better.
To answer your question: Think of what you want to write and don't let yourself be narrowed down by the set of tools and conventions provided by a framework, as they will surely be if you choose this path.
Buy yourself some books about Design Patterns and read some great software, and then you may understand why the authors did not choose an application framework. Indeed, they've developed their own conventions and defined the behavior of what it should do; and once again, the "Rails way" isn't the only way, which doesn't make it the bad way. :)
I would appreciate constructive criticism as I really tried to provide a good answer.
Have you looked at Monkeybars? It's a Rails-inspired MVC framework that runs on top of JRuby and Swing to build desktop applications. I like it a lot.
Locked. This question and its answers are locked because the question is off-topic but has historical significance. It is not currently accepting new answers or interactions.
Sometimes it feels that my company is the only company in the world using Ruby but not Ruby on Rails, to the point that Rails has almost become synonymous with Ruby.
I'm sure this isn't really true, but it'd be fun to hear some stories about non-Rails Ruby usage out there.
One of the huge benefits of Ruby is the ability to create DSLs very easily. Ruby allows you to create "business rules" in a natural language way that is usually easy enough for a business analyst to use. Many Ruby apps outside of web development exist for this purpose.
I highly recommend Googling "ruby dsl" for some excellent reading, but I would like to leave you with one post in particular. Russ Olsen wrote a two part blog post on DSLs. I saw him give a presentation on DSLs and it was very good. I highly recommend reading these posts.
I also found this excellent presentation on Ruby DSLs by Obie Fernandez. Highly recommended reading!
I use Ruby extensively in my work, and none of it is Rails (or even web) based.
My domain is usually client-side Windows applications (wxRuby GUI) and scripts, automating Excel, Internet Explorer, SQL Server queries and report generation (win32ole COM automation). I also use the sqlite, pdf-writer, and gruff libraries for various data munging and graph generation tasks.
Rails' success has been great for Ruby, but I agree that Rails has received so much attention that Ruby's value beyond the web is often overlooked.
We are mainly a C++ shop, but we've found several areas where Ruby has proven quite useful. Here are a few:
Code Generation - Built several DSLs to generate C++/Java/C# code from single input files
Build Support
scripts to generate Makefiles for unix from Visual Studio Project Files
scripts for building projects and formatting the output for Cruise Control
scripts for running our unit tests and formatting the output for Cruise Control
scripts for manipulating Visual Studio projects and solutions from the command line
Integration Tests - We can crank out tests much quicker and cleaner using Ruby than C++
QA's entire testing suite is written in Ruby
Ruby is basically my go to tool for where it makes sense. And it makes sense in a lot of places.
Google Sketchup uses Ruby as an embedded scripting language. You can use it to perform all sorts of 3d modeling and import/export tasks. The scripting works with the free version and there's even decent documentation.
Ruby with a homebrew extension written in C++ does all the heavy pixel pushing for my photography processing. I was using Python+numpy but when doing artsy stuff, Ruby is just more fun. Also the relative lack of, or lesser maturity of, good image processing libraries makes me feel less like i'm reinventing wheels. I am clueless about Rails, other than i've heard of it, have a fuzzy idea what it is, and actually have a book on it (unopened)
We use Watir (Ruby library) to test our .net web application.
Check out Shoes, a simple API for building GUIs in Ruby aimed at novice programmers.
Or you could use Ruby to make music ala Giles Bowkett's Archaeopteryx. This presentation by Giles about Archaeopteryx is one of the best presentations ever. I highly recommend it.
RubyCocoa and MacRuby. Possible to make full Cocoa-based GUI apps without Rails. And then you get to use Interface Builder, too.
I worked on a museum project last year that used a lot of Ruby. (http://http://ourspace.tepapa.com/home)
The part that I spent most of my time on was an interactive floor map. The Map on the floor has sensors so when people walk on it lights are triggered and displays in the wall show images or videos and audio tracks are played.
All the control code for this part of the exhibit is ruby. I wrote C interfaces with ruby wrappers to communicate with the floor sensors and the lighting controllers. The system queries a MYSQL database for the media files to be displayed and then tells computers in the walls to play the media via UDP.
It's the most reliable part of the entire exhibit.
Ruby was used for the other major part of the exhibit, the Wall though I didn't have much to do with that. Most of the graphics were prototyped in ruby using interfaces to OpenGL, a bit of Cocoa and a physics library before being ported to pure Obj-C.
Puppet and Chef: DevOps
I didn't see a mention of Puppet or Chef in the 30 answers that preceded my arrival. Ruby appears to dominate current work in cloud automation and is the base, extension, and templating language of these two big players. They are used primarily to distribute system and application configuration information for server arrays and for general IT workstation management.
The DevOps field is quite Ruby-aware. Today, Perl has a competitor. While a really simple script may often still be written directly for sh(1), a complex task now might be done in Ruby rather than Perl.
The only site I've done with Ruby at work is using Rails, but I'd like to try Merb.
Other than that I do a lot of little utility programs in Ruby - for instance an app that reads RSS feeds and imports new posts into a dabase.
It's fun, so I also write some dumb stuff just because it's so quick. Yesterday I wrote an app to play the Monty Hall problem 100,000 times to help a friend convince her professor that switching is the correct strategy.
I almost take insult that ruby is a rails thing. It is like back when CGI was the latest trend and everyone figured that if you knew perl you must be doing it only because you programmed CGI apps. Ruby is just a scripting language for me, although not as mature as python so I somewhat regret having to jump through some of its hoops and recent changes, I still like it and use it. Although I work in a java shop and therefore groovy is the ideal choice for a scripting language, I still use ruby at home and for throw away scripts that aren't needed to be shared at work.
I was considering getting into RoR from all the buzz and how quick/simple it is, but after looking over rails I didn't see anything at all that was amazing or even the least bit innovative or rapidly fast about its development compared to any other framework. The only benefit I saw was that I could code in ruby, which would be nice, but initial setup, server maintenance and scaling is more difficult, thus re-offsetting the pleasure of coding in ruby.
I created a presentation -- coincidentally named Off The Rails -- to discuss Rack-based web applications:
https://github.com/alexch/Off-The-Rails
The git repo includes slides in Markdown format and sample code (in the form of running applications and middleware). Here's the abstract:
Ruby on Rails is the most popular web application framework for Ruby. But it's not the only one! If you think Rails is too big, or too opinionated, or too anything, you might be happy to learn about the new generation of so-called microframeworks built on Rack. And since Rails 3 is itself a Rack app, you don't have to give up Rails to get the benefit of Sinatra routes or Grape APIs.
And here are some references:
This talk lives at https://github.com/alexch/off-the-rails
Yehuda's #10 Favorite Thing About Ruby
Rack
rack-test
rack-client
Sinatra
Grape
Vegas
Siesta
Rerun
Hope you find it useful!
I'm mostly a Web developer, and I learned Ruby to use Rails, but I like the language so much that I started developing a desktop Swing application in Ruby, using JRuby and Monkeybars. I'm competent in Java, but don't much like using it, and the Swing API is horrible, so putting Ruby on top has been a big win.
We mainly use rails, but we have plenty of other non-rails ruby things - for example a standalone authentication daemon thing for centralized authentication of users, and an 'image processing server' which runs arbitrary numbers of ruby processes to process images in parallel.
Oh, and don't forget good old Rake :-)
Ruby is also used for Desktop application. Especially the use of JRuby to develop Swing desktop application.
I've used Ruby at work for
A data extractor, generating csv files from binary output.
A .ini file generator, turning a simple syntax into a repetitive .ini format.
A simple TCP/IP server, acting as stand-in for the customer's system during testing.
We use Ruby to implement our test automation software. This includes test framework and driver code for Selenium RC, WATIR and AutoIT.
Ruby is powerful enough to create comprehensive applications that can interface with Test tools like Selenium or WATIR, while at the same time reading from data files, interacting with a remote Windows UI and performing near transparent network communication. All while running on Windows or Linux.
The uncluttered syntax makes it ideal for new and inexperienced programmers to read. While its totally OO nature makes it easy for these same programmers to apply good (recently learned) OO techniques, from the start.
The flexible nature of Ruby's syntax also makes the use and creation of DSLs much easier. This allows less-technical people to get invovled, read and possibly create there own tests.
I have used Ruby for code generation of C# and T-SQL stored procedures in a project with unstable requirements. The data model was encoded in a YAML file and .erb templates were used for the classes and stored procedures. It also allowed for a much more DRY solution than would have been possible with straight C# as repetitve code could be factored out into a single method in the code generator.
Where I work, we use Ruby to do a number of different one-off type batch jobs. One example of that is a job that interacts with Amazon's S3 service. At the time, the Ruby S3 library was probably the easiest one out there for us to get up and running in a short amount of time.
I wrote an order processing expert system (see DSL answer as well), converted 100k lines of customer specific perl into about 10k lines of ruby handling dozens of customers. No web components at all, no Rails.
I am a webdriver user. ruby is used by webdriver for automating the build process thanks to rake. see http://code.google.com/p/webdriver/ for details
Heh, great question.
I used Ruby to convert Excel spreadsheet airport facility data to sqlite3 for the android phone platform while making an app for pilots.
I use Ruby with Sinatra which is much simpler than Rails. I did use Rails but just found that it has turned into a bit of a monster, although Rails is still amazing compared to web frameworks available for Java.
The main feature of Ruby that I love however is "eval" and "method_missing", which Rails actually uses for example in ActiveRecord so that you can use the amazing "find_by-field-name-" queries.
I used Ruby for a lot of back-end code simply because I was the only person who was tasked to do it and needed a nice clean language that allowed me to be very productive and write easy to maintain code. I find Ruby allows me to do that easier than Perl and Python. Other people's mileage might vary on that but it works well for me.
Besides that, I like how Sequel and Nokogiri work. I also used ActiveRecord for a while separately from Rails.
We use some Ruby for file manipulation but have not been able to incorporate rails yet.
I've used Ruby a lot professionally for quick scripts for things like shuffling files around. I'm the same way in that I was using Ruby first before touching Rails at all.
In Boulder there was an excellent group of Ruby users who met monthly. This point was made - that Ruby does have an existence beside its use in Rails. Plain Ruby users do exist, are begging for attention, have neat things to show, and can find each other at user group meetings.
They also had better pizza than the Python group, who met also the same day of the month. Can only pick one...
While we do have several Rails apps at work, we also use Ruby for some fairly intensive non-web stuff.
We've got an SMS delivery daemon, which pulls messages from a queue and then delivers them, and credit card processing daemon which other apps can call out to, which makes sure there's a central audit trail.