Ruby Memory leak (MRI) - ruby

Question

I must be missing something but every single application I write in Ruby seems like leaking some memory. I use Ruby MRI 2.3 but I see the same behaviour with other versions.
Whenever I write a test application that does something inside a loop it is slowly leaking memory.
while true
#do something
sleep 0.1
end
For instance, I can write to array and then clean it in a loop, or just send http post request.
Here is just one example, but I have many examples like this:
require 'net/http'
require 'json'
require 'openssl'
class Tester
def send_http some_json
begin
#uri = URI('SERVER_URL')
#http = Net::HTTP.new(#uri.host, #uri.port)
#http.use_ssl = true
#http.keep_alive_timeout = 10
#http.verify_mode = OpenSSL::SSL::VERIFY_NONE
#http.read_timeout = 30
#req = Net::HTTP::Post.new(#uri.path, 'Content-Type' => 'application/json')
#req.body = some_json.to_json
res = #http.request(#req)
rescue Exception => e
puts e.message
puts e.backtrace.inspect
end
end
def run
while true
some_json = {"name": "My name"}
send_http(some_json)
sleep 0.1
end
end
end
Tester.new.run
The leak I see is very small, it can be 0.5 mb every hour.
I ran the code with MemoryProfiler and with GC::Profiler.enable and it shows that I have no leaks. So it must be 2 options:
There is a memory leak in C code. This might be possible but I don't use any external gems so I find it hard to believe that Ruby is leaking.
There is no memory leak and this is some sort of Ruby memory management mechanism. The thing is that I can defiantly see the memory growing. Until when will it grow? How much do I need to wait to know if it is a leak or now?
The same code runs perfectly fine with JRuby without any leaks.
I was amazed reading a post:
stack overlflow
from Joe Edgar:
Ruby’s history is mostly as a command line tool for text processing
and therefore it values quick startup and a small memory footprint. It
was not designed for long-running daemon/server processes
If what is written there is true and Ruby doesn't release memory back to OS then... We will always have a leak, right?
For instance:
Ruby asks for memory from OS.
OS provides the memory to Ruby.
Ruby frees the memory but GC still didn't run.
Ruby asks for more memory from OS.
OS provide more memory to Ruby.
Ruby runs GC but it is too late as Ruby already asked twice.
And so on and on.
What am I missing here?

Answer 1

Look Into GC Compaction and (Un)frozen String Literals
"Identical" Strings Aren't Necessarily Identical
Prior to Ruby 2.7.0, mainline Ruby didn't have compacting garbage collection. While I don't fully understand all the internals, the gist is that certain objects couldn't be garbage collected. Since you're using Ruby 2.3, that's something to keep in mind as you work on your memory allocation issues. Other non-YARV VMs may handle their internals differently, which is why you might see variation when using alternative engines like JRuby.
Even with Ruby 3.0.0-preview2, String literals aren't frozen by default, so your current implementation is creating a new String object with a unique object ID every tenth of a second. Consider the following:
3.times.map { 'foo'.__id__ }
#=> [240, 260, 280]
Even though the String objects seem identical, Ruby is actually allocating each one as a unique object in memory. Because a loop iteration is not a scope gate, those String objects can't be collected or compacted by YARV.
Enable Frozen String Literals by Default
You may have other issues as well, but it seems likely that your largest issue is keeping all of those String literals in scope indefinitely within your endless while-loop. You may be able to resolve your garbage collection problem (it's not a memory leak) by using frozen String literals instead. Consider the following:
# run irb with universally-frozen string literals
RUBYOPT="--enable-frozen-string-literal" irb
3.times.map { 'foo'.__id__ }
#=> [240, 240, 240]
You can solve this within your code in other ways as well, but reducing the number of String literals that remain in scope seems like a very sensible place to start.