05.02.07
JRuby Performance, Yowch!
I took JRuby 0.9.9 for a spin with the exceptionally-inefficient Boggle program from a few months back. Here are the numbers:
$ time ruby short.rb c a t d l i n e m a r o p e t s 2338 ruby short.rb c a t d l i n e m a r o p e t s 241.95s user 1.20s system 97% cpu 4:08.35 total
$ time jruby short.rb c a t d l i n e m a r o p e t s 2338 jruby short.rb c a t d l i n e m a r o p e t s 1178.86s user 40.84s system 108% cpu 18:44.44 total
I’d heard JRuby was slow, but this is spectacular. Four times slower than the already-slow Ruby?
I’d always thought that the point of JRuby was to run Ruby programs on the JVM, and hence get the benefits of the JVM’s JIT. I guess not. With that kind of performance, the only possible justification is the ability to use Java libraries.
A said,
May 2, 2007 at 11:42 pm
I know JRuby has a compiler; did you try that or are you just running it off the interpreter?
danvk said,
May 2, 2007 at 11:47 pm
Yeah, I did try that, jrubyc gave me:
Error — Not compilable: Can’t compile node: NextNode[]
Any ideas?
Charles Oliver Nutter said,
May 3, 2007 at 12:23 am
Your results aren’t too surprising. JRuby can run faster than MRI, but a few things have to line up:
1. Generally you need to run Java 6, which is by far the fastest release of Java yet.
2. You need to run the “server” more in Java, which runs considerably faster.
3. If possible, you need to enable the JRuby compiler to get the best performance.
4. Disabling objectspace can get you another big gain, since it’s just pure overhead for JRuby.
When these four line up, or even three (ObjectSpace isn’t crucial), JRuby consistently outperforms Ruby. And that’s good enough for a lot of people. But we’re working to make it faster an faster, so that folks not running Java 6 can get good performance, and so that the compiler is able to compile more code.
You would probably be able to do 1, 2, and 4 by using the following command line (with Java 6 installed):
jruby SERVER -O short.rb …..
Unfortunately there’s a number of things in your code golfed version that won’t compile, so you won’t get the largest performance boost you could. It’s also important to remember that although code running on the JVM will almost always run very fast, it rarely does so right away. It takes some time for code to get JITted by Java.
All that said, here’s my results for your test, using Java 6 server VM with objectspace disabled. I’ve also created a modified version of your script that could take better advantage of the compiler and compared that.
The primary two things in the golfed version that couldn’t compile is the swap (p, B[x][y] = …) and the each block that accepts two arguments. Both will be compilable soon in the future, but they’re not now. My modification did a slow swap (split it in to two lines) and move the each block out to a separate method. Then JRuby’s JIT is able to compile the “s” method.
ruby time: 8m45.336s
jruby time: 7m21.713s
Note that this would improve if the last each block could also be compiled, rather than moved out to a separate method, but it’s still pretty good. We’ll get everything compiling, hopefully soon after the 1.0 final release.
danvk said,
May 3, 2007 at 9:04 am
Thanks for the followup, Charles. You guys certainly get props for being on top of the JRuby blogosphere. I’m on Mac OS X, so my JVM build is 1.5.0_07-87, which I assume means Java 5.
I’m curious why the “do |a,b| … end” syntax isn’t supported yet. I’ve found it to be a major time saver in my Ruby coding.
Also, is there any official documentation for jrubyc? A google search asks me if I meant “ruby”, and shows lots of source code.
Charles Oliver Nutter said,
May 3, 2007 at 9:33 am
Yep, that would be Java 5. Java 6 is only available through the Apple Developer Connection (which isn’t hard to get into).
The details of why it can’t compile are a little tricky. Basically anything other than a single argument block parses to a multiple assignment. The variable-swapping code also compiles to multiple assignment. Multiassign is probably the nastiest bit of Ruby to get compiling right, because for each variable/element being assigned it may have to do something different. And basically, I haven’t subjected myself to that pain yet. But I came up with a new strategy for compiling it, so there’s a chance it could still be in 1.0 final.
And no, there’s no official documentation for jrubyc. I figured it was mostly experimental until more stuff compiles and people could really start using it regularly. For 1.0 it won’t be an officially supported feature, since it won’t be able to compile everything.
Kungen said,
July 15, 2007 at 12:07 pm
danvk: You don’t have the sources (or even clips) of your online php boggle-solver anywhere? I have tried to make one myself. I have worked on it for few weeks, but still it doesn’t work right. I really can’t think cpp-code as php, so I can’t use those example sources of yours?
Are you planning on publishing those sources someday?
danvk said,
July 17, 2007 at 9:04 am
Kungen: The PHP is just a simple wrapper around the C++ program. I’ve sent it to you as an email, but I have no plans to post the source here, since it’s not particularly interesting.