Mar 28, 2014

Follow-up: JavaScript on the JVM, experimenting with Nashorn

The release of JDK8 was a good opportunity to test again how fast the Nashorn JavaScript engine is in compiling Less style sheets.
I hoped that performance improved since I last tried an early access build, but alas...

Setup

I recently released a new version of lesscss that supports 3 execution engines:
  • rhino 1.7R4: it runs a pre-compiled version of less.js in the highest optimization level (9)
  • nashorn: the JDK8 built-in version, which does not (yet) support pre-compilation nor optimization
  • node.js lessc: prepare a commandline string and execute it with Runtime.exec
I compiled the Twitter Bootstrap style sheets using different engines.
I used my mac book (2.66GHz core i7 with 8GB memory and JVM args -Xms512m -Xmx1500m -XX:MaxPermSize=320m).
For each run I instantiated one engine for 15 consecutive compilations.

As you can see, both rhino and nashorn perform better when the JVM warmed up, but the differences in performance are huge:

Time needed to compile Twitter Bootstrap (seconds)
  The first row includes JavaScript compilation in case of Nashorn
# Rhino
(optimized)
Nashorn node.js
1 3.9 18.4 0.9
2 1.9 7.9 0.6
3 1.7 6.1 0.6
4 1.6 5.8 0.6
5 1.4 5.0 0.6
6 1.4 4.9 0.6
7 1.5 4.5 0.6
8 1.4 5.3 0.6
9 1.3 5.8 0.6
10 1.3 4.4 0.6
11 1.0 4.0 0.6
12 1.0 3.8 0.6
13 1.0 3.5 0.6
14 0.9 3.1 0.6
15 0.9 2.8 0.6

Conclusions

Although this is not a real benchmark, the figures differed less then 20% between runs and the trends are clear (at least for this use case):
  • Rhino with optimizations is still a lot faster then nashorn
  • While rhino got faster on JDK8, nashorn seems to got slower (see previous blog)
  • There is a big gap with node.js, despite the overhead of spawning a new process and writing its output to disk