JVM performance over time

Out of curiosity, I decided to run some quick (and non-repeated, largely unscientific) benchmarks of a Gentoo (-march=native,etc etc) OpenJDK/IcedTea build vs. actual Sun/Oracle JVM builds. Interestingly, the Gentoo builds are slower. Unsurprisingly, the JVM gets noticeably faster across versions from 1.6->1.7->1.8

Not all of the benchmarks passed locally, and one of them (batik) passed on the official builds but not on OpenJDK because of the lack of the com.sun JPEG encoder.

[nate@tesla /dev/shm] $ for a in /opt/*/bin/java /usr/lib/icedtea?/bin/java ; do $a -version ; $a -jar ~/Downloads/dacapo-9.12-bach.jar avrora h2 jython luindex lusearch pmd sunflow xalan 2>&1 | grep PASSED ; done

java version "1.6.0_45"
Java(TM) SE Runtime Environment (build 1.6.0_45-b06)
Java HotSpot(TM) 64-Bit Server VM (build 20.45-b01, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1869 msec =====
===== DaCapo 9.12 h2 PASSED in 4949 msec =====
===== DaCapo 9.12 jython PASSED in 8113 msec =====
===== DaCapo 9.12 luindex PASSED in 883 msec =====
===== DaCapo 9.12 lusearch PASSED in 3068 msec =====
===== DaCapo 9.12 pmd PASSED in 4061 msec =====
===== DaCapo 9.12 sunflow PASSED in 2476 msec =====
===== DaCapo 9.12 xalan PASSED in 2678 msec =====

java version "1.7.0_80"
Java(TM) SE Runtime Environment (build 1.7.0_80-b15)
Java HotSpot(TM) 64-Bit Server VM (build 24.80-b11, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1910 msec =====
===== DaCapo 9.12 h2 PASSED in 4270 msec =====
===== DaCapo 9.12 jython PASSED in 7551 msec =====
===== DaCapo 9.12 luindex PASSED in 991 msec =====
===== DaCapo 9.12 lusearch PASSED in 2027 msec =====
===== DaCapo 9.12 pmd PASSED in 3400 msec =====
===== DaCapo 9.12 sunflow PASSED in 2227 msec =====
===== DaCapo 9.12 xalan PASSED in 2283 msec =====

java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1882 msec =====
===== DaCapo 9.12 h2 PASSED in 3864 msec =====
===== DaCapo 9.12 jython PASSED in 4860 msec =====
===== DaCapo 9.12 luindex PASSED in 688 msec =====
===== DaCapo 9.12 lusearch PASSED in 1016 msec =====
===== DaCapo 9.12 pmd PASSED in 1929 msec =====
===== DaCapo 9.12 sunflow PASSED in 2235 msec =====
===== DaCapo 9.12 xalan PASSED in 1387 msec =====

(duplicate because of a path alias)
java version "1.8.0_92"
Java(TM) SE Runtime Environment (build 1.8.0_92-b14)
Java HotSpot(TM) 64-Bit Server VM (build 25.92-b14, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1921 msec =====
===== DaCapo 9.12 h2 PASSED in 4238 msec =====
===== DaCapo 9.12 jython PASSED in 5382 msec =====
===== DaCapo 9.12 luindex PASSED in 871 msec =====
===== DaCapo 9.12 lusearch PASSED in 1185 msec =====
===== DaCapo 9.12 pmd PASSED in 2020 msec =====
===== DaCapo 9.12 sunflow PASSED in 2254 msec =====
===== DaCapo 9.12 xalan PASSED in 1471 msec =====

java version "1.7.0_101"
OpenJDK Runtime Environment (IcedTea 2.6.6) (Gentoo icedtea-7.2.6.6)
OpenJDK 64-Bit Server VM (build 24.95-b01, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1969 msec =====
===== DaCapo 9.12 h2 PASSED in 4567 msec =====
===== DaCapo 9.12 jython PASSED in 7169 msec =====
===== DaCapo 9.12 luindex PASSED in 980 msec =====
===== DaCapo 9.12 lusearch PASSED in 1474 msec =====
===== DaCapo 9.12 pmd PASSED in 3223 msec =====
===== DaCapo 9.12 sunflow PASSED in 2467 msec =====
===== DaCapo 9.12 xalan PASSED in 2120 msec =====

openjdk version "1.8.0_91"
OpenJDK Runtime Environment (IcedTea 3.0.1) (Gentoo icedtea-3.0.1)
OpenJDK 64-Bit Server VM (build 25.91-b14, mixed mode)
===== DaCapo 9.12 avrora PASSED in 1875 msec =====
===== DaCapo 9.12 h2 PASSED in 4075 msec =====
===== DaCapo 9.12 jython PASSED in 5677 msec =====
===== DaCapo 9.12 luindex PASSED in 705 msec =====
===== DaCapo 9.12 lusearch PASSED in 1166 msec =====
===== DaCapo 9.12 pmd PASSED in 1785 msec =====
===== DaCapo 9.12 sunflow PASSED in 2331 msec =====
===== DaCapo 9.12 xalan PASSED in 1851 msec =====

(See also:
Blackburn, S. M., Garner, R., Hoffman, C., Khan, A. M., McKinley, K. S., Bentzur, R., Diwan, A., Feinberg, D., Frampton, D., Guyer, S. Z., Hirzel, M., Hosking, A., Jump, M., Lee, H., Moss, J. E. B., Phansalkar, A., Stefanovic, D., VanDrunen, T., von Dincklage, D., and Wiedermann, B. The DaCapo Benchmarks: Java Benchmarking Development and Analysis, OOPSLA ’06: Proceedings of the 21st annual ACM SIGPLAN conference on Object-Oriented Programing, Systems, Languages, and Applications, (Portland, OR, USA, October 22-26, 2006) (pdf, BibTeX).

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.