Benchmark Dalvik, Hard-Float OpenJDK for ARM and JAVA SE with SciMark
Why?
Benchmarking SciMark with the hard-float OpenJDK for ARM and Dalvik on the same platform is a means for approximating the amount of headroom available to optimize in Dalivk.
Context?
This is part of the larger ARM benchmarking effort and to inform a possible Dalvik optimization effort.
What gets produced?
A set of measurements on a Wiki, documentation for producing those measurements and a script to automate the measurements.
Where will the work get put?
Th scripts will be put in git and the docs on the Wiki or Google Doc.
Blueprint information
- Status:
- Complete
- Approver:
- Zach Pfeffer
- Priority:
- High
- Drafter:
- vishal
- Direction:
- Approved
- Assignee:
- vishal
- Definition:
- Approved
- Series goal:
- Accepted for 2012q3
- Implementation:
-
Implemented
- Milestone target:
-
12.10
- Started by
- vishal
- Completed by
- Zach Pfeffer
Related branches
Related bugs
Sprints
Whiteboard
Notes:
[2012/9/25 pfefferz] Please put notes here.
[2012/10/03 vishalbhoj] We can run the scimark2 jar on android after converting it to dex format.
[2012/10/04 vishalbhoj] Dalvik seems to beat OpenJDK comfortably (both in server configuration and client configuration) by 1.5x to 2x better performance by Dalvik and that is because OpenJDK is running in interpreter mode. I tried to use CACAO JVM (which has someJIT support) which crashed on OpenJDK 6 and it doesn't seem to be supported on OpenJDK 7 yet.
[2012/10/22 vishabhoj] Here are the results of the profiling for speed:
From current analysis this is what I see on Panda :
For OpenJDK Interpreter mode :
Composite Score: 36.30357859627802
FFT (1024): 24.552936620511193
SOR (100x100): 76.22448400556964
Monte Carlo : 9.044321200804992
Sparse matmult (N=1000, nz=5000): 33.921325587575936
LU (100x100): 37.77482556692837
On Dalvik with JIT:
Composite Score: 59.69456554120022
FFT (1024): 38.754014007974014
SOR (100x100): 147.5011473414838
Monte Carlo : 11.748750919975837
Sparse matmult (N=1000, nz=5000): 37.118259792851305
LU (100x100): 63.350655643716145
java SE Development Kit 7u6 :
Server Configuration:
Composite Score: 128.53424774750374
FFT (1024): 105.63472204418967
SOR (100x100): 231.49386412669065
Monte Carlo : 79.82023302651025
Sparse matmult (N=1000, nz=5000): 90.59441865315316
LU (100x100): 135.12800088697497
Client Configuration:
Composite Score: 108.99643713221958
FFT (1024): 104.67118856922178
SOR (100x100): 197.1107818625708
Monte Carlo : 29.266840166767814
Sparse matmult (N=1000, nz=5000): 78.42987125282515
LU (100x100): 135.50350380971238
Meta:
Headline: SciMark has been benchmarked against Dalvik and Hard-Float OpenJDK
Acceptance:
1. SciMark has been benchmarked against Dalvik
2. SciMark has been benchmarked against Hard Float OpenJDK
3. Scripts have been written that run the benchmarks
4. Results have been published
Work Items
Work items:
Run SciMark into Android builds: DONE
Benchmark SciMark against Dalvik for speed: DONE
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for speed: DONE
Benchmark Scimark against java SE Development Kit 7u6 : DONE
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for start-up time in long-time-
Benchmark SciMark against Hard Float OpenJDK for ARM on Ubuntu for start-up time in quickly-
