A few weeks ago I ran some tests pitting method handles against typical reflection to see what was faster. The result was that reflection was faster although since it was a quick test I don't have the result data or what run configuration I used. Now that I'm tasked with looking into porting our core systems to Java 7 I thought I'd take a more critical look.
Below is the source code I used to run the test. It calls a method on a class called Incrementer a million times. I didn't notice any difference when running the methods several times when running the test several times before timing values to "warm up" server mode.
Note that this code throws an exception when run from eclipse. I had to use the javac/java executables from the command line for this to work. When running the class in mixed mode I got the following results:
java -verbose:gc MethodHandleTest
[GC 4416K->141K(15872K), 0.0011714 secs]
[GC 4557K->141K(15872K), 0.0006216 secs]
[GC 4557K->141K(15872K), 0.0006387 secs]
reflection took 515ms, result is 1000000
method handles took 16ms, result is 1000000
There is plenty of garbage collection happening but not enough to account for the huge time difference. When running in server mode I got the following results:
java -verbose:gc -server MethodHandleTest
reflection took 47ms, result is 1000000
method handles took 15ms, result is 1000000
java -verbose:gc -server MethodHandleTest
reflection took 32ms, result is 1000000
method handles took 31ms, result is 1000000
All runs of the test gave almost identical results to the above where method handles would be twice as fast as reflection or equal with no garbage collection. Interestingly, if I comment out the increment of the long value the same result as with the increment is produced:
java -verbose:gc MethodHandleTest
[GC 4416K->141K(15872K), 0.0011488 secs]
[GC 4557K->141K(15872K), 0.0006595 secs]
[GC 4557K->141K(15872K), 0.0006482 secs]
reflection took 500ms, result is 0
method handles took 31ms, result is 0
This makes me think that incrementing the long is a very small proportion of the time it takes for the reflection example to run and the majority is for calling the method through reflection. In server mode the reflection call is JITted and almost identical results are produced. This is mostly conjecture since it is quite difficult to work out what the Hotspot compiler is doing.
In conclusion, method handles consistently performs as good as, or better than reflection no matter what run configuration you are using. Stability is an issue with the current tools, eclipse will have to support Java 7 before I could use method handles in our systems.
5 comments:
It would be interesting to know whether putting
method.invoke(increments, "12345");
and
methodHandle.invokeExact(increments, "12345");
into separate methods would make a difference. That would allow the JIT to optimise it better.
I retried the test using methodHandle.invoke instead of methodHandle.invokeExact and there was no difference.
You seem to be making a JIT warmup mistake. Run the whole test in a loop 100 times and you will see that Reflection is still way faster than MethodHandles.
Just wrap the test code in a loop:
public static void main(String[] args) throws Throwable {
for (int i = 0; i < 100; i++) {
usingReflection();
usingMethodHandles();
System.out.println();
System.gc();
}
}
reflection took 31ms, result is 1000000
method handles took 18ms, result is 1000000
..
..
reflection took 3ms, result is 1000000
method handles took 14ms, result is 1000000
...
reflection took 4ms, result is 1000000
method handles took 14ms, result is 1000000
reflection took 4ms, result is 1000000
method handles took 20ms, result is 1000000
BTW that was on JDK_1.7.03 64bit, Windows 7, i7 2.6 GHz
Hi Ashwin
It is suprising that hotspot is still optimising after more than a million method calls. Good job on finding this.
As a general statement, I don't believe however that reflection is way faster than method handles. Although you have found a situation where it is, hotspot may not be in the position where it has already executed the method 100 million times in server mode.
I will update the post to show that servers under high stress might still be better off with reflection.
Cheers!
Post a Comment