InputStream test. Updated readme.

[zapov]

Add missing test for reading off InputStream.
Updated DSL-JSON to v1.0
Updated README with latest run.

[cowtowncoder]

Hi there! I'd be happy to update the changes, minus result updates -- I can re-run them once everything works, but I don't want to include results that I did not run myself, at least on README.
But I would be open to adding result pages to wiki, where they could be contributed by anyone, if that would work?

[cowtowncoder]

Actually moving results from README makes sense in general, easier to update, add visualizations etc.
So would it be possible to just include code + config changes on this PR?

[zapov]

Np. Gl hf ;)

[zapov]

btw. I think you should change read test base to consume hashCode instead of the returned instance

This will force your lazy Map to actually build the structure instead of just faking it.
While I don't have anything against that kind of optimizations, those lazy actions just cause confusion in the benchmarks.

[cowtowncoder]

@zapov I don't think I have any lazy maps; maybe you are thinking of Boon, which has a set of aggressive optimizations? I definitely agree that leaving lazy optimizations in effect is unfair and not indicative of actual performance.

I'd be happy to change code to use hashCode() if it's the last thing after looping; for case of Maps it could otherwise become a hotspot of its own if included within busy loop?

[cowtowncoder]

@zapov thanks -- I hope to find time to look into this and other updates, and maybe get results updated. I don't want to have old information given changes and inclusion of many fast(er) implementations.

[zapov]

I'm pretty sure it was Jackson which was like 10x faster than others. When I added hashCode, it went down to like 20% faster than others. I didn't debug it for the actual cause, but I took a look at Jackson code and saw that DeferredMap. I see now that it's behind a flag, so I'm not sure anymore why that might be possible.

I'll take a look again, hopefully soon ;)

[cowtowncoder]

@zapov Ok: if it's with Jackson I'll be happy to change that as well, I just can't think of a place off the top of my head. :-D

It could be within jackson-jr as I did have some ideas there, based on what Boon was doing, and because there's less room for configurability.
Regardless, use of hashCode() sounds like a good idea regardless, or some kind of verification to ensure laziness does not skew results.

[zapov]

btw. the results I'm talking about are:
java -Xmx256m -jar target/microbenchmarks.jar "._DZoneReadMap._read100kFromStream.*" -wi 3 -i 3 -f 3

Benchmark Mode Cnt Score Error Units
DZoneReadMapBoon.read100kFromStream thrpt 9 3.798 ± 3.099 ops/s
DZoneReadMapDslJson.read100kFromStream thrpt 9 3.551 ± 1.429 ops/s
DZoneReadMapFastjson.read100kFromStream thrpt 9 4.886 ± 2.201 ops/s
DZoneReadMapGSON.read100kFromStream thrpt 9 3.903 ± 1.077 ops/s
DZoneReadMapJackson.read100kFromStream thrpt 9 6.996 ± 1.284 ops/s
DZoneReadMapJacksonJr.read100kFromStream thrpt 9 18.317 ± 1.331 ops/s
DZoneReadMapJohnzon.read100kFromStream thrpt 2 0.898 ops/s
DZoneReadMapJsonIO.read100kFromStream thrpt 9 1.297 ± 0.646 ops/s
DZoneReadMapJsonMoshi.read100kFromStream thrpt 9 2.187 ± 0.921 ops/s
DZoneReadMapJsonParse.read100kFromStream thrpt 9 2.314 ± 1.307 ops/s

with consume of hashCode

DZoneReadMapBoon.read100kFromStream thrpt 9 1.384 ± 0.068 ops/s
DZoneReadMapDslJson.read100kFromStream thrpt 9 3.712 ± 1.440 ops/s
DZoneReadMapFastjson.read100kFromStream thrpt 9 4.837 ± 0.094 ops/s
DZoneReadMapGSON.read100kFromStream thrpt 9 3.399 ± 0.866 ops/s
DZoneReadMapJackson.read100kFromStream thrpt 9 5.409 ± 1.204 ops/s
DZoneReadMapJacksonJr.read100kFromStream thrpt 9 3.971 ± 1.413 ops/s
DZoneReadMapJohnzon.read100kFromStream thrpt 9 0.929 ± 0.421 ops/s
DZoneReadMapJsonIO.read100kFromStream thrpt 9 1.331 ± 0.165 ops/s
DZoneReadMapJsonMoshi.read100kFromStream thrpt 9 1.888 ± 0.351 ops/s
DZoneReadMapJsonParse.read100kFromStream thrpt 9 2.702 ± 0.768 ops/s

Lazy map seemed like an obvious culprit. If you find out what it was, let me know. Otherwise I'll have to dig into it myself and see what it was.
Interestingly, it's only much faster on 100k, so it might be just an optimization within JVM (maybe JVM switched to lazy map when initialized through reflection. I heard they did something similar with array lists).

[cowtowncoder]

@zapov It looks like 2 impls that may indeed be using lazy instantiation (Boon, Jackson-jr) have difference, so your guess is probably good one. I was just confused because standard Jackson does not have anything special handling as it builds Maps eagerly.

WIll have a look at this later today. And also figure out how install mono etc. :)

[zapov]

I still think either Jackson or JVM is doing something lazy with Maps since Jackson has 30-40% difference with or without hashCode()

Did you managed to set everything up? Was it painful?

[cowtowncoder]

@zapov I haven't figured out Mono setup. As to settings of Map handling: fix went in 11 days ago:

8b9610e

[zapov]

That sounds... worrisome ;( I though setting up Mono should be a no brainer. There is installer for Mac and Linux has it through apt-get.
If people have issues with setting up Mono I should revisit my stance on that being problematic dependency ;(

As for the fix, yeah I did saw that, but here is still 10% difference with/without hashCode
I guess it comes down to Jackson caching map keys which gets free hashCode for keys.
hashCode on map results in iteration of all entries hash keys which are sum of key ^ value.
This results in Jackson having to calculate value hashCode which it doesn't have to if it just consumes it.

Not entirely sure if this is valid benchmark optimization, but I don't have strong objections to it at this point.

Let me know if there is anything I can help you with regards to setting up Mono. I use this installer for Mac: http://www.mono-project.com/download/#download-mac

I would love for you to finish up this bench and send it over to DZone ;)

[cowtowncoder]

@zapov sorry for not following up on this -- I hope to find time to figure things out, blocker is lack of time and not so much problems. Write up for DZone sounds like a great idea.

Also I think you may well be right wrt hash code: due to canonicalization (which is done at low level, regardless of what databinding does, unless explicitly disabled) String instances are reused and thus hashCode() re-calculation avoided. So in that sense accessing results may add unfair cost to non-Jackson implementations. On the other hand Jackson decoder is heavily optimized assuming canonicalization is enabled so disabling that is generally not recommended so there's some apples/oranges comparison in the end.
Whether results need to be accessed is an open question; I think Boon does default to lazy Lists/Maps.