Don't load the whole measurement file into memory at once -- it may be very large #38
Conversation
| Looks good. The performance improvements are excellent. |
In general, that's right. You have to rebuild it after this change though, due to the change in the signature of "invoked" |
| I forgot to update the tests; I'll fix (EDIT: now done) |
| Oh I see you changed it to a Set not a Seq. |
This is still the case for me: uncovered test reproducibly takes 2m48 +/- 5 and scovered test reproducibly takes 1m21 +/- 2. I have checked "inspect test" and "inspect scoverage:test", "inspect scoverage-test:test" in SBT and I can't see any extra work being done in test. AFAICT "scoverage:test" should do exactly the same stuff as test, except using a different classpath and with the extra "cleanup" step to generate the report. I have checked the detailed output: each individual test is faster uncovered, but the whole thing is slower. Something very funny is going on. It's probably not scoverage's fault: I reckon I have misconfigured SBT or something. |
| Are you sure that scoverage is running all the tests ok and not skipping On 15 May 2014 20:29, Richard Bradley notifications@github.com wrote:
|
Don't load the whole measurement file into memory at once -- it may be very large
| Found it -- tests are not being run in parallel in "Test". I tried to turn that on for our integration tests, but it looks like it's serial execution in Test and Integration, but parallel execution in scoverage-test. |
| That make sense. On 15 May 2014 21:13, Richard Bradley notifications@github.com wrote:
|
2 participants
This change is to scan the measurement files line-by-line, rather than loading the whole thing into memory as a giant string.
With a large test suite, the measurement files can get rather large (30MB for half of our suite).
If we change the "invoked" numbers to be
\ndelimited instead of;delimited, then we can use the standard line-by-line buffered file reading libs to read in the statement ids instead of needing to read a huge string then doing a (memory-expensive)splitinto a huge array.Note that this creates a binary incompatibility -- you will need to recompile
sbt-scoverage(no source changes are needed in that project).This takes my test run (sbt scoverage:test, without a "clean") from 1m28 to 1m22. This number may overstate the improvement (or hopefully understate it); I haven't done very many tests.
Performance measurements digression
Just for interest, while I'm measuring performance, here are some numbers from our fairly large multi-threaded test suite (477 tests). Note that this is just the "test" part, not the integration test part. This is on my Windows dev machine, using Java 6 and Scala 2.10.3
Time to run "
sbt scoverage:test" without a clean (i.e. after a previous run has done all the compilation):I don't quite believe this, but I have just checked and re-checked this, and it seems like "
sbt test" takes longer than "sbt scoverage:test", at about 2m06. I'm going to have to dig into that!(The variance of each of the measured numbers above looks to be around 5%)