Thursday, September 15, 2016

Follow-up on bechmarking 7-zip

Couple of notes on things I found out of during benchmarking 7-Zip:
  • Each combination of parameters was run 4 times and then averaged. I always checked the deviation, so I would catch the runs during which OS did something on background and slowed the benchmark. And boy, that happened a lot! I had to rerun almost half of ZIP format runs, just because one run was off by 10%. Thanks a lot, Windows 10. 
  • What I am particularly proud of is run with combination of ZIP format, highest compression level and 8GB image file. That took almost and hour and the difference between highest and lowest time was less than 1 second. It means less than 0.03% difference, which I think is quite impressive for machine which dynamically changes CPU speed (or just a coincidence, IDK).
  • My poor old laptop spent more than 10 hours running just the results I published. However, the whole benchmark took more than 3 days, as I ran couple of more runs (see bellow), had to re-run some runs and shut it down during nights so we both get some sleep.
  • Originally I wanted to include also test case with compressing video file. I used 120,43 MB video encoded in H.264 only to find out that the best compression 7-Zip can do is to 120,4 MB. So I guess H.264 is damn good optimized, 7-Zip tried for more than 80 seconds and it figured out how to make it smaller by just 0,025%.
  • As for other archive formats, I did run some tests using XZ format only to find out the results are exactly the same as 7Z format. It would be nice to look at the difference between these formats in the future.
  • (Most important for the future) Making graphs in Excel is really not an option for next time, I have to use some library for generating them automatically. So stay tuned for next benchmark with better graphs.
Edit 16 Sep 2016: As I have some DB schema changed in my mind, I might be truncating the DB in some time ahead. In order to preserve exact results from this benchmark ("for future generations"), I exported the data and uploaded it here.

No comments:

Post a Comment