Performance

In the source code download there is a document entitled "DeltaCodec Performance v1.3.pdf".

That document is quite large because it contains the comparative results for 32 prefabricated test groups that cover all included codec types, data types, and parameter settings in various combinations. While it serves as a useful reference, no one is expected to read straight through all of that information (except for me of course).

What I'll do here instead is show a few selected results from different groups that highlight key aspects of codec behavior.

To view the results presented here you will almost certainly have to adjust the ZOOM in your browser. The examples shown on the Home page don't include all of the fields of the actual test output. Let's go over the format and content of that output now.

Test Output

There are three areas of information available in the output for each test:
  • 1. Summary Header Block – Basic information about the data that was encoded.
  • 2. Results Table – Comparative performance results for each individual codec.
  • 3. Data Samples – A small sample of the data (values and bit strings).

The summary information at the top of the output currently looks something like this:

SummaryHeaderBlock.png

The Results Table contains lines for individual codecs, each configured with a specific set of parameters.

ResultsTable.png

The various fields provide a variety of information to help assess configuration and performance:
  • EncType: Base name of the codec. The suffix “BP” is added for block parallel encoding.
  • Factor: The type of factoring applied.
    • None – No factoring.
    • Auto – Granularity is calculated and used to factor the data.
    • Gran – Granularity is provided, so it doesn’t need to be calculated.
  • Level: The CompressionLevel used (translated roughly to third-party equivalents).
    • NoCompression – The finisher is by-passed entirely.
    • Fastest – Time emphasis.
    • Optimal – Space emphasis.
  • Parts: The number of blocks that are encoded in parallel (“numBlocks” parameter).
  • Ratio: Encoded bytes divided by raw bytes.
  • Multiple: Reciprocal of the ratio.
  • Enc(ms): Time spent encoding (milliseconds).
  • Dec(ms): Time spent decoding (milliseconds).
  • CmbsEnc: Megabytes per second reduced. Better measure than raw throughput.
  • CmbsDec: Megabytes per second restored. Better measure than raw throughput.
  • SpaceRank: Compression ratio compared to other codecs.
  • TRankEnc: Time rank encoding compared to other codecs.
  • TRankDec: Time rank decoding compared to other codecs.
  • Balance: Average of space and time rankings compared to other codecs.
  • Entropy: Shannon Entropy of encoded bytes.
Together these statistics give a fairly complete picture of relative performance within the group.

After the Results Table a small set of Sample Data is displayed (from a different test to make a point).

SampleData.png

This DateTime sample includes the normal display values, the tick values, and the bit strings. For integer or floating point data you will only see the values and the bit strings.

With that out of the way, we're now ready to look at a few interesting examples!

NOTE: Since it would be overwhelming to show results for every data type in this venue, I've just randomly chosen Int32 to be consistent in the following examples. That's not very representative, but hopefully you'll browse through more of the tests in the offline documentation.

NullTransform vs DeltaTransform

This test will show how Ratio/Multiple is improved when the data is first differenced before it is compressed with a finisher.

SerialNullTransformVersusDeltaFast_Int32.png

The easiest way to see the improvement is by looking at the paired Multiple statistics (normal vs delta).

Serial vs Parallel

This test will show how encoding multiple blocks in parallel greatly improves encoding and decoding speed.

SerialVersusParallelFast_Int32.png

Notice the Parts field in the results. Whenever the number of blocks is greater than 1, the codecs perform parallel block encoding.

Optimal vs Fast

This test shows how codec speed is improved at the expense of ratio based on the compression level used.

ParallelOptimalVersusFast_Int32.png

In this case, we have to look at both speed and ratio/multiple statistics to make comparisons.

Factoring Comparison

This test shows how different factoring modes affect performance.

ParallelFactoringComparisonOptimal_Int32.png

There seems to be no consistent pattern with factoring, except in the case of RWC which always benefits. Some codecs do better with it for certain data types but not for others. Other codecs can behave exactly the opposite. This particular transform choice must be tested thoroughly with the particular data you are targeting!


Last edited Jun 21, 2015 at 6:34 PM by bstabile, version 63