Viewing the results

You view the results in the line graph and the bar graph. The Time menu filters raw time. The Results menu selects the type of information that is displayed in the graphs.

Filtering raw time

The Time menu filters raw execution time for the line and bar graphs. The following mutually exclusive items are available from the menu:

Raw Time

When this is checked, the graphs show raw execution time. This is the actual time spent in the operation.

Execution Time

When this is checked, the graphs show the time spent actually running the code. This is the raw execution time minus the time to scavenge and perform global garbage collection.

Scavenge Time

When this is checked, the graphs show only the time spent scavenging.

Global GC Time

When this is checked, the graphs show only the time spent performing global garbage collection.

When optimizing code or testing a new algorithm, you typically view execution time. When reducing the amount of garbage, you typically view scavenge or global garbage collection time. Raw time may show spikes when garbage collection occurred.

Comparing benchmarks

The Results menu selects the comparison metric that the bar graph displays. The following mutually exclusive items are available from the menu:

Show Statistics

When this is checked, the bar graph displays the following statistics:

•

Mean

•

Median

•

Minimum

•

Maximum

•	Standard deviation

The bars show the mean time for each benchmark. The standard deviation as a percentage of the mean is a good measure of the stability of a benchmark. Values below 5% indicate a stable benchmark.

Show Garbage

When this is checked, the bar graph displays the following:

•

Raw time

•	Execution time

•	Scavenge time

•	Time for global garbage collection

The bars show the percentage of time spent collecting garbage for each benchmark. This option is useful when reducing the amount of garbage created by the application code. Usually, values below 5% are acceptable. Improving the algorithm or optimizing the code almost always has a greater affect than reducing garbage.

Show Percents

When this is checked, each benchmark is compared against the baseline; the percentage improvement is displayed in the bar graph. The baseline benchmark is always the first selected benchmark in the benchmark list.

The formula for the percentage improvement is (A-B)/A, where A is the baseline and B is the candidate for improvement. A positive value shows improvement in the code, negative shows degradation, and zero shows no change.

Show Difference

When this is checked, each benchmark is compared against the baseline; the difference in execution time is displayed in the bar graph.

The formula for the difference is A-B, where A is the baseline and B is the candidate for improvement.

Show Ratios

When this is checked, each benchmark is compared against the baseline; the benchmark ratio (also called the relative value) is displayed in the bar graph.

The formula for the ratio is A/B, where A is the baseline and B is the candidate for improvement. A value less than one shows degradation, greater than one shows improvement, and equal to one shows no change.

The percentage improvement and the benchmark ratio present the same information in different ways. An improvement of 50% in benchmark B over benchmark A means that B is two times faster than A.

Storing, loading, and clearing benchmarks

The Results menu also stores, loads, and clears benchmarks. The following menu items are available:

Store Benchmarks

Prompts for a file in which to store all benchmarks in the Benchmark Workshop. Selecting an existing file overwrites the file.

Load Benchmarks

Prompts for a file to be loaded into the Benchmark Workshop. Benchmarks can be merged or the current set replaced by those in the file.

Clear Benchmarks

Destroys all benchmarks in the Benchmark Workshop. You are prompted for confirmation before the benchmarks are destroyed.

The Stats tool does not save sample [S] or trace T] information in the benchmark file. To load a benchmark file, the classes and methods that were used to create the benchmarks must be in the image.