Writing the bench method

To write a bench method, you subclass the standard bench class EsbBench. The new subclass is stored in the library and is visible to all Smalltalk browsers. Next, you write a method in the new subclass. Bench methods must start with an uppercase letter or they will not be visible from the Stats user interface.

Because the Stats tool has fewer editing features than a Smalltalk browser, you might write the methods in a Smalltalk browser

A bench method is a Smalltalk method that contains selected code from an application program. Areas of interest are timed using instances of high-precision software timers, which are available as inherited instance variables of EsbBench.

Inherited instance variables

EsbBench provides inherited instance variables, which are automatically initialized and ready to use in all subclasses.

Note:

EsbBench reimplements the class method new to send initialize to the new instance after it has been created. Any subclass of EsbBench must send super new when reimplementing new, and super initialize when reimplementing initialize. Failure to do so causes the inherited instance variables of EsbBench to be uninitialized, and the Stats tool will not execute the method.

See Timing sections of code for an example of how to use the instance variables. The instance variables are as follows:

name

A string that provides a descriptive name for the bench method. The default is an empty string.

iterations

An integer used in conjunction with timesRepeat: to control the number of times an operation runs. The default is 10.

benchTimer

A high-precision software timer used to measure the operation. You can start and stop the timer multiple times as long as the start and stop operations match. You cannot start a timer that is already started or stop one that is already stopped.

differenceTimer

A high-precision software timer used to remove a fixed overhead from the operation. As with benchTimer, you can start and stop a differenceTimer.

Timing sections of code

The following is an example bench method that measures the speed of an operation to open a window:

EtClassesBrowserNewOpen

"Bench EtClassesBrowser new open."

| w |

name := 'EtClassesBrowser new open'.

iterations timesRepeat: [

benchTimer start.

w := EtClassesBrowser new open.

benchTimer stop.

w close].

Using benchTimer, you surround critical portions of the code. The above example includes the time to open a window but ignores the time to close it. The iterations loop surrounds the start and stop messages to benchTimer, allowing benchTimer to collect statistics on every iteration.

In the case of operations that take less than one millisecond, have the start and stop messages surround the iterations loop. The loop must be included in the operation because the time to execute the operation is so short that it cannot be measured from Smalltalk. To remove the cost of the loop and any other fixed cost of the operation, use the differenceTimer.

The following is a bench method that measures the speed of a very short operation:

SendDropDummy

"Send and drop #dummy for a specific number of iterations."

name := 'self dummy'.

self dummy.

benchTimer start.

iterations timesRepeat: [self dummy].

benchTimer stop.

differenceTimer start.

iterations timesRepeat: [].

differenceTimer stop.

 

Note:

Bench methods, such as the one above, are usually needed only when benchmarking primitives or low-level virtual-machine operations. Timing very short operations, such as a message-send operation, requires extra care.

Advantages of bench methods

Writing a bench method has the following advantages over repeatedly executing the expression Time millisecondsToRun: ...] in a workspace and viewing the results:

•	Bench code is written in a standard format.

This means that the code is managed by VA Smalltalk. You include any application code that is needed to set up the initial conditions for the benchmark and all code is captured in the database. This makes benchmarks accessible to other members of the team.

•	Benchmark execution is monitored and controlled by the Stats tool user interface.

Code is monitored by gathering statistics while the code is running. Execution is controlled by configuring the initial and current state of the Smalltalk virtual machine. The time to scavenge and to perform global garbage collection can be filtered from the raw time and viewed separately.

•	Results are stable and repeatable.

The Stats is unobtrusive; raw stack data is gathered while the benchmark code executes but is not analyzed until execution completes.

•	Bench code is independent of the user interface.

While bench code is often built and executed through the Stats user interface, it can also be constructed from other browsers and executed without the user interface. Over time, while optimizing the code, the programmer builds up a suite of benchmarks, which can be run periodically to ensure that code does not become slower.

•	On some virtual machines, code that is executed in a method may be slightly faster than the same code executed in an expression.

This means that code executed in a method produces a more accurate measure of the actual execution time than does the same code executed in an expression.