Computer Systems Performance Modeling And Evaluation - Part 3

Introduction

Now, after discussing performance evaluation, we move on to understanding workload and its characterization with real-life examples. Firstly, we learn what is workload and its construction.

Workload (WL)

The quantity and nature of requests submitted to the system during some given period of time to model or drive the system. E.g. the no. of requests submitted to the database server per second. i.e. the intensity of the workload. 

Workload also depends on the nature of the request. All database requests are not equivalent. 

No. of instructions per second and the mix of instruction types presented for execution per second.

Duration of the load

1. All at once, requiring the system to queue up the requests and perform them as resources become available.
2. Endless, with the load continually refreshed to provide a constant saturation load to the system.
3. Periodic, where the load is reentered after the prescribed period of time.

Construction of the load

The Workload is constructed based on the focus of the analysis. e.g., if we are interested in comparing the hardware of two computer systems, our focus may be on the low-level instructions. 

Another example is to measure transaction throughput through some database system. This WL would have transactional units of work that read and write data items from the DB system and do some additional computational work mimicking the real system.

Controllable vs uncontrollable parameters of workload

Parameters that can be controlled by the system designer/ administrator, e.g.,

• the scheduling disciplines,
• inter-connections between devices and
• resource allocation policies.

Uncontrollable parameters, e.g.,

• the inter-arrival times and
• service demands of incoming jobs.

These inputs can be used for driving the real system or its simulation model, and for determining distributions for analytic/simulation modeling.

There is a science to WL development and selection for the computer systems modeler. e.g., the database community has developed a set of transactional WLs (TP benchmarks) for testing a variety of DB systems configurations.

Likewise, the PC industry has developed a set of systems workloads allowing customers to assess the performance of one computer architecture against another.

Workload Characterization

It involves deciding about different aspects of the workload (controllable and uncontrollable parameters etc), level of detail for recording the workload, and representation of the workload. The WL characterization problem becomes more difficult when designing or evaluating a system for an anticipated need. In this case, we must generate the representative WL artificially.

Workload characterization only builds a model of real workload, since not every aspect of real WL may be captured or is relevant.

Analytic modeling usually requires a rather abstract WL model. E.g., characterization in terms of statistical properties. However, simulation can use a very detailed model. E.g., a trace of actual events.

For reasons of cost-effectiveness and robustness, it is desirable to keep the model (and hence the WL) as abstract as possible. For example, when evaluating the effectiveness of an instruction pipeline or a cache, we need information on instruction sequences in addition to the relative frequency of various instructions. However, the actual instruction traces are not necessary for effective WL characterization.

It may be adequate to capture only the first order sequencing effects represented by the probabilities (or relative frequencies) of the form Pij which gives the probability that the jth instruction will immediately follow the ith instruction.

A WL model may be executable or nonexecutable. For example, recording the arrival instants and service durations of jobs creates an executable model, whereas only determining the distributions creates a nonexecutable model. An executable model need not be a record of inputs, it can also be a program that generates the inputs.

Executable WLs are useful in direct measurements and trace-driven simulations.

Nonexecutable WLs are useful for analytic modeling and distribution-driven simulations.

You might like:

• Computer Systems Performance Modeling And Evaluation - Part 1
• Computer Systems Performance Modeling And Evaluation - Part 2
• Computer Systems Performance Modeling And Evaluation - Part 4

Thank you for reading!

This is PART 3 of the series, I have explained the workload and its characterization with real-life examples to improve the system's performance.

If you found this article useful, feel free to go for PART 4 of this awesome blog series.

Cheers!