On 29 Aug 98 at 21:59, Jun Qian wrote:

> could someone tell me how does benchmark software work?

  There are two basic strategies when writing benchmarks:

a.  See how long it takes to do some task n times.

b.  See how many times some task can be done in n seconds.

  As you can see, there are two parts to this.  There is some kind of
timer -- either a way to measure elapsed time, or something that will
"go off" like an alarm clock.
  And a "task", a collection of instructions and data for the CPU to
work on.  More in a moment....

> I lost count how many ad say whatever their graphics card beats
> voodoo 2 in 3D whatever test.  I'm that kind of person who doesn't
> believe these tests anymore and what to know how they work (because
> I want to know how could they give result like that).

  The first obvious way to get invalid results from a benchmark is to
change the task from one machine to the next.  [Note that some kinds
of changes, if they are, in fact, easy to do, may be allowed....]
  The second problem is that benchmark tasks typically assume[*] that
while a given system may be faster or slower, the ratio between
different parts of the system is constant.  NEC sold a lot of V20
chips, replacement for the 8088, on the claim that it was 3x faster
than an 8088.
  And 3.0x was the number that Norton's SI (System Information)
program would report, not pointing out that (a) I believe this was
8MHz versus the 8088 at 4.77MHz, and (b) Norton's task only used a
couple of instructions, one of which happened to be much faster on
the V20 than on the 8088.
  [*]  This assumption can be true within processors sharing similar
microcode.  The fact that performance of this one instruction took
such a radically different amount of time provides evidence that the
NEC chip was not a clone of the Intel design.  [As a bonus, the NEC
included an 8080 emulation mode in hardware, for those with legacy
8-bit CP/M applications.]

  There's a third obvious way to get invalid benchmark results:
context.  How long does it take to copy a file?  Depends not just on
performance of the drives and controllers, but on where the file
system has put the various pieces of the file.  Unless you can
duplicate that exactly, you can't do a real comparison between drives
or controllers.
  "Context" also covers the case where some parts of the benchmark
task might already be cached.  A common way to try to avoid this
issue is to run the benchmark task several times, and eliminate the
extremes.

  3D cards are often benchmarked using specific pre-defined game
situations.  You can base purchase decisions on those benchmarks IF
you plan to play the chosen game, or games that handle video in
exactly the same way.  [This is typically a type b benchmark, given
in fps:  How many times the CPU was able to (had time available to)
re-draw the screen during the sequence.

David G

                                  -----
       **Need help with PCBUILD mailing list? Send an Email to:**
        Bob Wright<[log in to unmask]> or Drew Dunn<[log in to unmask]>