 | Windows notebooks |
 | |
Performance and battery life are the two most important attributes of a notebook computer--and they significantly influence each other: While most notebooks modulate processor speed and other resource hogs to save energy, those with desktop processors typically sacrifice battery life for raw power. CNET Labs' flagship benchmark for notebooks is BAPCo's MobileMark 2002, which is designed to gauge notebook performance and battery life simultaneously. MobileMark 2002 is run with the notebook powered by its battery alone. In addition, for desktop-replacement notebooks, which often have high-speed CPUs, fast hard drives, and superfast graphics processors, we run BAPCo's SysMark 2002 and Futuremark's 3DMark2001 SE Pro while the notebook is plugged into AC power.
With the exception of 3D graphics testing, all tests are run with Windows' Display Properties set to a notebook's native resolution and at 32-bit color depth. Additionally, Windows' screensaver is disabled.
Before running our tests, we disable all applications that launch at start-up and uninstall any programs that conflict with our tests, including apps that will be installed and used by the benchmarks. We run Windows Update, then turn off automatic updating. We also disable Windows' System Restore, Error Reporting, Remote Assistance, and Remote Desktop. We run all tests with the notebook unattached to any network and with its wireless networking (if any) disabled.
CNET Labs runs MobileMark 2002 on every notebook we test. For notebooks that are considered to be in the desktop-replacement category, we also run SysMark 2002 and 3DMark2001 SE Pro.
According to BAPCo's recommended settings, notebooks that run MobileMark 2002 should be set at a brightness of 60 to 70 nits (or 60cd/m² to 70cd/m²). CNET Labs uses the Minolta CA-210 LCD color analyzer to calibrate notebook brightness.
A detailed description of the recommended system settings that CNET Labs follows when running MobileMark 2002 can be found on BAPCo's Web site (a PDF file). We use these same recommended settings when we run SysMark 2002 and 3DMark2001 SE Pro, with one exception: Instead of the Portable/Laptop power scheme that we use with MobileMark 2002, we use the Always On scheme with SysMark 2002 and 3DMark2001 SE Pro so that a notebook's processor runs at its highest possible speed for both tests.
MobileMark 2002 is an applications-based benchmark that measures both application performance and battery life simultaneously. CNET Labs uses the benchmark's Productivity workload when running MobileMark.
The benchmark runs the following applications: Microsoft Word 2002, Microsoft Excel 2002, Microsoft PowerPoint 2002, Microsoft Outlook 2002, Netscape Communicator (Mozilla 5), McAfee VirusScan 5.13, WinZip 8.0, Macromedia Flash 5.0, and Adobe Photoshop 6.0.1.
MobileMark measures the response time between the simulated user's submission of a task and the completion of that task by the application. It records only the response time for each operation, not the time to reposition the cursor to perform the operation. MobileMark includes application-load times as part of the measurement for its ratings. MobileMark also includes a conditioning run that prepares a notebook's battery for testing by draining the battery completely, then requiring that the battery be fully charged before an official test run can commence. The conditioning run also automatically loads Windows' ProcessIdleTasks routine on notebooks that are using Windows XP so that the system is fully optimized for all applications utilized by the benchmark.
The MobileMark 2002 performance rating is an average of the response times normalized against a fixed calibration platform: a 1GHz Pentium III with a 100MHz bus, 256MB of PC100 SDRAM, an ATI Rage P/M Mobility AGP2X 8MB graphics card, an ATA/66 hard drive, Windows XP Professional, Windows' Display Properties set to 1,024x768 and 32-bit color, and Windows' power scheme set to Always On. A rating of 100 means the test system's performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
The MobileMark 2002 battery rating, expressed in minutes, reflects how long a notebook with a fully charged battery is able to run the MobileMark test. The test concludes when the battery is drained.
A note about CPU throttling: As per BAPCo's recommendations, CNET Labs runs MobileMark 2002 using the Portable/Laptop power scheme. This power scheme allows a notebook to automatically throttle down its CPU speed in order to conserve battery life. The amount of CPU throttling is dependent on a how a manufacturer has configured that particular model and can therefore vary among models from different manufacturers. This can be a contributing factor as to why some notebooks with similar specifications generate different MobileMark 2002 application and battery-life performance scores.
CNET Labs runs SysMark 2002 on only the notebooks that are considered to be in the desktop-replacement category. This benchmark measures a system's overall performance using off-the-shelf applications. SysMark comprises two workload scenarios, reflecting usage models of mainstream business users ("office productivity") and Web designers ("Internet content creation"). The benchmark runs multiple applications simultaneously, continually switching between applications, with some tasks running in the background.
Each scenario rating is an average of the response times normalized against a fixed calibration platform: a 1GHz Pentium III, Intel's 815EEA motherboard chipset, 256MB of 133MHz SDRAM, a GeForce3 graphics card with 64MB of DDR, an ATA/100 hard drive using the FAT32 file system, Windows XP Professional, Windows' Display Properties set to 1,024x768 and 16-bit color at 75Hz, and Windows' power scheme set to Always On. A rating of 100 means the test system's performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
The SysMark 2002 rating is the geometric mean of the Internet-content-creation and office-productivity scenario ratings.
CNET Labs runs 3DMark2001 SE Pro on only the notebooks that are considered to be in the desktop-replacement category. 3DMark2001 measures a notebook's 3D graphics performance using an actual game engine (Remedy Entertainment's MAX-FX Technology). 3DMark2001 uses the DirectX 8 (DX8) interface, which includes features such as pixel shaders, vertex shaders, pixel/point sprites, and volumetric 3D textures. Some of 3DMark2001's tests require DX8 hardware support; a system that does not have DX8 hardware support will typically generate a lower score than one that does. We run 3DMark with Windows' Display Properties set to a 32-bit color depth and with the following benchmark settings:
| Test environment |
|
With the exception of 3D graphics testing, all tests are run with Windows' Display Properties set to a notebook's native resolution and at 32-bit color depth. Additionally, Windows' screensaver is disabled.
Before running our tests, we disable all applications that launch at start-up and uninstall any programs that conflict with our tests, including apps that will be installed and used by the benchmarks. We run Windows Update, then turn off automatic updating. We also disable Windows' System Restore, Error Reporting, Remote Assistance, and Remote Desktop. We run all tests with the notebook unattached to any network and with its wireless networking (if any) disabled.
CNET Labs runs MobileMark 2002 on every notebook we test. For notebooks that are considered to be in the desktop-replacement category, we also run SysMark 2002 and 3DMark2001 SE Pro.
According to BAPCo's recommended settings, notebooks that run MobileMark 2002 should be set at a brightness of 60 to 70 nits (or 60cd/m² to 70cd/m²). CNET Labs uses the Minolta CA-210 LCD color analyzer to calibrate notebook brightness.
A detailed description of the recommended system settings that CNET Labs follows when running MobileMark 2002 can be found on BAPCo's Web site (a PDF file). We use these same recommended settings when we run SysMark 2002 and 3DMark2001 SE Pro, with one exception: Instead of the Portable/Laptop power scheme that we use with MobileMark 2002, we use the Always On scheme with SysMark 2002 and 3DMark2001 SE Pro so that a notebook's processor runs at its highest possible speed for both tests.
| BAPCo MobileMark 2002 |
|
MobileMark 2002 is an applications-based benchmark that measures both application performance and battery life simultaneously. CNET Labs uses the benchmark's Productivity workload when running MobileMark.
The benchmark runs the following applications: Microsoft Word 2002, Microsoft Excel 2002, Microsoft PowerPoint 2002, Microsoft Outlook 2002, Netscape Communicator (Mozilla 5), McAfee VirusScan 5.13, WinZip 8.0, Macromedia Flash 5.0, and Adobe Photoshop 6.0.1.
 Click to enlarge. |
MobileMark measures the response time between the simulated user's submission of a task and the completion of that task by the application. It records only the response time for each operation, not the time to reposition the cursor to perform the operation. MobileMark includes application-load times as part of the measurement for its ratings. MobileMark also includes a conditioning run that prepares a notebook's battery for testing by draining the battery completely, then requiring that the battery be fully charged before an official test run can commence. The conditioning run also automatically loads Windows' ProcessIdleTasks routine on notebooks that are using Windows XP so that the system is fully optimized for all applications utilized by the benchmark.
The MobileMark 2002 performance rating is an average of the response times normalized against a fixed calibration platform: a 1GHz Pentium III with a 100MHz bus, 256MB of PC100 SDRAM, an ATI Rage P/M Mobility AGP2X 8MB graphics card, an ATA/66 hard drive, Windows XP Professional, Windows' Display Properties set to 1,024x768 and 32-bit color, and Windows' power scheme set to Always On. A rating of 100 means the test system's performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
The MobileMark 2002 battery rating, expressed in minutes, reflects how long a notebook with a fully charged battery is able to run the MobileMark test. The test concludes when the battery is drained.
A note about CPU throttling: As per BAPCo's recommendations, CNET Labs runs MobileMark 2002 using the Portable/Laptop power scheme. This power scheme allows a notebook to automatically throttle down its CPU speed in order to conserve battery life. The amount of CPU throttling is dependent on a how a manufacturer has configured that particular model and can therefore vary among models from different manufacturers. This can be a contributing factor as to why some notebooks with similar specifications generate different MobileMark 2002 application and battery-life performance scores.
| BAPCo SysMark 2002 |
|
CNET Labs runs SysMark 2002 on only the notebooks that are considered to be in the desktop-replacement category. This benchmark measures a system's overall performance using off-the-shelf applications. SysMark comprises two workload scenarios, reflecting usage models of mainstream business users ("office productivity") and Web designers ("Internet content creation"). The benchmark runs multiple applications simultaneously, continually switching between applications, with some tasks running in the background.
- The office-productivity scenario runs the following applications: Microsoft Word 2002, Microsoft Excel 2002, Microsoft PowerPoint 2002, Microsoft Access 2002, Microsoft Outlook 2002, Netscape Communicator 6, Dragon NaturallySpeaking Preferred 5.0, WinZip 8.0, and McAfee VirusScan 5.13. File compression, virus scan, and speech-to-text translation run in the background.
- The Internet-content-creation scenario runs the following applications: Adobe Photoshop 6.0.1, Adobe Premiere 6.0, Macromedia Dreamweaver 4.0, Macromedia Flash 5.0, and Microsoft Windows Media Encoder 7.1. Video encoding runs in the background.
 Click to enlarge. |
Each scenario rating is an average of the response times normalized against a fixed calibration platform: a 1GHz Pentium III, Intel's 815EEA motherboard chipset, 256MB of 133MHz SDRAM, a GeForce3 graphics card with 64MB of DDR, an ATA/100 hard drive using the FAT32 file system, Windows XP Professional, Windows' Display Properties set to 1,024x768 and 16-bit color at 75Hz, and Windows' power scheme set to Always On. A rating of 100 means the test system's performance equals that of the calibration platform. Scores above or below 100 indicate performance relative to the calibration platform.
The SysMark 2002 rating is the geometric mean of the Internet-content-creation and office-productivity scenario ratings.
| Futuremark's 3DMark2001 SE Pro (Build 330) |
|
CNET Labs runs 3DMark2001 SE Pro on only the notebooks that are considered to be in the desktop-replacement category. 3DMark2001 measures a notebook's 3D graphics performance using an actual game engine (Remedy Entertainment's MAX-FX Technology). 3DMark2001 uses the DirectX 8 (DX8) interface, which includes features such as pixel shaders, vertex shaders, pixel/point sprites, and volumetric 3D textures. Some of 3DMark2001's tests require DX8 hardware support; a system that does not have DX8 hardware support will typically generate a lower score than one that does. We run 3DMark with Windows' Display Properties set to a 32-bit color depth and with the following benchmark settings:
 Click to enlarge. |
- Resolution and color depth: 1,024x768; 32-bit color
- Antialiasing: none
- Frame buffer: double
- Texture format: compressed (if compressed is not available, we choose 16-bit)
- Z-buffer depth: 24-bit
- Rendering pipeline: D3D Hardware Transform and Lighting (if D3D Hardware T&L is not available, we choose D3D Software T&L)
