Version 1.30, Last modified 2009-11-20
These instructions assume that you are familiar with the following:
Web server Software configuration
Server and Network configuration
Installing and Using Open Source Software
Installing a JVM and running Java programs
The SPECweb2005 kit contains the following components:
The SPECweb2005 kit contains the following components:
Additional Software Requirements:
Hardware Requirements:
Note: For debugging purposes, you could run everything on 1 or 2 systems but these instructions assume you have at least 3.
Place a copy of the provided setup.jar or setup.exe on all systems that are part of your test setup. You will need to have a JVM installed; a JDK or JRE version 1.4.1 or later is recommended. Invoke the Java installer as follows:
Follow the menu prompts and select the type of installation appropriate for the role of the system being installed. Sample output from the Java installer is shown below:
SPECweb2005 - InstallShield Wizard
Choose the installation type that best suits your needs.
[X] 1 - Client / Prime Client Installation
The benchmark Java source code and class files for running the prime
client and client harness. Requires a Java Virtual Machine (JVM).
Benchmark documentation is included as well in this installation type.
[ ] 2 - Web server Installation
The Web server php, jsp and aspx script implementations that run on the System
Under Test (SUT), file set generator tool for script paddings, images,
and static content (requires a Java Virtual Machine), and benchmark
documentation are included in this installation type.
[ ] 3 - Backend Simulator (BeSim) Installation
Backend Simulator (BeSim); a Web server API that simulates an application
or database server. Must be installed on a separate machine than the SUT.
Benchmark documentation is included as well in this installation type.
[ ] 4 - Full Installation
The program will be installed with the suggested configuration.
Recommended for most users.
[ ] 5 - Custom
The program will be installed with the features you choose.
Recommended for advanced users.
If the install of the SPECweb2005 components fail with the following message:
Errors occurred during the installation.
- JVM not found
Please follow the directions below:
If installing with Java 1.5 ( JVM 5), set the JAVA_HOME variable to the root path of your JVM engine.
Ex: export JAVA_HOME=/usr/path_to_java
There are known capability issues with SPECweb2005 installer and Java 1.6 (JVM 6), please use Java 1.5 or earlier to complete the install.
General Directions:
test_besim_bank
, test_besim_ecom
,
and test_besim_support
scripts (included in the "besim"
subdirectory of the installation) to test whether you're getting valid BeSim responses. Both Perl scripts and Linux
bash scripts are provided. The Perl scripts require a Perl
interpreter as well as the following Perl modules: URI, HTML-Tagset, HTML-Parser, and libwww-perl.
The bash scripts require cURL to be in
the path.perl
test_besim_bank.pl
http://192.2.1.132:81/fcgi-bin/besim_fcgi.fcgi
bash test_besim_bank.sh http://bsim614:81/fcgi-bin/besim_fcgi.fcgi
*Note: For more FastCGI instructions (particularly for the Apache HTTP Server), see Appendix G.
If you notice that one BeSim system has a bottleneck (CPU, disk, network, etc.), there is now (as of 1.10 and above) an option to use multiple physical BeSim systems. Follow the same steps above to install BeSim on another system, and specify multiple BESIM_SERVERs (delimited by spaces) in Test.config. See the example Test.config files for more information.
Configure all the test bed systems so that they can communicate with one another; make sure to update your /etc/hosts file (or equivalent). The Prime must be able to communicate with each Client in the test environment. The Web server and the BeSim system must be able to communicate with each other. Finally, the Prime Client and BeSim system(s) must be able to communicate with each other.
Once the Web server, clients, and BeSim system have been configured and networked, the next step is to try running a test strobe for one of the workloads. To start with a short test, edit Test.config again and put these lines at the end of the file to override the defaults:
KILL_CLIENT
= 1
DEBUG_LEVEL = 2
RAMPUP_SECS = 60
WARMUP_SECONDS = 60
RUN_SECONDS = 300
SIMULTANEOUS_SESSIONS = 1
Also set TEST_TYPE to the desired workload (SPECweb_Banking, SPECweb_Ecommerce, or SPECweb_ Support ).
Next start the test using the commands below. Note, you may want to use 2 windows or redirect the output of the two Java commands to files or create a script run the harness and to save the output:
After
test has completed check the redirected output and the contents of the results.
Note: you may wish to set other Java parameters besides the heap parameters to
optimize the specwebclient processes as you increase
the test load. Check the documentation for your JVM.
Did that test work?
Yes? Great, so did these instructions “work” or were there missing details or do you have additional questions that you are now writing up to send to web2005support@spec.org?
No? Well, there should be enough information in the debug output to start debugging. Please send any corrections or requests for clarification to these instructions to the email address above. Continue reading and reviewing the Web server, BeSim, and Client Checks below, along with the related README files for the various components in the attached appendixes, particularly Appendix E for common errors and Appendix F for the Support FAQ.
Now, that the basics are working, you can try tests using more Sessions and adding more clients.
1. Have you configured your Web server to support the HTTP or HTTPS based on the workload you are running? Banking is all HTTPS (SSL), Ecommerce requires both HTTP and HTTPS, and Support uses only HTTP. Note: that you can temporarily override the use of HTTPS in Banking and Ecommerce by setting USE_SSL = 0 in the .config file. Results will be noncompliant but this may be useful when debugging.
2. Have you configured you Web server to support PHP, JSP or ASP.NET according to the directions provided with your Web server? If using JSP and a separate Java application server such as Tomcat also review its documentation and configure appropriately. Try running the PHP, JSP or ASP.NET "Hello World" equivalent to make sure your Web server's script engine is working.
3. Have you configured the wafgen .rc files for the workload and run Wafgen to create the fixed and scaling file sets? Did you set the number of SIMULTANEOUS_SESSIONS to a value equal or greater than the value you plan to use in your testing?
4. Can the Web server system ping the BeSim system? Connect to the BeSim Web server (telnet [BESIM_IP] [BESIM_port])?
TEST_TYPE=SPECweb_<test
type>
WEB_SERVER = <SUT>
BESIM_SERVER = <BeSim system>
BESIM_PORT = 81 [or the non-80 value set up on the BeSim Web server]
Starting with version 1.10, the prime client has the ability to poll clients for data at specified intervals during a run and display it as CSV or in a GUI. Data is collected and displayed only during the warm-up and benchmark run phases. The following configuration parameters in Test.config control the collection and display of this information:
Once the SPECweb2005 benchmark is running the workload(s) without errors, you will want to optimize your testbed for optimal performance. Here are some tips:
Here are the basics for setting up SPECweb2005 Clients:
First, you should at minimum have version 1.4.1 of the java jdk and/or jre
installed on the client machines running this code to take advantage of garbage
collection optimizations in the more recent jvms. JVM v1.4.n is strongly
recommended for best performance. However, older jvm versions back to 1.3.1
may work.
Next, there are five different .config files used for SPECweb2005. There is a
SPECweb_<workload>.config for each of the three workloads (Banking, Ecommerce,
and Support) that contain both configurable and fixed parameters specific to
the workload. Additionally, Test.config contains parameters that are common
to all workloads and includes configurable and fixed parameters. Lastly,
Testbed.config contains the testbed hardware/software configuration properties.
Note that Test.config and the workload-specific configuration files mentioned
above do not exist in a new installation. Instead, example workload and Test
configuration files are provided, and you will need to copy the sample file
most suited to your environment. Ex:
cp Test.Unix-PHP.config Test.config
cp SPECweb_Banking.Unix-PHP.config SPECweb_Banking.config
cp SPECweb_Ecommerce.Unix-PHP.config SPECweb_Ecommerce.config
cp SPECweb_Support.Unix-PHP.config SPECweb_Support.config
"specwebclient" is the SPECweb2005 client load generator. After editing the
.config files to match your test environment, you need to start this process on
all client machines. From the directory where specwebclient.jar is installed,
and at a command prompt type:
java -jar specwebclient.jar
Starting "specwebclient" with tuned heap sizes and garbage collection
parameters is also recommended. Specifically, for v1.4.1 of the jdk on a
multi-processor client, using parallel and CMS garbage collection is strongly
recommended. Example:
java -cp c:\sw200x -Xms512m -Xmx512m -XX:+UseParNewGC \
-XX:+UseConcMarkSweepGC -jar specwebclient.jar
The usage for specwebclient is:
java -jar specwebclient.jar -h
Usage: java -jar specwebclient.jar [-v] [-p port] [-n hostname]
[-s servername] [-le errorlogfilename] [-lo stdoutlogfilename]
Command line flags:
-v Print version string and exit
-p <port> Override default port number
(allows multiple instances on same machine)
-n <hostname> Override hostname (use Alias)
-s <servername> Overrides WEB_SERVER
-le <errorlogfilename> redirects error messages
-lo <stdoutlogfilename> redirects other logging messages
"specweb" is the SPECweb2005 test manager and is run on the Prime Client. It
is started after all the "specwebclient" processes have be started. Starting
specweb should begin the test. The "specweb" process exits at the end of the
test, and the "specwebclient" process can be configured to terminate as well
(see Test.config). To start the "specweb" process, from the directory where
specweb.jar is installed, at a command prompt type:
java -jar specweb.jar
The usage for specweb is:
java -jar specweb.jar -h
Usage: java -jar specweb.jar [-C] [-v] [-w workloadname] [-le errorlogfilename]
[-lo stdoutlogfilename]
Command line flags:
-h Print this message and exit
-v Print version string and exit
-C Overrides any non-compliant settings in the .config files with the
default compliant values
-w Overides the TEST_TYPE in the config file (SPECweb_Banking,
SPECweb_Ecommerce, SPECweb_Support)
-le <errorlogfilename> redirects error messages
-lo <stdoutlogfilename> redirects other logging messages
"reporter" is the SPECweb2005 report generator and is invoked by "specweb" at
the end of a test to produce .TXT and .HTML reports. It can also be invoked to
create a merged .raw file from results from running the 3 workloads on a given
testbed for a submission to SPEC; as well as generating combined .TXT and
.HTML report files. Here is the usage for the reporter:
java -jar reporter.jar -h
SPECweb2005 Reporter 1.10
Copyright (c) 2005 Standard Performance Evaluation Corporation
Command line flags:
-h Print this message and exit
-v Print version string and exit
-c Combine 3 separate workload files into one submittable .raw file
-R Regenerate HTML and ASCII reports from combined rawfile
Example Usage:
--------------
Regenerate reports from a single raw file:
java -jar reporter.jar results\SPECweb_Support.20050323-102204.raw
Combine three compliant workload raw files into one submittable raw file:
java -jar reporter.jar -c bank.raw ecom.raw support.raw submit.raw
Regenerate reports from a previously combined raw file:
java -jar reporter.jar -R submit.raw
SPECweb_Banking.config:
36c36
< PADDING_DIR = "/www/webroot/bank/dynamic_padding/"
---
> PADDING_DIR = "/www/bank/dynamic_padding/"
47,48c47,48
< CHECK_IMAGE_DIR = "c:/www/webroot/bank/images"
< CHECK_IMAGE_SUBDIRS = 10
---
> CHECK_IMAGE_DIR = "/www/bank/images"
> CHECK_IMAGE_SUBDIRS = 100
SPECweb_Ecommerce.config:
36c36
< PADDING_DIR = "/www/webroot/ecommerce/dynamic_padding/"
---
> PADDING_DIR = "/www/ecommerce/dynamic_padding/"
SPECweb_
Support
.config:
40c40
< PADDING_DIR = "c:/www/webroot/support/dynamic_padding/"
---
> PADDING_DIR = "/www/support/dynamic_padding/"
50c50
< DOWNLOAD_SUBDIRS = 0
---
> DOWNLOAD_SUBDIRS = 2
Test.config:
30c30
< CLIENTS = "localhost"
---
> CLIENTS = "client50:1095 client40:1094 client30:1093 client20:1092"
32c32
< SIMULTANEOUS_SESSIONS = 5
---
> SIMULTANEOUS_SESSIONS = 1100
35,36c35,36
< TEST_TYPE=SPECweb_Banking
< # TEST_TYPE=SPECweb_Ecommerce
---
> # TEST_TYPE=SPECweb_Banking
> TEST_TYPE=SPECweb_Ecommerce
41c41
< WEB_SERVER = localhost
---
> WEB_SERVER = sut2005
54c54
< BESIM_SERVER = "localhost"
---
> BESIM_SERVER = "BeSimbox"
56c56
< BESIM_PORT = 81
---
> BESIM_PORT = 82
63a64
> BESIM_INIT_SCRIPT = "/FastCGI/BESIM_fcgi.fcgi"
67,70c68,71
< SMARTY_DIR = "c:/www/php/Smarty-2.6.3/"
< SMARTY_BANK_DIR = "c:/www/php/Smarty-2.6.3/banking/"
< SMARTY_ECOMMERCE_DIR = "c:/www/php/Smarty-2.6.3/ecommerce/"
< SMARTY_SUPPORT_DIR = "c:/www/php/Smarty-2.6.3/support/"
---
> SMARTY_DIR = "/www/Smarty-2.6.9/"
> SMARTY_BANK_DIR = "./"
> SMARTY_ECOMMERCE_DIR = "./"
> SMARTY_SUPPORT_DIR = "./"
107c108
< KILL_CLIENT = 0
---
> KILL_CLIENT = 1
Invoking Wafgen:
================
Unix: Wafgen <workload_specific>.rc
Windows: Wafgen.bat <workload_specific>.rc
or alternately:
java -Xms384m -Xmx384m -jar Wafgen.jar <workload>.rc
or:
java -Xms384m -Xmx384m -classpath Wafgen.jar org.spec.wafgen.Wafgen <workload>.rc
Wafgen Overview
===============
Wafgen is used to create the file sets used by SPECweb2005. Each workload
has a "fixed" and a "scaling" file set. The fixed file set includes a set of
static elements that are commonly referenced by the pages that make up a given
workload. The scaling file set includes a set of directories each containing
additional static elements for the workload. The number of directories created
(or referenced by a test) is calculated by:
(SIMULTANEOUS_SESSIONS * DIRSCALING) = Total #directories
The total amount of storage required for a benchmark is also be dependent on
the total number of directories in the scaling file set and the size of the
contents of each directory (see Storage Requirement below). Since the total
number of directories needed could exceed system limits for the number of files
per directory (usually ~32,000), an additional level of subdirectories can be
added using the SUBDIR* parameters. By default, banking and ecommerce use
SUBDIR's. The scaling directories are round-robined across the specified
number of subdirectories.
This kit contains the resource files (.rc) for each workload that are used as
input to Wafgen. There is a set tailored for both Windows and Unix/Linux in
the windows and unix subdirectories. To use these files copy the appropriate
set into the upper directory and edit them as needed to match your test
environment. For conforming results - ONLY parameters listed under Section A
of the .rc file may be modified. Double check all section A parameters,
especially:
DOCROOT (if not set will default to current directory)
BASEDIRNAME (see examples in <workload_specific>.rc files)
BASESUBDIRNAME (if applicable)
SIMULTANEOUS_SESSIONS (may be larger than the corresponding value in
value in SPECweb_<workload>.config)
Also verify that the SPECweb_<workload>.config file parameters in the test
harness on the Prime Client are in agreement.
The .rc files for fixed file sets include:
bank_image_props.rc
ecommerce_image_props.rc
support_image_props.rc
The .rc files for scaling file sets include:
bank_usercheck_props.rc
ecommerce_productline_props.rc
support_downloads_props.rc
The .rc files are annotated to provide additional guidance.
Reminder: Altering .rc file parameters listed in section B will result
in non-conforming tests, so only modify parameters in section A.
Storage Requirements
====================
Since the benchmark requires that the System Under Test (SUT) contain the file
sets for all three workloads, it is important to understand the storage
requirements for each workload. Below are the data and formulas to help
estimate storage requirements for SPECweb2005.
Banking
=======
The fixed file set created by bank_image_props.rc includes:
o bank/images -
44 files; Average size: 3570 bytes; Total size: 157096 bytes
o bank/dynamic_padding -
15 files; Average size: 21800 bytes; Total size: 327000 bytes
The scaling file set created by bank_usercheck_props.rc creates a number
of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.
o Each bank/images/{subdir00mm}/user0000nnnnnn contains -
20 files; Average size: 10431 bytes; Total size: 208625 bytes
To estimate the storage requirement for the user check directories use
the following formula:
(SIMULTANEOUS_SESSIONS * CLIENT_SESSION_USER_IDS) * 208625
For Conforming results, CLIENT_SESSION_USER_IDS (alias for DIRSCALING)
must be 50.
Example:
((1000 SIMULTANEOUS_SESSIONS * 50 CLIENT_SESSION_USER_IDS) * 208625) =
10,431,250,000 bytes
Ecommerce
=========
The fixed file set created by ecommerce_image_props.rc includes:
o ecommerce/images -
172 files; Average size: 2032 bytes; Total size: 349557 bytes
o ecommerce/dynamic_padding -
11 files; Average size: 73000 bytes; Total size: 803000 bytes
The scaling file set created by ecommerce_productline_props.rc creates a number
of productline0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.
o Each ecommerce/images/{subdir00mm}/productline0000nnnnnn contains -
30 files; Average size: 18816 bytes; Total size: 564495 bytes
To estimate the storage requirement for the product line directories use
the following formula:
(SIMULTANEOUS_SESSIONS * DIRSCALING) * 564495
For Conforming results, DIRSCALING must be 5.
Example:
((1000 SIMULTANEOUS_SESSIONS * 5 DIRSCALING) * 564495) =
2,822,475,000 bytes
Support
=======
The fixed file set created by support_image_props.rc includes:
o support/images -
31 files; Average size: 728 bytes; Total size: 22580 bytes
Warning: this file set contains several very small files (30 bytes)
so the path name must not be longer than 14 characters. So,
"support/images" is OK but "supporting/image" is NOT.
o support/dynamic_padding -
6 files; Average size: 59616 bytes; Total size: 357700 bytes
The scaling file set created by support_downloads_props.rc creates a number
of user0000nnnnnn directories based on the SIMULTANEOUS_SESSIONS value.
o Each support/downloads/{subdir00mm}/dir0000nnnnnn contains -
16 files; Average size: 4279501 bytes; Total size: 68472021 bytes
To estimate the storage requirement for the download directories use
the following formula:
(SIMULTANEOUS_SESSIONS * DIRSCALING) * 68472021
For Conforming results, DIRSCALING must be 0.25.
Example:
((1000 SIMULTANEOUS_SESSIONS * .25 DIRSCALING) * 68472021) =
17,118,005,250 bytes
Note: The heap parameters used for Wafgen are required to support the
large files in the scaling file set for Support workload.
Tips for Building Large Filesets
================================
o To increase the size of an existing file set to support a larger number of
Simultaneous Sessions than used initially, the steps are:
Edit the <workload>.rc file for the scaling fileset create previously
and change these parameters:
SIMULTANEOUS_SESSIONS=<new higher value>
FIRSTDIRECTORY=<1+highest directory number previously create>
For example:
Originally a support_downloads_props.rc file used:
SIMULTANEOUS_SESSIONS=1000
FIRSTDIRECTORY=1
When a file set to support 2000 Simultaneous Sessions is needed,
change:
SIMULTANEOUS_SESSIONS=2000
FIRSTDIRECTORY=501
Then run: Wafgen <updated_workload>.rc
o To run several Wafgen's in parallel to create a large file_set, copies of i
the <workload>.rc file can be for each parallel wafgen to be run and make the
changes to the SIMULTANEOUS_SESSIONS and FIRSTDIRECTORY directories. To
calculate the correct values for FIRSTDIRECTORY in each copy use the formula:
Number of Directories = SIMULTANEOUS_SESSIONS * DIRSCALING
(or for banking the DIRSCALING alias CLIENT_SESSION_USER_IDS is used).
Use the directory scaling formula for the workload (shown above) to
calculate the correct values for FIRSTDIRECTORY in each copy.
For example, to create a support downloads file set for a total of
4000 Simultaneous Sessions:
support_downloads_props.rc_1:
SIMULTANEOUS_SESSIONS=1000
FIRSTDIRECTORY=1
support_downloads_props.rc_2:
SIMULTANEOUS_SESSIONS=2000
FIRSTDIRECTORY=501
support_downloads_props.rc_3:
SIMULTANEOUS_SESSIONS=3000
FIRSTDIRECTORY=1001
support_downloads_props.rc_4:
SIMULTANEOUS_SESSIONS=4000
FIRSTDIRECTORY=1501
Then run 4 parallel wafgens:
Wafgen support_downloads_props.rc_1 &
Wafgen support_downloads_props.rc_2 &
Wafgen support_downloads_props.rc_3 &
Wafgen support_downloads_props.rc_4 &
(Note: depending on the number of CPUs and the I/O subsystem being
used this sequence may not take significantly less time than just
running a single Wafgen for the whole file set.)
o To convert from a file set that was recreated without using *SUBDIR*
parameters or to add additional subdirectories, the original file set will
need to be re-created from scratch. The <workload>.rc file will need to
be edited to configure these parameters. The parmeters are preceeded by
the comment string "#subdirs#" if use of SUBDIRS is not the default
(i.e Support).
o Multiple disks or mount points may be used to hold subsets of a workload's
file sets, for example: /<docroot>/bank may have the images directory as
as separate mount point or /<docroot>/bank/images may have separate mount
points for each of 10 subdirectories (subdir0001 thru subdir0010).
Warning:
The path names have been standarize in the wafgen <workload>.rc files
and the SPECweb_<workload>.config files in the harness, so that
in the DOCROOT there would be these direcories:
bank/images
bank/dynamic_padding
ecommerce/images
ecommerce/dynamic_padding
support/images
support/dynamic_padding
support/downloads
While it is possible to change these names in the .rc and .config i
files with due care (and making sure not to exceed path length limits
for some very small files); it is recommended that users retain this
naming to avoid errors due to typos and to simplify debugging.
How to Build BESIM:
UNIX/LINUX:
Type: uname -s to get name of the operating system (i.e. HP-UX)
In the Makefiles directory use one of the MakeIncl.<type>.<OS>
files as a template for your MakeIncl.<type>.<your_OS> and
update to match you environment.
To Build Fast-cgi version of BESIM: make fcgi
To Build NSAPI version of BESIM: make nsapi
To Build Zeus ISAPI version of BESIM: make zisapi
Optionally you can specify other make targets by adding TARGET='list of targets'
For example:
make zisapi TARGET='clean'
make nsapi TARGET='clean all install'
The install target will use the path specified by DEST to install the besim
exeutatble or library, so it should be set to the appropriate place on your
Besim webserver directory tree (i.e. /www/fast-cgi).
If you plan to use Fast-cgi you must build and install the Fast-CGI development
kit which provides the fast-cgi library and include files before build BESIM.
If using Apache for the BESIM webserver, that will need to include mod_fastcgi.
See: http://www.fastcgi.com/
Note: Please send any new or updated MakeIncl.* files you want included in
a future release to web2005(at)spec.org.
Windows:
Project files are included as an example: BeSim.def, BeSim.dsp, BeSim.dsw
NOTE: For the latest version of this document, please visit http://www.spec.org/web2005/docs/support-faq.html.
Q: It's not working, what should I do?
A:
Reread this User's Guide and make sure that you've followed all the steps and
have run through the check lists in section 3. If you still haven't resolved
your problem, review this FAQ to see if there is anything similar to the
condition you are seeing.
If you have reviewed the User's Guide, various README files, and have tried
higher DEBUG_LEVEL values and are still unable to resolve your problem, then
write up all the details of the problem along with any logs your .config files
and send them to web2005support@spec.org.
Q: How long will it take to complete a full compliant 3 iteration run for one of the 3 workloads?
A: The Minimum required time is: ~144 minutes (2.4hrs), with each iteration at 48 minutes. If WARMUP_SECONDS or THREAD_RAMUP_SECONDS have been increased, the test will run somewhat longer.
Q: My clients are printing "Java.lang.OutOfMemoryError: Java heap space" errors. What can I do to eliminate these?
A:
Invoke the client JVMs with more heap space; for
example, assuming the clients have sufficient RAM, the initial and maximum heap
sizes can be forced to 512MB (these values are only suggestions, and will vary
depending upon system):
java -Xms512m -Xmx512m -XX:+UseParNewGC -XX:+UseConcMarkSweepGC specwebclient
More information about heap size and garbage collection can be found here:
http://java.sun.com/docs/hotspot/gc1.4.2/faq.html
http://java.sun.com/j2se/1.5.0/docs/guide/vm/gc-ergonomics.html
Q: When trying to manually access PHP pages (i.e. from a browser), the PHP error log or the response displays "failed to open stream: No such file or directory in [PATH]\inc.common.php"
A:
You must first initialize the benchmark by running 'java specweb
'
with all the variables in the .config files set properly. The init.php script for each workload will write these
variables to init_vars.php, so be sure user the Web
server runs as has write permissions for the SPECweb2005 directories.
Q: I'm running the PHP scripts on Apache (either as the primary Web server or as the BeSim server) and I'm seeing errors.
A:
If you have BESIM_PERSISTENT=1, try changing it to 0. During testing, Apache
was observed to have issues with attempting to use persistent sockets to BeSim. It is suspected that this is due to the Apache
architecture which forks multiple child processes to respond to requests; this
makes it difficult to reuse a connection across script executions.
More information about the issue can be found here (search on Apache):
http://www.php.net/pfsockopen
Q: When starting the benchmark, the prime client attempts to run the INIT_SCRIPT on the Web server but fails.
A: Ensure that the user the Web server process runs as has sufficient permissions to write to the appropriate SPECweb2005 directories. For PHP (all workloads), init.php attempts to write init_vars.php, which contains server-specific variables required to run the benchmark.
Q: Why do I get "access to globals failed" in BeSim responses?
A: For Unix, BeSim attempts to write a globals file to the /tmp directory; for Win32, the root of the C: drive. There are two possible solutions:
· Change the permissions on the appropriate directory so the user the Web server runs as can read and write this globals file.
·
Modify BESIM_banking.h
, BESIM_ecommerce.h
,
and BESIM_supportsite.h
to specify different BESIM_BANKING_GLOBALS_PATH, BESIM_ECOMMERCE_GLOBALS_PATH,
and BESIM_SUPPORT_GLOBALS_PATH variables respectively. You must then recompile
the API.
Q: The PHP scripts don't appear to be running properly, and I don't get any output from a browser when requesting the pages manually. How do I troubleshoot this?
A: PHP ships with two ini files, php.ini-dist and php.ini-recommended. While php.ini-recommended is optimized for performance and security, for initial troubleshooting, php.ini-dist may be a better choice. This will configure PHP to display errors in the browser. You may also want to set "error_reporting = E_ALL", which will display all errors and warnings.
Q: How can I test whether I've compiled and installed BeSim correctly?
A:
The test_besim_bank
, test_besim_ecom
,
and test_besim_support
scripts (included in the BeSim directory) good ways
to test whether you're getting valid BeSim responses.
Invoke these scripts with the URL to your compiled BeSim API, i.e.
perl
test_besim_bank.pl
http://192.2.1.132:81/fcgi-bin/besim_fcgi.fcgi
bash test_besim_bank.sh http://bsim614:81/fcgi-bin/besim_fcgi.fcgi
.
Q: I'm getting invalid account values in client logs and it used to work fine.
A: BeSim may need to be to restarted cleanly along with the rest of the testbed software. Try:
· stopping the webserver on the SUT
· stopping the BeSim webserver
· remove /tmp/besim_*.globals
· remove old besim access logs
· check that none of your clients are running "java specwebclient" processes (kill these processes if present)
· start the webserver on the SUT and on BeSim box
· re-run your test
Q: My clients are dying with "Exceeded max allowed overthink time" errors, how do I prevent these?
A: Check the following items:
· Ensure your clients are not overloaded.
· Ensure THINK_TIME to a reasonable level.
· Increase THREAD_RAMPUP_SECONDS in Test.config.
Q:
My client logs show sequences of errors that begin with:
2005-05-14 07:27:44:909 SPECweb_
Support
:
[ERROR] SocketTimeoutException encountered during run!
followed by a couple of other header and response
errors.
Occasionally my log also has error sequences that begin with:
2005-05-14 07:37:02:613 Connection:
[ERROR] Bad status: 503
A: Your server is overloaded or under-tuned for the number of Simultaneous Sessions the test is requesting. The SocketTimeoutException error occurs when the server was unable to send a response back before the PROTOCOL_TIMEOUT_SECONDS (60 sec) has expired.The 503 status is returned by some webservers to indicate that the service is temporarily unavailable. See if reducing the load on the server eliminates these messages, if so you may wish to investigate if there is addition tuning that could be done to the SUT.
Q: My clients reported errors during the test but they don't show up in my run results. Why?
A: If the errors occur before the line that reads "WorkloadScheduler: Clearing statistics...", then they occurred during warmup and thus are not counted as errors.
Q: When navigating the ecommerce pages in a browser, why does "Session does not exist" appear when I click the "Add to Cart" button?
A: Make sure that you are accessing the Web server with the same host name as you specified for the WEB_SERVER variable in Test.config. For example, if WEB_SERVER=192.168.0.1 in your Test.config, you should use a URI such as http://192.168.0.1/ecommerce/index.php (and not a hostname that resolves to that IP address). This is due to the way the Location redirects are done in the scripts.
Q: Why do I get a segmentation fault when using the Sun 1.4.x JVM on the x86_64 architecture?
A: This problem is not specific to SPECweb2005. It has to do with the processor having NX (No-Execute, also known as Execute Disabled) on. There are two known workarounds:
· Upgrade to a 1.5.x JVM
· Disable NX in your operating system. Under Linux, this can be done by passing the "noexec=off" command-line option to the kernel. If the kernel is 64-bit, "noexec32=off" should also be passed.
Q: How do I get the PHP scripts running on Windows Server 2003 / IIS 6.0?
A:
There is a helpful tutorial available here: http://www.peterguy.com/php/install_IIS6.html
You must also configure IIS to serve all files
regardless of file name extension by adding a wildcard character (*) MIME type.
NOTE: This should not be performed on production Web servers. More information
can be found here:
http://www.microsoft.com/resources/documentation/iis/6/all/proddocs/en-us/wsa_mimemapcfg.mspx
Q: I changed BASEDIRNAME in support_image_props_rc file from the default "support" to "supp1data". The wafgen was failing to create the image files under the image directory with an error message as: Error: path length too long for requested file size - what's wrong?
A: The message: Error: path length too long for requested file size has to do with very small files (like ccc which is 30 bytes). To be a specweb file it has to have the byte count, relative file name and at least 1 char. of data. So to handle support's smallest image file, the path can be <7chars>/<6chars>/<name>. While you can change the path to something else (sup/sup_images or sup1data/img01 or other as long as it fits into this footprint and the changes are reflected in the SPECweb_<workload>.config. The Wafgen README does include the following warning under the section on Support : Warning: this file set contains several very small files (30 bytes) so the path name must not be longer than 14 characters. So, "support/images" is OK but "supporting/image" is NOT.
Q: I am trying to initialize multiple physical web server systems, or multiple web server instances on one physical system, how do I do this?
A: List all web server instances (hostnames or IP addresses) in SERVER_LIST, and all ports in SERVER_PORT_LIST. For example, if you have two web server instances (192.2.1.1:80 and 192.2.2.1:81) on one machine, your entries should read:
SERVER_LIST = "192.2.1.1 192.2.2.1"
SERVER_PORT_LIST = "80 81"
Q: I am trying to run the ecommerce workload and get the following error message, how can I fix it?
2006-03-26 15:16:54:183 SPECweb_Ecommerce[1]: [ERROR] state = 8 response.indexOf fail to find unit price: NN.NN, scaled_load =1
A: One possible reason for this failure is a localization issue. Try changing the localization settings on the clients to US-format.
There are two pieces necessary to get BeSim for FastCGI working properly under Apache:
cd <path_to_SPECweb2005>/besim/fcgi-2.4.0/
./configure --libdir=/libmake
make install
NOTE: --libdir=/lib was added above due to the default FastCGI Makefile installing libraries to /usr/local/lib, which is not a default library path on Linux and could cause this error upon execution of the FastCGI:
besim_fcgi.fcgi: error while loading shared libraries: libfcgi.so.0: cannot open shared object file: No such file or directory
NOTE
#2: For x86_64 versions of Fedora Core 3 and 4, it was observed that --libdir=/lib64 should be specified instead of /lib.
SuSE: /srv/www/fcgi-bin
Red Hat: /var/www/fcgi-bin
Apache 2.x compiled from source: /usr/local/apache2/fcgi-bin
cd <SPECweb2005>/besim/
make fcgi TARGET='clean all install' DEST=/var/www/fcgi-bin/
This should create besim_fcgi.fcgi and install it into DEST. If not, check the make output for errors.
These
instructions show how to compile the FastCGI web server module for Apache (mod_fastcgi). For other web servers, please refer to their
documentation to see whether FastCGI support is available.
NOTE for SuSE: SUSE Linux (8.1 and above)
ships with a precompiled mod_fastcgi for Apache
2.x. To check if it is installed, do a rpm -q apache2-mod_fastcgi;
if it's not, install this package from the CDs (see your OS documentation for
more instructions). If this package is installed, you can skip the rest
of this section.
NOTE for Red Hat: If using the httpd package that ships with the distribution, the httpd-devel RPM must also be installed, as it contains the necessary utilities and headers to compile Apache modules. To check, do a rpm -q httpd-devel.
2. Red Hat:
# ls -l /usr/lib/httpd/mod_fastcgi.so
-rwxr-xr-x 1 root root 161408 Sep 16 13:47 /usr/lib/httpd/modules/mod_fastcgi.so
Compiled from source:
# ls -l /usr/local/apache2/modules/mod_fastcgi.so
-rwxr-xr-x 1 root root 156434 Sep 21 10:03 /usr/local/apache2/modules/mod_fastcgi.so
# tail -f <APACHE_ROOT>/logs/error_log
You should initially see some FastCGI initialization messages such as
[Wed Sep 21 10:36:39 2005] [notice] FastCGI: process manager initialized (pid 25700)
See BeSim sections above for additional information.
The SPECweb2005 benchmark utilizes the JFreeChart library, which is licensed under the Lesser General Public License (LGPL). Source code and a copy of the license are included with the benchmark kit.
Copyright © 2005-2007 Standard Performance Evaluation Corporation. All rights reserved.
Java® is a registered trademark of Sun Microsystems.