OSGeo - User contributions [en]

Benchmarking 2011

2011-09-02T08:18:07Z

Wiki-Mpd: /* Server Ports */

== Basic Premise ==

Following up on [[Benchmarking 2010|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2011]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net (you can use [http://webchat.freenode.net/ webchat] to connect in browser)

=== Next IRC Meeting ===

* Wed August 3rd, 2011 @ [http://timeanddate.com/worldclock/fixedtime.html?year=2011&month=08&day=03&hour=14&min=0&sec=0 14:00:00 UTC]
** Provisional Agenda:
*** Status of OSM vector styling from all teams
*** Discuss rasters to be used
*** Discuss testing methodology

=== Previous IRC Meetings ===
[http://reversephonelookuppages.com/ reverse phone lookup]

* [[Benchmarking_2011/MeetingLog | Meeting Log]]

== Participants ==

''Server Teams:''

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''Confirmed'''
|-
| Cadcorp GeognoSIS
| [[User:Mpd|Martin Daly]]
| Yes
|-
| Constellation-SDI
| [[User:Desruisseaux|Martin Desruisseaux]], [[User:Eclesia|Johann Sorel]]
| Yes
|-
| GeoServer
| Andrea Aime, [[User:groldan|Gabriel Roldan]]
| Yes
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| Yes
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]], [[User:msmitherdc|Michael Smith]]
| Yes
|-
| QGIS Server
| [[User:Pirmin_Kalberer|Pirmin Kalberer]], Marco Hugentobler
| Yes
|}

''Data Teams:''

{| border="1"
|-
| '''Data Package'''
| '''Team Leader'''
| '''Comment'''
|-
| [http://imposm.org/docs/imposm/latest/ Imposm]
| Oliver Tonnhofer
| tool to be used to import OSM data
|-
| SPOT imagery
| [[User:Jeffaudi|Jean-Francois (Jeff) Faudi]]
|
|}

''Not Participating:''

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''Comment'''
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie]
| Response from Luc Donea: Unable to participate
|-
| ESRI ArcServer
| Satish Sankaran
| discussing internally
|-
| MapGuide Open Source
| TBD (contacted mapguide-internals)
| Response from Jason Birch: Unable to participate
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2011
| begin contacting all mapping servers
|-
| March 1st, 2011
| commitment due by all interested mapping servers
|-
| March 2nd, 2011
| exercise begins (and weekly meetings)
|-
| June 1st, 2011
| final testing begins (no more changes to data/styles/hardware, but changes to software is allowed)
|-
| September 1st, 2011
| no further testing
|-
| September 2nd, 2011
| final results due from all teams
|-
| September 12-16, 2011
| present results at FOSS4G2011
|}

== Rules of Engagement (DRAFT) ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# One test will be run: a 'baseline' test with the data in any format desired, but teams cannot generalize or change the data's resolution from its raw values.
# Teams must document all steps they did to manipulate the data/server (such as spatial indexes created etc.). If a team does not document the steps on this wiki then that team's test results will not be used.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Setup ==

It is the responsibility of each team to document their setup with regard to configuration details, setup notes, and differences between other servers in how data might be accessed/indexed.

Notes for each server:

* [[Benchmarking_2011/Mapnik_notes | Mapnik]]
* [[Benchmarking_2011/Cadcorp_notes | Cadcorp]]
* [[Benchmarking_2011/MapServer_notes | MapServer]]

Stylesheets and scripts in svn using a directory named wms/{year}/{servername}:

* [http://svn.osgeo.org/osgeo/foss4g/benchmarking/wms/2011/mapnik Mapnik]
* [http://svn.osgeo.org/osgeo/foss4g/benchmarking/wms/2011/cadcorp Cadcorp]
* [http://svn.osgeo.org/osgeo/foss4g/benchmarking/wms/2011/mapserver MapServer]

== Testing Tool ==

JMeter is used since it can read a list of bbox's via CSV, and the machine that applies load has been updated with Jmeter 2.5 (the latest stable release at the time of the benchmark): http://www.reverse.net/pub/apache//jakarta/jmeter/binaries/jakarta-jmeter-2.5.zip

Jmeter can be run from the command line like:

/home/jmeterusr/jakarta-jmeter-2.5/bin/jmeter

== Server Ports ==

Each server has access to a range of ports for testing:

Mapserver - 8080 - 8089

Mapnik - 8090 - 8099

Cadcorp - 4326 - 4335

== Datasets ==

=== OSM Vectors ===
* [[Benchmarking_2011/DB_Schema]]
* [[Benchmarking_2011/Imposm]]

CSV Files created with

<code> ./run_wms_request.sh -count 2200 -region -109 37 -102 41 -minsize 64 64 -maxsize 1024 768 -minres "2.5e-06" -maxres 0.000755 -srs 4326 -srs2 3857 -filter_within colorado.shp </code>

Sample styling:

* [http://12.189.158.78:8081/cgi-bin/mapserv6?MAP=/benchmarking/wms/2011/mapserver/osm-google.map&layers=default&mode=map&map_imagetype=png&mapext=-11690811.758434+4826833.1534376+-11687378.549609+4830266.3622626&imgext=-11690811.758434+4826833.1534376+-11687378.549609+4830266.3622626&map_size=800+800&imgx=400&imgy=400&imgxy=800+800 mapserv6]
** [http://12.189.158.78:8081/cgi-bin/mapserv6?MAP=/benchmarking/wms/2011/mapserver/osm-google.map&mode=browse&template=openlayers&layers=default MapServer Linux (with OpenLayers)]
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/wms/2011/mapserver/osm-google.map&mode=browse&template=openlayers&layers=default MapServer Win64 (with OpenLayers)]

* [http://12.189.158.78:8091/?SRS=EPSG:3857&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=-11690811.758434,4826833.1534376,-11687378.549609,4830266.3622626&WIDTH=800&HEIGHT=800 mapnik]

=== DEM Hillshading ===

Sample styling:

* [http://12.189.158.78:8081/cgi-bin/mapserv6?MAP=/benchmarking/wms/2011/mapserver/hill-google.map&layers=hillshade&mode=map&map_imagetype=png&mapext=-11947611.487019+4524873.854257+-11508160.757419+4964324.583857&imgext=-11947611.487019+4524873.854257+-11508160.757419+4964324.583857&map_size=800+800&imgx=400&imgy=400&imgxy=800+800 MapServer 6 Hillshade]

=== DEM Hillshading + OSM Vectors ===

* MapServer 6 Hillshade + OSM Vectors - [http://12.189.158.78:8081/cgi-bin/mapserv6?MAP=/benchmarking/wms/2011/mapserver/hill-google.map&layers=hillshade%20default&mode=map&map_imagetype=png&mapext=-11741344.300813+4772064.889657+-11631481.618413+4881927.572057&imgext=-11741344.300813+4772064.889657+-11631481.618413+4881927.572057&map_size=800+800&imgx=400&imgy=400&imgxy=800+800 Linux] [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/wms/2011/mapserver/hill-google.map&layers=hillshade%20default&mode=map&map_imagetype=png&mapext=-11741344.300813+4772064.889657+-11631481.618413+4881927.572057&imgext=-11741344.300813+4772064.889657+-11631481.618413+4881927.572057&map_size=800+800&imgx=400&imgy=400&imgxy=800+800 Win64]

OSM Vector Style changes:
LandN layers turned off (they provide background color)
LanduseN layers set to 70% opacity

== SVN ==

* The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/wms/2011/). If you need commit access contact jmckenna on IRC.

* Data are only stored on the server at '/benchmarking/wms/2011/data/'

* ''existing committers can add new OSGeoID users through this page: https://www.osgeo.org/cgi-bin/auth/ldap_group.py?group=osgeo''

== Hardware ==

Contact [User:msmitherdc|Michael Smith] with any questions about this hardware or for login credentials.

windows_wms_bm (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

linux_wms_bm (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

== External Related Links ==

* [http://www.slideshare.net/gatewaygeomatics.com/wms-performance-shootout-2010 FOSS4G2010 Benchmarking Presentation]
* [http://www.slideshare.net/gatewaygeomatics.com/wms-performance-shootout FOSS4G2009 Benchmarking Presentation]
* [[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
* [http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2011]] [[Category:FOSS4G]]

Benchmarking 2011/Cadcorp notes

2011-08-10T09:16:28Z

Wiki-Mpd: Created page with "== Data == The "UNION ALL" views used by, for example, the "osm_roads" table are not usable by GeognoSIS because: * No spatial index can be detected (this restriction could be ..."

== Data ==

The "UNION ALL" views used by, for example, the "osm_roads" table are not usable by GeognoSIS because:

* No spatial index can be detected (this restriction could be relaxed, because PostGIS should work OK, albeit potentially slowly)
* No primary key/oid/unique id column can be detected

The views were converted to tables, and indexed, as follows:

<pre>
create table osm_railways_gen1_cadcorp as (select * from osm_railways_gen1);
alter table osm_railways_gen1_cadcorp add column cadcorp_id serial;
create index osm_railways_gen1_cadcorp_geometry_sidx on osm_railways_gen1_cadcorp using gist(geometry);
insert into geometry_columns(f_table_catalog, f_table_schema, f_table_name, f_geometry_column, coord_dimension, srid, "type") SELECT '', 'public', 'osm_railways_gen1_cadcorp', 'geometry', ST_CoordDim(geometry), ST_SRID(geometry), GeometryType(geometry) FROM public.osm_railways_gen1_cadcorp LIMIT 1;
create unique index osm_railways_gen1_cadcorp_unqidx on osm_railways_gen1_cadcorp(cadcorp_id);
</pre>

This was done for the following [[Benchmarking_2011/Imposm|Imposm]] tables:

* osm_railways_gen1
* osm_roads_gen0
* osm_roads_gen1
* osm_roads

Giving the following additional tables:

* osm_railways_gen1_cadcorp
* osm_roads_gen0_cadcorp
* osm_roads_gen1_cadcorp
* osm_roads_cadcorp

=== Styling ===

In order to mimic the layered styling the "osm_roads_gen1_cadcorp" and "osm_roads_cadcorp" tables have additional columns added, as follows:

<pre>
alter table osm_roads_gen1_cadcorp add column cadcorp_simplefc int4;
alter table osm_roads_cadcorp add column cadcorp_simplefc int4;
alter table osm_roads_cadcorp add column cadcorp_complexfc int4;
</pre>

== mapserver2geognosis ==

The MapServer MAP file was copied, edited to fix the "PROJ_LIB" setting to match the machine configuration (a freshly downloaded MS4W), then parsed via MapScript, in Python 2.6.

The output of the Python program is:

* A JSON file (approximately) describing the MapServer MAP file LAYER/STYLE/CLASS hierarchy
* A SQL file for adding styling codes to the "osm_roads_cadcorp" and "osm_roads_gen1_cadcorp" tables (see below)
* A JSON file containing styling lookup tables to match the styling codes

The first JSON file is then re-parsed and the contents used to create the GeognoSIS equivalent of a MapServer MAP file.

=== Style lookup codes ===

The following SQL is used to set styling codes on the "osm_roads_cadcorp" and "osm_roads_gen1_cadcorp" tables:

<pre>
update osm_roads_cadcorp set cadcorp_simplefc=5 where type='trunk_link';
update osm_roads_cadcorp set cadcorp_simplefc=4 where type='motorway_link';
update osm_roads_cadcorp set cadcorp_simplefc=9 where type='track';
update osm_roads_gen1_cadcorp set cadcorp_simplefc=2 where type='trunk';
update osm_roads_cadcorp set cadcorp_simplefc=10 where type='pedestrian';
update osm_roads_cadcorp set cadcorp_simplefc=7 where type='tertiary';
update osm_roads_gen1_cadcorp set cadcorp_simplefc=3 where type='primary';
update osm_roads_gen1_cadcorp set cadcorp_simplefc=1 where type='motorway';
update osm_roads_cadcorp set cadcorp_simplefc=8 where type in ('unclassified','residential','service','road','living_street');
update osm_roads_cadcorp set cadcorp_simplefc=6 where type='secondary';
update osm_roads_cadcorp set cadcorp_complexfc=23 where type='secondary';
update osm_roads_cadcorp set cadcorp_complexfc=21 where type='trunk_link';
update osm_roads_cadcorp set cadcorp_complexfc=24 where type='tertiary';
update osm_roads_cadcorp set cadcorp_complexfc=20 where type='trunk';
update osm_roads_cadcorp set cadcorp_complexfc=22 where type='primary';
update osm_roads_cadcorp set cadcorp_complexfc=28 where type='footway';
update osm_roads_cadcorp set cadcorp_complexfc=25 where type in ('unclassified','residential','service','road','living_street');
update osm_roads_cadcorp set cadcorp_complexfc=19 where type='motorway_link';
update osm_roads_cadcorp set cadcorp_complexfc=26 where type='track';
update osm_roads_cadcorp set cadcorp_complexfc=27 where type='pedestrian';
update osm_roads_cadcorp set cadcorp_complexfc=18 where type='motorway';
update osm_roads_cadcorp set cadcorp_complexfc=8 where bridge=1 and type in ('classified','residential','service','road','living_street');
update osm_roads_cadcorp set cadcorp_complexfc=9 where bridge=1 and type='pedestrian';
update osm_roads_cadcorp set cadcorp_complexfc=6 where bridge=1 and type in ('secondary','secondary_link');
update osm_roads_cadcorp set cadcorp_complexfc=3 where bridge=1 and type='trunk';
update osm_roads_cadcorp set cadcorp_complexfc=5 where bridge=1 and type in ('primary','primary_link');
update osm_roads_cadcorp set cadcorp_complexfc=1 where bridge=1 and type='motorway';
update osm_roads_cadcorp set cadcorp_complexfc=7 where bridge=1 and type in ('tertiary','tertiary_link');
update osm_roads_cadcorp set cadcorp_complexfc=4 where bridge=1 and type='trunk_link';
update osm_roads_cadcorp set cadcorp_complexfc=2 where bridge=1 and type='motorway_link';
update osm_roads_cadcorp set cadcorp_complexfc=10 where type='motorway' and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=16 where type in ('tertiary','tertiary_link') and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=12 where type='trunk' and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=11 where type='motorway_link' and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=14 where type in ('primary','primary_link') and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=17 where type in ('unclassified','residential','service','road','living_street') and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=15 where type in ('secondary','secondary_link') and tunnel=1;
update osm_roads_cadcorp set cadcorp_complexfc=13 where type='trunk_link' and tunnel=1;
</pre>

Benchmarking 2011

2010-12-09T17:05:18Z

Wiki-Mpd: /* Potential Participants */

Benchmarking 2010

2010-09-07T07:09:39Z

Wiki-Mpd: Added link to Cadcorp notes page

== Basic Premise ==

Following up on [[Benchmarking 2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''IRC nick'''
| '''Server status'''
|-
| Cadcorp GeognoSIS
| [mailto:martin.daly--@--cadcorp.com Martin Daly]
| mpdaly
| Production release of server installed with preliminary configuration for SHP, PostGIS, TIF and ECW
|-
| Constellation-SDI
| [[User:Acuster|Adrian Custer]] [[User:Cedricbr|Cédric Briançon]]
| acuster cedricbr
| Running on port 8280. Installation done for Shapefiles and Rasters.
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting]
| dmonie
| Install and configuration on-going on windows server, ecw WMS configuration done, shape and DB WMS configuration on going. Port 80, port 8080
|-
| GeoServer
| Andrea Aime
| aaime
| Normally running on port 8080 on the Linux server. Shapefiles and TIFF mosaic have been configured.
|-
| <s>MapGuide Open Source</s>
| <s>Zac Spitzer</s>
| <s>zacspitzer</s>
| ''inactive''
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| springmeyer
| Paleoserver is running on port 8090(shape) / 8091(postgis) / 8092(raster - no reprojection). (mod_mapnik_wms is an alternate being tested, will run on same ports if active).
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| jmckenna
| linux_wms_bm serving WMS requests for shps, on port 8081. windows wms_bm serving requests for shps, on port 8081.
|-
| Oracle MapViewer
| [[User:msmitherdc|Michael Smith]]
| msmith_
| Set up on port 8088
|-
| QGIS mapserver
| Marco Hugentobler
| marco___
| Set up on port 8081 (Linux server). Shapefiles and TIF (without reprojection) configured
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2010
| begin contacting all mapping servers
|-
| June 1st, 2010
| commitment due by all interested mapping servers
|-
| June 2nd, 2010
| exercise begins (and weekly meetings)
|-
| August 1st, 2010
| final testing begins
|-
| August 23rd, 2010
| no further testing
|-
| September 6-9, 2010
| present results at FOSS4G2010
|}

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
# Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
# Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Details and Differences ==

It is the responsibility of each team to document their setup with regard to data.

Please keep your notes in your servers directory in svn: http://svn.osgeo.org/osgeo/foss4g/benchmarking/

One particular thing that will differ is each team's use of spatial indexes on shapefiles.

Teams are '''not''' allowed to modify the .shp, .shx, .dbf or .prj files of the vector data (for the baseline benchmark). They can, however, create auxiliary files for spatial indexes and the like.

{| border="1"
| '''Mapping Server''' || '''ext''' || '''type''' || '''command to create'''
|-
| MapServer || .qix || quadtree || shptree <shpfile> ([[shptree notes]])
|-
| GeoServer || .qix || quadtree || Using the ones generated for MapServer atm
|-
| Mapnik || .index || quadtree || shapeindex <shpfile> ([[Benchmarking_2010/Mapnik_notes#shpindex_depth]])
|-
|Erdas || .rtr || RTree || RTreeBuilder basedir shapename max min ([[RTreeBuilder notes]])
|-
|Cadcorp || *.shp.idx || RTree || GUI command in desktop SIS ([[Benchmarking_2010/Cadcorp_notes]])
|-
| Constellation-SDI || .qix || quadtree || Using the ones generated by MapServer
|-
| MapViewer || .oix || RTree || GUI option in MapBuilder.jar, java map/theme/style builder
|}

* Note: in your server setup documentation be sure to record the EXACT command used to create the indexes, with appropriate options passed (e.g depth)

== Datasets ==

A set of data published by the spanish National Mapping Agency will be used. This data is free for non-commercial use, so it's perfectly OK to use it in the benchmark.

Vector data will be a topographic map, composed of several shapefiles (one shapefile per theme). Raster data will be 50cm/px aerial photography.

We still do not have the definitive datasets, due to their size and the processing time needed to put them together. You can get some sample data by reading [[Benchmarking 2010/How to get some sample data]].

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Hardware ==

Machine A (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

Machine B (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

Machine C (DB Host)
* System Type: Gateway E6610D
* Processor, Intel Core2 Duo - E6750 2.66 Ghz
* 4Gb Ram
* 250Gb Hard Drive 7200 rpm
* OS: Centos 5.5 x86-64

Machine D (testing server (JMeter))
* System Type: Dell Precision Workstation 390
* Ship Date: 9/7/2006
* Processor, 6300, 1.86, 2M, Core Duo-conroe, Burn 2
* 2Gb RAM
* 160 Gb Hard drive 7200 rpm
* Service Tag: 5Q5LQB1
* OS: Centos 5.5 i386

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed August 18th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=18&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** baseline test styling equivalence: are we there yet?
*** starting and stopping all servers
*** slotting times for each team to run benchmarks
*** cleaning up the rendering benchmark blank tiles
*** graphic access to the JMeter machine
*** last testing date

=== Previous IRC meeting ===

* Wed August 11th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=11&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subsets?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
*** Status report from each team
*** Testing deadline: shall we extend it?
*** Can all server provide links to the sample requests and allow to build more? (style comparison, are we all doing the same work?)
*** Vector data
**** Changes in rendering (missing values for some attributes)
**** Gaps: what to do with the 9 missing sheets?
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-08-11.log log]

* Wed August 4th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=4&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Machine status / issues
*** Data status / issues
**** Characters mis-encoded in shapefiles (e.g. catalan in labels): fixable? impact on PostGIS database?
**** Raster conversion to GeoTIFF (torrent of results?)
*** Styles status / issues
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subset?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
** Summary: [[Benchmarking 2010 IRC meeting logs Aug 4th]]

* Wed July 28th
(no info)

* Wed July 21st @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=21&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** update on data styling
*** checkin on server installs / library dependencies

* Wed July 14th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=14&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** data styling discussions
**** scales to use for layers
**** missing full road network
**** layer to use for labels along lines
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-07-14.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Dimitri Monie (dmonie)
**** Dane Springmeyer (springmeyer)
*** Discussion:
**** data:
***** full dataset of Spain, vector and raster, has been processed by IvanSanchez, and he is checking it and will create a torrent for transfer, by the end of the week
***** because of shapefile size limit some layers may have to be split (contour-1.shp, contour-2.shp)
**** servers:
***** msmith has received both server machines
***** T1 router is being installed, it will be a dedicated line for this project
***** he is adding 2 TB drives to each
**** layer styling:
***** we notice that the Spain dataset we are using (BCN25) does not include street files (zoomed in road network)
***** we agreed to use contours for curved label tests instead
***** jmckenna will test the layer scales provided by Marco (QGIS mapserver)
***** aaime pointed out problem with the posted SLDs, in terms of the label property (jmckenna to look into it)
***** aaime pointed out that we should make sure that all servers compute the scales the same way
****** OGC recommends 90 DPI (GeoServer, Mapnik use the recommended 90...MapServer defaults to 72, but should be configurable)
****** all teams should look into how their engine handles this and report to group
**** other:
***** IvanSanchez reported that a Spanish custom-made WMS server may join the exercise (they access raster files only)
***** dmonie pointed out that Erdas has been quiet lately because one of the team leaders is on holiday

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 23rd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=23&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on proposed layer styling
*** update on available server specs
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-23.log log])

* Wed June 16th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=16&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on server infrastructure
*** update on OSGeo-es dataset ([[Benchmarking 2010/How to get some sample data|Sample data download instructions]])
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-16.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Cédric Briançon (cedricbr)
*** Discussion:
**** "base" run will use only shapefiles, "best" run will be any format (raster/vector)
**** teams can optionally test Oracle, PostGIS, Ingres connections during the "best" run
**** layers tested will be: labelled roads (with labels that follow the lines), thematic polygon map, and a test with labelled roads + thematic polygon + point layer in a single request (3 separate runs)
***** msmith and jmckenna to examine styling for one sheet (Barcelona, sheet# 420)
**** each base and best run will test data in raw projection (ETRS89 + UTM28/29/30/31) and then reprojected on-the-fly to the Google Mercator projection
**** threads (number of concurrent requests) will be: 1-2-4-8-16-32-64
**** US Army Corps is proposing to provide 4 desktop machines. 2 identical for testing ( win/lin ) and 1 for jmeter and 1 for databases
***** msmith will find out availability of 2-cpu machines this week

* Wed June 2nd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=02&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** team representative introductions
*** Establish this year's 'Rules of Engagement' ([http://wiki.osgeo.org/wiki/Benchmarking_2009#Rules_of_Engagement previous RoE])
*** Discuss dataset to be used (including how to share the large dataset, for running local tests)
*** Discuss server infrastructure
*** Discuss possible changes to the test script parameters
*** Set next meeting date
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-02.log log])
*** Attendance:
**** Jeff McKenna, MapServer (jmckenna)
**** Cédric Briançon - Constellation (cedricbr)
**** Michael Smith - Oracle Mapviewer and Oracle connections in MapServer (msmith_)
**** Zac Spitzer - MapGuide (zacspitzer)
**** Martin Daly - Cadcorp (mpdaly)
**** Anne-Sophie Collignon - ERDAS (ascollignon)
**** Marco Hugentobler - QGIS mapserver (marco___)
**** Andrea Aime - GeoServer (aaime)
**** Dimitri Monie - ERDAS (dmonie)
**** Frank Warmerdam - MapServer, possibly Ingres as well (FrankW)
**** Iván Sánchez - provide geodata from Spanish sources and help out Mapnik (IvanSanchez)
**** Dane Springmeyer - Mapnik (springmeyer)
**** Johann Sorel - GeotoolKit / Constellation (Eclesia)
**** Adrian Custer - Constellation (acuster)
**** Pirmin Kalberer - QGIS mapserver (pirmin_k)
**** Trevor Wekel - MapGuide (trevorw)
**** Daniel Morissette - MapServer (danmo)
*** Discussion:
**** All agreed to this year's "Rules of Engagement"
***** All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
***** Comparisons will be made of the best available version of the software, be it a formal release or a development version.
***** Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
***** Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
***** Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
***** WMS output formats to be used will be png8 and png24 where possible
**** Dataset
***** Iván Sánchez Ortega from OSGeo-es is willing to provide local Spain datasets
***** data is for non-commercial use only
***** 10 GB of shapefiles
***** 100 GB of uncompressed rasters
**** Server infrastructure
***** possible hosts are msmith (US Army Corps) or Skygone (http://www.skygoneinc.com/)
***** need two identical servers, one running Unix and another Windows, plus a client machine
***** specs could be: quad core, 8GB of memory, 1TB hard drive
***** need fast pipe to server, as large datasets will be transferred
**** Next meeting date
***** all agreed to use same day and time next week, but limit the meeting length to one hour

== Sample Dataset Styling ==

* download [http://download.osgeo.org/benchmarking/barcelona-sample-dataset.zip sample dataset]
* layer [http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Vector_data_issues styling rules]
{| border="1"
|-
| '''Mapping Server'''
| '''GetMap'''
| '''SLD'''
|-
| MapServer
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 all layers]
| --
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=settlement&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 settlement]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/settlement-sld.xml settlement-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=building&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 building]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/building-sld.xml building-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=industry&SRS=EPSG:25831&BBOX=424932,4575961,427328,4577812&FORMAT=image/png&WIDTH=950&HEIGHT=600 industry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/industry-sld.xml industry-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=contour_lines&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 contour lines]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/contour-sld.xml contour-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=motorway&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 motorway]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/motorway-sld.xml motorway-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=road&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 road]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/road-sld.xml road-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=ramp&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 ramp]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/ramp-sld.xml ramp-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=track&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 track]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/track-sld.xml track-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_geometry&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels with geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-geom-sld.xml point-geom-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_nogeom&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels without geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-nogeom-sld.xml point-nogeom-sld.xml]
|}

=== Scale dependent rules checking requests ===

These requests show the behavior of scale dependent styling around the critical scales.
The geographic area requested is always the same:

<tt>BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102</tt>

The image size changes instead to alter the scale:

* roughly 1:29k: <tt>WIDTH=824&HEIGHT=472</tt>
* roughly 1:31k: <tt>WIDTH=771&HEIGHT=442</tt>
* roughly 1:49k: <tt>WIDTH=488&HEIGHT=280</tt>
* roughly 1.51k: <tt>WIDTH=469&HEIGHT=269</tt>

The table allows to compare the various servers output. The GeoServer links use a vendor option to get the actual scale, as computed by the server, displayed at the bottom/right of the map.

{| border="1"
|-
| '''Server'''
| '''1:29k'''
| '''1:31k'''
| '''1:49k'''
| '''1:51k'''
|-
| Constellation-SDI
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| GeoServer
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&format_options=layout:scale 1:29k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&format_options=layout:scale 1:31k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&format_options=layout:scale 1:49k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&format_options=layout:scale 1:51k]
|-
| Mapnik
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapServer
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapViewer
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| GeognoSIS
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| QGIS mapserver
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| ERDAS APOLLO
| [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&srs=EPSG:4326&FORMAT=image/png 1:29k]
| [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&srs=EPSG:4326&FORMAT=image/png 1:31k]
| [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&height=600&srs=EPSG:4326&FORMAT=image/png 1:49k]
| [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&srs=EPSG:4326&FORMAT=image/png 1:51k]
|}

== Benchmarking setup ==

Tests are going to be run using JMeter, using a progression of 1, 2, 4, 8, 16, 32, and 64 threads, each thread group doing 100, 200, 200, 400, 400, 800 requests respectively, for a total of 2200 requests.
The requests bounds and sizes are going to be picked up from a csv file generated by a random generator script.

The scripts to be run for the baseline tests, and repeated for the best effort one, are:
* raster data, JPEG output, EPSG:25831, scales between 1:1M and 1:10k
* raster data, JPEG output, EPSG:3857, scales between 1:1M and 1:10k
* vector data, PNG24 output, EPSG:4326, scales between 1:300k and 1:1k
* vector data, PNG24 output, EPSG:3875, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:25831, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:3785, scales between 1:300k and 1:1k

The areas to be tested are:
* 6 E, 38 N, -1 E, 43 N for vector tests
* 372360,4557880,484490,4631460 for raster tests

The raster layer in the baseline test is going to be the mosaic of GeoTIFF files Frank prepared out of the ECW files on the Linux server.

The vector data set is composed of a stack of individual layers to be stacked in the following order (bottom to top):
* settlements
* contour-0
* contour-1
* contour-2
* contour-3
* contour-4
* contour-5
* contour-6
* contour-7
* building
* industry
* track
* ramp
* road
* motorway
* point-labels-for-geometry
* point-labels-no-geometry

The best effort will use its own custom set of layers (e.g., it's likely contours will be merged into one, and so on)

== Live Benchmark WMS GetMap Requests ==

Note: sample bbox's fixed on Aug 12 to fit the proper aspect ratio of an 800/600 image request (springmeyer). Previous bbox for barcelona request was '1.8,41,2.4,42', new one used below is: '1.43333333333,41.0,2.76666666667,42.0'. Previous request for focused view was '2.1076723642349,41.407828508849,2.1178733654021,41.414271246429', new one is '2.10767236423,41.4072245022,2.1178733654,41.4148752531'. This was done to sidestep a deficiency in mapnik for affine transformation of geometries according to the WMS spec when the image w/h aspect does not match the bbox aspect, but was agreed upon as useful for these demo links by jeff and andrea in IRC.

=== MapServer ===

* Windows 64bit Server
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=postgis_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png PostGIS, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=postgis_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png PostGIS, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=oracle_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Oracle, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=oracle_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Oracle, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (zoomed-in)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (mid-zoom)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=316295,4519985.6640625,540555,4669784.3359375&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (whole-area)

* Linux
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (labelled contours, larger view)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=postgis_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png PostGIS, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=postgis_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png PostGIS, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=oracle_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Oracle, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=oracle_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Oracle, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (zoomed-in)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (mid-zoom)
** [http://12.189.158.78:8081/cgi-bin/mapservfull?MAP=/benchmarking/mapserver/spain-raster.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=raster_benchmarking_2010&STYLES=&BBOX=316295,4519985.6640625,540555,4669784.3359375&width=800&height=600&srs=EPSG:25831&FORMAT=image/jpeg GeoTIFF, MapServer 5.7-dev] (whole-area)

=== GeoServer ===

* [http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (Barcelona extents)
*[http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles GeoServer 2.1.x] (with labelled contours)
* [http://12.189.158.78:8080/geoserver/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (labelled contours, larger view)
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeoServer 2.1.x]

=== Mapnik ===
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik2 / paleoserver] (Barcelona extents)
* [http://12.189.158.78:8091/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik0.7.2 / mod_mapnik_wms] (Barcelona extents)
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles Mapnik2 / paleoserver] (larger view - labeled contours)

=== Constellation-SDI ===
* [http://12.189.158.78:8280/constellation/WS/wms?SRS=EPSG%3A4326&LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (Barcelona extents)
* [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (labelled contours, larger view)

=== Oracle MapViewer ===
*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (Barcelona Extents)

*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (with labeled contours)

=== Cadcorp GeognoSIS ===
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Shapefiles, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles, GeognoSIS 7.0] (with labelled contours)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 PostGIS, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 PostGIS, GeognoSIS 7.0] (with labelled contours)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Oracle, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Oracle, GeognoSIS 7.0] ('''no contours''' - under investigation)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, whole area, GeognoSIS 7.0]

=== QGIS mapserver ===
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SRS=EPSG:4326&LAYERS=shp_vector&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles, QGIS mapserver] (Barcelona Extents)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, QGIS mapserver] (with labeled contours)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/pg.qgs&SRS=EPSG:4326&LAYERS=shp_vector&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 PostGIS, QGIS mapserver] (Barcelona Extents)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/pg.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png PostGIS, QGIS mapserver] (with labeled contours)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector_raster.qgs&LAYERS=vrt&STYLES=&FORMAT=image/jpg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG:3043&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, QGIS mapserver]
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector_raster.qgs&LAYERS=vrt&STYLES=&FORMAT=image/jpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG:3043&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, QGIS mapserver]
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector_raster.qgs&LAYERS=vrt&STYLES=&FORMAT=image/jpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG:3043&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, QGIS mapserver]

=== ERDAS APOLLO ===

Windows 64bit Server
* [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, APOLLO 10.1.2] (Barcelona extents)
* [http://12.189.158.75:8080/erdas-apollo/vector/FOSS4GSHAPES?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shape_vector&STYLES=prop&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, APOLLO 10.1.2] (with labelled contours)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=TIFF&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 Geotiff, APOLLO 10.1.2] (Zoomed-in)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=TIFF&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 Geotiff, APOLLO 10.1.2](mid-zoom)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=TIFF&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 Geotiff, APOLLO 10.1.2] (Whole Area)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=BENCHMARK&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, APOLLO 10.1.2] (Zoomed-in)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=BENCHMARK&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, APOLLO 10.1.2] (mid-zoom)
* [http://12.189.158.75/ImageX/ecw_wms.dll?LAYERS=BENCHMARK&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, APOLLO 10.1.2] (Whole Area)

== Starting and stopping the various servers ==

* GeoServer on the Linux box:
** <tt>cd /benchmarking/geoserver</tt>
** starting: <tt>/benchmarking/geoserver/start_geoserver.sh</tt>
** stopping gracefully: <tt>/benchmarking/geoserver/stop_geoserver.sh</tt>
** check it's really dead: <tt>ps aux | grep GEOSERVER</tt> . If it hasn't stopped gracefully use <tt>kill -9 <pid></tt> to terminate the process

* MapServer and QGIS Mapserver on the Linux box (both are running in the same Apache instance):
** starting: /opt/mapserver/bin/apachectl start
** stopping: /opt/mapserver/bin/apachectl stop

* MapServer on Windows box:
** Command prompt (right-click "Run as Administrator")
** cd \ms4w
** apache-stop.bat
** apache-start.bat

* Mapnik:
** paleoserver (standalone daemon):
*** stopping: /opt/mapnik/paleoserver_stop.sh
*** starting: /opt/mapnik/paleoserver_start.sh
** mod_mapnik_wms (runs within apache):
** starting: /opt/mapnik/mod_mapnik_start.sh
** stopping: /opt/mapnik/mod_mapnik_stop.sh

* Mapviewer:
** <tt>cd /benchmarking/mapviewer</tt>
** starting: <tt> /opt/mapviewer/start_mv.sh </tt>
** stopping: <tt> /opt/mapviewer/stop_mv.sh </tt>
** check it's really dead: <tt>ps aux | grep java | grep mapviewer</tt> . If it hasn't stopped gracefully use <tt>kill -9 <pid></tt> to terminate the process

* Cadcorp GeognoSIS
** 7.0
*** Command line (with "Run as Administrator")
**** starting: net start geognosis-70
**** stopping: net stop geognosis-70
*** Or use services.msc, service name geognosis-70
** 7.1
*** Same as 7.0, but with "-71" suffix

* Erdas Windows Server
** APOLLO SDI (vector services)
*** Command Line (with "Run as Administrator")
**** starting : cd C:\opt\ERDAS\APOLLOSDI\tomcat\bin\startup.bat
**** stopping : cd C:\opt\ERDAS\APOLLOSDI\tomcat\bin\shutdown.bat
*** check the java process for erdas user has been killed
** APOLLO IWS (raster services)
*** Open IIS (Control Panel\Administrative Tools --> Internet Information Services (IIS) Manager)
**** starting : select under "Site" "Default Web Site" --> Manage Web Site : Start
**** Stopping : select under "Site" "Default Web Site" --> Manage Web Site : Stop

* Oracle Database
** Login to database machine and sudo to user oracle
***Starting: lsntctl start; dbstart
***Stopping: lsntcrl stop: dbshut

== List of Software Enhancements ==

=== MapServer ===
* large DBF support ([https://trac.osgeo.org/mapserver/ticket/3514 ticket])
* improving labels on curved lines ([http://trac.osgeo.org/mapserver/ticket/3523 ticket])
* handling label precision ([http://trac.osgeo.org/mapserver/ticket/2950 ticket])

=== Constellation-SDI ===

Detailed in the [[Benchmarking_2010/Constellation-SDI|Constellation-SDI]] page.

=== Cadcorp GeognoSIS ===

Detailed in the [[Benchmarking_2010/Cadcorp_notes|Cadcorp notes]] page.

== List of Performance Results ==

=== Constellation-SDI ===

Detailed in the [[Benchmarking_2010/Constellation-SDI|Constellation-SDI]] page.

== External Related Links ==

*[[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
*[http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010/Cadcorp notes

2010-08-20T10:44:25Z

Wiki-Mpd: /* Enhancements */

= Cadcorp Notes =

''(for the [[Benchmarking_2010]] exercise)''

== Software ==

=== Cadcorp SIS 7.0 ===

==== Bugfixes ====

* Handle DBF files > 2Gb
* Support PostGIS geometry columns with 4D (XYZM) geometry type
* Improved DBF string encoding handling

==== Enhancements ====

* Cache DBF record data for the current SHP record
* Allow Label Theme to use fill brush and/or outline pen from other Overlay Themes
* Improved Label Theme "along lines" option, to better suit contour data

== Data ==

=== Vector ===

==== SHP ====

* Created *.shp.idx Rtree spatial files via SIS UI

==== PostGIS ====

* "contours" table
** No spatial index - created by [[User:Pwramsey3]]
** No primary key/unique index column - created by [[User:Pwramsey3]]

==== Oracle ====

* No changes
* '''N.B.''' CONTOURS table not used: needs investigation

==== Cadcorp Feature Database (FDB) ====

* Created from SHP files
** One per SHP file, except for contour-* which were merged into a single file
* Columns not used for styling or labelling ignored, to reduce data transfer sizes
* Added FC ("feature code") column

=== Raster ===

==== TIF ====

* No changes

=== ECW ===

* No changes

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-20T10:42:38Z

Wiki-Mpd: Removed contour labelling nore

Benchmarking 2010

2010-08-19T18:43:58Z

Wiki-Mpd: Added start/stop instructions for GeognoSIS

Benchmarking 2010/Cadcorp notes

2010-08-19T09:22:42Z

Wiki-Mpd:

= Cadcorp Notes =

''(for the [[Benchmarking_2010]] exercise)''

== Software ==

=== Cadcorp SIS 7.0 ===

==== Bugfixes ====

* Handle DBF files > 2Gb
* Support PostGIS geometry columns with 4D (XYZM) geometry type
* Improved DBF string encoding handling

==== Enhancements ====

* Cache DBF record data for the current SHP record
* Allow Label Theme to use fill brush and/or outline pen from other Overlay Themes

== Data ==

=== Vector ===

==== SHP ====

* Created *.shp.idx Rtree spatial files via SIS UI

==== PostGIS ====

* "contours" table
** No spatial index - created by [[User:Pwramsey3]]
** No primary key/unique index column - created by [[User:Pwramsey3]]

==== Oracle ====

* No changes
* '''N.B.''' CONTOURS table not used: needs investigation

==== Cadcorp Feature Database (FDB) ====

* Created from SHP files
** One per SHP file, except for contour-* which were merged into a single file
* Columns not used for styling or labelling ignored, to reduce data transfer sizes
* Added FC ("feature code") column

=== Raster ===

==== TIF ====

* No changes

=== ECW ===

* No changes

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-19T09:11:48Z

Wiki-Mpd: Added link to Cadcorp notes page

== Basic Premise ==

Following up on [[Benchmarking 2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''IRC nick'''
| '''Server status'''
|-
| Cadcorp GeognoSIS
| [mailto:martin.daly--@--cadcorp.com Martin Daly]
| mpdaly
| Production release of server installed with preliminary configuration for SHP, PostGIS, TIF and ECW
|-
| Constellation-SDI
| [[User:Cedricbr|Cédric Briançon]]
| cedricbr
| Running on port 8280. Installation done for Shapefiles.
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting]
| dmonie
| Install and configuration on-going on windows server, ecw WMS configuration done, shape and DB WMS configuration on going. Port 80, port 8080
|-
| GeoServer
| Andrea Aime
| aaime
| Normally running on port 8080 on the Linux server. Shapefiles and TIFF mosaic have been configured.
|-
| <s>MapGuide Open Source</s>
| <s>Zac Spitzer</s>
| <s>zacspitzer</s>
| ''inactive''
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| springmeyer
| Running on port 8090 (paleoserver) 8091 (mod_mapnik_wms). Shapefiles styles ready, postgis and raster still to come.
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| jmckenna
| linux_wms_bm serving WMS requests for shps, on port 8081. windows wms_bm serving requests for shps, on port 8081.
|-
| Oracle MapViewer
| [[User:msmitherdc|Michael Smith]]
| msmith_
| Set up on port 8088
|-
| QGIS mapserver
| Marco Hugentobler
| marco___
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2010
| begin contacting all mapping servers
|-
| June 1st, 2010
| commitment due by all interested mapping servers
|-
| June 2nd, 2010
| exercise begins (and weekly meetings)
|-
| August 1st, 2010
| final testing begins
|-
| August 23rd, 2010
| no further testing
|-
| September 6-9, 2010
| present results at FOSS4G2010
|}

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
# Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
# Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Details and Differences ==

It is the responsibility of each team to document their setup with regard to data.

Please keep your notes in your servers directory in svn: http://svn.osgeo.org/osgeo/foss4g/benchmarking/

One particular thing that will differ is each team's use of spatial indexes on shapefiles.

Teams are '''not''' allowed to modify the .shp, .shx, .dbf or .prj files of the vector data (for the baseline benchmark). They can, however, create auxiliary files for spatial indexes and the like.

{| border="1"
| '''Mapping Server''' || '''ext''' || '''type''' || '''command to create'''
|-
| MapServer || .qix || quadtree || shptree <shpfile> ([[shptree notes]])
|-
| GeoServer || .qix || quadtree || Using the ones generated for MapServer atm
|-
| Mapnik || .index || quadtree || shapeindex <shpfile> ([[Benchmarking_2010/Mapnik_notes#shpindex_depth]])
|-
|Erdas || .rtr || RTree || RTreeBuilder basedir shapename max min ([[RTreeBuilder notes]])
|-
|Cadcorp || *.shp.idx || RTree || GUI command in desktop SIS ([[Benchmarking_2010/Cadcorp_notes]])
|-
| Constellation || .qix || quadtree || Using the ones generated by MapServer
|-
| MapViewer || .oix || RTree || GUI option in MapBuilder.jar, java map/theme/style builder
|}

* Note: in your server setup documentation be sure to record the EXACT command used to create the indexes, with appropriate options passed (e.g depth)

== Datasets ==

A set of data published by the spanish National Mapping Agency will be used. This data is free for non-commercial use, so it's perfectly OK to use it in the benchmark.

Vector data will be a topographic map, composed of several shapefiles (one shapefile per theme). Raster data will be 50cm/px aerial photography.

We still do not have the definitive datasets, due to their size and the processing time needed to put them together. You can get some sample data by reading [[Benchmarking 2010/How to get some sample data]].

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Hardware ==

Machine A (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

Machine B (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

Machine C (DB Host)
* System Type: Gateway E6610D
* Processor, Intel Core2 Duo - E6750 2.66 Ghz
* 4Gb Ram
* 250Gb Hard Drive 7200 rpm
* OS: Centos 5.5 x86-64

Machine D (testing server (JMeter))
* System Type: Dell Precision Workstation 390
* Ship Date: 9/7/2006
* Processor, 6300, 1.86, 2M, Core Duo-conroe, Burn 2
* 2Gb RAM
* 160 Gb Hard drive 7200 rpm
* Service Tag: 5Q5LQB1
* OS: Centos 5.5 i386

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed August 18th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=18&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** baseline test styling equivalence: are we there yet?
*** starting and stopping all servers
*** slotting times for each team to run benchmarks
*** cleaning up the rendering benchmark blank tiles
*** graphic access to the JMeter machine
*** last testing date

=== Previous IRC meeting ===

* Wed August 11th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=11&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subsets?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
*** Status report from each team
*** Testing deadline: shall we extend it?
*** Can all server provide links to the sample requests and allow to build more? (style comparison, are we all doing the same work?)
*** Vector data
**** Changes in rendering (missing values for some attributes)
**** Gaps: what to do with the 9 missing sheets?
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-08-11.log log]

* Wed August 4th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=4&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Machine status / issues
*** Data status / issues
**** Characters mis-encoded in shapefiles (e.g. catalan in labels): fixable? impact on PostGIS database?
**** Raster conversion to GeoTIFF (torrent of results?)
*** Styles status / issues
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subset?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
** Summary: [[Benchmarking 2010 IRC meeting logs Aug 4th]]

* Wed July 28th
(no info)

* Wed July 21st @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=21&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** update on data styling
*** checkin on server installs / library dependencies

* Wed July 14th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=14&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** data styling discussions
**** scales to use for layers
**** missing full road network
**** layer to use for labels along lines
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-07-14.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Dimitri Monie (dmonie)
**** Dane Springmeyer (springmeyer)
*** Discussion:
**** data:
***** full dataset of Spain, vector and raster, has been processed by IvanSanchez, and he is checking it and will create a torrent for transfer, by the end of the week
***** because of shapefile size limit some layers may have to be split (contour-1.shp, contour-2.shp)
**** servers:
***** msmith has received both server machines
***** T1 router is being installed, it will be a dedicated line for this project
***** he is adding 2 TB drives to each
**** layer styling:
***** we notice that the Spain dataset we are using (BCN25) does not include street files (zoomed in road network)
***** we agreed to use contours for curved label tests instead
***** jmckenna will test the layer scales provided by Marco (QGIS mapserver)
***** aaime pointed out problem with the posted SLDs, in terms of the label property (jmckenna to look into it)
***** aaime pointed out that we should make sure that all servers compute the scales the same way
****** OGC recommends 90 DPI (GeoServer, Mapnik use the recommended 90...MapServer defaults to 72, but should be configurable)
****** all teams should look into how their engine handles this and report to group
**** other:
***** IvanSanchez reported that a Spanish custom-made WMS server may join the exercise (they access raster files only)
***** dmonie pointed out that Erdas has been quiet lately because one of the team leaders is on holiday

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 23rd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=23&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on proposed layer styling
*** update on available server specs
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-23.log log])

* Wed June 16th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=16&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on server infrastructure
*** update on OSGeo-es dataset ([[Benchmarking 2010/How to get some sample data|Sample data download instructions]])
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-16.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Cédric Briançon (cedricbr)
*** Discussion:
**** "base" run will use only shapefiles, "best" run will be any format (raster/vector)
**** teams can optionally test Oracle, PostGIS, Ingres connections during the "best" run
**** layers tested will be: labelled roads (with labels that follow the lines), thematic polygon map, and a test with labelled roads + thematic polygon + point layer in a single request (3 separate runs)
***** msmith and jmckenna to examine styling for one sheet (Barcelona, sheet# 420)
**** each base and best run will test data in raw projection (ETRS89 + UTM28/29/30/31) and then reprojected on-the-fly to the Google Mercator projection
**** threads (number of concurrent requests) will be: 1-2-4-8-16-32-64
**** US Army Corps is proposing to provide 4 desktop machines. 2 identical for testing ( win/lin ) and 1 for jmeter and 1 for databases
***** msmith will find out availability of 2-cpu machines this week

* Wed June 2nd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=02&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** team representative introductions
*** Establish this year's 'Rules of Engagement' ([http://wiki.osgeo.org/wiki/Benchmarking_2009#Rules_of_Engagement previous RoE])
*** Discuss dataset to be used (including how to share the large dataset, for running local tests)
*** Discuss server infrastructure
*** Discuss possible changes to the test script parameters
*** Set next meeting date
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-02.log log])
*** Attendance:
**** Jeff McKenna, MapServer (jmckenna)
**** Cédric Briançon - Constellation (cedricbr)
**** Michael Smith - Oracle Mapviewer and Oracle connections in MapServer (msmith_)
**** Zac Spitzer - MapGuide (zacspitzer)
**** Martin Daly - Cadcorp (mpdaly)
**** Anne-Sophie Collignon - ERDAS (ascollignon)
**** Marco Hugentobler - QGIS mapserver (marco___)
**** Andrea Aime - GeoServer (aaime)
**** Dimitri Monie - ERDAS (dmonie)
**** Frank Warmerdam - MapServer, possibly Ingres as well (FrankW)
**** Iván Sánchez - provide geodata from Spanish sources and help out Mapnik (IvanSanchez)
**** Dane Springmeyer - Mapnik (springmeyer)
**** Johann Sorel - GeotoolKit / Constellation (Eclesia)
**** Adrian Custer - Constellation (acuster)
**** Pirmin Kalberer - QGIS mapserver (pirmin_k)
**** Trevor Wekel - MapGuide (trevorw)
**** Daniel Morissette - MapServer (danmo)
*** Discussion:
**** All agreed to this year's "Rules of Engagement"
***** All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
***** Comparisons will be made of the best available version of the software, be it a formal release or a development version.
***** Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
***** Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
***** Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
***** WMS output formats to be used will be png8 and png24 where possible
**** Dataset
***** Iván Sánchez Ortega from OSGeo-es is willing to provide local Spain datasets
***** data is for non-commercial use only
***** 10 GB of shapefiles
***** 100 GB of uncompressed rasters
**** Server infrastructure
***** possible hosts are msmith (US Army Corps) or Skygone (http://www.skygoneinc.com/)
***** need two identical servers, one running Unix and another Windows, plus a client machine
***** specs could be: quad core, 8GB of memory, 1TB hard drive
***** need fast pipe to server, as large datasets will be transferred
**** Next meeting date
***** all agreed to use same day and time next week, but limit the meeting length to one hour

== Sample Dataset Styling ==

* download [http://download.osgeo.org/benchmarking/barcelona-sample-dataset.zip sample dataset]
* layer [http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Vector_data_issues styling rules]
{| border="1"
|-
| '''Mapping Server'''
| '''GetMap'''
| '''SLD'''
|-
| MapServer
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 all layers]
| --
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=settlement&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 settlement]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/settlement-sld.xml settlement-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=building&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 building]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/building-sld.xml building-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=industry&SRS=EPSG:25831&BBOX=424932,4575961,427328,4577812&FORMAT=image/png&WIDTH=950&HEIGHT=600 industry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/industry-sld.xml industry-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=contour_lines&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 contour lines]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/contour-sld.xml contour-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=motorway&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 motorway]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/motorway-sld.xml motorway-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=road&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 road]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/road-sld.xml road-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=ramp&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 ramp]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/ramp-sld.xml ramp-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=track&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 track]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/track-sld.xml track-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_geometry&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels with geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-geom-sld.xml point-geom-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_nogeom&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels without geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-nogeom-sld.xml point-nogeom-sld.xml]
|}

== Benchmarking setup ==

Tests are going to be run using JMeter, using a progression of 1, 2, 4, 8, 16, 32, and 64 threads, each thread group doing 100, 200, 200, 400, 400, 800 requests respectively, for a total of 2200 requests.
The requests bounds and sizes are going to be picked up from a csv file generated by a random generator script.

The scripts to be run for the baseline tests, and repeated for the best effort one, are:
* raster data, JPEG output, EPSG:25831, scales between 1:1M and 1:10k
* raster data, JPEG output, EPSG:3857, scales between 1:1M and 1:10k
* vector data, PNG24 output, EPSG:4326, scales between 1:300k and 1:1k
* vector data, PNG24 output, EPSG:3875, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:25831, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:3785, scales between 1:300k and 1:1k

The areas to be tested are:
* 6 E, 38 N, -1 E, 43 N for vector tests
* 372360,4557880,484490,4631460 for raster tests

The raster layer in the baseline test is going to be the mosaic of GeoTIFF files Frank prepared out of the ECW files on the Linux server.

The vector data set is composed of a stack of individual layers to be stacked in the following order (bottom to top):
* settlements
* contour-0
* contour-1
* contour-2
* contour-3
* contour-4
* contour-5
* contour-6
* contour-7
* building
* industry
* track
* ramp
* road
* motorway
* point-labels-for-geometry
* point-labels-no-geometry

The best effort will use its own custom set of layers (e.g., it's likely contours will be merged into one, and so on)

== Live Benchmark WMS GetMap Requests ==

Note: sample bbox's fixed on Aug 12 to fit the proper aspect ratio of an 800/600 image request (springmeyer). Previous bbox for barcelona request was '1.8,41,2.4,42', new one used below is: '1.43333333333,41.0,2.76666666667,42.0'. Previous request for focused view was '2.1076723642349,41.407828508849,2.1178733654021,41.414271246429', new one is '2.10767236423,41.4072245022,2.1178733654,41.4148752531'. This was done to sidestep a deficiency in mapnik for affine transformation of geometries according to the WMS spec when the image w/h aspect does not match the bbox aspect, but was agreed upon as useful for these demo links by jeff and andrea in IRC.

=== MapServer ===

* Windows 64bit Server
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)

* Linux
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (labelled contours, larger view)

=== GeoServer ===

* [http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (Barcelona extents)
*[http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles GeoServer 2.1.x] (with labelled contours)
* [http://12.189.158.78:8080/geoserver/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (labelled contours, larger view)
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeoServer 2.1.x]

=== Mapnik ===
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik2 / paleoserver] (Barcelona extents)
* [http://12.189.158.78:8091/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik0.7.2 / mod_mapnik_wms] (Barcelona extents)
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles Mapnik2 / paleoserver] (larger view - labeled contours)

=== Constellation ===
* [http://12.189.158.78:8280/constellation/WS/wms?SRS=EPSG%3A4326&LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (Barcelona extents)
* [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (labelled contours, larger view)

=== Oracle MapViewer ===
*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (Barcelona Extents)

*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (with labeled contours)

=== Cadcorp GeognoSIS ===
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Shapefiles, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 PostGIS, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 PostGIS, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Oracle, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Oracle, GeognoSIS 7.0] ('''no contours''' - under investigation)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, whole area, GeognoSIS 7.0]

'''N.B.''' The current GeognoSIS Label Theme "label along lines" option does not work well with this data. We hope to have an alternative option ready in time for the tests

=== QGIS mapserver ===
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SRS=EPSG:4326&LAYERS=shp_vector&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 QGIS mapserver shapefile] (Barcelona Extents)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png QGIS mapserver shapefile] (with labeled contours)

=== Scale dependent rules checking requests ===

These requests show the behavior of scale dependent styling around the critical scales.
The geographic area requested is always the same:

<tt>BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102</tt>

The image size changes instead to alter the scale:

* roughly 1:29k: <tt>WIDTH=824&HEIGHT=472</tt>
* roughly 1:31k: <tt>WIDTH=771&HEIGHT=442</tt>
* roughly 1:49k: <tt>WIDTH=488&HEIGHT=280</tt>
* roughly 1.51k: <tt>WIDTH=469&HEIGHT=269</tt>

The table allows to compare the various servers output. The GeoServer links use a vendor option to get the actual scale, as computed by the server, displayed at the bottom/right of the map.

{| border="1"
|-
| '''Server'''
| '''1:29k'''
| '''1:31k'''
| '''1:49k'''
| '''1:51k'''
|-
| Constellation
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| GeoServer
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&format_options=layout:scale 1:29k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&format_options=layout:scale 1:31k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&format_options=layout:scale 1:49k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&format_options=layout:scale 1:51k]
|-
| Mapnik
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapServer
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapViewer
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| GeognoSIS
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| QGIS mapserver
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|}

== External Related Links ==

*[[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
*[http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010/Cadcorp notes

2010-08-19T09:10:45Z

Wiki-Mpd: Created page with '= Cadcorp Notes = ''(for the Benchmarking_2010 exercise)'' == Software == === Cadcorp SIS 7.0 === ==== Bugfixes ==== * Handle DBF files > 2Gb * Support PostGIS geometry …'

= Cadcorp Notes =

''(for the [[Benchmarking_2010]] exercise)''

== Software ==

=== Cadcorp SIS 7.0 ===

==== Bugfixes ====

* Handle DBF files > 2Gb
* Support PostGIS geometry columns with 4D (XYZM) geometry type
* Improved DBF string encoding handling

==== Enhancements ====

* Cache DBF record data for the current feature
* Allow Label Theme to use fill brush and/or outline pen from other Overlay Themes

== Data ==

=== Vector ===

==== SHP ====

* Created *.shp.idx Rtree spatial files via SIS UI

==== PostGIS ====

* "contours" table
** No spatial index - created by [[User:Pwramsey3]]
** No primary key/unique index column - created by [[User:Pwramsey3]]

==== Oracle ====

* No changes
* '''N.B.''' CONTOURS table not used: needs investigation

==== Cadcorp Feature Database (FDB) ====

* Created from SHP files
** One per SHP file, except for contour-* which were merged into a single file
* Columns not used for styling or labelling ignored, to reduce data transfer sizes
* Added FC ("feature code") column

=== Raster ===

==== TIF ====

* No changes

=== ECW ===

* No changes

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-17T16:06:06Z

Wiki-Mpd: Added Oracle + GeognoSIS URLs

== Basic Premise ==

Following up on [[Benchmarking 2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''IRC nick'''
| '''Server status'''
|-
| Cadcorp GeognoSIS
| [mailto:martin.daly--@--cadcorp.com Martin Daly]
| mpdaly
| Production release of server installed with preliminary configuration for SHP, PostGIS, TIF and ECW
|-
| Constellation-SDI
| [[User:Cedricbr|Cédric Briançon]]
| cedricbr
| Running on port 8280. Installation done for Shapefiles.
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting]
| dmonie
| Install and configuration on-going on windows server, ecw WMS configuration done, shape and DB WMS configuration on going. Port 80, port 8080
|-
| GeoServer
| Andrea Aime
| aaime
| Normally running on port 8080 on the Linux server. Shapefiles and TIFF mosaic have been configured.
|-
| MapGuide Open Source
| Zac Spitzer
| zacspitzer
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| springmeyer
| Running on port 8090 (paleoserver) 8091 (mod_mapnik_wms). Shapefiles styles ready, postgis and raster still to come.
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| jmckenna
| linux_wms_bm serving WMS requests for shps, on port 8081. windows wms_bm serving requests for shps, on port 8081.
|-
| Oracle MapViewer
| [[User:msmitherdc|Michael Smith]]
| msmith_
| Set up on port 8088
|-
| QGIS mapserver
| Marco Hugentobler
| marco___
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2010
| begin contacting all mapping servers
|-
| June 1st, 2010
| commitment due by all interested mapping servers
|-
| June 2nd, 2010
| exercise begins (and weekly meetings)
|-
| August 1st, 2010
| final testing begins
|-
| August 23rd, 2010
| no further testing
|-
| September 6-9, 2010
| present results at FOSS4G2010
|}

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
# Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
# Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Details and Differences ==

It is the responsibility of each team to document their setup with regard to data.

Please keep your notes in your servers directory in svn: http://svn.osgeo.org/osgeo/foss4g/benchmarking/

One particular thing that will differ is each team's use of spatial indexes on shapefiles.

Teams are '''not''' allowed to modify the .shp, .shx, .dbf or .prj files of the vector data (for the baseline benchmark). They can, however, create auxiliary files for spatial indexes and the like.

{| border="1"
| '''Mapping Server''' || '''ext''' || '''type''' || '''command to create'''
|-
| MapServer || .qix || quadtree || shptree <shpfile> ([[shptree notes]])
|-
| GeoServer || .qix || quadtree || Using the ones generated for MapServer atm
|-
| Mapnik || .index || quadtree || shapeindex <shpfile> ([[Benchmarking_2010/Mapnik_notes#shpindex_depth]])
|-
|Erdas || .rtr || RTree || RTreeBuilder basedir shapename max min ([[RTreeBuilder notes]])
|-
|Cadcorp || *.shp.idx || RTree || GUI command in desktop SIS
|-
| Constellation || .qix || quadtree || Using the ones generated by MapServer
|-
| MapViewer || .oix || RTree || GUI option in MapBuilder.jar, java map/theme/style builder
|}

* Note: in your server setup documentation be sure to record the EXACT command used to create the indexes, with appropriate options passed (e.g depth)

== Datasets ==

A set of data published by the spanish National Mapping Agency will be used. This data is free for non-commercial use, so it's perfectly OK to use it in the benchmark.

Vector data will be a topographic map, composed of several shapefiles (one shapefile per theme). Raster data will be 50cm/px aerial photography.

We still do not have the definitive datasets, due to their size and the processing time needed to put them together. You can get some sample data by reading [[Benchmarking 2010/How to get some sample data]].

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Hardware ==

Machine A (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

Machine B (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

Machine C (DB Host)
* System Type: Gateway E6610D
* Processor, Intel Core2 Duo - E6750 2.66 Ghz
* 4Gb Ram
* 250Gb Hard Drive 7200 rpm
* OS: Centos 5.5 x86-64

Machine D (testing server (JMeter))
* System Type: Dell Precision Workstation 390
* Ship Date: 9/7/2006
* Processor, 6300, 1.86, 2M, Core Duo-conroe, Burn 2
* 2Gb RAM
* 160 Gb Hard drive 7200 rpm
* Service Tag: 5Q5LQB1
* OS: Centos 5.5 i386

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed August 11th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=11&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subsets?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
*** Status report from each team
*** Testing deadline: shall we extend it?
*** Can all server provide links to the sample requests and allow to build more? (style comparison, are we all doing the same work?)
*** Vector data
**** Changes in rendering (missing values for some attributes)
**** Gaps: what to do with the 9 missing sheets?

=== Previous IRC meeting ===

* Wed August 4th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=4&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Machine status / issues
*** Data status / issues
**** Characters mis-encoded in shapefiles (e.g. catalan in labels): fixable? impact on PostGIS database?
**** Raster conversion to GeoTIFF (torrent of results?)
*** Styles status / issues
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subset?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
** Summary: [[Benchmarking 2010 IRC meeting logs Aug 4th]]

* Wed July 28th
(no info)

* Wed July 21st @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=21&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** update on data styling
*** checkin on server installs / library dependencies

* Wed July 14th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=14&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** data styling discussions
**** scales to use for layers
**** missing full road network
**** layer to use for labels along lines
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-07-14.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Dimitri Monie (dmonie)
**** Dane Springmeyer (springmeyer)
*** Discussion:
**** data:
***** full dataset of Spain, vector and raster, has been processed by IvanSanchez, and he is checking it and will create a torrent for transfer, by the end of the week
***** because of shapefile size limit some layers may have to be split (contour-1.shp, contour-2.shp)
**** servers:
***** msmith has received both server machines
***** T1 router is being installed, it will be a dedicated line for this project
***** he is adding 2 TB drives to each
**** layer styling:
***** we notice that the Spain dataset we are using (BCN25) does not include street files (zoomed in road network)
***** we agreed to use contours for curved label tests instead
***** jmckenna will test the layer scales provided by Marco (QGIS mapserver)
***** aaime pointed out problem with the posted SLDs, in terms of the label property (jmckenna to look into it)
***** aaime pointed out that we should make sure that all servers compute the scales the same way
****** OGC recommends 90 DPI (GeoServer, Mapnik use the recommended 90...MapServer defaults to 72, but should be configurable)
****** all teams should look into how their engine handles this and report to group
**** other:
***** IvanSanchez reported that a Spanish custom-made WMS server may join the exercise (they access raster files only)
***** dmonie pointed out that Erdas has been quiet lately because one of the team leaders is on holiday

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 23rd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=23&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on proposed layer styling
*** update on available server specs
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-23.log log])

* Wed June 16th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=16&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on server infrastructure
*** update on OSGeo-es dataset ([[Benchmarking 2010/How to get some sample data|Sample data download instructions]])
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-16.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Cédric Briançon (cedricbr)
*** Discussion:
**** "base" run will use only shapefiles, "best" run will be any format (raster/vector)
**** teams can optionally test Oracle, PostGIS, Ingres connections during the "best" run
**** layers tested will be: labelled roads (with labels that follow the lines), thematic polygon map, and a test with labelled roads + thematic polygon + point layer in a single request (3 separate runs)
***** msmith and jmckenna to examine styling for one sheet (Barcelona, sheet# 420)
**** each base and best run will test data in raw projection (ETRS89 + UTM28/29/30/31) and then reprojected on-the-fly to the Google Mercator projection
**** threads (number of concurrent requests) will be: 1-2-4-8-16-32-64
**** US Army Corps is proposing to provide 4 desktop machines. 2 identical for testing ( win/lin ) and 1 for jmeter and 1 for databases
***** msmith will find out availability of 2-cpu machines this week

* Wed June 2nd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=02&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** team representative introductions
*** Establish this year's 'Rules of Engagement' ([http://wiki.osgeo.org/wiki/Benchmarking_2009#Rules_of_Engagement previous RoE])
*** Discuss dataset to be used (including how to share the large dataset, for running local tests)
*** Discuss server infrastructure
*** Discuss possible changes to the test script parameters
*** Set next meeting date
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-02.log log])
*** Attendance:
**** Jeff McKenna, MapServer (jmckenna)
**** Cédric Briançon - Constellation (cedricbr)
**** Michael Smith - Oracle Mapviewer and Oracle connections in MapServer (msmith_)
**** Zac Spitzer - MapGuide (zacspitzer)
**** Martin Daly - Cadcorp (mpdaly)
**** Anne-Sophie Collignon - ERDAS (ascollignon)
**** Marco Hugentobler - QGIS mapserver (marco___)
**** Andrea Aime - GeoServer (aaime)
**** Dimitri Monie - ERDAS (dmonie)
**** Frank Warmerdam - MapServer, possibly Ingres as well (FrankW)
**** Iván Sánchez - provide geodata from Spanish sources and help out Mapnik (IvanSanchez)
**** Dane Springmeyer - Mapnik (springmeyer)
**** Johann Sorel - GeotoolKit / Constellation (Eclesia)
**** Adrian Custer - Constellation (acuster)
**** Pirmin Kalberer - QGIS mapserver (pirmin_k)
**** Trevor Wekel - MapGuide (trevorw)
**** Daniel Morissette - MapServer (danmo)
*** Discussion:
**** All agreed to this year's "Rules of Engagement"
***** All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
***** Comparisons will be made of the best available version of the software, be it a formal release or a development version.
***** Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
***** Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
***** Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
***** WMS output formats to be used will be png8 and png24 where possible
**** Dataset
***** Iván Sánchez Ortega from OSGeo-es is willing to provide local Spain datasets
***** data is for non-commercial use only
***** 10 GB of shapefiles
***** 100 GB of uncompressed rasters
**** Server infrastructure
***** possible hosts are msmith (US Army Corps) or Skygone (http://www.skygoneinc.com/)
***** need two identical servers, one running Unix and another Windows, plus a client machine
***** specs could be: quad core, 8GB of memory, 1TB hard drive
***** need fast pipe to server, as large datasets will be transferred
**** Next meeting date
***** all agreed to use same day and time next week, but limit the meeting length to one hour

== Sample Dataset Styling ==

* download [http://download.osgeo.org/benchmarking/barcelona-sample-dataset.zip sample dataset]
* layer [http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Vector_data_issues styling rules]
{| border="1"
|-
| '''Mapping Server'''
| '''GetMap'''
| '''SLD'''
|-
| MapServer
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 all layers]
| --
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=settlement&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 settlement]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/settlement-sld.xml settlement-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=building&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 building]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/building-sld.xml building-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=industry&SRS=EPSG:25831&BBOX=424932,4575961,427328,4577812&FORMAT=image/png&WIDTH=950&HEIGHT=600 industry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/industry-sld.xml industry-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=contour_lines&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 contour lines]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/contour-sld.xml contour-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=motorway&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 motorway]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/motorway-sld.xml motorway-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=road&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 road]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/road-sld.xml road-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=ramp&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 ramp]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/ramp-sld.xml ramp-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=track&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 track]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/track-sld.xml track-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_geometry&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels with geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-geom-sld.xml point-geom-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_nogeom&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels without geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-nogeom-sld.xml point-nogeom-sld.xml]
|}

== Benchmarking setup ==

Tests are going to be run using JMeter, using a progression of 1, 2, 4, 8, 16, 32, and 64 threads, each thread group doing 100, 200, 200, 400, 400, 800 requests respectively, for a total of 2200 requests.
The requests bounds and sizes are going to be picked up from a csv file generated by a random generator script.

The scripts to be run for the baseline tests, and repeated for the best effort one, are:
* raster data, JPEG output, EPSG:25831, scales between 1:1M and 1:10k
* raster data, JPEG output, EPSG:3857, scales between 1:1M and 1:10k
* vector data, PNG24 output, EPSG:4326, scales between 1:300k and 1:1k
* vector data, PNG24 output, EPSG:3875, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:25831, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:3785, scales between 1:300k and 1:1k

The areas to be tested are:
* 6 E, 38 N, -1 E, 43 N for vector tests
* 372360,4557880,484490,4631460 for raster tests

The raster layer in the baseline test is going to be the mosaic of GeoTIFF files Frank prepared out of the ECW files on the Linux server.

The vector data set is composed of a stack of individual layers to be stacked in the following order (bottom to top):
* settlements
* contour-0
* contour-1
* contour-2
* contour-3
* contour-4
* contour-5
* contour-6
* contour-7
* building
* industry
* track
* ramp
* road
* motorway
* point-labels-for-geometry
* point-labels-no-geometry

The best effort will use its own custom set of layers (e.g., it's likely contours will be merged into one, and so on)

== Live Benchmark WMS GetMap Requests ==

Note: sample bbox's fixed on Aug 12 to fit the proper aspect ratio of an 800/600 image request (springmeyer). Previous bbox for barcelona request was '1.8,41,2.4,42', new one used below is: '1.43333333333,41.0,2.76666666667,42.0'. Previous request for focused view was '2.1076723642349,41.407828508849,2.1178733654021,41.414271246429', new one is '2.10767236423,41.4072245022,2.1178733654,41.4148752531'. This was done to sidestep a deficiency in mapnik for affine transformation of geometries according to the WMS spec when the image w/h aspect does not match the bbox aspect, but was agreed upon as useful for these demo links by jeff and andrea in IRC.

=== MapServer ===

* Windows 64bit Server
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)

* Linux
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (labelled contours, larger view)

=== GeoServer ===

* [http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (Barcelona extents)
*[http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles GeoServer 2.1.x] (with labelled contours)
* [http://12.189.158.78:8080/geoserver/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (labelled contours, larger view)
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeoServer 2.1.x]

=== Mapnik ===
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik2 / paleoserver] (Barcelona extents)
* [http://12.189.158.78:8091/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik0.7.2 / mod_mapnik_wms] (Barcelona extents)
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles Mapnik2 / paleoserver] (larger view - no labels yet)

=== Constellation ===
* [http://12.189.158.78:8280/constellation/WS/wms?SRS=EPSG%3A4326&LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (Barcelona extents)
* [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (labelled contours, larger view)

=== Oracle MapViewer ===
*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (Barcelona Extents)

*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (with labeled contours)

=== Cadcorp GeognoSIS ===
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Shapefiles, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 PostGIS, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 PostGIS, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Oracle, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-oracle&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Oracle, GeognoSIS 7.0] ('''no contours''' - under investigation)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, whole area, GeognoSIS 7.0]

'''N.B.''' The current GeognoSIS Label Theme "label along lines" option does not work well with this data. We hope to have an alternative option ready in time for the tests

=== QGIS mapserver ===
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SRS=EPSG:4326&LAYERS=shp_vector&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 QGIS mapserver shapefile] (Barcelona Extents)
* [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png QGIS mapserver shapefile] (with labeled contours)

=== Scale dependent rules checking requests ===

These requests show the behavior of scale dependent styling around the critical scales.
The geographic area requested is always the same:

<tt>BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102</tt>

The image size changes instead to alter the scale:

* roughly 1:29k: <tt>WIDTH=824&HEIGHT=472</tt>
* roughly 1:31k: <tt>WIDTH=771&HEIGHT=442</tt>
* roughly 1:49k: <tt>WIDTH=488&HEIGHT=280</tt>
* roughly 1.51k: <tt>WIDTH=469&HEIGHT=269</tt>

The table allows to compare the various servers output. The GeoServer links use a vendor option to get the actual scale, as computed by the server, displayed at the bottom/right of the map.

{| border="1"
|-
| '''Server'''
| '''1:29k'''
| '''1:31k'''
| '''1:49k'''
| '''1:51k'''
|-
| Constellation
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| GeoServer
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&format_options=layout:scale 1:29k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&format_options=layout:scale 1:31k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&format_options=layout:scale 1:49k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&format_options=layout:scale 1:51k]
|-
| Mapnik
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapServer
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapViewer
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| GeognoSIS
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| QGIS mapserver
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/fcgi-bin/qgis_mapserv.fcgi?MAP=/opt/qgisms/fcgi-bin/vector.qgs&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=shp_vector&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|}

== External Related Links ==

*[[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
*[http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-16T15:33:45Z

Wiki-Mpd: /* Participants */

== Basic Premise ==

Following up on [[Benchmarking 2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''IRC nick'''
| '''Server status'''
|-
| Cadcorp GeognoSIS
| [mailto:martin.daly--@--cadcorp.com Martin Daly]
| mpdaly
| Production release of server installed with preliminary configuration for SHP, PostGIS, TIF and ECW
|-
| Constellation-SDI
| [[User:Cedricbr|Cédric Briançon]]
| cedricbr
| Running on port 8280. Installation done for Shapefiles.
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting]
| dmonie
| Install and configuration on-going on windows server, ecw WMS configuration done, shape and DB WMS configuration on going. Port 80, port 8080
|-
| GeoServer
| Andrea Aime
| aaime
| Normally running on port 8080 on the Linux server. Shapefiles and TIFF mosaic have been configured.
|-
| MapGuide Open Source
| Zac Spitzer
| zacspitzer
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| springmeyer
| Running on port 8090 (paleoserver) 8091 (mod_mapnik_wms). Shapefiles styles ready, postgis and raster still to come.
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| jmckenna
| linux_wms_bm serving WMS requests for shps, on port 8081. windows wms_bm serving requests for shps, on port 8081.
|-
| Oracle MapViewer
| [[User:msmitherdc|Michael Smith]]
| msmith_
| Set up on port 8088
|-
| QGIS mapserver
| Marco Hugentobler
| marco___
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2010
| begin contacting all mapping servers
|-
| June 1st, 2010
| commitment due by all interested mapping servers
|-
| June 2nd, 2010
| exercise begins (and weekly meetings)
|-
| August 1st, 2010
| final testing begins
|-
| August 23rd, 2010
| no further testing
|-
| September 6-9, 2010
| present results at FOSS4G2010
|}

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
# Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
# Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Details and Differences ==

It is the responsibility of each team to document their setup with regard to data.

Please keep your notes in your servers directory in svn: http://svn.osgeo.org/osgeo/foss4g/benchmarking/

One particular thing that will differ is each team's use of spatial indexes on shapefiles.

Teams are '''not''' allowed to modify the .shp, .shx, .dbf or .prj files of the vector data (for the baseline benchmark). They can, however, create auxiliary files for spatial indexes and the like.

{| border="1"
| '''Mapping Server''' || '''ext''' || '''type''' || '''command to create'''
|-
| MapServer || .qix || quadtree || shptree <shpfile> ([[shptree notes]])
|-
| GeoServer || .qix || quadtree || Using the ones generated for MapServer atm
|-
| Mapnik || .index || quadtree || shapeindex <shpfile> ([[Benchmarking_2010/Mapnik_notes#shpindex_depth]])
|-
|Erdas || .rtr || RTree || RTreeBuilder basedir shapename max min ([[RTreeBuilder notes]])
|-
|Cadcorp || *.shp.idx || RTree || GUI command in desktop SIS
|-
| Constellation || .qix || quadtree || Using the ones generated by MapServer
|}

* Note: in your server setup documentation be sure to record the EXACT command used to create the indexes, with appropriate options passed (e.g depth)

== Datasets ==

A set of data published by the spanish National Mapping Agency will be used. This data is free for non-commercial use, so it's perfectly OK to use it in the benchmark.

Vector data will be a topographic map, composed of several shapefiles (one shapefile per theme). Raster data will be 50cm/px aerial photography.

We still do not have the definitive datasets, due to their size and the processing time needed to put them together. You can get some sample data by reading [[Benchmarking 2010/How to get some sample data]].

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Hardware ==

Machine A (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

Machine B (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

Machine C (DB Host)
* System Type: Gateway E6610D
* Processor, Intel Core2 Duo - E6750 2.66 Ghz
* 4Gb Ram
* 250Gb Hard Drive 7200 rpm
* OS: Centos 5.5 x86-64

Machine D (testing server (JMeter))
* System Type: Dell Precision Workstation 390
* Ship Date: 9/7/2006
* Processor, 6300, 1.86, 2M, Core Duo-conroe, Burn 2
* 2Gb RAM
* 160 Gb Hard drive 7200 rpm
* Service Tag: 5Q5LQB1
* OS: Centos 5.5 i386

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed August 11th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=11&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subsets?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
*** Status report from each team
*** Testing deadline: shall we extend it?
*** Can all server provide links to the sample requests and allow to build more? (style comparison, are we all doing the same work?)
*** Vector data
**** Changes in rendering (missing values for some attributes)
**** Gaps: what to do with the 9 missing sheets?

=== Previous IRC meeting ===

* Wed August 4th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=4&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Machine status / issues
*** Data status / issues
**** Characters mis-encoded in shapefiles (e.g. catalan in labels): fixable? impact on PostGIS database?
**** Raster conversion to GeoTIFF (torrent of results?)
*** Styles status / issues
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subset?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
** Summary: [[Benchmarking 2010 IRC meeting logs Aug 4th]]

* Wed July 28th
(no info)

* Wed July 21st @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=21&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** update on data styling
*** checkin on server installs / library dependencies

* Wed July 14th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=14&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** data styling discussions
**** scales to use for layers
**** missing full road network
**** layer to use for labels along lines
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-07-14.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Dimitri Monie (dmonie)
**** Dane Springmeyer (springmeyer)
*** Discussion:
**** data:
***** full dataset of Spain, vector and raster, has been processed by IvanSanchez, and he is checking it and will create a torrent for transfer, by the end of the week
***** because of shapefile size limit some layers may have to be split (contour-1.shp, contour-2.shp)
**** servers:
***** msmith has received both server machines
***** T1 router is being installed, it will be a dedicated line for this project
***** he is adding 2 TB drives to each
**** layer styling:
***** we notice that the Spain dataset we are using (BCN25) does not include street files (zoomed in road network)
***** we agreed to use contours for curved label tests instead
***** jmckenna will test the layer scales provided by Marco (QGIS mapserver)
***** aaime pointed out problem with the posted SLDs, in terms of the label property (jmckenna to look into it)
***** aaime pointed out that we should make sure that all servers compute the scales the same way
****** OGC recommends 90 DPI (GeoServer, Mapnik use the recommended 90...MapServer defaults to 72, but should be configurable)
****** all teams should look into how their engine handles this and report to group
**** other:
***** IvanSanchez reported that a Spanish custom-made WMS server may join the exercise (they access raster files only)
***** dmonie pointed out that Erdas has been quiet lately because one of the team leaders is on holiday

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 23rd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=23&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on proposed layer styling
*** update on available server specs
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-23.log log])

* Wed June 16th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=16&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on server infrastructure
*** update on OSGeo-es dataset ([[Benchmarking 2010/How to get some sample data|Sample data download instructions]])
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-16.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Cédric Briançon (cedricbr)
*** Discussion:
**** "base" run will use only shapefiles, "best" run will be any format (raster/vector)
**** teams can optionally test Oracle, PostGIS, Ingres connections during the "best" run
**** layers tested will be: labelled roads (with labels that follow the lines), thematic polygon map, and a test with labelled roads + thematic polygon + point layer in a single request (3 separate runs)
***** msmith and jmckenna to examine styling for one sheet (Barcelona, sheet# 420)
**** each base and best run will test data in raw projection (ETRS89 + UTM28/29/30/31) and then reprojected on-the-fly to the Google Mercator projection
**** threads (number of concurrent requests) will be: 1-2-4-8-16-32-64
**** US Army Corps is proposing to provide 4 desktop machines. 2 identical for testing ( win/lin ) and 1 for jmeter and 1 for databases
***** msmith will find out availability of 2-cpu machines this week

* Wed June 2nd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=02&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** team representative introductions
*** Establish this year's 'Rules of Engagement' ([http://wiki.osgeo.org/wiki/Benchmarking_2009#Rules_of_Engagement previous RoE])
*** Discuss dataset to be used (including how to share the large dataset, for running local tests)
*** Discuss server infrastructure
*** Discuss possible changes to the test script parameters
*** Set next meeting date
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-02.log log])
*** Attendance:
**** Jeff McKenna, MapServer (jmckenna)
**** Cédric Briançon - Constellation (cedricbr)
**** Michael Smith - Oracle Mapviewer and Oracle connections in MapServer (msmith_)
**** Zac Spitzer - MapGuide (zacspitzer)
**** Martin Daly - Cadcorp (mpdaly)
**** Anne-Sophie Collignon - ERDAS (ascollignon)
**** Marco Hugentobler - QGIS mapserver (marco___)
**** Andrea Aime - GeoServer (aaime)
**** Dimitri Monie - ERDAS (dmonie)
**** Frank Warmerdam - MapServer, possibly Ingres as well (FrankW)
**** Iván Sánchez - provide geodata from Spanish sources and help out Mapnik (IvanSanchez)
**** Dane Springmeyer - Mapnik (springmeyer)
**** Johann Sorel - GeotoolKit / Constellation (Eclesia)
**** Adrian Custer - Constellation (acuster)
**** Pirmin Kalberer - QGIS mapserver (pirmin_k)
**** Trevor Wekel - MapGuide (trevorw)
**** Daniel Morissette - MapServer (danmo)
*** Discussion:
**** All agreed to this year's "Rules of Engagement"
***** All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
***** Comparisons will be made of the best available version of the software, be it a formal release or a development version.
***** Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
***** Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
***** Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
***** WMS output formats to be used will be png8 and png24 where possible
**** Dataset
***** Iván Sánchez Ortega from OSGeo-es is willing to provide local Spain datasets
***** data is for non-commercial use only
***** 10 GB of shapefiles
***** 100 GB of uncompressed rasters
**** Server infrastructure
***** possible hosts are msmith (US Army Corps) or Skygone (http://www.skygoneinc.com/)
***** need two identical servers, one running Unix and another Windows, plus a client machine
***** specs could be: quad core, 8GB of memory, 1TB hard drive
***** need fast pipe to server, as large datasets will be transferred
**** Next meeting date
***** all agreed to use same day and time next week, but limit the meeting length to one hour

== Sample Dataset Styling ==

* download [http://download.osgeo.org/benchmarking/barcelona-sample-dataset.zip sample dataset]
* layer [http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Vector_data_issues styling rules]
{| border="1"
|-
| '''Mapping Server'''
| '''GetMap'''
| '''SLD'''
|-
| MapServer
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 all layers]
| --
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=settlement&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 settlement]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/settlement-sld.xml settlement-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=building&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 building]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/building-sld.xml building-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=industry&SRS=EPSG:25831&BBOX=424932,4575961,427328,4577812&FORMAT=image/png&WIDTH=950&HEIGHT=600 industry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/industry-sld.xml industry-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=contour_lines&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 contour lines]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/contour-sld.xml contour-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=motorway&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 motorway]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/motorway-sld.xml motorway-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=road&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 road]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/road-sld.xml road-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=ramp&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 ramp]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/ramp-sld.xml ramp-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=track&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 track]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/track-sld.xml track-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_geometry&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels with geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-geom-sld.xml point-geom-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_nogeom&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels without geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-nogeom-sld.xml point-nogeom-sld.xml]
|}

== Benchmarking setup ==

Tests are going to be run using JMeter, using a progression of 1, 2, 4, 8, 16, 32, and 64 threads, each thread group doing 100, 200, 200, 400, 400, 800 requests respectively, for a total of 2200 requests.
The requests bounds and sizes are going to be picked up from a csv file generated by a random generator script.

The scripts to be run for the baseline tests, and repeated for the best effort one, are:
* raster data, JPEG output, EPSG:25831, scales between 1:1M and 1:10k
* raster data, JPEG output, EPSG:3857, scales between 1:1M and 1:10k
* vector data, PNG24 output, EPSG:4326, scales between 1:300k and 1:1k
* vector data, PNG24 output, EPSG:3875, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:25831, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:3785, scales between 1:300k and 1:1k

The areas to be tested are:
* 6 E, 38 N, -1 E, 43 N for vector tests
* 372360,4557880,484490,4631460 for raster tests

The raster layer in the baseline test is going to be the mosaic of GeoTIFF files Frank prepared out of the ECW files on the Linux server.

The vector data set is composed of a stack of individual layers to be stacked in the following order (bottom to top):
* settlements
* contour-0
* contour-1
* contour-2
* contour-3
* contour-4
* contour-5
* contour-6
* contour-7
* building
* industry
* track
* ramp
* road
* motorway
* point-labels-for-geometry
* point-labels-no-geometry

The best effort will use its own custom set of layers (e.g., it's likely contours will be merged into one, and so on)

== Live Benchmark WMS GetMap Requests ==

Note: sample bbox's fixed on Aug 12 to fit the proper aspect ratio of an 800/600 image request (springmeyer). Previous bbox for barcelona request was '1.8,41,2.4,42', new one used below is: '1.43333333333,41.0,2.76666666667,42.0'. Previous request for focused view was '2.1076723642349,41.407828508849,2.1178733654021,41.414271246429', new one is '2.10767236423,41.4072245022,2.1178733654,41.4148752531'. This was done to sidestep a deficiency in mapnik for affine transformation of geometries according to the WMS spec when the image w/h aspect does not match the bbox aspect, but was agreed upon as useful for these demo links by jeff and andrea in IRC.

=== MapServer ===

* Windows 64bit Server
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)

* Linux
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (labelled contours, larger view)

=== GeoServer ===

* [http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (Barcelona extents)
*[http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles GeoServer 2.1.x] (with labelled contours)
* [http://12.189.158.78:8080/geoserver/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (labelled contours, larger view)
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeoServer 2.1.x]

=== Mapnik ===
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik2 / paleoserver] (Barcelona extents)
* [http://12.189.158.78:8091/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik0.7.2 / mod_mapnik_wms] (Barcelona extents)
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles Mapnik2 / paleoserver] (larger view - no labels yet)

=== Constellation ===
* [http://12.189.158.78:8280/constellation/WS/wms?SRS=EPSG%3A4326&LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (Barcelona extents)
* [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (labelled contours, larger view)

=== Oracle MapViewer ===
*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (Barcelona Extents)

*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (with labeled contours)

=== Cadcorp GeognoSIS ===
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Shapefiles, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 PostGIS, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 PostGIS, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, whole area, GeognoSIS 7.0]

'''N.B.''' The current GeognoSIS Label Theme "label along lines" option does not work well with this data. We hope to have an alternative option ready in time for the tests

=== Scale dependent rules checking requests ===

These requests show the behavior of scale dependent styling around the critical scales.
The geographic area requested is always the same:

<tt>BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102</tt>

The image size changes instead to alter the scale:

* roughly 1:29k: <tt>WIDTH=824&HEIGHT=472</tt>
* roughly 1:31k: <tt>WIDTH=771&HEIGHT=442</tt>
* roughly 1:49k: <tt>WIDTH=488&HEIGHT=280</tt>
* roughly 1.51k: <tt>WIDTH=469&HEIGHT=269</tt>

The table allows to compare the various servers output. The GeoServer links use a vendor option to get the actual scale, as computed by the server, displayed at the bottom/right of the map.

{| border="1"
|-
| '''Server'''
| '''1:29k'''
| '''1:31k'''
| '''1:49k'''
| '''1:51k'''
|-
| Constellation
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| GeoServer
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&format_options=layout:scale 1:29k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&format_options=layout:scale 1:31k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&format_options=layout:scale 1:49k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&format_options=layout:scale 1:51k]
|-
| Mapnik
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapServer
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapViewer
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| GeognoSIS
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|}

== External Related Links ==

*[[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
*[http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-16T15:33:13Z

Wiki-Mpd: Added GeognoSIS URLs

== Basic Premise ==

Following up on [[Benchmarking 2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Development Team Leader'''
| '''IRC nick'''
| '''Server status'''
|-
| Cadcorp GeognoSIS
| [mailto:martin.daly--@--cadcorp.com Martin Daly]
| mpdaly
| Production release of server installed with preliminary configuration for SHP, TIF and ECW
|-
| Constellation-SDI
| [[User:Cedricbr|Cédric Briançon]]
| cedricbr
| Running on port 8280. Installation done for Shapefiles.
|-
| Erdas Apollo
| [mailto:dimitri.monie@erdas.com Dimitri Monie begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting begin_of_the_skype_highlighting end_of_the_skype_highlighting]
| dmonie
| Install and configuration on-going on windows server, ecw WMS configuration done, shape and DB WMS configuration on going. Port 80, port 8080
|-
| GeoServer
| Andrea Aime
| aaime
| Normally running on port 8080 on the Linux server. Shapefiles and TIFF mosaic have been configured.
|-
| MapGuide Open Source
| Zac Spitzer
| zacspitzer
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| springmeyer
| Running on port 8090 (paleoserver) 8091 (mod_mapnik_wms). Shapefiles styles ready, postgis and raster still to come.
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| jmckenna
| linux_wms_bm serving WMS requests for shps, on port 8081. windows wms_bm serving requests for shps, on port 8081.
|-
| Oracle MapViewer
| [[User:msmitherdc|Michael Smith]]
| msmith_
| Set up on port 8088
|-
| QGIS mapserver
| Marco Hugentobler
| marco___
|}

== Timeline ==

{| border="1"
|-
| January 1st, 2010
| begin contacting all mapping servers
|-
| June 1st, 2010
| commitment due by all interested mapping servers
|-
| June 2nd, 2010
| exercise begins (and weekly meetings)
|-
| August 1st, 2010
| final testing begins
|-
| August 23rd, 2010
| no further testing
|-
| September 6-9, 2010
| present results at FOSS4G2010
|}

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version.
# Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
# Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
# Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
# WMS output formats to be used will be png8 and png24 where possible

== Documenting Server Details and Differences ==

It is the responsibility of each team to document their setup with regard to data.

Please keep your notes in your servers directory in svn: http://svn.osgeo.org/osgeo/foss4g/benchmarking/

One particular thing that will differ is each team's use of spatial indexes on shapefiles.

Teams are '''not''' allowed to modify the .shp, .shx, .dbf or .prj files of the vector data (for the baseline benchmark). They can, however, create auxiliary files for spatial indexes and the like.

{| border="1"
| '''Mapping Server''' || '''ext''' || '''type''' || '''command to create'''
|-
| MapServer || .qix || quadtree || shptree <shpfile> ([[shptree notes]])
|-
| GeoServer || .qix || quadtree || Using the ones generated for MapServer atm
|-
| Mapnik || .index || quadtree || shapeindex <shpfile> ([[Benchmarking_2010/Mapnik_notes#shpindex_depth]])
|-
|Erdas || .rtr || RTree || RTreeBuilder basedir shapename max min ([[RTreeBuilder notes]])
|-
|Cadcorp || *.shp.idx || RTree || GUI command in desktop SIS
|-
| Constellation || .qix || quadtree || Using the ones generated by MapServer
|}

* Note: in your server setup documentation be sure to record the EXACT command used to create the indexes, with appropriate options passed (e.g depth)

== Datasets ==

A set of data published by the spanish National Mapping Agency will be used. This data is free for non-commercial use, so it's perfectly OK to use it in the benchmark.

Vector data will be a topographic map, composed of several shapefiles (one shapefile per theme). Raster data will be 50cm/px aerial photography.

We still do not have the definitive datasets, due to their size and the processing time needed to put them together. You can get some sample data by reading [[Benchmarking 2010/How to get some sample data]].

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Hardware ==

Machine A (windows server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Windows Server 64bit

Machine B (linux server)
* System Type: Dell PowerEdge R410
* Ship Date: 7/7/2010
* Processor: Intel® Xeon® E5630 2.53Ghz, 12M Cache,Turbo, HT, 1066MHz Max Mem
* 8GB Memory (4x2GB), 1333MHz Dual Ranked RDIMMs for 1Processor, Optimized
* 2TB 7.2K RPM SATA
* OS: Centos 5.5 x86-64

Machine C (DB Host)
* System Type: Gateway E6610D
* Processor, Intel Core2 Duo - E6750 2.66 Ghz
* 4Gb Ram
* 250Gb Hard Drive 7200 rpm
* OS: Centos 5.5 x86-64

Machine D (testing server (JMeter))
* System Type: Dell Precision Workstation 390
* Ship Date: 9/7/2006
* Processor, 6300, 1.86, 2M, Core Duo-conroe, Burn 2
* 2Gb RAM
* 160 Gb Hard drive 7200 rpm
* Service Tag: 5Q5LQB1
* OS: Centos 5.5 i386

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed August 11th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=11&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subsets?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
*** Status report from each team
*** Testing deadline: shall we extend it?
*** Can all server provide links to the sample requests and allow to build more? (style comparison, are we all doing the same work?)
*** Vector data
**** Changes in rendering (missing values for some attributes)
**** Gaps: what to do with the 9 missing sheets?

=== Previous IRC meeting ===

* Wed August 4th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=08&day=4&hour=15&min=0&sec=0 15:00:00 UTC]
** Provisional Agenda:
*** Machine status / issues
*** Data status / issues
**** Characters mis-encoded in shapefiles (e.g. catalan in labels): fixable? impact on PostGIS database?
**** Raster conversion to GeoTIFF (torrent of results?)
*** Styles status / issues
*** Jmeter testing
**** what output formats will be requested?
**** what projections will be requested? (Platte-carré is not conformal)
**** what layer combinations will be requested? (All vector layers or various mixed subset?)
**** what envelopes will be issued? (Frank's 2009 query generation script forces a hard cutoff at the largest scale)
**** what output metrics will be created?
**** how will the jmx file be designed? (need one .csv file per thread so all requests are different)
** Summary: [[Benchmarking 2010 IRC meeting logs Aug 4th]]

* Wed July 28th
(no info)

* Wed July 21st @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=21&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** update on data styling
*** checkin on server installs / library dependencies

* Wed July 14th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=07&day=14&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on data processing for all of Spain
*** update on server setup
*** data styling discussions
**** scales to use for layers
**** missing full road network
**** layer to use for labels along lines
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-07-14.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Dimitri Monie (dmonie)
**** Dane Springmeyer (springmeyer)
*** Discussion:
**** data:
***** full dataset of Spain, vector and raster, has been processed by IvanSanchez, and he is checking it and will create a torrent for transfer, by the end of the week
***** because of shapefile size limit some layers may have to be split (contour-1.shp, contour-2.shp)
**** servers:
***** msmith has received both server machines
***** T1 router is being installed, it will be a dedicated line for this project
***** he is adding 2 TB drives to each
**** layer styling:
***** we notice that the Spain dataset we are using (BCN25) does not include street files (zoomed in road network)
***** we agreed to use contours for curved label tests instead
***** jmckenna will test the layer scales provided by Marco (QGIS mapserver)
***** aaime pointed out problem with the posted SLDs, in terms of the label property (jmckenna to look into it)
***** aaime pointed out that we should make sure that all servers compute the scales the same way
****** OGC recommends 90 DPI (GeoServer, Mapnik use the recommended 90...MapServer defaults to 72, but should be configurable)
****** all teams should look into how their engine handles this and report to group
**** other:
***** IvanSanchez reported that a Spanish custom-made WMS server may join the exercise (they access raster files only)
***** dmonie pointed out that Erdas has been quiet lately because one of the team leaders is on holiday

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 30th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=30&hour=15&min=0&sec=0 15:00:00 UTC]
** Summary: [http://logs.qgis.org/foss4g/%23foss4g.2010-06-30.log log]

* Wed June 23rd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=23&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on proposed layer styling
*** update on available server specs
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-23.log log])

* Wed June 16th @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=16&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** update on server infrastructure
*** update on OSGeo-es dataset ([[Benchmarking 2010/How to get some sample data|Sample data download instructions]])
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-16.log log])
*** Attendance:
**** Jeff McKenna (jmckenna)
**** Andrea Aime (aaime)
**** Iván Sánchez (IvanSanchez)
**** Michael Smith (msmith_)
**** Cédric Briançon (cedricbr)
*** Discussion:
**** "base" run will use only shapefiles, "best" run will be any format (raster/vector)
**** teams can optionally test Oracle, PostGIS, Ingres connections during the "best" run
**** layers tested will be: labelled roads (with labels that follow the lines), thematic polygon map, and a test with labelled roads + thematic polygon + point layer in a single request (3 separate runs)
***** msmith and jmckenna to examine styling for one sheet (Barcelona, sheet# 420)
**** each base and best run will test data in raw projection (ETRS89 + UTM28/29/30/31) and then reprojected on-the-fly to the Google Mercator projection
**** threads (number of concurrent requests) will be: 1-2-4-8-16-32-64
**** US Army Corps is proposing to provide 4 desktop machines. 2 identical for testing ( win/lin ) and 1 for jmeter and 1 for databases
***** msmith will find out availability of 2-cpu machines this week

* Wed June 2nd @ [http://timeanddate.com/worldclock/fixedtime.html?year=2010&month=06&day=02&hour=15&min=0&sec=0 15:00:00 UTC]
** Agenda:
*** team representative introductions
*** Establish this year's 'Rules of Engagement' ([http://wiki.osgeo.org/wiki/Benchmarking_2009#Rules_of_Engagement previous RoE])
*** Discuss dataset to be used (including how to share the large dataset, for running local tests)
*** Discuss server infrastructure
*** Discuss possible changes to the test script parameters
*** Set next meeting date
** Summary: ([http://logs.qgis.org/foss4g/%23foss4g.2010-06-02.log log])
*** Attendance:
**** Jeff McKenna, MapServer (jmckenna)
**** Cédric Briançon - Constellation (cedricbr)
**** Michael Smith - Oracle Mapviewer and Oracle connections in MapServer (msmith_)
**** Zac Spitzer - MapGuide (zacspitzer)
**** Martin Daly - Cadcorp (mpdaly)
**** Anne-Sophie Collignon - ERDAS (ascollignon)
**** Marco Hugentobler - QGIS mapserver (marco___)
**** Andrea Aime - GeoServer (aaime)
**** Dimitri Monie - ERDAS (dmonie)
**** Frank Warmerdam - MapServer, possibly Ingres as well (FrankW)
**** Iván Sánchez - provide geodata from Spanish sources and help out Mapnik (IvanSanchez)
**** Dane Springmeyer - Mapnik (springmeyer)
**** Johann Sorel - GeotoolKit / Constellation (Eclesia)
**** Adrian Custer - Constellation (acuster)
**** Pirmin Kalberer - QGIS mapserver (pirmin_k)
**** Trevor Wekel - MapGuide (trevorw)
**** Daniel Morissette - MapServer (danmo)
*** Discussion:
**** All agreed to this year's "Rules of Engagement"
***** All parties must contribute any changes that they make to their software for this exercise, back to their community. Note that the changes don't have to be contributed before the conference, just in a reasonable period of time.
***** Comparisons will be made of the best available version of the software, be it a formal release or a development version.
***** Two tests will be run: one 'baseline' test with the data in its raw format (with spatial indexes), and another 'best effort' test where 'the sky is the limit' for what changes you want to make to the data (change format, generalize, etc)
***** Teams must document all steps they did to manipulate the data/server for both the 'baseline' and 'best effort' tests. If a team does not document the steps on this wiki then that team's test results will not be used.
***** Data formats to be used will be shapefiles for vectors, and uncompressed geotiffs for rasters.
***** WMS output formats to be used will be png8 and png24 where possible
**** Dataset
***** Iván Sánchez Ortega from OSGeo-es is willing to provide local Spain datasets
***** data is for non-commercial use only
***** 10 GB of shapefiles
***** 100 GB of uncompressed rasters
**** Server infrastructure
***** possible hosts are msmith (US Army Corps) or Skygone (http://www.skygoneinc.com/)
***** need two identical servers, one running Unix and another Windows, plus a client machine
***** specs could be: quad core, 8GB of memory, 1TB hard drive
***** need fast pipe to server, as large datasets will be transferred
**** Next meeting date
***** all agreed to use same day and time next week, but limit the meeting length to one hour

== Sample Dataset Styling ==

* download [http://download.osgeo.org/benchmarking/barcelona-sample-dataset.zip sample dataset]
* layer [http://wiki.osgeo.org/wiki/Benchmarking_2010/How_to_get_some_sample_data#Vector_data_issues styling rules]
{| border="1"
|-
| '''Mapping Server'''
| '''GetMap'''
| '''SLD'''
|-
| MapServer
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 all layers]
| --
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=settlement&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 settlement]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/settlement-sld.xml settlement-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=building&SRS=EPSG:25831&BBOX=426633,4579926,429740,4582528&FORMAT=image/png&WIDTH=950&HEIGHT=600 building]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/building-sld.xml building-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=industry&SRS=EPSG:25831&BBOX=424932,4575961,427328,4577812&FORMAT=image/png&WIDTH=950&HEIGHT=600 industry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/industry-sld.xml industry-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=contour_lines&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 contour lines]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/contour-sld.xml contour-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=motorway&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 motorway]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/motorway-sld.xml motorway-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=road&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 road]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/road-sld.xml road-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=ramp&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 ramp]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/ramp-sld.xml ramp-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=track&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 track]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/track-sld.xml track-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_geometry&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels with geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-geom-sld.xml point-geom-sld.xml]
|-
|
| [http://labs.gatewaygeomatics.com/cgi-bin/benchmark2010?SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=point_nogeom&SRS=EPSG:25831&BBOX=400637.456357,4576080.998882,456535.105266,4594947.942275&FORMAT=image/png&WIDTH=950&HEIGHT=600 point labels without geometry]
| [http://labs.gatewaygeomatics.com/benchmarking/sld/point-nogeom-sld.xml point-nogeom-sld.xml]
|}

== Benchmarking setup ==

Tests are going to be run using JMeter, using a progression of 1, 2, 4, 8, 16, 32, and 64 threads, each thread group doing 100, 200, 200, 400, 400, 800 requests respectively, for a total of 2200 requests.
The requests bounds and sizes are going to be picked up from a csv file generated by a random generator script.

The scripts to be run for the baseline tests, and repeated for the best effort one, are:
* raster data, JPEG output, EPSG:25831, scales between 1:1M and 1:10k
* raster data, JPEG output, EPSG:3857, scales between 1:1M and 1:10k
* vector data, PNG24 output, EPSG:4326, scales between 1:300k and 1:1k
* vector data, PNG24 output, EPSG:3875, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:25831, scales between 1:300k and 1:1k
* raster + vector, JPEG output, EPSG:3785, scales between 1:300k and 1:1k

The areas to be tested are:
* 6 E, 38 N, -1 E, 43 N for vector tests
* 372360,4557880,484490,4631460 for raster tests

The raster layer in the baseline test is going to be the mosaic of GeoTIFF files Frank prepared out of the ECW files on the Linux server.

The vector data set is composed of a stack of individual layers to be stacked in the following order (bottom to top):
* settlements
* contour-0
* contour-1
* contour-2
* contour-3
* contour-4
* contour-5
* contour-6
* contour-7
* building
* industry
* track
* ramp
* road
* motorway
* point-labels-for-geometry
* point-labels-no-geometry

The best effort will use its own custom set of layers (e.g., it's likely contours will be merged into one, and so on)

== Live Benchmark WMS GetMap Requests ==

Note: sample bbox's fixed on Aug 12 to fit the proper aspect ratio of an 800/600 image request (springmeyer). Previous bbox for barcelona request was '1.8,41,2.4,42', new one used below is: '1.43333333333,41.0,2.76666666667,42.0'. Previous request for focused view was '2.1076723642349,41.407828508849,2.1178733654021,41.414271246429', new one is '2.10767236423,41.4072245022,2.1178733654,41.4148752531'. This was done to sidestep a deficiency in mapnik for affine transformation of geometries according to the WMS spec when the image w/h aspect does not match the bbox aspect, but was agreed upon as useful for these demo links by jeff and andrea in IRC.

=== MapServer ===

* Windows 64bit Server
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.75:8081/cgi-bin/mapserv.exe?MAP=C:/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)

* Linux
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (Barcelona extents)
** [http://12.189.158.78:8081/cgi-bin/mapserv?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (with labelled contours)
** [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png Shapefiles, MapServer 5.7-dev-label-fix] (labelled contours, larger view)

=== GeoServer ===

* [http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (Barcelona extents)
*[http://12.189.158.78:8080/geoserver/wms?SRS=EPSG%3A4326&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles GeoServer 2.1.x] (with labelled contours)
* [http://12.189.158.78:8080/geoserver/wms?EXCEPTIONS=application%2Fvnd.ogc.se_inimage&LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles GeoServer 2.1.x] (labelled contours, larger view)
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeoServer 2.1.x]
* [http://12.189.158.78:8080/geoserver/wms?LAYERS=ras%3Amosaic_tif&STYLES=&FORMAT=image%2Fjpeg&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeoServer 2.1.x]

=== Mapnik ===
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik2 / paleoserver] (Barcelona extents)
* [http://12.189.158.78:8091/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Mapnik0.7.2 / mod_mapnik_wms] (Barcelona extents)
* [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles Mapnik2 / paleoserver] (larger view - no labels yet)

=== Constellation ===
* [http://12.189.158.78:8280/constellation/WS/wms?SRS=EPSG%3A4326&LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=1.43333333333,41.0,2.76666666667,42.0&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (Barcelona extents)
* [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=2.10767236423,41.4072245022,2.1178733654,41.41487525316&WIDTH=800&HEIGHT=600 Shapefiles Constellation] (labelled contours, larger view)

=== Oracle MapViewer ===
*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (Barcelona Extents)

*[http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600&srs=EPSG:4326&FORMAT=image/png&aa=TRUE Oracle DB / MapViewer] (with labeled contours)

=== Cadcorp GeognoSIS ===
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 Shapefiles, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 Shapefiles, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=1.43333333333,41.0,2.76666666667,42.0&width=800&height=600 PostGIS, GeognoSIS 7.0] (Barcelona Extents)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-postgis&BGCOLOR=0xFFFFFF&BBOX=2.10767236423,41.4072245022,2.1178733654,41.4148752531&width=800&height=600 PostGIS, GeognoSIS 7.0] (with labelled contours - see note)
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 TIFF mosaic, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 TIFF mosaic, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-tif&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 TIFF mosaic, whole area, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=449647.84729003,4594968.4096903,450085.85510253,4595260.9852213&WIDTH=512&HEIGHT=342 ECW, zoomed in, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=441127.2265625,4590970.3051752,455143.4765625,4600332.7221674&WIDTH=512&HEIGHT=342 ECW, mid zoom, GeognoSIS 7.0]
* [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=bcn-ecw&BGCOLOR=0xFFFFFF&SRS=EPSG%3A25831&BBOX=316295,4519985.6640625,540555,4669784.3359375&WIDTH=512&HEIGHT=342 ECW, whole area, GeognoSIS 7.0]

'''N.B.''' The current GeognoSIS Label Theme "label along lines" option does not work well with this data. We hope to have an alternative option ready in time for the tests

=== Scale dependent rules checking requests ===

These requests show the behavior of scale dependent styling around the critical scales.
The geographic area requested is always the same:

<tt>BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102</tt>

The image size changes instead to alter the scale:

* roughly 1:29k: <tt>WIDTH=824&HEIGHT=472</tt>
* roughly 1:31k: <tt>WIDTH=771&HEIGHT=442</tt>
* roughly 1:49k: <tt>WIDTH=488&HEIGHT=280</tt>
* roughly 1.51k: <tt>WIDTH=469&HEIGHT=269</tt>

The table allows to compare the various servers output. The GeoServer links use a vendor option to get the actual scale, as computed by the server, displayed at the bottom/right of the map.

{| border="1"
|-
| '''Server'''
| '''1:29k'''
| '''1:31k'''
| '''1:49k'''
| '''1:51k'''
|-
| Constellation
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8280/constellation/WS/wms?LAYERS=shp:settlement,shp:contour-0,shp:contour-1,shp:contour-2,shp:contour-3,shp:contour-4,shp:contour-5,shp:contour-6,shp:contour-7,shp:building,shp:industry,shp:track,shp:ramp,shp:road,shp:motorway,shp:point-labels-for-geometry,shp:point-labels-no-geometry&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| GeoServer
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472&format_options=layout:scale 1:29k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442&format_options=layout:scale 1:31k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280&format_options=layout:scale 1:49k]
| [http://12.189.158.78:8080/geoserver/wms?LAYERS=shp_alllayers_label&STYLES=&FORMAT=image%2Fpng&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&SRS=EPSG%3A4326&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269&format_options=layout:scale 1:51k]
|-
| Mapnik
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8090/?SRS=EPSG%3A4326&LAYERS=__all__&STYLES=&FORMAT=image/png&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&EXCEPTIONS=application%2Fvnd.ogc.se_inimage&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapServer
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8081/cgi-bin/mapserv_label?MAP=/benchmarking/mapserver/spain-shapefiles.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=GetMap&LAYERS=vector_benchmarking_2010&STYLES=&srs=EPSG:4326&FORMAT=image/png&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
| MapViewer
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.78:8088/mapviewer/wms?REQUEST=GetMap&BASEMAP=spain_oracle&LAYERS=&SERVICE=WMS&VERSION=1.1.1&STYLES=&srs=EPSG:4326&FORMAT=image/png&aa=TRUE&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|-
| GeognoSIS
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=824&HEIGHT=472 1:29k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=771&HEIGHT=442 1:31k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=488&HEIGHT=280 1:49k]
| [http://12.189.158.75/cadcorp/geognosis-70.exe?VERSION=1.1.1&SERVICE=WMS&REQUEST=GetMap&STYLES=&FORMAT=image%2fpng%3bPhotometricInterpretation=RGB&SRS=EPSG:4326&LAYERS=spain-shp&BGCOLOR=0xFFFFFF&BBOX=0.58671954345704,40.73922127533,0.64684991455078,40.773670967102&WIDTH=469&HEIGHT=269 1:51k]
|-
|}

== External Related Links ==

*[[FOSS4G Benchmark|FOSS4G WMS Benchmark]]
*[http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-08-12T07:38:26Z

Wiki-Mpd: /* Participants */

Benchmarking 2010

2010-08-04T13:06:07Z

Wiki-Mpd: /* Participants */

Benchmarking 2010

2010-08-04T13:04:37Z

Wiki-Mpd: /* Documenting Server Details and Differences */

Benchmarking 2010

2010-04-21T06:53:42Z

Wiki-Mpd: Added Cadcorp to Potential Participants

Tile Map Service Specification

2006-11-17T13:22:45Z

Wiki-Mpd:

This document is the work of a loose community of participants interested in client/server mapping solutions that use multi-resolution image pyramids. It is meant to be used as a baseline for the implementation of client/server mapping software. It is not an "official standard" nor is it endorsed by OSGeo as an official project or work product of the Foundation.

This document is currently in ''active revision'', edits are accepted from any user at any time. Please join the mailing list http://lists.eogeo.org/mailman/listinfo/tiling to discuss your ideas before applying them to the draft. The document will first move from ''active revision'' to ''final review'' and then finally to ''numbered final version'', at which point it will be locked.

==Introduction==

===Document Scope===

A Tile Map Service provides access to cartographic maps of geo-referenced data, not direct access to the data itself. This document standardizes the way in which map tiles are requested by clients, and the ways that servers describe their holdings.

===Document Form===

This document will proceed from a description of general resources provided by the server to particular resources (such as map tiles) providing examples of access URLs and return values at each stage.

==Specification==

The Tiled Web Service provides access to resources, in particular, to rendered cartographic tiles at fixed scales. Access to these resources is provided via a "REST" interface, starting with a root resource describing available layers, then map resources with a set of scales, then scales holding sets of tiles.

Each resource contains the descriptive information and links to further resources. Note that while the URLs used to access resources may appear to have internal meaning (the resource for version 1.0.0 of the service has "1.0.0" in it's URL) such structure is ''not required'' of them.

The only requirement is that the resource be referenced by a URL (<nowiki>http://tms.osgeo.org/1.0.0/ could be http://tms.osgeo.org/onepointzeropointzero or http://tms.osgeo.org/flipper.xml as long as the value appeared in the href of the <TileMapService> element</nowiki>).

The value of an "href" '''must''' be an absolute URL (starting with "http://"). For example: href="<nowiki>http://www.service.org/subdirectory/tilemap.xml</nowiki>"

===Root Resource===

The root resource describes the available versions of the <TileMapService> (and possibly other services as well).

Request:

<nowiki>http://tms.osgeo.org/</nowiki>

Response (Content-type: text/xml):

<nowiki><?xml version="1.0" encoding="UTF-8" ?>
<Services>
<TileMapService version="1.0.0" href="http://tms.osgeo.org/1.0.0/" />
<TileMapService version="1.1.0" href="http://tms.osgeo.org/1.1.0/" />
<FancyFeatureService version="0.9" href="http://ffs.osgeo.org/0.9/" />
</Services></nowiki>

Request:

<nowiki>http://www.osgeo.org/services/root.xml</nowiki>

Response:

<nowiki><?xml version="1.0" ?>
<Services>
<TileMapService version="1.0.0" href="http://www.osgeo.org/services/tilemapservice.xml" />
</Services></nowiki>

===TileMapService Resource===

The <TileMapService> resource provides description metadata about the service and lists the available <TileMaps>.

Optional elements in the resource are called out below using the pipe character. All other elements are mandatory.

Request:

<nowiki>http://tms.osgeo.org/1.0.0/</nowiki>

Response (Content-type: text/xml):

<nowiki><?xml version="1.0" encoding="UTF-8" ?>
<TileMapService version="1.0.0">
<Title>Example Tile Map Service</Title>
<Abstract>This is a longer description of the example tiling map service.</Abstract>
| <KeywordList>example tile service</KeywordList>
| <ContactInformation>
| <ContactPersonPrimary>
| <ContactPerson>Paul Ramsey</ContactPerson>
| <ContactOrganization>Refractions Research</ContactOrganization>
| </ContactPersonPrimary>
| <ContactPosition>Manager</ContactPosition>
| <ContactAddress>
| <AddressType>postal</AddressType>
| <Address>300 - 1207 Douglas Street</Address>
| <City>Victoria</City>
| <StateOrProvince>British Columbia</StateOrProvince>
| <PostCode>V8W2E7</PostCode>
| <Country>Canada</Country>
| </ContactAddress>
| <ContactVoiceTelephone>12503833022</ContactVoiceTelephone>
| <ContactFacsimileTelephone>12503832140</ContactFacsimileTelephone>
| <ContactElectronicMailAddress>pramsey@refractions.net</ContactElectronicMailAddress>
| </ContactInformation>

<TileMaps>

<TileMap>
<Title>VMAP0 World Map</Title>
<Abstract>A map of the world built from the NGA VMAP0 vector data set.</Abstract>
| <KeywordList></KeywordList>
| <Metadata type="TC211" mime-type="text/xml" href="http://www.org" />
| <Attribution>
| <Title>National Geospatial Intelligence Agency</Title>
| <Logo width="10" height="10" href="http://nga.mil/logo.gif" mime-type="image/gif" />
| </Attribution>
| <WebMapServer href="http://wms.org" />
| <Face>0</Face>
| <Profile type="global-geodetic" href="http://tms.osgeo.org/1.0.0/vmap0/" />
<SRS>EPSG:4326</SRS>
<BoundingBox minx="-180" miny="-90" maxx="180" maxy="90" />
<Origin x="-180" y="-180" />
<TileFormat width="256" height="256" mime-type="image/jpeg" extension="jpg" />
<TileSets>
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/0" units-per-pixel="0.703125" order="0" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/1" units-per-pixel="0.3515625" order="1" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/2" units-per-pixel="0.17578125" order="2" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/3" units-per-pixel="0.08789063" order="3" />
</TileSets>
</TileMap>

<TileMap>
<Title>British Columbia Landsat Imagery (2000)</Title>
<Abstract>Landsat data collected in the year 2000 over British Columbia.</Abstract>
| <KeywordList></KeywordList>
| <Metadata type="TC211" mime-type="text/xml" href="http://www.org" />
| <Attribution>
| <Title>Government of British Columbia</Title>
| <Logo width="10" height="10" href="http://gov.bc.ca/logo.png" mime-type="image/png" />
| </Attribution>
| <WebMapServer href="http://wms.gov.bc.ca" />
<Profile type="local" href="http://tms.osgeo.org/1.0.0/landsat2000/" />
<SRS>EPSG:3005</SRS>
<BoundingBox minx="100000" miny="100000" maxx="1800000" maxy="1800000" />
<Origin x="100000" y="100000" />
<TileFormat width="256" height="256" mime-type="image/png" extension="png" />
<TileSets>
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/2048" units-per-pixel="2048" order="0" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/1024" units-per-pixel="1024" order="1" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/512" units-per-pixel="512" order="2" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/256" units-per-pixel="256" order="3" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/128" units-per-pixel="128" order="4" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/64" units-per-pixel="64" order="5" />
</TileSets>
</TileMap>

</TileMaps>
</TileMapService></nowiki>

Request:

<nowiki>http://www.osgeo.org/services/tilemapservice.xml</nowiki>

Response (Content-type: text/xml):

<nowiki><?xml version="1.0" encoding="UTF-8" ?>
<TileMapService version="1.0.0">
<Title>Example Static Tile Map Service</Title>
<Abstract>This is a longer description of the static tiling map service.</Abstract>
| <KeywordList>example tile service static</KeywordList>
| <ContactInformation>
| <ContactPersonPrimary>
| <ContactPerson>Paul Ramsey</ContactPerson>
| <ContactOrganization>Refractions Research</ContactOrganization>
| </ContactPersonPrimary>
| <ContactPosition>Manager</ContactPosition>
| <ContactAddress>
| <AddressType>postal</AddressType>
| <Address>300 - 1207 Douglas Street</Address>
| <City>Victoria</City>
| <StateOrProvince>British Columbia</StateOrProvince>
| <PostCode>V8W2E7</PostCode>
| <Country>Canada</Country>
| </ContactAddress>
| <ContactVoiceTelephone>12503833022</ContactVoiceTelephone>
| <ContactFacsimileTelephone>12503832140</ContactFacsimileTelephone>
| <ContactElectronicMailAddress>pramsey@refractions.net</ContactElectronicMailAddress>
| </ContactInformation>
<TileMaps>
<TileMap>
<Title>Vancouver Island Base Map</Title>
<Abstract>A map of the Vancouver Island built from British Columbia planimetric mapping
data and digital elevation hill shading.</Abstract>
| <KeywordList></KeywordList>
| <Metadata type="TC211" mime-type="text/xml" href="http://www.org" />
| <Attribution>
| <Title>Goverment of British Columbia</Title>
| <Logo width="10" height="10" href="http://www.gov.bc.ca/logo.gif" mime-type="image/gif" />
| </Attribution>
| <WebMapServer href="http://openmaps.gov.bc.ca" />
<SRS>EPSG:26910</SRS>
| <Face>0</Face>
<BoundingBox minx="500000" miny="4800000" maxx="700000" maxy="5500000" />
<Origin x="500000" y="4800000" />
<TileFormat width="256" height="256" mime-type="image/png" extension="png" />
<TileSets>
<TileSet href="http://www.osgeo.org/services/basemap/L1" units-per-pixel="6400" order="0" />
<TileSet href="http://www.osgeo.org/services/basemap/L2" units-per-pixel="1600" order="1" />
<TileSet href="http://www.osgeo.org/services/basemap/L3" units-per-pixel="400" order="2" />
<TileSet href="http://www.osgeo.org/services/basemap/L4" units-per-pixel="100" order="3" />
<TileSet href="http://www.osgeo.org/services/basemap/L5" units-per-pixel="25" order="4" />
</TileSets>
</TileMap>
</TileMaps>
</TileMapService></nowiki>

A <TileMap> is a (usually) cartographically complete map representation. Sometimes <TileMap>s are built to be used in conjunction, as a set of stacked layers, but they are generally visually complete on their own.

<TileMap>s are composed of a set of scale-appropriate cartographic renderings, each divided up into regularly spaced image tiles, called <TileSet>s. Small-scale (eg, 1:10000000) tile sets may only contain a handful of tiles. Large-scale tile sets (eg, 1:10000) may contain millions of tiles.

At a particular scale, and in a particular cartographic projection, a <TileMap> is represented by a <TileSet>, a coverage of regularly sized and spaced images that taken together form a complete visual representation of the entire area of coverage of the <TileMap>.

Each <TileMap> supports one <SRS> and one image format. To support more than one SRS or image format, define extra <TileMaps> in your <TileMapService> for each combination you want.

<TileMap>s have both a <BoundingBox> and an <Origin>. The <BoundingBox> is the extent of the data of interest -- it might be used by a client to set an initial spatial extent. The <Origin> is the lower-left corner of the 0/0 tile, and the upper right corner of tile -1/-1 (if you choose to configure your service so that negative tiles are required). The <Origin> may be outside of the visual region of interest (the <BoundingBox>), for reasons of implementation convenience.

<TileMap>s may participate as a <Face> of a larger complex of <TileMap>s. The OSGPlanet and GeoFusion clients both use separate polar faces in conjuction with equatorial faces (an "earth cube") to create a single world view from multiple tile maps. See [[#Using Faces]].

<TileMap>s may implement one of three <Profile>s, two global profiles in common global projections, or a local profile in an arbitrary projection. All <Profile>s restrict the service to a fixed set of scales, to allow services from different services to be more easily overlaid.

====Profiles====

Using this server specification will ensure that clients can easily consume your tiled map data. However, it will not guarantee that clients can efficiently overlay your data with data from other tile map servers. In order to maximize the interoperability of your tile map with other tile maps, you must implement the a "profile" for your tile map.

Services implementing a "profile" should be flagged as such by using the <Profile> element in the <TileMap>.

If <Profile> is not present, then the <TileMap> is not implementing a profile.

=====global-geodetic=====

<nowiki><Profile type="global-geodetic" href="http://tms.osgeo.org/1.0.0/vmap0/" /></nowiki>

If the <Profile> type is set to "global-geodetic", the <TileMap> must meet the following requirements:

* Must use <SRS>EPSG:4326</SRS>
* Must provide <TileSet>s with units-per-pixel meeting the following formula for any integral value of "n" greater than or equal to 0: units-per-pixel = 0.703125 / 2^n
** This scaling allows an initial zoom level that consists of 2 256x256 pixel tiles covering the whole earth, with an <Origin> of (-180,-90). Other combinations of tile size and <Origin> are also possible at this scale.
* Must provide <TileSet> sub-directories below the <Profile> href value, using the value of "n" appropriate for that <TileSet> as the sub-directory name.

<nowiki><TileSet href="http://tms.osgeo.org/1.0.0/vmap0/0" units-per-pixel="0.703125" order="0" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/1" units-per-pixel="0.3515625" order="1" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/2" units-per-pixel="0.17578125" order="2" />
<TileSet href="http://tms.osgeo.org/1.0.0/vmap0/3" units-per-pixel="0.08789063" order="3" /></nowiki>

=====global-mercator=====

<nowiki><Profile type="global-mercator" href="http://tms.osgeo.org/1.0.0/vmap0-merc/" /></nowiki>

If the <Profile> type is set to "global-mercator", the <TileMap> must meet the following requirements:

* Must use <SRS>OSGEO:41001</SRS>
* Must provide <TileSet>s with units-per-pixel meeting the following formula for any integral value of "n" greater than or equal to 0: units-per-pixel = 78271.516 / 2^n
** This scaling allows an initial zoom level that consists of four 256x256 pixel tiles covering the whole earth, with an <Origin> of (-20037508.34, -20037508.34). Other combinations of tile size and <Origin> are also possible at this scale.
* Must provide <TileSet> sub-directories below the <Profile> href value, using the value of "n" appropriate for that <TileSet> as the sub-directory name.

=====local=====

Unlike the global profiles, the "local" profile is built from the bottom up, starting with a smallest possible scale of one unit per pixel and compounding upwards from there. Local profiles can be in any projection, but are at a fixed set of scales.

<nowiki><SRS>EPSG:3005</SRS>
<BoundingBox minx="100000" miny="100000" maxx="1800000" maxy="1800000" />
<Origin x="100000" y="100000" />
<Profile type="local" href="http://tms.osgeo.org/1.0.0/landsat2000/" /></nowiki>

If the <Profile> type is set to "local", the <TileMap> must meet the following requirements:

* May use any coordinate system, and must identify that coordinate system in the <SRS>.
* Must provide <TileSet>s with units-per-pixel meeting the following formula for any integral value of "n" greater than or equal to 0: units-per-pixel = 2^n
* Must provide <TileSet> sub-directories below the <Profile> href value, using the value of "n" appropriate for that <TileSet> as the sub-directory name.

<nowiki><TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/2048" units-per-pixel="2048" order="0" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/1024" units-per-pixel="1024" order="1" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/512" units-per-pixel="512" order="2" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/256" units-per-pixel="256" order="3" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/128" units-per-pixel="128" order="4" />
<TileSet href="http://tms.osgeo.org/1.0.0/landsat2000/64" units-per-pixel="64" order="5" /></nowiki>

===Tile Resources===

The origin of a <TileMap> is defined in the coordinates of the spatial reference system of the <TileMap>. The x-coordinate of the tile numbers increases with the x-coordinate of the spatial reference system, and the y-coordinate of the tile numbers also increases with the y-coordinate of the spatial reference system.

Tiles are addressed under the "href" specified in the <TileSet> appending the "x" tile coordinate as a directory name and using the "y" tile coordinate as the file name, with the file "extension" from the <TileFormat>.

Example:

The tile at the origin of the tile set in the first zoom level of vmap0.
<nowiki>http://tms.osgeo.org/1.0.0/vmap0/levelzero/0/0.jpg</nowiki>

The tile near the middle of the tile set in the third zoom level of vmap0.
<nowiki>http://tms.osgeo.org/1.0.0/vmap0/leveltwo/3/4.jpg</nowiki>

The tile near the middle of the tile set in the fifth zoom level of landsat2000.
<nowiki>http://tms.osgeo.org/1.0.0/landsat2000/1/8500/8500.png</nowiki>

The tile at the origin of the tile set in the first zoom level of basemap.
<nowiki>http://www.osgeo.org/services/basemap/L1/0/0.png</nowiki>

===TileMap Diagram===

[[Image:Tms.png]]

===Error Handling===

When an error occurs in the server, it is important that the client be able easily notice that an error has occurred, and ascertain ''why'' the error occured so the user can be notified if necessary.

The tile map server uses HTTP error codes to relay the general reason for an error condition, and an XML payload to communicate the specific reason for the failure in human readable language.

Only HTTP error codes given in this specification should be used to return errors to the client.

* The client requests a nonexistent resource URL. Return HTTP error code 404 (Not Found)
* The server fails in processing a response for a valid resource URL. Return HTTP error code 500 (Internal Server Error)

Servers are optionally allowed to return content, even when throwning an error code. The following is the XML format for a tile map server error message. If the Content-type of the return on an error is set to text/xml, this format is the required form of the response.

<nowiki><?xml version="1.0" ?>
<TileMapServerError>
<Message>The requested tile is outside the bounding box of the tile map.</Message>
</TileMapServerError></nowiki>

==Implementation Advice==

===Spatial Referencing Systems===

Spatial referencing systems for the tile map service will be defined using the EPSG database as a reference for "well-known" projections, subject to interpretations, given below.

There are two substantial implementation issues with using the EPSG database as a source of truth for spatial reference systems:

* Firstly, the EPSG database has some specific definitions for commonly used geodetic coordinate systems, in particular EPSG:4326 -- geodetic coordinates relative to the WGS84 spheroid. The EPSG definition for 4326 says that the coordinate order is latitude, longitude and that the units are degrees, minutes, seconds. However, common usage of EPSG:4326 in web mapping says that the coordinate order is longitude, latitude and the units are decimal degrees.
* Secondly, the EPSG database does not include every commonly used spatial reference system. There are still many local systems which are not included in the database, though the EPSG does make an effort to include new systems as they are brought to their attention. However, the EPSG does not catalogue commonly used global and large area systems, presumably as a matter of policy. For example, no EPSG identifier is provided for a Mercator projection of the world, or an Albers projection of North America.

The issues will be dealt with by fiat in this specification, matching implementation practice rather than following the database definition:

* For all geodetic coordinate systems in the EPSG database, the tile map service specification will treat the coordinate order as longitude, latitude and the units as decimal degrees.
* Spatial reference systems not defined in the EPSG database may be defined in the tile map service specification itself, using an OSGEO authority string.
** '''OSGEO:41001''' <nowiki>PROJCS["WGS84 / Simple Mercator", GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["WGS_1984",6378137,298.257223563]], PRIMEM["Greenwich",0], UNIT["Decimal_Degree", 0.0174532925199433]], PROJECTION["Mercator_1SP"], PARAMETER["central_meridian",0], PARAMETER["false_easting",0], PARAMETER["false_northing",0], UNIT["Meter",1]]</nowiki>
** '''OSGEO:42310''' <nowiki>PROJCS["WGS84+GRS80 / Mercator", GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["GRS 1980",6378137,298.257222101], TOWGS84[0,0,0]], PRIMEM["Greenwich",0], UNIT["Decimal_Degree",0.0174532925199433]], PROJECTION["Mercator_1SP"], PARAMETER["central_meridian",0], PARAMETER["false_easting",0], PARAMETER["false_northing",0], UNIT["Meter",1]]</nowiki>
** '''OSGEO:42101''' <nowiki>PROJCS["WGS 84 / LCC Canada", GEOGCS["WGS 84", DATUM["WGS_1984", SPHEROID["WGS_1984",6378137,298.257223563]], PRIMEM["Greenwich",0], UNIT["Decimal_Degree",0.0174532925199433]], PROJECTION["Lambert_Conformal_Conic_2SP"], PARAMETER["central_meridian",-95.0], PARAMETER["latitude_of_origin",0], PARAMETER["standard_parallel_1",49.0], PARAMETER["standard_parallel_2",77.0], PARAMETER["false_easting",0.0], PARAMETER["false_northing",-8000000.0], UNIT["Meter",1]]</nowiki>
** '''OSGEO:42304''' <nowiki>PROJCS["NAD83 / NRCan LCC Canada", GEOGCS["NAD83", DATUM["North_American_Datum_1983", SPHEROID["GRS_1980",6378137,298.257222101], TOWGS84[0,0,0]], PRIMEM["Greenwich",0], UNIT["Decimal_Degree",0.0174532925199433]], PROJECTION["Lambert_Conformal_Conic_2SP"], PARAMETER["central_meridian",-95.0], PARAMETER["latitude_of_origin",49.0], PARAMETER["standard_parallel_1",49.0], PARAMETER["standard_parallel_2",77.0], PARAMETER["false_easting",0.0], PARAMETER["false_northing",0.0], UNIT["Meter",1]]</nowiki>

===Maximizing Cacheability===

Tile maps are usually base maps, and usually represent data that changes on a very slow cycle. They are also usually large in volume, comprising potentially millions of different tiles. Given these basic facts, the aggressive use of caching strategies can optimize performance of tile map services.

Caching can happen at multiple layers between the server and the client:

* At the client itself, as the user-agent caches results on the local disk.
* In a shared cache at an intermediate ISP, allowing multiple users of the ISP to pull data from the cache.
* In a cache on the server itself, to move load from the tile generator to a simpler caching process.

In order for caching to occur at any of these layers, the caching mechanisms need to know when a resource is cachable.

If your tile server is written using a scripting or programming language, you will probably be constructing your HTTP headers yourself, and it is important to include cache control headers when doing so to allow caching to occur.

There are different cache headers for HTTP 1.0 and HTTP 1.1, and because both protocols are in active use, it is important to include both.

For HTTP 1.0, use the "Expires" header. If you expect your data to change no more than once per week, set your Expires header to one week in the future. For example, if it is January 1, 2007, and you wanted your tiles to expire no more than one week after they are retreived, you would set your header using this PHP invocation:

header('Expires: Mon, 8 Jan 2007 14:57:12 GMT');

Or, to always set the Expires header to one week in the future:

header('Expires: ' . gmdate('D, j M Y H:i:s T', time() + 7 * 24 * 60 * 60)); // time + 7 days worth of seconds

For HTTP 1.1, use the "Cache-control" header. Unlike the older "Expires" header, "Cache-control" does not have a clock reference, just a time period to reatain the data, thereby avoiding the clock sychronization issues of "Expires".

header('Cache-Control: max-age=86400, must-revalidate');
header('Cache-Control: ' . 7 * 24 * 60 * 60 );

Read about HTTP 1.1 cache control headers in the [http://www.w3.org/Protocols/rfc2616/rfc2616-sec14.html#sec14.9 W3.org specification].

===Implementing Cacheability===

You can trust that the somewhere on the internet, someone will respect your cache control headers and your content will be cached, ''or'' you can set up your own cache. If you are running Apache 2.0 adding [http://httpd.apache.org/docs/2.0/mod/mod_cache.html mod_cache caching] to your tms is laughably easy.

Just add a mod_cache directive inside your <VirtualHost> definition:

<IfModule mod_disk_cache.c>
CacheRoot /tmp/apache-cache
CacheSize 1024000
CacheEnable disk /cgi-bin/tms
CacheDirLevels 5
CacheDirLength 3
</IfModule>

This example is for a disk cache, probably what you will use for your TMS, since the data volumes tend to be high. Note how the CacheEnable directive allows you to very precisely control which content you are going to cache. In my case, I am only caching the output from my TMS server, nothing else. If I wanted, I could be even narrower and restrict caching to just one tile map inside my service, or just one tile set.

===URLs That are Actually Scripts===

For large implementations of the tile map server specification, the data will not be statically pre-built, but will be demand-generated by some kind of backend service. That means that URLs that appear to be static may actually be dynamic.

The [http://hoohoo.ncsa.uiuc.edu/cgi/env.html CGI specification] allows this trivially, by passing any path information after the CGI executable in the URL back to the executable in the PATH_INFO environment variable:

<nowiki>http://tms.osgeo.org/cgi-bin/tms/1.0.0/vmap0</nowiki>

PATH_INFO = 1.0.0/vmap0

If "tms" is the CGI executable, it can easily extract the remaining path information and use that for processing purposes.

Note that by default some versions of Microsoft IIS do not conform to the CGI specification for this behavior (Apache does). See the note at http://support.microsoft.com/kb/q184320/ for information on how to enable this bahavior in IIS.

Note that it is allowable for URLs to include "."s in the middle of paths, so that executable scripts (like PHP files) can be legally used as TMS servers.

Here is a root resource:
<nowiki>http://tms.osgeo.org/tms.php</nowiki>

Here is the a tile request on that server:
<nowiki>http://tms.osgeo.org/tms.php/1.0.0/thetilemap/firstlevel/2/1.jpg</nowiki>

In general, the simplest way to extract information from the incoming script invocation is to take the incoming PATH_INFO environment variable, strip the "/" character from the start and end of the string, and then split the string into an array using the "/" character. In this manner, the first element of the array will be the version, the second element will be the tile map, the third will be the level, the fourth will be the tile "x" and the fifth will be the tile "y" (with a .extension).

===Using Faces===

The "face-id" attribute of the <TileMap> referenced in the <TileMapService> is used for some specialized clients. How that attribute is used by each client is described here.

''To be filled in by implementation knowledge...''

===Reference Implementations===

====Servers====

WARNING, CURRENTLY BOTH OUT OF DATE DUE TO RECENT SPECIFICATION CHANGE

* This is a first attempt at a reference server, it might not work yet, and has no error handling at all. And it is probably very slow, it just wraps up the NASA WMS Global Mosaic within the Tile Map Service specification.
** http://mapserver.refractions.net/cgi-bin/tms
* Metacarta has created another reference implementation in Python.
** http://labs.metacarta.com/wms-c/#TileCache

====Clients====

* Worldkit: http://worldkit.org/tilemap/
* Openlayers: http://dev.openlayers.org/sandbox/crschmidt/refractions/examples/refractions.html
* Cadcorp SIS: [http://wiki.osgeo.org/images/a/a5/CadcorpTmsClient.png Screenshot]

===Returning Error Codes===

If your tile map server is a static set of files, you will find that your web server sets the appropriate error codes automatically when people ask for resources that do not exist, or the server suffers a failure.

However, if your tile map server is dynamic, you will have to set the HTTP status codes yourself, otherwise the HTTP server will assign a code of 200 (OK) for your error message XML document, which would be wrong. In PHP, an error return function might look like this:

header("HTTP/1.0 404 Not Found");
header("Content-type: text/xml");
print "<?xml version='1.0' ?>";
print "<TileMapServerErrror>";
print "<Message>You requested a map tile [ $path_info ] that does not exist.</Message>";
print "</TileMapServerError>";

Note that in addition to setting the error code the Content-type was also correctly set. Also note that cache headers were ''not'' set, since errors are not a good thing to cache.

==See Also==

* [http://www.peej.co.uk/articles/http-caching.html HTTP Caching]
* [http://www.peej.co.uk/articles/rest.html What is REST]

File:CadcorpTmsClient.png

2006-11-17T13:17:22Z

Wiki-Mpd: