OSGeo - User contributions [en]

Benchmarking 2010

2010-04-18T22:56:44Z

Wiki-Jencek: /* Links */

== Basic Premise ==

Following up on [[Benchmarking_2009|last year's exercise]], the performance shoot-out presentation at [[FOSS4G2010]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Timeline ==

{| border="1"
|January 1st, 2010
|begin contacting all mapping servers
|-
|April 1st, 2010
|commitment due by all interested mapping servers
|-
|April 1st, 2010
|exercise begins (and weekly meetings)
|-
|August 1st, 2010
|final testing begins
|-
|August 23rd, 2010
|no further testing
|-
|September 6-9, 2010
|present results at FOSS4G2010

|}

== Communication ==

Coordination/communication is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

== Potential Participants ==

{| border="1"
|-
| '''Mapping Server'''
| '''Volunteer to contact Dev Team'''
| '''Contacted Server's Development Team'''
| '''Confirmed by Server's Development Team'''
| '''Development Team Leader'''
|-
| Constellation-SDI
| [[User:Acuster |Adrian Custer]]
| yes
| yes
| [mailto:cedric.briancon--@--geomatys.fr Cédric Briançon]
|-
| Deegree
| Simone Giannecchini
| yes
|
|
|-
| Erdas Apollo
|
|
|
|
|-
| ESRI ArcGIS Server
| Joel Schlagel
|
|
|
|-
| GeoServer
| Andrea Aime
| yes
| yes
| Andrea Aime
|-
| MapGuide Open Source
| DanickVenne?
|
|
|
|-
| Luciad Web Services Suite
| [[User:LeifGruenwoldt|Leif Gruenwoldt]]
| yes
|
|
|-
| Mapnik
| [[User:Springmeyer|Dane Springmeyer]]
| yes
| yes
| [[User:Springmeyer|Dane Springmeyer]]
|-
| MapServer
| [[User:Jmckenna|Jeff McKenna]]
| yes
| yes
| [[User:Jmckenna|Jeff McKenna]]
|-
| Oracle MapViewer
| Mike Smith
|
|
|
|}

== Links ==

* [[FOSS4G_Benchmark | FOSS4G WMS Benchmark]]
* [http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]
----

[[Category:FOSS4G2010]] [[Category:FOSS4G]]

Benchmarking 2010

2010-04-17T06:09:02Z

Wiki-Jencek: /* Links */

Benchmarking 2009

2010-04-17T06:02:43Z

Wiki-Jencek: /* Links */

== Overview ==

{| border="1"
|Description
|The performance shoot-out will see mapping servers compared in terms of how long they take to generate a map image, from common set of spatial data.
|-
|Participants
|development teams from GeoServer, MapServer
|-
|Results Announced
|[http://2009.foss4g.org/ FOSS4G2009] event, 20-23 October, 2009, in Sydney Australia
|-
|Press Release
|[[FOSS4G_2009_Press_Release_32]]

|}

== Participants ==

* Geoserver
** Andrea Aime
** [[User:simboss | Simone Giannecchini]]
* MapServer
** [[User:Dmorissette | Daniel Morissette]]
** [[User:jmckenna | Jeff McKenna]]
** Paul Ramsey
** Frank Warmerdam (raster)
* PostGIS
** Paul Ramsey
* Oracle
** Mike Smith
* SDE
** Mike Smith
* ArcGIS Server (withdrew from exercise)
** Satish Sankaran

== Basic Premise ==

The performance shoot-out presentation at [[FOSS4G2009]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version. For example, MapServer 5.6.0 (betas) will be tested, while GeoServer 2.0 (dev) will be tested.
# Data formats used must be shapefiles for vectors, geotiff files for rasters, and PostgreSQL/PostGIS for database. If you wish to test other formats/backends (such as SDE or ECW) you may do so, but that is optional for all parties. See below for access to the required data.
# Output formats are: jpeg for imagery, gif for non-antialised vector provided we do any, and png24 for antialiased
# Nearest Neighbor resampling should be used for raster image display

== Coordination ==

Coordination is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed Oct 6th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=10&day=6&hour=15&min=0&sec=0
** Agenda:
***

=== Previous Meetings ===

* Wed Sept 23rd: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=23&hour=15&min=0&sec=0
** Agenda:
*** moving scripts/files into SVN on BenchmarkingA server
*** LD_LIBRARY_PATH issue on BenchmarkingA server
*** FYI data download available
*** new vector data available on BenchmarkingA server (/opt/data/TIGER-2008)
**** what layers to use
*** what scales should be styled
**** e.g. MapServer scale of [http://labs.gatewaygeomatics.com/cgi-bin/wms_benchmarking_by_merged?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=vector_benchmarking&STYLES=&SRS=EPSG:4269&BBOX=-97.038385,32.545214,-96.516866,32.989369&WIDTH=950&HEIGHT=600&FORMAT=image/png 1:200,000 of Dallas]
**** e.g. MapServer scale of [http://labs.gatewaygeomatics.com/cgi-bin/wms_benchmarking_by_merged?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=vector_benchmarking&STYLES=&SRS=EPSG:4269&BBOX=-97.0286,32.5511,-96.8724,32.6925&WIDTH=950&HEIGHT=600&FORMAT=image/png 1:75,000 of Duncanville/Dallas] (with some ESRI icons)
* Wed Sept 16th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=16&hour=15&min=0&sec=0
** Agenda:
*** Directory structure in /opt/ ?
*** Status of ERDAS
*** status of home for the benchmarking files: http://trac.osgeo.org/osgeo/ticket/435
*** feedback on vector styling
** Minutes
*** msmith: will setup SDE+Oracle on RedHat by Tues
*** augusttown: will draft gnis_pop & tiger_trac styling
*** jmckenna: possibly have maptools.org host the benchmarking files?
*** frankw: press release should for now include GeoServer, MapServer, ESRI, and 'other servers' (until ERDAS confirms)

* Wed Sept 9th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=09&hour=15&min=0&sec=0
** Minutes:
*** jmckenna: Need confirmation from ERDAS so press release can move forward
*** msmith: can setup rhel 5 64 bit with sde 9.3.1 and oracle 11g 64 bit for SDE (need confirmation from ESRI that this setup is OK)
*** aaime: need feedback on styling (icon for point styling, for example)
*** aaime: all should use mailing list more often, for feedback
*** aaime: proposed roads style on mailing list
*** augusttown: are we ok with having a separate machine for database?
*** aaime: postgres should be on that same machine (jmckenna agrees)
*** augusttown: please post possible operating system options for database machine and we will get back to you
*** msmith: pramsey do you want to install postgres/postgis on new db machine?
*** jmckenna: setup an OSGeo server for downloading benchmarking data ([http://trac.osgeo.org/osgeo/ticket/435 ticket])
*** msmith: local port forwarding is ok for doing the testing or do I need to set up the NAT to handle some of the web traffic out?

* Wed Sept 2nd: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=02&hour=14&min=0&sec=0
** Agenda:
*** update on ERDAS involvement
*** what other web server teams have been contacted? Should more be contacted?
*** work on a press release: http://lists.osgeo.org/pipermail/benchmarking/2009-August/000107.html
*** output format (jpeg, png 24 bit, png 8 bit, gif?)
*** antialiasing (enabled, or not?)
*** interpolation (nearest neighbour, bilinear, bicubic?)
*** specifics on raster formats tests, especially the TIFF part (multilayers, mosaicked, ...)

* Wed Aug 26
** discussion logs: http://logs.qgis.org/foss4g/%23foss4g.2009-08-26.log
** summary:
*** Frank: to confirm with ESRI about Operating System (RHEL or CentOS)
*** Mike: to update wiki with specs when OS switch is done (if required)
*** discussion on how to test (OGC versus native access)...WMS was agreed upon
*** Daniel: volunteers to look at MapServer's WMS vs CGI performance (which is optional)
*** Data: we'll use same data as previous years (but plan to update data to next TIGER or OSM release next year)
**** all agreed to use shapefiles, geotiffs, and PostGIS. to be verified with ESRI.
**** Jeff: to review Andrea's progress re: layer stying
*** foss4g2009: Andrea and Jeff will co-present. (plus an ESRI representative??)
**** press release needs to be drafted: http://lists.osgeo.org/pipermail/benchmarking/2009-August/000107.html

== Hardware ==

Machine A (server)
* System Type: Dell PowerEdge 2850
* Ship Date: 10/21/2004
* 2 X Processor, 80546K, 3.4G, 1M, Xeon Nocona, 800
* 4 X DIMM, 2G 400M, 128X72, 8, 240, 2RX4
* 6 X Hard Drive, 73GB, SCSI, U320, 15K
* Service Tag: 16GRV51
* OS: Centos 5.3 + updates

Machine B (testing box)
* System Type: Dell PowerEdge 1750
* Ship Date: 9/23/2003
* 1 x Processor, 80532K, 3.06GHz, 512K 533
* 2 x Dimm, 512, 266M, 64X72, 8K, 184, 1U
* 2 x Hard Drive, 300Gb, SCSI, U320, 10K
* OS: Centos 5.3 + updates

Machine C (DB host)
* Dell Optiplex 755 tower
* 1 x Intel Core2 Duo CPU E6750 @ 2.66GHz / Dual Core
* 100Gb SATA 3.0Gb/s 7200 rpm HD
* 4Gb RAM
* OS: RedHat 5.3 + updates

== Layers ==

* '''[[texas_roads_styled]]''' This test exercises the renderer a bit more. The lines will be "pipe-styled" and different road types will have different styles. ''Data: edges_merge''
* '''[[point_layer_styled]]''' This test exercises the ability of the renderer to build a map in which points are symbolized with a certain number of externally provided icons (png/svg) depending on some point attribute. ''Data: gnis_names09''
* '''[[polygon_layer_styled]]''' Tests polygon rendering, with a solid fill. For this test the type of classification matters less than the number and complexity of the polygons (number of vertices and rings) which is the aspect we should work on. e.g. render maps with a few complex polygons vs other maps with lots of small polygons. ''Data: areawater_merge''
* '''single big ECW file''' (actual data TBD). This test exercises the ability to read a single wavelet compressed file and return it in JPEG format.
* '''file system mosaic of TIFF tiles'''. Generated out of the ECW file by splitting it into a sizeable amount of tiles, this test checks the ability of the server to efficiently retrieve data from a large collection of images and deal the associated file handling issues (ulimit and the like). It can also serve as a comparison with ECW results.

== Data ==

Data will reside in '''/opt/data''' on the testing server. Do not add any data to this directory without also describing it fully on this page.

* Vector data (original):
** '''gnis_names.shp''' (POINT, EPSG:4326) US named feature points.
** '''states.shp''' (POLYGON, EPSG:4326) US states and demographics.
** '''tiger_shp.shp''' (LINESTRING, EPSG:4326) TIGER roads for Texas.
** '''tiger_tracts.shp''' (POLYGON, EPSG:4326) TIGER census tracts for USA.
* Vector data (GNIS Names 2009)
** '''/GNIS-2009/gnis_names09.shp''' (POINT, EPSG:4269) US named feature points, for 2009.
* Vector data (TIGER 2008 of Texas)
** separated into individual counties (like the TIGER release)
*** '''/TIGER-2008/48_TEXAS/County/arealm.shp''' (POLYGON, EPSG:4269) area landmarks (e.g. parks) for counties.
*** '''/TIGER-2008/48_TEXAS/County/areawater.shp''' (POLYGON, EPSG:4269) area water(e.g. lakes) for counties.
*** '''/TIGER-2008/48_TEXAS/County/edges.shp''' (LINE, EPSG:4269) linework (e.g. roads, rivers) for counties.
*** '''/TIGER-2008/48_TEXAS/County/pointlm.shp''' (POINT, EPSG:4269) point landmarks (e.g. hospital, airport) for counties.
*** '''/TIGER-2008/48_TEXAS/county_tileindex.shp''' (POLYGON, EPSG:4269) county index file used by MapServer.
** merged into single files for the entire state (required for servers that cannot load many .shp files as a single entity)
*** '''/TIGER-2008/48_TEXAS/arealm_merge.shp''' (POLYGON, EPSG:4269) area landmarks (e.g. parks) for entire state.
*** '''/TIGER-2008/48_TEXAS/areawater_merge.shp''' (POLYGON, EPSG:4269) area water(e.g. lakes) for entire state.
*** '''/TIGER-2008/48_TEXAS/edges_merge.shp''' (LINE, EPSG:4269) linework (e.g. roads, rivers) for entire state.
*** '''/TIGER-2008/48_TEXAS/pointlm_merge.shp''' (POINT, EPSG:4269) point landmarks (e.g. hospital, airport) for entire state.
** general
*** '''/TIGER-2008/48_TEXAS/tl_2008_48_place.shp''' (POLYGON, EPSG:4269) places (populated areas) for entire state.
*** '''/TIGER-2008/tl_2008_us_county.shp''' (POLYGON, EPSG:4269) county outlines for entire state.
*** '''/TIGER-2008/tl_2008_us_state.shp''' (POLYGON, EPSG:4269) US state outlines.
* Raster data
** '''world-topo-bathy-200409-3x86400x43200.ecw''' (ECW, EPSG:4326) Medium size ECW file (Bluemarble Next Generation, whole world)
*** '''world-topo-bathy-200409-3x86400x43200.tif''' The same as a tiled BigTIFF, with overviews.
*** '''bluemarble_512/*.tif''' The same split into 512 GeoTIFF tiles, each with overviews.
* PostGIS data
** dbname=benchmark user=postgres password=postgres port=5432 host=localhost
** tables: public.gnis_names public.states public.tiger_shp public.tiger_tracts public.edges_merge public.pointlm_merge public.arealm_merge public.areawater_merge public.gnis_names09
** geometry column: "the_geom" in all cases
** srid: 4326 in all cases

=== Download ===

* http://www.maptools.org/foss4g/ (user/pass: foss4g/foss4g)
** raster-data.zip (.4 GB)
** vector-data-original.zip (.5 GB) (old tiger data, merged, for Texas)
** vector-data-tiger08-tx-counties.zip (1.1 GB) (TIGER 08, stored by county, for Texas)
** vector-data-tiger08-tx-merged.zip (1.1 GB) (TIGER 08, merged, for Texas)
** GNIS-2009.zip (7 MB) (2009 GNIS point data for Texas)

== Live BenchmarkA WMS GetMap Requests ==

=== MapServer ===

* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png PostGIS]

* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png PostGIS fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-merged.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Merged Shapefiles]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-merged.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=roads-merged&STYLES=&SRS=EPSG:4269&WIDTH=950&HEIGHT=600&FORMAT=image/png&BBOX=-96.79344550384396,32.76888304968294,-96.78588564285151,32.77365769873079 Merged Shapefiles - Local Roads labelled]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-county.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-county,roads-county,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png County-based Shapefiles]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oraclespatial,roads-merged-oraclespatial,gnis-names-oraclespatial&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png OracleSpatial]
* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oraclespatial,roads-merged-oraclespatial,gnis-names-oraclespatial&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png OracleSpatial fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oracle-ogr,roads-merged-oracle-ogr,gnis-names-oracle-ogr&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Oracle through OGR]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde,roads-merged-sde,gnis-names-sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE]
* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde,roads-merged-sde,gnis-names-sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde-ogr,roads-merged-sde-ogr,gnis-names-sde-ogr&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE through OGR]

=== GeoServer ===

* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=postgis&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Postgis group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=shapefiles&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Shapefiles group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=shp:edges_merge&STYLES=roads_classified_shp_label&SRS=EPSG:4269&WIDTH=950&HEIGHT=600&FORMAT=image/png&BBOX=-96.79344550384396,32.76888304968294,-96.78588564285151,32.77365769873079 Shapefiles, labelled]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=oracle&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Oracle group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE group]

== Software ==

For those of us who have to build our software, the builds will be staged in '''/opt/build'''

=== Libraries ===

* GEOS 3.1.1 installed in /usr/local (pramsey)
* Proj4 SVN trunk installed in /usr/local (pramsey)
* libecw installed in /usr/local (pramsey)
* GDAL 1.6.1 installed in /usr/local (pramsey)
* AGG 2.5 installed in /usr/local (pramsey)
* FastCGI installed in /usr/local (pramsey)

=== Databases ===

* PostgreSQL installed in /usr/local/pgsql (pramsey)
** PostgreSQL 8.3.7
** PostGIS 1.4.0
** PGDATA in /home/postgres/data

=== GeoServer related ===
* Sun JDK 1.6.0_16 with JAI, JAI Image-IO and ImageIO-ext native extensions installed in (/opt/jdk1.6.0_16) (aaime)
* GeoServer binaries and configurations (/opt/geoserver) (still incomplete at the moment) (aaime)

=== Testing machine ===

* Plain Sun JDK 1.6.0_16 (/opt/jdk1.6.0_16) (aaime)
* JMeter (/opt/jakarta-jmeter-2.3.4) (aaime)
* symbolic link to start jmeter in /usr/local/bin/jmeter (instructions on how to use it will follow) (aaime)
* /usr/local/bin/wms_requests.py, simple tool to generate random bounding boxes and sizes to drive JMeter (aaime, tool developed by Frank W.)

=== Starting / Stopping Services ===

To shutdown apache, and associated mapserver fastcgi processes:

sudo /usr/sbin/apachectl stop

For MapServer team: you might want to do a graceful restart of Apache, before running tests

sudo /usr/sbin/apachectl graceful

To shutdown arcgis:

sudo su - arcgis
cd /arcgis/scripts
./stopserver

To shutdown GeoServer do the following

sudo /opt/geoserver/gs20x/catalina.sh stop
ps aux | grep java | grep geoserver

The second command can be used to confirm GeoServer is really down, if not, kill it manually ('kill -9 PID').

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Press Release ==

[[FOSS4G_2009_Press_Release_32]]

== Results ==

Slides of the presentation of the results can be
[http://svn.osgeo.org/osgeo/foss4g/benchmarking/scripts/results/benchmarking2009.odp pulled from the code repository] or can be
[http://www.slideshare.net/gatewaygeomatics.com/wms-performance-shootout viewed online].

Other results obtained during the process can be seen [http://svn.osgeo.org/osgeo/foss4g/benchmarking/scripts/results/ in the svn repository].

== Links ==

* [[FOSS4G_Benchmark | FOSS4G WMS Benchmark]]
* [[Benchmarking_2010 | Benchmarking 2010]]
* [http://sourceforge.net/projects/wmstester/ WMSTester - tool for testing not from OSGeo]

----

[[Category:FOSS4G2009]]
[[Category:FOSS4G]]

Benchmarking 2010

2010-04-17T06:02:18Z

Wiki-Jencek: /* Links */

Benchmarking 2009

2010-04-17T05:59:54Z

Wiki-Jencek: /* Links */

== Overview ==

{| border="1"
|Description
|The performance shoot-out will see mapping servers compared in terms of how long they take to generate a map image, from common set of spatial data.
|-
|Participants
|development teams from GeoServer, MapServer
|-
|Results Announced
|[http://2009.foss4g.org/ FOSS4G2009] event, 20-23 October, 2009, in Sydney Australia
|-
|Press Release
|[[FOSS4G_2009_Press_Release_32]]

|}

== Participants ==

* Geoserver
** Andrea Aime
** [[User:simboss | Simone Giannecchini]]
* MapServer
** [[User:Dmorissette | Daniel Morissette]]
** [[User:jmckenna | Jeff McKenna]]
** Paul Ramsey
** Frank Warmerdam (raster)
* PostGIS
** Paul Ramsey
* Oracle
** Mike Smith
* SDE
** Mike Smith
* ArcGIS Server (withdrew from exercise)
** Satish Sankaran

== Basic Premise ==

The performance shoot-out presentation at [[FOSS4G2009]] will test how long each Web mapping server takes to generate a map image, from a common set of spatial data, on a common platform. The data will be served by each Web mapping server through the WMS standard, which will serve exactly the same set of LAYERS. A JMeter load will be run on the testing box to measure various aspects of those layers.

== Rules of Engagement ==

# All parties must contribute any changes that they make to their software for this exercise, back to their community.
# Comparisons will be made of the best available version of the software, be it a formal release or a development version. For example, MapServer 5.6.0 (betas) will be tested, while GeoServer 2.0 (dev) will be tested.
# Data formats used must be shapefiles for vectors, geotiff files for rasters, and PostgreSQL/PostGIS for database. If you wish to test other formats/backends (such as SDE or ECW) you may do so, but that is optional for all parties. See below for access to the required data.
# Output formats are: jpeg for imagery, gif for non-antialised vector provided we do any, and png24 for antialiased
# Nearest Neighbor resampling should be used for raster image display

== Coordination ==

Coordination is primarily via the Benchmarking mailing list: http://lists.osgeo.org/mailman/listinfo/benchmarking

Weekly meetings will occur through IRC chat in the #foss4g channel on irc.freenode.net

=== Next IRC meeting ===

* Wed Oct 6th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=10&day=6&hour=15&min=0&sec=0
** Agenda:
***

=== Previous Meetings ===

* Wed Sept 23rd: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=23&hour=15&min=0&sec=0
** Agenda:
*** moving scripts/files into SVN on BenchmarkingA server
*** LD_LIBRARY_PATH issue on BenchmarkingA server
*** FYI data download available
*** new vector data available on BenchmarkingA server (/opt/data/TIGER-2008)
**** what layers to use
*** what scales should be styled
**** e.g. MapServer scale of [http://labs.gatewaygeomatics.com/cgi-bin/wms_benchmarking_by_merged?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=vector_benchmarking&STYLES=&SRS=EPSG:4269&BBOX=-97.038385,32.545214,-96.516866,32.989369&WIDTH=950&HEIGHT=600&FORMAT=image/png 1:200,000 of Dallas]
**** e.g. MapServer scale of [http://labs.gatewaygeomatics.com/cgi-bin/wms_benchmarking_by_merged?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=vector_benchmarking&STYLES=&SRS=EPSG:4269&BBOX=-97.0286,32.5511,-96.8724,32.6925&WIDTH=950&HEIGHT=600&FORMAT=image/png 1:75,000 of Duncanville/Dallas] (with some ESRI icons)
* Wed Sept 16th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=16&hour=15&min=0&sec=0
** Agenda:
*** Directory structure in /opt/ ?
*** Status of ERDAS
*** status of home for the benchmarking files: http://trac.osgeo.org/osgeo/ticket/435
*** feedback on vector styling
** Minutes
*** msmith: will setup SDE+Oracle on RedHat by Tues
*** augusttown: will draft gnis_pop & tiger_trac styling
*** jmckenna: possibly have maptools.org host the benchmarking files?
*** frankw: press release should for now include GeoServer, MapServer, ESRI, and 'other servers' (until ERDAS confirms)

* Wed Sept 9th: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=09&hour=15&min=0&sec=0
** Minutes:
*** jmckenna: Need confirmation from ERDAS so press release can move forward
*** msmith: can setup rhel 5 64 bit with sde 9.3.1 and oracle 11g 64 bit for SDE (need confirmation from ESRI that this setup is OK)
*** aaime: need feedback on styling (icon for point styling, for example)
*** aaime: all should use mailing list more often, for feedback
*** aaime: proposed roads style on mailing list
*** augusttown: are we ok with having a separate machine for database?
*** aaime: postgres should be on that same machine (jmckenna agrees)
*** augusttown: please post possible operating system options for database machine and we will get back to you
*** msmith: pramsey do you want to install postgres/postgis on new db machine?
*** jmckenna: setup an OSGeo server for downloading benchmarking data ([http://trac.osgeo.org/osgeo/ticket/435 ticket])
*** msmith: local port forwarding is ok for doing the testing or do I need to set up the NAT to handle some of the web traffic out?

* Wed Sept 2nd: http://timeanddate.com/worldclock/fixedtime.html?year=2009&month=09&day=02&hour=14&min=0&sec=0
** Agenda:
*** update on ERDAS involvement
*** what other web server teams have been contacted? Should more be contacted?
*** work on a press release: http://lists.osgeo.org/pipermail/benchmarking/2009-August/000107.html
*** output format (jpeg, png 24 bit, png 8 bit, gif?)
*** antialiasing (enabled, or not?)
*** interpolation (nearest neighbour, bilinear, bicubic?)
*** specifics on raster formats tests, especially the TIFF part (multilayers, mosaicked, ...)

* Wed Aug 26
** discussion logs: http://logs.qgis.org/foss4g/%23foss4g.2009-08-26.log
** summary:
*** Frank: to confirm with ESRI about Operating System (RHEL or CentOS)
*** Mike: to update wiki with specs when OS switch is done (if required)
*** discussion on how to test (OGC versus native access)...WMS was agreed upon
*** Daniel: volunteers to look at MapServer's WMS vs CGI performance (which is optional)
*** Data: we'll use same data as previous years (but plan to update data to next TIGER or OSM release next year)
**** all agreed to use shapefiles, geotiffs, and PostGIS. to be verified with ESRI.
**** Jeff: to review Andrea's progress re: layer stying
*** foss4g2009: Andrea and Jeff will co-present. (plus an ESRI representative??)
**** press release needs to be drafted: http://lists.osgeo.org/pipermail/benchmarking/2009-August/000107.html

== Hardware ==

Machine A (server)
* System Type: Dell PowerEdge 2850
* Ship Date: 10/21/2004
* 2 X Processor, 80546K, 3.4G, 1M, Xeon Nocona, 800
* 4 X DIMM, 2G 400M, 128X72, 8, 240, 2RX4
* 6 X Hard Drive, 73GB, SCSI, U320, 15K
* Service Tag: 16GRV51
* OS: Centos 5.3 + updates

Machine B (testing box)
* System Type: Dell PowerEdge 1750
* Ship Date: 9/23/2003
* 1 x Processor, 80532K, 3.06GHz, 512K 533
* 2 x Dimm, 512, 266M, 64X72, 8K, 184, 1U
* 2 x Hard Drive, 300Gb, SCSI, U320, 10K
* OS: Centos 5.3 + updates

Machine C (DB host)
* Dell Optiplex 755 tower
* 1 x Intel Core2 Duo CPU E6750 @ 2.66GHz / Dual Core
* 100Gb SATA 3.0Gb/s 7200 rpm HD
* 4Gb RAM
* OS: RedHat 5.3 + updates

== Layers ==

* '''[[texas_roads_styled]]''' This test exercises the renderer a bit more. The lines will be "pipe-styled" and different road types will have different styles. ''Data: edges_merge''
* '''[[point_layer_styled]]''' This test exercises the ability of the renderer to build a map in which points are symbolized with a certain number of externally provided icons (png/svg) depending on some point attribute. ''Data: gnis_names09''
* '''[[polygon_layer_styled]]''' Tests polygon rendering, with a solid fill. For this test the type of classification matters less than the number and complexity of the polygons (number of vertices and rings) which is the aspect we should work on. e.g. render maps with a few complex polygons vs other maps with lots of small polygons. ''Data: areawater_merge''
* '''single big ECW file''' (actual data TBD). This test exercises the ability to read a single wavelet compressed file and return it in JPEG format.
* '''file system mosaic of TIFF tiles'''. Generated out of the ECW file by splitting it into a sizeable amount of tiles, this test checks the ability of the server to efficiently retrieve data from a large collection of images and deal the associated file handling issues (ulimit and the like). It can also serve as a comparison with ECW results.

== Data ==

Data will reside in '''/opt/data''' on the testing server. Do not add any data to this directory without also describing it fully on this page.

* Vector data (original):
** '''gnis_names.shp''' (POINT, EPSG:4326) US named feature points.
** '''states.shp''' (POLYGON, EPSG:4326) US states and demographics.
** '''tiger_shp.shp''' (LINESTRING, EPSG:4326) TIGER roads for Texas.
** '''tiger_tracts.shp''' (POLYGON, EPSG:4326) TIGER census tracts for USA.
* Vector data (GNIS Names 2009)
** '''/GNIS-2009/gnis_names09.shp''' (POINT, EPSG:4269) US named feature points, for 2009.
* Vector data (TIGER 2008 of Texas)
** separated into individual counties (like the TIGER release)
*** '''/TIGER-2008/48_TEXAS/County/arealm.shp''' (POLYGON, EPSG:4269) area landmarks (e.g. parks) for counties.
*** '''/TIGER-2008/48_TEXAS/County/areawater.shp''' (POLYGON, EPSG:4269) area water(e.g. lakes) for counties.
*** '''/TIGER-2008/48_TEXAS/County/edges.shp''' (LINE, EPSG:4269) linework (e.g. roads, rivers) for counties.
*** '''/TIGER-2008/48_TEXAS/County/pointlm.shp''' (POINT, EPSG:4269) point landmarks (e.g. hospital, airport) for counties.
*** '''/TIGER-2008/48_TEXAS/county_tileindex.shp''' (POLYGON, EPSG:4269) county index file used by MapServer.
** merged into single files for the entire state (required for servers that cannot load many .shp files as a single entity)
*** '''/TIGER-2008/48_TEXAS/arealm_merge.shp''' (POLYGON, EPSG:4269) area landmarks (e.g. parks) for entire state.
*** '''/TIGER-2008/48_TEXAS/areawater_merge.shp''' (POLYGON, EPSG:4269) area water(e.g. lakes) for entire state.
*** '''/TIGER-2008/48_TEXAS/edges_merge.shp''' (LINE, EPSG:4269) linework (e.g. roads, rivers) for entire state.
*** '''/TIGER-2008/48_TEXAS/pointlm_merge.shp''' (POINT, EPSG:4269) point landmarks (e.g. hospital, airport) for entire state.
** general
*** '''/TIGER-2008/48_TEXAS/tl_2008_48_place.shp''' (POLYGON, EPSG:4269) places (populated areas) for entire state.
*** '''/TIGER-2008/tl_2008_us_county.shp''' (POLYGON, EPSG:4269) county outlines for entire state.
*** '''/TIGER-2008/tl_2008_us_state.shp''' (POLYGON, EPSG:4269) US state outlines.
* Raster data
** '''world-topo-bathy-200409-3x86400x43200.ecw''' (ECW, EPSG:4326) Medium size ECW file (Bluemarble Next Generation, whole world)
*** '''world-topo-bathy-200409-3x86400x43200.tif''' The same as a tiled BigTIFF, with overviews.
*** '''bluemarble_512/*.tif''' The same split into 512 GeoTIFF tiles, each with overviews.
* PostGIS data
** dbname=benchmark user=postgres password=postgres port=5432 host=localhost
** tables: public.gnis_names public.states public.tiger_shp public.tiger_tracts public.edges_merge public.pointlm_merge public.arealm_merge public.areawater_merge public.gnis_names09
** geometry column: "the_geom" in all cases
** srid: 4326 in all cases

=== Download ===

* http://www.maptools.org/foss4g/ (user/pass: foss4g/foss4g)
** raster-data.zip (.4 GB)
** vector-data-original.zip (.5 GB) (old tiger data, merged, for Texas)
** vector-data-tiger08-tx-counties.zip (1.1 GB) (TIGER 08, stored by county, for Texas)
** vector-data-tiger08-tx-merged.zip (1.1 GB) (TIGER 08, merged, for Texas)
** GNIS-2009.zip (7 MB) (2009 GNIS point data for Texas)

== Live BenchmarkA WMS GetMap Requests ==

=== MapServer ===

* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png PostGIS]

* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/postgis.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png PostGIS fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-merged.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged,roads-merged,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Merged Shapefiles]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-merged.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=roads-merged&STYLES=&SRS=EPSG:4269&WIDTH=950&HEIGHT=600&FORMAT=image/png&BBOX=-96.79344550384396,32.76888304968294,-96.78588564285151,32.77365769873079 Merged Shapefiles - Local Roads labelled]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/shapefile-county.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-county,roads-county,gnis_names&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png County-based Shapefiles]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oraclespatial,roads-merged-oraclespatial,gnis-names-oraclespatial&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png OracleSpatial]
* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oraclespatial,roads-merged-oraclespatial,gnis-names-oraclespatial&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png OracleSpatial fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/oracle.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-oracle-ogr,roads-merged-oracle-ogr,gnis-names-oracle-ogr&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Oracle through OGR]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde,roads-merged-sde,gnis-names-sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE]
* [http://64.222.187.168/cgi-bin/mapserv560beta3.fcgi?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde,roads-merged-sde,gnis-names-sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE fcgi]
* [http://64.222.187.168/cgi-bin/mapserv560beta3?MAP=/opt/benchmarking/mapserver/sde.map&SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=areawater-merged-sde-ogr,roads-merged-sde-ogr,gnis-names-sde-ogr&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE through OGR]

=== GeoServer ===

* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=postgis&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Postgis group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=shapefiles&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Shapefiles group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=shp:edges_merge&STYLES=roads_classified_shp_label&SRS=EPSG:4269&WIDTH=950&HEIGHT=600&FORMAT=image/png&BBOX=-96.79344550384396,32.76888304968294,-96.78588564285151,32.77365769873079 Shapefiles, labelled]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=oracle&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png Oracle group]
* [http://64.222.187.168:8080/gs20x/ows?SERVICE=WMS&VERSION=1.1.1&REQUEST=getmap&layers=sde&STYLES=&SRS=EPSG:4269&BBOX=-96.980629943844,29.854114257812,-96.79038641357,29.974268066406&WIDTH=950&HEIGHT=600&FORMAT=image/png SDE group]

== Software ==

For those of us who have to build our software, the builds will be staged in '''/opt/build'''

=== Libraries ===

* GEOS 3.1.1 installed in /usr/local (pramsey)
* Proj4 SVN trunk installed in /usr/local (pramsey)
* libecw installed in /usr/local (pramsey)
* GDAL 1.6.1 installed in /usr/local (pramsey)
* AGG 2.5 installed in /usr/local (pramsey)
* FastCGI installed in /usr/local (pramsey)

=== Databases ===

* PostgreSQL installed in /usr/local/pgsql (pramsey)
** PostgreSQL 8.3.7
** PostGIS 1.4.0
** PGDATA in /home/postgres/data

=== GeoServer related ===
* Sun JDK 1.6.0_16 with JAI, JAI Image-IO and ImageIO-ext native extensions installed in (/opt/jdk1.6.0_16) (aaime)
* GeoServer binaries and configurations (/opt/geoserver) (still incomplete at the moment) (aaime)

=== Testing machine ===

* Plain Sun JDK 1.6.0_16 (/opt/jdk1.6.0_16) (aaime)
* JMeter (/opt/jakarta-jmeter-2.3.4) (aaime)
* symbolic link to start jmeter in /usr/local/bin/jmeter (instructions on how to use it will follow) (aaime)
* /usr/local/bin/wms_requests.py, simple tool to generate random bounding boxes and sizes to drive JMeter (aaime, tool developed by Frank W.)

=== Starting / Stopping Services ===

To shutdown apache, and associated mapserver fastcgi processes:

sudo /usr/sbin/apachectl stop

For MapServer team: you might want to do a graceful restart of Apache, before running tests

sudo /usr/sbin/apachectl graceful

To shutdown arcgis:

sudo su - arcgis
cd /arcgis/scripts
./stopserver

To shutdown GeoServer do the following

sudo /opt/geoserver/gs20x/catalina.sh stop
ps aux | grep java | grep geoserver

The second command can be used to confirm GeoServer is really down, if not, kill it manually ('kill -9 PID').

== SVN ==

The project files (minus data) are stored in Subversion (http://svn.osgeo.org/osgeo/foss4g/benchmarking/).

== Press Release ==

[[FOSS4G_2009_Press_Release_32]]

== Results ==

Slides of the presentation of the results can be
[http://svn.osgeo.org/osgeo/foss4g/benchmarking/scripts/results/benchmarking2009.odp pulled from the code repository] or can be
[http://www.slideshare.net/gatewaygeomatics.com/wms-performance-shootout viewed online].

Other results obtained during the process can be seen [http://svn.osgeo.org/osgeo/foss4g/benchmarking/scripts/results/ in the svn repository].

== Links ==

* [[FOSS4G_Benchmark | FOSS4G WMS Benchmark]]
* [[Benchmarking_2010 | Benchmarking 2010]]
* [http://sourceforge.net/projects/wmstester/ WMSTester]

----

[[Category:FOSS4G2009]]
[[Category:FOSS4G]]

Reading the INSPIRE Metadata Draft

2007-03-24T22:12:28Z

Wiki-Jencek: /* Too general */

Metadata about geographic data is at the heart of INSPIRE. The metadata draft is the first in the set of "implementing rules" and it will underpin all the other implementing rules. The consultation process is open until 2007-03-30. While the documents are open access, comments can only be offered through an SDIC or Spatial Data Interest Community.

The Free and Open Source Geospatial Community has a voice through one of these SDICs thanks to Markus Neteler. This page contains preparatory material for a collective response through the FOSS GIS SDIC, from the POV of people implementing and managing metadata creation, collection and search services, working closely with many different data user communities.

* The response proper will live at [[Response to INSPIRE Metadata Draft]]. Initial notes are included below in the Issues section.
* It is interesting to read this in parallel with the North American Metadata Profile draft which is also currently in consultation. It's hoped the OSGeo community will also be able to contribute to a [[Response to NAP Metadata Draft]] and get the [http://geodatacommons.umaine.edu geodata commons] project involved in this.

== Reading the draft ==

* [http://www.ec-gis.org/inspire/reports/ImplementingRules/draftINSPIREMetadataIRv2_20070202.pdf the Implementing Rules for Metadata Draft] (pdf)
* [http://www.ec-gis.org/inspire/whatsnew.cfm#1590 supporting / background material]
* Pages 1-17 are metadata about the document itself, intentions and history, and can be safely skipped. Pages 43-104 are the Annexes.
* Annex A is particularly interesting as there are details of the thinking exposed in the mapping to ISO19115/39 that aren't set out in the implementing rules. ''If you want to know what's likely to affect you but are short on time, at minimum read section 5 and Annex A'''.

== Lightning Summary of the draft ==

The draft establishes a basic information model for metadata which is close to, but not specific to, ISO19115 and OGC Web Services.

It only mandates what metadata is published by and for public authorities covered by INSPIRE - it does not try to cover repository management or internal processes.

It separates out metadata properties into those useful for 'discovery', 'evaluation', and 'use'. It identifies one very high level "use case" for spatial data search services built from metadata being shared at this level.

It differentiates between properties useful for 'non-specialist' and 'expert' users into 2 Levels, 1 and 2. Level 1 is always mandatory. This *includes* classification according to the data themes in the INSPIRE annexes, and keywords from controlled vocabularies which are not covered by the IR document but are left to Spatial Data Theme Communities. (How these communities are found, selected, and make their decisions, is unknown to us at this time.)

= Issues =

'''This list is an overview of what jumped out at me as something to address. I don't know how much of this is appropriate to send back, or how much can be fixed. - [[User:JoWalsh]]''' I've added some random musings also, again, no idea if they are of use or even valid questions, but at least they are there and can be edited out, or used as the basis of discussions - [[User:IanIbbo]].

== Conceptual overview ==

The model maps quite well to the minimum useful subset identified in [[DCLite4G]]. It looks like a lightweight core. But, the model and the draft break down the problem space of metadata in a way that is a reaction to artificial scarcity of data. It identifies three phases of the metadata use cycle:
* discovery (of what data is out there)
* evaluation (of whether the data will be useful for specific purpose)
* use (once access gained, how to best use the data)

It is illuminates to compare this with the [[Reading the NAP Metadata Draft|North American Profile]] metadata draft which talks about

* discovery
* access
* fitness for use (e.g. evaluation)
* transfer

So the IRs both don't address how to make the data more useful via metadata, and are vague about how much a minimal subset is going to provide enough information to evaluate utility on. Generally the draft dances around data licensing access issues, and glosses over the over-engineering needed to work around artificial constraints on availability. IRs for evaluation and use of data based on metadata are not covered by this draft at all, but left up to the Spatial Data Theme communities for each of the 35 data themes identified in Annexes I-III of the INSPIRE text.

== Issues with specific metadata properties ==

The model maps quite well to [[DCLite4G]]. It looks like a lightweight core.

=== Things that aren't there that should be ===

'''5.2.8 Resource responsible party'''. Each dataset *must* have one or more people/organisations responsible for it. The IR says that this can be freetext or can be in more structured form. This '''only''' includes the responsible party's name, but NOT any form of contact details.

'''Some form of electronic or telephonic contact address should be mandatory, if the org/person's details are mandatory.''' Why publish ownership information - especially if there are constraints on access and reuse of the described data - if you can't immediately get in personal contact with someone who can make assurances about the data?

I agree, but not only for data, btu for metadata as well:
Page 53. sec A.2.8.1 and A2.8.2 and A.4.1: Other information for identification is necessary like e-mail, URL, voice number.

Annex A on mapping to IS019115 mandates that contact persons and organisations be free text, not resource identifiers. 2 serious problems with the ISO 19115 mapping:

* It does not ask or provide for contact details.
* It looks *mandatory* that the reponsible party be given a role, which in turn is one of N codes published by the Library of Congress to describe people's roles within organisations.

'''No discussion of formalising dataset accuracy / completeness - crucial for cost-benefit evaluation / evaluation of suitability for combining with other data sets.'''

I agree:
Page 31. sec 5.2.12: The lineage element is not enough for data evaluation. The same problem is on Page 55. sec A.3.2.

* Page 7. par. 6: "Separate IRs for discovery services are being prepared and are not the subject of this document". But I have found a lot of information about discovery (search) services in this document and I think that information about discovery services should be a part of this document. There is a conflict. We can not be sure how to evaluate this document when we do not know how will look IR for services.
* Page 17. sec. 4.1. Note 6, par 4 and 5: This part of service evaluation must be solved for the infrastructure and the INSPIRE must give IR for this part of the infrastructure. Evaluation of the service must be based on the purpose of the usage. I believe that case studies support system can help with it. There is a gap in IR metadata from this point of view.
* Page 22. sec 4.6., Par 6: "The content of these repositories need to be accessible. This should happen preferably via a standard interface and/or standard encoding format such as XML". Must be specified how, if there is not exact information how to connect then is not useful for implementation.
* Page 30. sec 5.2.7.: There is not a list of types. The INSPIRE should define authority that can define types of services. For example you have in examples an identifier WMS, but we use an identifier OGC:WMS in our catalogue. This can lead to inconsistency.
* Page 30. sec 5.2.3.: There is vertical extend missing, necessary for geology, meteorology and climatology studies.
* Page 57. sec A.3.6: Code List is good, but I can not find it in the document.
* Page 21. sec 4.6. Par 1 (numbering 1): "Search engine connected to a set of metadata repositories". How the repositories will be connected, how selected, real-time, replication, harvesting. There is no information about it in the document here on later in the text, but they should be defined.
* Page 8:There is no information about cooperation with FGDC, FAO or other subjects that should be later connected to the INSPIRE infrastructure. Does it mean that EU want to stay out of USA and UN infrastructures?
* Page 64. sec B.8: Resource provider is not defined. Why not dc: publisher?
* Page 77. Annex G: There is not information about Usage of a resource that is very necessary to evaluate quality of a resource.
* Page 101. Annex I: There are no guidelines, just some unclear information.

=== Things that are there that probably shouldn't be ===

Every 'dataset or dataset series' published under INSPIRE *must* include both a '''Resource topic category''' and a set of resource '''keywords'''.

Topic categories are very high-level classifications which correspond to each of the Spatial Data Themes identified in Annexes I, II and III of the INSPIRE Directive.

* Which topic category data fits in will often be a property of an organisation not any published data sets.

From an implementor's POV this will involve something like selecting a topic category for data at install time of metadata publishing engine, and forgetting about it. The IRs place a lot of faith in the ability of simple keyword / classification code matches to enhance utility of search and discovery services for users.

But. this already raises the bar for non-expert users (the domain vocabulary is jargon specific or oriented towards specialist codes)

The IRs emphasise the fact that keywords should originate from a ''controlled vocabulary''. The reponsibility for creating one is not in the hands of the Drafting Teams but in the hands of Spatial Data Theme Communities. How these are constituted and how their decisions become binding are unclear.

Again, faith in keywords for search utility is misplaced. Reliance on them may lead to false negatives. Again assumes familiarity with, or time and ability to learn about, what to expect in the domain from a non-expert user, and an expert will need a better level of detail. Pitfalls of 'controlled' keywording:
* intentional misclassification
* lazy/default misclassification

Both of these are at 'Level 1 for discovery metadata' which implies that any INSPIRE compliant metadata set MUST have both topic category and associated keywords.

I partly agree: It is true that for INSPIRE should be defined thesaurus for keywords. INSPIRE is mainly oriented to environment protection. There is GEMET thesaurus, but there is not a common agreement to use it. This part of metadata is not so simple and I understand that definition of a IR for this part is very difficult.

Other issues:
* Page 60. sec B.2.1: Temporal extent is meaningful always for discovering and searching.
* Page 85. sec H.5.2: Too difficult for implementation and not very useful, there are other ways how to support multilingual descriptions.
* Page 38. sec 6.2: Identifier are of two types. This is problematic. It is usually better to have only one type of identifier. The preferred one should be URL (URI). The document says that URL can be in some cases not unique. If you want to make identification more unique than mix URL with UUID. Like this: http://gis.vsb.cz/01f8da38-10d7-11da-b569-000f1f1a7b03

== Areas which are unclear ==

=== Conformity ===

This is an IR and obligatory to deal with. But 5.3.4 just says "see Annex F".
Annex F '''in its entirety''' says:

The way in which conformity is expressed in the INSPIRE IR will be defined in a subsequent draft based on discussions with the Drafting Team on Data specifications and harmonization.

(Is this where accuracy/completeness comes in? How can we know?)

=== Dataset series / Aggregate data ===

IR talks about dataset series. Some of the diagrams talk about 'MD_Aggregates'- this term isn't used elsewhere. No conception in this model of one UrDataSet with many different potential sources according to how they are packaged or processed. As the IRs mandate properties for dataset series, really need more clarity / examples about what they actually are.

=== Other ===

* Page 17. sec. 4.1. Note 6, par. 2: "Services, including web services, are routinely measured in terms of availability and performance. These parameters are easily quantified and users can easily agree on their value: services that are available more often (seeking the elusive 99.9% "up-time") are more desirable than services that are available less often, and services that provide faster response time are more desirable than similar services that are slower to respond.". This is not exact. When a user wants to use a service he must know more than about availability and speed of the service. For example cost, quality of used geodata, used algorithms, quality of used algorithms, possibility of chaining, quality of self-description of the process, development, update, security, type of call (synchronous x asynchronous). Perhaps all that you have to know about software, because a service is a software and a user will not use it once but many times and perhaps for a long time.
* Page 30. sec 5.2.4.: Does it mean that the resource for INSPIRE can not be encoded in other encoding such as Windows 1250? This is a problem that we have found in ISO 19115 and we still do not know how to handle it. We have a lot of geodata encoded in Windows-1250 and nobody is going to change the encoding, and it means that we can not describe them using ISO 19115 (or we do not know how, except extending the list).
* Page 33. sec 5.3.2: The resource responsible party must be searchable. The abstract should be searchable.
* Page 39. sec 6.3: The formulation are not clear. They need to be specified better with examples and schemas.
* Page 58. A.3.8.2: Not conform with A.3.8.1. Why A.3.8.2 is conditional and A.3.8.2 is mandatory?
* Page 64. sec B.6: The keyword element must be mandatory for services as well.
* Page 72. sec D4.2: I think that ISO 19139 must be adopted always not only when there is not any other XML schema, I can prepare my own schema compatible with ISO 19115 but not with ISO 19139, but this is not useful for INSPIRE.
* Page 19. sec. 4.3. Par 4 (numbering 3): This is not right. When there is not ISO 19139 yet final we can not talk about maturity. Some of the services have adopted ISO 19115, but there is not a conformity in adoption and many adoptions are probably wrong.
* Page 27. sec 5.1. Par 2 (bullet 1): "the user query expressed through the search interface of the search engine and provided in a form compatible with the metadata repository interface". This is unclear, what does it mean? Any repository can have different interface? That is horrible if you know that there is CSW 2.0 specified now.
* Page 28. sec 5.1. Par 9: "The discovery metadata elements are defined at an abstract level in order to make the Implementing Rules independent of ...". This is often wrong, if there is not specified encoding. In this case the IR for metadata can be useless.
* Page 33. sec 5.3.3: The Operation name should be moved to Discovery level 1 and must be searchable, this is necessary for services searching.
* Page 44. sec A1.2.1 Note 3: This kind of condition is problematic. Who will decide that particular statement or quality report is required? I assure you, nobody will do more than is specified in the IR for metadata.
* Page 49. sec A.2.3.1: EX_GeographicBoundingBox must be defined always for services too. Not conditional. When the service is not based on GeographicBoundingBox then it should be defined for a whole world.
* Page 45. sec A.1.2.2: “identifier: MD_identifier. condition: if the identifier is available”. Identifier will be probably available always and if not it should be generated.
* Page 58. A.4: Why are metadata on metadata in level 2 and why not searchable.
* Page 59. sec A.4.3: Free text. I do not understand why when there is a list. Does not list include all languages?
* Page 62. sec B.3: Do not understand the extent definition, description looks very strange.

== General concerns ==

=== Search / discovery services ===

The preamble (p.7) states that "separate IRs for discovery services are being prepared and are not the subject of this document." But the INSPIRE use case is predicated on the availablity of 'Geoportal' style search services. What else *are* discovery services if they are not the search services treated of here? If there is only going to be an abstract model for discovery, and these IRs are careful to avoid imposing any constraints on internal data repository management, how much more can a discovery services draft provide?

II: I think this observation is spot on, but for different reasons perhaps. I'm finding it difficult to express concrete concerns.. but Section 5.2 "Discovery metadata elements" starts to set out a list of concepts seen to be (The document hints at, but does not directly say) core to the discovery process. Section 5.3 then sets out "Abstract discovery metadata element set". I *guess* the implication is that the concepts laid out in 5.2 are in some way even more abstract than those set out in 5.3. The document really isn't clear about what the abstract model is, or what it is for, before it starts enumerating the concepts. Your later comment about being tied to web services is spot on also here, I'm really not sure "Service type version", "Operation name" and "Distributed computing platform" belong in an abstract discovery model (The probably *do* belong in some result record schema). These three attributes seem to belong specifically to a particular (And I would guess already existing) service binding (Or as already said, to a very specific kind of returned result record). What I'd really like to see is a much clearer statement of what the purpose of the abstract discovery model is. Hopefully, once that is tightly defined, it should become easier to decide what lies inside the boundary of the abstract model, and what belongs in the domain of specific realisations of the abstract model. (Actually.. I should say that I'm baised by the information retrieval community generally, in that it's considered really important to have a seperate abstract model for discovery (The search access points) and then bind that model on to as many backend schemas as needed.. this decoupling is seen as best practice in the information retrival domain, and most of my concerns here are that because of the apparent 1:1 mapping between the abstract model and the implementation. This is the approach taken in the [[Z3950 GEO profile] http://www.blueangeltech.com/standards/GeoProfile/geo22.htm]).

I'm a bit confused by the "Temporal Reference" Element... 5.2.2. Talks about what I would expect to see from a temporal reference, but 5.3.2 maps temporal reference on to "One of the dates of publication, last revision or creation of the resource". These three elements are already well defined by dublin core attributes... Maybe I've misunderstood whats implied by table 1 in 5.3.2. Also, similar issues to the spaital access point arise (With structured data, as opposed to text queries). In some UK datasets, periods such as "Neolithic" can be used instead of an ISO 19108 Date Time. (I seen note 11 under 5.3.4 talks about this, which is good. Whats important is that regardless of the outcome of the study, the IR are extensible enough to cope with the eventual decision). I'd consider seperate access points for controlled vocabulary time period and structured temporal data. This seems a specific example where the abstract IR model needs to go beyond what is defined in the A2 binding.

Geographic Extent.. the doc seems a bit bounding box heavy. Would be nice to understand (have examples of) specification of interior/exterioir polygons. Servers only supporting minimal bounding boxes can gracefully degrade (Since it's easy to calculate a MBR from a polygon) whilst allowing other servers to retain the full richness of polygons. It's not clear (in the abstract model, it is in the A2 binding) where the semantics for parsing these strings will be defined.. for example should geographic extent be encoded as OpenGIS strings (Which seems to make sense to me, but I'm biased by Oracle and MySQL's spatial functions). This might seem a bit extreme for the abstract part of the document, but it's one of those make-or-break issues for interoperability, and might be worth the pain. Also, I think it's worth entertaining the idea that spatial specifications such as MBRs and polygons (Structured spatial constructs) might be better exposed using their own abstract access point, and "Place Name" having it's own access point. This will help server implementors avoid problems with disambiguation of search terms.

I'm interested in what the expected semantics of resource language are on retrieval of language-neutral data sets.... Should a result record not be selected because the user specified "Nor" as the search language, but resources matching other criteria (Geo Extent for example) do match. Normally in Info Retrieval this is a no-brainer, of course it shouldn't, but I'm a bit less certain when we talk about result records that aren't primarily "Text" based. (Actually, this is a slightly wider concernn about annex A and those "CharacterString" elements... In IEEE LOM for example we have "LangString" element that has a "Lang" attribute. That community chose to allow language variants of a resource to be expressed within one record by allowing an element to hold all language variants, for example
<pre>
<title>
<langstring lang="En">Hello</langstring>
<langstring lang="Dk">Hej</langstring>
</title>
</pre>
The presence of a "Lang" attribute at the "Dataset" level might mean the intention is to support multi-language datasets by having several dataset records, one for each language, which is OK, but possibly not optimal for datasets that aren't prmarily language based. If this is the case, is the "CharacterString" element in Annex A just redundant payload?)

=== Lack of machine-reusable data in general ===

Dataset 'lineage' is only a full-text field. If datasets result from recombination, that should be machine-traversable. Human descriptions of lineage will be so different that they won't be useful for building search / evaluation services.

II: It does tend to talk about "Lineage statement"... would making it (More along the conceptual lines of)
<pre>
<lineage>
<dc:description>Text</dc:description>
</lineage>
</pre>
Give you the extensibility to either use private extensions, or to specify recombination elements at a later date (I didn't think this through in terms of the *actual* recombination operations, just wanted to show how we might make lineage extensible without specifying it.
<pre>
<lineage>
<dc:description>This dataset is a recombination of X and Y</dc:description>
<Jo:recombination>
<Jo:source>X-URI</Jo:source>
<Jo:source>Y-URI</Jo:source>
<Jo:Rules>Overlap</Jo:Rules>
<Jo:recombination>
</lineage>
</pre>
Should the lineage search point be called "LineageDescription" (I think thats what I'll do in my SRW profile).

=== Lack of engagement with packaging and re-use issues ===

Cf. Dataset series / aggregates. The examples have 'MasterMap' as one potential dataset! Real world use cases are going to need subsets of such huge data sets broken down into packages with smaller spatial extents or with less layers.

II: Indeed, as well as the srw/sru binding experiment, I've been wondering about the OAI binding, which I know you've already discussed elsewhere. What might be generally useful (And maybe this already exists) is a set of TREC style test data. Setting up a static gateway OAI server wouldn't be so hard, and might give us some valuable real-world information about this problem. I now the records won't be in the right schema, but something we can try and munge into gmd would be a real help.

=== Bypassing of feature-level metadata from consideration ===

Once we get down to the feature level the interesting European problems appear - the fact that every local area may have its own classification schemes, even inside one language community the same word is used to describe different looking things, and across language barriers mappings from words to things don't tend to be 1-1. But by disregarding feature-level metadata - partly because it can't be mandated when the underlying geospatial objects aren't publically inspectable and a certain amount of feature level metadata would mean the data itself is essentially public...

II: Aye, generally for discovery services it's nice to try and avoid mandating that users understand predefined controlled vocabularies, whilst allowing users who do know terms to qualify their discovery process, for example, in CQL I'd be tempted to allow a user to say "dc.subject=Something" or (The equivalent of) "authority=19115:2003 and dc.subject=Something" for users who know a specific term.

There's quite a lot of work going on around europe at the moment covering crosswalks of controlled vocabularies (Mostly I know about crosswalking euroopean educational levels, but It seems to be the same problem cast in a different way). If we can arrange for someone to do the intellectual work of cross-mapping, and make the data publically available, then it becomes a "Turning-the-handle" job for providers to support cross vocab retrieval. Standards such as ZThes are being used quite a lot in the learning domain to transport this data around. The only effect on reviwing the IR is that it's important that the IR does not preclude this at a future date? (The whole design for unforseen use thing.. specifically, I think mandating a specific vocab in the IR might not be the right thing to do, and giving users a way to say which vocab they are using in the description and discovery process is a better way to go....)

=== Overspecificness about internet- and webservices- based distribution models ===

Actually causing ourselves unnesc problems by putting everything on the Internet. Data sharing agreements over publically maintained private networks with flat-rate membership are a clear potential future and 'middle way' in this domain. The draft now is all about making access/use contraints *specific to data sets* and not specific to the relationship between the data provider or broker, the data user and the transport network between them.

So we have a 'distributed computing platform' metadata property that is required by the IRs. In the ISO19915 mapping in Annex A this is a '''free text field''', yet 5.2.15 states that the property "is necessary for a client to bind to the service". If it must be mandated, it should be as a URI. It would be wonderful to have examples of what other than HTTP or OGC web services is envisaged NOW as a means of access to the backend of a distributed computing platform.

= Feelings =
== Too general ==
I am now finishing the reading and my first feeling from the IR document is that there are some parts too general and not helpful for implementation. It is like ISO 19115 or CSW 2.0. When you start to implement it you have to find your own way how to do it. This will probably lead in inconsistency of catalogues interfaces and content. I know that something is better than nothing and I welcome INSPIRE metadata IR, but I know that there should be specified more that is in this document. We are now testing CSW 2.0 implementation in GeoNetwork Open Source and we have found a lot of problematic parts (in implementation and in specification as well). I have some concrete comments to IR metadata, but I have to sumarised them first to publish them here. Some of them are probably mentioned here (in that case I will just comment that I agree with mentioned issues).

== List of issues ==
I have tried to put what I have find it the IR document to this page, but you can find different form of structure in http://gis.vsb.cz/ruzicka/down/ruzne/inspire/CommentsMetadata.pdf

Reading the INSPIRE Metadata Draft

2007-03-24T22:05:58Z

Wiki-Jencek: /* Areas which are unclear */

Reading the INSPIRE Metadata Draft

2007-03-24T22:02:44Z

Wiki-Jencek: /* Things that are there that probably shouldn't be */

Metadata about geographic data is at the heart of INSPIRE. The metadata draft is the first in the set of "implementing rules" and it will underpin all the other implementing rules. The consultation process is open until 2007-03-30. While the documents are open access, comments can only be offered through an SDIC or Spatial Data Interest Community.

The Free and Open Source Geospatial Community has a voice through one of these SDICs thanks to Markus Neteler. This page contains preparatory material for a collective response through the FOSS GIS SDIC, from the POV of people implementing and managing metadata creation, collection and search services, working closely with many different data user communities.

* The response proper will live at [[Response to INSPIRE Metadata Draft]]. Initial notes are included below in the Issues section.
* It is interesting to read this in parallel with the North American Metadata Profile draft which is also currently in consultation. It's hoped the OSGeo community will also be able to contribute to a [[Response to NAP Metadata Draft]] and get the [http://geodatacommons.umaine.edu geodata commons] project involved in this.

== Reading the draft ==

* [http://www.ec-gis.org/inspire/reports/ImplementingRules/draftINSPIREMetadataIRv2_20070202.pdf the Implementing Rules for Metadata Draft] (pdf)
* [http://www.ec-gis.org/inspire/whatsnew.cfm#1590 supporting / background material]
* Pages 1-17 are metadata about the document itself, intentions and history, and can be safely skipped. Pages 43-104 are the Annexes.
* Annex A is particularly interesting as there are details of the thinking exposed in the mapping to ISO19115/39 that aren't set out in the implementing rules. ''If you want to know what's likely to affect you but are short on time, at minimum read section 5 and Annex A'''.

== Lightning Summary of the draft ==

The draft establishes a basic information model for metadata which is close to, but not specific to, ISO19115 and OGC Web Services.

It only mandates what metadata is published by and for public authorities covered by INSPIRE - it does not try to cover repository management or internal processes.

It separates out metadata properties into those useful for 'discovery', 'evaluation', and 'use'. It identifies one very high level "use case" for spatial data search services built from metadata being shared at this level.

It differentiates between properties useful for 'non-specialist' and 'expert' users into 2 Levels, 1 and 2. Level 1 is always mandatory. This *includes* classification according to the data themes in the INSPIRE annexes, and keywords from controlled vocabularies which are not covered by the IR document but are left to Spatial Data Theme Communities. (How these communities are found, selected, and make their decisions, is unknown to us at this time.)

= Issues =

'''This list is an overview of what jumped out at me as something to address. I don't know how much of this is appropriate to send back, or how much can be fixed. - [[User:JoWalsh]]''' I've added some random musings also, again, no idea if they are of use or even valid questions, but at least they are there and can be edited out, or used as the basis of discussions - [[User:IanIbbo]].

== Conceptual overview ==

The model maps quite well to the minimum useful subset identified in [[DCLite4G]]. It looks like a lightweight core. But, the model and the draft break down the problem space of metadata in a way that is a reaction to artificial scarcity of data. It identifies three phases of the metadata use cycle:
* discovery (of what data is out there)
* evaluation (of whether the data will be useful for specific purpose)
* use (once access gained, how to best use the data)

It is illuminates to compare this with the [[Reading the NAP Metadata Draft|North American Profile]] metadata draft which talks about

* discovery
* access
* fitness for use (e.g. evaluation)
* transfer

So the IRs both don't address how to make the data more useful via metadata, and are vague about how much a minimal subset is going to provide enough information to evaluate utility on. Generally the draft dances around data licensing access issues, and glosses over the over-engineering needed to work around artificial constraints on availability. IRs for evaluation and use of data based on metadata are not covered by this draft at all, but left up to the Spatial Data Theme communities for each of the 35 data themes identified in Annexes I-III of the INSPIRE text.

== Issues with specific metadata properties ==

The model maps quite well to [[DCLite4G]]. It looks like a lightweight core.

=== Things that aren't there that should be ===

'''5.2.8 Resource responsible party'''. Each dataset *must* have one or more people/organisations responsible for it. The IR says that this can be freetext or can be in more structured form. This '''only''' includes the responsible party's name, but NOT any form of contact details.

'''Some form of electronic or telephonic contact address should be mandatory, if the org/person's details are mandatory.''' Why publish ownership information - especially if there are constraints on access and reuse of the described data - if you can't immediately get in personal contact with someone who can make assurances about the data?

I agree, but not only for data, btu for metadata as well:
Page 53. sec A.2.8.1 and A2.8.2 and A.4.1: Other information for identification is necessary like e-mail, URL, voice number.

Annex A on mapping to IS019115 mandates that contact persons and organisations be free text, not resource identifiers. 2 serious problems with the ISO 19115 mapping:

* It does not ask or provide for contact details.
* It looks *mandatory* that the reponsible party be given a role, which in turn is one of N codes published by the Library of Congress to describe people's roles within organisations.

'''No discussion of formalising dataset accuracy / completeness - crucial for cost-benefit evaluation / evaluation of suitability for combining with other data sets.'''

I agree:
Page 31. sec 5.2.12: The lineage element is not enough for data evaluation. The same problem is on Page 55. sec A.3.2.

* Page 7. par. 6: "Separate IRs for discovery services are being prepared and are not the subject of this document". But I have found a lot of information about discovery (search) services in this document and I think that information about discovery services should be a part of this document. There is a conflict. We can not be sure how to evaluate this document when we do not know how will look IR for services.
* Page 17. sec. 4.1. Note 6, par 4 and 5: This part of service evaluation must be solved for the infrastructure and the INSPIRE must give IR for this part of the infrastructure. Evaluation of the service must be based on the purpose of the usage. I believe that case studies support system can help with it. There is a gap in IR metadata from this point of view.
* Page 22. sec 4.6., Par 6: "The content of these repositories need to be accessible. This should happen preferably via a standard interface and/or standard encoding format such as XML". Must be specified how, if there is not exact information how to connect then is not useful for implementation.
* Page 30. sec 5.2.7.: There is not a list of types. The INSPIRE should define authority that can define types of services. For example you have in examples an identifier WMS, but we use an identifier OGC:WMS in our catalogue. This can lead to inconsistency.
* Page 30. sec 5.2.3.: There is vertical extend missing, necessary for geology, meteorology and climatology studies.
* Page 57. sec A.3.6: Code List is good, but I can not find it in the document.
* Page 21. sec 4.6. Par 1 (numbering 1): "Search engine connected to a set of metadata repositories". How the repositories will be connected, how selected, real-time, replication, harvesting. There is no information about it in the document here on later in the text, but they should be defined.
* Page 8:There is no information about cooperation with FGDC, FAO or other subjects that should be later connected to the INSPIRE infrastructure. Does it mean that EU want to stay out of USA and UN infrastructures?
* Page 64. sec B.8: Resource provider is not defined. Why not dc: publisher?
* Page 77. Annex G: There is not information about Usage of a resource that is very necessary to evaluate quality of a resource.
* Page 101. Annex I: There are no guidelines, just some unclear information.

=== Things that are there that probably shouldn't be ===

Every 'dataset or dataset series' published under INSPIRE *must* include both a '''Resource topic category''' and a set of resource '''keywords'''.

Topic categories are very high-level classifications which correspond to each of the Spatial Data Themes identified in Annexes I, II and III of the INSPIRE Directive.

* Which topic category data fits in will often be a property of an organisation not any published data sets.

From an implementor's POV this will involve something like selecting a topic category for data at install time of metadata publishing engine, and forgetting about it. The IRs place a lot of faith in the ability of simple keyword / classification code matches to enhance utility of search and discovery services for users.

But. this already raises the bar for non-expert users (the domain vocabulary is jargon specific or oriented towards specialist codes)

The IRs emphasise the fact that keywords should originate from a ''controlled vocabulary''. The reponsibility for creating one is not in the hands of the Drafting Teams but in the hands of Spatial Data Theme Communities. How these are constituted and how their decisions become binding are unclear.

Again, faith in keywords for search utility is misplaced. Reliance on them may lead to false negatives. Again assumes familiarity with, or time and ability to learn about, what to expect in the domain from a non-expert user, and an expert will need a better level of detail. Pitfalls of 'controlled' keywording:
* intentional misclassification
* lazy/default misclassification

Both of these are at 'Level 1 for discovery metadata' which implies that any INSPIRE compliant metadata set MUST have both topic category and associated keywords.

I partly agree: It is true that for INSPIRE should be defined thesaurus for keywords. INSPIRE is mainly oriented to environment protection. There is GEMET thesaurus, but there is not a common agreement to use it. This part of metadata is not so simple and I understand that definition of a IR for this part is very difficult.

Other issues:
* Page 60. sec B.2.1: Temporal extent is meaningful always for discovering and searching.
* Page 85. sec H.5.2: Too difficult for implementation and not very useful, there are other ways how to support multilingual descriptions.
* Page 38. sec 6.2: Identifier are of two types. This is problematic. It is usually better to have only one type of identifier. The preferred one should be URL (URI). The document says that URL can be in some cases not unique. If you want to make identification more unique than mix URL with UUID. Like this: http://gis.vsb.cz/01f8da38-10d7-11da-b569-000f1f1a7b03

== Areas which are unclear ==

=== Conformity ===

This is an IR and obligatory to deal with. But 5.3.4 just says "see Annex F".
Annex F '''in its entirety''' says:

The way in which conformity is expressed in the INSPIRE IR will be defined in a subsequent draft based on discussions with the Drafting Team on Data specifications and harmonization.

(Is this where accuracy/completeness comes in? How can we know?)

=== Dataset series / Aggregate data ===

IR talks about dataset series. Some of the diagrams talk about 'MD_Aggregates'- this term isn't used elsewhere. No conception in this model of one UrDataSet with many different potential sources according to how they are packaged or processed. As the IRs mandate properties for dataset series, really need more clarity / examples about what they actually are.

== General concerns ==

=== Search / discovery services ===

The preamble (p.7) states that "separate IRs for discovery services are being prepared and are not the subject of this document." But the INSPIRE use case is predicated on the availablity of 'Geoportal' style search services. What else *are* discovery services if they are not the search services treated of here? If there is only going to be an abstract model for discovery, and these IRs are careful to avoid imposing any constraints on internal data repository management, how much more can a discovery services draft provide?

II: I think this observation is spot on, but for different reasons perhaps. I'm finding it difficult to express concrete concerns.. but Section 5.2 "Discovery metadata elements" starts to set out a list of concepts seen to be (The document hints at, but does not directly say) core to the discovery process. Section 5.3 then sets out "Abstract discovery metadata element set". I *guess* the implication is that the concepts laid out in 5.2 are in some way even more abstract than those set out in 5.3. The document really isn't clear about what the abstract model is, or what it is for, before it starts enumerating the concepts. Your later comment about being tied to web services is spot on also here, I'm really not sure "Service type version", "Operation name" and "Distributed computing platform" belong in an abstract discovery model (The probably *do* belong in some result record schema). These three attributes seem to belong specifically to a particular (And I would guess already existing) service binding (Or as already said, to a very specific kind of returned result record). What I'd really like to see is a much clearer statement of what the purpose of the abstract discovery model is. Hopefully, once that is tightly defined, it should become easier to decide what lies inside the boundary of the abstract model, and what belongs in the domain of specific realisations of the abstract model. (Actually.. I should say that I'm baised by the information retrieval community generally, in that it's considered really important to have a seperate abstract model for discovery (The search access points) and then bind that model on to as many backend schemas as needed.. this decoupling is seen as best practice in the information retrival domain, and most of my concerns here are that because of the apparent 1:1 mapping between the abstract model and the implementation. This is the approach taken in the [[Z3950 GEO profile] http://www.blueangeltech.com/standards/GeoProfile/geo22.htm]).

I'm a bit confused by the "Temporal Reference" Element... 5.2.2. Talks about what I would expect to see from a temporal reference, but 5.3.2 maps temporal reference on to "One of the dates of publication, last revision or creation of the resource". These three elements are already well defined by dublin core attributes... Maybe I've misunderstood whats implied by table 1 in 5.3.2. Also, similar issues to the spaital access point arise (With structured data, as opposed to text queries). In some UK datasets, periods such as "Neolithic" can be used instead of an ISO 19108 Date Time. (I seen note 11 under 5.3.4 talks about this, which is good. Whats important is that regardless of the outcome of the study, the IR are extensible enough to cope with the eventual decision). I'd consider seperate access points for controlled vocabulary time period and structured temporal data. This seems a specific example where the abstract IR model needs to go beyond what is defined in the A2 binding.

Geographic Extent.. the doc seems a bit bounding box heavy. Would be nice to understand (have examples of) specification of interior/exterioir polygons. Servers only supporting minimal bounding boxes can gracefully degrade (Since it's easy to calculate a MBR from a polygon) whilst allowing other servers to retain the full richness of polygons. It's not clear (in the abstract model, it is in the A2 binding) where the semantics for parsing these strings will be defined.. for example should geographic extent be encoded as OpenGIS strings (Which seems to make sense to me, but I'm biased by Oracle and MySQL's spatial functions). This might seem a bit extreme for the abstract part of the document, but it's one of those make-or-break issues for interoperability, and might be worth the pain. Also, I think it's worth entertaining the idea that spatial specifications such as MBRs and polygons (Structured spatial constructs) might be better exposed using their own abstract access point, and "Place Name" having it's own access point. This will help server implementors avoid problems with disambiguation of search terms.

I'm interested in what the expected semantics of resource language are on retrieval of language-neutral data sets.... Should a result record not be selected because the user specified "Nor" as the search language, but resources matching other criteria (Geo Extent for example) do match. Normally in Info Retrieval this is a no-brainer, of course it shouldn't, but I'm a bit less certain when we talk about result records that aren't primarily "Text" based. (Actually, this is a slightly wider concernn about annex A and those "CharacterString" elements... In IEEE LOM for example we have "LangString" element that has a "Lang" attribute. That community chose to allow language variants of a resource to be expressed within one record by allowing an element to hold all language variants, for example
<pre>
<title>
<langstring lang="En">Hello</langstring>
<langstring lang="Dk">Hej</langstring>
</title>
</pre>
The presence of a "Lang" attribute at the "Dataset" level might mean the intention is to support multi-language datasets by having several dataset records, one for each language, which is OK, but possibly not optimal for datasets that aren't prmarily language based. If this is the case, is the "CharacterString" element in Annex A just redundant payload?)

=== Lack of machine-reusable data in general ===

Dataset 'lineage' is only a full-text field. If datasets result from recombination, that should be machine-traversable. Human descriptions of lineage will be so different that they won't be useful for building search / evaluation services.

II: It does tend to talk about "Lineage statement"... would making it (More along the conceptual lines of)
<pre>
<lineage>
<dc:description>Text</dc:description>
</lineage>
</pre>
Give you the extensibility to either use private extensions, or to specify recombination elements at a later date (I didn't think this through in terms of the *actual* recombination operations, just wanted to show how we might make lineage extensible without specifying it.
<pre>
<lineage>
<dc:description>This dataset is a recombination of X and Y</dc:description>
<Jo:recombination>
<Jo:source>X-URI</Jo:source>
<Jo:source>Y-URI</Jo:source>
<Jo:Rules>Overlap</Jo:Rules>
<Jo:recombination>
</lineage>
</pre>
Should the lineage search point be called "LineageDescription" (I think thats what I'll do in my SRW profile).

=== Lack of engagement with packaging and re-use issues ===

Cf. Dataset series / aggregates. The examples have 'MasterMap' as one potential dataset! Real world use cases are going to need subsets of such huge data sets broken down into packages with smaller spatial extents or with less layers.

II: Indeed, as well as the srw/sru binding experiment, I've been wondering about the OAI binding, which I know you've already discussed elsewhere. What might be generally useful (And maybe this already exists) is a set of TREC style test data. Setting up a static gateway OAI server wouldn't be so hard, and might give us some valuable real-world information about this problem. I now the records won't be in the right schema, but something we can try and munge into gmd would be a real help.

=== Bypassing of feature-level metadata from consideration ===

Once we get down to the feature level the interesting European problems appear - the fact that every local area may have its own classification schemes, even inside one language community the same word is used to describe different looking things, and across language barriers mappings from words to things don't tend to be 1-1. But by disregarding feature-level metadata - partly because it can't be mandated when the underlying geospatial objects aren't publically inspectable and a certain amount of feature level metadata would mean the data itself is essentially public...

II: Aye, generally for discovery services it's nice to try and avoid mandating that users understand predefined controlled vocabularies, whilst allowing users who do know terms to qualify their discovery process, for example, in CQL I'd be tempted to allow a user to say "dc.subject=Something" or (The equivalent of) "authority=19115:2003 and dc.subject=Something" for users who know a specific term.

There's quite a lot of work going on around europe at the moment covering crosswalks of controlled vocabularies (Mostly I know about crosswalking euroopean educational levels, but It seems to be the same problem cast in a different way). If we can arrange for someone to do the intellectual work of cross-mapping, and make the data publically available, then it becomes a "Turning-the-handle" job for providers to support cross vocab retrieval. Standards such as ZThes are being used quite a lot in the learning domain to transport this data around. The only effect on reviwing the IR is that it's important that the IR does not preclude this at a future date? (The whole design for unforseen use thing.. specifically, I think mandating a specific vocab in the IR might not be the right thing to do, and giving users a way to say which vocab they are using in the description and discovery process is a better way to go....)

=== Overspecificness about internet- and webservices- based distribution models ===

Actually causing ourselves unnesc problems by putting everything on the Internet. Data sharing agreements over publically maintained private networks with flat-rate membership are a clear potential future and 'middle way' in this domain. The draft now is all about making access/use contraints *specific to data sets* and not specific to the relationship between the data provider or broker, the data user and the transport network between them.

So we have a 'distributed computing platform' metadata property that is required by the IRs. In the ISO19915 mapping in Annex A this is a '''free text field''', yet 5.2.15 states that the property "is necessary for a client to bind to the service". If it must be mandated, it should be as a URI. It would be wonderful to have examples of what other than HTTP or OGC web services is envisaged NOW as a means of access to the backend of a distributed computing platform.

= Feelings =
== Too general ==
I am now finishing the reading and my first feeling from the IR document is that there are some parts too general and not helpful for implementation. It is like ISO 19115 or CSW 2.0. When you start to implement it you have to find your own way how to do it. This will probably lead in inconsistency of catalogues interfaces and content. I know that something is better than nothing and I welcome INSPIRE metadata IR, but I know that there should be specified more that is in this document. We are now testing CSW 2.0 implementation in GeoNetwork Open Source and we have found a lot of problematic parts (in implementation and in specification as well). I have some concrete comments to IR metadata, but I have to sumarised them first to publish them here. Some of them are probably mentioned here (in that case I will just comment that I agree with mentioned issues).

Reading the INSPIRE Metadata Draft

2007-03-24T21:25:01Z

Wiki-Jencek: /* Things that aren't there that should be */

Metadata about geographic data is at the heart of INSPIRE. The metadata draft is the first in the set of "implementing rules" and it will underpin all the other implementing rules. The consultation process is open until 2007-03-30. While the documents are open access, comments can only be offered through an SDIC or Spatial Data Interest Community.

The Free and Open Source Geospatial Community has a voice through one of these SDICs thanks to Markus Neteler. This page contains preparatory material for a collective response through the FOSS GIS SDIC, from the POV of people implementing and managing metadata creation, collection and search services, working closely with many different data user communities.

* The response proper will live at [[Response to INSPIRE Metadata Draft]]. Initial notes are included below in the Issues section.
* It is interesting to read this in parallel with the North American Metadata Profile draft which is also currently in consultation. It's hoped the OSGeo community will also be able to contribute to a [[Response to NAP Metadata Draft]] and get the [http://geodatacommons.umaine.edu geodata commons] project involved in this.

== Reading the draft ==

* [http://www.ec-gis.org/inspire/reports/ImplementingRules/draftINSPIREMetadataIRv2_20070202.pdf the Implementing Rules for Metadata Draft] (pdf)
* [http://www.ec-gis.org/inspire/whatsnew.cfm#1590 supporting / background material]
* Pages 1-17 are metadata about the document itself, intentions and history, and can be safely skipped. Pages 43-104 are the Annexes.
* Annex A is particularly interesting as there are details of the thinking exposed in the mapping to ISO19115/39 that aren't set out in the implementing rules. ''If you want to know what's likely to affect you but are short on time, at minimum read section 5 and Annex A'''.

== Lightning Summary of the draft ==

The draft establishes a basic information model for metadata which is close to, but not specific to, ISO19115 and OGC Web Services.

It only mandates what metadata is published by and for public authorities covered by INSPIRE - it does not try to cover repository management or internal processes.

It separates out metadata properties into those useful for 'discovery', 'evaluation', and 'use'. It identifies one very high level "use case" for spatial data search services built from metadata being shared at this level.

It differentiates between properties useful for 'non-specialist' and 'expert' users into 2 Levels, 1 and 2. Level 1 is always mandatory. This *includes* classification according to the data themes in the INSPIRE annexes, and keywords from controlled vocabularies which are not covered by the IR document but are left to Spatial Data Theme Communities. (How these communities are found, selected, and make their decisions, is unknown to us at this time.)

= Issues =

'''This list is an overview of what jumped out at me as something to address. I don't know how much of this is appropriate to send back, or how much can be fixed. - [[User:JoWalsh]]''' I've added some random musings also, again, no idea if they are of use or even valid questions, but at least they are there and can be edited out, or used as the basis of discussions - [[User:IanIbbo]].

== Conceptual overview ==

The model maps quite well to the minimum useful subset identified in [[DCLite4G]]. It looks like a lightweight core. But, the model and the draft break down the problem space of metadata in a way that is a reaction to artificial scarcity of data. It identifies three phases of the metadata use cycle:
* discovery (of what data is out there)
* evaluation (of whether the data will be useful for specific purpose)
* use (once access gained, how to best use the data)

It is illuminates to compare this with the [[Reading the NAP Metadata Draft|North American Profile]] metadata draft which talks about

* discovery
* access
* fitness for use (e.g. evaluation)
* transfer

So the IRs both don't address how to make the data more useful via metadata, and are vague about how much a minimal subset is going to provide enough information to evaluate utility on. Generally the draft dances around data licensing access issues, and glosses over the over-engineering needed to work around artificial constraints on availability. IRs for evaluation and use of data based on metadata are not covered by this draft at all, but left up to the Spatial Data Theme communities for each of the 35 data themes identified in Annexes I-III of the INSPIRE text.

== Issues with specific metadata properties ==

The model maps quite well to [[DCLite4G]]. It looks like a lightweight core.

=== Things that aren't there that should be ===

'''5.2.8 Resource responsible party'''. Each dataset *must* have one or more people/organisations responsible for it. The IR says that this can be freetext or can be in more structured form. This '''only''' includes the responsible party's name, but NOT any form of contact details.

'''Some form of electronic or telephonic contact address should be mandatory, if the org/person's details are mandatory.''' Why publish ownership information - especially if there are constraints on access and reuse of the described data - if you can't immediately get in personal contact with someone who can make assurances about the data?

I agree, but not only for data, btu for metadata as well:
Page 53. sec A.2.8.1 and A2.8.2 and A.4.1: Other information for identification is necessary like e-mail, URL, voice number.

Annex A on mapping to IS019115 mandates that contact persons and organisations be free text, not resource identifiers. 2 serious problems with the ISO 19115 mapping:

* It does not ask or provide for contact details.
* It looks *mandatory* that the reponsible party be given a role, which in turn is one of N codes published by the Library of Congress to describe people's roles within organisations.

'''No discussion of formalising dataset accuracy / completeness - crucial for cost-benefit evaluation / evaluation of suitability for combining with other data sets.'''

I agree:
Page 31. sec 5.2.12: The lineage element is not enough for data evaluation. The same problem is on Page 55. sec A.3.2.

* Page 7. par. 6: "Separate IRs for discovery services are being prepared and are not the subject of this document". But I have found a lot of information about discovery (search) services in this document and I think that information about discovery services should be a part of this document. There is a conflict. We can not be sure how to evaluate this document when we do not know how will look IR for services.
* Page 17. sec. 4.1. Note 6, par 4 and 5: This part of service evaluation must be solved for the infrastructure and the INSPIRE must give IR for this part of the infrastructure. Evaluation of the service must be based on the purpose of the usage. I believe that case studies support system can help with it. There is a gap in IR metadata from this point of view.
* Page 22. sec 4.6., Par 6: "The content of these repositories need to be accessible. This should happen preferably via a standard interface and/or standard encoding format such as XML". Must be specified how, if there is not exact information how to connect then is not useful for implementation.
* Page 30. sec 5.2.7.: There is not a list of types. The INSPIRE should define authority that can define types of services. For example you have in examples an identifier WMS, but we use an identifier OGC:WMS in our catalogue. This can lead to inconsistency.
* Page 30. sec 5.2.3.: There is vertical extend missing, necessary for geology, meteorology and climatology studies.
* Page 57. sec A.3.6: Code List is good, but I can not find it in the document.
* Page 21. sec 4.6. Par 1 (numbering 1): "Search engine connected to a set of metadata repositories". How the repositories will be connected, how selected, real-time, replication, harvesting. There is no information about it in the document here on later in the text, but they should be defined.
* Page 8:There is no information about cooperation with FGDC, FAO or other subjects that should be later connected to the INSPIRE infrastructure. Does it mean that EU want to stay out of USA and UN infrastructures?
* Page 64. sec B.8: Resource provider is not defined. Why not dc: publisher?
* Page 77. Annex G: There is not information about Usage of a resource that is very necessary to evaluate quality of a resource.
* Page 101. Annex I: There are no guidelines, just some unclear information.

=== Things that are there that probably shouldn't be ===

Every 'dataset or dataset series' published under INSPIRE *must* include both a '''Resource topic category''' and a set of resource '''keywords'''.

Topic categories are very high-level classifications which correspond to each of the Spatial Data Themes identified in Annexes I, II and III of the INSPIRE Directive.

* Which topic category data fits in will often be a property of an organisation not any published data sets.

From an implementor's POV this will involve something like selecting a topic category for data at install time of metadata publishing engine, and forgetting about it. The IRs place a lot of faith in the ability of simple keyword / classification code matches to enhance utility of search and discovery services for users.

But. this already raises the bar for non-expert users (the domain vocabulary is jargon specific or oriented towards specialist codes)

The IRs emphasise the fact that keywords should originate from a ''controlled vocabulary''. The reponsibility for creating one is not in the hands of the Drafting Teams but in the hands of Spatial Data Theme Communities. How these are constituted and how their decisions become binding are unclear.

Again, faith in keywords for search utility is misplaced. Reliance on them may lead to false negatives. Again assumes familiarity with, or time and ability to learn about, what to expect in the domain from a non-expert user, and an expert will need a better level of detail. Pitfalls of 'controlled' keywording:
* intentional misclassification
* lazy/default misclassification

Both of these are at 'Level 1 for discovery metadata' which implies that any INSPIRE compliant metadata set MUST have both topic category and associated keywords.

== Areas which are unclear ==

=== Conformity ===

This is an IR and obligatory to deal with. But 5.3.4 just says "see Annex F".
Annex F '''in its entirety''' says:

The way in which conformity is expressed in the INSPIRE IR will be defined in a subsequent draft based on discussions with the Drafting Team on Data specifications and harmonization.

(Is this where accuracy/completeness comes in? How can we know?)

=== Dataset series / Aggregate data ===

IR talks about dataset series. Some of the diagrams talk about 'MD_Aggregates'- this term isn't used elsewhere. No conception in this model of one UrDataSet with many different potential sources according to how they are packaged or processed. As the IRs mandate properties for dataset series, really need more clarity / examples about what they actually are.

== General concerns ==

=== Search / discovery services ===

The preamble (p.7) states that "separate IRs for discovery services are being prepared and are not the subject of this document." But the INSPIRE use case is predicated on the availablity of 'Geoportal' style search services. What else *are* discovery services if they are not the search services treated of here? If there is only going to be an abstract model for discovery, and these IRs are careful to avoid imposing any constraints on internal data repository management, how much more can a discovery services draft provide?

II: I think this observation is spot on, but for different reasons perhaps. I'm finding it difficult to express concrete concerns.. but Section 5.2 "Discovery metadata elements" starts to set out a list of concepts seen to be (The document hints at, but does not directly say) core to the discovery process. Section 5.3 then sets out "Abstract discovery metadata element set". I *guess* the implication is that the concepts laid out in 5.2 are in some way even more abstract than those set out in 5.3. The document really isn't clear about what the abstract model is, or what it is for, before it starts enumerating the concepts. Your later comment about being tied to web services is spot on also here, I'm really not sure "Service type version", "Operation name" and "Distributed computing platform" belong in an abstract discovery model (The probably *do* belong in some result record schema). These three attributes seem to belong specifically to a particular (And I would guess already existing) service binding (Or as already said, to a very specific kind of returned result record). What I'd really like to see is a much clearer statement of what the purpose of the abstract discovery model is. Hopefully, once that is tightly defined, it should become easier to decide what lies inside the boundary of the abstract model, and what belongs in the domain of specific realisations of the abstract model. (Actually.. I should say that I'm baised by the information retrieval community generally, in that it's considered really important to have a seperate abstract model for discovery (The search access points) and then bind that model on to as many backend schemas as needed.. this decoupling is seen as best practice in the information retrival domain, and most of my concerns here are that because of the apparent 1:1 mapping between the abstract model and the implementation. This is the approach taken in the [[Z3950 GEO profile] http://www.blueangeltech.com/standards/GeoProfile/geo22.htm]).

I'm a bit confused by the "Temporal Reference" Element... 5.2.2. Talks about what I would expect to see from a temporal reference, but 5.3.2 maps temporal reference on to "One of the dates of publication, last revision or creation of the resource". These three elements are already well defined by dublin core attributes... Maybe I've misunderstood whats implied by table 1 in 5.3.2. Also, similar issues to the spaital access point arise (With structured data, as opposed to text queries). In some UK datasets, periods such as "Neolithic" can be used instead of an ISO 19108 Date Time. (I seen note 11 under 5.3.4 talks about this, which is good. Whats important is that regardless of the outcome of the study, the IR are extensible enough to cope with the eventual decision). I'd consider seperate access points for controlled vocabulary time period and structured temporal data. This seems a specific example where the abstract IR model needs to go beyond what is defined in the A2 binding.

Geographic Extent.. the doc seems a bit bounding box heavy. Would be nice to understand (have examples of) specification of interior/exterioir polygons. Servers only supporting minimal bounding boxes can gracefully degrade (Since it's easy to calculate a MBR from a polygon) whilst allowing other servers to retain the full richness of polygons. It's not clear (in the abstract model, it is in the A2 binding) where the semantics for parsing these strings will be defined.. for example should geographic extent be encoded as OpenGIS strings (Which seems to make sense to me, but I'm biased by Oracle and MySQL's spatial functions). This might seem a bit extreme for the abstract part of the document, but it's one of those make-or-break issues for interoperability, and might be worth the pain. Also, I think it's worth entertaining the idea that spatial specifications such as MBRs and polygons (Structured spatial constructs) might be better exposed using their own abstract access point, and "Place Name" having it's own access point. This will help server implementors avoid problems with disambiguation of search terms.

I'm interested in what the expected semantics of resource language are on retrieval of language-neutral data sets.... Should a result record not be selected because the user specified "Nor" as the search language, but resources matching other criteria (Geo Extent for example) do match. Normally in Info Retrieval this is a no-brainer, of course it shouldn't, but I'm a bit less certain when we talk about result records that aren't primarily "Text" based. (Actually, this is a slightly wider concernn about annex A and those "CharacterString" elements... In IEEE LOM for example we have "LangString" element that has a "Lang" attribute. That community chose to allow language variants of a resource to be expressed within one record by allowing an element to hold all language variants, for example
<pre>
<title>
<langstring lang="En">Hello</langstring>
<langstring lang="Dk">Hej</langstring>
</title>
</pre>
The presence of a "Lang" attribute at the "Dataset" level might mean the intention is to support multi-language datasets by having several dataset records, one for each language, which is OK, but possibly not optimal for datasets that aren't prmarily language based. If this is the case, is the "CharacterString" element in Annex A just redundant payload?)

=== Lack of machine-reusable data in general ===

Dataset 'lineage' is only a full-text field. If datasets result from recombination, that should be machine-traversable. Human descriptions of lineage will be so different that they won't be useful for building search / evaluation services.

II: It does tend to talk about "Lineage statement"... would making it (More along the conceptual lines of)
<pre>
<lineage>
<dc:description>Text</dc:description>
</lineage>
</pre>
Give you the extensibility to either use private extensions, or to specify recombination elements at a later date (I didn't think this through in terms of the *actual* recombination operations, just wanted to show how we might make lineage extensible without specifying it.
<pre>
<lineage>
<dc:description>This dataset is a recombination of X and Y</dc:description>
<Jo:recombination>
<Jo:source>X-URI</Jo:source>
<Jo:source>Y-URI</Jo:source>
<Jo:Rules>Overlap</Jo:Rules>
<Jo:recombination>
</lineage>
</pre>
Should the lineage search point be called "LineageDescription" (I think thats what I'll do in my SRW profile).

=== Lack of engagement with packaging and re-use issues ===

Cf. Dataset series / aggregates. The examples have 'MasterMap' as one potential dataset! Real world use cases are going to need subsets of such huge data sets broken down into packages with smaller spatial extents or with less layers.

II: Indeed, as well as the srw/sru binding experiment, I've been wondering about the OAI binding, which I know you've already discussed elsewhere. What might be generally useful (And maybe this already exists) is a set of TREC style test data. Setting up a static gateway OAI server wouldn't be so hard, and might give us some valuable real-world information about this problem. I now the records won't be in the right schema, but something we can try and munge into gmd would be a real help.

=== Bypassing of feature-level metadata from consideration ===

Once we get down to the feature level the interesting European problems appear - the fact that every local area may have its own classification schemes, even inside one language community the same word is used to describe different looking things, and across language barriers mappings from words to things don't tend to be 1-1. But by disregarding feature-level metadata - partly because it can't be mandated when the underlying geospatial objects aren't publically inspectable and a certain amount of feature level metadata would mean the data itself is essentially public...

II: Aye, generally for discovery services it's nice to try and avoid mandating that users understand predefined controlled vocabularies, whilst allowing users who do know terms to qualify their discovery process, for example, in CQL I'd be tempted to allow a user to say "dc.subject=Something" or (The equivalent of) "authority=19115:2003 and dc.subject=Something" for users who know a specific term.

There's quite a lot of work going on around europe at the moment covering crosswalks of controlled vocabularies (Mostly I know about crosswalking euroopean educational levels, but It seems to be the same problem cast in a different way). If we can arrange for someone to do the intellectual work of cross-mapping, and make the data publically available, then it becomes a "Turning-the-handle" job for providers to support cross vocab retrieval. Standards such as ZThes are being used quite a lot in the learning domain to transport this data around. The only effect on reviwing the IR is that it's important that the IR does not preclude this at a future date? (The whole design for unforseen use thing.. specifically, I think mandating a specific vocab in the IR might not be the right thing to do, and giving users a way to say which vocab they are using in the description and discovery process is a better way to go....)

=== Overspecificness about internet- and webservices- based distribution models ===

Actually causing ourselves unnesc problems by putting everything on the Internet. Data sharing agreements over publically maintained private networks with flat-rate membership are a clear potential future and 'middle way' in this domain. The draft now is all about making access/use contraints *specific to data sets* and not specific to the relationship between the data provider or broker, the data user and the transport network between them.

So we have a 'distributed computing platform' metadata property that is required by the IRs. In the ISO19915 mapping in Annex A this is a '''free text field''', yet 5.2.15 states that the property "is necessary for a client to bind to the service". If it must be mandated, it should be as a URI. It would be wonderful to have examples of what other than HTTP or OGC web services is envisaged NOW as a means of access to the backend of a distributed computing platform.

= Feelings =
== Too general ==
I am now finishing the reading and my first feeling from the IR document is that there are some parts too general and not helpful for implementation. It is like ISO 19115 or CSW 2.0. When you start to implement it you have to find your own way how to do it. This will probably lead in inconsistency of catalogues interfaces and content. I know that something is better than nothing and I welcome INSPIRE metadata IR, but I know that there should be specified more that is in this document. We are now testing CSW 2.0 implementation in GeoNetwork Open Source and we have found a lot of problematic parts (in implementation and in specification as well). I have some concrete comments to IR metadata, but I have to sumarised them first to publish them here. Some of them are probably mentioned here (in that case I will just comment that I agree with mentioned issues).

Reading the INSPIRE Metadata Draft

2007-03-24T21:23:00Z

Wiki-Jencek: /* Things that aren't there that should be */

Metadata about geographic data is at the heart of INSPIRE. The metadata draft is the first in the set of "implementing rules" and it will underpin all the other implementing rules. The consultation process is open until 2007-03-30. While the documents are open access, comments can only be offered through an SDIC or Spatial Data Interest Community.

The Free and Open Source Geospatial Community has a voice through one of these SDICs thanks to Markus Neteler. This page contains preparatory material for a collective response through the FOSS GIS SDIC, from the POV of people implementing and managing metadata creation, collection and search services, working closely with many different data user communities.

* The response proper will live at [[Response to INSPIRE Metadata Draft]]. Initial notes are included below in the Issues section.
* It is interesting to read this in parallel with the North American Metadata Profile draft which is also currently in consultation. It's hoped the OSGeo community will also be able to contribute to a [[Response to NAP Metadata Draft]] and get the [http://geodatacommons.umaine.edu geodata commons] project involved in this.

== Reading the draft ==

* [http://www.ec-gis.org/inspire/reports/ImplementingRules/draftINSPIREMetadataIRv2_20070202.pdf the Implementing Rules for Metadata Draft] (pdf)
* [http://www.ec-gis.org/inspire/whatsnew.cfm#1590 supporting / background material]
* Pages 1-17 are metadata about the document itself, intentions and history, and can be safely skipped. Pages 43-104 are the Annexes.
* Annex A is particularly interesting as there are details of the thinking exposed in the mapping to ISO19115/39 that aren't set out in the implementing rules. ''If you want to know what's likely to affect you but are short on time, at minimum read section 5 and Annex A'''.

== Lightning Summary of the draft ==

The draft establishes a basic information model for metadata which is close to, but not specific to, ISO19115 and OGC Web Services.

It only mandates what metadata is published by and for public authorities covered by INSPIRE - it does not try to cover repository management or internal processes.

It separates out metadata properties into those useful for 'discovery', 'evaluation', and 'use'. It identifies one very high level "use case" for spatial data search services built from metadata being shared at this level.

It differentiates between properties useful for 'non-specialist' and 'expert' users into 2 Levels, 1 and 2. Level 1 is always mandatory. This *includes* classification according to the data themes in the INSPIRE annexes, and keywords from controlled vocabularies which are not covered by the IR document but are left to Spatial Data Theme Communities. (How these communities are found, selected, and make their decisions, is unknown to us at this time.)

= Issues =

'''This list is an overview of what jumped out at me as something to address. I don't know how much of this is appropriate to send back, or how much can be fixed. - [[User:JoWalsh]]''' I've added some random musings also, again, no idea if they are of use or even valid questions, but at least they are there and can be edited out, or used as the basis of discussions - [[User:IanIbbo]].

== Conceptual overview ==

The model maps quite well to the minimum useful subset identified in [[DCLite4G]]. It looks like a lightweight core. But, the model and the draft break down the problem space of metadata in a way that is a reaction to artificial scarcity of data. It identifies three phases of the metadata use cycle:
* discovery (of what data is out there)
* evaluation (of whether the data will be useful for specific purpose)
* use (once access gained, how to best use the data)

It is illuminates to compare this with the [[Reading the NAP Metadata Draft|North American Profile]] metadata draft which talks about

* discovery
* access
* fitness for use (e.g. evaluation)
* transfer

So the IRs both don't address how to make the data more useful via metadata, and are vague about how much a minimal subset is going to provide enough information to evaluate utility on. Generally the draft dances around data licensing access issues, and glosses over the over-engineering needed to work around artificial constraints on availability. IRs for evaluation and use of data based on metadata are not covered by this draft at all, but left up to the Spatial Data Theme communities for each of the 35 data themes identified in Annexes I-III of the INSPIRE text.

== Issues with specific metadata properties ==

The model maps quite well to [[DCLite4G]]. It looks like a lightweight core.

=== Things that aren't there that should be ===

'''5.2.8 Resource responsible party'''. Each dataset *must* have one or more people/organisations responsible for it. The IR says that this can be freetext or can be in more structured form. This '''only''' includes the responsible party's name, but NOT any form of contact details.

'''Some form of electronic or telephonic contact address should be mandatory, if the org/person's details are mandatory.''' Why publish ownership information - especially if there are constraints on access and reuse of the described data - if you can't immediately get in personal contact with someone who can make assurances about the data?

I agree, but no only for data, btu for metadata as well:
Page 53. sec A.2.8.1 and A2.8.2 and A.4.1: Other information for identification is necessary like e-mail, URL, voice number.

Annex A on mapping to IS019115 mandates that contact persons and organisations be free text, not resource identifiers. 2 serious problems with the ISO 19115 mapping:

* It does not ask or provide for contact details.
* It looks *mandatory* that the reponsible party be given a role, which in turn is one of N codes published by the Library of Congress to describe people's roles within organisations.

'''No discussion of formalising dataset accuracy / completeness - crucial for cost-benefit evaluation / evaluation of suitability for combining with other data sets.'''

I agree:
Page 31. sec 5.2.12: The lineage element is not enough for data evaluation. The same problem is on Page 55. sec A.3.2.

=== Things that are there that probably shouldn't be ===

Every 'dataset or dataset series' published under INSPIRE *must* include both a '''Resource topic category''' and a set of resource '''keywords'''.

Topic categories are very high-level classifications which correspond to each of the Spatial Data Themes identified in Annexes I, II and III of the INSPIRE Directive.

* Which topic category data fits in will often be a property of an organisation not any published data sets.

From an implementor's POV this will involve something like selecting a topic category for data at install time of metadata publishing engine, and forgetting about it. The IRs place a lot of faith in the ability of simple keyword / classification code matches to enhance utility of search and discovery services for users.

But. this already raises the bar for non-expert users (the domain vocabulary is jargon specific or oriented towards specialist codes)

The IRs emphasise the fact that keywords should originate from a ''controlled vocabulary''. The reponsibility for creating one is not in the hands of the Drafting Teams but in the hands of Spatial Data Theme Communities. How these are constituted and how their decisions become binding are unclear.

Again, faith in keywords for search utility is misplaced. Reliance on them may lead to false negatives. Again assumes familiarity with, or time and ability to learn about, what to expect in the domain from a non-expert user, and an expert will need a better level of detail. Pitfalls of 'controlled' keywording:
* intentional misclassification
* lazy/default misclassification

Both of these are at 'Level 1 for discovery metadata' which implies that any INSPIRE compliant metadata set MUST have both topic category and associated keywords.

== Areas which are unclear ==

=== Conformity ===

This is an IR and obligatory to deal with. But 5.3.4 just says "see Annex F".
Annex F '''in its entirety''' says:

The way in which conformity is expressed in the INSPIRE IR will be defined in a subsequent draft based on discussions with the Drafting Team on Data specifications and harmonization.

(Is this where accuracy/completeness comes in? How can we know?)

=== Dataset series / Aggregate data ===

IR talks about dataset series. Some of the diagrams talk about 'MD_Aggregates'- this term isn't used elsewhere. No conception in this model of one UrDataSet with many different potential sources according to how they are packaged or processed. As the IRs mandate properties for dataset series, really need more clarity / examples about what they actually are.

== General concerns ==

=== Search / discovery services ===

The preamble (p.7) states that "separate IRs for discovery services are being prepared and are not the subject of this document." But the INSPIRE use case is predicated on the availablity of 'Geoportal' style search services. What else *are* discovery services if they are not the search services treated of here? If there is only going to be an abstract model for discovery, and these IRs are careful to avoid imposing any constraints on internal data repository management, how much more can a discovery services draft provide?

II: I think this observation is spot on, but for different reasons perhaps. I'm finding it difficult to express concrete concerns.. but Section 5.2 "Discovery metadata elements" starts to set out a list of concepts seen to be (The document hints at, but does not directly say) core to the discovery process. Section 5.3 then sets out "Abstract discovery metadata element set". I *guess* the implication is that the concepts laid out in 5.2 are in some way even more abstract than those set out in 5.3. The document really isn't clear about what the abstract model is, or what it is for, before it starts enumerating the concepts. Your later comment about being tied to web services is spot on also here, I'm really not sure "Service type version", "Operation name" and "Distributed computing platform" belong in an abstract discovery model (The probably *do* belong in some result record schema). These three attributes seem to belong specifically to a particular (And I would guess already existing) service binding (Or as already said, to a very specific kind of returned result record). What I'd really like to see is a much clearer statement of what the purpose of the abstract discovery model is. Hopefully, once that is tightly defined, it should become easier to decide what lies inside the boundary of the abstract model, and what belongs in the domain of specific realisations of the abstract model. (Actually.. I should say that I'm baised by the information retrieval community generally, in that it's considered really important to have a seperate abstract model for discovery (The search access points) and then bind that model on to as many backend schemas as needed.. this decoupling is seen as best practice in the information retrival domain, and most of my concerns here are that because of the apparent 1:1 mapping between the abstract model and the implementation. This is the approach taken in the [[Z3950 GEO profile] http://www.blueangeltech.com/standards/GeoProfile/geo22.htm]).

I'm a bit confused by the "Temporal Reference" Element... 5.2.2. Talks about what I would expect to see from a temporal reference, but 5.3.2 maps temporal reference on to "One of the dates of publication, last revision or creation of the resource". These three elements are already well defined by dublin core attributes... Maybe I've misunderstood whats implied by table 1 in 5.3.2. Also, similar issues to the spaital access point arise (With structured data, as opposed to text queries). In some UK datasets, periods such as "Neolithic" can be used instead of an ISO 19108 Date Time. (I seen note 11 under 5.3.4 talks about this, which is good. Whats important is that regardless of the outcome of the study, the IR are extensible enough to cope with the eventual decision). I'd consider seperate access points for controlled vocabulary time period and structured temporal data. This seems a specific example where the abstract IR model needs to go beyond what is defined in the A2 binding.

Geographic Extent.. the doc seems a bit bounding box heavy. Would be nice to understand (have examples of) specification of interior/exterioir polygons. Servers only supporting minimal bounding boxes can gracefully degrade (Since it's easy to calculate a MBR from a polygon) whilst allowing other servers to retain the full richness of polygons. It's not clear (in the abstract model, it is in the A2 binding) where the semantics for parsing these strings will be defined.. for example should geographic extent be encoded as OpenGIS strings (Which seems to make sense to me, but I'm biased by Oracle and MySQL's spatial functions). This might seem a bit extreme for the abstract part of the document, but it's one of those make-or-break issues for interoperability, and might be worth the pain. Also, I think it's worth entertaining the idea that spatial specifications such as MBRs and polygons (Structured spatial constructs) might be better exposed using their own abstract access point, and "Place Name" having it's own access point. This will help server implementors avoid problems with disambiguation of search terms.

I'm interested in what the expected semantics of resource language are on retrieval of language-neutral data sets.... Should a result record not be selected because the user specified "Nor" as the search language, but resources matching other criteria (Geo Extent for example) do match. Normally in Info Retrieval this is a no-brainer, of course it shouldn't, but I'm a bit less certain when we talk about result records that aren't primarily "Text" based. (Actually, this is a slightly wider concernn about annex A and those "CharacterString" elements... In IEEE LOM for example we have "LangString" element that has a "Lang" attribute. That community chose to allow language variants of a resource to be expressed within one record by allowing an element to hold all language variants, for example
<pre>
<title>
<langstring lang="En">Hello</langstring>
<langstring lang="Dk">Hej</langstring>
</title>
</pre>
The presence of a "Lang" attribute at the "Dataset" level might mean the intention is to support multi-language datasets by having several dataset records, one for each language, which is OK, but possibly not optimal for datasets that aren't prmarily language based. If this is the case, is the "CharacterString" element in Annex A just redundant payload?)

=== Lack of machine-reusable data in general ===

Dataset 'lineage' is only a full-text field. If datasets result from recombination, that should be machine-traversable. Human descriptions of lineage will be so different that they won't be useful for building search / evaluation services.

II: It does tend to talk about "Lineage statement"... would making it (More along the conceptual lines of)
<pre>
<lineage>
<dc:description>Text</dc:description>
</lineage>
</pre>
Give you the extensibility to either use private extensions, or to specify recombination elements at a later date (I didn't think this through in terms of the *actual* recombination operations, just wanted to show how we might make lineage extensible without specifying it.
<pre>
<lineage>
<dc:description>This dataset is a recombination of X and Y</dc:description>
<Jo:recombination>
<Jo:source>X-URI</Jo:source>
<Jo:source>Y-URI</Jo:source>
<Jo:Rules>Overlap</Jo:Rules>
<Jo:recombination>
</lineage>
</pre>
Should the lineage search point be called "LineageDescription" (I think thats what I'll do in my SRW profile).

=== Lack of engagement with packaging and re-use issues ===

Cf. Dataset series / aggregates. The examples have 'MasterMap' as one potential dataset! Real world use cases are going to need subsets of such huge data sets broken down into packages with smaller spatial extents or with less layers.

II: Indeed, as well as the srw/sru binding experiment, I've been wondering about the OAI binding, which I know you've already discussed elsewhere. What might be generally useful (And maybe this already exists) is a set of TREC style test data. Setting up a static gateway OAI server wouldn't be so hard, and might give us some valuable real-world information about this problem. I now the records won't be in the right schema, but something we can try and munge into gmd would be a real help.

=== Bypassing of feature-level metadata from consideration ===

Once we get down to the feature level the interesting European problems appear - the fact that every local area may have its own classification schemes, even inside one language community the same word is used to describe different looking things, and across language barriers mappings from words to things don't tend to be 1-1. But by disregarding feature-level metadata - partly because it can't be mandated when the underlying geospatial objects aren't publically inspectable and a certain amount of feature level metadata would mean the data itself is essentially public...

II: Aye, generally for discovery services it's nice to try and avoid mandating that users understand predefined controlled vocabularies, whilst allowing users who do know terms to qualify their discovery process, for example, in CQL I'd be tempted to allow a user to say "dc.subject=Something" or (The equivalent of) "authority=19115:2003 and dc.subject=Something" for users who know a specific term.

There's quite a lot of work going on around europe at the moment covering crosswalks of controlled vocabularies (Mostly I know about crosswalking euroopean educational levels, but It seems to be the same problem cast in a different way). If we can arrange for someone to do the intellectual work of cross-mapping, and make the data publically available, then it becomes a "Turning-the-handle" job for providers to support cross vocab retrieval. Standards such as ZThes are being used quite a lot in the learning domain to transport this data around. The only effect on reviwing the IR is that it's important that the IR does not preclude this at a future date? (The whole design for unforseen use thing.. specifically, I think mandating a specific vocab in the IR might not be the right thing to do, and giving users a way to say which vocab they are using in the description and discovery process is a better way to go....)

=== Overspecificness about internet- and webservices- based distribution models ===

Actually causing ourselves unnesc problems by putting everything on the Internet. Data sharing agreements over publically maintained private networks with flat-rate membership are a clear potential future and 'middle way' in this domain. The draft now is all about making access/use contraints *specific to data sets* and not specific to the relationship between the data provider or broker, the data user and the transport network between them.

So we have a 'distributed computing platform' metadata property that is required by the IRs. In the ISO19915 mapping in Annex A this is a '''free text field''', yet 5.2.15 states that the property "is necessary for a client to bind to the service". If it must be mandated, it should be as a URI. It would be wonderful to have examples of what other than HTTP or OGC web services is envisaged NOW as a means of access to the backend of a distributed computing platform.

= Feelings =
== Too general ==
I am now finishing the reading and my first feeling from the IR document is that there are some parts too general and not helpful for implementation. It is like ISO 19115 or CSW 2.0. When you start to implement it you have to find your own way how to do it. This will probably lead in inconsistency of catalogues interfaces and content. I know that something is better than nothing and I welcome INSPIRE metadata IR, but I know that there should be specified more that is in this document. We are now testing CSW 2.0 implementation in GeoNetwork Open Source and we have found a lot of problematic parts (in implementation and in specification as well). I have some concrete comments to IR metadata, but I have to sumarised them first to publish them here. Some of them are probably mentioned here (in that case I will just comment that I agree with mentioned issues).

Reading the INSPIRE Metadata Draft

2007-03-19T21:48:02Z

Wiki-Jencek: /* Too general */

Metadata about geographic data is at the heart of INSPIRE. The metadata draft is the first in the set of "implementing rules" and it will underpin all the other implementing rules. The consultation process is open until 2007-03-30. While the documents are open access, comments can only be offered through an SDIC or Spatial Data Interest Community.

The Free and Open Source Geospatial Community has a voice through one of these SDICs thanks to Markus Neteler. This page contains preparatory material for a collective response through the FOSS GIS SDIC, from the POV of people implementing and managing metadata creation, collection and search services, working closely with many different data user communities.

* The response proper will live at [[Response to INSPIRE Metadata Draft]]. Initial notes are included below in the Issues section.
* It is interesting to read this in parallel with the North American Metadata Profile draft which is also currently in consultation. It's hoped the OSGeo community will also be able to contribute to a [[Response to NAP Metadata Draft]] and get the [http://geodatacommons.umaine.edu geodata commons] project involved in this.

== Reading the draft ==

* [http://www.ec-gis.org/inspire/reports/ImplementingRules/draftINSPIREMetadataIRv2_20070202.pdf the Implementing Rules for Metadata Draft] (pdf)
* [http://www.ec-gis.org/inspire/whatsnew.cfm#1590 supporting / background material]
* Pages 1-17 are metadata about the document itself, intentions and history, and can be safely skipped. Pages 43-104 are the Annexes.
* Annex A is particularly interesting as there are details of the thinking exposed in the mapping to ISO19115/39 that aren't set out in the implementing rules. ''If you want to know what's likely to affect you but are short on time, at minimum read section 5 and Annex A'''.

== Lightning Summary of the draft ==

The draft establishes a basic information model for metadata which is close to, but not specific to, ISO19115 and OGC Web Services.

It only mandates what metadata is published by and for public authorities covered by INSPIRE - it does not try to cover repository management or internal processes.

It separates out metadata properties into those useful for 'discovery', 'evaluation', and 'use'. It identifies one very high level "use case" for spatial data search services built from metadata being shared at this level.

It differentiates between properties useful for 'non-specialist' and 'expert' users into 2 Levels, 1 and 2. Level 1 is always mandatory. This *includes* classification according to the data themes in the INSPIRE annexes, and keywords from controlled vocabularies which are not covered by the IR document but are left to Spatial Data Theme Communities. (How these communities are found, selected, and make their decisions, is unknown to us at this time.)

= Issues =

'''This list is an overview of what jumped out at me as something to address. I don't know how much of this is appropriate to send back, or how much can be fixed. - [[User:JoWalsh]]''' I've added some random musings also, again, no idea if they are of use or even valid questions, but at least they are there and can be edited out, or used as the basis of discussions - [[User:IanIbbo]].

== Conceptual overview ==

The model maps quite well to the minimum useful subset identified in [[DCLite4G]]. It looks like a lightweight core. But, the model and the draft break down the problem space of metadata in a way that is a reaction to artificial scarcity of data. It identifies three phases of the metadata use cycle:
* discovery (of what data is out there)
* evaluation (of whether the data will be useful for specific purpose)
* use (once access gained, how to best use the data)

It is illuminates to compare this with the [[Reading the NAP Metadata Draft|North American Profile]] metadata draft which talks about

* discovery
* access
* fitness for use (e.g. evaluation)
* transfer

So the IRs both don't address how to make the data more useful via metadata, and are vague about how much a minimal subset is going to provide enough information to evaluate utility on. Generally the draft dances around data licensing access issues, and glosses over the over-engineering needed to work around artificial constraints on availability. IRs for evaluation and use of data based on metadata are not covered by this draft at all, but left up to the Spatial Data Theme communities for each of the 35 data themes identified in Annexes I-III of the INSPIRE text.

== Issues with specific metadata properties ==

The model maps quite well to [[DCLite4G]]. It looks like a lightweight core.

=== Things that aren't there that should be ===

'''5.2.8 Resource responsible party'''. Each dataset *must* have one or more people/organisations responsible for it. The IR says that this can be freetext or can be in more structured form. This '''only''' includes the responsible party's name, but NOT any form of contact details.

'''Some form of electronic or telephonic contact address should be mandatory, if the org/person's details are mandatory.''' Why publish ownership information - especially if there are constraints on access and reuse of the described data - if you can't immediately get in personal contact with someone who can make assurances about the data?

Annex A on mapping to IS019115 mandates that contact persons and organisations be free text, not resource identifiers. 2 serious problems with the ISO 19115 mapping:

* It does not ask or provide for contact details.
* It looks *mandatory* that the reponsible party be given a role, which in turn is one of N codes published by the Library of Congress to describe people's roles within organisations.

'''No discussion of formalising dataset accuracy / completeness - crucial for cost-benefit evaluation / evaluation of suitability for combining with other data sets.'''

=== Things that are there that probably shouldn't be ===

Every 'dataset or dataset series' published under INSPIRE *must* include both a '''Resource topic category''' and a set of resource '''keywords'''.

Topic categories are very high-level classifications which correspond to each of the Spatial Data Themes identified in Annexes I, II and III of the INSPIRE Directive.

* Which topic category data fits in will often be a property of an organisation not any published data sets.

From an implementor's POV this will involve something like selecting a topic category for data at install time of metadata publishing engine, and forgetting about it. The IRs place a lot of faith in the ability of simple keyword / classification code matches to enhance utility of search and discovery services for users.

But. this already raises the bar for non-expert users (the domain vocabulary is jargon specific or oriented towards specialist codes)

The IRs emphasise the fact that keywords should originate from a ''controlled vocabulary''. The reponsibility for creating one is not in the hands of the Drafting Teams but in the hands of Spatial Data Theme Communities. How these are constituted and how their decisions become binding are unclear.

Again, faith in keywords for search utility is misplaced. Reliance on them may lead to false negatives. Again assumes familiarity with, or time and ability to learn about, what to expect in the domain from a non-expert user, and an expert will need a better level of detail. Pitfalls of 'controlled' keywording:
* intentional misclassification
* lazy/default misclassification

Both of these are at 'Level 1 for discovery metadata' which implies that any INSPIRE compliant metadata set MUST have both topic category and associated keywords.

== Areas which are unclear ==

=== Conformity ===

This is an IR and obligatory to deal with. But 5.3.4 just says "see Annex F".
Annex F '''in its entirety''' says:

The way in which conformity is expressed in the INSPIRE IR will be defined in a subsequent draft based on discussions with the Drafting Team on Data specifications and harmonization.

(Is this where accuracy/completeness comes in? How can we know?)

=== Dataset series / Aggregate data ===

IR talks about dataset series. Some of the diagrams talk about 'MD_Aggregates'- this term isn't used elsewhere. No conception in this model of one UrDataSet with many different potential sources according to how they are packaged or processed. As the IRs mandate properties for dataset series, really need more clarity / examples about what they actually are.

== General concerns ==

=== Search / discovery services ===

The preamble (p.7) states that "separate IRs for discovery services are being prepared and are not the subject of this document." But the INSPIRE use case is predicated on the availablity of 'Geoportal' style search services. What else *are* discovery services if they are not the search services treated of here? If there is only going to be an abstract model for discovery, and these IRs are careful to avoid imposing any constraints on internal data repository management, how much more can a discovery services draft provide?

II: I think this observation is spot on, but for different reasons perhaps. I'm finding it difficult to express concrete concerns.. but Section 5.2 "Discovery metadata elements" starts to set out a list of concepts seen to be (The document hints at, but does not directly say) core to the discovery process. Section 5.3 then sets out "Abstract discovery metadata element set". I *guess* the implication is that the concepts laid out in 5.2 are in some way even more abstract than those set out in 5.3. The document really isn't clear about what the abstract model is, or what it is for, before it starts enumerating the concepts. Your later comment about being tied to web services is spot on also here, I'm really not sure "Service type version", "Operation name" and "Distributed computing platform" belong in an abstract discovery model (The probably *do* belong in some result record schema). These three attributes seem to belong specifically to a particular (And I would guess already existing) service binding (Or as already said, to a very specific kind of returned result record). What I'd really like to see is a much clearer statement of what the purpose of the abstract discovery model is. Hopefully, once that is tightly defined, it should become easier to decide what lies inside the boundary of the abstract model, and what belongs in the domain of specific realisations of the abstract model. (Actually.. I should say that I'm baised by the information retrieval community generally, in that it's considered really important to have a seperate abstract model for discovery (The search access points) and then bind that model on to as many backend schemas as needed.. this decoupling is seen as best practice in the information retrival domain, and most of my concerns here are that because of the apparent 1:1 mapping between the abstract model and the implementation. This is the approach taken in the [[Z3950 GEO profile] http://www.blueangeltech.com/standards/GeoProfile/geo22.htm]).

I'm a bit confused by the "Temporal Reference" Element... 5.2.2. Talks about what I would expect to see from a temporal reference, but 5.3.2 maps temporal reference on to "One of the dates of publication, last revision or creation of the resource". These three elements are already well defined by dublin core attributes... Maybe I've misunderstood whats implied by table 1 in 5.3.2. Also, similar issues to the spaital access point arise (With structured data, as opposed to text queries). In some UK datasets, periods such as "Neolithic" can be used instead of an ISO 19108 Date Time. (I seen note 11 under 5.3.4 talks about this, which is good. Whats important is that regardless of the outcome of the study, the IR are extensible enough to cope with the eventual decision). I'd consider seperate access points for controlled vocabulary time period and structured temporal data. This seems a specific example where the abstract IR model needs to go beyond what is defined in the A2 binding.

Geographic Extent.. the doc seems a bit bounding box heavy. Would be nice to understand (have examples of) specification of interior/exterioir polygons. Servers only supporting minimal bounding boxes can gracefully degrade (Since it's easy to calculate a MBR from a polygon) whilst allowing other servers to retain the full richness of polygons. It's not clear (in the abstract model, it is in the A2 binding) where the semantics for parsing these strings will be defined.. for example should geographic extent be encoded as OpenGIS strings (Which seems to make sense to me, but I'm biased by Oracle and MySQL's spatial functions). This might seem a bit extreme for the abstract part of the document, but it's one of those make-or-break issues for interoperability, and might be worth the pain. Also, I think it's worth entertaining the idea that spatial specifications such as MBRs and polygons (Structured spatial constructs) might be better exposed using their own abstract access point, and "Place Name" having it's own access point. This will help server implementors avoid problems with disambiguation of search terms.

I'm interested in what the expected semantics of resource language are on retrieval of language-neutral data sets.... Should a result record not be selected because the user specified "Nor" as the search language, but resources matching other criteria (Geo Extent for example) do match. Normally in Info Retrieval this is a no-brainer, of course it shouldn't, but I'm a bit less certain when we talk about result records that aren't primarily "Text" based. (Actually, this is a slightly wider concernn about annex A and those "CharacterString" elements... In IEEE LOM for example we have "LangString" element that has a "Lang" attribute. That community chose to allow language variants of a resource to be expressed within one record by allowing an element to hold all language variants, for example
<pre>
<title>
<langstring lang="En">Hello</langstring>
<langstring lang="Dk">Hej</langstring>
</title>
</pre>
The presence of a "Lang" attribute at the "Dataset" level might mean the intention is to support multi-language datasets by having several dataset records, one for each language, which is OK, but possibly not optimal for datasets that aren't prmarily language based. If this is the case, is the "CharacterString" element in Annex A just redundant payload?)

=== Lack of machine-reusable data in general ===

Dataset 'lineage' is only a full-text field. If datasets result from recombination, that should be machine-traversable. Human descriptions of lineage will be so different that they won't be useful for building search / evaluation services.

II: It does tend to talk about "Lineage statement"... would making it (More along the conceptual lines of)
<pre>
<lineage>
<dc:description>Text</dc:description>
</lineage>
</pre>
Give you the extensibility to either use private extensions, or to specify recombination elements at a later date (I didn't think this through in terms of the *actual* recombination operations, just wanted to show how we might make lineage extensible without specifying it.
<pre>
<lineage>
<dc:description>This dataset is a recombination of X and Y</dc:description>
<Jo:recombination>
<Jo:source>X-URI</Jo:source>
<Jo:source>Y-URI</Jo:source>
<Jo:Rules>Overlap</Jo:Rules>
<Jo:recombination>
</lineage>
</pre>
Should the lineage search point be called "LineageDescription" (I think thats what I'll do in my SRW profile).

=== Lack of engagement with packaging and re-use issues ===

Cf. Dataset series / aggregates. The examples have 'MasterMap' as one potential dataset! Real world use cases are going to need subsets of such huge data sets broken down into packages with smaller spatial extents or with less layers.

II: Indeed, as well as the srw/sru binding experiment, I've been wondering about the OAI binding, which I know you've already discussed elsewhere. What might be generally useful (And maybe this already exists) is a set of TREC style test data. Setting up a static gateway OAI server wouldn't be so hard, and might give us some valuable real-world information about this problem. I now the records won't be in the right schema, but something we can try and munge into gmd would be a real help.

=== Bypassing of feature-level metadata from consideration ===

Once we get down to the feature level the interesting European problems appear - the fact that every local area may have its own classification schemes, even inside one language community the same word is used to describe different looking things, and across language barriers mappings from words to things don't tend to be 1-1. But by disregarding feature-level metadata - partly because it can't be mandated when the underlying geospatial objects aren't publically inspectable and a certain amount of feature level metadata would mean the data itself is essentially public...

II: Aye, generally for discovery services it's nice to try and avoid mandating that users understand predefined controlled vocabularies, whilst allowing users who do know terms to qualify their discovery process, for example, in CQL I'd be tempted to allow a user to say "dc.subject=Something" or (The equivalent of) "authority=19115:2003 and dc.subject=Something" for users who know a specific term.

There's quite a lot of work going on around europe at the moment covering crosswalks of controlled vocabularies (Mostly I know about crosswalking euroopean educational levels, but It seems to be the same problem cast in a different way). If we can arrange for someone to do the intellectual work of cross-mapping, and make the data publically available, then it becomes a "Turning-the-handle" job for providers to support cross vocab retrieval. Standards such as ZThes are being used quite a lot in the learning domain to transport this data around. The only effect on reviwing the IR is that it's important that the IR does not preclude this at a future date? (The whole design for unforseen use thing.. specifically, I think mandating a specific vocab in the IR might not be the right thing to do, and giving users a way to say which vocab they are using in the description and discovery process is a better way to go....)

=== Overspecificness about internet- and webservices- based distribution models ===

Actually causing ourselves unnesc problems by putting everything on the Internet. Data sharing agreements over publically maintained private networks with flat-rate membership are a clear potential future and 'middle way' in this domain. The draft now is all about making access/use contraints *specific to data sets* and not specific to the relationship between the data provider or broker, the data user and the transport network between them.

So we have a 'distributed computing platform' metadata property that is required by the IRs. In the ISO19915 mapping in Annex A this is a '''free text field''', yet 5.2.15 states that the property "is necessary for a client to bind to the service". If it must be mandated, it should be as a URI. It would be wonderful to have examples of what other than HTTP or OGC web services is envisaged NOW as a means of access to the backend of a distributed computing platform.

= Feelings =
== Too general ==
I am now finishing the reading and my first feeling from the IR document is that there are some parts too general and not helpful for implementation. It is like ISO 19115 or CSW 2.0. When you start to implement it you have to find your own way how to do it. This will probably lead in inconsistency of catalogues interfaces and content. I know that something is better than nothing and I welcome INSPIRE metadata IR, but I know that there should be specified more that is in this document. We are now testing CSW 2.0 implementation in GeoNetwork Open Source and we have found a lot of problematic parts (in implementation and in specification as well). I have some concrete comments to IR metadata, but I have to sumarised them first to publish them here. Some of them are probably mentioned here (in that case I will just comment that I agree with mentioned issues).

Reading the INSPIRE Metadata Draft

2007-03-19T21:36:54Z

Wiki-Jencek: /* Issues */

User:Jencek

2007-03-09T20:10:15Z

Wiki-Jencek:

Contact: jan.ruzicka@vsb.cz

Response to INSPIRE Metadata Draft

2007-03-09T19:59:09Z

Wiki-Jencek: /* Participants */

* See [[Reading the INSPIRE Metadata Draft]] for preparatory material.
* See [http://inspire.jrc.it/ir/sdic_view_step1_only.cfm?id=2163 FOSS SDIC]

Any member of the Free and Open Source Geospatial software community is welcome to participate in creating this response. Please add your name, and contact details on your userpage, to the list of participants and at any key stage (first stable draft; when sending the response).

= Participants =

* [[User:JoWalsh|Jo Walsh]]
* [[User:Neteler|Markus Neteler]]
* [[User:Ianibbo|Ian Ibbotson]]
* [[User:Jencek|Jan Růžička]]

Reading the INSPIRE Metadata Draft

2007-02-16T13:34:02Z

Wiki-Jencek: /* Things that aren't there that should be */

Reading the INSPIRE Metadata Draft

2007-02-16T13:28:04Z

Wiki-Jencek: /* Reading the draft */

Czech

2007-02-07T07:06:39Z

Wiki-Jencek: /* Možné společné téma a nástroj */

TODO

== Interested? ==
''Put you name / wiki account down below to show support moving forward''

;Martin [[User:Landa|Landa]]
:PhD student
:Department of Mapping and Cartography, [http://www.fsv.cvut.cz Faculty of Civil Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Jencek Růžička [[User:Jencek|Jencek]]
:lecturer
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics], [http://gis.vsb.cz/ruzicka Home Page]

;František Klímek [[User:frkl|frkl]]
:PhD student
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics]

;Tomáš Richta [[User:squatter|squatter]]
:PhD student, assistant professor
:[http://cs.felk.cvut.cz/webis/en/ Department of Computer Science and Engineering], [http://www.fel.cvut.cz/en/ Faculty of Electrical Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Aleš Daněk [[User:aldama|aldama]]
:analyst, coder
:[http://www.cad-programs.com/ CAD Programs - Ing. Jan Vlčinský]

;[[User:cepek|Aleš Čepek]]
:[http://gama.fsv.cvut.cz/ Geodesy and Cartography, CTU Prague]

;[[User:Jezekjan|Jan Ježek]]
:[http://www.kma.zcu.cz/main.php Department of Mathematics, University of West Bohemia in Pilsen]

;Stanislav Grill [[User:grillst|grillst]]
:PhD student
:[http://web.natur.cuni.cz/gis Department of Applied Geoinformatics and Cartography], [http://www.natur.cuni.cz Faculty of science Charles university in Prague]

== Základní vysvětelní cílů ==
Zatím prvotní vysvětlení cílů připravil Martin: http://grass.fsv.cvut.cz/wiki/index.php/OSGeo_Local_Chapter

== Zamyšlení od Tomáše Richty ==
ahoj,
nedalo mi to, a neodpustil jsem si vcera cestou z ostravy takovou malou uvahu,
tak vam ji dnes posilam, napadl me totiz takovy pekny primer...

doktori drive byli jini, dobre rozumeli lidskemu telu i duchu, mohli tak cloveka lecit zevrubnym pruzkumem toho, co mu chybi, nebo prebyva a naslednym doporucenim, jak telo a ducha primet k tomu, aby se vse zase vratilo do puvodniho rovnovazneho stavu a clovek se uzdravil...

dnes uz doktori leci jinak, podle priznaku pacienta vyberou vhodny prasek, ktery tyto priznaky potlaci, nebo pozmeni, tak aby byl pacient spokjen, ze se mu ulevilo, pri vyberu prasku je doktorum casto dobrym navodem castka, kterou dostali od sve oblibene farmaceuticke firmy za to, ze prave jeji prasky budou pacientum predepisovat, farmaceuticke firmy maji sve doktory rady, dobre si je zivi a dobre se o ne staraji, zasobuji je dobrymi prasky, letaky s obrazky s fotkami stastnych pacientu, kteri jedi prasky, tu a tam i nejakymi peknymi vylety k mori, ci jinymi darecky...

farmaceuticke firmy take zvou doktory na ruzne konference, na kterych jim predavdeji jak jejich prasky dobre funguji, a take kolik ruznych skvelych a barevnych prasku vyrabeji, farmaceuticke firmy jsou bohate a tak se na konfencich dobre ji a pije, a i na nejakou tu zabavu se penize najdou, doktori pak odjizdeji domu s taskami plnymi letaku, plakatu, prospektu a vzorku, plni krasneho pocitu, ze vse je v poradku, protoze na kazdy neduh existuje prasek a staci si jen pamatovat, ktery je na ktery, pripadne jak jednotlive prasky dobre zkombinovat, aby vyledna chemicka reakce nejspis probehla spravne...

nekdy si take doktori zacnou na konferencich povidat o tom, jakych zajimavych vysledku dosahli, kdyz tuhle zkusili na nejakem pacientovi zkombinovat ty a ty prasky, nebo kdyz nejaky prasek nezabral, jak se jim povedlo situaci vyresit tim, ze podali jiny, mnohem silnejsi prasek, a nebo co vsechno se da s timhle praskem delat...

dnesnim doktorum pomalu umira mozek a zrejme uz se to ani neda zastavit, tolikrat se v posleni dobe setkavam se situacemi, kdy doktori nejsou schopni vubec zjistit, co dotycnemu je a jen neustale opakovane zkouseji dalsi a dalsi prasky, az nejaky treba zabere, cit je nahrazovan hamiznosti, rozum tuposti...

jiste si snadno dokazete v predchozich odstavcich zamenit za doktora informatika a uvidite, ze rozdilu mnoho neni, dnesni informatik se podle meho nazoru pomalu ale jiste riti do stejne dusevni zahuby, do ktere se uz drive dostali doktori, stava se z nej bezduchy prostrednik s vybornymi znalostmi seznamu prasku, dobrymi vztahy s farmaceutickymi firmami a s prazdnym prostorem v sekci napady a inovace, v cechach to zrejme plati dvojnasob...

jiste mi muzete namitnout, ze je to vse nadsazene a velmi zobecnene, urcite se i dnes najdou doktori, kteri sve praci rozumi, a take leky, ktere opravdu pomohou, to je fakt, ale trend je myslim celkem jasny...

zaslechl jsem vcera, ze se planuje (nebo snad, ze uz dokonce probehlo), zalozeni gis-open-source komunity, jsem tim velmi nadsen! pevne verim, ze prave takovato iniciativa muze byt jednim z impulzu, ktery ceske informatiky muze zachranit, verim take, ze prave z takove komunity, a ji podobnych, mohou plynout myslenky hluboke i vzletne, ktere ve forme prispevku pozvednou uroven i takovych konferenci jako je treba ta prave skoncena...

je treba hledat a podporovat budouci doktory, kteri telu i duchu zase dobre porozumi a budou vedet kam sahnout, aby vse dobre fungovalo, aby se soukoli zase rozebehlo, vzdycky bude dost obchodniku s prasky, ale rozum by jen proto, ze je to trend, nemel vymrit...

tom

p.s. prikladam muj oblibeny citat naseho smalltalkerskeho poloboha:
<pre>
The real romance is out ahead and yet to come.
The computer revolution hasn't started yet.
Don't be misled by the enormous flow of money
into bad defacto standards
for unsophisticated buyers
using poor adaptations of
incomplete ideas.

Alan Kay
</pre>

== Diskusní skupina ==

Domlouvali jsme se taky, že by měla vzniknout diskusní e-mailová skupina. Napadlo mě že bychom zatím mohli využívat FreeGeoCZ http://mailman.fsv.cvut.cz/mailman/listinfo/freegeocz

== Možné společné téma a nástroj ==

Jako možné společné téma zatím krystalizuje práce s rastry v PostGIS a PostGIS obecně jako datový sklad.

Včera Tonda říkal, že hlavním tématem sdružení je sdružovat se. A druhým tématem je propagace OS GIS, tak aby se nástroje, které my používáme dostaly k běžným uživatelům GIS. Tato dvě témata jsou určitě to co bychom dělat měli a dnes už děláme. Síla sdružení je v tom, že teď už můžeme propagovat cíleněji a s určitou podporou (minimálně psychologickou), že za námi stojí určitá skupina lidí se stejným zájmem.

Přidejte co byste chtěli řešit vy.

[[Category:Local Chapters]]

Czech

2007-02-05T11:08:55Z

Wiki-Jencek: /* Zamyšlení od Tomáše Richty */

TODO

== Interested? ==
''Put you name / wiki account down below to show support moving forward''

;Martin [[User:Landa|Landa]]
:PhD student
:Department of Mapping and Cartography, [http://www.fsv.cvut.cz Faculty of Civil Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Jencek Růžička [[User:Jencek|Jencek]]
:lecturer
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics], [http://gis.vsb.cz/ruzicka Home Page]

;František Klímek [[User:frkl|frkl]]
:PhD student
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics]

;Tomáš Richta [[User:squatter|squatter]]
:PhD student, assistant professor
:[http://cs.felk.cvut.cz/webis/en/ Department of Computer Science and Engineering], [http://www.fel.cvut.cz/en/ Faculty of Electrical Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Aleš Daněk [[User:aldama|aldama]]
:analyst, coder
:[http://www.cad-programs.com/ CAD Programs - Ing. Jan Vlčinský]

;[[User:cepek|Aleš Čepek]]
:[http://gama.fsv.cvut.cz/ Geodesy and Cartography, CTU Prague]

== Základní vysvětelní cílů ==
Zatím prvotní vysvětlení cílů připravil Martin: http://grass.fsv.cvut.cz/wiki/index.php/OSGeo_Local_Chapter

== Zamyšlení od Tomáše Richty ==
ahoj,
nedalo mi to, a neodpustil jsem si vcera cestou z ostravy takovou malou uvahu,
tak vam ji dnes posilam, napadl me totiz takovy pekny primer...

doktori drive byli jini, dobre rozumeli lidskemu telu i duchu, mohli tak cloveka lecit zevrubnym pruzkumem toho, co mu chybi, nebo prebyva a naslednym doporucenim, jak telo a ducha primet k tomu, aby se vse zase vratilo do puvodniho rovnovazneho stavu a clovek se uzdravil...

dnes uz doktori leci jinak, podle priznaku pacienta vyberou vhodny prasek, ktery tyto priznaky potlaci, nebo pozmeni, tak aby byl pacient spokjen, ze se mu ulevilo, pri vyberu prasku je doktorum casto dobrym navodem castka, kterou dostali od sve oblibene farmaceuticke firmy za to, ze prave jeji prasky budou pacientum predepisovat, farmaceuticke firmy maji sve doktory rady, dobre si je zivi a dobre se o ne staraji, zasobuji je dobrymi prasky, letaky s obrazky s fotkami stastnych pacientu, kteri jedi prasky, tu a tam i nejakymi peknymi vylety k mori, ci jinymi darecky...

farmaceuticke firmy take zvou doktory na ruzne konference, na kterych jim predavdeji jak jejich prasky dobre funguji, a take kolik ruznych skvelych a barevnych prasku vyrabeji, farmaceuticke firmy jsou bohate a tak se na konfencich dobre ji a pije, a i na nejakou tu zabavu se penize najdou, doktori pak odjizdeji domu s taskami plnymi letaku, plakatu, prospektu a vzorku, plni krasneho pocitu, ze vse je v poradku, protoze na kazdy neduh existuje prasek a staci si jen pamatovat, ktery je na ktery, pripadne jak jednotlive prasky dobre zkombinovat, aby vyledna chemicka reakce nejspis probehla spravne...

nekdy si take doktori zacnou na konferencich povidat o tom, jakych zajimavych vysledku dosahli, kdyz tuhle zkusili na nejakem pacientovi zkombinovat ty a ty prasky, nebo kdyz nejaky prasek nezabral, jak se jim povedlo situaci vyresit tim, ze podali jiny, mnohem silnejsi prasek, a nebo co vsechno se da s timhle praskem delat...

dnesnim doktorum pomalu umira mozek a zrejme uz se to ani neda zastavit, tolikrat se v posleni dobe setkavam se situacemi, kdy doktori nejsou schopni vubec zjistit, co dotycnemu je a jen neustale opakovane zkouseji dalsi a dalsi prasky, az nejaky treba zabere, cit je nahrazovan hamiznosti, rozum tuposti...

jiste si snadno dokazete v predchozich odstavcich zamenit za doktora informatika a uvidite, ze rozdilu mnoho neni, dnesni informatik se podle meho nazoru pomalu ale jiste riti do stejne dusevni zahuby, do ktere se uz drive dostali doktori, stava se z nej bezduchy prostrednik s vybornymi znalostmi seznamu prasku, dobrymi vztahy s farmaceutickymi firmami a s prazdnym prostorem v sekci napady a inovace, v cechach to zrejme plati dvojnasob...

jiste mi muzete namitnout, ze je to vse nadsazene a velmi zobecnene, urcite se i dnes najdou doktori, kteri sve praci rozumi, a take leky, ktere opravdu pomohou, to je fakt, ale trend je myslim celkem jasny...

zaslechl jsem vcera, ze se planuje (nebo snad, ze uz dokonce probehlo), zalozeni gis-open-source komunity, jsem tim velmi nadsen! pevne verim, ze prave takovato iniciativa muze byt jednim z impulzu, ktery ceske informatiky muze zachranit, verim take, ze prave z takove komunity, a ji podobnych, mohou plynout myslenky hluboke i vzletne, ktere ve forme prispevku pozvednou uroven i takovych konferenci jako je treba ta prave skoncena...

je treba hledat a podporovat budouci doktory, kteri telu i duchu zase dobre porozumi a budou vedet kam sahnout, aby vse dobre fungovalo, aby se soukoli zase rozebehlo, vzdycky bude dost obchodniku s prasky, ale rozum by jen proto, ze je to trend, nemel vymrit...

tom

p.s. prikladam muj oblibeny citat naseho smalltalkerskeho poloboha:
<pre>
The real romance is out ahead and yet to come.
The computer revolution hasn't started yet.
Don't be misled by the enormous flow of money
into bad defacto standards
for unsophisticated buyers
using poor adaptations of
incomplete ideas.

Alan Kay
</pre>

== Diskusní skupina ==

Domlouvali jsme se taky, že by měla vzniknout diskusní e-mailová skupina. Napadlo mě že bychom zatím mohli využívat FreeGeoCZ http://mailman.fsv.cvut.cz/mailman/listinfo/freegeocz

== Možné společné téma a nástroj ==

Jako možné společné téma zatím krystalizuje PostGIS a v něm prostorová indexace vektorů a rastrů (třeba na sféroidu - asi sférický grid, "koule" rozdělená na plochy (nejčastěji trojúhelníky)). Přidejte co byste chtěli řešit vy.

[[Category:Local Chapters]]

Czech

2007-02-05T08:32:20Z

Wiki-Jencek: /* Diskusní skupina */

TODO

== Interested? ==
''Put you name / wiki account down below to show support moving forward''

;Martin [[User:Landa|Landa]]
:PhD student
:Department of Mapping and Cartography, [http://www.fsv.cvut.cz Faculty of Civil Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Jencek Růžička [[User:Jencek|Jencek]]
:lecturer
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics], [http://gis.vsb.cz/ruzicka Home Page]

;František Klímek [[User:frkl|frkl]]
:PhD student
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics]

;Tomáš Richta [[User:squatter|squatter]]
:PhD student, assistant professor
:[http://cs.felk.cvut.cz/webis/en/ Department of Computer Science and Engineering], [http://www.fel.cvut.cz/en/ Faculty of Electrical Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Aleš Daněk [[User:aldama|aldama]]
:analyst, coder
:[http://www.cad-programs.com/ CAD Programs - Ing. Jan Vlčinský]

== Zamyšlení od Tomáše Richty ==
ahoj,
nedalo mi to, a neodpustil jsem si vcera cestou z ostravy takovou malou uvahu,
tak vam ji dnes posilam, napadl me totiz takovy pekny primer...

doktori drive byli jini, dobre rozumeli lidskemu telu i duchu, mohli tak cloveka lecit zevrubnym pruzkumem toho, co mu chybi, nebo prebyva a naslednym doporucenim, jak telo a ducha primet k tomu, aby se vse zase vratilo do puvodniho rovnovazneho stavu a clovek se uzdravil...

dnes uz doktori leci jinak, podle priznaku pacienta vyberou vhodny prasek, ktery tyto priznaky potlaci, nebo pozmeni, tak aby byl pacient spokjen, ze se mu ulevilo, pri vyberu prasku je doktorum casto dobrym navodem castka, kterou dostali od sve oblibene farmaceuticke firmy za to, ze prave jeji prasky budou pacientum predepisovat, farmaceuticke firmy maji sve doktory rady, dobre si je zivi a dobre se o ne staraji, zasobuji je dobrymi prasky, letaky s obrazky s fotkami stastnych pacientu, kteri jedi prasky, tu a tam i nejakymi peknymi vylety k mori, ci jinymi darecky...

farmaceuticke firmy take zvou doktory na ruzne konference, na kterych jim predavdeji jak jejich prasky dobre funguji, a take kolik ruznych skvelych a barevnych prasku vyrabeji, farmaceuticke firmy jsou bohate a tak se na konfencich dobre ji a pije, a i na nejakou tu zabavu se penize najdou, doktori pak odjizdeji domu s taskami plnymi letaku, plakatu, prospektu a vzorku, plni krasneho pocitu, ze vse je v poradku, protoze na kazdy neduh existuje prasek a staci si jen pamatovat, ktery je na ktery, pripadne jak jednotlive prasky dobre zkombinovat, aby vyledna chemicka reakce nejspis probehla spravne...

nekdy si take doktori zacnou na konferencich povidat o tom, jakych zajimavych vysledku dosahli, kdyz tuhle zkusili na nejakem pacientovi zkombinovat ty a ty prasky, nebo kdyz nejaky prasek nezabral, jak se jim povedlo situaci vyresit tim, ze podali jiny, mnohem silnejsi prasek, a nebo co vsechno se da s timhle praskem delat...

dnesnim doktorum pomalu umira mozek a zrejme uz se to ani neda zastavit, tolikrat se v posleni dobe setkavam se situacemi, kdy doktori nejsou schopni vubec zjistit, co dotycnemu je a jen neustale opakovane zkouseji dalsi a dalsi prasky, az nejaky treba zabere, cit je nahrazovan hamiznosti, rozum tuposti...

jiste si snadno dokazete v predchozich odstavcich zamenit za doktora informatika a uvidite, ze rozdilu mnoho neni, dnesni informatik se podle meho nazoru pomalu ale jiste riti do stejne dusevni zahuby, do ktere se uz drive dostali doktori, stava se z nej bezduchy prostrednik s vybornymi znalostmi seznamu prasku, dobrymi vztahy s farmaceutickymi firmami a s prazdnym prostorem v sekci napady a inovace, v cechach to zrejme plati dvojnasob...

jiste mi muzete namitnout, ze je to vse nadsazene a velmi zobecnene, urcite se i dnes najdou doktori, kteri sve praci rozumi, a take leky, ktere opravdu pomohou, to je fakt, ale trend je myslim celkem jasny...

zaslechl jsem vcera, ze se planuje (nebo snad, ze uz dokonce probehlo), zalozeni gis-open-source komunity, jsem tim velmi nadsen! pevne verim, ze prave takovato iniciativa muze byt jednim z impulzu, ktery ceske informatiky muze zachranit, verim take, ze prave z takove komunity, a ji podobnych, mohou plynout myslenky hluboke i vzletne, ktere ve forme prispevku pozvednou uroven i takovych konferenci jako je treba ta prave skoncena...

je treba hledat a podporovat budouci doktory, kteri telu i duchu zase dobre porozumi a budou vedet kam sahnout, aby vse dobre fungovalo, aby se soukoli zase rozebehlo, vzdycky bude dost obchodniku s prasky, ale rozum by jen proto, ze je to trend, nemel vymrit...

tom

p.s. prikladam muj oblibeny citat naseho smalltalkerskeho poloboha:
<pre>
The real romance is out ahead and yet to come.
The computer revolution hasn't started yet.
Don't be misled by the enormous flow of money
into bad defacto standards
for unsophisticated buyers
using poor adaptations of
incomplete ideas.

Alan Kay
</pre>

== Diskusní skupina ==

Domlouvali jsme se taky, že by měla vzniknout diskusní e-mailová skupina. Napadlo mě že bychom zatím mohli využívat FreeGeoCZ http://mailman.fsv.cvut.cz/mailman/listinfo/freegeocz

== Možné společné téma a nástroj ==

Jako možné společné téma zatím krystalizuje PostGIS a v něm prostorová indexace vektorů a rastrů (třeba na sféroidu - asi sférický grid, "koule" rozdělená na plochy (nejčastěji trojúhelníky)). Přidejte co byste chtěli řešit vy.

[[Category:Local Chapters]]

Czech

2007-02-05T08:28:41Z

Wiki-Jencek: /* Zamyšlení od Tomáše Richty */

TODO

== Interested? ==
''Put you name / wiki account down below to show support moving forward''

;Martin [[User:Landa|Landa]]
:PhD student
:Department of Mapping and Cartography, [http://www.fsv.cvut.cz Faculty of Civil Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Jencek Růžička [[User:Jencek|Jencek]]
:lecturer
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics], [http://gis.vsb.cz/ruzicka Home Page]

;František Klímek [[User:frkl|frkl]]
:PhD student
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics]

;Tomáš Richta [[User:squatter|squatter]]
:PhD student, assistant professor
:[http://cs.felk.cvut.cz/webis/en/ Department of Computer Science and Engineering], [http://www.fel.cvut.cz/en/ Faculty of Electrical Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Aleš Daněk [[User:aldama|aldama]]
:analyst, coder
:[http://www.cad-programs.com/ CAD Programs - Ing. Jan Vlčinský]

== Zamyšlení od Tomáše Richty ==
ahoj,
nedalo mi to, a neodpustil jsem si vcera cestou z ostravy takovou malou uvahu,
tak vam ji dnes posilam, napadl me totiz takovy pekny primer...

doktori drive byli jini, dobre rozumeli lidskemu telu i duchu, mohli tak cloveka lecit zevrubnym pruzkumem toho, co mu chybi, nebo prebyva a naslednym doporucenim, jak telo a ducha primet k tomu, aby se vse zase vratilo do puvodniho rovnovazneho stavu a clovek se uzdravil...

dnes uz doktori leci jinak, podle priznaku pacienta vyberou vhodny prasek, ktery tyto priznaky potlaci, nebo pozmeni, tak aby byl pacient spokjen, ze se mu ulevilo, pri vyberu prasku je doktorum casto dobrym navodem castka, kterou dostali od sve oblibene farmaceuticke firmy za to, ze prave jeji prasky budou pacientum predepisovat, farmaceuticke firmy maji sve doktory rady, dobre si je zivi a dobre se o ne staraji, zasobuji je dobrymi prasky, letaky s obrazky s fotkami stastnych pacientu, kteri jedi prasky, tu a tam i nejakymi peknymi vylety k mori, ci jinymi darecky...

farmaceuticke firmy take zvou doktory na ruzne konference, na kterych jim predavdeji jak jejich prasky dobre funguji, a take kolik ruznych skvelych a barevnych prasku vyrabeji, farmaceuticke firmy jsou bohate a tak se na konfencich dobre ji a pije, a i na nejakou tu zabavu se penize najdou, doktori pak odjizdeji domu s taskami plnymi letaku, plakatu, prospektu a vzorku, plni krasneho pocitu, ze vse je v poradku, protoze na kazdy neduh existuje prasek a staci si jen pamatovat, ktery je na ktery, pripadne jak jednotlive prasky dobre zkombinovat, aby vyledna chemicka reakce nejspis probehla spravne...

nekdy si take doktori zacnou na konferencich povidat o tom, jakych zajimavych vysledku dosahli, kdyz tuhle zkusili na nejakem pacientovi zkombinovat ty a ty prasky, nebo kdyz nejaky prasek nezabral, jak se jim povedlo situaci vyresit tim, ze podali jiny, mnohem silnejsi prasek, a nebo co vsechno se da s timhle praskem delat...

dnesnim doktorum pomalu umira mozek a zrejme uz se to ani neda zastavit, tolikrat se v posleni dobe setkavam se situacemi, kdy doktori nejsou schopni vubec zjistit, co dotycnemu je a jen neustale opakovane zkouseji dalsi a dalsi prasky, az nejaky treba zabere, cit je nahrazovan hamiznosti, rozum tuposti...

jiste si snadno dokazete v predchozich odstavcich zamenit za doktora informatika a uvidite, ze rozdilu mnoho neni, dnesni informatik se podle meho nazoru pomalu ale jiste riti do stejne dusevni zahuby, do ktere se uz drive dostali doktori, stava se z nej bezduchy prostrednik s vybornymi znalostmi seznamu prasku, dobrymi vztahy s farmaceutickymi firmami a s prazdnym prostorem v sekci napady a inovace, v cechach to zrejme plati dvojnasob...

jiste mi muzete namitnout, ze je to vse nadsazene a velmi zobecnene, urcite se i dnes najdou doktori, kteri sve praci rozumi, a take leky, ktere opravdu pomohou, to je fakt, ale trend je myslim celkem jasny...

zaslechl jsem vcera, ze se planuje (nebo snad, ze uz dokonce probehlo), zalozeni gis-open-source komunity, jsem tim velmi nadsen! pevne verim, ze prave takovato iniciativa muze byt jednim z impulzu, ktery ceske informatiky muze zachranit, verim take, ze prave z takove komunity, a ji podobnych, mohou plynout myslenky hluboke i vzletne, ktere ve forme prispevku pozvednou uroven i takovych konferenci jako je treba ta prave skoncena...

je treba hledat a podporovat budouci doktory, kteri telu i duchu zase dobre porozumi a budou vedet kam sahnout, aby vse dobre fungovalo, aby se soukoli zase rozebehlo, vzdycky bude dost obchodniku s prasky, ale rozum by jen proto, ze je to trend, nemel vymrit...

tom

p.s. prikladam muj oblibeny citat naseho smalltalkerskeho poloboha:
<pre>
The real romance is out ahead and yet to come.
The computer revolution hasn't started yet.
Don't be misled by the enormous flow of money
into bad defacto standards
for unsophisticated buyers
using poor adaptations of
incomplete ideas.

Alan Kay
</pre>

== Diskusní skupina ==

Domlouvali jsme se taky, že by měla vzniknout diskusní e-mailová skupina. Napadlo mě že bychom zatím mohli využívat FreeGeoCZ http://mailman.fsv.cvut.cz/mailman/listinfo/freegeocz

[[Category:Local Chapters]]

Czech

2007-02-01T10:31:08Z

Wiki-Jencek: /* Interested? */

TODO

== Interested? ==
''Put you name / wiki account down below to show support moving forward''

;Martin [[User:Landa|Landa]]
:PhD student
:Department of Mapping and Cartography, [http://www.fsv.cvut.cz Faculty of Civil Engineering], [http://www.cvut.cz Czech Technical University in Prague]

;Jencek Růžička [[User:Jencek|Jencek]]
:lecturer
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics], [http://gis.vsb.cz/ruzicka Home Page]

;František Klímek [[User:frkl|frkl]]
;PhD student
:Institute of Geoinformatics, [http://gis.vsb.cz Institute of Geoinformatics]

== Zamyšlení od Tomáše Richty ==
ahoj,
nedalo mi to, a neodpustil jsem si vcera cestou z ostravy takovou malou uvahu,
tak vam ji dnes posilam, napadl me totiz takovy pekny primer...

doktori drive byli jini, dobre rozumeli lidskemu telu i duchu, mohli tak cloveka lecit zevrubnym pruzkumem toho, co mu chybi, nebo prebyva a naslednym doporucenim, jak telo a ducha primet k tomu, aby se vse zase vratilo do puvodniho rovnovazneho stavu a clovek se uzdravil...

dnes uz doktori leci jinak, podle priznaku pacienta vyberou vhodny prasek, ktery tyto priznaky potlaci, nebo pozmeni, tak aby byl pacient spokjen, ze se mu ulevilo, pri vyberu prasku je doktorum casto dobrym navodem castka, kterou dostali od sve oblibene farmaceuticke firmy za to, ze prave jeji prasky budou pacientum predepisovat, farmaceuticke firmy maji sve doktory rady, dobre si je zivi a dobre se o ne staraji, zasobuji je dobrymi prasky, letaky s obrazky s fotkami stastnych pacientu, kteri jedi prasky, tu a tam i nejakymi peknymi vylety k mori, ci jinymi darecky...

farmaceuticke firmy take zvou doktory na ruzne konference, na kterych jim predavdeji jak jejich prasky dobre funguji, a take kolik ruznych skvelych a barevnych prasku vyrabeji, farmaceuticke firmy jsou bohate a tak se na konfencich dobre ji a pije, a i na nejakou tu zabavu se penize najdou, doktori pak odjizdeji domu s taskami plnymi letaku, plakatu, prospektu a vzorku, plni krasneho pocitu, ze vse je v poradku, protoze na kazdy neduh existuje prasek a staci si jen pamatovat, ktery je na ktery, pripadne jak jednotlive prasky dobre zkombinovat, aby vyledna chemicka reakce nejspis probehla spravne...

nekdy si take doktori zacnou na konferencich povidat o tom, jakych zajimavych vysledku dosahli, kdyz tuhle zkusili na nejakem pacientovi zkombinovat ty a ty prasky, nebo kdyz nejaky prasek nezabral, jak se jim povedlo situaci vyresit tim, ze podali jiny, mnohem silnejsi prasek, a nebo co vsechno se da s timhle praskem delat...

dnesnim doktorum pomalu umira mozek a zrejme uz se to ani neda zastavit, tolikrat se v posleni dobe setkavam se situacemi, kdy doktori nejsou schopni vubec zjistit, co dotycnemu je a jen neustale opakovane zkouseji dalsi a dalsi prasky, az nejaky treba zabere, cit je nahrazovan hamiznosti, rozum tuposti...

jiste si snadno dokazete v predchozich odstavcich zamenit za doktora informatika a uvidite, ze rozdilu mnoho neni, dnesni informatik se podle meho nazoru pomalu ale jiste riti do stejne dusevni zahuby, do ktere se uz drive dostali doktori, stava se z nej bezduchy prostrednik s vybornymi znalostmi seznamu prasku, dobrymi vztahy s farmaceutickymi firmami a s prazdnym prostorem v sekci napady a inovace, v cechach to zrejme plati dvojnasob...

jiste mi muzete namitnout, ze je to vse nadsazene a velmi zobecnene, urcite se i dnes najdou doktori, kteri sve praci rozumi, a take leky, ktere opravdu pomohou, to je fakt, ale trend je myslim celkem jasny...

zaslechl jsem vcera, ze se planuje (nebo snad, ze uz dokonce probehlo), zalozeni gis-open-source komunity, jsem tim velmi nadsen! pevne verim, ze prave takovato iniciativa muze byt jednim z impulzu, ktery ceske informatiky muze zachranit, verim take, ze prave z takove komunity, a ji podobnych, mohou plynout myslenky hluboke i vzletne, ktere ve forme prispevku pozvednou uroven i takovych konferenci jako je treba ta prave skoncena...

je treba hledat a podporovat budouci doktory, kteri telu i duchu zase dobre porozumi a budou vedet kam sahnout, aby vse dobre fungovalo, aby se soukoli zase rozebehlo, vzdycky bude dost obchodniku s prasky, ale rozum by jen proto, ze je to trend, nemel vymrit...

tom

p.s. prikladam muj oblibeny citat naseho smalltalkerskeho poloboha:

"The real romance is out ahead and yet to come.
The computer revolution hasn't started yet.
Don't be misled by the enormous flow of money
into bad defacto standards
for unsophisticated buyers
using poor adaptations of
incomplete ideas."

[[Category:Local Chapters]]

Czech

2007-02-01T10:27:27Z

Wiki-Jencek: /* Interested? */

Czech

2007-02-01T10:26:50Z

Wiki-Jencek: /* Interested? */