OSGeo - User contributions [en]

Geodata Discovery Working Group

2007-04-04T06:51:57Z

Wiki-K1 kabiri: /* Participants */

This is a [[Working Group]] started by the [[Public Geospatial Data Committee]].

''NOTICE'': There are two BOFs on this topic at [[OSGeo_at_FOSS4G2006#BOFs|FOSS4G2006]] on ''Geodata'' and about ''Geodata Discovery and Metadata Models''.

While OSGeo can consider hosting sets of public geodata, (and for the purposes of offering maintained [[Geodata Packaging Working Group|packages]] of high-quality data for education and demonstration, it probably will), we can compile a set of best practises and references to good prior art for building and maintaining collections of data and metadata.

== Focus ==

* Metadata for collections of geographic information.
* Providing best practise and prototypes for geodata search facilities
* Help with providing input into the OSGeo repository/services at http://osgeo.telascience.org/
* Providing 'catalog'/discovery/search services as part of OSGeo activities in promoting access to public domain and open licensed bodies of data.

== Participants ==

* [[User:JoWalsh|Jo Walsh]]
* [[User:SchuylerErle|Schuyler Erle]]
* [[User:bitner|David Bitner]]
* [[User:Pnaciona|Perry Nacionales]]
* [[User:Neteler|Markus Neteler]]
* [[User:Arnulf Christl|Arnulf Christl]]
* [[User:virtualmemorys|Bob Wang]]
* [[User:Ocantan|Oscar Cantán]]
* [[User:Sfkeller|Stefan Keller]]
* [[User:candrsn|Carl Anderson]]
* [[User:K1_Kabiri|Keivan Kabiri]]
* ''please add yourself''

== Existing (Meta) Search Projects and Related Efforts ==

More on metadata models at [[Geodata_Metadata_Requirements]]

Search services:
* See [[Geodata_Metadata_Requirements#Discovery]]
* (Meta) Data Catalogues / Data Repositories
** [http://www.mapdex.org/search/ Mapdex] is "a global Index of 2,018 servers, serving 37,168 map services, containing over 400,000 GIS Layers, covering more than 4,000,000 columns!"
** [http://www.geotorrent.org/ GeoTorrent] hosts BitTorrents of shapefile/TIFF format data.
** [http://www.fao.org/geonetwork/ GeoNetwork] "Find Interactive Maps, GIS datasets, Satellite Imagery and Related Applications"
** [http://www.dlese.org/libdev/dcs_overview.html DLESE Collection System] offers a metadata repository and editor and supports OAI-PMH as data provider and harvester
** [http://gis.hsr.ch/gmdb/ Geo Meta Data Base (GMDB)] - Metadata repository and editor based on ISO 19115/119 Metadata and Dublin Core with XML export and OAI-PMH data provider support (german only)
** GDI Portals (country specific): [http://www.geodata.gov US], [http://www.gogeo.ac.uk UK], [http://www.geoportal.bund.de D], [http://www.geoland.at/ A], [http://www.geocat.ch CH], [http://geodaten.llv.li/geoshop/public.html LI], [http://www.geoportail.fr/ F], ...
* (Meta) Search engines:
** [http://geometa.info/ geometa.info] - search engine for geospatial services, data and documents combining structured and unstructured data (currently german only)
** [http://www.google.com/search?q=allinurl%3A+getcapabilities Google search] using 'allinurl' parameter

Lists about data sources and services:
* Lists on web services:
** http://www.skylab-mobilesystems.com/ger/wms_serverlist.html simple WMS service URL list
** http://www.refractions.net/white_papers/ogcsurvey/ Automated OGC Survey on the Web (WMF, WFS, ...)
** http://geometa.info/search.jsp?query=type%3Awms WMS search engine
** http://exploreourpla.net/maps/
** http://wms-sites.com/ catalog of WMS sites
** [[Orthophotography]] Annotated list of Orthophotography WMS Services
* Lists on data services:
** http://datagateway.nrcs.usda.gov/ US data servers

== Public Datasets for Possible Mirroring or Distribution ==

=== Gazetteer Data ===
... basically placenames with coordinates ...

* [http://www.geonames.org/ Geonames.org] gazetteer (includes [http://earth-info.nga.mil/gns/html/ NGA GNS Gazetteer])
* [http://prdownloads.sourceforge.net/nasa-exp/BlueMarble-Placenames.zip?download WorldWind Placenames] (markusN can provide PERL to read this)
* [http://lists.burri.to/pipermail/geowanking/2007-March/003772.html Geowanking-List: Toponymic accurracy of Google and Yahoo Maps] => a very good example why releasing this data is useful for public authorities and the society

=== Vector data ===
* VMAP0 - 1:1Mio, very generalised vector data for many features, around 70 [http://www.terragear.org/docs/vmap0/coverage.html layers] (already at telascience) - see also [http://www.mapability.com/info/vmap0_index.html here] for docs
* [http://aprsworld.net/gisdata/world/ World map for APRS], 1:1.1mio
* [http://www.soest.hawaii.edu/wessel/gshhs/gshhs.html GSHHS] - world coastlines, different scales available
* TIGER/Line (USA) - street and addressing data for the US
* RNF (Canada) - street and addressing data for Canada
* [http://www.openstreetmap.org OpenStreetMap] - data available through a [http://wiki.openstreetmap.org/index.php/REST HTTP based XML interface]. They've also been getting requests for shapefile and GML output of their data.
* [http://www.maplibrary.org/stacks/Africa_lite.asp African geodata]

=== Raster data ===
* DEM
** [ftp://e0srp01u.ecs.nasa.gov/srtm/version2 SRTM V2]
** [http://www.ngdc.noaa.gov/mgg/image/2minrelief.html ETOPO2]
** [http://topex.ucsd.edu/WWW_html/srtm30_plus.html TOPEX]: SRTM30 (with GTOPO30 data for high latitudes where SRTM data are not available)
** [http://ekman.csc.noaa.gov/TCM/ NOAA TCM] LIDAR / IfSAR
* Satellite Data
** Landsat ([http://onearth.jpl.nasa.gov/ OnEarth] - [http://telascience.sdsc.edu/tela_data/onearth/ mirror already at telascience])
** [http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ Blue Marble] / [http://www.iac.ethz.ch/staff/stockli/bmng/ Blue Marble Next Generation] (mirror already at telascience)
** ASTER (from GLCF)
** MODIS (from NASA EDC)
** USGS Urban Orthos (URL?)
* Maps derived from Satellite Data
** Nested Imagery Layer (Best of -- resolution -- with seamless zoom through multiple levels)
** [http://glcf.umiacs.umd.edu/data/landcover/ AVHRR Landuse Map] from GLCF
** [http://www.shadedrelief.com/natural/ natural earth]
** [http://dmsp.ngdc.noaa.gov/html/projects/lights.html City lights]
** [http://www.worldwildlife.org/freshwater/hydrosheds.cfm HydroSHEDS] - Hydrological data and maps based on SHuttle Elevation Derivatives at multiple Scales
** [http://unosat.web.cern.ch/unosat/ UNOSAT imagery] http://unosat.web.cern.ch/unosat/asp/prod_free.asp

== Wishlist for Public Datasets ==

* Orthophotos
** Possible sources:
** http://www.fgdl.org/download/index.html
** http://datagateway.nrcs.usda.gov/

== Search Protocols ==

* GIS specific:
** OGC's CSW 2.0 (work in progress...)
*** [http://www.opengeospatial.org/standards/requests/29 OpenGIS Catalogue Services - ebRIM (ISO/TS 15000-3) profile of CSW]
*** (ISO/OGC's "ISO19115/ISO19119 Application Profile for CSW 2.0")
*** See a comparison between [http://www.gis.hsr.ch/wiki/OAI-PMH CSW and OAI-PMH]
** OGC's WFS (profile) serving (profiled) ISO 19115/19119 metadata format...?
** 'Lean and mean' proposals on project level...
*** [[Simple Catalog Interface]]
* Related search protocols - possibly to be profiled or specialized by geoinformation requirements:
** [http://www.openarchives.org/OAI/openarchivesprotocol.html Open Archives Initiative Protocol for Metadata Harvesting 2.0] (OAI-PMH)
** [http://www.loc.gov/standards/sru/ Search/Retrieve via URL] (SRU/SRW)
** [http://en.wikipedia.org/wiki/OpenSearch OpenSearch] proposal from A9
** [http://lcweb.loc.gov/z3950/agency/document.html Z39.50 ANSI/NISO standard protocol]
** [http://www.ebxml.org/ ebXML]/ebRIM Messaging Service 2.0

== Hosting Plan ==

(placeholder for an initial hosting plan sketch)

== See Also ==

* [[Geodata Packaging Working Group]]
* [[Geodata Licensing Working Group]]
* [[WMS Tile Caching]] proposal
* [[Geodata processing at telascience.org]]

Geodata Discovery Working Group

2007-04-04T06:51:14Z

Wiki-K1 kabiri: /* Participants */

This is a [[Working Group]] started by the [[Public Geospatial Data Committee]].

''NOTICE'': There are two BOFs on this topic at [[OSGeo_at_FOSS4G2006#BOFs|FOSS4G2006]] on ''Geodata'' and about ''Geodata Discovery and Metadata Models''.

While OSGeo can consider hosting sets of public geodata, (and for the purposes of offering maintained [[Geodata Packaging Working Group|packages]] of high-quality data for education and demonstration, it probably will), we can compile a set of best practises and references to good prior art for building and maintaining collections of data and metadata.

== Focus ==

* Metadata for collections of geographic information.
* Providing best practise and prototypes for geodata search facilities
* Help with providing input into the OSGeo repository/services at http://osgeo.telascience.org/
* Providing 'catalog'/discovery/search services as part of OSGeo activities in promoting access to public domain and open licensed bodies of data.

== Participants ==

* [[User:JoWalsh|Jo Walsh]]
* [[User:SchuylerErle|Schuyler Erle]]
* [[User:bitner|David Bitner]]
* [[User:Pnaciona|Perry Nacionales]]
* [[User:Neteler|Markus Neteler]]
* [[User:Arnulf Christl|Arnulf Christl]]
* [[User:virtualmemorys|Bob Wang]]
* [[User:Ocantan|Oscar Cantán]]
* [[User:Sfkeller|Stefan Keller]]
* [[User:candrsn|Carl Anderson]]
* [[User:candrsn|Keivan Kabiri]]
* ''please add yourself''

== Existing (Meta) Search Projects and Related Efforts ==

More on metadata models at [[Geodata_Metadata_Requirements]]

Search services:
* See [[Geodata_Metadata_Requirements#Discovery]]
* (Meta) Data Catalogues / Data Repositories
** [http://www.mapdex.org/search/ Mapdex] is "a global Index of 2,018 servers, serving 37,168 map services, containing over 400,000 GIS Layers, covering more than 4,000,000 columns!"
** [http://www.geotorrent.org/ GeoTorrent] hosts BitTorrents of shapefile/TIFF format data.
** [http://www.fao.org/geonetwork/ GeoNetwork] "Find Interactive Maps, GIS datasets, Satellite Imagery and Related Applications"
** [http://www.dlese.org/libdev/dcs_overview.html DLESE Collection System] offers a metadata repository and editor and supports OAI-PMH as data provider and harvester
** [http://gis.hsr.ch/gmdb/ Geo Meta Data Base (GMDB)] - Metadata repository and editor based on ISO 19115/119 Metadata and Dublin Core with XML export and OAI-PMH data provider support (german only)
** GDI Portals (country specific): [http://www.geodata.gov US], [http://www.gogeo.ac.uk UK], [http://www.geoportal.bund.de D], [http://www.geoland.at/ A], [http://www.geocat.ch CH], [http://geodaten.llv.li/geoshop/public.html LI], [http://www.geoportail.fr/ F], ...
* (Meta) Search engines:
** [http://geometa.info/ geometa.info] - search engine for geospatial services, data and documents combining structured and unstructured data (currently german only)
** [http://www.google.com/search?q=allinurl%3A+getcapabilities Google search] using 'allinurl' parameter

Lists about data sources and services:
* Lists on web services:
** http://www.skylab-mobilesystems.com/ger/wms_serverlist.html simple WMS service URL list
** http://www.refractions.net/white_papers/ogcsurvey/ Automated OGC Survey on the Web (WMF, WFS, ...)
** http://geometa.info/search.jsp?query=type%3Awms WMS search engine
** http://exploreourpla.net/maps/
** http://wms-sites.com/ catalog of WMS sites
** [[Orthophotography]] Annotated list of Orthophotography WMS Services
* Lists on data services:
** http://datagateway.nrcs.usda.gov/ US data servers

== Public Datasets for Possible Mirroring or Distribution ==

=== Gazetteer Data ===
... basically placenames with coordinates ...

* [http://www.geonames.org/ Geonames.org] gazetteer (includes [http://earth-info.nga.mil/gns/html/ NGA GNS Gazetteer])
* [http://prdownloads.sourceforge.net/nasa-exp/BlueMarble-Placenames.zip?download WorldWind Placenames] (markusN can provide PERL to read this)
* [http://lists.burri.to/pipermail/geowanking/2007-March/003772.html Geowanking-List: Toponymic accurracy of Google and Yahoo Maps] => a very good example why releasing this data is useful for public authorities and the society

=== Vector data ===
* VMAP0 - 1:1Mio, very generalised vector data for many features, around 70 [http://www.terragear.org/docs/vmap0/coverage.html layers] (already at telascience) - see also [http://www.mapability.com/info/vmap0_index.html here] for docs
* [http://aprsworld.net/gisdata/world/ World map for APRS], 1:1.1mio
* [http://www.soest.hawaii.edu/wessel/gshhs/gshhs.html GSHHS] - world coastlines, different scales available
* TIGER/Line (USA) - street and addressing data for the US
* RNF (Canada) - street and addressing data for Canada
* [http://www.openstreetmap.org OpenStreetMap] - data available through a [http://wiki.openstreetmap.org/index.php/REST HTTP based XML interface]. They've also been getting requests for shapefile and GML output of their data.
* [http://www.maplibrary.org/stacks/Africa_lite.asp African geodata]

=== Raster data ===
* DEM
** [ftp://e0srp01u.ecs.nasa.gov/srtm/version2 SRTM V2]
** [http://www.ngdc.noaa.gov/mgg/image/2minrelief.html ETOPO2]
** [http://topex.ucsd.edu/WWW_html/srtm30_plus.html TOPEX]: SRTM30 (with GTOPO30 data for high latitudes where SRTM data are not available)
** [http://ekman.csc.noaa.gov/TCM/ NOAA TCM] LIDAR / IfSAR
* Satellite Data
** Landsat ([http://onearth.jpl.nasa.gov/ OnEarth] - [http://telascience.sdsc.edu/tela_data/onearth/ mirror already at telascience])
** [http://earthobservatory.nasa.gov/Newsroom/BlueMarble/ Blue Marble] / [http://www.iac.ethz.ch/staff/stockli/bmng/ Blue Marble Next Generation] (mirror already at telascience)
** ASTER (from GLCF)
** MODIS (from NASA EDC)
** USGS Urban Orthos (URL?)
* Maps derived from Satellite Data
** Nested Imagery Layer (Best of -- resolution -- with seamless zoom through multiple levels)
** [http://glcf.umiacs.umd.edu/data/landcover/ AVHRR Landuse Map] from GLCF
** [http://www.shadedrelief.com/natural/ natural earth]
** [http://dmsp.ngdc.noaa.gov/html/projects/lights.html City lights]
** [http://www.worldwildlife.org/freshwater/hydrosheds.cfm HydroSHEDS] - Hydrological data and maps based on SHuttle Elevation Derivatives at multiple Scales
** [http://unosat.web.cern.ch/unosat/ UNOSAT imagery] http://unosat.web.cern.ch/unosat/asp/prod_free.asp

== Wishlist for Public Datasets ==

* Orthophotos
** Possible sources:
** http://www.fgdl.org/download/index.html
** http://datagateway.nrcs.usda.gov/

== Search Protocols ==

* GIS specific:
** OGC's CSW 2.0 (work in progress...)
*** [http://www.opengeospatial.org/standards/requests/29 OpenGIS Catalogue Services - ebRIM (ISO/TS 15000-3) profile of CSW]
*** (ISO/OGC's "ISO19115/ISO19119 Application Profile for CSW 2.0")
*** See a comparison between [http://www.gis.hsr.ch/wiki/OAI-PMH CSW and OAI-PMH]
** OGC's WFS (profile) serving (profiled) ISO 19115/19119 metadata format...?
** 'Lean and mean' proposals on project level...
*** [[Simple Catalog Interface]]
* Related search protocols - possibly to be profiled or specialized by geoinformation requirements:
** [http://www.openarchives.org/OAI/openarchivesprotocol.html Open Archives Initiative Protocol for Metadata Harvesting 2.0] (OAI-PMH)
** [http://www.loc.gov/standards/sru/ Search/Retrieve via URL] (SRU/SRW)
** [http://en.wikipedia.org/wiki/OpenSearch OpenSearch] proposal from A9
** [http://lcweb.loc.gov/z3950/agency/document.html Z39.50 ANSI/NISO standard protocol]
** [http://www.ebxml.org/ ebXML]/ebRIM Messaging Service 2.0

== Hosting Plan ==

(placeholder for an initial hosting plan sketch)

== See Also ==

* [[Geodata Packaging Working Group]]
* [[Geodata Licensing Working Group]]
* [[WMS Tile Caching]] proposal
* [[Geodata processing at telascience.org]]