Training Material for UN Open GIS OpenData - Revision history

Codrina: /* License= */

2019-02-20T10:54:35Z

License=

Codrina at 10:54, 20 February 2019

2019-02-20T10:54:16Z

Codrina: /* Educational Material */

2019-02-12T18:11:26Z

Educational Material

Codrina: /* Open geospatial information and its role in answering UN Sustainable Development Goals */

2019-02-11T13:26:46Z

Open geospatial information and its role in answering UN Sustainable Development Goals

Codrina: /* Educational Material */

2019-02-11T12:07:32Z

Educational Material

Show changes

Codrina: /* Educational Material */

2019-02-11T11:42:32Z

Educational Material

Codrina: /* Data and software used */

2019-01-29T12:15:34Z

Data and software used

Codrina at 12:15, 29 January 2019

2019-01-29T12:15:04Z

Codrina at 10:41, 21 January 2019

2019-01-21T10:41:30Z

Codrina at 10:39, 21 January 2019

2019-01-21T10:39:50Z

@@ Line 10: / Line 10: @@
 We have prepared this educational material for researchers, educators and professionals in local, regional, national or international agencies with minimal to intermediate geospatial information knowledge.
 We assume our audience has already basic knowledge of geospatial data structures, formats and that they have already used a GIS software, as to have basic skills and understanding of how to work with geospatial and tabular data. In that respect, we have limited the interactions with the command line, however we have inserted references to it.
-===License====
+===License===
 This educational material was prepared by Codrina Maria Ilie under the mentorship of prof. Maria Brovelli, Department of Civil and Environmental Engineering, Vice Rector for Como Campus at Politecnico di Milano and prof. Serena Coetzee, professor and Head of Department Geography, Geoinformatics and Meteorology, Director at the Centre for Geoinformation Science at University of Pretoria, within the [https://www.osgeo.org/foundation-news/osgeo-un-committee-educational-challenge/ OSGeo UN Committee Educational Challenge]. The material is distributed under [https://creativecommons.org/licenses/by-sa/4.0/legalcode Attribution-ShareAlike 4.0 International (CC BY-SA 4.0)] license.  In summary, according to CC BY-SA 4.0, you are free to:
 # Share — copy and redistribute the material in any medium or format
@@ Line 17: / Line 17: @@
 # Attribution — You must give appropriate credit, provide a link to the license, and indicate if changes were made. You may do so in any reasonable manner, but not in any way that suggests the licensor endorses you or your use,
 # ShareAlike — If you remix, transform, or build upon the material, you must distribute your contributions under the same license as the original.
 This work has been supported by the [https://www.osgeo.org/ Open Source Geospatial Foundation] - OSGeo - within the framework of the [https://www.osgeo.org/initiatives/un-initiative/ UN Open GIS Initiative].
 === Software used ===
 For the calculation of the SDG indicator, we have only used QGIS 3.4. We have also taken into consideration that changes that might occur from one version to another and thus focused on the functions used more than on a step-by-step guide.

@@ Line 10: / Line 10: @@
 We have prepared this educational material for researchers, educators and professionals in local, regional, national or international agencies with minimal to intermediate geospatial information knowledge.
 We assume our audience has already basic knowledge of geospatial data structures, formats and that they have already used a GIS software, as to have basic skills and understanding of how to work with geospatial and tabular data. In that respect, we have limited the interactions with the command line, however we have inserted references to it.
-=== Software used ===
+===License====
 For the calculation of the SDG indicator, we have only used QGIS 3.4. We have also taken into consideration that changes that might occur from one version to another and thus focused on the functions used more than on a step-by-step guide.

@@ Line 25: / Line 25: @@
 {| class="wikitable"
 |-
-! No. !! Topic !! Name collection/dataset !! Abstract !! indicators !! Producer/collector !! Owner !! License !! Type of data !! Format !! Scale/spatial resolution !! Edition !! CRS !! Geographic coverage !! URL
+! No. !! Topic !! Name collection/dataset !!  style="width: 60%;" | Abstract !! indicators !! Producer/collector !! Owner !! License !! Type of data !! Format !! Scale/spatial resolution !! Edition !! CRS !! Geographic coverage !! URL
 |-
-| 1 || Water || World Hydrogeological Map || style="width: 1000px;" | The objectives of the World-wide Hydrogeological Mapping and Assessment Programme (WHYMAP) are to summarise groundwater information on global scale, show groundwater data on maps and map applications. || groundwater resources, river and groundwater basins, karst aquifer || UNESCO&all || BGR and UNESCO own the copyright on the data and maps provided here.  ||  style="width: 200px;"The maps may be reproduced without further permission from the copyright owner provided that an acknowledgement to BGR & UNESCO is included in presentations and publications. || vector || Esri Geodatabase, SHP || Example || 2008 || Geographic WGS84 || Example || Example
+| 1 || Water || World Hydrogeological Map || The objectives of the World-wide Hydrogeological Mapping and Assessment Programme (WHYMAP) are to summarise groundwater information on global scale, show groundwater data on maps and map applications. || groundwater resources, river and groundwater basins, karst aquifer || UNESCO&all || BGR and UNESCO own the copyright on the data and maps provided here.  ||  style="width: 200px;"The maps may be reproduced without further permission from the copyright owner provided that an acknowledgement to BGR & UNESCO is included in presentations and publications. || vector || Esri Geodatabase, SHP || Example || 2008 || Geographic WGS84 || Example || Example
 |-
 | 2 || Water || Global Dataset of River Widths || Global Dataset of River Widths Developed from Landsat Imagery || river widths || Allen, George H., & Pavelsky, Tamlin M. || n/a || CC BY 4.0 || vector, raster || ESRI shapefile, TIFF || 30 m || 2018 || Geographic WGS84 || Global || https://zenodo.org/record/1297434

@@ Line 23: / Line 23: @@
 The idea of this training material is based on the fact that in the last two decades significant amounts of geospatial data have become freely available and accessible. This situation translated into a multitude of worldwide initiatives, as well as local web services and projects on an array of topics and, as an cyclical cause-effect relationship, this lead to a open data movement that spread from research data to public data, to community driven data. The following table presents a few freely available geospatial datasets that have been considered, with respect to their applicability for a geospatial SDG indicator analysis. Please be aware, this table is far from exhaustive, but more a live one. We invite anyone to add other available geospatial datasets.
-{| class="wikitable" style="width: 60%; height: 14em;"
+{| class="wikitable"
 |-
 ! No. !! Topic !! Name collection/dataset !! Abstract !! indicators !! Producer/collector !! Owner !! License !! Type of data !! Format !! Scale/spatial resolution !! Edition !! CRS !! Geographic coverage !! URL
 |-
-| 1 || Water || World Hydrogeological Map || The objectives of the World-wide Hydrogeological Mapping and Assessment Programme (WHYMAP) are to summarise groundwater information on global scale, show groundwater data on maps and map applications. || groundwater resources, river and groundwater basins, karst aquifer || UNESCO&all || BGR and UNESCO own the copyright on the data and maps provided here.  ||  The maps may be reproduced without further permission from the copyright owner provided that an acknowledgement to BGR & UNESCO is included in presentations and publications. || vector || Esri Geodatabase, SHP || Example || 2008 || Geographic WGS84 || Example || Example
+| 1 || Water || World Hydrogeological Map || style="width: 1000px;" | The objectives of the World-wide Hydrogeological Mapping and Assessment Programme (WHYMAP) are to summarise groundwater information on global scale, show groundwater data on maps and map applications. || groundwater resources, river and groundwater basins, karst aquifer || UNESCO&all || BGR and UNESCO own the copyright on the data and maps provided here.  ||  style="width: 200px;"The maps may be reproduced without further permission from the copyright owner provided that an acknowledgement to BGR & UNESCO is included in presentations and publications. || vector || Esri Geodatabase, SHP || Example || 2008 || Geographic WGS84 || Example || Example
 |-
 | 2 || Water || Global Dataset of River Widths || Global Dataset of River Widths Developed from Landsat Imagery || river widths || Allen, George H., & Pavelsky, Tamlin M. || n/a || CC BY 4.0 || vector, raster || ESRI shapefile, TIFF || 30 m || 2018 || Geographic WGS84 || Global || https://zenodo.org/record/1297434
@@ Line 68: / Line 68: @@
 Yet, given our area of interest, the Tabora county from Tanzania, we must take into consideration specific developments for Africa, more precisely, the [https://wiki.openstreetmap.org/wiki/Highway_Tag_Africa Highway Tag Africa - Topology of Road Network in African countries], and furthermore, the [https://wiki.openstreetmap.org/wiki/East_Africa_Tagging_Guidelines East Africa Tagging Guidelines]. <br/>
 However, with consideration to the global replicability of our educational material, we will also insert specifications on a more general scale. Of course, it must be acknowledged that the workflow presented here could require other adjustments with respect to the specificity of the road dataset used in calculation.
 ===Preparing the geospatial data===
 For the scope of this exercise we have chosen the Tabora county of Tanzania. As we strive to create an educational material that can be applied no matter the region of interest, a decision was made to use the available datasets, on a global level.

@@ Line 21: / Line 21: @@
 ==Educational Material==
 ===Open geospatial information and its role in answering UN Sustainable Development Goals===
 ''Population dataset description''<br />
 ''Global Administrative Units Dataset description''<br />
@@ Line 249: / Line 290: @@
 [[File:MapTemplate.qpt|thumbnail|Map template ]]
 [[File:MapTemplateLupe.qpt|thumbnail|The map template for all maps with lupe detail]]
 ==Known limitations of the material==
 The scope of this material is to show that we have entered in a time when using FOSS and open data we can initiate complex analysis allowing understanding of various facets of our environment. Works in this direction are already underway with a high focus on the knowledge we can extract from satellite imagery that have an open data policy, such as the Landsat or Copernicus programs. These works translate into global initiatives such as the [https://www.globalforestwatch.org/ Global Forest Watch] or [https://www.earthobservations.org/activity.php?id=122 Geo Wetlands Initiative] and so many more. Yet, we must not underestimate the power of what collaborative open mapping efforts can bring to our knowledge ecosystem and the OpenStreetMap project is a successful exemplification of that. Of course, one must not overlook matters related to the quality of data when it comes to the open collaborative environments, but as we have proven through this material, they provide a very good starting point that must not be neglected. As done in this study, careful analysis on understanding the datasets we are working with is crucial for eliminating as much of artificial errors of the final results as possible. Such decision is as the one we made, to eliminate roads that are shorter than 2 km and don't have more than 2 segments that would have given an artificial value of the Rural Access Index in a multitude of situations, as we presented. Nonetheless, we are aware that this decision might have lead to delete segments of roads in good condition that would be viable in the index calculation. It was a compromise that we acknowledge, after the analysing the data. Yet, it must be highlighted that this material is a first step to calculate a Sustainable Development Goal indicator, when official data is not available. Thus, in a specific case a more thorough and manual inspection of the open datasets is highly recommended. <br/>
 When it comes to software, FOSS is a stable choice as it has reached a maturity level that covers all sections of a geospatial analysis, from storing, to processing to visualization, be it online or offline, dynamic and interactive or static. For our material we have used a very popular GIS open source solution - QGIS 3, but it must be highlighted that it is by far the only solution. Most, if not all steps in the processing chain we made can be accomplished using other open source solutions, as well. Our choice to use QGIS was based on the idea of using an outright, intuitive and user friendly framework.

@@ Line 11: / Line 11: @@
 We assume our audience has already basic knowledge of geospatial data structures, formats and that they have already used a GIS software, as to have basic skills and understanding of how to work with geospatial and tabular data. In that respect, we have limited the interactions with the command line, however we have inserted references to it.
 === Data and software used ===
 For the calculation of the SDG indicator, we have only used QGIS 3.4. We have also taken into consideration that changes that might occur from one version to another and thus focused on the functions used more than on a step-by-step guide.
-The datasets used for our exercise are the following:
 === Acquired knowledge ===
 After going through the entire educational material, one will be able to:

@@ Line 177: / Line 177: @@
 [[File:Fig18_buffer-2km.jpg| thumb| center| 800px|Fig. 10 Exemplification of issues determined by the poor connectivity of road network]]
-Why not just delete the unconnected segments? Because there are such road segments that would be in the 2km buffer from the road network but stretch beyond the buffer, so if we were to delete all together, we would lose the buffer region of the unconnected segment. This case is exemplified in fig. 19.
+Why not just delete the unconnected segments? Because there are such road segments that would be in the 2km buffer from the road network but stretch beyond the buffer, so if we were to delete all together, we would lose the buffer region of the unconnected segment. This case is exemplified in fig. 11.
 [[File:fig19_buffer-2km.jpg|thumb| center| 800px|Fig.11 Unconnected road segment to be considered in the Tabora district RAI calculation]]

@@ Line 126: / Line 126: @@
 GroupStats is a plugin for easy calculation of statistics for features in a vector layer. The plugin has a control panel that allows the user to make calculations (as in our example, length) for specific group features, as can be seen in attached file 1.
-[[File:fig7_groupStats.png|thumb|center|400px|Example of calculation the length for roads, per categories]]
+[[File:fig7_groupStats.png|thumb|center|400px|Fig. 5 Example of calculation the length for roads, per categories]]
 A look at the complete dataset tags shows that we can find any kind of combination of the tags we are interested in. We can observe that, for our test dataset, we have 100% the HIGHWAY tag completed for all roads segments.
@@ Line 156: / Line 156: @@
 [[File:fig12_length2000_initial_Stats.png|thumb|center|500px|The total length of road segments shorter than 2 km, by road, surface and smoothness type, before the merge operation]]
-[[File:fig13_length2000_initial.jpg|thumb|center|800px|Spatial distribution of road segments shorter than 2 km]]
+[[File:fig13_length2000_initial.jpg|thumb|center|800px| Fig. 7 Spatial distribution of road segments shorter than 2 km]]
 However, by looking at the geometry, we can see that a significant part of these short segments are actually completing the longer road segments. Thus, we need to connect all these segments, in order to be able to delete the features that are shorter than 2km AND not connected to any other road feature.
@@ Line 167: / Line 167: @@
 Next, we will create the 2km buffer zone around the road network. We need to identify whether there are any buffers disconnected from the network buffer. Such a case would indicate that again, there are roads in good or fair condition, that are not connected to the main road network, thus including into the RAI calculation would offer an artificial result, as it can be seen in the map below.
-[[File:fig16_buffer_initial.jpeg|thumb|center|800px|Spatial distribution of unconnected road buffers]]
+[[File:fig16_buffer_initial.jpeg|thumb|center|800px|Fig. 8 Spatial distribution of unconnected road buffers]]
 Before deleting the unconnected buffers, we will check to see whether the corresponding roads are not on the border. In this case, we decide not to remove them because, they might be connected to roads outside the Tabora county.
-[[File:fig17_buffer_RAI.jpeg|thumb|center|800px|Example of unconnected road buffers that are intersecting the county border]]
+[[File:fig17_buffer_RAI.jpeg|thumb|center|800px|Fig. 9 Example of unconnected road buffers that are intersecting the county border]]
 After deleting the disconnected buffer zones, we can identify a new situation that may lead to an artificial result for RAI. As shown in fig. 18, the issue can occur when we have 2 disconnected road segments that have overlapping 2km buffers and yet, the distance that one must walk from segment A to segment B is between 2.01 km and 3.99 km. Hence, the buffer should not include this area.
-[[File:Fig18_buffer-2km.jpg| thumb| center| 800px| Exemplification of issues determined by the poor connectivity of road network]]
+[[File:Fig18_buffer-2km.jpg| thumb| center| 800px|Fig. 10 Exemplification of issues determined by the poor connectivity of road network]]
 Why not just delete the unconnected segments? Because there are such road segments that would be in the 2km buffer from the road network but stretch beyond the buffer, so if we were to delete all together, we would lose the buffer region of the unconnected segment. This case is exemplified in fig. 19.
-[[File:fig19_buffer-2km.jpg|thumb| center| 800px|Fig.19 Unconnected road segment to be considered in the Tabora district RAI calculation]]
+[[File:fig19_buffer-2km.jpg|thumb| center| 800px|Fig.11 Unconnected road segment to be considered in the Tabora district RAI calculation]]
 To overcome this issue, we will identify all connected segments of the road and create a 2km buffer zone for the entire network. As our dataset is substantial (over 13000 features), we will split all vectors by a 50X50 grid.
@@ Line 186: / Line 186: @@
 [Vector]--> [Data Management Tools] --> [Split vector layer] - creates a road layer for each grid cells.
-[[File:fig20_roads_GRID.jpg|thumb| center| 800px|Fig. 20 The 50X50KM grid and the corresponding road segments]]
+[[File:fig20_roads_GRID.jpg|thumb| center| 800px|Fig. 12 The 50X50KM grid and the corresponding road segments]]
 Now that we have prepared the road dataset, we will run [https://drive.google.com/file/d/16u_plF7q2TH0HuzWLi_iQvV6C3tkmndL/view the script] (by [https://gis.stackexchange.com/users/63747/jochen-schwarze Jochen Schwarze]) that creates networks, by merging adjacent lines. The script produces a new dataset from the input data with an attribute “subnet” that contains an incremented ID for each subnetwork created.
-[[File:fig21_roads_GRID_results.jpg|thumb| center| 800px|Fig. 21 Result of the subnet creation script]]
+[[File:fig21_roads_GRID_results.jpg|thumb| center| 800px|Fig. 13 Result of the subnet creation script]]
 [Vector]--> [Geoprocessing Tools] --> [Dissolve] - the roads that all connected will be dissolved into one feature - we will run the algorithm in batch processing mode.
@@ Line 206: / Line 206: @@
 A quick analysis of the spatial distribution shows that the short road segments do not represent small disconnected parts of the road network, but rather disparate segments that do not bring a significant improvement to the road network (fig.22, fig.23).
-[[File:fig22_networks-2000.jpg|thumb| center| 800px| Fig. 22 Spatial distribution of road segments of computed networks, by lengths (km)]]
+[[File:fig22_networks-2000.jpg|thumb| center| 800px| Fig. 14 Spatial distribution of road segments of computed networks, by lengths (km)]]
-[[File:fig23_segments-2000.jpg|thumb| center| 800px|Fig. 23 Spatial distribution of the shorter than 2 km road segments]]
+[[File:fig23_segments-2000.jpg|thumb| center| 800px|Fig. 15 Spatial distribution of the shorter than 2 km road segments]]
 To clean the road network in preparation for the 2km buffer calculation, we will eliminate the roads that have 1 or 2 segments, with a total length below 2 km.
@@ Line 239: / Line 239: @@
 We do that within the [Properties] of the Administrative units level 3 vector layer [Joins] --> [join attribute values] from the 2 km buffer zone by the name of the administrative unit.
-[[File:fig24_RAI.jpg|thumb| center| 800px|Fig. 24 Rural Access Index calculated for Tabora county, Tanzania]]
+[[File:fig24_RAI.jpg|thumb| center| 800px|Fig. 16 Rural Access Index calculated for Tabora county, Tanzania]]
 We have enclosed all steps of the processing chain detailed above in a conceptual schema of the workflow necessary to calculate the Rural Access Index using open data and open source software.
-[[File:fig25_workflow.jpg|thumb| center| 1000px|Fig. 25 Conceptual schema of the employed workflow]]
+[[File:fig25_workflow.jpg|thumb| center| 1000px|Fig. 17 Conceptual schema of the employed workflow]]
 === Geospatial data representations===