QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (2024)

IFPB

Ricardo Topo 09/10/2024

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (2)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (3)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (4)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (5)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (6)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (7)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (8)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (9)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (10)

QGIS 2 Cookbook Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (11)

Prévia do material em texto

<p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS 2 Cookbook</p><p>Become a QGIS power user and master QGIS</p><p>data management, visualization, and spatial</p><p>analysis techniques</p><p>Alex Mandel</p><p>Víctor Olaya Ferrero</p><p>Anita Graser</p><p>Alexander Bruy</p><p>BIRMINGHAM - MUMBAI</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS 2 Cookbook</p><p>Copyright © 2016 Packt Publishing</p><p>All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or</p><p>transmitted in any form or by any means, without the prior written permission of the publisher,</p><p>except in the case of brief quotations embedded in critical articles or reviews.</p><p>Every effort has been made in the preparation of this book to ensure the accuracy of the</p><p>information presented. However, the information contained in this book is sold without</p><p>warranty, either express or implied. Neither the authors, nor Packt Publishing, and its dealers</p><p>and distributors will be held liable for any damages caused or alleged to be caused directly</p><p>or indirectly by this book.</p><p>Packt Publishing has endeavored to provide trademark information about all of the companies</p><p>and products mentioned in this book by the appropriate use of capitals. However, Packt</p><p>Publishing cannot guarantee the accuracy of this information.</p><p>First published: April 2016</p><p>Production reference: 1250416</p><p>Published by Packt Publishing Ltd.</p><p>Livery Place</p><p>35 Livery Street</p><p>Birmingham B3 2PB, UK.</p><p>ISBN 978-1-78398-496-1</p><p>www.packtpub.com</p><p>www.it-ebooks.info</p><p>www.packtpub.com</p><p>http://www.it-ebooks.info/</p><p>Credits</p><p>Authors</p><p>Alex Mandel</p><p>Víctor Olaya Ferrero</p><p>Anita Graser</p><p>Alexander Bruy</p><p>Reviewers</p><p>Jorge Arévalo</p><p>Olivier Dalang</p><p>Ben Mearns</p><p>Commissioning Editor</p><p>Pramila Balan</p><p>Acquisition Editor</p><p>Vinay Argekar</p><p>Content Development Editor</p><p>Amey Varangaonkar</p><p>Merint Mathew</p><p>Technical Editor</p><p>Dhiraj Chandanshive</p><p>Copy Editor</p><p>Priyanka Ravi</p><p>Project Coordinator</p><p>Suzanne Coutinho</p><p>Proofreader</p><p>Safis Editing</p><p>Indexer</p><p>Rekha Nair</p><p>Production Coordinator</p><p>Aparna Bhagat</p><p>Cover Work</p><p>Aparna Bhagat</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>About the Authors</p><p>Alex Mandel is a geospatial scientist who has a PhD in geography and more than 12 years</p><p>of experience in applying GIS to a variety of projects. He has also taught courses on GIS,</p><p>geospatial programming, and Geoweb.</p><p>Víctor Olaya Ferrero is a GIS developer. He is the creator and main developer of the QGIS</p><p>Processing Framework. He is also the author of Sistemas de Información Geográfica, a free</p><p>book about the fundamentals of GIS.</p><p>I would like to thank my co-authors and everyone else in the QGIS</p><p>community.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Anita Graser studied geomatics at the University of Applied Sciences Wiener Neustadt,</p><p>Austria, from where she graduated with a master's degree in 2010. During her study, she</p><p>gained hands-on experience in the fields of geo-marketing and transportation research. Since</p><p>2007, she has been working as a geographic information system (GIS) expert with the</p><p>dynamic transportation systems group at the Austrian Institute of Technology (AIT), where</p><p>she focuses on analyzing and visualizing spatio-temporal data.</p><p>Anita serves on the OSGeo board of directors and the QGIS project steering committee. She</p><p>has been working with GIS since 2005, provides QGIS training courses, and writes a popular</p><p>blog on open source GIS at anitagraser.com.</p><p>I would like to thank my family, partner, and coworkers for their support</p><p>and encouragement. Of course, I also want to thank the whole QGIS</p><p>community for their continued efforts to provide the best open source</p><p>GIS experience possible.</p><p>Alexander Bruy is a GFOSS advocate and open source developer working on the QGIS</p><p>project. He has also maintained a collection of his own open source projects. He has been</p><p>working with QGIS since 2006, and now he is an OSGeo charter member and QGIS core</p><p>developer. He is also the author of QGIS By Example, Packt Publishing.</p><p>Alexander is currently a freelance GIS developer and works for various companies worldwide.</p><p>www.it-ebooks.info</p><p>anitagraser.com</p><p>http://www.it-ebooks.info/</p><p>About the Reviewers</p><p>Jorge Arévalo is a computer engineer from Universidad Autónoma de Madrid, UAM. He</p><p>started developing web applications with JS, PHP, and Python in 2007. In 2010, he began</p><p>collaborating with PostGIS and GDAL projects after participating in GSoC 2009, creating the</p><p>PostGIS Raster GDAL driver. He currently works as a technology trainer and Python/Django</p><p>developer. He also organizes hackathons with others at http://hackathonlovers.com/.</p><p>Jorge Arévalo has co-written the book Instant Zurb Foundation 4 for Packt Publishing. He has</p><p>also worked as reviewer for the books PostGIS Cookbook, OpenLayers 3 Beginner's Guide,</p><p>and Getting Started with Memcached, all of them for Packt Publishing.</p><p>I want to thank my wife Elena for her continuous love and support while</p><p>reviewing this book.</p><p>Olivier Dalang completed his master's degree in architecture and urban planning from</p><p>EPFL, Switzerland. He then worked as an urban planner at Team+ as a volunteer for the</p><p>NGO urbaMonde, which is active in Senegal, and as a researcher and lecturer at EPFL on</p><p>the Venice Time Machine project. He got more and more acquainted with QGIS through the</p><p>different positions he worked in. Being a programmer, he developed a few plugins, of which</p><p>a few are now in the core.</p><p>www.it-ebooks.info</p><p>http://hackathonlovers.com/</p><p>http://www.it-ebooks.info/</p><p>Ben Mearns lives in Philly, PA, where he consults, teaches, advises, speaks, and creates</p><p>geographical information. In private practice, he has previously been the lead geospatial</p><p>information consultant and instructor on GIS for natural resource management at the University</p><p>of Delaware. He has held other GIS and data positions at the Cartographic Modeling Lab at the</p><p>University of Pennsylvania, Princeton University, and Macalester College. He is currently writing</p><p>QGIS Blueprints with Packt Publishing, which will soon be available in the market.</p><p>I would like to thank my girlfriend, Catherine Moore, and mother, Raiana</p><p>Mearns, for their support during the review of this book.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>www.PacktPub.com</p><p>eBooks, discount offers, and more</p><p>Did you know that Packt offers eBook versions of every book published, with PDF and ePub</p><p>files available? You can upgrade to the eBook version at www.PacktPub.com and as a print</p><p>book customer, you are entitled to a discount on the eBook copy. Get in touch with us at</p><p>customercare@packtpub.com for more details.</p><p>At www.PacktPub.com, you can also read a collection of free technical articles, sign up</p><p>for a range of free newsletters and receive exclusive discounts and offers on Packt books</p><p>and eBooks.</p><p>TM</p><p>https://www2.packtpub.com/books/subscription/packtlib</p><p>Do you need instant solutions to your IT questions? PacktLib is Packt's online digital book</p><p>library. Here, you can search, access, and read Packt's entire library of books.</p><p>Why Subscribe?</p><p>f Fully searchable across every book published by Packt</p><p>f Copy and paste, print, and bookmark content</p><p>f On demand and accessible via a web browser</p><p>www.it-ebooks.info</p><p>www.PacktPub.com</p><p>www.PacktPub.com</p><p>https://www2.packtpub.com/books/subscription/packtlib</p><p>http://www.it-ebooks.info/</p><p>i</p><p>Table of Contents</p><p>Preface v</p><p>Chapter 1: Data Input and Output 1</p><p>Introduction 1</p><p>Finding geospatial data on your computer 2</p><p>Describing data sources 6</p><p>Importing data from text files 10</p><p>Importing KML/KMZ files 16</p><p>Importing DXF/DWG files 17</p><p>Opening a NetCDF file 19</p><p>Saving a vector layer 21</p><p>Saving a raster layer 22</p><p>Reprojecting a layer 24</p><p>Batch format conversion 25</p><p>Batch reprojection 29</p><p>Loading vector layers into SpatiaLite 31</p><p>Loading vector layers into PostGIS 34</p><p>Chapter 2: Data Management 39</p><p>Introduction 39</p><p>Joining layer data 40</p><p>Cleaning up the attribute table 42</p><p>Configuring relations 45</p><p>Joining tables in databases 47</p><p>Creating views in SpatiaLite 48</p><p>Creating views in PostGIS 51</p><p>Creating spatial indexes 54</p><p>Georeferencing rasters</p><p>that</p><p>the names of the joined columns are too long for the 10 character-name length limit of the</p><p>shapefile format. QGIS ensures that all columns in the exported shapefiles have unique</p><p>names even after the names have been shortened to only 10 characters. To do this, QGIS</p><p>adds incrementing numbers to the end of, otherwise, duplicate column names. If you save</p><p>the join from this example as a shapefile, you will see that the column names are altered to</p><p>census_w_1, census_w_2, and so on. Of course, these names are less than optimal to</p><p>continue working with the data. As described in How it works... in this recipe, the names for</p><p>the joined columns are a combination of joined layer name and column name. Therefore, we</p><p>can use the following trick if we want to create a shapefile from the join: we can shorten the</p><p>layer name. Just rename the layer in the layer list. You can even have a completely empty</p><p>layer name! If you change the joined layer name to an empty string, the joined column</p><p>names will be _STATE, _COUNTY, and so on instead of census_wake2000_pop_STATE</p><p>and census_wake2000_pop_COUNTY. In any case, it is good practice to document your</p><p>data and provide a description of the attribute table columns in the metadata.</p><p>In any case, it is very likely that you will want to clean up the attribute table of the new</p><p>dataset, and this is exactly what we are going to do in the next exercise.</p><p>Cleaning up the attribute table</p><p>There are many reasons why we need to clean up attribute tables every now and then.</p><p>These may be because we receive badly structured or named data from external sources, or</p><p>because data processing, such as the layer joins that we performed in the previous exercise,</p><p>require some post processing. This recipe shows us how to use attribute table and the Table</p><p>Manager plugin to rename, delete, and reorder columns, as well as how to convert between</p><p>different data types using Field Calculator.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>43</p><p>Getting ready</p><p>If you performed the previous recipe, just save the joined layer to a new shapefile; otherwise,</p><p>load census_wake2000_pop.shp. In any case, you will notice that the dataset contains a</p><p>lot of duplicate information, and the column names could use some love as well. To follow</p><p>this recipe, you should also install and enable the Table Manager plugin by navigating to</p><p>Plugins | Manage and Install Plugins.</p><p>How to do it…</p><p>1. Our first step to clean up this dataset is to delete duplicated information. From all</p><p>available columns, we only want to keep _STATE, _COUNTY, _TRACT, FIPSSTCO,</p><p>TRT2000, STFID, _POP2000, AREA, and PERIMETER.</p><p>2. To delete the other columns, enable editing using the Toggle editing mode button in</p><p>the upper-left corner of the attribute table or by pressing Ctrl + E. This activates the</p><p>Delete column button.</p><p>3. Alternatively, you can also press Ctrl + L to open the Delete attributes dialog. This</p><p>dialog allows us to delete multiple columns at once. Just select all the columns that</p><p>you want to be deleted, press OK, and QGIS will display the reduced attribute table.</p><p>It's worth noting that the changes will only be permanent</p><p>once you use the Save edits button or disable the editing</p><p>mode and confirm that you want to save the changes.</p><p>4. Next, we will rename columns to remove the leading underscores in some of the</p><p>column names. This can be done using the Table Manager plugin.</p><p>5. When you start the plugin (edit mode should be disabled), you will see a list of the</p><p>layer columns. The plugin allows you to change the order of columns, as well as</p><p>rename, insert, clone, and delete columns.</p><p>6. To rename a column, just select it in the list and press the Rename button. You'll then</p><p>be asked to provide a new name. Go ahead and remove the leading underscores</p><p>from _STATE, _COUNTY, _TRACT, and _POP2000.</p><p>7. Finally, using the Move up and Move down buttons, you can also rearrange the</p><p>column order to something more intuitive. We'd suggest moving STFID to the first</p><p>position and AREA and PERIMETER to the last.</p><p>8. If you press Save, the changes will be saved back to the layer source file. Alternatively,</p><p>you can also create a new file using Save as....</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>44</p><p>How it works…</p><p>The steps provided in this exercise are mostly limited to layers with shapefile sources. If you</p><p>use other input data formats, such as MIF, GML, or GeoJSON files, you will notice that the</p><p>Toggle editing button is grayed out because these files cannot be edited in QGIS. Whether</p><p>a certain format can be edited in QGIS or not depends on which functionality has been</p><p>implemented in the respective GDAL/OGR driver.</p><p>The GDAL/OGR version that is used by QGIS is either part of the QGIS</p><p>package (as in the case of the Windows installers) or QGIS uses the GDAL</p><p>library existing in your system (on Linux and Mac). To get access to specific</p><p>drivers that are not supported by the provided GDAL/OGR version, it is</p><p>possible to compile custom versions of GDAL/OGR, but the details of</p><p>doing this are out of the scope of this cookbook.</p><p>There's more…</p><p>Another common task while dealing with attribute table management is changing column</p><p>data types. Currently, it is not possible to simply change the data type directly. Instead, we</p><p>have to use Field Calculator (which is directly accessible through the corresponding button</p><p>in the Attributes toolbar or from the attribute table dialog) to perform conversions and create</p><p>a new column for the result.</p><p>In our census_wake2000_pop.shp file, for example, the tract ID, TRACT, is stored in a</p><p>REAL type column with a precision of 15 digits even though it may be preferable to simply</p><p>have it in a STRING column and formatted to two digits after the decimal separator. To</p><p>create such a column using Field Calculator, we can use the following expression:</p><p>format_number("TRACT",2)</p><p>Compared to a simple conversion (which would be simple, use tostring("TRACT"),</p><p>format_number("TRACT",2) offers the advantage that all values will be formatted to</p><p>display two digits after the decimal separator, while a simple conversion would drop these</p><p>digits if they are zeros.</p><p>Of course, it's also common to convert from text to numerical. In this case, you can chose</p><p>between toint() and toreal().</p><p>See also</p><p>f Have a look through the conversion functions in the Field Calculator Function list</p><p>to see the other available functions that can deal with date and time data types.</p><p>Usage of all these functions is explained in Selected function help directly in the</p><p>calculator dialog.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>45</p><p>Configuring relations</p><p>In the Joining layer data recipe, we discussed that joins only append additional columns</p><p>to existing features (1:1 or n:1 relationships). Using joins, it is, therefore, not possible to</p><p>model 1:n relationships, such as "one zip code area containing n schools". These kinds of</p><p>relationships can instead be modeled using relations. This recipe introduces the concept of</p><p>relations and shows how you can put them to use.</p><p>Getting ready</p><p>To follow this exercise, load zip code areas and schools from zipcodes_wake.shp and</p><p>schools_wake.shp.</p><p>How to do it…</p><p>Relations are configured in Project Properties. The dialog is very similar to the join dialog:</p><p>1. Define the two layers (Referencing/Child and Referenced/Parent), as well as the</p><p>fields containing the common values/IDs. As you want to model "one zip code area</p><p>contains n schools," the zip code dataset is the parent layer and the school dataset is</p><p>the child layer. The connection between both datasets is established based on the zip</p><p>code fields (ADDRZIPCOD and ZIPNUM), as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>46</p><p>2. To verify that the relation is set up and working, you can either check the attribute</p><p>table in form view (button in the lower-right corner), as shown in the following</p><p>screenshot, or open an individual</p><p>feature form. You will find that the relation</p><p>information has been appended at the end of the form:</p><p>As the preceding screenshot shows, setting up this relation enables you to get access</p><p>to all schools within a certain zip code in a very convenient way. As the edit button</p><p>suggests, it is even possible to edit the school data from this view. You can simply edit</p><p>the values in the table view. You can add and delete schools from the dataset using</p><p>the + and X buttons. The next two buttons enable you to quickly add new entries to</p><p>the relation or to remove them.</p><p>How it works…</p><p>In this example, removing a school from the dataset works just fine, but adding a school via this</p><p>dialog makes less sense because you cannot create a point geometry through this process.</p><p>If you press the button to add to the relation, you will get a dialog that allows you to choose</p><p>which existing school you want to add. In the background, the school's ADDRZIPCOD value is</p><p>updated to match the zip code we just assigned it to.</p><p>Similarly, if you select a school and press the button to remove the relation, what actually</p><p>happens is that the school's ADDRZIPCOD value is set to NULL.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>47</p><p>Joining tables in databases</p><p>If you use a database (SpatiaLite or PostGIS) to store your data, vector and nonspatial, then</p><p>you also have the option of using the database and SQL to perform tables joins. The primary</p><p>advantages of this method include being able to filter data before loading in the map, perform</p><p>multitable joins (three or more), and have full control over the details of the join via queries.</p><p>Getting ready</p><p>You'll need at least two layers in either a SpatiaLite or PostGIS database. These two layers</p><p>need at least one column in common, and the column in common should contain unique</p><p>values in at least one table. In this case, our example uses the census_wake_2000 polygon</p><p>layer and census_wake_2000_pop.csv.</p><p>How to do it…</p><p>1. Open the DB Manager plugin that comes with QGIS. You can find this in the</p><p>Database menu.</p><p>2. Select your database from the tree on the left-hand side, use cookbook.db in</p><p>SpatiaLite (which was created in Chapter 1, Data Input and Output).</p><p>If you don't see this database listed, use Add SpatiaLite</p><p>Layer (the icon or the menu item), or right-click on SpatiaLite</p><p>in the Browser window to make a new connection and add it</p><p>to an existing database.</p><p>3. Now, open the SQL window (the second icon from the left in top toolbar of the</p><p>plugin window).</p><p>4. Put in the following SQL code to query and JOIN the tables:</p><p>SELECT *</p><p>FROM census_wake2000 Sas a</p><p>JOIN census_wake2000_pop AS b</p><p>ON a.stfid = b.stfid;</p><p>How it works…</p><p>SELECT lists all the columns that you want from the source tables; in this case, * means</p><p>everything. FROM is the first (left) table, as a is an alias, which is used so that there's less</p><p>typing later. JOIN is the second (right) table, and ON indicates which columns to should be</p><p>matched between the two tables. The rest of how this works in relational database theory is</p><p>best explained in other texts.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>48</p><p>There's more…</p><p>In databases, there's more than one type of join. You can perform a join where you retain</p><p>only the matches in both tables, or you can retain all content from the left (first table) and any</p><p>matches from the right. You can also control how a one-to-many relationship is summarized or</p><p>select specific records instead of aggregating.</p><p>If you want to save the results of a query you have two options. You can make a view or a new</p><p>table. A view is a saved copy of your query. Every time you open it, the query will be rerun. This</p><p>is great if your data changes because it will always be up-to-date, and this doesn't use any</p><p>additional disk space. On the other hand, a table is like saving a new file; it becomes a static</p><p>new copy of the results. This is good to repeatedly access the same answer, and it is usually</p><p>faster to use, especially for large tables.</p><p>See also</p><p>f Refer to the Creating views in SpatiaLite and Creating views in PostGIS sections in</p><p>this chapter to learn how to make views of the query results.</p><p>f For more general information on writing SQL queries refer to http://sqlzoo.net/</p><p>f Refer to Chapter 1, Data Input and Output, about using the cookbook.db database</p><p>Creating views in SpatiaLite</p><p>In a database, view is a stored query. Every time you open it, the query is run and fresh results</p><p>are generated. To use views as layers in QGIS takes a couple of steps.</p><p>Getting ready</p><p>For this recipe, you'll need a query that returns results containing a geometry. The example</p><p>that we'll use is the query from the Joining tables in databases recipe (the previous recipe)</p><p>where attributes were joined 1:1 between the census polygons and the population CSV. The</p><p>QSpatiaLite plugin is recommended for this recipe.</p><p>How to do it…</p><p>The GUI method is described as follows:</p><p>1. Using the QspatiaLite plugin (which is in the Database menu, if you've activated it)</p><p>place the following in the query:</p><p>SELECT *</p><p>FROM census_wake2000 as a</p><p>www.it-ebooks.info</p><p>http://sqlzoo.net/</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>49</p><p>JOIN census_wake2000_pop as b</p><p>ON a.stfid = b.stfid;</p><p>2. From the Option dropdown, select the last choice, Create Spatial View & Load in</p><p>QGIS, and set the Geometry field box value to the name of your geometry field from</p><p>your spatial layer. In this example, this is geom.</p><p>You can explore your data table fields in the left-hand side</p><p>to check the name of the fields that you need.</p><p>The SQL method is as described, as follows:</p><p>1. In Database | DB Manager, open SQL Window.</p><p>2. Write a query. In this example, this is the Join query from the previous recipe.</p><p>3. Convert this query to a view by adding CREATE VIEW as SELECT:</p><p>CREATE VIEW census_wake2000_pop_join AS</p><p>SELECT *</p><p>FROM census_wake2000 as a</p><p>JOIN census_wake2000_pop as b</p><p>ON a.stfid = b.stfid;</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>50</p><p>4. Register the view with the SpatiaLite metadata backend with a follow up query. This</p><p>function is case sensitive:</p><p>CREATE VIEW census_wake2000_pop_join AS</p><p>INSERT INTO views_geometry_columns</p><p>(view_name, view_geometry, view_rowid, f_table_name,</p><p>f_geometry_column,read_only)</p><p>VALUES ('census_wake2000_pop_join', 'geom', 'rowid',</p><p>'census_wake2000', 'geom',1);</p><p>This only works when the view geometry is based on the</p><p>geometry of a single table. If you need to generate new</p><p>geometries, you probably need a table.</p><p>5. The pattern is ('name of view','name of view geometry field','A Unique ID','name</p><p>of table the view gets its geometry from','name of geometry field in the original</p><p>table',read-only (1) or writable(0)).</p><p>6. After running the second query, you should be able to load the view in QGIS and see</p><p>the same fields as the join query.</p><p>How it works…</p><p>A view is actually stored in the database and is triggered when you load it. In this way, if you</p><p>change the original data tables, the view will always be up to date. By comparison, creating new</p><p>tables makes copies of the existing data, which is stored in a new place, or creates a snapshot</p><p>or freeze of the values at that time. It also increases the database's size by replicating data.</p><p>Whereas, a view is just the SQL text itself and doesn't store any additional data.</p><p>QGIS reads the metadata tables of SpatiaLite in order to figure out what layers contain</p><p>spatial data, what kind of spatial data they contain, and which column contains the geometry</p><p>definition. Without creating entries in the metadata, the tables appear as normal SQLite</p><p>tables, and you can only load attribute data without spatial representation.</p><p>As it's a view, it's really reading the geometries from the original tables. Therefore, any edits</p><p>to the original table will show up. New in SpatiaLite 4.x series, this makes it easier to create</p><p>writable views. If you use the spatialite-gui standalone application, it registers all the database</p><p>triggers needed</p><p>to make it work, and the changes made will affect the original tables.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>51</p><p>There's more…</p><p>You don't have to use ROWID as unique id, but this is a convenient handle that always exists</p><p>in SQLite, and unlike an ID from the original table, there's no chance of duplication in an</p><p>aggregating query.</p><p>See also</p><p>f Read more about writable-view at https://www.gaia-gis.it/fossil/</p><p>libspatialite/wiki?name=writable-view. This recipe is extremely similar to</p><p>the next one on PostGIS and demonstrates how interchangeable the two can be if you</p><p>are aware of the slight differences.</p><p>Creating views in PostGIS</p><p>In a database, a view is a stored query. Every time that you open it, the query is run and</p><p>fresh results are generated. To use views as layers in QGIS takes a couple of steps.</p><p>Getting ready</p><p>For this recipe, you'll need a query that returns results containing a geometry. The example</p><p>that we'll use here is the query from the Joining tables in databases recipe where attributes</p><p>were joined 1:1 between the census polygons and the population CSV.</p><p>How to do it…</p><p>The SQL method is described as follows:</p><p>1. In Database | DB Manager, open SQL Window.</p><p>2. Write a query; in this example, this is the join query that was written in the previous</p><p>exercise. If you want to see it right away but not necessarily retain it, check the Load</p><p>as new layer checkbox near the bottom:</p><p>SELECT *</p><p>FROM census_wake2000 as a</p><p>JOIN census_wake2000_pop as b</p><p>ON a.stfid = b."STFID";</p><p>www.it-ebooks.info</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=writable-view</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=writable-view</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>52</p><p>3. Now, execute the query by clicking on the Execute (F5) button:</p><p>4. After executing the query, to load it to the map check the Load as new layer box,</p><p>which will expand some additional options. Pick your unique integer (id_0) for</p><p>Column with unique integer values and geom for Geometry column. Name your</p><p>result in the Layer name (prefix) textbox and click on Load now!.</p><p>If you only needed to see this data in this particular QGIS</p><p>project, you can stop here. In order to make the database store</p><p>this query for other projects and users, continue this recipe.</p><p>5. Convert this query to a view by adding CREATE VIEW AS SELECT:</p><p>CREATE VIEW census_wake2000_pop_join AS SELECT *</p><p>FROM census_wake2000 as a</p><p>JOIN census_wake2000_pop as b</p><p>ON a.stfid = b."STFID";</p><p>6. Go back to DB Manager and hit the Refresh button (on the left). You should now see</p><p>your new view listed and be able to add it to the map.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>53</p><p>How it works…</p><p>QGIS reads the metadata tables or views of PostGIS in order to figure out what layers contain</p><p>spatial data, what kind of spatial data they contain, and which column contains the geometry</p><p>definition. Without creating entries in the metadata, the tables appear as normal PostgreSQL</p><p>tables, and you can only load attribute data without spatial representation.</p><p>As this is a view, it's really reading the geometries from the original tables. Therefore, any edits</p><p>to the original table will also show up.</p><p>There's more…</p><p>QGIS is really picky about having a unique ID for PostGIS tables and views. There are a few</p><p>tips to make this always work. Always include a numeric unique ID (as the first column is</p><p>recommended but not required, IDs must be integer columns (usually int4, but int8 should</p><p>work now too). Autoincrementing IDs are good idea. When you don't have such an ID field to</p><p>use from one of the underlying tables, you can add an ID on the fly with the following:</p><p>SELECT row_number() OVER() AS id_qgis, FROM table;</p><p>The downside of this is that you now have to list out all the fields that you want to use in the</p><p>view rather than using *. When creating tables, you'll want to turn this id_qgis field into an</p><p>auto-incrementing field if you plan to add records.</p><p>The other big catch is that if you make a new geometry by manipulating existing geometries,</p><p>QGIS isn't always aware of the results. In the previous example, the geometry is just</p><p>passed from the original table to the view unchanged, so it is properly registered in the</p><p>geometry_columns metadata of PostGIS. However, a new geometry doesn't exist in</p><p>the original table, so the trick is to cast the geometry result, as follows:</p><p>CREATE VIEW census_wake2000_4326 AS</p><p>SELECT id_0,</p><p>stfid,tractid,ST_Transform(geom,4326)::geometry(GeometryZ, 4326)</p><p>As geom</p><p>FROM census_wake2000;</p><p>QGIS doesn't always think that this is a valid spatial layer but adding to the Canvas should work.</p><p>The more specific you can be, the better. If you're not sure what</p><p>geometry type it is or if you have 3D (aka Z), check the entries in</p><p>the geometry_columns view.</p><p>Also, keep your eyes on Postgres's relatively new feature called Materialized Views. This is a</p><p>method of caching view results that don't update automatically, but they also don't require</p><p>whole new tables.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>54</p><p>See also</p><p>f Finer details from the PostGIS manual can be read at http://postgis.</p><p>refractions.net/docs/using_postgis_dbmanagement.html#Manual_</p><p>Register_Spatial_Column. This recipe is extremely similar to the previous one</p><p>and demonstrates how interchangeable these two can be if you are aware of the</p><p>slight differences.</p><p>f Read more about Materialized Views at http://www.postgresql.org/</p><p>docs/9.3/static/rules-materializedviews.html</p><p>Creating spatial indexes</p><p>Spatial indexes are methods to speed up queries of geometries. This includes speeding up</p><p>the display of database layers in QGIS when you zoom in close (it has no effect on viewing</p><p>entire layers).</p><p>This recipe applies to SpatiaLite and PostGIS databases. In the event that you've made a new</p><p>table or you have imported some data and didn't create a spatial index, it's usually a good</p><p>idea to add this.</p><p>You can also create a spatial index for shapefile layers. Take a look at</p><p>Layer Properties | General for the Create Spatial Index button. This will</p><p>create a .qix file that works with QGIS, Mapserver, GDAL/OGR, and other</p><p>open source applications. Refer to https://en.wikipedia.org/</p><p>wiki/Shapefile.</p><p>Getting ready</p><p>You'll need a SpatiaLite and a Postgis database. For ease, import a vector layer from the</p><p>provided sample data and do not select the Create spatial index option when importing. (Not</p><p>sure how to import data? Refer to Chapter 1, Data Input and Output, for how to do this.)</p><p>How to do it…</p><p>Using the DB Manager plugin (in the Database menu), perform the following steps:</p><p>1. Check whether the index does not exist. In DB Manager, open the database and then</p><p>open the table that you want to check. Looking at the properties on the right, you</p><p>should see a message just above Fields that looks like this:</p><p>www.it-ebooks.info</p><p>http://postgis.refractions.net/docs/using_postgis_dbmanagement.html#Manual_Register_Spatial_Column</p><p>http://postgis.refractions.net/docs/using_postgis_dbmanagement.html#Manual_Register_Spatial_Column</p><p>http://postgis.refractions.net/docs/using_postgis_dbmanagement.html#Manual_Register_Spatial_Column</p><p>http://www.postgresql.org/docs/9.3/static/rules-materializedviews.html</p><p>http://www.postgresql.org/docs/9.3/static/rules-materializedviews.html</p><p>https://en.wikipedia.org/wiki/Shapefile</p><p>https://en.wikipedia.org/wiki/Shapefile</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>55</p><p>2. However, what if no index was listed for the geom column? Then, we can make one</p><p>just by clicking the create it link. Or you can do this in a SQL window, as follows:</p><p>� For SpatiaLite, use the following:</p><p>SELECT CreateSpatialIndex('schools_wake', 'geom');</p><p>� For PostGIS, use the following:</p><p>CREATE INDEX sidx_census_wake2000_geom</p><p>ON public.census_wake2000 USING gist(geom);</p><p>3. Verify that the index exists, as follows:</p><p>� For PostGIS (the left-hand side of the following screenshot), on the right-hand</p><p>side, scroll to the</p><p>bottom looking for the Indexes section</p><p>� For SpatiaLite (the right-hand side of the following screenshot), you can see</p><p>the idx_nameoftable_geomcolumn listed as a table:</p><p>How it works…</p><p>When you create a spatial index, the database stores a bounding box rectangle for every</p><p>spatial object in the geometry column. These boxes are also sorted so that boxes near each</p><p>other in coordinate space are also near each other in the index.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>56</p><p>When queries are run involving a location, a comparison is made against the boxes, which is a</p><p>simple math comparison. Rows with boxes that match the area in question are then selected</p><p>to be tested in depth for a precise match, based on their real geometries. This method of</p><p>searching for intersection is faster than testing complex geometries one by one because it</p><p>quickly eliminates items that are clearly not near the area of interest.</p><p>There's more…</p><p>Spatial indexes are really important to speed up the loading time of database spatial layers</p><p>in QGIS. They also play a critical role in the speed of spatial queries (such as intersects).</p><p>Note that PostGIS will automatically use a spatial index if one is present. SpatiaLite requires</p><p>that you write queries that intentionally call a particular spatial index (Refer to Haute Cuisine</p><p>examples from the SpatiaLite Cookbook)</p><p>Also, keep in mind that only one spatial index per table can be used in a single query. This</p><p>really comes into play if you happen to have more than one spatial column or create a spatial</p><p>index in a different projection than the geometry (check out the PostGIS Cookbook by Packt</p><p>Publishing for more information).</p><p>If you plan to insert many records into a table with an existing spatial index,</p><p>you may want to disable or drop the index and recreate it after the import is</p><p>done. Otherwise, the index will be recalculated after each row is inserted. This</p><p>applies to nonspatial indexes too.</p><p>Do you want to check lots of tables at once? You can list all GIST indexes in PostGIS at once:</p><p>SELECT i.relname as indexname, idx.indrelid::regclass as</p><p>tablename,</p><p>am.amname as typename,</p><p>ARRAY(SELECT pg_get_indexdef(idx.indexrelid, k + 1, true)</p><p>FROM generate_subscripts(idx.indkey, 1) as k</p><p>ORDER BY k</p><p>) as indkey_names</p><p>FROM pg_index as idx</p><p>JOIN pg_class as i ON i.oid = idx.indexrelid</p><p>JOIN pg_am as am ON i.relam = am.oid</p><p>JOIN pg_namespace as ns ON ns.oid = i.relnamespace</p><p>AND ns.nspname = ANY(current_schemas(false))</p><p>Where am.amname Like 'gist';</p><p>To do something similar in SpatiaLite, use the following:</p><p>SELECT * FROM geometry_columns WHERE spatial_index_enabled = 1;</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>57</p><p>See also</p><p>f Information on SpatiaLite spatial index implementation can be found at https://</p><p>www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>f More details on using spatial indexes can be found at https://www.gaia-gis.</p><p>it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>f Information about PostGIS implementation is at http://postgis.net/docs/</p><p>manual-2.0/using_postgis_dbmanagement.html#gist_indexes</p><p>f You can also check out Chapter 10, Maintenance, Optimization, and Performance</p><p>Tuning, of PostGIS Cookbook by Packt Publishing,</p><p>Georeferencing rasters</p><p>Sometimes, you have a paper map, an image of a map from the Internet, or even a raster</p><p>file with projection data included. When working with these types of data, the first thing you'll</p><p>need to do is reference them to existing spatial data so that they will work with your other data</p><p>and GIS tools. This recipe will walk you through the process to reference your raster (image)</p><p>data, called georeferencing.</p><p>Getting ready</p><p>You'll need a raster that lacks spatial reference information; that is, unknown projection</p><p>according to QGIS. You'll also need a second layer (reference map) that is known and you can</p><p>use for reference points. The exception to this is, if you have a paper map that has coordinates</p><p>marked on it or a spatial dataset that just didn't come with a reference file but you happen to</p><p>know its CRS/SRS definition. Load your reference map in QGIS.</p><p>This book's data includes a scanned USGS topographic map that's missing its o38121e7.tif</p><p>projection information. This map is from Davis, CA, so the example data has plenty of other</p><p>possible reference layers you could use, for example, the streets would be a good choice.</p><p>Actually, the world file was just renamed to o38121e7.tfw.orig</p><p>so that QGIS wouldn't detect it. You can use this later to compare your</p><p>georeference quality.</p><p>www.it-ebooks.info</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>https://www.gaia-gis.it/fossil/libspatialite/wiki?name=SpatialIndex</p><p>http://postgis.net/docs/manual-2.0/using_postgis_dbmanagement.html#gist_indexes</p><p>http://postgis.net/docs/manual-2.0/using_postgis_dbmanagement.html#gist_indexes</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>58</p><p>How to do it…</p><p>On the Raster menu, open the Georeferencing tool and perform the following steps:</p><p>1. Use the file dialog to open your unknown map in the Georeferencing tool.</p><p>2. Create a Ground Control Point (GCP) of matches between your start coordinates and</p><p>end coordinates.</p><p>Building corners, street intersections, and things where line</p><p>features intersect or significant edge features can be found.</p><p>3. Add a point in your unknown map with GCP Add +. You can now enter the coordinates</p><p>(that is, if it's a paper map with known coordinates marked on it), or you can select a</p><p>match from the main QGIS window reference layer.</p><p>4. Repeat this process to find at least four matches. If you want to get a really good fit</p><p>do between 10-20 matches.</p><p>5. (Optional) Save your GCPs as a text file for future reference and troubleshooting:</p><p>Try to spread out your control points so that you have good</p><p>coverage of the whole map. It's all about averaging the</p><p>differences.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>59</p><p>6. Now, choose Transformation Settings, as follows:</p><p>7. You have a choice here. Generally, you'll want to use Polynomial. If you set 4+ points</p><p>for the first order, 6+ points for the second order and 10+ points for the third order,</p><p>The second order is the currently recommend one. This will be discussed in the</p><p>There's more… section of this recipe.</p><p>8. Set Target SRS to the same projection as the reference layer. (In this case, this is</p><p>EPSG:26910 UTM Zone 10n)</p><p>9. Output Raster should be a different name from the original so that you can easily</p><p>identify it.</p><p>Save your GCP list to the file. If you don't like the results,</p><p>come back and try a different algorithm or change</p><p>the number of GCPs used. If you want a reference for</p><p>comparison, look at the text o38121e6.tif.points</p><p>file in this book's data folder.</p><p>10. When you're happy with your list of GCPs click on Start Georeferencing in File or on</p><p>the green triangular button.</p><p>How it works…</p><p>A mathematical function is created based on the differences between your two sets of points.</p><p>This function is then applied to the whole image, stretching it in an attempt to fit. This is</p><p>basically a translation or projection from one coordinate system to another.</p><p>There's more…</p><p>Picking transformation types can be a little tricky, the list in QGIS is currently in alphabetical</p><p>order and not the recommended order. Polynomial 2 and Thin-plate-spline (TPS) are probably</p><p>the two most common choices. Polynomial 1 is great when you just have minor shift, zooming</p><p>(scale), and rotation. When you have old well-made maps in consistent projections, this will</p><p>apply the least amount of change. Polynomial 2 picks up from here and handles consistent</p><p>distortion. Both of these provide you with an error estimate as the Residual or RMSE (Root</p><p>Mean Square Error). TPS handles variable distortion, varying it's correction around each</p><p>control point. This will</p><p>almost always result in the best fit, at least through the GCPs that</p><p>you provide. However, because it varies at every GCP location, you can't calculate an error</p><p>estimate and it may actually overfit (create new distortion). TPS is best for hand-drawn maps,</p><p>nonflat scans of maps, or other variable distorted sources. Polynomial methods are good for</p><p>sources that had high accuracy and reference marks to begin with.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>60</p><p>If you really want a good match, once you have all your points, check the RMSE values in the</p><p>table at the bottom. Generally, you want this near or less than 1. If you have a point with a</p><p>huge value, consider deleting it or redoing it. You can move existing points, and a line will be</p><p>drawn in the direction of the estimated error. So, go back over the high values, zoom in extra</p><p>close, and use the GCP move option.</p><p>Sometimes, just changing your transformation type will help, as shown in the following</p><p>screenshot that compares Polynomial 1 versus Polynomial 2 for the same set of GCP:</p><p>Polynomial 1</p><p>Note the residual values difference when changing to Polynomial 2 (assuming that you have</p><p>the minimum number of points to use Polynomial 2):</p><p>Polynomial 2</p><p>Resampling methods can also have a big impact on how the output looks.</p><p>Some of the methods are more aggressive about trying to smooth out</p><p>distortions. If you're not sure, stick with the default nearest neighbor. This</p><p>will copy the value of the nearest pixel from the original to a new square</p><p>pixel in the output.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>61</p><p>See also</p><p>f When performing georeferencing in a setting where you need it to be very accurate</p><p>(science and surveying), you should read up on the different transformations and</p><p>what RMSE values are good for your type of data. Refer to the general GIS or Remote</p><p>Sensing textbooks for more information.</p><p>f For full details of all the features of the QGIS georeferencer, refer to the online</p><p>manual at http://docs.qgis.org/2.8/en/docs/user_manual/plugins/</p><p>plugins_georeferencer.html.</p><p>f The QGIS documentation has some basic information about how to pick</p><p>transformation type at http://docs.qgis.org/2.8/en/docs/user_manual/</p><p>plugins/plugins_georeferencer.html#available-transformation-</p><p>algorithms.</p><p>Georeferencing vector layers</p><p>For various reasons, sometimes you have a vector layer that lacks projection information.</p><p>This is often the case with CAD layers that were created only in local coordinates. When it is</p><p>possible, try to track down the original projection information. As a last resort, you can attempt</p><p>to warp the vector layer to match a known reference layer with the recipe described here.</p><p>Getting ready</p><p>You can open two instances of QGIS (or use one as you'll just be zooming back and forth a</p><p>lot). In one instance, load a reference layer, something in the projection that you want your</p><p>data to be in. Activate Coordinate Capture Plugin from the Manage Plugins menu.</p><p>In Windows, you need the osgeo4w shell for this recipe. If you don't have</p><p>a start menu item, look for the OSGeo4W.bat launcher in your QGIS or</p><p>OSGeo4w installation folder.</p><p>This example uses cad-lines-only.shp, which is the line layer extracted from the</p><p>CSS-SITE-CIV.dxf file. This file is a CAD rendering of design plans for Academy St.</p><p>in the town of Cary, Wake County, North Carolina.</p><p>www.it-ebooks.info</p><p>https://www.qgis.org/en/docs/user_manual/plugins/plugins_georeferencer.html</p><p>https://www.qgis.org/en/docs/user_manual/plugins/plugins_georeferencer.html</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/plugins/plugins_georeferencer.html#available-transformation-algorithms</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/plugins/plugins_georeferencer.html#available-transformation-algorithms</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/plugins/plugins_georeferencer.html#available-transformation-algorithms</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>62</p><p>How to do it…</p><p>1. Create a list of GCP matches between your unknown layer (cad-lines-only.shp)</p><p>and your reference layer (CarystreetsND83NC.shp).</p><p>2. Here are some specific adjustments to help with cad-lines-only.shp referenced</p><p>to CarystreetsND83NC.shp. These will make it easier to find matches between</p><p>the two layers:</p><p>1. Load cad-lines-only.shp, and adjust its style properties using a</p><p>rule-based style. Use the "Layer" = 'C-ROAD-CNTR' rule, which will only</p><p>show you street centerlines.</p><p>2. In your other QGIS session, load CarystreetsND83NC.shp in order to</p><p>find the matching area, open the attribute table, and apply the following</p><p>select expression: "Street" LIKE '%N ACADEMY%' OR "Street"</p><p>LIKE '%S ACADEMY%' OR "Street" LIKE '%CHATHAM%'. The filter</p><p>here highlights the three main streets of the original project, which is at the</p><p>intersection of Chatham and N/S Academy streets in the center of the town.</p><p>This may also be useful to change the color of the selected features to</p><p>make it easier to find. The traffic circles at either end of the project are</p><p>good landmarks:</p><p>3. Find an easy-to-identify feature that matches in both layers</p><p>(street intersections).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>63</p><p>4. Use the coordinate capture plugin to copy the x,y value for the point in</p><p>both layers.</p><p>5. Save the coordinates in a text editor while you work.</p><p>6. Repeat this procedure until you have at least four pairs of points. Try to pick</p><p>points spread out across the whole layer:</p><p>There is currently no graphical interface in QGIS for the next</p><p>step, which uses the OGR library that comes with QGIS. Take</p><p>the list of points and using the ogr2ogr command-line, you're</p><p>going to apply the GCP to the unknown layer.</p><p>3. Each set of coordinate pairs will look as follows:</p><p>-gcp sourceX sourceY destinationX destinationY</p><p>4. Open a terminal (Mac or Linux) or an OSGeo4w shell (Windows).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>64</p><p>5. Change to the directory where you have the data (Hint: cd /home/user/</p><p>Qgis2Cookbook/):</p><p>ogr2ogr -a_srs EPSG:3358 -gcp 2064886.09740 741552.90836</p><p>629378.595 226024.853 -gcp 2066610.97021 741674.39817</p><p>629903.420 226064.049 -gcp 2064904.46214 743055.63847</p><p>629384.784 226485.725 -gcp 2062863.85707 741337.65243</p><p>628762.587 225960.900 cad_lines_nd83nc.shp cad-lines-only.shp</p><p>-a_srs is the proj code for your reference layer.</p><p>The command pattern is ogr2ogr .</p><p>Other useful advanced options include -order to</p><p>indicate polynomial level (default is based on the number</p><p>of GCPs) or -tps to use Thin-plate-spline instead of</p><p>polynomial. For more options refer to http://www.</p><p>gdal.org/ogr2ogr.html.</p><p>6. Now, load your new cad_lines_nd83nc.shp file in the same project, as</p><p>CarystreetsND83NC.shp. They should line up without the need to enable</p><p>projection-on-the-fly:</p><p>www.it-ebooks.info</p><p>http://www.gdal.org/ogr2ogr.html</p><p>http://www.gdal.org/ogr2ogr.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>65</p><p>How it works…</p><p>Given the list of input coordinates and matching output coordinates, a math formula is derived</p><p>to translate between the two sets. This formula is then applied to all the points in the original</p><p>data. The result of this is a reprojected dataset from an unknown projection to a known</p><p>projection.</p><p>The original data is actually EPSG:102719, but we're pretending that we didn't</p><p>have this piece of information to demonstrate this example.</p><p>There's more…</p><p>When picking a reference layer, try to pick something in the projection that you want to use</p><p>for your maps and analysis. That way you only have to reproject once, as each additional</p><p>transformation can add an error. There's also more than one way to go about accomplishing</p><p>this task, including moving the data by hand.</p><p>In this particular, example the transformation is autoselected based on the number of GCP</p><p>point pairs. 4-5 is the first order polynomial, 6-9 is the second order polynomial, and 10+ is</p><p>the third order polynomial. Refer to the previous recipe</p><p>in this chapter for more information.</p><p>A related topic is Affine transformations when you simply want to shift or rotate a vector layer</p><p>by a known amount. The QgsAffine plugin is great if you already know the parameters, or</p><p>roughly know how far you want to rotate and shift the vector layer, as it then just needs</p><p>some math to get the parameters.</p><p>Maybe by the time you read this, all of the difficult things here will be worked</p><p>in a plugin. Keep an eye open, and try the experimental plugins Vector</p><p>Bender, vectorgeoref, and Affine Transformations.</p><p>See also</p><p>f This method is very similar to the Georeferencing Rasters recipe and many of the</p><p>same tips apply to both</p><p>f If you want to see how we got the CAD file into an SHP to begin with, look at Importing</p><p>DXF/DWG files in the Chapter 1, Data Input and Output</p><p>f See the Using Rule Based Rendering recipe in Chapter 10, Cartography Tips, for tips</p><p>on how to visualize the resulting CAD import better by applying attribute based rule</p><p>filtering</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>66</p><p>f Too lazy to do the math? You can also just use GvSig to do the math and make a</p><p>world file; refer to http://foss4gis.blogspot.com/2011/05/computing-</p><p>and-applying-affine.html</p><p>f If you want to do the math yourself see http://press.underdiverwaterman.</p><p>com/rotating-a-point-grid-in-qgis/</p><p>Creating raster overviews (pyramids)</p><p>Overviews, or pyramids, and resampling are all about making raster layers load faster when</p><p>zooming and panning in your map canvas, by reducing the amount of data loaded when not</p><p>zoomed in all the way.</p><p>Getting ready</p><p>You will need a large raster image.</p><p>Generally, you want to make a copy of the data as this method will likely</p><p>alter the original file if you choose to make 'internal' pyramids (easy to do</p><p>on accident).</p><p>How to do it…</p><p>1. Load your raster in QGIS. elev_lid792_1m.tif will work fine for this example.</p><p>2. Right-click on the layer name and open Properties.</p><p>3. Go to the Pyramids item on the left:</p><p>www.it-ebooks.info</p><p>ftp://199.72.17.76/Engineering/Academy%20Street%20Improvements%202013/DWGs/CSS-WATER%20&%20SEWER/CSS-SITE-CIV.dwg</p><p>ftp://199.72.17.76/Engineering/Academy%20Street%20Improvements%202013/DWGs/CSS-WATER%20&%20SEWER/CSS-SITE-CIV.dwg</p><p>ftp://199.72.17.76/Engineering/Academy%20Street%20Improvements%202013/DWGs/CSS-WATER%20&%20SEWER/CSS-SITE-CIV.dwg</p><p>http://foss4gis.blogspot.com/2011/05/computing-and-applying-affine.html</p><p>http://foss4gis.blogspot.com/2011/05/computing-and-applying-affine.html</p><p>http://press.underdiverwaterman.com/rotating-a-point-grid-in-qgis/</p><p>http://press.underdiverwaterman.com/rotating-a-point-grid-in-qgis/</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>67</p><p>4. Select the image sizes that you want to create pyramids for:</p><p>� Optionally, choose whether to store externally (safer) or internally (less files</p><p>to keep track of).</p><p>� Optionally, choose a resampling algorithm; Nearest Neighbor is the simplest,</p><p>but other methods may look smoother at the cost of more data manipulation</p><p>and compute time</p><p>5. Click on Build pyramids.</p><p>6. When this is completed, you'll notice the red X on the sizes that you picked will now</p><p>show a pyramid.</p><p>How it works…</p><p>Generating pyramids essentially makes copies of your original data resized for different</p><p>zoom levels. As you zoom out, the original data is resampled to fit the size of the screen. The</p><p>pyramids do the same thing, but they let you decide what resampling method to use and</p><p>generate this overview ahead of time. By generating them ahead of time, QGIS can load the</p><p>image faster when you change zoom levels.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>68</p><p>There's more…</p><p>Resampling is a fancy way of saying that at each zoom level that is now 1 pixel is more than 1</p><p>pixel from the original data, so they need to be averaged in some way and the result assigned</p><p>to the 1 pixel that is now available. Each of the different methods uses a different math</p><p>formula to decide the new value and how much to smooth that value with neighboring pixels</p><p>(so that it looks aesthetically pleasing). This is the same concept as when you shrink pictures</p><p>so that you can e-mail them to your friends.</p><p>If you chose to save them externally, your overviews are stored in elev_lid792_1m.tif.</p><p>ovr. Some other programs store the same thing in the .aux files; however, pyramid formats</p><p>are not universally compatible between GIS applications.</p><p>See also</p><p>f This is the same effect as using the GDAL gdaladdo command; refer to</p><p>http://gdal.org/gdaladdo.html</p><p>f More details from the QGIS documentation can be found at https://docs.</p><p>qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_</p><p>properties.html</p><p>Building virtual rasters (catalogs)</p><p>When you have a lot of rasters (instead of one big raster) that are all part of the same dataset</p><p>(typically adjacent to each other), you don't want to load each file individually and then style</p><p>it. It's much easier to load one file and treat it as one layer. This recipe lets you do this without</p><p>actually creating a single monstrous raster, which can be difficult to work with.</p><p>Getting ready</p><p>You will need two or more raster files that have adjacent extents or only overlap partially</p><p>around the edges and are in the same projection. Ideally, the files should be of the same type,</p><p>such as all elevations, all air photos, and so on. For this recipe, the elevation rasters from the</p><p>OSGeo EDU (North Carolina) dataset will work.</p><p>How to do it…</p><p>1. (Optional) Load the elevation rasters to your current map.</p><p>2. Go to Raster Menu | Miscellanous | Build Virtual Raster (Catalog).</p><p>3. Check the Use visible raster layers checkbox or choose SELECT, browse to the</p><p>example data, and select all four.</p><p>www.it-ebooks.info</p><p>http://gdal.org/gdaladdo.html</p><p>https://docs.qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_properties.html</p><p>https://docs.qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_properties.html</p><p>https://docs.qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_properties.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>69</p><p>4. SELECT and name an output file using the .vrt extension.</p><p>5. (Optional) Check the Load into canvas when finished checkbox if you want to see the</p><p>results immediately:</p><p>GDAL command line equivalent:</p><p>For example, gdalbuildvrt elevlid.vrt elevlid_</p><p>D782_6.tif elevlid_D783_6m.tif elevlid_</p><p>D792_6m.tif elevlid_D793_6m.tif.</p><p>How it works…</p><p>GDAL VRT format is an XML file that defines the location of each raster file relative to an</p><p>anchor file. It uses the existing spatial extent information of the rasters to figure out their</p><p>positions relative to each other and then anchors the set in the given coordinate system.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>70</p><p>There's more…</p><p>Using a VRT is all about saving time. When you have hundreds of raster files for one particular</p><p>dataset, you can combine them into a single file. However, this file could be gigantic in size and</p><p>somewhat impossible to work with. This is a quick way to be able use the files as a seamless</p><p>background layer. If you need to perform analysis, you'll likely need to either combine the layers</p><p>or loop over them individually.</p><p>You could also generate Tile Index (also in the Miscellaneous menu), which makes a shapefile</p><p>of the outlines of the rasters and puts the ID and path of the raster in the attribute table. This</p><p>would allow you to figure out which image you want to load for a given map without having to</p><p>load them all.</p><p>Finally, if you really want to make all of the files a single large file, use the context menu</p><p>(right-click on the loaded VRT layer and choose Save As). If you have overlaps, more</p><p>complicated situations, or want to merge without loading the files, first use the Merge</p><p>tool (also in the Miscellaneous menu). This can be tricky if your files overlap, you'll need</p><p>to decide how to handle the double data.</p><p>See also</p><p>f For another example,</p><p>please refer to http://manual.linfiniti.com/en/</p><p>rasters/data_manipulation.html#basic-fa-create-a-virtual-raster</p><p>f GDAL's gdalbuildvrt is the underlying tool; it's documentation can be found at</p><p>http://gdal.org/gdalbuildvrt.html</p><p>www.it-ebooks.info</p><p>http://manual.linfiniti.com/en/rasters/data_manipulation.html#basic-fa-create-a-virtual-raster</p><p>http://manual.linfiniti.com/en/rasters/data_manipulation.html#basic-fa-create-a-virtual-raster</p><p>http://gdal.org/gdalbuildvrt.html</p><p>http://www.it-ebooks.info/</p><p>71</p><p>3</p><p>Common Data</p><p>Preprocessing Steps</p><p>In this chapter, we will cover the following recipes:</p><p>f Converting points to lines to polygons and back – QGIS</p><p>f Converting points to lines to polygons and back – SpatiaLite</p><p>f Converting points to lines to polygons and back – PostGIS</p><p>f Cropping rasters</p><p>f Clipping vectors</p><p>f Extracting vectors</p><p>f Converting rasters to vectors</p><p>f Converting vectors to rasters</p><p>f Building DateTime strings</p><p>f Geotagging photos</p><p>Introduction</p><p>When working with other people's data, it is often not the exact format that you need for a</p><p>particular use. This chapter is all about taking the data that you do have and converting it to</p><p>what you actually need. It covers converting between different types of vectors (points, lines,</p><p>and polygons), between vectors and polygons, and cutting out only the parts that you need.</p><p>Taking data from how you get it and converting it to the format and layout that you need in</p><p>order to work with is often called 'data preprocessing'.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>72</p><p>Converting points to lines to polygons and</p><p>back – QGIS</p><p>Sometimes your data is vector formatted (point, line, or polygon), but it is not the right kind</p><p>of vector for a particular type of analysis. Or perhaps you need to split a vector in a particular</p><p>way to facilitate some analysis or cartography. Thankfully, all vector formats are related, lines</p><p>are two or more connected points, polygons are lines whose first and last point are the same,</p><p>multipolygons are two or more polygons for the same record, and rings are nested polygons</p><p>where the inner polygon outlines an area to be excluded. This recipe covers how to convert</p><p>between the different vector types using built-in QGIS methods.</p><p>Getting ready</p><p>To convert points to lines or polygons, you will need a shapefile with an ID column that has a</p><p>single value shared between the points of the same line or polygon. In the following example,</p><p>we will use census_wake_2000_points.shp.</p><p>You will also need to install and activate the Points2One plugin. Refer to the following</p><p>website for how install plugins, http://docs.qgis.org/2.8/en/docs/user_manual/</p><p>plugins/plugins.html.</p><p>How to do it…</p><p>The following instructions show four different conversion methods, depending on the starting</p><p>data and the end data type. All of the tools are in the Vector menu:</p><p>Start by loading the census_wake_2000_points.shp layer.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/plugins/plugins.html</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/plugins/plugins.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>73</p><p>Converting points to lines (or polygons)</p><p>1. Go to Vector | Points2One.</p><p>2. Choose to create either lines or polygons.</p><p>3. Pick the group ID; in this case, this is STFID.</p><p>4. Create the output filename: census_wake_2000_pt2lines.shp:</p><p>Converting lines to polygons</p><p>1. Go to Vector | Geometry Tools | Lines to Polygons.</p><p>2. Create the output filename: census_wake_2000_lines2poly.shp.</p><p>Converting polygons to lines</p><p>1. Go to Vector | Geometry Tools | Polygons to Lines.</p><p>2. Create the output filename: census_wake_2000_poly2lines.shp.</p><p>Converting polygons or lines to points</p><p>1. Go to Vector | Geometry Tools | Extract Nodes.</p><p>2. Create the output filename: census_wake_2000_poly2pts.shp.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>74</p><p>How it works…</p><p>Converting to simpler types from more complex ones is fairly straightforward in simple cases.</p><p>Lines are just multiple points connected together and polygons are lines that start and end</p><p>with the same point. So, it's pretty easy to see how to deconstruct one geometry to simpler</p><p>geometries.</p><p>It's building up from points, which is a little trickier. In a line with three or more points, you</p><p>need to make sure that you have them in the correct order; otherwise, you'll end up with a</p><p>squiggle. When going to polygons, this can create bigger issues by leaving you with invalid</p><p>polygons that self-intersect. So, it's really important to order your points in your source table in</p><p>the same order that they will be combined. Reordering your data can be somewhat tricky. The</p><p>Points2One plugin now includes a sort order option; to use this, make sure that your attribute</p><p>table has a numeric column with the order of the points specified per group (you can restart</p><p>the numbering at 1 for each distinct grouping).</p><p>There's more…</p><p>You can also split or combine multipolygons with the Singleparts to Multiparts and</p><p>Multiparts to Singleparts commands.</p><p>When things get really tricky, you may need to switch to editing the shapes by hand or</p><p>custom scripts. A good example of this is when you want a polygon with a hole in the middle.</p><p>If you do go the route of editing by hand, make sure to turn on snapping so that your lines</p><p>are automatically snapped to existing points. The official documentation on snapping</p><p>can be found at http://docs.qgis.org/2.8/en/docs/user_manual/working_</p><p>with_vector/editing_geometry_attributes.html#setting-the-snapping-</p><p>tolerance-and-search-radius.</p><p>The Editing and Advanced Editing toolbars and additional editing related plugins offer the</p><p>ability to manipulate particularly tricky geometries, one at a time, if you need to.</p><p>Converting points to lines to polygons and</p><p>back – SpatiaLite</p><p>The goal of this recipe is identical to the previous recipe, but it covers how to perform the</p><p>process with data in a SpatiaLite database. You will to turn points into lines and lines into</p><p>polygons.</p><p>Not all methods are available; for those that are not available, you can use the previous</p><p>recipe. It will also work on a database layer; it just doesn't save the results to the database.</p><p>So, the results will need to be imported to the database after completion.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#setting-the-snapping-tolerance-and-search-radius</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#setting-the-snapping-tolerance-and-search-radius</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#setting-the-snapping-tolerance-and-search-radius</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>75</p><p>Getting ready</p><p>You need to load a vector layer of points with a numeric ID indicating order, and an identifier</p><p>of unique lines or polygons that is shared between points of the same geometry. For example,</p><p>you can use census_wake_2000_points loaded into SpatiaLite with the geometry field</p><p>called geom.</p><p>How to do it…</p><p>Using DB Manager Plugin (comes with QGIS and is in the Database menu), the QspatiaLite</p><p>plugin, or an alternate SpatiaLite SQL application (command line or GUI), the following SQL</p><p>examples will perform the conversions between vector types.</p><p>Points to lines</p><p>1. Create a table with points grouped by common ID:</p><p>--Create table grouping points with shared stfid into lines</p><p>CREATE Table census_pts2lines AS</p><p>SELECT stfid,MakeLine(geom) as geom</p><p>FROM census_wake_2000_points</p><p>GROUP BY stfid;</p><p>The following screenshot shows what the screen will look like:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>76</p><p>2. Register the new table as spatial:</p><p>--Register the new table's geometry so QGIS knows its a</p><p>spatial layer</p><p>SELECT</p><p>RecoverGeometryColumn('census_pts2lines','geom',3358,'LINES</p><p>TRING',2);</p><p>Some SQL interfaces can run multiple SQL statements in a row,</p><p>separated by a semicolon. However, there are also many interfaces</p><p>that can only perform one query at a time. Generally, run one query</p><p>at a time unless you know your software supports multiple queries;</p><p>otherwise, this may fail or silently only run the first query.</p><p>Lines to polygons</p><p>1. Create a table with lines grouped by common ID:</p><p>--Create table grouping lines with shared stfid into polygons</p><p>CREATE Table census_line2poly AS</p><p>SELECT stfid,ST_Polygonize(geom) as geom</p><p>FROM census_pts2lines</p><p>GROUP BY stfid;</p><p>2. Register the new table as spatial:</p><p>--Register the new table's geometry so QGIS knows its a</p><p>spatial layer</p><p>SELECT</p><p>RecoverGeometryColumn('census_line2poly','geom',3358,'POLYGON',2);</p><p>Double dashes (--) is the SQL character for a comment line.</p><p>It is used to include descriptive text that is ignored in a query.</p><p>How it works…</p><p>Based on the common identifier specified in GROUP BY, the SQL statement aggregates</p><p>multiple points into a new geometry of the type specified. After creating the new geometry and</p><p>saving the results to a table, registration of the spatial metadata allows Spatialite and QGIS to</p><p>know the table is a spatial layer.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>77</p><p>There's more…</p><p>In the second example, lines were converted to polygons. You could also go directly from</p><p>points to polygons with ST_Polygonize(ST_MakeLine(geom)).</p><p>Under the current versions of SpatiaLite, only aggregation to higher levels is fully supported.</p><p>If you wish to disaggregate geometries, you can use the QGIS vector tools from the previous</p><p>recipe.</p><p>See also</p><p>f SpatiaLite does have functions to dump specific points by first, last, or ID, one at a</p><p>time. Refer to the index of functions (Reference Guide) online for details at https://</p><p>www.gaia-gis.it/fossil/libspatialite/index</p><p>f The Converting points to lines to polygons and back – QGIS recipe in this chapter for</p><p>nondatabase methods</p><p>Converting points to lines to polygons and</p><p>back – PostGIS</p><p>The goal of this recipe is identical to the previous two recipes, but it covers how to perform</p><p>the process with data in a PostGIS database. You will use it to turn points into lines, and lines</p><p>into polygons.</p><p>Not all methods are available; for those not available, you can use the previous recipe. It will</p><p>also work on a database layer; it just doesn't save the results to the database. So, the results</p><p>will need to be imported to the database after completion.</p><p>Getting ready</p><p>You need to load a vector layer of points with a numeric ID indicating order, and an identifier</p><p>of unique lines or polygons that is shared between points of the same geometry. For example,</p><p>you can use census_wake_2000_points loaded into PostGIS with the geometry field called</p><p>geom. (Refer to Chapter 1, Data Input and Output, the Loading Vector Data into PostGIS recipe</p><p>to see how to load data into PostGIS.)</p><p>Import as single not multigeometries. Otherwise, you'll need to carry out</p><p>some extra steps in the queries to split the multigeometries before they</p><p>can be converted.</p><p>www.it-ebooks.info</p><p>https://www.gaia-gis.it/fossil/libspatialite/index</p><p>https://www.gaia-gis.it/fossil/libspatialite/index</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>78</p><p>How to do it…</p><p>Using DB Manager Plugin (this comes with QGIS and is in the Database menu) or</p><p>an alternate PostGIS SQL application (command line—pgsql or GUI—pgadmin III), the</p><p>following SQL examples will perform the conversions between vector types.</p><p>Converting points to lines</p><p>1. Run the following query:</p><p>CREATE VIEW pts2line AS</p><p>SELECT ROW_NUMBER() over (order by census_wake_2000_points</p><p>.stfid) as id, stfid, ST_MakeLine(geom) as geom</p><p>FROM census_wake_2000_points</p><p>GROUP BY stfid;</p><p>The following screenshot shows what the screen will look like:</p><p>To test the creation of new geometries, wrap the queries in CREATE VIEW,</p><p>as demonstrated in Chapter 2, Data Management. If the data is large or you</p><p>are happy with the results, you can swap in CREATE TABLE to make a new</p><p>table for more permanent storage.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>79</p><p>Converting lines to polygons</p><p>Run the following query:</p><p>CREATE VIEW line2poly AS</p><p>SELECT id,stfid,ST_MakePolygon(geom) as geom</p><p>FROM pts2line;</p><p>Want to go straight from point to polygons? Try ST_</p><p>MakePolygon(ST_MakeLine(geom)); the rest is</p><p>as shown in the first example: points to lines query.</p><p>Converting lines or polygons to points</p><p>Run the following query:</p><p>CREATE VIEW pts AS</p><p>SELECT ROW_NUMBER() over (order by a.id_0) as id,id_0 as</p><p>grpid,(a.a_geom).path[2] as path,</p><p>ST_GeometryType((a.a_geom).geom), ((a.a_geom).geom) as geom</p><p>FROM (SELECT id_0,(ST_DumpPoints(geom)) as a_geom FROM</p><p>"census_wake2000") as a;</p><p>What's ROW_NUMBER() about? This is a trick to ensure a</p><p>unique integer for each row. Some tools complain if you don't</p><p>have this; for example, DB Manager won't preview or load the</p><p>layer, even though direct loading in QGIS works fine.</p><p>How it works…</p><p>Based on the common identifier specified in GROUP BY, the SQL statement aggregates</p><p>multiple points into a new geometry of the specified type.</p><p>When dumping geometries to points, PostGIS actually dumps an array, including ID</p><p>information. This is why the example query is actually a nested set of queries. The first is to</p><p>dump the array of geometry information, and the second to extract the relevant parts of the</p><p>results in the format that we want them in.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>80</p><p>There's more…</p><p>PostGIS has a few dump functions with different purposes in mind. Splitting geometries is</p><p>apparently a difficult concept for databases because aggregation is usually the only direction</p><p>functions can logically go. Disaggregation is claimed by some to be counter to how SQL</p><p>conceptually works and would require non-SQL logic.</p><p>See also</p><p>f For more details on the dump functions of PostGIS (ST_Dump, ST_DumpPoints, and</p><p>ST_DumpRings), refer to the PostGIS manual at http://postgis.net/docs/</p><p>manual-2.1/reference.html</p><p>f Refer to the Converting points to lines to polygons and back – QGIS recipe in this</p><p>chapter for the non-database methods</p><p>Cropping rasters</p><p>Sometimes, the raster data you have for a theme is just much larger than the actual extent of</p><p>your study area or map. Or, in the case of scanned maps, you have extra nonmap information</p><p>around the outside edge. In these cases, you want to cut out a portion of your raster.</p><p>Getting ready</p><p>You'll need a raster file that you want to cut a portion of. In this example, we will use the North</p><p>Carolina whole state elevation model (elev_state_500m.tif) and cut it with the outline of</p><p>Wake County (county_wake.shp). Load both of these files in a fresh QGIS project.</p><p>How to do it…</p><p>The easiest way to do this is to use a polygon mask layer. The vector mask can be a rectangle,</p><p>but it doesn't have to be. The outline of a single polygon works best, though.</p><p>An alternate method would be to determine the bounding box (bbox)</p><p>coordinates of the extent that you want with the Capture Coordinate</p><p>tool or to draw the rectangle directly on the map.</p><p>1. Go to Raster | Extraction | Clipper.</p><p>2. Set Input file (raster) as elev_state_500m.tif.</p><p>3. Set Output file using the Select button to pick a directory, and name the output</p><p>elev_wake_500m.tif.</p><p>www.it-ebooks.info</p><p>http://postgis.net/docs/manual-2.1/reference.html</p><p>http://postgis.net/docs/manual-2.1/reference.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>81</p><p>4. Set No data value to -9999.</p><p>Why -9999? Setting No data value to something impossible</p><p>makes it more obvious later to other users. The value 0 is a</p><p>really bad choice as data can legitimately have a value of zero.</p><p>As some raster formats only support numbers and, in particular,</p><p>integers, a large negative number is a common choice.</p><p>5. Now change Clipping mode to Mask Layer and select county_wake.shp as</p><p>Mask Layer:</p><p>How it works…</p><p>The shape of your mask and the size of the raster cells in the source data will determine how</p><p>pixelated</p><p>the resulting raster will be. Zoom in to the results and compare the edge of the new</p><p>raster to the vector outline of the county. You'll notice that because of the 500 m wide pixels,</p><p>it's hard to exactly match the edge of the county exactly with whole pixels.</p><p>Note that this tool, as with all other tools in the GDAL Tools menu, is actually a graphical</p><p>interface to GDAL command-line tools.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>82</p><p>There's more…</p><p>You'll notice that with this particular example, an issue that arises with converting rasters to</p><p>nonrectangular shapes; the edges are jagged as compared to the vector. If you need it to be</p><p>really smooth, there are a few options. You can decrease the pixel size, splitting current pixels</p><p>into multiple pixels using the -tr option. As with other Raster tools, you can use the pencil</p><p>icon to override the GDAL command-line options to add features not included in the interface.</p><p>In this case, the -tr option inline with the rest of the already formatted command:</p><p>gdalwarp -tr 100 100</p><p>This would make each pixel 100 units instead of the current 500 x 500.</p><p>Some important options to remember when saving TIFF files with GDAL are</p><p>number type (Integer versus Float) and compression. Both of these can</p><p>greatly impact the final file size. Refer to the Converting Vectors to Rasters</p><p>recipe later in this chapter for an example. Also, if you have a multicore CPU,</p><p>add -multi to take advantage of your CPU cores for faster processing of</p><p>most raster operations.</p><p>See also</p><p>f For a full list of the gdalwarp options refer to http://gdal.org/gdalwarp.html</p><p>Clipping vectors</p><p>Like rasters, occasionally you only need vector data to cover a certain area of study (area of</p><p>interest). Also, like rasters, you can use a layer defining the extent that you want to select only</p><p>for a portion of a vector layer to make a new layer. The tool that is used for this job is Clip; that</p><p>is, 'Cookie Cutter' because of how the results look afterwards.</p><p>Getting ready</p><p>For the example in this recipe, we will use geology.shp and clip it to the extent of Wake</p><p>County using census_wake2000.shp. Any vector layer with the aggregation of polygons</p><p>covering all of the county will work.</p><p>www.it-ebooks.info</p><p>http://gdal.org/gdalwarp.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>83</p><p>How to do it…</p><p>1. Load the two, geology.shp and census_wake2000.shp, layers.</p><p>2. Open the clipping tool from Vector | Geoprocessing Tools | Clip:</p><p>3. Input layer is the layer that has to be cut; this is geology.shp.</p><p>4. Clip layer defines the boundaries that have to be cut.</p><p>There is no requirement that clip layer be contiguous. You</p><p>can cut any combination of shapes that you want, circles,</p><p>squares, triangles, and so on. They just need to be polygons.</p><p>5. (Optional) Check Add result to canvas so that you see the results immediately.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>84</p><p>6. Select OK to run the tool:</p><p>How it works…</p><p>All clipping is based on the principle of intersection features. For each feature in Input, the</p><p>tool checks to see whether it intersects with the overall shape of clip layer. When it does</p><p>intersect, the algorithm then checks whether any part falls outside the intersection. When</p><p>a part lands outside, it is cut off.</p><p>There's more…</p><p>You have to be careful when using clip. If the original table contained columns that included</p><p>measurements such as area and perimeter, these values are copied from the original.</p><p>Therefore, they may not reflect the new size of and shapes that were cut.</p><p>Generally, all geometry operations and analysis should be done with layers in the same</p><p>projection in order to ensure consistent results. Also, many tools are not projection aware and</p><p>won't compensate for two source layers being in different projections.</p><p>Tools that create the intersection of objects (for example, in PostGIS's and SpatiaLite's</p><p>ST_Intersection) can provide you with similar results. However, you may need to perform</p><p>multiple steps: Intersect, then select by contains or intersection to eliminate unwanted data.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>85</p><p>See also</p><p>Refer to the next recipe in the chapter if you want a way to limit features without altering the</p><p>original spatial data</p><p>Extracting vectors</p><p>Clipping is great, except when you don't want to alter the original geometries, such as when</p><p>you want to select overlapping features. Or, in other cases, you just want filter the geometries</p><p>based on nonspatial attributes. To achieve both of these results, you can utilize the Selection</p><p>tools in combination with Save Layer As.. to extract just the features of interest. This recipe</p><p>uses spatial selection methods to extract a subset of original polygons without altering them.</p><p>Getting ready</p><p>We'll use the same data as the previous recipe, geology.shp and census_wake2000.shp.</p><p>How to do it…</p><p>1. Select polygons from geology.shp that overlap with Wake County</p><p>(census_wake2000.shp) by navigating to Vector | Research Tools |</p><p>Select by location.</p><p>2. Select the feature in geology.</p><p>3. Intersect the features in census_wake2000.</p><p>4. Modify the current selection by creating new selection.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>86</p><p>5. Click on OK:</p><p>6. Now, you will see the matching features highlighted (by default in yellow):</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>87</p><p>7. If the selection looks good, use the Layer context menu, right-click, and click on</p><p>Save As... to create a file containing only the selection.</p><p>When in the Save As... (as described in Chapter 1, Data</p><p>Input and Output) dialog make sure to check the box next</p><p>to Save only selected features.</p><p>How it works…</p><p>This really goes back to the same fundamental concept of Intersection that most vector</p><p>analysis rely on. When you can test whether two features overlap, there are many different</p><p>operations possible based on the answer. You can select, deselect, or, as in the previous</p><p>recipe, select then cut to fit. In these cases, each polygon is tested for at least a partial</p><p>intersection, and the matches are then highlighted as the results.</p><p>There's more…</p><p>While this recipe demonstrates how to select a subset of data based on location, you can also</p><p>do the same thing based on attributes of the features with a query on the attribute table. Or,</p><p>you can combine attribute based selection, spatial selection, and hand selection graphically</p><p>on the map—any selection combination that you want can be saved as a new layer.</p><p>You may also notice in the Select by location tool that vectors can also be added or removed</p><p>from existing selections in case you want to perform more complicated operations involving</p><p>more than one type of criteria.</p><p>See also</p><p>f Refer to the documentation on PostGIS, SpatiaLite, or the PostGIS Cookbook,</p><p>by Packt Publishing, for how to perform similar operations using SQL in PostGIS</p><p>(SpatiaLite, queries are very similar)</p><p>Converting rasters to vectors</p><p>Sometimes, you need to convert data that is originally in raster format to a vector format</p><p>in order to perform vector-based analysis methods. Generally speaking, as rasters are</p><p>continuous datasets, converting them to polygons is more common than converting them to</p><p>lines or points.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>88</p><p>Getting ready</p><p>You need a raster layer, preferably one with groups of the same valued pixels next to each</p><p>other. For this example, we'll use geology_30m.tif, as a 30 meter x 30 meter pixel should</p><p>give decent results.</p><p>The smaller the pixels, the smoother looking the resulting</p><p>vector will appear when zoomed out.</p><p>How to do it…</p><p>1. Load geology_30m.tif.</p><p>2. Go to Raster | Conversion | Polygonize:</p><p>3. Name the output geology_30m.shp.</p><p>4. (Optional) Name the output column geology, class or value.</p><p>5. Press OK to run the process.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter</p><p>3</p><p>89</p><p>6. Compare the results (colors are in a similar but different scale):</p><p>How it works…</p><p>For each pixel, the value is compared to its neighbors (there are different neighbor</p><p>algorithms). When two pixels next to each other have the same value, they are lumped into</p><p>a polygon. Additional neighbors of the same value get added to the polygon until pixels of</p><p>differing values are encountered. As it's pixel-based, the edge of the result will usually follow</p><p>the outline of the pixels, making for a jagged edge. This edge can be smoothed into straight</p><p>lines with additional options or other tools, such as the QGIS smoothing tool.</p><p>There's more…</p><p>The minimum number of pixels required to make a polygon or the maximum allowed value</p><p>difference to be counted as the same can be altered to drop out isolated pixels or to allow for</p><p>a range of values to be counted together.</p><p>Note that if each pixel is unique as compared to its adjacent neighbors, then you'll just end up</p><p>with a polygon for each pixel. Or if your raster is sufficiently large and varied, this process could</p><p>take days. You may want to reconsider whether you really need to convert or whether your</p><p>analysis can be done in raster. Another option would be to resample or reclassify the raster to</p><p>larger polygons first to decrease the data density. Or, you need to investigate remote sensing</p><p>type tools that perform classification to create related groupings of pixels based on similarity.</p><p>If you want to convert raster data to points, then you probably want to use the points sampling</p><p>tool. If you want to convert some portion of a raster to lines then you may need more</p><p>sophisticated feature extraction tools found; for example, in SAGA and GRASS, either through</p><p>the Processing toolbox or as standalone software. Or, you may even need to result to a mix of</p><p>pixel extraction and hand digitizing.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>90</p><p>See also</p><p>f Chapter 7, Raster Analysis I, and Chapter 8, Raster Analysis II, on further raster</p><p>methods</p><p>Converting vectors to rasters</p><p>Occasionally, you want to convert vectors to rasters to facilitate using raster analysis tools</p><p>such as the raster calculator.</p><p>Getting ready</p><p>You'll need a vector layer; this can be a point, line or polygon layer. The best results generally</p><p>come from polygon layers. We will use geology_wake2000reclass.shp. This file is the</p><p>result of the earlier clipping vectors recipe with a new column added that codes the geology</p><p>types as integers. For reference, you'll also use elev_wake_500m.tif as a matching raster</p><p>for the area of interest.</p><p>How to do it…</p><p>In order to be useful in analysis, here's a checklist:</p><p>1. Is the vector data in the same projection as the rest of the raster analysis data?</p><p>If not, reproject it first. Check the following URL for help, https://docs.qgis.</p><p>org/2.8/en/docs/training_manual/vector_analysis/reproject_</p><p>transform.html.</p><p>2. Clip the vector data to the analysis extent. You may need to convert a raster into a</p><p>polygon mask to clip it (refer to the clipping vectors recipe earlier in this chapter).</p><p>You can only pick numeric fields as raster formats only store</p><p>a single number per cell. In order to keep the attribute that</p><p>you want, you may need to use the field calculator to create</p><p>a new field that reclassifies categories of text into a numeric</p><p>scheme (for example, 1 = water, 2 = land, and so on) before</p><p>performing the conversion. If you copy a unique ID as the</p><p>attribute, there are some tools later that let you rejoin the</p><p>original attribute table as a value attribute table (refer to the</p><p>GRASS functions in Processing Toolbox).</p><p>3. Load geology_wake2000reclass.shp and elev_wake_500m.tif.</p><p>www.it-ebooks.info</p><p>https://docs.qgis.org/2.8/en/docs/training_manual/vector_analysis/reproject_transform.html</p><p>https://docs.qgis.org/2.8/en/docs/training_manual/vector_analysis/reproject_transform.html</p><p>https://docs.qgis.org/2.8/en/docs/training_manual/vector_analysis/reproject_transform.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>91</p><p>4. Open Properties of elev_wake_500m.tif:</p><p>1. In the Metadata section, scroll down to Dimensions. You will want to match</p><p>either the dimensions or resolution so that your new raster will match the</p><p>existing elevation data pixels.</p><p>2. Note that Dimensions are X: 134, Y: 124 and Bands: 1. The resolution is</p><p>499.637,-498.342 pixel size.</p><p>3. Close the dialog.</p><p>5. Now, open the conversion dialog by navigating to Raster | Conversion | Rasterize</p><p>and follow these steps:</p><p>1. Input the file as geology_wake2000reclass.</p><p>2. Name your output geology_wake.tif (you will get a warning to set the</p><p>size or resolution).</p><p>3. Set the raster size: Width to 134 and Height to 124.</p><p>4. Click on OK to run the process.</p><p>The following screenshot shows how the screen will look:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>92</p><p>How it works…</p><p>A grid of pixels is created at the specified width, height, and extent. For each cell, an</p><p>intersection is performed with the underlying vector layer. If more than 50% of the cell</p><p>intersects with the vector, it's designated attribute is assigned to the cell.</p><p>There's more…</p><p>It's really important to match projection and extent before converting to raster. If you fail to do</p><p>so, then your pixels in different raster layers won't line up perfectly with each other, and either</p><p>tools won't work or they will introduce a resampling error. If this looks too pixelated (squares)</p><p>for your liking, consider creating the raster at a higher pixel density.</p><p>If you compare the vector version to the new raster, you'll notice that the area in the middle all</p><p>came out a similar color. This is due to the values used for classification, where the geology</p><p>that started with the same major component was given the same starting value (for example,</p><p>PZ all start with 40, and the last number changes based on the letters after PZ).</p><p>Looking at the new layer and want to get rid of the black surrounding the real data? This area</p><p>is no-data, refer to Chapter 8, Raster Analysis II.</p><p>See also</p><p>f There are other methods to calculate the new value of a pixel to make smoother</p><p>transitions or intermediate values when multiple polygons are with the same</p><p>pixel. Refer to the GRASS and SAGA methods in Processing Toolbox for more</p><p>sophisticated alternatives.</p><p>Building DateTime strings</p><p>Date and time data get stored in all sorts of ways. One of the more frustrating issues is that</p><p>some common GIS formats (Shapefiles) can't store date and time in the same field without</p><p>making it a string. This is fine for visual display but terrible for use with tools that use DateTime</p><p>for their functionality, such as the TimeManager Plugin (refer to Chapter 4, Data Exploration).</p><p>Getting ready</p><p>Use datetime-example.shp, which contains a variety of date and time representations to</p><p>play with.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>93</p><p>How to do it…</p><p>1. Load datetime-example.shp.</p><p>2. Open the attribute table of datetime-example.</p><p>3. Create a new field. As this is a shapefile, we'll need to use a String of length 30 (or</p><p>you can use the empty field called calculated).</p><p>4. Turn on layer editing (this is the pencil icon, which is the first icon to the left of the</p><p>window toolbar).</p><p>5. From the drop-down list, select Calculated.</p><p>6. Now press the Calculation button.</p><p>In older versions of QGIS, you'll need to open the Field</p><p>calculator, which also works in newer versions but has</p><p>slightly more steps.</p><p>7. In the calculator, we'll use the substr String operation (that is, Substring) in</p><p>combination with the || concatenation to rearrange the values from existing fields</p><p>into a valid DateTime:</p><p>1. The simplest variant is just to combine shpDate with Time and put a space</p><p>in between:</p><p>"shpdate" || ' '|| "Time"</p><p>2. For a bit more of a challenge, use Date:</p><p>substr("Date",7,4) ||'-'||substr("Date",1,2)||'-'||</p><p>substr("Date",4,2) || ' '|| "Time"</p><p>Note the use of single quotes (') to denote a string</p><p>as opposed</p><p>57</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>ii</p><p>Table of Contents</p><p>Georeferencing vector layers 61</p><p>Creating raster overviews (pyramids) 66</p><p>Building virtual rasters (catalogs) 68</p><p>Chapter 3: Common Data Preprocessing Steps 71</p><p>Introduction 71</p><p>Converting points to lines to polygons and back – QGIS 72</p><p>Converting points to lines to polygons and back – SpatiaLite 74</p><p>Converting points to lines to polygons and back – PostGIS 77</p><p>Cropping rasters 80</p><p>Clipping vectors 82</p><p>Extracting vectors 85</p><p>Converting rasters to vectors 87</p><p>Converting vectors to rasters 90</p><p>Building DateTime strings 92</p><p>Geotagging photos 95</p><p>Chapter 4: Data Exploration 99</p><p>Introduction 99</p><p>Listing unique values in a column 100</p><p>Exploring numeric value distribution in a column 102</p><p>Exploring spatiotemporal vector data using Time Manager 105</p><p>Creating animations using Time Manager 108</p><p>Designing time-dependent styles 110</p><p>Loading BaseMaps with the QuickMapServices plugin 112</p><p>Loading BaseMaps with the OpenLayers plugin 116</p><p>Viewing geotagged photos 119</p><p>Chapter 5: Classic Vector Analysis 125</p><p>Introduction 125</p><p>Selecting optimum sites 125</p><p>Dasymetric mapping 131</p><p>Calculating regional statistics 135</p><p>Estimating density heatmaps 137</p><p>Estimating values based on samples 139</p><p>Chapter 6: Network Analysis 143</p><p>Introduction 143</p><p>Creating a simple routing network 144</p><p>Calculating the shortest paths using the Road graph plugin 148</p><p>Routing with one-way streets in the Road graph plugin 150</p><p>Calculating the shortest paths with the QGIS network analysis library 152</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>iii</p><p>Table of Contents</p><p>Routing point sequences 156</p><p>Automating multiple route computation using batch processing 158</p><p>Matching points to the nearest line 159</p><p>Creating a routing network for pgRouting 160</p><p>Visualizing the pgRouting results in QGIS 163</p><p>Using the pgRoutingLayer plugin for convenience 166</p><p>Getting network data from the OSM 168</p><p>Chapter 7: Raster Analysis I 171</p><p>Introduction 171</p><p>Using the raster calculator 172</p><p>Preparing elevation data 175</p><p>Calculating a slope 177</p><p>Calculating a hillshade layer 180</p><p>Analyzing hydrology 183</p><p>Calculating a topographic index 190</p><p>Automating analysis tasks using the graphical modeler 192</p><p>Chapter 8: Raster Analysis II 199</p><p>Introduction 199</p><p>Calculating NDVI 199</p><p>Handling null values 203</p><p>Setting extents with masks 207</p><p>Sampling a raster layer 210</p><p>Visualizing multispectral layers 212</p><p>Modifying and reclassifying values in raster layers 216</p><p>Performing supervised classification of raster layers 219</p><p>Chapter 9: QGIS and the Web 223</p><p>Introduction 223</p><p>Using web services 224</p><p>Using WFS and WFS-T 226</p><p>Searching CSW 229</p><p>Using WMS and WMS Tiles 231</p><p>Using WCS 235</p><p>Using GDAL 238</p><p>Serving web maps with the QGIS server 242</p><p>Scale-dependent rendering 247</p><p>Hooking up web clients 251</p><p>Managing GeoServer from QGIS 256</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>iv</p><p>Table of Contents</p><p>Chapter 10: Cartography Tips 259</p><p>Introduction 259</p><p>Using Rule Based Rendering 260</p><p>Handling transparencies 266</p><p>Understanding the feature and layer blending modes 269</p><p>Saving and loading styles 273</p><p>Configuring data-defined labels 277</p><p>Creating custom SVG graphics 282</p><p>Making pretty graticules in any projection 286</p><p>Making useful graticules in printed maps 291</p><p>Creating a map series using Atlas 295</p><p>Chapter 11: Extending QGIS 299</p><p>Introduction 299</p><p>Defining custom projections 300</p><p>Working near the dateline 304</p><p>Working offline 309</p><p>Using the QspatiaLite plugin 311</p><p>Adding plugins with Python dependencies 313</p><p>Using the Python console 315</p><p>Writing Processing algorithms 318</p><p>Writing QGIS plugins 322</p><p>Using external tools 327</p><p>Chapter 12: Up and Coming 333</p><p>Introduction 333</p><p>Preparing LiDAR data 334</p><p>Opening File Geodatabases with the OpenFileGDB driver 337</p><p>Using Geopackages 339</p><p>The PostGIS Topology Editor plugin 342</p><p>The Topology Checker plugin 346</p><p>GRASS Topology tools 350</p><p>Hunting for bugs 354</p><p>Reporting bugs 358</p><p>Index 363</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>v</p><p>Preface</p><p>Location-based technology is the latest buzzword to explain tools related to spatial knowledge</p><p>and analysis. For those who work on map making, geospatial science, or any number of other</p><p>things with spatial data, Geographic Information Systems (GIS), which is the more traditional</p><p>name for such tools, is a field of study with decades of innovation.</p><p>QGIS (previously known as Quantum GIS), a cross-platform, free, and open source software,</p><p>provides a traditional desktop-based geographic information system. Unlike a traditional</p><p>system though, it is highly customizable, extendable, and, by design, works in tandem</p><p>with a ton of other GIS-related tools (more are added all the time).</p><p>QGIS is a crossing point of the free and open source geospatial world. While there are a</p><p>great many tools in QGIS, it is not one massive application that does everything, and it was</p><p>never really designed to be that from the beginning. It is rather a visual interface to much of</p><p>the open source geospatial world. You can load data from proprietary and open formats into</p><p>spatial databases of various flavors and then analyze the data with well-known analytical</p><p>backends before creating a printed or web-based map to display and interact with your</p><p>results. What's QGIS's role in all this? It's the place where you check your data along the way,</p><p>build and queue the analysis, visualize the results, and develop cartographic end products.</p><p>If you need to test modify one layer before doing a batch of 1000, use QGIS. Want to make</p><p>sure the results of that SQL query or script make sense, use QGIS. Need to tinker with</p><p>alternative methods of displaying your data to find the right colors, lines, and layers to convey</p><p>your message, use QGIS. Find something QGIS can't do, look at other tools it works with,</p><p>search the plugin list, write a plugin, or submit a new feature request. With such a mentality,</p><p>everything is possible—it just takes an understanding of how to get there.</p><p>This book is all about showing you how to do all these great things and all the new cool things</p><p>you didn't know you wanted to do. When QGIS doesn't do what you need or doesn't work with</p><p>the tools you want to use, we'll show you where to go and who to talk to about making your</p><p>dream of new functionality a reality.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>vi</p><p>What this book covers</p><p>Over the course of 12 chapters, this book will take you from data input and output, through</p><p>data management and analysis, to creating print and web output, as well as extending QGIS.</p><p>Chapter 1, Data Input and Output, covers loading and saving data with special instructions</p><p>for trickier formats, batch conversions, and databases.</p><p>Chapter 2, Data Management, describes the basic manipulation of attributes, indexes, and</p><p>queries to make the use of your data more efficient.</p><p>Chapter 3, Common Data Preprocessing Steps, deals with converting data into the formats</p><p>you need for analysis, including vector to and from raster, transitioning through different</p><p>types of vectors, and cutting your data to just the important areas.</p><p>Chapter 4, Data Exploration, explores methods for visualizing and understanding the</p><p>information in your data.</p><p>Chapter 5, Classic Vector Analysis, shows the QGIS way of performing traditional analysis</p><p>methods of vector layers.</p><p>Chapter 6, Network Analysis, dives into the methods for analyzing routes and networks.</p><p>Chapter 7, Raster Analysis I, covers raster analysis that is primarily related to topography</p><p>and hydrology.</p><p>Chapter 8, Raster Analysis II, covers common raster analysis methods and introduces more</p><p>advanced multispectral and classification data handling.</p><p>Chapter 9, QGIS and the Web, explores the use of live data from the Web and how to put up</p><p>your own web map based on a QGIS project.</p><p>Chapter 10, Cartography Tips, reveals advanced tips and tricks to get the most out of the</p><p>cartographic tools in QGIS.</p><p>Chapter 11, Extending QGIS, shows you how to take QGIS beyond the out-of-the-box features</p><p>with plugins, customization, and add-on tools.</p><p>Chapter 12, Up</p><p>to double quotes ("), which indicate a</p><p>field name.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>94</p><p>8. Experiment with the formulas, checking the results with the calculator preview at the</p><p>bottom. When satisfied, select OK, and then click on the Update All button:</p><p>9. If you're happy with the results, save the edits; if not, toggle editing off and choose</p><p>Close without saving.</p><p>How it works…</p><p>Substr takes three arguments: the field name, the starting position, and the number of</p><p>characters after this to include. The position index starts at 1.</p><p>There's more…</p><p>Using string manipulation, we've combined multiple fields into an ISO standard format for</p><p>DateTime, which other tools will recognize and be able to utilize.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>95</p><p>The fields included in this example data are as follows:</p><p>Field Type Explanation</p><p>Time String This is time in a 24 hour format: hours:minutes:seconds.</p><p>datetime String This is a DateTime String from a typical GPS GPX file.</p><p>calculated String This is a String that is long enough to hold date and time with</p><p>padding characters and a timezone UTC offset.</p><p>Date String This is a typical date format coming out of a spreadsheet.</p><p>shpdate Date This is a date format in the Date type within a shapefile, and</p><p>it can not hold time.</p><p>See also</p><p>f If using database layers, this can all be performed with SQL in SpatiaLite or PostGIS.</p><p>The ISO date time standard is 8601, and can be found at http://en.wikipedia.</p><p>org/wiki/ISO_8601.</p><p>Geotagging photos</p><p>Newer cameras and phones with built-in GPS can be wonderful tools for data collection, as</p><p>they help keep track of exactly where and when a picture was taken. However, not all cameras</p><p>have a built-in GPS. You can add geotags afterwards, either with a GPS log from a separate</p><p>GPS unit or just using a reference map and your memory or notes.</p><p>Getting ready</p><p>For this recipe, you'll need a some photos and either a GPS log (*.gpx), reference vector,</p><p>reference raster, or coordinates. We've provided centerofcalifornia.jpg in the</p><p>geotag folder, and the coordinates are in the image itself but also included as a point in</p><p>centerofcalifornia.shp.</p><p>You will also need the Geotag photos plugin, which requires the exiftool program to be</p><p>installed on your system. If exiftool didn't come with your install, you can easily get it from</p><p>the Web at http://www.sno.phy.queensu.ca/~phil/exiftool/ or at package</p><p>repositories (Linux).</p><p>www.it-ebooks.info</p><p>http://en.wikipedia.org/wiki/ISO_8601</p><p>http://en.wikipedia.org/wiki/ISO_8601</p><p>http://www.sno.phy.queensu.ca/~phil/exiftool/</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>96</p><p>How to do it…</p><p>This particular plugin assigns location per folder, so all photos in a folder will get the same</p><p>coordinates. This works well for batch assigning of general coordinates:</p><p>1. Start by loading your GPS log, or creating a new vector layer and digitizing the points</p><p>that you want to assign to photos. In this case, load centerofcalifornia.shp.</p><p>Don't have a lot of locations or just have coordinates written</p><p>down by hand? You can manually enter the information into</p><p>the plugin interface without an existing layer.</p><p>2. (Optional) Load a background reference layer.</p><p>3. Now that you have the layer that you want to associate with photos open the plugin in</p><p>Vector | Geotag and import photos | Geotag photos.</p><p>4. Select the layer and then the field (location) that you want to use as the label.</p><p>5. Now, click on Populate Table. One row should have been added to the interface.</p><p>6. Now, pick the photos by clicking under Path to Folder in the empty box for the row</p><p>that you want to assign:</p><p>1. You can type in the path or browse by clicking on ….</p><p>2. Pick the geotag folder.</p><p>3. Make sure to press Enter or click outside the box once you are back in the</p><p>main screen:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 3</p><p>97</p><p>7. (Optional) Check the box to rename geotagged files if you want the geotagged version</p><p>to be a new copy of the file instead of altering the original files (always keep an</p><p>original backup).</p><p>8. Click on OK.</p><p>How it works…</p><p>Exiftools writes to the built-in metadata of an image file to a section called EXIF. It's a standard</p><p>in photography to store extra data about photos that many software management tools can</p><p>easily read from. Latitude and Longitude in WGS 84 coordinates are the standard method</p><p>of encoding GPS data within the EXIF section.</p><p>There's more…</p><p>Now that you have a geotagged photo, you can upload it to sites such as Flickr, which will</p><p>display it on a map, or skip to the recipe Viewing Geotagged Photos in Chapter 4, Data</p><p>Exploration, for how to make a map in QGIS.</p><p>This plugin is very manual and assigns location per folder as it was created to work specifically</p><p>with camera traps. Instead, if you were travelling between each photo location and have a GPS</p><p>log, there are other non-QGIS tools to help you match GPS points with your photos. Digikam</p><p>(a photography management tool) has a function to geotag based on timestamp matches.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Common Data Preprocessing Steps</p><p>98</p><p>See also</p><p>f The OpenStreetMap wiki lists other free and paid options out there at</p><p>http://wiki.openstreetmap.org/wiki/Geotagging_Source_Photos</p><p>www.it-ebooks.info</p><p>http://wiki.openstreetmap.org/wiki/Geotagging_Source_Photos</p><p>http://www.it-ebooks.info/</p><p>99</p><p>4</p><p>Data Exploration</p><p>In this chapter, we will cover the following recipes:</p><p>f Listing unique values in a column</p><p>f Exploring numeric value distribution in a column</p><p>f Exploring spatiotemporal data using Time Manager</p><p>f Creating animations using Time Manager</p><p>f Designing time-dependent styles</p><p>f Loading BaseMaps with the QuickMapServices plugin</p><p>f Loading BaseMaps with the OpenLayers plugin</p><p>f Viewing geotagged photos</p><p>Introduction</p><p>This chapter focuses on recipes that will help you visually inspect and better comprehend</p><p>your data. The recipes in this chapter include methods of summarizing, inspecting, filtering,</p><p>and styling data, based on spatial and temporal attributes so that you can get a better feeling</p><p>for your data before you perform analysis. The primary goal is to create some visuals or</p><p>summaries of data that allow you, the human, to utilize your brain's ability to identify patterns</p><p>of interest. The better you understand your data, the easier it is to pick appropriate analysis</p><p>methods later.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>100</p><p>Listing unique values in a column</p><p>When investigating a new dataset, it is very helpful to have a way to quickly check which</p><p>values a column contains. In this recipe, we will use different approaches using both the GUI</p><p>and the Python console to list the unique values of POI classes in our sample POI dataset.</p><p>Getting ready</p><p>To follow this recipe, please load poi_names_wake.shp.</p><p>How to do it…</p><p>If you are simply looking for a solution based on the GUI, the List unique values tool is</p><p>available both in Vector | Analysis Tools as well as in the Processing Toolbox menu. You can</p><p>use either one of these to get a list of the unique values in a column. Having this tool available</p><p>in the Processing Toolbox menu makes it possible to include it in processing models and,</p><p>thus, automate the process. The following steps to list unique values use the Processing</p><p>Toolbox menu:</p><p>1. Start List unique values from the Processing Toolbox menu.</p><p>2. Select the poi_names_wake layer as Input layer and the class attribute as</p><p>Target field.</p><p>3. Click on Run and wait for the tool to finish. The results will be displayed in the</p><p>Results view, which will open automatically.</p><p>If you want to further customize this task, for example, by counting how often the values</p><p>appear in this dataset, it's time to fire up Python console:</p><p>1. Start Python Console, which you will find in the Plugins menu, and click on the Show</p><p>editor button (in the toolbar to the left of Python console) to open the editor window.</p><p>2. Paste the following short script into the editor, save it, and then click on the Run</p><p>script button. (Make sure that the POI layer is selected in the layer list.) It loops</p><p>through all features in the active layer and creates a dictionary object, which contains</p><p>all unique values and the corresponding counts:</p><p>import processing</p><p>layer = iface.activeLayer()</p><p>classes = {}</p><p>features = processing.features(layer)</p><p>for f in features:</p><p>attrs = f.attributes()</p><p>class_value = f['class']</p><p>if class_value in classes:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>101</p><p>classes[class_value] += 1</p><p>else:</p><p>classes[class_value] = 1</p><p>print classes</p><p>The following screenshot shows what the screen looks like:</p><p>How it works…</p><p>In the first line, we use import processing because it offers a very handy and convenient</p><p>function, processing.features(), to access layer features, which we use in line 7. It</p><p>is worth noting that, if there is a selection, processing.features() will only return an</p><p>iterator of the selected features.</p><p>In line 3, we get the currently active layer object. Line 5 creates the empty dictionary object,</p><p>which we will fill with the unique values and corresponding counts.</p><p>The for loop, starting on line 5, loops through all features of the layer. For each feature, we</p><p>get the value in the classification field (line 7). You can change the column name to analyze</p><p>other columns. Then, we only need to check whether this value is already present in the</p><p>classes dictionary (line 8) or whether we have to add it (line 11).</p><p>There's more…</p><p>If you are using an SQL database, such as Spatialite or PostGIS, you can achieve similar</p><p>results with a query using the COUNT and GROUP BY functions to count the number of</p><p>features or records per class:</p><p>SELECT class, COUNT(*)</p><p>FROM poi_name_wake</p><p>GROUP BY class</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>102</p><p>Exploring numeric value distribution in a</p><p>column</p><p>In this recipe, we will look at how to explore the properties of a column of numeric values. We</p><p>will look at the tools that QGIS offers and apply them to analyze the elevation values in our</p><p>sample POI dataset.</p><p>Getting ready</p><p>To follow this recipe, please load poi_names_wake.shp. If you followed the previous recipe,</p><p>Listing unique values in a column, you can continue directly from there.</p><p>How to do it…</p><p>A good way to get a first impression of the properties of a numeric column is using the</p><p>Basic Statistics tool from Vector. This allows you to calculate statistical values, such as the</p><p>minimum and maximum values, mean and median, standard deviation, and sum.</p><p>If you want to examine the distribution of elevation values, there is the handy Statist plugin.</p><p>Statist generates an interactive histogram representation of the value distribution:</p><p>1. Install Statist using Plugin Manager.</p><p>2. Start Statist from the Vector menu.</p><p>3. Specify Input vector layer and the attribute that you want to analyze (Target field),</p><p>then click on OK to compute the statistics.</p><p>4. Using the buttons below the diagram, you can zoom and pan the diagram, as well as</p><p>save the diagram image.</p><p>5. You can even customize the diagram by changing the title and axis labels and</p><p>ranges. Just use the right-most button with the green tick mark on it to open the</p><p>customization dialog:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>103</p><p>How it works...</p><p>Thanks to Python Console and the editor, we are not limited to the existing tools and plugins.</p><p>Instead, we can create or own specialized scripts such as the following one. This script creates</p><p>a short layer statistics report using HTML and the Google Charts Javascript API (for more</p><p>information and API docs refer to https://developers.google.com/chart/), which it</p><p>then displays in a QWebView window. Of course, you can use any other JavaScript charting API</p><p>as well. (Note that you need to be connected to the Internet for this script to work because it</p><p>has to download the Javascript.) We recommend using the editor that was introduced in the</p><p>previous recipe. Don't forget to select the layer in the legend:</p><p>import processing</p><p>from PyQt4.QtWebKit import QWebView</p><p>layer = iface.activeLayer()</p><p>values = []</p><p>features = processing.features(layer)</p><p>for f in features:</p><p>values.append( f['elev_m'])</p><p>myWV = wQWebView(None)</p><p>html=''</p><p>html+=''</p><p>html+='google.load("visualization","1",{packages:["corechart"]});'</p><p>html+='google.setOnLoadCallback(drawChart);'</p><p>www.it-ebooks.info</p><p>https://developers.google.com/chart/</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>104</p><p>html+='function drawChart() { '</p><p>html+='var data = google.visualization.arrayToDataTable(['</p><p>html+='["%s"],' % (field_name)</p><p>for value in values:</p><p>html+='[%f],' % (value)</p><p>html+=']);'</p><p>html+='var chart = new google.visualization.Histogram('</p><p>html+='document.getElementById("chart_div"));'</p><p>html+='chart.draw(data, {title: "Histogram"});}'</p><p>html+='Layer: %s' % (layer.name())</p><p>html+='Values for %s range from: ' % (field_name)</p><p>html+='%d to %d' % (min(values),max(values))</p><p>html+=''</p><p>html+=''</p><p>myWV.setHtml(html)</p><p>myWV.show()</p><p>Of course, custom reports such as this one lend themselves to adding more details. For</p><p>example, we can create separate histograms for each POI class or add other types of charts,</p><p>such as scatter charts:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>105</p><p>The first part (lines 1 to 12) is very similar to the script explained in the previous recipe,</p><p>Listing unique values in a column: We get the active layer and collect all elevation values in</p><p>the values list.</p><p>The QWebView created on line 14 enables us to display the HTML content, which we then</p><p>generate in the following section (lines 16 to 34). First, we load the Google Charts Javascript.</p><p>The actual magic happens in the drawChart() function starting on line 21. Lines 22 to 26</p><p>create the data object, which is filled with the elevation values from our values list. The last</p><p>three lines of the function (lines 27 to 29) finally create and draw the histogram chart. Finally,</p><p>lines 30 to 34 contain the HTML body definition with the header stating the layer name and a</p><p>short introduction text that states the min and max elevation values.</p><p>See also</p><p>f For those who want to perform more advanced graphing and numerical analysis,</p><p>consider using the matplotlib python library or reading your data sources into R.</p><p>Aggregate functions in SatialLitetgis PostGIS can also provide you with min, max,</p><p>average, sum, and other summarization functions. For PostGIS, refer to http://</p><p>www.postgresql.org/docs/9.1/static/functions-aggregate.html.</p><p>Exploring spatiotemporal vector data using</p><p>Time Manager</p><p>In this recipe, we will look at exploring spatiotemporal vector data using the Time Manager</p><p>plugin. We'll use event data from the ACLED (Armed Conflict Location and Event Data</p><p>Project) at http://www.acleddata.com/about-acled/.</p><p>Getting ready</p><p>To follow this recipe, please load ACLED_africa_fatalities_dec2013.shp. The layer</p><p>style that you will see in the following screenshots consists of a simple circle marker at 50%</p><p>transparency with the data-defined size set to the number of fatalities of the incident. (You</p><p>can read more about styling in Chapter 10, Cartography Tips, and Learning QGIS book by</p><p>Packt Publishing.) If you want some additional geographic context, you can also load NE_</p><p>africa.shp, which contains the outline of Africa.</p><p>www.it-ebooks.info</p><p>http://www.postgresql.org/docs/9.1/static/functions-aggregate.html</p><p>http://www.postgresql.org/docs/9.1/static/functions-aggregate.html</p><p>http://www.acleddata.com/about-acled/</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>106</p><p>How to do it…</p><p>Once the data is loaded, all event positions will be displayed. The default way to filter the</p><p>events, for example, to only see the</p><p>events from December 1, is to use Layer | Query and</p><p>enter a filter expression or query, such as the following:</p><p>"EVENT_DATE" >= '2013-12-01' AND "EVENT_DATE"</p><p>size.</p><p>For the time-dependent style, we will add a new definition to the Fill color property,</p><p>as shown in the following screenshot:</p><p>2. Confirm the changes and start the animation to watch the effect.</p><p>How it works…</p><p>Here is our color expression in more detail:</p><p>color_hsla(</p><p>0,</p><p>scale_linear(</p><p>day(age(todatetime(animation_datetime()),</p><p>todatetime("EVENT_DATE"))),</p><p>0,31,</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>112</p><p>100,0</p><p>),</p><p>50,</p><p>128</p><p>)</p><p>The expression consists of multiple parts, as follows:</p><p>f day(age(todatetime(animation_datetime()),todatetime("EVENT_</p><p>DATE"))): This calculates the number of days between the current animation time</p><p>given by animation_datetime() and calculates EVENT_DATE</p><p>f scale_linear: This transforms the age value (in days between 0 to 31 days) into</p><p>a value between 100 and 0 which is suitable for the saturation parameter of the</p><p>following color function</p><p>f color_hsla: This is one of many functions that are available in QGIS to create</p><p>colors. It returns a string representation of a color, based on its attributes for hue</p><p>(0 equals red), saturation (between 0 and 100 depending on our age function),</p><p>lightness (50 equals medium lightness) and alpha (128 equals 50% transparency)</p><p>You can speed up the fading effect by reducing the scale_linear parameter, domain_max,</p><p>from 31 days to a smaller value, such as 7, for a complete fade to gray within one week.</p><p>See also</p><p>f If you are interested in learning more about color models, such as the HSL color</p><p>model used in this recipe, we recommend the Wikibook on color models at</p><p>http://en.wikibooks.org/wiki/Color_Models:_RGB,_HSV,_HSL</p><p>Loading BaseMaps with the</p><p>QuickMapServices plugin</p><p>Often, when exploring your data, you may feel somewhat lost. Without the context of the</p><p>known world, a layer can seem like a blob of information floating in space. By adding an</p><p>atlas-style map, air photos, or another BaseMap, you can begin to see how your data fits in</p><p>the on-the-ground reality. However, adding such layers often takes considerable preprocessing</p><p>work; sometimes, you just don't want to go through this until you know you need it. What's the</p><p>solution? Use a premade layer, preferably fast-loading tiles, from a web service.</p><p>www.it-ebooks.info</p><p>http://en.wikibooks.org/wiki/Color_Models:_RGB,_HSV,_HSL</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>113</p><p>Getting ready</p><p>The QuickMapServices plugin works best when you have another dataset that you want to</p><p>provide extra context for. Start by first loading such a layer and then zooming in to its extent.</p><p>You will need the following:</p><p>f A layer of interest to overlay (you can use Davis_DBO_Centerline-wgs84.shp)</p><p>f An active Internet connection (this may not work behind corporate proxies)</p><p>f You will need to install and activate the QuickMapServices plugin</p><p>How to do it…</p><p>Starting with a new QGIS project, follow these instructions to load BaseMap from the web with</p><p>the QuickMapServices plugin:</p><p>1. Start by first loading your local map layers:</p><p>2. Verify that the projection definition is correctly identified by QGIS.</p><p>The plugin will not turn on projection-on-the-fly for you</p><p>unless you change its settings. However, in order for most</p><p>tile services to work in QGIS, projection-on-the-fly must</p><p>be enabled and set to EPSG:3857 Psuedo/Web/Popular</p><p>Mercator. Other data will fail to line up if their projection is</p><p>not defined or read properly by QGIS.</p><p>3. Go to Web | QuickMapServices and select a layer to load from the Web. Wait a few</p><p>seconds for the tiles to be loaded:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>114</p><p>The default list of services is open and free. If you</p><p>want to use other services that have more limited</p><p>licensing restrictions, such as Google and Bing, you</p><p>need to change some of the plugin's settings. Refer</p><p>to the There's more… section of this recipe.</p><p>4. (Optional) Temporarily disable Rendering (the checkbox in the bottom panel) to avoid</p><p>constant redrawing while rearranging the layer order.</p><p>5. Rearrange your layers to move the new QuickMapService added layer to the bottom</p><p>of your layer list.</p><p>6. Zoom to your original layer's extent.</p><p>7. (Optional) If you turned Rendering off, reactivate Rendering now.</p><p>Are things not lining up? Try zooming in a little more or</p><p>panning slightly. Most of all, be patient! Depending on</p><p>your Internet connection, it can take a while to retrieve</p><p>the tiles.</p><p>The following screenshot shows how the screen will look:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>115</p><p>How it works…</p><p>The QuickMapServices plugin is a web-based tool. All of the BaseMaps come from the Internet</p><p>as you pan and zoom; none of the data comes from your computer or QGIS itself.</p><p>There are a few things to be cautious of when using the QuickMapServices plugin. It doesn't</p><p>always line up quite right, especially when zoomed out to big areas. First, check whether your</p><p>other layers' projections are defined correctly and then try to reset the map by slightly panning</p><p>to the side. The key idea to remember is that tiled services generally only exist for EPSG:3857</p><p>and at a very specific set of scales. QGIS will attempt to pick the closest matching scale and</p><p>resample the scale to make it fit. This also explains why loading such layers can sometimes</p><p>be slow.</p><p>There's more…</p><p>To add more restricted services, such as Google. Bing, and so on, perform the following steps:</p><p>1. Go to Web | QuickMapServices | Settings | Contributed Services.</p><p>2. Click on the Get contributed pack button:</p><p>While it may be legal to view the maps (most of the time), depending on layers that are</p><p>selected, it may not be legal to digitize maps based on them, print them, or, otherwise, save</p><p>them for offline use. The license varies by data source. So, make sure to check this for the</p><p>sources you want to use by going online and reading the Terms of Service on their websites.</p><p>If your use case is outside of generally viewing for quick reference, you will probably need to</p><p>spend some time obtaining a license or permission for your use.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>116</p><p>OpenStreetMap-based sources are often good choices as the licenses typically just require</p><p>attribution with no restrictions on use. The main layers that originally come with the plugin</p><p>are there because they have less restrictive licenses.</p><p>Finally, you may be wondering how QuickMapServices differs from the OpenLayers plugin</p><p>mentioned in the next recipe. For starters, this plugin is newer and currently supported. It</p><p>also solves some long-standing issues, especially in regards to printing. There is also the</p><p>contributed layers GitHub repository, which should make it easier for people to contribute</p><p>new layer definitions.</p><p>See also</p><p>f Additional tile services can be added by hacking the plugin code or using a GDAL</p><p>TMS layer (Refer to Chapter 9, QGIS and the Web). You can also substitute in WMS</p><p>services to serve a similar role without some of the same limitations of tiles. Refer to</p><p>Chapter 9, QGIS and the Web, for information about creating your own web services.</p><p>Loading BaseMaps with the OpenLayers</p><p>plugin</p><p>Often, when exploring your data, you may feel somewhat lost. Without the context of the</p><p>known world, a layer can seem like a blob of information floating in space. By adding an atlas-</p><p>style map, air photos, or another BaseMap, you can begin to see how your data fits in the</p><p>on-the-ground reality. However, adding such layers often takes considerable preprocessing</p><p>work; sometimes, you just don't want to go through this until you know you need it. What's the</p><p>solution? Use a premade layer from a web service.</p><p>This recipe is almost identical to the previous recipe. QuickMapServices</p><p>is a replacement for the OpenLayers plugin, which is being discontinued</p><p>(deprecated). We kept this recipe because it's still a commonly-mentioned</p><p>plugin and works slightly differently. However, please consider using</p><p>QuickMapServices.</p><p>Getting ready</p><p>The</p><p>Openlayers plugin works best when you have another dataset that you want to provide</p><p>extra context for. Start by first loading such a layer and then zooming in to its extent.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>117</p><p>You will need the following:</p><p>f A layer of interest to overlay (you can use Davis_DBO_Centerline-wgs84.shp)</p><p>f An active Internet connection (this may not work behind corporate proxies)</p><p>f You will need to install and activate the OpenLayers plugin</p><p>How to do it…</p><p>Starting with a new QGIS project, follow these instructions to load BaseMap from the Web with</p><p>the Openlayers plugin:</p><p>1. Start by first loading your local map layers.</p><p>2. Verify that the projection definition is correctly identified by QGIS.</p><p>The plugin will set and turn on projection-on-the-fly for you. In order</p><p>for most tile services to work in QGIS, projection-on-the-fly must be</p><p>enabled and set to EPSG:3857 Psuedo/Web/Popular Mercator.</p><p>Other data will fail to line up if their projection is not defined or read</p><p>properly by QGIS.</p><p>3. Go to Vector | OpenLayers Plugin and select a layer to load from the Web. Wait a few</p><p>seconds for the tiles to be loaded.</p><p>4. (Optional) Temporarily disable Rendering (the checkbox in the bottom panel) to avoid</p><p>constant redrawing while rearranging layer order.</p><p>5. Rearrange your layers to move the new OpenLayers added layer to the bottom of your</p><p>layer list.</p><p>6. Zoom to your original layer's extent.</p><p>7. (Optional) If you turned Rendering off, reactivate Rendering now.</p><p>Are things not lining up? Try zooming in a little more or</p><p>panning slightly. Most of all, be patient! Depending on</p><p>your Internet connection, it can take a while to retrieve</p><p>the tiles.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>118</p><p>The following screenshot shows how the screen will look:</p><p>How it works…</p><p>The OpenLayers plugin is a web-based tool, based on the similarly named OpenLayers library</p><p>to create web-based maps in an Internet browser. However, instead of displaying the maps in</p><p>the browser, this plugin renders them into an active QGIS canvas (that is, a map).</p><p>There are a few things to be cautious of when using the OpenLayers plugin. It doesn't always</p><p>line up quite right, especially when zoomed out to big areas. First, check whether your other</p><p>layers' projections are defined correctly and then try to reset the map by panning slightly to</p><p>the side. The key idea to remember is that tiled services generally only exist for EPSG:3857</p><p>and at a very specific set of scales. QGIS will attempt to pick the closest matching scale and</p><p>resample the scale to make it fit. This also explains why loading such layers can sometimes</p><p>be slow.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>119</p><p>There's more…</p><p>While it may be legal to view the maps, depending on layers selected, it may not be legal to</p><p>digitize maps based on them, print them, or, otherwise, save them for offline use. The license</p><p>varies by data source. So, make sure to check for the sources you want to use by going online</p><p>and reading the Terms of Service on their websites. If your use case is outside of generally</p><p>viewing for quick reference, you will probably need to spend some time obtaining a license</p><p>or permission for your use. OpenStreetMap-based sources are often good choices as the</p><p>licenses typically just require attribution.</p><p>See also</p><p>f Additional tile services can be added by hacking the plugin code or using a GDAL</p><p>TMS layer (Refer to Chapter 9, QGIS and the Web). You can also substitute in WMS</p><p>services to serve a similar role without some of the same limitations of tiles. Refer to</p><p>Chapter 9, QGIS and the Web, for information about creating your own web services.</p><p>Viewing geotagged photos</p><p>Keeping track of photographs by location can be an extremely useful tool, enabling you to</p><p>easily pull up relevant photos of a place and time. They provide local context about other data</p><p>collected in the same place, and they can provide office staff with a view of what people in the</p><p>field saw. You can think of this as your own personal Street View, which is just more focused</p><p>than Google's version.</p><p>Getting ready</p><p>For this recipe, you'll need a set of geotagged photos. We've included a set a photos in this</p><p>book's data for you to learn with. This is a collection of photos from downtown Davis that</p><p>highlights the density and variety of public art along several blocks.</p><p>This recipe also takes advantage of several plugins, as follows:</p><p>f Install and activate Photo2Shape</p><p>f Activate the core plugin, eVis (Event Visualization)</p><p>f (Optional) Install and activate OpenLayers Plugin</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>120</p><p>How to do it…</p><p>Follow these steps to view geotagged photo locations in QGIS:</p><p>1. In a QGIS project, enable the plugins listed in the Getting ready section.</p><p>2. (Optional) Load a reference layer to help you see the local context</p><p>(Davis_DBO_Centerline.shp and/or OpenStreetMap/Google Streets</p><p>via OpenLayers Plugin).</p><p>Keep in mind that GPS locations and geotagged photos are almost</p><p>always in Latitude and Longitude WGS84 coordinates (that is,</p><p>EPSG:4326). So, you'll need to turn on projection-on-the-fly to</p><p>make them line up with your reference layers.</p><p>3. Go to the Vector menu or locate the icon on the toolbar for Photo2Shape:</p><p>4. This will ask to you select the directory in which you have the geotagged photos and</p><p>set an output shapefile. (Use the davis-art folder as the input directory.)</p><p>5. You should now get a new shapefile of the point locations of your photos loaded in</p><p>the map. Use Zoom to Layer Extent to zoom in on the locations. You should see a</p><p>camera icon at the location of each photo.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>121</p><p>6. Looking at the attribute table, you can see all the information about the photos pulled</p><p>into the table, including the path to the photos on your computer:</p><p>Going a little further</p><p>If you want to be able to see the actual photos in QGIS and not just the locations, continue</p><p>with the next section of steps:</p><p>1. Enable the eVis plugin.</p><p>2. Once activated go to Database | eVis | eVis Event Browser.</p><p>3. In the new window that pops up, you can see the attributes in the bottom box</p><p>the photo:</p><p>1. If this is blank, go to the Options tab and check whether the correct field is</p><p>selected for the path to the photo, in this case, this is filepath.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Exploration</p><p>122</p><p>2. To make the tool remember this change, check the Remember This box and</p><p>click on the Save button at the bottom:</p><p>How it works…</p><p>In photography, there is a standard metadata format written by most cameras called Exif,</p><p>which is stored as part of the image file format. Normally, all images store the timestamp,</p><p>camera model, camera settings, and other general information about an image. When you</p><p>take a picture with a GPS enabled camera, it should write the latitude and longitude to the</p><p>photo's metadata. Other programs that are metadata-aware can then read this information at</p><p>any time. If you happen to touch up these photos, make sure to tell your software to keep or</p><p>copy the metadata from the original so that you retain the location information.</p><p>There's more…</p><p>Don't have a camera or phone with built-in geotagging? This is not a problem. There are many</p><p>ways to add location information by yourself. One such method is with the Geotag and import</p><p>photos plugin that lets you link photo data to known locations, and this can be found at</p><p>http://hub.qgis.org/projects/geotagphotos/wiki.</p><p>If you need something more sophisticated, there are many other tools out there. Digikam,</p><p>an open source photo management program, includes a geotagging tool that will attempt to</p><p>automatch a GPX file from a GPS to your photos, based on timestamps.</p><p>www.it-ebooks.info</p><p>http://hub.qgis.org/projects/geotagphotos/wiki</p><p>http://www.it-ebooks.info/</p><p>Chapter 4</p><p>123</p><p>Geotagged photos are also supported by many online photos</p><p>services, so you can easily</p><p>browse a map of the photos that you've uploaded. Flickr is probably the most well-known for</p><p>this, and it also includes a concept of geo-fences, where you can exclude certain locations</p><p>from being publicly known.</p><p>On the flip-side, you now have an idea about how to remove geotags from photos in case you</p><p>don't want their locations known if you share them online.</p><p>See also</p><p>f There are other methods of seeing photos in the map besides eVis, including HTML</p><p>map tips. Refer to Nathan's blog at http://nathanw.net/2012/08/05/html-</p><p>map-tips-in-qgis/.</p><p>f More information about geotagging with Digikam can be found at http://</p><p>docs.kde.org/development/en/extragear-graphics/kipi-plugins/</p><p>geolocation.html.</p><p>f You can also use Flickr to geotag and re-export your images, you can or create online</p><p>map mash-ups with their API.</p><p>www.it-ebooks.info</p><p>http://nathanw.net/2012/08/05/html-map-tips-in-qgis/</p><p>http://nathanw.net/2012/08/05/html-map-tips-in-qgis/</p><p>http://docs.kde.org/development/en/extragear-graphics/kipi-plugins/geolocation.html</p><p>http://docs.kde.org/development/en/extragear-graphics/kipi-plugins/geolocation.html</p><p>http://docs.kde.org/development/en/extragear-graphics/kipi-plugins/geolocation.html</p><p>http://www.it-ebooks.info/</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>125</p><p>5</p><p>Classic Vector Analysis</p><p>In this chapter, we will cover the following recipes:</p><p>f Selecting optimum sites</p><p>f Dasymetric mapping</p><p>f Calculating regional statistics</p><p>f Estimating density using heatmaps</p><p>f Estimating values based on samples</p><p>Introduction</p><p>This chapter will provide you with an introduction to some of the most-common GIS analysis</p><p>use cases. The recipes focus on step-by-step instructions, as well as a closer explanation of</p><p>the tools that are used to achieve the desired analysis results. This chapter includes recipes</p><p>on optimum site selection, using interpolation, and creating heat maps, as well as calculating</p><p>regional statistics.</p><p>Selecting optimum sites</p><p>Optimum site selection is a pretty common problem, for example, when planning shop or</p><p>warehouse locations or when looking for a new apartment. In this recipe, you will learn how to</p><p>perform optimum site selection manually using tools from the Processing Toolbox option, but</p><p>you will also see how to automate this workflow by creating a Processing model.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>126</p><p>In the optimum site selection in this recipe, we will combine different vector analysis tools to</p><p>find potential locations in Wake County that match the following criteria:</p><p>f Locations are near a big lake (up to 500 m)</p><p>f Locations are close to an elementary school (up to 500 m)</p><p>f Locations are within a reasonable distance (up to 2 km) from a high school</p><p>f Locations are at least 1 km from a main road</p><p>Getting ready</p><p>To follow this exercise, load the following datasets, lakes.shp, schools_wake.shp, and</p><p>roadsmajor.shp.</p><p>As all datasets in our test data already use the same CRS, we can get right to the analysis. If</p><p>you are using different data, you may have to get all your datasets into the same CRS first. In</p><p>this case, please refer to Chapter 1, Data Input and Output.</p><p>How to do it…</p><p>The following steps show you how to perform optimum site selection using the Processing</p><p>Toolbox option:</p><p>1. First, we have to filter the lakes layer for big lakes. To do this, we use the Select</p><p>by expression tool from the Processing toolbox, select the lakes layer, and enter</p><p>"AREA" > 1000000 AND "FTYPE" = 'LAKE/POND' in the Expression textbox,</p><p>as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>127</p><p>2. Next, we create the buffers that will represent the proximity areas around lakes,</p><p>schools, and roads. Use Fixed distance buffer from the Processing Toolbox</p><p>option to create the following buffers:</p><p>1. For the lakes, select Distance of 500 meters and set Dissolve result by</p><p>checking the box as shown in the following screenshot. By dissolving the</p><p>result, we can make sure that the overlapping buffer areas will be combined</p><p>into one polygon. Otherwise, each buffer will remain as a separate feature</p><p>in the resulting layer:</p><p>It's your choice whether you want to save the</p><p>buffer results permanently by specifying an output</p><p>file, or you just want to work with temporary files</p><p>by leaving the Buffer output file field empty.</p><p>2. To create the elementary school buffers, first select only the schools with</p><p>"GLEVEL" = 'E' using the Select by Expression tool like we did for the</p><p>lakes buffer. Then, use the buffer tool like we just did for the lakes buffer.</p><p>3. Repeat the process for the high schools using "GLEVEL" = 'H' and a</p><p>buffer distance of 2,000 meters.</p><p>4. Finally, for the roads, create a buffer with a distance of 1,000 meters.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>128</p><p>3. With all these buffers ready, we can now combine them to fulfill these rules:</p><p>1. Use the Intersection tool from the Processing Toolbox option on the buffers</p><p>around elementary and high schools to get the areas that are within the</p><p>vicinity of both school types.</p><p>2. Use the Intersection tool on the buffers around the lakes and the result of</p><p>the previous step to limit the results to lakeside areas. Use the Difference</p><p>tool to remove areas around major roads (that is, the buffered road layer)</p><p>from the result of the previous (Intersection) steps.</p><p>4. Check the resulting layer to view the potential sites that fit all the criteria that we</p><p>previously specified. You'll find that there is only one area close to WAKEFIELD</p><p>ELEMENTARY and WAKEFIELD HIGH that fits the bill, as shown in the following</p><p>screenshot:</p><p>How it works…</p><p>In step 1, we used Intersection to model the requirement that our preferred site would be</p><p>near both an elementary and a high school. Later, in step 3, the Difference tool enabled us to</p><p>remove areas close to major roads. The following figure gives us an overview of the available</p><p>vector analysis tools that can be useful for similar analyses. For example, Union could be</p><p>used to model requirements, such as "close to at least an elementary or a high school".</p><p>Symmetrical Difference, on the other hand, would result in "close to an elementary or a high</p><p>school but not both", as illustrated in the following figure:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>129</p><p>Dissolve</p><p>Intersection Symmetrical Difference</p><p>DifferenceUnion</p><p>There's more…</p><p>We were lucky and found a potential site that matched all criteria. Of course, this is not always</p><p>the case, and you will have to try and adjust your criteria to find a matching site. As you can</p><p>imagine, it can be very tedious and time-consuming to repeat these steps again and again</p><p>with different settings. Therefore, it's a good idea to create a Processing model to automate</p><p>this task.</p><p>The model (as shown in the following screenshot) basically contains the same tools that we</p><p>used in the manual process, as follows:</p><p>f Use two select by expression instances to select elementary and high schools. As you</p><p>can see in the following screenshot, we used the descriptions Select "GLEVEL" = 'E'</p><p>and Select "GLEVEL" = 'H' to name these model steps.</p><p>f For elementary schools, compute fixed distance buffers of 500 meters. This step is</p><p>called Buffer "GLEVEL" = 'E'.</p><p>f For high schools, compute fixed distance buffers of 2,000 meters. This step is called</p><p>Buffer "GLEVEL" = 'H'.</p><p>f Select the big lakes using Select by expression (refer to the Select big lakes step)</p><p>and buffer them using fixed distance buffer of 500 meters (refer to the Buffer</p><p>lakes step).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>130</p><p>f Buffer the roads using Fixed distance buffer (refer to the Buffer roads step). The</p><p>buffer size is controlled by the number model input called road_buffer_size. You can</p><p>extend this approach of controlling the model parameters using additional inputs to</p><p>all the other buffer steps</p><p>in this model. (We chose to show only one example in order</p><p>to keep the model screenshot readable.)</p><p>f Use Intersection to get areas near schools (refer to the Intersection: near schools</p><p>step).</p><p>f Use Intersection to get areas near schools and lakes (refer to the Intersection:</p><p>schools and lakes step).</p><p>f Use Difference to remove areas near roads (refer to the Difference: avoid roads</p><p>step).</p><p>This is how the final model looks like:</p><p>You can run this model from the Processing Toolbox option, or you can even use it as a</p><p>building block in other models. It is worth noting that this model produces intermediate results</p><p>in the form of buffer results (near_elementary, near highschool, and so on). While</p><p>these intermediate results are useful while developing and debugging the model, you may</p><p>eventually want to remove them. This can be done by editing the buffer steps and removing</p><p>the Buffer names.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>131</p><p>Dasymetric mapping</p><p>Dasymetric mapping is a technique that is commonly used to improve population distribution</p><p>maps. By default, population is displayed using census data, which is usually available for</p><p>geographic units, such as census tracts whose boundaries don't necessarily reflect the actual</p><p>distribution of the population. To be able to model population distribution better, Dasymetric</p><p>mapping enables us to map population density relative to land use. For example, population</p><p>counts that are organized by census tracts can be more accurately distributed by removing</p><p>unpopulated areas, such as water bodies or vacant land, from the census tract areas.</p><p>In this recipe, we will use data about populated urban areas, as well as data about water</p><p>bodies to refine our census tract population data.</p><p>Getting ready</p><p>To follow this exercise, please load the population data from census_wake2000_pop.shp</p><p>(the file that we created in Chapter 2, Data Management), as well as the urban areas from</p><p>urbanarea.shp, and the lakes from lakes.shp.</p><p>As all the datasets in our sample data already use the same CRS, we can get right into the</p><p>analysis. If you are using different data, you may have to first get all datasets into the same</p><p>CRS. In this case, please refer to Chapter 1, Data Input and Output, for details.</p><p>How to do it…</p><p>To create a new and improved population distribution map, we will first remove the</p><p>unpopulated areas from the census tracts. Then, we will recalculate the population density</p><p>values to reflect the changes to the area geometries by performing the following steps:</p><p>1. Use Clip from the Processing Toolbox option (or Clip by navigating to Vector |</p><p>Geoprocessing tools if you prefer this option—the results will be identical) on the</p><p>census tracts and urban area layers to create a new dataset, containing only those</p><p>parts of the census tracts that are within urban areas.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>132</p><p>2. Refine the results of the previous step further by removing the water bodies (the</p><p>lakes layer) using the Difference tool. The following screenshot shows the results</p><p>of this so far:</p><p>3. Now, we can calculate the population density of the resulting areas, as follows:</p><p>1. Enable editing.</p><p>2. Open Field calculator.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>133</p><p>3. Calculate a new population density (inhabitants per square km) using the</p><p>formula, "_POP2000" / ($area / 1000000):</p><p>4. Deactivate editing and save the changes.</p><p>It is worth noting that you don't necessarily have to make a new</p><p>column. If you only want to use the density values for styling</p><p>purposes, you can also enter the expression directly in the style</p><p>configuration. On the other hand, if you create a new column, you</p><p>can inspect the density values in the attribute table, export them,</p><p>or analyze them further.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>134</p><p>We are done, and you can now visualize the results using a Graduated renderer with, for</p><p>example, the Natural Breaks (Jenks) classification mode. The Jenks Natural Breaks classification</p><p>is designed to arrange values into "natural" classes by maximizing the variance between</p><p>different classes while reducing the variance within the generated classes. The following figure</p><p>shows the population density based on the original census data (on the left) and the results</p><p>after Dasymetric mapping (on the right):</p><p>How it works…</p><p>In the first step of this recipe, we used the Clip operation. As you most likely noticed, the</p><p>results of a Clip operation look very similar to the results of the Intersection tool, which</p><p>we used in the previous recipe of this chapter, Selecting optimum sites. Compare both the</p><p>results, and you will see the following differences:</p><p>f The layer resulting from an Intersection operation contains attributes from both</p><p>input layers, while the result of a Clip operation only contains attributes of the first</p><p>input layer.</p><p>f This also means that the layer order is important when using Clip, but this does not</p><p>change the output of Intersection (except for the attribute order in the attribute table).</p><p>f The Intersection result is also very likely to contain more features than the Clip result</p><p>(164 instead of 105 if you use our sample data census tracts and urban areas). This</p><p>is because the Intersect tool needs to create a new feature for every combination of</p><p>intersecting census tracts and urban areas, while the Clip tool only removes the parts</p><p>of the census tracts that are not within any urban area.</p><p>A popular way of thinking about the Clip operation is to imagine one layer as the cookie cutter</p><p>and the other layer as the cookie dough.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>135</p><p>Calculating regional statistics</p><p>Another classic spatial analysis task is calculating the areas of a certain type within regions,</p><p>for example, the area within a county that is covered by certain land use types, or the share of</p><p>different crops that is farmed in given municipalities.</p><p>In this recipe, we will calculate statistics of geological data for zip code areas. In particular, we</p><p>will calculate the total area of each type of rock per zip code area.</p><p>Getting ready</p><p>To follow this recipe, load zipcodes_wake.shp and geology.shp from our sample data.</p><p>Additionally, install and activate the Group Stats plugin using Plugin Manager.</p><p>How to do it…</p><p>Using the following steps, we can calculate the areas of certain rock types per zip code area:</p><p>1. Calculate the intersections between zip code areas and geological areas using the</p><p>Intersection tool in the Processing Toolbox option or from the Vector menu.</p><p>2. Using the Group Stats plugin, you can now calculate the total area per rock type</p><p>and zip code area, as follows:</p><p>1. Select the Intersection result layer as the input Layer.</p><p>2. Drag the ZIPCODE field to the Rows input area and the GEO_NAME field</p><p>to the Columns input area.</p><p>3. Drag the sum function and the Area value to the Value input area.</p><p>4. Click on Calculate to start the calculations.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>136</p><p>The following screenshot displays the complete configuration, as well as the results:</p><p>How it works…</p><p>The Group Stats plugin brings functionality, which is commonly known as pivot tables, to</p><p>QGIS. A pivot table is a data summarization tool, which is commonly found in applications,</p><p>such as spreadsheets or business intelligence software. As shown in this example, pivot</p><p>tables can aggregate data from an input table. Additionally, the Group Stats plugin offers</p><p>extended geometry functions, such as, Area and Perimeter for polygon input layers, or</p><p>Length for line layers. This makes using the plugin more convenient because it is not</p><p>necessary to first use Field calculator to add these geometric values to the attribute table.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>137</p><p>It is worth noting that you always need to put the</p><p>following two entries into the Value input area:</p><p>f An aggregation function, such as sum, average, or count</p><p>f A value field (from the input layer's attribute table) or geometry function, which should</p><p>be aggregated</p><p>Estimating density heatmaps</p><p>Whether they are animal sightings, accident locations, or general points of interest, many</p><p>point datasets can be interpreted more easily by visualizing the point density using a</p><p>heatmap. In this recipe, we will estimate the density of POIs in Wake county to find areas with</p><p>a high density.</p><p>Getting ready</p><p>Load the poi_names_wake.shp POI dataset from our sample data. Make sure that the</p><p>Heatmap plugin, which comes with QGIS by default, is enabled in Plugin Manager.</p><p>How to do it…</p><p>Using the following steps, we can calculate the POI heatmap:</p><p>1. Start the Heatmap plugin from the Raster menu.</p><p>2. Make sure that poi_names_wake is selected as Input point layer.</p><p>3. Select a location and filename for Output raster. You don't need to specify the file</p><p>extension because this will be added automatically, based on the selected Output</p><p>format. GeoTIFF is usually the first choice.</p><p>4. Select a search Radius of 1000 meters.</p><p>5. The Add generated file to map option should be activated by default. Click on OK to</p><p>create the default heatmap.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>138</p><p>6. By default, the heatmap layer will be rendered using the Singleband gray render</p><p>type. Change the render type to Singleband pseudocolor and apply a color ramp that</p><p>you like to improve the visualization, as show in the following screenshot:</p><p>If you want to control the size of the output raster, just enable the</p><p>Advanced section and adjust the number of Rows and Columns or Cell</p><p>size X and Cell size Y, accordingly. Note that changing rows and columns</p><p>will automatically recalculate the size of the cell and vice versa.</p><p>How it works…</p><p>The search radius, which is also known as the kernel bandwidth, determines how smooth the</p><p>heatmap will look because it sets the distance around each point at which the influence of the</p><p>point will be felt. Therefore, smaller radius values result in heatmaps that display finer details,</p><p>while larger values result in smoother heatmaps.</p><p>Besides the kernel bandwidth, there are also different kernel shapes to choose from. The</p><p>kernel shape controls the rate at which the influence of a point decreases with increasing</p><p>distance from the point. The kernel shapes that are available in the Heatmap plugin can be</p><p>seen in the following figures. For example, a Triweight kernel (the first on the bottom row)</p><p>creates smaller hotspots than the Epanechnikov kernel (the second on the bottom) because</p><p>the Triweight shape gives features a higher influence for distances that are closer to the point:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>139</p><p>The triangular kernel shape can be further adjusted using the Decay ratio setting. In the</p><p>preceding figure, you can see the shape for ratios of 0 (a solid red line), 0.5 (a dashed black</p><p>line), and 1 (a dotted black line), which is equal to the uniform kernel shape. You can even</p><p>specify values greater than 1. In this case, the influence of a feature will increase with the</p><p>distance from the point.</p><p>Estimating values based on samples</p><p>Interpolation is the idea that, with a set of known values, you can estimate the values of</p><p>additional points based on their proximity to these known values. This recipe shows you</p><p>how to use known values at point locations to create a continuous surface (raster) of value</p><p>estimates. Classic examples include weather data estimations that are based on weather</p><p>station data (think temperature or rainfall maps), crop yield estimates that are based on</p><p>sampling parts of a field, and like in this example in this recipe, elevation estimations that</p><p>are based on the elevation of sampled points.</p><p>Getting ready</p><p>Activate Interpolation Plugin via Plugin Manager.</p><p>Load a point layer with numeric columns, representing the feature of interest. For this recipe,</p><p>use the poi_names_wake.shp, and the elev_m column, which contains elevation in meters</p><p>for each point.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>140</p><p>How to do it…</p><p>1. Start by loading poi_name_wake.</p><p>2. Zoom to the layer extent.</p><p>3. Open the Interpolation tool by navigating to Raster | Interpolation | Interpolation.</p><p>Yes, it's on the Raster menu; the source data must be</p><p>a vector, but the results are a raster.</p><p>4. Select poi_names_wake for Input.</p><p>5. Select elev_m for Interpolation attribute.</p><p>6. Click on the Add button, your selection should appear in the box on the left-hand side.</p><p>7. Select Inverse Distance Weighted (IDW) for Interpolation Method.</p><p>8. Now, set the Extent and Cell Size properties. In Cellsize X and Cellsize Y, enter 100</p><p>and 100. This forces the output cells to be 100x100 units of the current projection.</p><p>Generally, if this was for analysis, you would attempt to</p><p>match the region of interest or other raster layers. In this</p><p>case, we just want to go for sensibly-sized cells. As the map</p><p>is in UTM, we will want cells to be integers that represent</p><p>metric units; 100 meters by 100 meters makes interpreting</p><p>the results easier.</p><p>9. Click on the Set to current extent button in the middle.</p><p>10. Next to Output file box, click on the button labeled ... to set the output path to save</p><p>the results:</p><p>11. Pick the folder and type in a name with no file extension, such as idw100m (the result</p><p>will be an ASCII raster .asc file), as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 5</p><p>141</p><p>The wrench tool in the upper-right corner will let you change</p><p>the P value, which is the exponent in the denominator and</p><p>directly sets how much a point influences a nearby location,</p><p>as compared against more distanced points.</p><p>12. Check all your settings and then click on the OK button.</p><p>13. Now, wait patiently for your results, the smaller the size of the cell and the larger the</p><p>number of columns and data points, the longer the calculation will take, as shown in</p><p>the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Classic Vector Analysis</p><p>142</p><p>How it works…</p><p>The basic idea is that, at a given cell, you take the average of all the nearby points that are</p><p>weighted by their distance to the cell in order to estimate the value at your current location.</p><p>Inverse Distance Weighted (IDW) takes this one step further by giving more weight to values</p><p>that are closer to the given cell and less weight to values that are further. This function uses</p><p>an exponent factor P in order to greatly increase the role of closer points over distant points.</p><p>There's more…</p><p>Are the results not quite what you expected? There are a few parameters that can be adjusted;</p><p>these are primarily the P value and the size of the cell. Is this still not coming out the way that</p><p>you want? There are several other Interpolation tools that are accessible in Processing under</p><p>the SAGA, GRASS, and GDAL toolboxes, which allow you to manipulate more of the formula</p><p>parameters to refine the results.</p><p>Finally, depending on your data, IDW may not do a good job of interpolating. In the example</p><p>here, you can actually see how there are distinct circles around isolated points. This is generally</p><p>not a good result, and this needs a smoother transition to nearby points. If you have any control</p><p>over field sampling to begin with, keep in mind that regularly-spaced grids will usually provide</p><p>better results.</p><p>Do you not have control over the source data or you didn't get good results? Then, you may need</p><p>to look into other more complicated formulas that compensate for skew, strong directionality,</p><p>obstructions, and non-regular spacing of samples, such as Splines or Kriging, or Triangulated</p><p>Irregular Networks (TINs). There is lot of science and statistics behind the methods and</p><p>diagnostic tools to determine the best parameters. This is far too complicated a topic for this</p><p>recipe,</p><p>but it is well-covered in books on geostatistics.</p><p>See also</p><p>f http://docs.qgis.org/2.2/en/docs/user_manual/plugins/plugins_</p><p>interpolation.html</p><p>f http://en.wikipedia.org/wiki/Inverse_distance_weightinging</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/plugins/plugins_interpolation.html</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/plugins/plugins_interpolation.html</p><p>http://en.wikipedia.org/wiki/Inverse_distance_weightinging</p><p>http://www.it-ebooks.info/</p><p>143</p><p>Network Analysis</p><p>In this chapter, we will cover the following recipes:</p><p>f Creating a simple routing network</p><p>f Calculating the shortest paths using the Road graph plugin</p><p>f Routing with one-way streets in the Road graph plugin</p><p>f Calculating the shortest paths with the QGIS network analysis library</p><p>f Routing point sequences</p><p>f Automating multiple route computation using batch processing</p><p>f Matching points to the nearest line</p><p>f Creating a network for pgRouting</p><p>f Visualizing the pgRouting results in QGIS</p><p>f Using the pgRoutingLayer plugin for convenience</p><p>f Getting network data from the OSM</p><p>Introduction</p><p>This chapter focuses on the common use cases that are related to routing within networks.</p><p>By far, the most common networks that are used to route are street networks. Other less</p><p>common cases include networks for indoor routing, that is, through rooms inside buildings,</p><p>or networks of shipping routes.</p><p>6</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>144</p><p>Networks and routing are in no way a GIS-only topic. You will find a lot of math literature</p><p>related to this, called Graph Theory. In this chapter, we will use the following terms to talk</p><p>about networks:</p><p>f A network (also known as graph) is a collection of connected objects</p><p>f These objects are called nodes (also known as vertices)</p><p>f The connections between nodes are called links (also known as edges)</p><p>The following figure explains these terms:</p><p>nodes (or vertices)</p><p>edges</p><p>(or links)</p><p>The two routing tools that are commonly used with QGIS are as follows:</p><p>f The Road graph plugin, which is one of the QGIS core plugins; that is, this plugin is</p><p>available in every QGIS installation, but you may have to activate it in Plugin Manager</p><p>f The PostGIS extension pgRouting, which can be used directly through the QGIS DB</p><p>Manager, or more comfortably through the pgRoutingLayer plugin, which can be</p><p>installed from the QGIS plugin repository using Plugin Manager</p><p>Creating a simple routing network</p><p>In this recipe, we will create a routing network from scratch using the QGIS editing tools. Even</p><p>though more and more open network data is available, there will still be numerous use cases</p><p>where necessary network data does not exist or is not available for use. Therefore, it is good to</p><p>know how to create a network and what to pay attention to in order to avoid common pitfalls.</p><p>For the task of network creation, the main difference between the Road graph plugin and</p><p>pgRouting is that pgRouting needs a network node (that is, link start or end node) at each</p><p>intersection while the Road graph plugin will also use intermediate link geometry nodes</p><p>to infer intersections if two links share a node. In this recipe, we will create a network,</p><p>which can be used in both tools.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>145</p><p>Getting ready</p><p>To follow this recipe, you only need a new empty QGIS project. Additionally, make sure you</p><p>have the Digitizing toolbar enabled (as shown in the following screenshot). We will create</p><p>an imaginary network, but if you want you can load a background map and digitize this:</p><p>How to do it…</p><p>Before we can start to create the network, there are a few things that need to be set up first:</p><p>1. Create a new shapefile line layer for the network. You don't need to add any extra</p><p>attributes besides the default ID attribute yet.</p><p>You can read more about creating new shapefiles in the Learning QGIS</p><p>book by Packt Publishing and the QGIS user guide at http://docs.</p><p>qgis.org/2.2/en/docs/user_manual/working_with_</p><p>vector/editing_geometry_attributes.html#creating-</p><p>a-new-shapefile-layer.</p><p>2. To ensure that we can digitize the network with valid topology, we'll activate snapping</p><p>next. Go to Settings | Snapping Options and activate snapping for your line layer</p><p>by enabling the checkbox to the left of it. Additionally, set the mode to to vertex and</p><p>choose a tolerance of at least 5.00000 pixels:</p><p>3. Now, we can enable editing for the line layer and then select the Add Feature tool</p><p>from the Digitizing toolbar to start digitizing.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#creating-a-new-shapefile-layer</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#creating-a-new-shapefile-layer</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#creating-a-new-shapefile-layer</p><p>http://docs.qgis.org/2.2/en/docs/user_manual/working_with_vector/editing_geometry_attributes.html#creating-a-new-shapefile-layer</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>146</p><p>4. Create the first line feature now, and give it the ID number 1. The line can have</p><p>as many nodes as you wish. We'll create a line with four nodes, as shown in the</p><p>following screenshot:</p><p>5. To draw the second line feature, start at the first or the last node of line 1. As we have</p><p>activated snapping, you will see that the node is being highlighted if you hover over it</p><p>with the mouse cursor. Draw a second line and give it the ID number 2.</p><p>The line in the preceding screenshot is drawn with a style that has circles</p><p>on the starting and ending points. You can reproduce this style by adding</p><p>the Marker line symbol levels to the line style or load network_links.</p><p>qml from our sample data. For more details about styling features, please</p><p>refer to Chapter 10, Cartography Tips.</p><p>6. Draw a few more lines (around 12 in total) forming a network. Make sure to pay</p><p>attention to the snapping and assign link IDs:</p><p>7. Disable editing, and confirm that you want to save the changes.</p><p>We will use this basic network as a starting point for the remaining recipes in this chapter.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>147</p><p>How it works…</p><p>By setting the snapping mode to to vertex, we made it possible to digitize the line network in</p><p>a way that ensures that lines, which should be connected, really contain a node at the exact</p><p>same position.</p><p>There's more…</p><p>You can validate the network topology by running the Topology Checker plugin, which is</p><p>installed with QGIS by default (you can read more about Topology Checker in Chapter 12,</p><p>Up and Coming):</p><p>1. Start Topology Checker from the Vector menu.</p><p>2. Click Configure to set up a topology rule, as shown in the following screenshot, and</p><p>click on Add Rule to add it to the list of rules to check. Then, close the settings by</p><p>clicking on OK:</p><p>3. Once this tool is configured, click on Validate All (the button with the checkmark) to</p><p>initiate the check. You will see the list of discovered errors displayed in the list above</p><p>the buttons, as shown in the following screenshot. Additionally, the dangling ends are</p><p>highlighted in red in the map:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>148</p><p>4. You can select the error entries in the list to jump to the line features that failed the</p><p>check. In our network, only lines with dead-ends should be listed. If you see an error</p><p>at an intersection, you should zoom closer and try to correct the node positions.</p><p>Calculating the shortest paths using the</p><p>Road graph plugin</p><p>This recipe shows you how to use the built-in Road graph plugin to calculate the shortest</p><p>paths in a network.</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp from the sample data. Additionally, make sure</p><p>that the Road graph plugin is enabled in Plugin Manager.</p><p>How to do it…</p><p>The Road graph plugin enables us to route between two points that are</p><p>selected on the map.</p><p>Before we can use this, we have to configure it first, as follows:</p><p>1. Enable the Shortest path panel by navigating to View | Panels. This should add the</p><p>plugin panel to the user interface.</p><p>2. Go to Vector | Road graph | Settings to get to the configuration dialog. For now, the</p><p>default settings, as shown in the following screenshot, should be fine. Note that the</p><p>network layer is selected as Transportation layer. Click on OK to confirm the settings:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>149</p><p>3. Once the settings are configured, we can calculate our first route. Select the Start</p><p>and Stop locations using the buttons marked with crosshair icons. Activate the</p><p>crosshair button and then click in the map to select a location. This location will be</p><p>marked on the map, and the coordinates will be automatically inserted into the Start</p><p>or Stop input field.</p><p>4. Click on Calculate to initiate route computation. Depending on the size of the</p><p>network used, this step will either be very fast or it can take much more time. The</p><p>route will be highlighted in the map, and the route length and travel time will be</p><p>displayed, as shown in the following screenshot:</p><p>5. If you want to store the computed shortest path, click on Export and you will be able</p><p>to choose whether you want to create a new layer for the path or add the route to an</p><p>existing line layer.</p><p>6. To compute a new route, simply change the start and stop locations and click on</p><p>Calculate again.</p><p>7. Click on Clear to remove the route highlights when you are done.</p><p>How it works…</p><p>The Road graph plugin uses the QGIS network analysis library, which implements Dijkstra's</p><p>algorithm. For a given starting node, the algorithm finds the path with the lowest cost (that is,</p><p>the shortest path if the cost criterion is length or the fastest path if the cost criterion is time)</p><p>between this node and every other node in the network. This can also be used to find costs of</p><p>the shortest paths from a start node to a destination node by stopping the algorithm once the</p><p>shortest path to the destination node has been determined.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>150</p><p>In contrast to many simple routing tools, the Road graph plugin builds the network topology</p><p>automatically. As our network dataset is topologically sound (that is, there are no tiny</p><p>gaps where network edges meet), we can set up Road graph plugin settings with Topology</p><p>tolerance as 0. If you are using a network from a different source, it may not have been</p><p>created with the same attention to detail, and you may have to increase Topology tolerance</p><p>to get routing to work.</p><p>See also</p><p>f If you are interested in learning more about this algorithm, you can start at</p><p>http://wiki.gis.com/wiki/index.php/Dijkstra's_algorithm</p><p>Routing with one-way streets in the Road</p><p>graph plugin</p><p>When it comes to vehicle routing, it is often necessary to go into more detail and consider</p><p>driving restrictions, such as one-way streets. This recipe shows you how to use one-way street</p><p>information to route with the Road graph plugin.</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp from the sample data. Additionally, make sure</p><p>that the Road graph plugin is enabled in Plugin Manager.</p><p>How to do it…</p><p>To demonstrate routing with one-way street information, we will first visualize the one-way</p><p>values, and then we will configure the Road graph plugin to use the one-way information,</p><p>as follows:</p><p>1. Before we start routing with one-way information, it is helpful to visualize the one-</p><p>way streets. It is worth noting that the one-way direction will depend on the direction</p><p>of the link geometry (that is, the direction the link was digitized in). The best way to</p><p>visualize the link direction is by assigning arrow symbols, as shown in the following</p><p>screenshot. You can load network_pgr.qml from our sample data to get the style:</p><p>www.it-ebooks.info</p><p>http://wiki.gis.com/wiki/index.php/Dijkstra's_algorithm</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>151</p><p>There are many different ways to encode one-way information. In our dataset,</p><p>a forward direction is encoded as FT for "from-to", a backward direction as TF</p><p>for "to-from", and both ways as B for "both".</p><p>2. Then, we can configure the Road graph plugin to use the one-way information. To do</p><p>this, we have to choose the dir attribute as Direction field and enter the values for</p><p>forward (in link geometry) direction and reverse (against link geometry) direction:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>152</p><p>3. Once the plugin is configured, you can compute the shortest path as described in</p><p>the previous recipe, Calculating the shortest paths using the Road graph plugin. You</p><p>will see how the resulting routes differ from the normal (without one-way restrictions)</p><p>paths, as shown in the following screenshot where the algorithm avoids the one-way</p><p>links on the direct route and takes the longer route instead:</p><p>How it works…</p><p>When we use the default two-way setting, each network link is interpreted as a connection</p><p>from the start to end node, as well as a connection from the end to start node. By adding</p><p>one-way restrictions, this changes and the link is only interpreted as one connection now.</p><p>Besides FT, TF, and B, another common way to encode one-ways is 1 for in-link direction,</p><p>-1 for against-link direction, and 0 for both ways. In OpenStreetMap, you will find yes for the</p><p>in-link direction, no for both ways and -1 for the against-link direction (refer to http://wiki.</p><p>openstreetmap.org/wiki/Key:oneway for more details).</p><p>Calculating the shortest paths with the</p><p>QGIS network analysis library</p><p>As mentioned in the recipe, Calculating the shortest paths using the Road graph plugin, QGIS</p><p>comes with a network analysis library, which can be used from the Python console, inside</p><p>plugins, to process scripts, and basically anything else that you can think of. In this recipe,</p><p>we will introduce the usage of the network analysis to compute the shortest paths in the</p><p>Python console.</p><p>www.it-ebooks.info</p><p>http://wiki.openstreetmap.org/wiki/Key:oneway</p><p>http://wiki.openstreetmap.org/wiki/Key:oneway</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>153</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp from the sample data.</p><p>How to do it…</p><p>Instead of typing or copying the following script directly in the Python console, we recommend</p><p>opening the Python console editor using the Show editor button on the left-hand side of the</p><p>Python console:</p><p>1. Paste the following script into the editor:</p><p>import processing</p><p>from processing.tools.vector import VectorWriter</p><p>from PyQt4.QtCore import *</p><p>from qgis.core import *</p><p>from qgis.networkanalysis import *</p><p># create the graph</p><p>layer = processing.getObject('network_pgr')</p><p>director = QgsLineVectorLayerDirector(layer,-1,'','','',3)</p><p>director.addProperter(QgsDistanceArcProperter())</p><p>builder = QgsGraphBuilder(layer.crs())</p><p>from_point = QgsPoint(2.73343,3.00581)</p><p>to_point = QgsPoint(0.483584,2.01487)</p><p>tied_points =</p><p>director.makeGraph(builder,[from_point,to_point])</p><p>graph = builder.graph()</p><p># compute the route from from_id to to_id</p><p>from_id = graph.findVertex(tied_points[0])</p><p>to_id = graph.findVertex(tied_points[1])</p><p>(tree,cost) = QgsGraphAnalyzer.dijkstra(graph,from_id,0)</p><p># assemble the route</p><p>route_points = []</p><p>curPos = to_id</p><p>while (curPos != from_id):</p><p>in_vertex = graph.arc(tree[curPos]).inVertex()</p><p>route_points.append(graph.vertex(in_vertex).point())</p><p>curPos = graph.arc(tree[curPos]).outVertex()</p><p>route_points.append(from_point)</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>154</p><p># write the results to a Shapefile</p><p>result = 'C:\\temp\\route.shp'</p><p>writer = VectorWriter(result,None,[],2,layer.crs())</p><p>fet = QgsFeature()</p><p>fet.setGeometry(QgsGeometry.fromPolyline(route_points))</p><p>writer.addFeature(fet)</p><p>del writer</p><p>processing.load(result)</p><p>2. If you are using your own network dataset instead of network_pgr.shp, which is</p><p>provided with this book, adjust the coordinates of from_point</p><p>and Coming, hints at the future with cutting-edge plugins and how to</p><p>participate in the future development of QGIS.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>vii</p><p>What you need for this book</p><p>We recommend installing QGIS 2.8 or later; you will need at least QGIS 2.4. During the writing</p><p>of this book, several new versions were released, approximately every 4 months, and most</p><p>recently, 2.14 was released. Most of the recipes will work on older versions, but some may</p><p>require 2.6 or newer. In general, if you can, upgrade to the latest stable release or Long Term</p><p>Support (LTS) version.</p><p>There are also a lot of side interactions with other software throughout many of these</p><p>recipes, including—but not limited to—Postgis 2+, GRASS 6.4+, SAGA 2.0.8+, and Spatialite</p><p>4+. On Windows, most of these can be installed using OSGeo4W; on Mac, you may need</p><p>some additional frameworks from Kyngchaos, or if you're familiar with Brew, you can use the</p><p>OSGeo4Mac Tap. For Linux users, in particular Ubuntu and Debian, refer to the UbuntuGIS</p><p>PPA and the DebianGIS blend.</p><p>Does all of this sound a little too complicated? If yes, then consider using a virtual machine</p><p>that runs OSGeo-Live (http://live.osgeo.org). All the software is preinstalled for you</p><p>and is known to work together.</p><p>Lastly, you will need data. For the most part, we've provided a lot of free and open data</p><p>from a variety of sources, including the OSGeo Educational dataset (North Carolina), Natural</p><p>Earth Data, OpenFlights, Wake County, City of Davis, and Armed Conflict Location & Event</p><p>Data Project (ACLED). A full list of our data sources is provided here if you would like</p><p>additional data.</p><p>We recommend that you try methods with the sample data first, only because we tested it.</p><p>Feel free to try using your own data to test many of the recipes; however, just remember that</p><p>you might need to alter the structure to make it work. After all, that's what you'll be working</p><p>with normally.</p><p>The following are the data sources for this book:</p><p>f OSGeo Educational Data: http://grass.osgeo.org/download/sample-data/</p><p>f Wake County, USA: http://www.wakegov.com/gis/services/pages/data.</p><p>aspx</p><p>f Natural Earth Data: http://www.naturalearthdata.com/</p><p>f City of Davis, USA: http://maps.cityofdavis.org/library</p><p>f Stamen Designs: http://stamen.com/</p><p>f Armed Conflict Location & Event Data Project: http://www.acleddata.com/</p><p>www.it-ebooks.info</p><p>http://live.osgeo.org</p><p>http://grass.osgeo.org/download/sample-data/</p><p>http://www.wakegov.com/gis/services/pages/data.aspx</p><p>http://www.wakegov.com/gis/services/pages/data.aspx</p><p>http://www.naturalearthdata.com/</p><p>http://maps.cityofdavis.org/library</p><p>http://stamen.com/</p><p>http://www.acleddata.com/</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>viii</p><p>Who this book is for</p><p>This book is for anyone who wants to do more with QGIS. It's aimed at an intermediate to</p><p>advanced audience that already has some experience using GIS (any). The goal is to master</p><p>the fundamentals of QGIS and launch you, the reader, to the next level of being a QGIS power</p><p>user and community member.</p><p>Whatever your current level of experience with QGIS, you will find a lot of new ways to put your</p><p>skills to good use. For those who are new to GIS, this book shows you many of the things that</p><p>are possible with GIS. For those of you who are new to QGIS, this book is a guide on how to do</p><p>the things you know from elsewhere with QGIS. For advanced users, this book is a reference</p><p>and cheat sheet to the common tools you use often but can't quite remember how they work.</p><p>And for all users, this book is filled with unknown and barely documented aspects of QGIS</p><p>that you didn't think possible but likely want to use.</p><p>Sections</p><p>In this book, you will find several headings that appear frequently (Getting ready, How to do it,</p><p>How it works, There's more, and See also).</p><p>To give clear instructions on how to complete a recipe, we use these sections as follows:</p><p>Getting ready</p><p>This section tells you what to expect in the recipe, and describes how to set up any software or</p><p>any preliminary settings required for the recipe.</p><p>How to do it…</p><p>This section contains the steps required to follow the recipe.</p><p>How it works…</p><p>This section usually consists of a detailed explanation of what happened in the previous</p><p>section.</p><p>There's more…</p><p>This section consists of additional information about the recipe in order to make the reader</p><p>more knowledgeable about the recipe.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>ix</p><p>See also</p><p>This section provides helpful links to other useful information for the recipe.</p><p>Conventions</p><p>In this book, you will find a number of text styles that distinguish between different kinds of</p><p>information. Here are some examples of these styles and an explanation of their meaning.</p><p>Code words in text, database table names, folder names, filenames, file extensions,</p><p>pathnames, dummy URLs, user input, and Twitter handles are shown as follows: "Create a</p><p>Spatialite database if you don't already have one and name it cookbook.db."</p><p>A block of code is set as follows:</p><p>geom,id,elevation</p><p>LINESTRING(0 1, 0 2, 1 3),1,50</p><p>LINESTRING(0 -1, 0 -2, 1 -3),2,60</p><p>LINESTRING(0 1, 0 3, 5 4),3,70</p><p>When we wish to draw your attention to a particular part of a code block, the relevant lines or</p><p>items are set in bold:</p><p>geom,id,elevation</p><p>LINESTRING(0 1, 0 2, 1 3),1,50</p><p>LINESTRING(0 -1, 0 -2, 1 -3),2,60</p><p>LINESTRING(0 1, 0 3, 5 4),3,70</p><p>Any command-line input or output is written as follows:</p><p>-gcp sourceX sourceY destinationX destinationY</p><p>New terms and important words are shown in bold. Words that you see on the screen, for</p><p>example, in menus or dialog boxes, appear in the text like this: "Check the Use visible raster</p><p>layers checkbox or choose SELECT."</p><p>Warnings or important notes appear in a box like this.</p><p>Tips and tricks appear like this.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>x</p><p>Reader feedback</p><p>Feedback from our readers is always welcome. Let us know what you think about this book—</p><p>what you liked or disliked. Reader feedback is important for us as it helps us develop titles</p><p>that you will really get the most out of.</p><p>To send us general feedback, simply e-mail feedback@packtpub.com, and mention the</p><p>book's title in the subject of your message.</p><p>If there is a topic that you have expertise in and you are interested in either writing or</p><p>contributing to a book, see our author guide at www.packtpub.com/authors.</p><p>Customer support</p><p>Now that you are the proud owner of a Packt book, we have a number of things to help you to</p><p>get the most from your purchase.</p><p>Downloading the example code</p><p>You can download the example code files for this book from your account at http://</p><p>www.packtpub.com. If you purchased this book elsewhere, you can visit http://www.</p><p>packtpub.com/support and register to have the files e-mailed directly to you.</p><p>You can download the code files by following these steps:</p><p>1. Log in or register to our website using your e-mail address and password.</p><p>2. Hover the mouse pointer on the SUPPORT tab at the top.</p><p>3. Click on Code Downloads & Errata.</p><p>4. Enter the name of the book in the Search box.</p><p>5. Select the book for which you're looking to download the code files.</p><p>6. Choose from the drop-down menu where you purchased this book from.</p><p>7. Click on Code Download.</p><p>Once the file is downloaded, please make sure that you unzip or extract the folder using the</p><p>latest version of:</p><p>f WinRAR / 7-Zip for Windows</p><p>f Zipeg / iZip / UnRarX for Mac</p><p>f 7-Zip / PeaZip for Linux</p><p>www.it-ebooks.info</p><p>www.packtpub.com/authors</p><p>http://www.packtpub.com</p><p>http://www.packtpub.com</p><p>http://www.packtpub.com/support</p><p>http://www.packtpub.com/support</p><p>http://www.it-ebooks.info/</p><p>Preface</p><p>xi</p><p>Downloading the color images of this book</p><p>We also provide you with a PDF file that has color images of the screenshots/diagrams used</p><p>in this book. The color images will help you better understand the changes in the output.</p><p>You can download this file from http://www.packtpub.com/sites/default/files/</p><p>downloads/QGIS2Cookbook_ColorImages.pdf.</p><p>and to_point for</p><p>the route's starting and ending points.</p><p>3. Change the file paths for the result layer depending on your operating system.</p><p>4. Make sure that the network layer is loaded and selected in the QGIS layer list.</p><p>5. Save the script and run it.</p><p>How it works…</p><p>On line 8, we created a QgsLineVectorLayerDirector object (http://qgis.org/api/</p><p>classQgsLineVectorLayerDirector.html), which contains the network configuration.</p><p>The constructor (QgsLineVectorLayerDirector(layer,-1,'','','',3)) parameters</p><p>are as follows:</p><p>f The network line layer</p><p>f The ID of the direction field: we set it to -1 because this script does not consider</p><p>one-ways</p><p>f The following three parameters are the values for the in link direction: reverse link</p><p>direction, and two-way</p><p>f The last parameter is the default direction: 1 for the in link direction, 2 for the reverse</p><p>direction, and 3 for the two-way</p><p>Line 10 creates the QgsGraphBuilder (http://qgis.org/api/</p><p>classQgsGraphBuilder.html) instance, which will be used to create the routing graph</p><p>on line 14.</p><p>On lines 11 and 12, we defined the starting and ending points of our route. To be able to route</p><p>between these two points, they have to be matched to the nearest network link. This happens</p><p>on line 13 in the makeGraph() function, which returns the so-called tied_points.</p><p>The actual route computation takes place on line 18 in the QgsGraphAnalyzer.</p><p>dijkstra() (http://qgis.org/api/classQgsGraphAnalyzer.html) function.</p><p>www.it-ebooks.info</p><p>http://qgis.org/api/classQgsLineVectorLayerDirector.html</p><p>http://qgis.org/api/classQgsLineVectorLayerDirector.html</p><p>http://qgis.org/api/classQgsGraphBuilder.html</p><p>http://qgis.org/api/classQgsGraphBuilder.html</p><p>http://qgis.org/api/classQgsGraphAnalyzer.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>155</p><p>The while loop, starting on line 22, is where the script moves through the tree created by</p><p>Dijkstra's algorithm to collect all the vertices on the way and add them to the route_points</p><p>list, which becomes the resulting route geometry on line 31.</p><p>The writer for output route line layer is created on line 29, where we pass the file path, None</p><p>for default encoding, the [] for empty fields list, and the 2 for geometry type, which equals to</p><p>lines as well as the resulting layer CRS. The following lines, 30 to 32, create the route feature</p><p>and add it to the writer.</p><p>Finally, the last line loads the resulting shapefile, and this is displayed on the map, as</p><p>illustrated by the following screenshot:</p><p>See also</p><p>You can read more about QGIS's network analysis library online in the PyQGIS Developer</p><p>Cookbook at http://docs.qgis.org/testing/en/docs/pyqgis_developer_</p><p>cookbook/network_analysis.html.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/network_analysis.html</p><p>http://docs.qgis.org/testing/en/docs/pyqgis_developer_cookbook/network_analysis.html</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>156</p><p>Routing point sequences</p><p>In the recipes so far, we routed from one starting point to one destination point. Another use</p><p>case is when we want to compute routes that connect a sequence of points, such as the</p><p>points in a GPS track. In this recipe, we will use the point layer to route processing script to</p><p>compute a route for a point sequence. At its core, this script uses the same idea that was</p><p>introduced in the previous recipe, Calculating the shortest paths with the QGIS network</p><p>analysis library, but this computes several shortest paths one after the other.</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp and sample_pts_for_routing.shp,</p><p>which contains a point layer that should be routed from the sample dataset.</p><p>Additionally, you need to get the point layer to route script from https://raw.</p><p>githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.6/</p><p>scripts/point_layer_to_route.py and save it in the Processing script folder, which</p><p>is set to C:\Users\youruser\qgis2\processing\scripts (on Windows), /home/</p><p>youruser/.qgis2/processing/scripts (on Linux), and /Users/youruser/.qgis2/</p><p>processing/scripts (on Mac) by default. Alternatively, save the point layer to route to the</p><p>folder configured in the Processing menu under Options | Scripts | Scripts folder.</p><p>How to do it…</p><p>To compute the route between the input points, you need to perform the following tasks:</p><p>1. Load the network and the point layer.</p><p>2. If you are using your own data, make sure that both layers are in the same CRS.</p><p>If they are in different CRS, you need to reproject them (for example, using the</p><p>Reproject layer tool from the Processing Toolbox option) before you continue.</p><p>3. Start the point layer to route tool from the Processing Toolbox option.</p><p>4. Pick the points and network input layers, make sure that Open output file</p><p>after running algorithm option is activated, and click on Run to start the route</p><p>computation. The resulting route layer will be loaded automatically:</p><p>www.it-ebooks.info</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.6/scripts/point_layer_to_route.py</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.6/scripts/point_layer_to_route.py</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.6/scripts/point_layer_to_route.py</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>157</p><p>How it works…</p><p>The point layer to route tool uses the QGIS network analysis library. We already discussed the</p><p>basic use of this library in the previous recipe, Calculating the shortest paths with the QGIS</p><p>network analysis library. The main difference is that we now have to handle more than two</p><p>points. Therefore, the script fetches all points from the input point layer and ties or matches</p><p>them to the graph:</p><p>points = []</p><p>features = processing.features(point_layer)</p><p>for f in features:</p><p>points.append(f.geometry().asPoint())</p><p>tiedPoints = director.makeGraph(builder, points)</p><p>For each pair of consecutive points, the script then computes the route between the two</p><p>points just like we did in the Calculating the shortest paths with the QGIS network analysis</p><p>library recipe:</p><p>point_count = point_layer.featureCount()</p><p>for i in range(0,point_count-1):</p><p># compute the route between two consecutive points</p><p>The resulting route line layer contains one line feature for each consecutive point pair.</p><p>There's more…</p><p>Of course, you can also use the point layer to route tool to route between only two points</p><p>as well.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>158</p><p>See also</p><p>There is also a version of this script, which takes one-way information into account at</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/</p><p>master/2.2/scripts/point_layer_to_route_with_oneways.py.</p><p>Automating multiple route computation</p><p>using batch processing</p><p>If you have multiple input point layers, you can use Processing's batch processing capabilities</p><p>to speed up the process. In this recipe, we will compute routes for two input point layers at</p><p>once, but the same approach can be applied to many more layers.</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp and the two point layers, sample_pts_for_</p><p>routing.shp and sample_pts_for_routing2.shp.</p><p>How to do it…</p><p>To get started, right-click on the point layer to route tool in the Processing Toolbox option</p><p>and select Execute as batch process. Then perform the following steps:</p><p>1. In the points column, click the … button and use the Select from open layers option</p><p>to select sample_pts_for_routing and sample_pts_for_routing2.</p><p>2. Select and remove the third line using the Delete row button at the bottom of</p><p>the dialog.</p><p>3. In the network column, click the … button and use the Select from open layers</p><p>option to select network_pgr. You can avoid having to pick the file twice by double-</p><p>clicking on the table header entry (where it reads network). This will autofill all rows</p><p>with the same network file path.</p><p>4. In the routes column, you need to pick a path for the resulting route files. In the Save</p><p>file dialog, which opens when you</p><p>click on the … button, you can specify one base</p><p>filename and click on Save. The following Autofill settings dialog lets you specify if</p><p>and how you want to have the rows filled. Use Autofill mode Fill with numbers and</p><p>Processing will automatically append a running number to the filename that you</p><p>specified. You can see an example in the following screenshot where we specified</p><p>route as the base filename.</p><p>www.it-ebooks.info</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.2/scripts/point_layer_to_route_with_oneways.py</p><p>https://raw.githubusercontent.com/anitagraser/QGIS-Processing-tools/master/2.2/scripts/point_layer_to_route_with_oneways.py</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>159</p><p>5. Click on Run to execute the batch process. Both routes will be computed and loaded</p><p>automatically:</p><p>Matching points to the nearest line</p><p>In this recipe, we will use the QGIS network analysis library from Python console to match</p><p>points to the nearest line. This is the simplest form of what is also known as map matching.</p><p>Getting ready</p><p>To follow this recipe, load network_pgr.shp from the sample data.</p><p>How to do it…</p><p>The following script will match three points, QgsPoint(3.63715,3.60401),</p><p>QgsPoint(3.86250,1.58906), and QgsPoint(0.42913,2.26512), to the network:</p><p>1. Open Python console and its editor and then load or paste the following network_</p><p>analysis_match_points.py script:</p><p>import processing</p><p>from processing.tools.vector import VectorWriter</p><p>from PyQt4.QtCore import *</p><p>from qgis.core import *</p><p>from qgis.networkanalysis import *</p><p>layer = processing.getObject('network_pgr')</p><p>director = QgsLineVectorLayerDirector(layer,-1,'','','',3)</p><p>director.addProperter(QgsDistanceArcProperter())</p><p>builder = QgsGraphBuilder(layer.crs())</p><p>additional_points =</p><p>[QgsPoint(3.63715,3.60401),QgsPoint(3.86250,1.58906),QgsPoi</p><p>nt(0.42913,2.26512)]</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>160</p><p>tied_points = director.makeGraph(builder,additional_points)</p><p>result = 'C:\\temp\\matched_pts.shp'</p><p>writer = VectorWriter(result,None,[],1,layer.crs())</p><p>fet = QgsFeature()</p><p>for pt in tied_points:</p><p>fet.setGeometry(QgsGeometry.fromPoint(pt))</p><p>writer.addFeature(fet)</p><p>del writer</p><p>processing.load(result)</p><p>2. Make sure that the network layer is selected in the layer list.</p><p>3. Run the script. The results should be loaded automatically.</p><p>How it works…</p><p>This script uses the QGIS network analysis library's ability to match points to lines using the</p><p>makeGraph() function. The resulting tied_points list contains the coordinates of the</p><p>points on the network that are closest to the input points.</p><p>The 1 option on line 15 specifies that the output layer is of type point.</p><p>The for loop finally goes through all points in the tied_points list and creates point</p><p>features, which are then added to the result writer.</p><p>Creating a routing network for pgRouting</p><p>This recipe shows you how to import a line layer into PostGIS and create a routable network</p><p>out of it, which can be used by PostGIS's routing library, pgRouting. (For details about</p><p>pgRouting, please visit the project website at http://docs.pgrouting.org.)</p><p>The installation of PostGIS with pgRouting won't be covered in detail here because</p><p>instructions for the different operating systems can be found on the project's website at</p><p>http://docs.pgrouting.org/2.0/en/doc/src/installation/index.html.</p><p>If you are using Windows, both PostGIS and pgRouting can be installed directly from the Stack</p><p>Builder application, which is provided by the standard PostgreSQL installation, as described</p><p>at http://anitagraser.com/2013/07/06/pgrouting-2-0-for-windows-quick-</p><p>guide/.</p><p>www.it-ebooks.info</p><p>http://docs.pgrouting.org</p><p>http://docs.pgrouting.org/2.0/en/doc/src/installation/index.html</p><p>http://anitagraser.com/2013/07/06/pgrouting-2-0-for-windows-quick-guide/</p><p>http://anitagraser.com/2013/07/06/pgrouting-2-0-for-windows-quick-guide/</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>161</p><p>Getting ready</p><p>To follow this exercise, you need a PostGIS database with pgRouting enabled. In QGIS, you</p><p>should set up the connection to the database using the New button in the Add PostGIS</p><p>Layers dialog. Additionally, you should load network_pgr.shp from the sample data.</p><p>How to do it…</p><p>These steps will create a routable network table in your PostGIS database:</p><p>1. Open DB Manager by navigating to Database | DB Manager.</p><p>2. In Tree on the left-hand side of the dialog, select the database that you want to load</p><p>the network to.</p><p>3. Go to Table | Import Layer/File to load the network_pgr layer into your database,</p><p>as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>162</p><p>4. After network_pgr has been imported successfully, open the SQL window of DB</p><p>Manager by pressing F2, clicking on the corresponding toolbar button, or in the</p><p>Database menu.</p><p>5. pgRouting is a little picky when it comes to column data types. You will notice this when</p><p>you see Error, columns 'source', 'target' must be of type int4, 'cost' must be of type</p><p>float8. When we import network_pgr with QGIS's DB Manager, it creates the cost</p><p>column as numeric. As pgRouting won't accept numeric, we will use Table | Edit Table</p><p>in DB Manager to edit the cost column. Click on the Edit column button and change</p><p>Type from numeric to double precision (which equals the required float8).</p><p>6. Now that the data is loaded and ready, we can build the network topology. This will</p><p>create a new network_pgr_vertices_pgr table, which contains the computed</p><p>network nodes:</p><p>SELECT pgr_createTopology('network_pgr',0.001);</p><p>7. Once this topology is ready, we can test the network by calculating a simple shortest</p><p>path from the node number 16 to the node number 9:</p><p>SELECT pgr_dijkstra('SELECT id, source, target, cost</p><p>FROM network_pgr', 16, 9, false, false);</p><p>This will result in the following:</p><p>(0,16,6,1)</p><p>(1,17,7,1)</p><p>(2,5,8,1)</p><p>(3,6,9,1)</p><p>(4,11,15,1)</p><p>(5,9,-1,0)</p><p>How it works…</p><p>The preceding pgr_dijkstra query consists of the following parts:</p><p>f 'SELECT id, source, target, cost FROM network_pgr': This is a SQL</p><p>query, which returns a set of rows with the following columns:</p><p>f id: This is the unique edge ID (type int4)</p><p>f source: This is the ID of the edge source node (type int4)</p><p>f target: This is the ID of the edge target node (type int4)</p><p>f cost: This is the cost of the edge traversal (type float8)</p><p>f 16, 9: These are the IDs of the route source and target nodes (type int4)</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>163</p><p>f false: This is true if the graph is directed</p><p>f false: If true, the reverse_cost column of the SQL-generated set of rows will be</p><p>used for the cost of the traversal of the edge in the opposite direction</p><p>The results of pgr_dijkstra contain the list of network links that our route uses to get from</p><p>the start to the destination. The four values in reach result row are as follows:</p><p>f seq: This is the sequence number, which tells us the order of the links within the</p><p>route starting from 0</p><p>f id1: This is the node ID</p><p>f id2: This is the edge ID</p><p>f cost: This is the cost of the link (can be distance, travel time, a monetary value,</p><p>or any other measure that you chose)</p><p>See also</p><p>In the following recipe, Visualizing pgRouting results in QGIS, we will see how to use the</p><p>results of pgr_dijkstra to visualize the route on a map.</p><p>If you are interested in more pgRouting SQL recipes, you will find a whole chapter on this topic</p><p>in PostGIS Cookbook by Packt Publishing.</p><p>Visualizing the pgRouting results in QGIS</p><p>In the previous recipe, Creating a routing network for pgRouting, we imported a network layer,</p><p>built the topology, and finally tested the routing. Building on these results, this recipe will show</p><p>you how to visualize the routing results on a map in QGIS.</p><p>Getting ready</p><p>You should first go through the previous recipe, Creating a routing network for pgRouting, to</p><p>set up the necessary PostGIS tables. Alternatively, you can use your own network tables, but</p><p>be aware that you may have to alter</p><p>some of the SQL statements if your table uses different</p><p>column names.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>164</p><p>How to do it…</p><p>To visualize the results in QGIS, we can use the DB Manager SQL window, as shown in the</p><p>following screenshot. The extended query that we use here joins the routing results back to the</p><p>original network table to get the route link geometries, which we want to display on the map:</p><p>1. Open DB Manager by navigating to Database | DB Manager.</p><p>2. In Tree to the left of the dialog, select the database that you want to load the</p><p>network to.</p><p>3. Open the SQL window of DB Manager and configure it, as shown in the following</p><p>screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>165</p><p>Note that there must not be a semicolon at the end of the SQL statement.</p><p>Otherwise, loading the results as a new layer will fail.</p><p>4. Make sure that the Geometry column is selected correctly and click on the Load now!</p><p>button to load the query result as a new layer, as shown in the following screenshot:</p><p>How it works…</p><p>As pgr_dijkstra only returns a list with the IDs of the route edges, we need to get the edge</p><p>geometries from the original network table in order to display the route on the map. Therefore,</p><p>we join the routing results with the network table on id2 (which contains the edge ID) and the</p><p>network table's id column.</p><p>See also</p><p>To make using pgRouting from within QGIS more convenient, the pgRoutingLayer plugin</p><p>provides a GUI to access many of pgRouting's functions. You will find an introduction</p><p>to this plugin in the Using the pgRoutingLayer plugin for convenience recipe.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>166</p><p>Using the pgRoutingLayer plugin for</p><p>convenience</p><p>The previous recipe, Visualizing pgRouting results in QGIS, showed you how to manually</p><p>add pgRouting results to the map. In this chapter, we will use the pgRoutingLayer plugin</p><p>to get more convenient access to the functions that pgRouting offers, including the most</p><p>basic algorithms, such as Dijkstra's algorithm, which we have used so far, to more complex</p><p>algorithms, such as drivingDistance and alphashape, which can be used to visualize</p><p>catchment zones, also known as service areas.</p><p>Getting ready</p><p>You should first go through the previous recipe, Creating a routing network for pgRouting, to</p><p>set up the necessary PostGIS tables. Alternatively, you can use your own network tables, but</p><p>be aware that you may have to alter some of the SQL statements if your table uses different</p><p>column names.</p><p>Additionally, install the pgRoutingLayer plugin from Plugin Installer. You will need to enable</p><p>experimental plugins in Settings to view this.</p><p>How to do it…</p><p>The pgRoutingLayer plugin adds a new panel to the QGIS GUI, which allows convenient access</p><p>to the available routing functions. The following steps show you how to use this plugin:</p><p>1. First, you should select a database from the Database field that contains your routing</p><p>network table. The drop-down list contains all the configured PostGIS connections.</p><p>2. Next, you can select a function from the Function field that you want to use.</p><p>Let's try Dijkstra's algorithm first; select the dijkstra function. You will recognize</p><p>the parameters from the previous recipes where we wrote the pgRouting SQL</p><p>query manually.</p><p>3. Specify the parameters for the network table (edge_table) and the geometry, id,</p><p>source, target, and cost columns, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>167</p><p>4. Now, you can use the green + buttons beside the source_id and target_id input</p><p>fields to select the source and target nodes in the map.</p><p>5. When everything is configured, you can click on the Run button to compute and</p><p>display the route.</p><p>6. Next, you can switch functions and compute a service area. Select the alphashape</p><p>function. The rest of the input fields adapt automatically to the selected function.</p><p>7. Now, you can use the green + button right beside the source_id input field to select</p><p>the starting or center node of the service area.</p><p>8. Then, select the size of the service area by specifying the distance limit.</p><p>9. Finally, click on the Run button to compute and display the service area, as shown in</p><p>the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>168</p><p>How it works…</p><p>When we click on the Run button, the query results are visualized as a temporary overlay</p><p>on the map. If you want to save the output permanently, you can click on the Export button.</p><p>Currently, the Export button is only available for the routing functions but not for the service</p><p>area functions.</p><p>See also</p><p>For a detailed documentation on the pgRouting algorithms, refer to the project documentation</p><p>website at http://docs.pgrouting.org/2.0/en/doc/index.html.</p><p>Getting network data from the OSM</p><p>A popular data source for real-world routing applications is OpenStreetMap (OSM). This recipe</p><p>shows you how to prepare OSM data for usage with pgRouting using the osm2po command-</p><p>line tool to convert OSM data to an insert script for PostGIS. Finally, we will test the data</p><p>import using the pgRoutingLayer plugin.</p><p>Getting ready</p><p>Download osm2po from http://osm2po.de and unpack the download. Note that osm2po</p><p>requires Java to be installed on your machine.</p><p>You also need a pgRouting-enabled database to follow this recipe.</p><p>Additionally, you should have the pgRoutingLayer plugin installed and enabled because we will</p><p>use this to test the OSM data import.</p><p>You can use the wake.pbf OSM file from our sample data, or download your own data from</p><p>services such as http://download.geofabrik.de.</p><p>How to do it…</p><p>Open the command line to perform the following steps. If you are working on Windows, we</p><p>recommend using the osgeo4W Shell:</p><p>1. Go to the osm2po folder and open osm2po.config in a text editor. Look for the</p><p>following configuration line and remove the # at the beginning of the line to activate</p><p>the pgRouting export:</p><p>postp.0.class = de.cm.osm2po.plugins.postp.PgRoutingWriter</p><p>www.it-ebooks.info</p><p>http://docs.pgrouting.org/2.0/en/doc/index.html</p><p>http://osm2po.de</p><p>http://download.geofabrik.de</p><p>http://www.it-ebooks.info/</p><p>Chapter 6</p><p>169</p><p>2. Now use osm2po to convert the OSM .pbf file to SQL. Adjust the file paths for your</p><p>system, as follows:</p><p>D:\osm2po-5.1.0>java -jar osm2po-core-5.1.0-signed.jar prefix=wake</p><p>"C:\tmp\OSM_NorthCarolina\wake.pbf"</p><p>3. When osm2po is finished, you should see the following:</p><p>INFO Services started. Waiting for requests at</p><p>http://localhost:8888/Osm2poService</p><p>4. You should now find a folder with the name of the prefix (that is, wake) inside the</p><p>osm2po folder. This contains a log file, which in turn provides a command-line</p><p>template to import the OSM network to PostGIS:</p><p>INFO commandline template:</p><p>psql -U [username] -d [dbname] -q -f</p><p>"D:\osm2po-5.1.0\wake\wake_2po_4pgr.sql"</p><p>5. Using this template, we can easily import the .sql file into an existing database,</p><p>as follows:</p><p>D:\osm2po-5.1.0\wake>psql -U [username] -d cookbook -q -f D:\</p><p>osm2po-5.1.0\wake\wake_2po_4pgr.sql</p><p>6. Now, the data is ready for use in QGIS. When we connect to the cookbook database,</p><p>we can see the wake_2po_4pgr table:</p><p>7. Finally, we can use the pgRouting Layer plugin to test the OSM data import by</p><p>calculating a service area of 0.1 hours (the distance value) around the 43679</p><p>(source_id) source node number:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Network Analysis</p><p>170</p><p>How it works…</p><p>The network table created by osm2po contains, among others, the following useful columns:</p><p>f km: This is the length of the network edge</p><p>f kmh: The is the speed on the edge, depending on the street class and the values</p><p>specified in the osm2po configuration</p><p>f cost: This is the travel time computed using km/kmh</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>171</p><p>7</p><p>Raster Analysis I</p><p>In this chapter, we will cover the following recipes:</p><p>f Using the raster calculator</p><p>f Preparing elevation data</p><p>f Calculating a slope</p><p>f Calculating a hillshade layer</p><p>f Analyzing hydrology</p><p>f Calculating a topographic index</p><p>f Automate analysis tasks using the graphical modeler</p><p>Introduction</p><p>Raster analysis is a classic area in GIS analysis. This chapter shows you some of the most</p><p>important and common tasks of raster analysis. Elevation data is commonly stored as raster</p><p>layers, and in this format, it is particularly suitable to run a large variety of analysis. For this</p><p>reason, terrain analysis has traditionally been one of the main areas of raster analysis, and</p><p>we will show you some of the most common operations that are related to Digital Elevation</p><p>Models (DEM), from simple analysis, such as slope calculation, to more complex ones, such</p><p>as drainage network delineation or watershed extraction.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>172</p><p>Using the raster calculator</p><p>The raster calculator is one of the most flexible and versatile tools in QGIS. This allows you</p><p>to perform algebraic operations based on raster layers, and compute new layers. This recipe</p><p>shows you how to use it.</p><p>Getting ready</p><p>Open the catchment_area.tif file. The file should look like the following screenshot:</p><p>How to do it…</p><p>1. Open the Processing Toolbox option and find the algorithm called Raster calculator</p><p>by searching for it using the search box. Double-click on the algorithm item to execute</p><p>it, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>173</p><p>2. Click on the button in the Input layers field to open the layer selector. There is only</p><p>one layer available: the catchment_area layer. Select this layer.</p><p>3. In the Formula field, enter ln(a).</p><p>4. Click on Run to run the algorithm. The resulting layer will be added to the QGIS</p><p>project, as follows:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>174</p><p>How it works…</p><p>The layers selected in the layer selector are referred to using a single letter in alphabetical</p><p>order (a for the first one, b for the second one, and so on). In this case, we selected just one</p><p>layer, so we can refer to it as a in the formula.</p><p>The formula calculates a natural logarithm of the values in the catchment area layer. The</p><p>distribution of values in this layer is not homogeneous because it contains a large number</p><p>of cells with low values and just a few of them with very large values. This causes the</p><p>rendering of the layer to be not very informative with most of the colors in the color ramp</p><p>not even being used.</p><p>The resulting layer is much more informative because applying the logarithm alters the</p><p>distribution of values, resulting in a more explicit rendering.</p><p>There's more…</p><p>QGIS contains a raster calculator module outside of Processing. You can find this by</p><p>navigating to Raster | Raster calculator...:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>175</p><p>This interface resembles an actual calculator, and it is more intuitive and user friendly. On</p><p>the other hand, this lacks many of the functions that are available in the Processing raster</p><p>calculator (the logarithm that we have computed, for instance, is not available). This also</p><p>cannot be used in automated processes, such as scripts or graphical models, which are</p><p>only available for the Processing algorithms.</p><p>On the other hand, the QGIS built-in calculator supports multiband layers, while the</p><p>Processing one is limited to single-band ones.</p><p>See also</p><p>f The QGIS raster calculator is described in more detail in the QGIS manual at</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_</p><p>raster/raster_calculator.html</p><p>Preparing elevation data</p><p>In this recipe, we will show you how to perform terrain analysis in QGIS. Terrain analysis</p><p>algorithms assume certain characteristics in the DEMs that are used as inputs, so it is</p><p>important to know them and prepare these DEMs if they are needed. This recipe shows</p><p>you how to do this.</p><p>Getting ready</p><p>Open the dem_to_prepare.tif layer. This layer contains a DEM in the EPSG:4326 CRS</p><p>and elevation data in feet. These characteristics are unsuitable to run most terrain analysis</p><p>algorithms, so we will modify this layer to get a suitable one.</p><p>How to do it…</p><p>1. Reproject the layer to the EPSG:3857 CRS, using the Save as... entry in the context</p><p>menu that appears by right-clicking on the layer name.</p><p>2. Open the resulting reprojected layer.</p><p>3. Open the Processing raster calculator and select the reprojected layer as the only</p><p>raster input in the Input layers field. Enter a * 0.3048 in the Formula field. Run</p><p>the algorithm.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_calculator.html</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_raster/raster_calculator.html</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>176</p><p>How it works…</p><p>Most of the algorithms that we are going to use assume that the horizontal units (the unit</p><p>used to measure the size of the cell) are the same as the units used in the elevation values</p><p>that are contained in the layer. If the layer does not meet this requirement, the result of the</p><p>analysis will be wrong.</p><p>Our input layer uses a CRS with geographic coordinates (degrees). As elevation cannot</p><p>be measured in degrees, the layer cannot have the same units for horizontal and vertical</p><p>distances, and it is not ready to be used for terrain analysis.</p><p>By reprojecting the layer to the EPSG:3857 CRS, we get a new layer in which coordinates are</p><p>expressed in meters. This is a unit that is more suitable for the type of analysis that we plan to</p><p>run. Actually, after the reprojection, the units are meters only near the equator, but this gives</p><p>us enough precision for this case. If more precise calculations are needed, a local projection</p><p>system should be used.</p><p>The next step is converting the elevation values in feet to elevation values in meters. Knowing</p><p>that 1 foot = 0.3048 meter, we just have to use the calculator to apply this formula and</p><p>convert the values in the reprojected layer.</p><p>There's more…</p><p>There are other things that must be taken into account when running a terrain analysis</p><p>algorithm to ensure that results are correct.</p><p>One common problem is dealing with different cell sizes. An assumption that is made by most</p><p>terrain analysis algorithms (and also most of the ones not related to terrain analysis) is that</p><p>cells are square. That is, their horizontal and vertical values are the same. This is the case in</p><p>our input layer (you can verify this by checking the layer properties), but it may not be true for</p><p>other layers.</p><p>In this case, you should export the layer and define the sizes of the cells of the exported layer</p><p>to have the same value. Right-click on the layer name and select Save as.... In the save dialog</p><p>that will appear, enter the new sizes of the cells in the lower part of the dialog:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>177</p><p>Calculating a slope</p><p>Slope is one of the most basic parameters that can be derived from a DEM. It corresponds</p><p>to the first derivative of the DEM, and it represents the rate of change of the elevation. It is</p><p>computed by analyzing the elevation of each cell and comparing this with the elevation of</p><p>the surrounding ones. This recipe shows you how to compute slope in QGIS.</p><p>Getting ready</p><p>Open the DEM that we prepared in the previous recipe.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>178</p><p>How to do it…</p><p>1. In the Processing Toolbox option, find the Slope algorithm and double-click on it</p><p>to open it:</p><p>2. Select the DEM in the Input layer field.</p><p>3. Click on Run to run the algorithm.</p><p>The slope layer will be added to the QGIS project.</p><p>How it works…</p><p>Slope is calculated from a DEM elevation model by analyzing the cells around a given one.</p><p>This analysis is performed by the slope algorithm from the GDAL library.</p><p>There's more…</p><p>There are several ways of using the slope algorithms in QGIS. Here are some comments and</p><p>ideas about this.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>179</p><p>Using a ratio for elevation values</p><p>If the units of elevation are not the same as the horizontal units, you can convert them,</p><p>as we did in the previous recipe, using the raster calculator. However, the slope module</p><p>contains an option to convert them on-the-fly by entering the conversion factor in the Scale</p><p>field. Note that this option is not available in other terrain analysis modules that we will use,</p><p>so it's still good practice to create a layer with the correct units, which can be used without</p><p>any further processing.</p><p>Other slope algorithms</p><p>The Processing framework contains algorithms that rely on several external applications and</p><p>libraries. These libraries sometimes contain similar algorithms, so there is more than one</p><p>option for a given analysis.</p><p>If you switch the presentation mode of the toolbox from simplified to advanced using</p><p>the lower part of the drop-down list and then type slope in the search box, you will see</p><p>something like the following screenshot:</p><p>Calculating the slope</p><p>Try using the GRASS or SAGA algorithm to calculate the slope. Each of them has different</p><p>parameters and options, but all of them perform similar calculations and create slope layers.</p><p>Apart from Processing, you can also perform analysis using the Raster Terrain Analysis plugin.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>180</p><p>See also</p><p>f The Using the raster calculator recipe in the beginning of this chapter</p><p>Calculating a hillshade layer</p><p>A hillshade layer is commonly used to enhance the appearance of a map and display</p><p>topography in an intuitive way, by simulating a light source and the shadows it casts.</p><p>This can be computed from a DEM by using this recipe.</p><p>Getting ready</p><p>Open the DEM that we prepared in the Preparing elevation data recipe.</p><p>How to do it…</p><p>1. In the Processing Toolbox option, find the Hillshade algorithm and double-click on it</p><p>to open it:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>181</p><p>2. Select the DEM in the Input layer field. Leave the rest of the parameters with their</p><p>default values.</p><p>3. Click on Run to run the algorithm.</p><p>The hillshade layer will be added to the QGIS project, as follows:</p><p>How it works…</p><p>As in the case of the slope, the algorithm is part of the GDAL library. You will see that the</p><p>parameters are very similar to the slope case. This is because slope is used to compute the</p><p>hillshade layer. Based on the slope and the aspect of the terrain in each cell and using the</p><p>position of the sun that is defined by the Azimuth and Altitude fields, the algorithm computes</p><p>the illumination that the cell will receive. This is based on a focal analysis, so shadows are not</p><p>considered and are not a real illumination value, but they can be used to render and to display</p><p>the topography of the terrain.</p><p>You can try changing the values of these parameters to alter the appearance of the layer.</p><p>There's more…</p><p>As in the case of slope, there are alternative options to compute the hillshade. The SAGA one</p><p>in the Processing Toolbox option has a feature that is worth mentioning.</p><p>The SAGA hillshade algorithm contains a field named method. This field is used to select the</p><p>method that is used to compute the hillshade value, and the last method that is available.</p><p>Raytracing differs from the other ones as it models the real behavior of light, making an</p><p>analysis that is not local but that uses the full information of the DEM instead because it</p><p>takes into account the shadows that are cast by the surrounding relief. This renders more</p><p>precise hillshade layers, but the processing time can be notably larger.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>182</p><p>Enhancing your map view with a hillshade layer</p><p>You can combine the hillshade layer with your other layers to enhance their appearance.</p><p>As you used a DEM to compute the hillshade layer, it should be already in your QGIS project</p><p>along with the hillshade itself. However, this will be covered by the hillshade because of the</p><p>new layers produced by Processing are added on top of the existing ones in the layers list.</p><p>Move it to the top of the layer list so that you can see the DEM (and not the hillshade layer)</p><p>and style it to something like the following screenshot:</p><p>Lets see the steps to enhance the map view with a hillshade layer:</p><p>1. In the Properties dialog of the layer, move to the Transparency section, and set the</p><p>Global transparency value to 50%, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>183</p><p>2. Now, you should see the hillshade layer through the DEM, and the combination of</p><p>both of them will look like the following screenshot:</p><p>Another way of doing this is using the blending modes in QGIS. You can find more information</p><p>about this in the recipe, Understanding the feature and layer blending modes of Chapter 10,</p><p>Cartography Tips, or in the QGIS manual at http://docs.qgis.org/2.8/en/docs/</p><p>user_manual/working_with_vector/vector_properties.html#style-menu.</p><p>Analyzing hydrology</p><p>A common analysis from a DEM is to compute hydrological elements, such as the channel</p><p>network or the set of watersheds. This recipe shows you the steps to do these analysis.</p><p>Getting ready</p><p>Open the DEM that we prepared in the Preparing elevation data recipe.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/vector_properties.html#style-menu</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/vector_properties.html#style-menu</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>184</p><p>How to do it…</p><p>1. In the Processing Toolbox option, find the Fill Sinks algorithm and double-click on it</p><p>to open it:</p><p>2. Select the DEM in the DEM field and run the algorithm. This will generate a new</p><p>filtered DEM layer. From now on, we will just use this DEM in the recipe and not</p><p>the original one.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>185</p><p>3. Open Catchment Area and select the filtered DEM in the Elevation field:</p><p>4. Run the algorithm. This will generate a catchment area layer:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>186</p><p>5. Open the Channel network algorithm and fill it in, as shown in the following</p><p>screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>187</p><p>6. Run the algorithm. This will extract the channel network from the DEM, based on the</p><p>catchment area, and it will then generate it as both a raster and vector layer:</p><p>7. Open the Watershed basins algorithm and fill it in, as shown in the following</p><p>screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>188</p><p>8. Run the algorithm. This will generate a raster layer with the watersheds calculated</p><p>from the DEM and the channel network. Each watershed is a hydrological unit that</p><p>represents the area that flows into a junction, which is defined by the channel network:</p><p>How it works…</p><p>Starting from the DEM, the preceding steps follow a typical workflow for hydrological analysis:</p><p>f First, the sinks are filled. This is a required preparation whenever you plan to perform</p><p>a hydrological analysis. The DEM may contain sinks where a flow direction cannot be</p><p>computed, which represents a problem to model the movement of water across these</p><p>cells. Removing these sinks solves this problem.</p><p>f The catchment area is computed from the DEM. The values in the catchment area</p><p>layer represent the area that is upstream of each cell. That is, the total area in which</p><p>if water is dropped, it will eventually pass through the cell.</p><p>f Cells with high values of the catchment area will likely contain a river, while cells</p><p>with lower values will have overland flow. By setting a threshold on the catchment</p><p>area values, we can separate the river cells (the ones above the threshold) from</p><p>the remaining ones and extract the channel network.</p><p>f Finally, we compute the watersheds associated with each junction in the channel</p><p>network that was extracted</p><p>in the last step.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>189</p><p>There's more…</p><p>The key parameter in the preceding workflow is the catchment area threshold. If a larger</p><p>threshold is used, fewer cells will be considered as river cells, and the resulting channel</p><p>network will be sparser. As the watershed is computed based on the channel network,</p><p>this will result in a lower number of watersheds.</p><p>You can try this yourself with different values of the catchment area threshold. Here, you can</p><p>see the result for threshold is equal to 1,000,000 in the following screenshot:</p><p>The channel network has been added to help you understand the structure of the resulting set</p><p>of watersheds.</p><p>Here, you can see the result for a threshold of 50,000,000 in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>190</p><p>Note that in this last case, with a higher threshold value, there is only one single watershed in</p><p>the resulting layer.</p><p>The threshold values are expressed in the units of the catchment area which, as the size of</p><p>the cell is assumed to be in meters, are in square meters.</p><p>Calculating a topographic index</p><p>As the topography defines and influences most of the processes that take place in a given</p><p>terrain, the DEM can be used to extract many different parameters, which give us information</p><p>about these processes. This recipe shows you how to calculate a popular one, which is called</p><p>the Topographic wetness index, which estimates the soil wetness based on the topography.</p><p>Getting ready</p><p>Open the DEM that we prepared in the Preparing elevation data recipe.</p><p>How to do it…</p><p>1. Calculate a slope layer using the Slope, aspect, curvature algorithm from SAGA in the</p><p>Processing Toolbox option. Calculate a catchment area layer using the Catchment</p><p>area algorithm from the Processing Toolbox option. Note that you must use a</p><p>sink-less DEM, such as the one that we generated in the previous recipe with</p><p>the Fill sinks algorithm.</p><p>Open the Topographic wetness index algorithm from the Processing Toolbox option</p><p>and fill it in, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>191</p><p>2. Run the algorithm. This will create a layer with the topographic wetness index,</p><p>indicating the soil wetness in each cell:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>192</p><p>How it works…</p><p>The index combines slope and catchment area, two parameters that influence the soil</p><p>wetness. If the catchment area value is high, this means that more water will flow into the cell,</p><p>thus, increasing its soil wetness. A low value of slope will have a similar effect because the</p><p>water that flows into the cell will not flow out of it quickly.</p><p>This algorithm expects the slope to be expressed in radians. This is the reason why the Slope,</p><p>aspect, curvature algorithm has to be used because it produces its slope output in radians.</p><p>The other Slope algorithm that you will find, which is based on the GDAL library, creates a</p><p>slope layer with values expressed in degrees. You can use this layer if you convert its units</p><p>using the raster calculator.</p><p>There's more…</p><p>Other indices that are based on the same input layers can be found in different algorithms</p><p>in the Processing Toolbox option. The Stream Power Index and the LS factor both use the</p><p>slope and catchment area as inputs as well, and they can be related to potential erosion.</p><p>Automating analysis tasks using the</p><p>graphical modeler</p><p>Most analysis tasks involve using several algorithms. Repeating the same analysis with a</p><p>different dataset or different input parameters requires using them one by one, making this</p><p>task tedious and error-prone. You can automate analysis workflows using the Processing</p><p>graphical modeler, which allows you to define a workflow graphically and wrap it in a single</p><p>algorithm. This recipe introduces the main ideas about the modeler and creates a simple</p><p>model as an example.</p><p>Getting ready</p><p>No special preparation is needed in QGIS for this recipe, but make sure that you have read the</p><p>previous recipe about computing a topographic index. This recipe will create a model based</p><p>on the workflow in that recipe, so it is important that you understand it.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>193</p><p>How to do it…</p><p>1. Open the graphical modeler by navigating to Processing | Graphical modeler:</p><p>2. Double-click on the Raster Layer item to add a raster input. In the dialog that will</p><p>appear to define the input, name it DEM and set it as mandatory:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>194</p><p>3. Click on OK to add the input to the canvas:</p><p>4. Move to the Algorithms tab. Double-click on the Slope, aspect, curvature algorithm</p><p>and set the algorithm definition, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>195</p><p>5. Close the dialog by clicking on the OK button. This will be added to the modeler</p><p>canvas, as follows:</p><p>6. Add the Catchment area algorithm to the model by double-clicking on it in the</p><p>algorithm list and filling in the dialog, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis I</p><p>196</p><p>7. Finally, add the Topographic wetness index algorithm, defining it as shown in the</p><p>following screenshot:</p><p>8. The final model should look like the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 7</p><p>197</p><p>9. Enter a name and a group to identify the model and save it by clicking on the Save</p><p>button. Do not change the save location folder, because Processing will only look</p><p>for it in the default location, you can however change the name of the model. Close</p><p>the modeler dialog. If you now go to the Processing Toolbox option, you will find a</p><p>new algorithm in the Models section, which corresponds to the workflow that you</p><p>have just defined:</p><p>How it works…</p><p>The model automates the workflow and wraps all the steps into a single one.</p><p>By saving the model in the models folder, Processing will see this when updating the toolbox</p><p>and will include it along with the rest of algorithms so that it can be executed normally.</p><p>See also</p><p>f More information about the graphical modeler can be found in the Processing</p><p>chapter of the QGIS manual at http://docs.qgis.org/2.8/en/docs/user_</p><p>manual/processing/modeler.html</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/processing/modeler.html</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/processing/modeler.html</p><p>http://www.it-ebooks.info/</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>199</p><p>8</p><p>Raster Analysis II</p><p>In this chapter, we will cover the following recipes:</p><p>f Calculating NDVI</p><p>f Handling null values</p><p>f Setting extents with masks</p><p>f Sampling a raster layer</p><p>f Visualizing multispectral layers</p><p>f Modifying and reclassifying values in raster layers</p><p>f Performing supervised classification of raster layers</p><p>Introduction</p><p>Following the previous chapter, this chapter introduces some additional techniques for raster</p><p>analysis. This chapter will show you how to work with images, how to modify raster values and</p><p>classify them, and how raster layers can be used along with vector layers, thus extending the</p><p>set of tools that were introduced in the recipes in the previous chapter.</p><p>Calculating NDVI</p><p>The Normalized Differential Vegetation Index is a very popular vegetation index that gives us</p><p>useful information about the presence or absence of live green vegetation.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>200</p><p>Getting ready</p><p>NDVI is calculated using a band with red spectral reflectance values, and another one with</p><p>near-infrared reflectance values. In the sample dataset, you will find two image files named</p><p>red.tif and nir.tif that can be used to compute NDVI. A project named ndvi.qgs is</p><p>available, which contains these two layers and a landsat image corresponding to this same</p><p>area. Open this project.</p><p>How to do it…</p><p>1. Open the Processing</p><p>Toolbox menu and find the algorithm called Vegetation</p><p>index[slope based]. Double-click on the algorithm item to execute it:</p><p>2. Select the red.tif layer in the Red Band field and the nir.tif layer in the Near</p><p>Infrared Band field. Click on Run to run the algorithm.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>201</p><p>3. The algorithm will produce a set of layers with different vegetation indices, NDVI is</p><p>among them:</p><p>How it works…</p><p>All vegetation indices that are computed by the algorithm are based on the relation between</p><p>red and near-infrared reflectances. Leaf cells scatter solar radiation in the near-infrared</p><p>reflectance and absorb radiation in the red reflectance, which can be used to predict the</p><p>location of healthy green vegetation based on these two values.</p><p>NDVI is computed with the formula given in the following section.</p><p>There's more…</p><p>As the formula of the NDVI is rather simple, you can calculate it without using a specific</p><p>algorithm, just by going to the raster calculator. You can use the one integrated in the</p><p>Processing Framework or the QGIS built-in on. You can see how you should fill the parameters</p><p>in the QGIS Raster calculator to compute the NDVI, based on the two proposed sample layers</p><p>in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>202</p><p>Extracting bands</p><p>The vegetation indices algorithm requires the red and infrared values to be in two separate</p><p>layers, each of them with a single band. However, it's common to have both of them in a</p><p>multiband image. To be able to use these bands, you must separate them, extracting them</p><p>into two separate files.</p><p>This can be done using the GDAL translate algorithm. The project contains a multiband</p><p>image named landsat.tif with the red band in band number 3 and infrared band in</p><p>band number 4:</p><p>1. Open the Translate algorithm in the Processing Toolbox menu.</p><p>2. Fill its parameters, as shown in the following screenshot, to export the infrared band:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>203</p><p>3. Run this again, as shown in the following screenshot, to export the red band:</p><p>The Translate algorithm uses the GDAL library underneath. You can also use this library as an</p><p>independent tool from the console. At the lower part of the algorithm dialog, you will find a text</p><p>field where you will see the equivalent console call to your current algorithm configuration.</p><p>Handling null values</p><p>Null values are a particular type of values that are used to indicate cells where the value for a</p><p>given layer is not defined. Understanding how to use them is important to avoid wrong results</p><p>when performing analyses but also to use them as a tool to get better and more correct</p><p>results. This recipe explains some of the fundamental ideas about null values in raster layers.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>204</p><p>How to do it…</p><p>The watershed.tif layer contains the area of a watershed. Cells inside the watershed</p><p>are cells from which water will eventually flow into the outlet point of the watershed. The</p><p>remaining cells belong to a different watershed. To mask the DEM with the watershed mask,</p><p>follow these steps:</p><p>1. Open the watershed.tif layer.</p><p>2. Open the identify tool and check whether the cells that belong to the watershed have</p><p>a value of 1, and the ones outside, have a value of no data.</p><p>3. Try clicking inside and outside the watershed; in your Identify Results dialog, you will</p><p>see the results, as shown in the following screenshot:</p><p>4. Now, let's calculate some statistics of the raster layer. Open the Raster layer</p><p>statistics algorithm in the Processing Toolbox menu.</p><p>5. Select the watershed layer in the input layer field and click on OK to run the</p><p>algorithm. The result is a short text output that looks like the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>205</p><p>Only the cells with a value of 1 have been considered, and the average value in the layer is</p><p>equal to 1.</p><p>The layer has 610 columns and 401 rows, but the total number of valid cells is much lower</p><p>than 610 x 410. These are the cells that have been used to compute the statistics.</p><p>How it works…</p><p>Raster layers always cover a rectangular region. However, in some circumstances, the land</p><p>object that the layer represents might not be rectangular. This might be due to a purely</p><p>geophysical reason (imagine a layer with water temperature that contains non-water cells),</p><p>political ones (a layer with a DEM of a given country with no data available for a neighboring</p><p>country), or many others. In any case, a value is needed for these cells to indicate that no data</p><p>is available. An arbitrary value is selected and used. As such, this is usually a value that is not</p><p>a logical and/or feasible value for the variable that is stored in the layer.</p><p>In the case of the example layer, the value used is -99999, which is the default value set for</p><p>no-data values. This means that, when the identify tool shows no data, it has actually selected</p><p>a value of -99999 in this case.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>206</p><p>Algorithms in the Processing framework systematically ignore no-data cells, and do not use</p><p>their values. You can clearly see this in the preceding example. A large part of the cells in</p><p>the layer have a value of 1 (the ones that belong to the watershed), but many of them have</p><p>a value of -99999. The average value of the cells should then be different from 1, but as</p><p>-99999 is defined as the no-data value, all cells with this value are ignored. The average</p><p>of the layer is, therefore, equal to 1.</p><p>There's more…</p><p>Null values should be considered not only when performing an analysis, but also when we just</p><p>want to render a layer that contains them.</p><p>Controlling the rendering of null values</p><p>Null values are also considered separately when rendering a raster layer. You can choose to</p><p>select them using a given color (as set by the current color palette), or to not render them</p><p>at all. To make all cells with null values transparent, open the layer properties and go to the</p><p>Transparency section. Make sure that the No data value checkbox is checked, as shown in</p><p>the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>207</p><p>Setting extents with masks</p><p>The extent of a layer can be set using a second layer, which acts as a mask. This recipe shows</p><p>you how to do this.</p><p>How to do it…</p><p>To mask the DEM with the watershed mask, follow these steps:</p><p>1. Open the watershed.tif layer and the dem.tif layer.</p><p>2. Open the Raster Calculator algorithm present in the Processing Toolbox menu.</p><p>3. In Main input layer, select the DEM, and in the Additional layers field, select the</p><p>watershed layer.</p><p>4. In the Formula field, enter the formula, a*b.</p><p>5. Click on Run to run the algorithm. You will get a masked DEM, as follows:</p><p>How it works…</p><p>When using the raster calculator, all operations involving a no-data value will result in another</p><p>no-data value. This means that, when multiplying the DEM layer and the mask layer, in the</p><p>cells that contain no-data values in the mask layer, the value in the resulting layer will be</p><p>a no-data value, no matter which elevation value is found in the DEM layer for this cell.</p><p>As cells inside the watershed in the mask layer have a value of 1, the result is a layer with</p><p>elevation values for watershed cells and no-data values for the remaining ones.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>208</p><p>There's more…</p><p>Here are some additional ideas about masks.</p><p>Restricting analysis to a given area</p><p>Once we have masked the area of interest (in this case, the watershed), all analysis that</p><p>we perform will be restricted to this. For instance, let's calculate the average elevation of</p><p>the watershed:</p><p>1. Open the Basic statistics for raster layers algorithm.</p><p>2. Select the masked DEM in the Input layer field and click on Run to run the algorithm.</p><p>You will get the statistics on the</p><p>Results window:</p><p>These values have been computed using only valid cell values and ignoring the no-data ones,</p><p>which means that they refer to the watershed and not the the full extent of the raster layer.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>209</p><p>Removing superfluous no-data values</p><p>Sometimes, you might have more no-data values that are needed in a raster layer, as in the</p><p>case of the proposed watershed layer. To reduce the extent of the layer and just have the</p><p>minimum extent that covers the valid data, you can use the Crop to data algorithm:</p><p>1. Open the Crop to data algorithm in the Processing Toolbox menu.</p><p>2. Enter the masked DEM in the Input layer field and click on Run to run the algorithm.</p><p>The resulting layer should look like this when you disable transparency for no-data</p><p>values:</p><p>Note that, if you have opted to render no-data cells as transparent pixels, you</p><p>will see no visual difference between the original and the cropped layer.</p><p>Masking using a vector mask</p><p>Masking a raster layer can also be done using a polygon vector layer. The watershed.shp</p><p>file contains a single polygon with the area of the watershed that we have already used to</p><p>mask the DEM. Here is how to use this to mask that DEM without using the raster mask:</p><p>1. Open the Clip grid with the polygon algorithm.</p><p>2. Select the DEM in the Input field.</p><p>3. Select the vector layer in the Polygons field.</p><p>4. Click on Run to run the algorithm. The clipped layer will be added to the QGIS project.</p><p>In this case, the clip algorithm automatically reduces the extent of the output layer to the</p><p>minimum extent defined by the polygon layer, so there is no need to run the Crop to data</p><p>algorithm afterwards.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>210</p><p>Sampling a raster layer</p><p>Data from a raster layer can be added to a points layer by querying the value of the layer in</p><p>the coordinates of the points. This process is known as sampling, and this recipe explains</p><p>how to perform it.</p><p>Getting ready</p><p>Open the dem.tif raster layer and the dem_points.shp vector layer:</p><p>How to do it…</p><p>1. In the Processing Toolbox menu, find the Add grid values to points algorithm and</p><p>double-click on it to open it:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>211</p><p>2. Select the DEM in the Grids field.</p><p>3. Select the point layer in the Points field.</p><p>4. Click on Run to run the algorithm.</p><p>A new vector layer will be created. This contains the same points as the input layer, but the</p><p>attribute table will have an additional field with the name of the selected raster layer and</p><p>the values corresponding to this layer in each point:</p><p>How it works…</p><p>The coordinates of the points are taken, and the value of the pixel in which the layer falls is</p><p>added to the resulting points layer.</p><p>This method assumes that the value of a cell is constant in all the area covered by this cell. A</p><p>different approach is to consider that the value of the cell represents its value only in the center</p><p>of the cell and perform additional calculations to compute the value at the exact sampling</p><p>point using the values of the surrounding cells as well. This can be done using several different</p><p>interpolation methods, which can be selected in the Interpolation method selector, changing</p><p>the default value, which only uses the value of the cell where the sampling point falls.</p><p>Layers are assumed to be in the same CRS and no reprojection is performed. If this is not the</p><p>case, the value added to the vector layer might not be correct.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>212</p><p>There's more…</p><p>Here, you can find some ideas about how to combine a raster and vector layer in different</p><p>situations.</p><p>Other raster-vector data transfer operations</p><p>Data coming from a raster layer can also be added to other types of vector layers. In the</p><p>case of a vector layer with polygons, the Grid statistics for polygons algorithm can be used,</p><p>as follows:</p><p>1. Open the watershed.shp file that we used in the previous recipe.</p><p>2. Open the Grid statistics in the Polygons field.</p><p>3. Select the raster layer to clip in the Grids field.</p><p>4. Select the polygon layer with the mask in the Polygons field.</p><p>5. Select the statistics to be calculated from the remaining parameters. For instance,</p><p>to calculate just the mean elevation, leave the Mean field selected and unselect</p><p>the others.</p><p>6. Click on Run to run the algorithm.</p><p>The resulting layer is a new polygon layer with the watershed and an additional field in the</p><p>attributes table, containing the mean elevation value for each polygon.</p><p>If more statistics are selected, the result will have a larger number of additional fields added,</p><p>one for each new parameter computed and each selected grid.</p><p>Visualizing multispectral layers</p><p>Multispectral layers can be rendered in different ways depending on how bands are used. This</p><p>recipe shows you how to do this and discusses the theory behind it.</p><p>Getting ready</p><p>Open the landsat.qgs project.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>213</p><p>How to do it…</p><p>1. The Landsat image, when opened with the default configuration, looks something like</p><p>the following screenshot:</p><p>2. Double-click on the layer to open its properties and move to the Style section:</p><p>1. Select the band number 4 in the Red band field.</p><p>2. Select the band number 3 in the Green band field.</p><p>3. Select the band number 2 in the Blue band field.</p><p>Your style configuration should be like the following:</p><p>3. Click on OK.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>214</p><p>The image should now look like the following:</p><p>How it works…</p><p>Colors representing a given pixel are defined using the RGB color space, which requires three</p><p>different components. A normal image (such as the one you will get from a digital camera)</p><p>has three bands containing the intensity for each one of these three components: red, green,</p><p>and blue.</p><p>Multispectral bands, such as the one used in this recipe, have more than three bands and</p><p>provide more detail in different regions of the electromagnetic spectrum. To visualize these,</p><p>three bands from the total number of available bands have to be chosen and their intensities</p><p>have to be used as intensities of the basic red, green, and blue components (although they</p><p>might correspond to a different region of the spectrum, even outside the visible range). This is</p><p>known as a false color image.</p><p>Depending on the combination of the bands that are used, the resulting image will convey</p><p>a different type of information. The combination chosen is frequently used for vegetation</p><p>studies, as it allows you to separate coniferous from hardwood vegetation as well as</p><p>providing information about vegetation health.</p><p>The combination is applied, in this case, to a Landsat 7 image, which is taken with the</p><p>ETM+ sensor. The wavelengths covered by each band are as follows (in micrometers):</p><p>f Band 1: 0.45 - 0.515</p><p>f Band 2: 0.525 - 0.605</p><p>f Band 3: 0.63 - 0.69</p><p>f Band 4: 0.75 - 0.90</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 8</p><p>215</p><p>f Band 5: 1.55 - 1.75</p><p>f Band 6: 10.40 - 12.5</p><p>f Band 7: 2.09 - 2.35</p><p>There's more…</p><p>Different combinations are frequently used for Landsat layers. One of them is the following:</p><p>f Select the band number 3 in the Red band field</p><p>f Select the band number 2 in the Green band field</p><p>f Select band number 1 in the Blue band field</p><p>This is a natural color combination, as the bands used for the R, G, and B components</p><p>actually have the wavelengths corresponding to the colors red, green, and blue:</p><p>If you are using an image that is not a Landsat 7 one, each band will have a different</p><p>meaning, and using the same combination of band numbers will yield different results. The</p><p>meaning of each band must be checked in order to understand the information displayed by</p><p>the rendered image.</p><p>See also</p><p>f Landsat data is freely available. If you want to download Landsat data corresponding</p><p>to a given region, visit http://landsat.gsfc.nasa.gov/.</p><p>Here, you can find</p><p>more information about where and how you can download it.</p><p>www.it-ebooks.info</p><p>http://landsat.gsfc.nasa.gov/</p><p>http://www.it-ebooks.info/</p><p>Raster Analysis II</p><p>216</p><p>Modifying and reclassifying values in raster</p><p>layers</p><p>A very useful technique to work with raster data is changing their values or grouping them into</p><p>categories. In this recipe, we will see how to do this.</p><p>Getting ready</p><p>Open the DEM file that we used in previous recipes.</p><p>How to do it…</p><p>We will classify the elevation in three groups:</p><p>f Lower than 1,000m</p><p>f Between 1,000 and 2,000m</p><p>f Higher than 2,000m</p><p>To do this, follow these steps:</p><p>1. Open the Change grid values algorithm from the Processing Toolbox menu. Set the</p><p>Replace condition parameter to Low Value</p><p>for geographic data and a few methods to put your maps online.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>224</p><p>Using web services</p><p>There are quite a few different types of web-based map service that can be loaded in QGIS.</p><p>Each type of web service provides data; often, this is the same data in different ways. This</p><p>recipe is about helping you figure out what type of web service you want to consume, and</p><p>conversely what type of web service you may want to create for others to use.</p><p>Getting ready</p><p>This recipe is all about thinking, so you don't need anything in particular to start. It does help</p><p>if you have a project in mind and some type of data you are interested in using or creating.</p><p>Most, if not all, of these methods require an Internet connection or a local server, providing</p><p>these services. Of course, if you have the data locally, you should probably just load it directly.</p><p>To help with the following section, here's a list of acronyms:</p><p>f CSW: This is Catalog Service for the Web</p><p>f WFS: This is Web Feature Service</p><p>f WFS-T: This is Web Feature Service, Transactional</p><p>f WCS: This is Web Coverage Service</p><p>f WMS: This is Web Map Service</p><p>f WMTS: This is Web Map Tile Service</p><p>f TMS: This is Tile Map Service</p><p>f XYZ: This is an "X Y Z" Service (there really isn't a formal name for this one because</p><p>it's not an official standard)</p><p>How to do it…</p><p>1. Start by answering the following questions with regards to using data over</p><p>web services:</p><p>� Do you already know where to find the data that you want?</p><p>� Do you need to edit or apply custom styling to this data?</p><p>� Do you care if the data is vector or raster?</p><p>� Do you need the data at its original resolution or quality?</p><p>� Do you need data at specific resolutions or in a specific projection?</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>225</p><p>2. Use the following decision matrix to pick out which services are appropriate to your</p><p>use case:</p><p>Criteria CSW WFS WFS-T WCS WMS Tiles (WMTS,</p><p>XYZ, TMS)</p><p>Do you already know where to</p><p>find the data you want?</p><p>No Yes Yes Yes Yes Yes</p><p>Do you need to edit or apply</p><p>custom styling to the data?</p><p>Yes Yes Yes No No</p><p>Do you care if the data is</p><p>vector or raster?</p><p>Vector Vector Raster Raster Raster</p><p>Do you need the data at its</p><p>original resolution or quality?</p><p>Yes Yes Yes No No</p><p>Do you need data at specific</p><p>resolutions or in a specific</p><p>projection?</p><p>Yes Yes Yes Yes No</p><p>Recipe number in this chapter. 3 2 2 5 4 4, 6</p><p>Now that you've found the appropriate recipe for the web service that you want to use, jump</p><p>to this recipe later in this chapter. If you're still not sure, read on for more hints on how to pick</p><p>the correct service. This recipe applies both to how you use web services and how to decide</p><p>what web services to offer (if you put up a web service for other people to use).</p><p>Generally speaking, from left (WFS) to right (Tiles) the speed of the service increases. Tiles</p><p>serve data the fastest to end users but with the most limitations.</p><p>How it works…</p><p>For each of the services, there is a QGIS tool (built-in or plugin). This tool stores your list of web</p><p>servers and connection settings for each service. When you go to load a layer from a chosen</p><p>server for a particular protocol an up-to-date list of layers is requested from the server (that is,</p><p>the GetCapabilities XML). You then get to pick off this list the layers that you would like to add</p><p>to the map canvas (and depending on service type, the projection, and the file type).</p><p>There's more…</p><p>Vector is generally slower, as more data needs to be transmitted as the data grows. Raster</p><p>formats are a fixed number of pixels onscreen, so it's always approximately the same amount</p><p>of data per screen load.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>226</p><p>WMS, WMTS, WFS, and WCS are sometimes referred to as W*S as a</p><p>collective group of related services that behave similarly.</p><p>Each situation will have additional considerations. For example, if you need a specific</p><p>projection, you probably can't use a Tile service because these are usually only in very</p><p>specific projections (Web/Spherical Mercator). Or perhaps, you want to print large paper</p><p>maps. Then, you probably want WFS or WCS in order to get the full resolution possible over</p><p>your entire region.</p><p>One of the most common mistakes is to think that you need vector data when you actually just</p><p>need a background tile that incorporates vector data. A great example of this is road data. If</p><p>you don't actually need to style, select, or individually manipulate road data, and then a Tile</p><p>or WMS type layer will be much faster.</p><p>See also</p><p>f For more information, read the standards at the OGC website, http://</p><p>opengeospatial.org</p><p>Using WFS and WFS-T</p><p>Web Feature Services (WFS) is an OGC standard method to access and, in some cases, edit</p><p>(WFS-T) vector data over the Internet. When you need full attribute tables, local style control,</p><p>or editing, WFS is the way to go. Like most other web services, the biggest advantage over a</p><p>local layer is that you don't have to copy or load the whole layer at once.</p><p>If you just need to view the layer, often WMS or a Tile service</p><p>(described in other recipes within this chapter) are more efficient.</p><p>Getting ready</p><p>You need the URL of a WFS service to use and a working Internet connection. We will use the</p><p>public Mapserver demo website (http://demo.mapserver.org/).</p><p>To try WFS-T, which involves editing, you will need to get access to a</p><p>service (typically, password protected) or make one yourself. Do you</p><p>need a WFS-T test server? This is a great case where OSGeo-Live</p><p>comes in handy, as you can run your own WFS-T server in a virtual</p><p>machine at http://live.osgeo.org.</p><p>www.it-ebooks.info</p><p>http://opengeospatial.org</p><p>http://opengeospatial.org</p><p>http://demo.mapserver.org/</p><p>http://live.osgeo.org</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>227</p><p>How to do it…</p><p>1. Find a WFS service that you want to use and copy the GetCapabilities URL. For this</p><p>example, we will use the Mapserver demo website, http://demo.mapserver.org/.</p><p>As with other web services, it's more efficient if you load your local</p><p>layers and zoom to their extent first. This enables you to not waste time</p><p>loading data from web services for extents outside your area of interest.</p><p>2. Open the add WFS dialog by clicking on the following icon:</p><p>3. Create a New connection.</p><p>4. Assign Name of your choosing and paste in the URL field for the WFS service,</p><p>(http://demo.mapserver.org/cgi-bin/wfs?SERVICE=WFS&VERSION=1.0.</p><p>0&REQUEST=GetCapabilities):</p><p>On future usage of the same service, this will already be in your list of</p><p>services, so you only have to add it once.</p><p>5. Save your edits by clicking on the OK button.</p><p>6. Now select the service from the drop-down menu.</p><p>7. Query for a list of layers by clicking on the Connect button.</p><p>www.it-ebooks.info</p><p>http://demo.mapserver.org/</p><p>http://demo.mapserver.org/cgi-bin/wfs?SERVICE=WFS&VERSION=1.0.0&REQUEST=GetCapabilities</p><p>http://demo.mapserver.org/cgi-bin/wfs?SERVICE=WFS&VERSION=1.0.0&REQUEST=GetCapabilities</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>228</p><p>8. Select the layer or layers that you want to add to the map; either continents or</p><p>cities works for this example:</p><p>9. When your selection is complete, use the Add button to place the layer in the map.</p><p>10. Rearrange the render order of the map by dragging layers up and down in the list.</p><p>11. Pan and zoom to make fresh requests for WFS data to be loaded to the view.</p><p>WFS layers can be restyled with standard layer properties. Also,</p><p>the information tool and the attribute table will appear as other</p><p>vector layers.</p><p>How it works…</p><p>For each web service, there is a main URL. When you browse to this URL and add the</p><p>GetCapabilities parameter (QGIS does this for you), the returned result is an XML file,</p><p>which describes the services that are offered by the server. The client, QGIS, parses the list of</p><p>layers for you to choose from, and once you pick the layer(s), uses the additional information</p><p>in the</p><p>Errata</p><p>Although we have taken every care to ensure the accuracy of our content, mistakes do</p><p>happen. If you find a mistake in one of our books—maybe a mistake in the text or the</p><p>code—we would be grateful if you could report this to us. By doing so, you can save other</p><p>readers from frustration and help us improve subsequent versions of this book. If you find</p><p>any errata, please report them by visiting http://www.packtpub.com/submit-errata,</p><p>selecting your book, clicking on the Errata Submission Form link, and entering the details of</p><p>your errata. Once your errata are verified, your submission will be accepted and the errata will</p><p>be uploaded to our website or added to any list of existing errata under the Errata section of</p><p>that title.</p><p>To view the previously submitted errata, go to https://www.packtpub.com/books/</p><p>content/support and enter the name of the book in the search field. The required</p><p>information will appear under the Errata section.</p><p>Piracy</p><p>Piracy of copyrighted material on the Internet is an ongoing problem across all media. At</p><p>Packt, we take the protection of our copyright and licenses very seriously. If you come across</p><p>any illegal copies of our works in any form on the Internet, please provide us with the location</p><p>address or website name immediately so that we can pursue a remedy.</p><p>Please contact us at copyright@packtpub.com with a link to the suspected pirated material.</p><p>We appreciate your help in protecting our authors and our ability to bring you valuable content.</p><p>Questions</p><p>If you have a problem with any aspect of this book, you can contact us at questions@</p><p>packtpub.com, and we will do our best to address the problem.</p><p>www.it-ebooks.info</p><p>http://www.packtpub.com/sites/default/files/downloads/QGIS2Cookbook_ColorImages.pdf</p><p>http://www.packtpub.com/sites/default/files/downloads/QGIS2Cookbook_ColorImages.pdf</p><p>http://www.packtpub.com/submit-errata</p><p>https://www.packtpub.com/books/content/support</p><p>https://www.packtpub.com/books/content/support</p><p>http://www.it-ebooks.info/</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>1</p><p>1</p><p>Data Input and Output</p><p>In this chapter, we will cover the following recipes:</p><p>f Finding geospatial data on your computer</p><p>f Describing data sources</p><p>f Importing data from text files</p><p>f Importing KML/KMZ files</p><p>f Importing DXF/DWG files</p><p>f Opening a NetCDF file</p><p>f Saving a vector layer</p><p>f Saving a raster layer</p><p>f Reprojecting a layer</p><p>f Batch format conversion</p><p>f Batch reprojection</p><p>f Loading vector layers into SpatiaLite</p><p>f Loading vector layers into PostGIS</p><p>Introduction</p><p>If you want to work with QGIS, the first thing you need is spatial data. Whether you want to</p><p>prepare a nice-looking map layout or perform spatial analysis, you need to open some data to</p><p>work with. This chapter deals with the basic input and output commands, which will allow you</p><p>to use data in several different formats and also export to the most convenient format</p><p>in case you want to use it in different applications or share with others.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>2</p><p>Automation is possible for many of the operations that you will see in this cookbook. This</p><p>chapter contains some recipes that use automation to process a set of input files.</p><p>Finding geospatial data on your computer</p><p>This recipe shows you how to use the QGIS browser to locate and open spatial data.</p><p>Getting ready</p><p>Before you start working, make sure that you have copied the sample dataset to your</p><p>filesystem and you have it located.</p><p>How to do it…</p><p>There are several ways of locating and opening a data file to open it in QGIS, but the most</p><p>convenient of these is the QGIS browser:</p><p>1. To enable this, go to the View | Panels menu and enable the Browser checkbox in</p><p>it. The browser will be shown by default in the left-hand side of the QGIS window, as</p><p>shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>3</p><p>Browser contains a tree with all the available sources of spatial data. This includes</p><p>data files in your filesystem, databases, and remote services.</p><p>2. Navigate to the folder where you copied the sample dataset, and you will see a list of</p><p>available data files, as shown in the following screenshot:</p><p>Not all files are shown but just the ones that are identified as valid</p><p>data sources.</p><p>3. To add a file to your project, just right-click on it and select Add Layer:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>4</p><p>4. Multiple selections are allowed. In that case, select the Add selected layer menu.</p><p>Another way of opening a file is by just dragging it and dropping it into the QGIS</p><p>canvas. Dragging multiple files is allowed, as well, as shown in the following</p><p>screenshot:</p><p>How it works…</p><p>The browser acts as a file explorer that is directly linked to QGIS, which only shows valid data</p><p>files and can be used to easily add them to a QGIS project.</p><p>There's more…</p><p>There are a few more things that you need to know that are related to this recipe. They are</p><p>explained in the following sections.</p><p>Adding layers with the Layer menu</p><p>As an alternative to the browser, the Layer menu contains a set of entries. Each of them deals</p><p>with a different type of data. They give you some additional options, and they might allow you</p><p>to work with formats that are not directly supported by the browser.</p><p>Adding a folder to Favorites</p><p>Navigating to the folder where your data is located can be tedious. If you use a given folder</p><p>regularly, you can right-click on it and select Add as favorite. The folder will appear on the</p><p>Favorites section at the top of the browser tree.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>5</p><p>Nonfile data sources</p><p>The browser also shows non-file data, such as remote services. Services have to be defined</p><p>before they appear on the corresponding section in the browser. To add a service, right-click</p><p>on the service name and select New connection.... A dialog will appear to define the service</p><p>connection parameters.</p><p>As an example, try adding the following WMS service, using the WMS entry in the browser,</p><p>as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>6</p><p>A new entry will appear, containing the layers offered by the service, as shown in the</p><p>following screenshot:</p><p>Describing data sources</p><p>You can get additional information about a data file before opening it. This recipe shows you</p><p>how to explore the properties of a data origin.</p><p>Getting ready</p><p>Before you start working, make sure that you have copied the sample dataset to your</p><p>filesystem and that you have it located.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>7</p><p>How to do it…</p><p>1. In the QGIS browser, navigate to the folder with your sample dataset. Select the</p><p>elev_lid792_1m file and right-click on it. In the context menu, select Properties.</p><p>A dialog like the one in the following screenshot will appear:</p><p>This dialog displays the properties of a raster layer.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>8</p><p>2. Now, let's select a vector layer instead. Select the elev_lid792_randpts.shp file,</p><p>right-click on it, and select Properties. The information dialog will look like</p><p>the following:</p><p>How it works…</p><p>In the upper part of the description window, you will see a field named Provider. Provider</p><p>defines the type or data origin and who takes care of reading the data and passing it to QGIS.</p><p>For raster layers, you will see gdal as Provider. For most file-based vector layers, ogr will</p><p>be the provider that will appear. They refer to the GDAL and OGR libraries, two open-source</p><p>libraries that are used by many GIS programs to access both raster and vector data.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>9</p><p>There's more…</p><p>If the data is already loaded in QGIS, you can access the information about it in the Properties</p><p>section of the layer (right-click on the layer name to select the Properties entry in the context</p><p>menu). In</p><p>XML to look up the data at the specific URL.</p><p>Data requests are limited to the visible bounding box of the map canvas. This limits the</p><p>amount of data that is requested. At least this is how it should work. However, features that go</p><p>off screen will likely be included in their entirety to maintain geometry integrity. So, expect that</p><p>loading large vector layers over WFS has the potential to be extremely slow.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>229</p><p>There's more…</p><p>WFS-T services typically require passwords and are designed to work over the Internet. If you</p><p>are working within a local network, you may consider just using PostGIS layers. Either way,</p><p>it should also be noted that versioning and conflict resolution are not automatic, requiring</p><p>the service backend to be configured to support such features.</p><p>Searching CSW</p><p>CSW is a catalog web service. Its main function is to provide discoverability of geographic data</p><p>and link you to usable data either by download or by any other of the web services that are</p><p>mentioned in this chapter.</p><p>Getting ready</p><p>This recipe uses the MetaSearch plugin. It requires the pycsw and owslib libraries installed</p><p>in your system's Python. Refer to the Adding plugins with Python dependencies recipe of</p><p>Chapter 11, Extending QGIS, for help on installing pycsw and owslib if you don't know how</p><p>to do this.</p><p>How to do it…</p><p>1. Open the MetaSearch plugin by navigating to Web | MetaSearch | MetaSearch.</p><p>If you don't see the Web menu, check the plugin manager and</p><p>ensure that MetaSearch is enabled (checkmark).</p><p>2. Pick a service from the dropdown on the right: UK Location Catalogue</p><p>Publishing Service.</p><p>If you don't see any services in the drop-down list, click on the</p><p>Services tab and use the Add default services button.</p><p>3. Type a search term in box on the left: Park.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>230</p><p>4. (Optional) Set an extent to limit the search. In this case, use Map Extent.</p><p>Global searches often return too many results, or they cause the</p><p>connection to time out while waiting for all the results. As with other</p><p>web services, it is advisable to load a reference layer and zoom to</p><p>the area of interest first before trying to search them. The third tab,</p><p>Settings, allows you to adjust the timeout. Increase this if you're</p><p>getting too many timeout errors.</p><p>5. Click on the Search button and wait for the results:</p><p>1. Double-click on any of the results to see additional details.</p><p>2. If a selected result is available as a loadable layer, one or more of the service</p><p>buttons at the bottom of the screen will be enabled. To understand more</p><p>about how to use each of these choices, refer to the other recipes in this</p><p>chapter on WMS, WFS, and WCS:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>231</p><p>How it works…</p><p>MetaSearch queries websites that provide catalogs in the CSW standard, which is defined</p><p>by the OGC. Once your request parameters are sent, the receiving website queries its online</p><p>database for matches. If matches are found, metadata about the results is sent back to the</p><p>client, in this case, QGIS.</p><p>CSW currently includes options to search by keyword and spatial extent. Future versions may</p><p>enable setting time frames.</p><p>There's more…</p><p>If you pick opening an additional service that is based on the results, Metasearch will create a</p><p>temporary service registration and open the correct service dialog. Unfortunately, at this time,</p><p>you need to then scroll through the available layers to find the one that you want and actually</p><p>add it to the map.</p><p>Additional future CSW searches will ask if you want to override the</p><p>existing connection. You must say yes. If you find yourself using the</p><p>same W*S service, consider copying the GetCapabilities URL and</p><p>making a new permanent entry in the correct service dialog.</p><p>You can add more catalogs to search on the Services tab within the plugin. You will need</p><p>to find the CSW GetCapabilities URL on the website that you want to query. Most of the</p><p>common geoportal-type websites now support CSW, including (but not limited to) Geonode,</p><p>Geonetwork, and the ESRI Geoportal.</p><p>CSW is a relatively new standard when compared to some of the others, and it seems to be</p><p>hard to find services that consistently work and actually offer WMS, WCS, or WFS of the layers</p><p>in their catalog.</p><p>See also</p><p>f The Adding plugins with Python dependencies recipe of Chapter 11, Extending QGIS,</p><p>for help on installing pycsw and owslib if you don't know how</p><p>Using WMS and WMS Tiles</p><p>Web Map Services (WMS), one of the first OGC web services created, provides a method for</p><p>dynamic raster generation served over the Web. They are a compromise between the flexibility</p><p>of WFS and the speed of Tile services.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>232</p><p>Getting ready</p><p>There are several iterations of WMS, and QGIS supports most of them. To use a WMS, you</p><p>need to give QGIS the GetCapabilities URL of the service that you want to view data from.</p><p>How to do it…</p><p>1. Find a WMS service that you want to use and copy the GetCapabilities URL. In this</p><p>recipe, we can use the Geoserver demo website (http://demo.opengeo.org/</p><p>geoserver/web/).</p><p>As with other web services, it's more efficient if you load your</p><p>local layers and zoom to their extent first. This enables you to not</p><p>waste time loading data from web services for extents outside</p><p>your area of interest.</p><p>2. Open the Add WMS dialog.</p><p>3. Create a New connection.</p><p>4. Assign a Name of your choosing and paste in the URL (http://demo.opengeo.</p><p>org/geoserver/ows?service=wms&version=1.3.0&reque</p><p>st=GetCapabilities):</p><p>www.it-ebooks.info</p><p>http://demo.opengeo.org/geoserver/web/</p><p>http://demo.opengeo.org/geoserver/web/</p><p>http://demo.opengeo.org/geoserver/ows?service=wms&version=1.3.0&reque st=GetCapabilities</p><p>http://demo.opengeo.org/geoserver/ows?service=wms&version=1.3.0&reque st=GetCapabilities</p><p>http://demo.opengeo.org/geoserver/ows?service=wms&version=1.3.0&reque st=GetCapabilities</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>233</p><p>On future usage of the same service, this will already be in</p><p>your list of services, so you only have to add it once.</p><p>5. Save your edits by clicking on OK.</p><p>6. Now select the service from the drop-down list.</p><p>7. Query for a list of layers using the Connect button.</p><p>8. Select the layer or layers that you want to add to the map:</p><p>You can select one or more layers. If you select multiple layers, they</p><p>will be merged and only appear as a single layer in the QGIS Layers</p><p>list. The Layer Order tab lets you arrange the WMS layers within the</p><p>combined layer. This is important when one of the layers is opaque</p><p>and has 100% continuous data, allowing you to put other data on</p><p>top of it visually.</p><p>9. There are several other options, including image type and projection:</p><p>� For an image type, PNG is a good default as it supports lossless compression</p><p>and transparency. If you don't need transparency and are okay with a little</p><p>data loss, JPG can be used for smaller files, so they are faster to load.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>234</p><p>� When picking projection, if you can use the original projection of the</p><p>data (if you know it), you will get the least resampling. Otherwise, pick</p><p>something that matches the other data that you plan to use in conjunction</p><p>with the WMS.</p><p>Not all image types and projections are available; this depends</p><p>on what the server offers. If one image type doesn't seem to</p><p>work, try a different one before reporting a bad server.</p><p>10. When your selection is complete, use the Add button to place the layer in the map.</p><p>11. Rearrange the render order of the map by dragging layers up and down in the list.</p><p>12. Pan and zoom to make fresh requests for WMS data to be loaded to the view.</p><p>How it works…</p><p>When you pan and zoom the map, a request with the bounding box of the viewable extent and</p><p>scale is sent to the service. The server then renders an image that matches the request and</p><p>passes it back</p><p>to the client (in this case, QGIS).</p><p>There's more…</p><p>Some WMS services now also support tiling under the Web Map Tiling Service (WMTS)</p><p>protocol. From the client's perspective, this not really different from WMS in usage. On the</p><p>server side, after each request the results are cached so that if the same extent and scale is</p><p>requested, the cached version can be delivered instead of creating the results from scratch.</p><p>For you, the end user, this should result in faster loading if a service provides WMTS.</p><p>When configuring a WMTS, use the WMTS URL instead of the WMS URL. One example would</p><p>be the Geoserver demo site's WMTS:</p><p>http://demo.opengeo.org/geoserver/gwc/service/</p><p>wmts?REQUEST=GetCapabilities</p><p>Once successful, this will take you to the Tilesets tab, where you can pick which layer and</p><p>projection of the available options you want to load. As the Tiles are premade or cached, you</p><p>will usually not have the option to combine multiple layers at once and will need to load them</p><p>one at a time:</p><p>www.it-ebooks.info</p><p>http://demo.opengeo.org/geoserver/gwc/service/wmts?REQUEST=GetCapabilities</p><p>http://demo.opengeo.org/geoserver/gwc/service/wmts?REQUEST=GetCapabilities</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>235</p><p>WMS-C is an earlier version of the WMTS standard. In usage, it's pretty</p><p>much the same, though the URL pattern may look more similar to the WMS.</p><p>See also</p><p>f See the QGIS documentation for more information about the WMS capabilities of</p><p>QGIS at http://docs.qgis.org/2.8/en/docs/user_manual/working_</p><p>with_ogc/ogc_client_support.html#ogc-wms</p><p>Using WCS</p><p>A Web Coverage Service (WCS) differs greatly in use case from the other services, but it</p><p>behaves very similarly. The goal of WCS is to allow users to extract a region of interest from</p><p>a large raster data that is hosted remotely. Unlike a WMS or Tiled set, WCS is a clip of the</p><p>original data in full resolution and usually in the original projection. This format is ideal if</p><p>you need the raster data for analysis purposes and not just visualization.</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_ogc/ogc_client_support.html#ogc-wms</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_ogc/ogc_client_support.html#ogc-wms</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>236</p><p>Getting ready</p><p>For this recipe, you need a WCS to connect to. Check with your data providers to see</p><p>whether they offer WCS. For this recipe, we can use the OpenGeo Geoserver Demo site</p><p>at http://demo.opengeo.org/geoserver/web/.</p><p>How to do it…</p><p>1. Open a web browser and go to http://demo.opengeo.org/geoserver/web/.</p><p>2. On the right-hand side, you'll see a list of web services that are available; right-click</p><p>on WCS 1.1.1 and copy the link.</p><p>3. In QGIS, open the WCS dialog.</p><p>4. Select New to create a new server entry.</p><p>5. In the boxes, perform the following:</p><p>1. Provide a name so that you remember which service this is.</p><p>2. Paste the URL that you copied earlier in the URL box (http://demo.</p><p>opengeo.org/geoserver/ows?service=wcs&version=1.1.1&requ</p><p>est=GetCapabilities):</p><p>6. Click on the OK button.</p><p>www.it-ebooks.info</p><p>http://demo.opengeo.org/geoserver/web/</p><p>http://demo.opengeo.org/geoserver/web/</p><p>http://demo.opengeo.org/geoserver/ows?service=wcs&version=1.1.1&request=GetCapabilities</p><p>http://demo.opengeo.org/geoserver/ows?service=wcs&version=1.1.1&request=GetCapabilities</p><p>http://demo.opengeo.org/geoserver/ows?service=wcs&version=1.1.1&request=GetCapabilities</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>237</p><p>7. Now, you'll be back on the Add Layer(s) from a WCS Server dialog:</p><p>1. Click on the Connect button.</p><p>2. After the list is populated, select a layer to add to the map. Click on the</p><p>Add button. Try the Blue Marble layer.</p><p>3. Now, click on the Close button to return to your map:</p><p>8. You should now see the Blue Marble layer loaded.</p><p>If you zoom in to the level of a US State or a European</p><p>country, you will see the image start to pixelate. Blue Marble</p><p>is a low resolution image put together by NASA that roughly</p><p>shows what the whole world looks like from space, cloud</p><p>free. It is meant as a general view of the whole world and</p><p>does not contain fine details.</p><p>9. (Optional) Use the Save As option to download a portion or the entire WCS layer at its</p><p>full original resolution.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>238</p><p>How it works…</p><p>WCS, like other web services, sends a bounding box request to the server, which in turn</p><p>delivers the raster data to QGIS. Unlike WMS, no rendering is done on the server side,</p><p>the raw original raster data is sent. This could mean the following:</p><p>f There was no resampling of the image before it was sent</p><p>f You can apply your own styling to the data that is delivered</p><p>Keep in mind that requesting the full extent of a high resolution raster will result in a large</p><p>amount of data transfer. This is unlike Tiles or WMS, which at most return the exact number</p><p>of pixels in the viewable area at the resolution that is requested.</p><p>As with other web services, it is recommended that you zoom in to your area</p><p>of interest before loading a WCS.</p><p>There's more…</p><p>Another bonus of WCS over WMS is that because WCS delivers the original data, it is not</p><p>limited to a 3-band RGB image. You can use WCS to view and download Multi or Hyperspectral</p><p>data (4+ bands common in remote-sensing applications).</p><p>Currently, QGIS only supports 1.0.x and 1.1.x, not WCS 2.x; at least not yet!</p><p>Using GDAL</p><p>The QuickMapsServices and OpenLayers plugins, as described in the Loading BaseMaps with</p><p>the QuickMapServices plugin and Loading BaseMaps with the OpenLayers plugin recipes in</p><p>Chapter 4, Data Exploration, are awesome as they put a reference layer in your map session.</p><p>The one downside, however, is that it is a hassle to add new layers. So, if you come across or</p><p>build your own Tile service and want to use it in QGIS, this recipe will let you use almost any</p><p>Tile service.</p><p>Getting ready</p><p>You will need a web browser, text editor, and the URL of a web-based XYZ (sometimes called</p><p>TMS) service—one that allows you to make requests without an API key. We're going to use</p><p>the maps at http://www.opencyclemap.org/.</p><p>Viewing the JavaScript source (a good tool for this is Firebug, or other web-developer tools for</p><p>the browser), we can view the source URLs for the tiles.</p><p>www.it-ebooks.info</p><p>http://www.opencyclemap.org/</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>239</p><p>How to do it…</p><p>1. Open http://www.opencyclemap.org/ in a web browser.</p><p>2. Now, figure out the URL for the tiles by looking at the source code:</p><p>1. Look in map.js and you'll see the layer definition:</p><p>var cycle = new OpenLayers.Layer.OSM("OpenCycleMap",</p><p>["https://a.tile.thunderforest.com/cycle/${z}/${x}/${y}.</p><p>png",</p><p>"https://b.tile.thunderforest.com/cycle/${z}/${x}/${y}.</p><p>png",</p><p>"https://c.tile.thunderforest.com/cycle/${z}/${x}/${y}.</p><p>png"],</p><p>{ displayOutsideMaxExtent: true,</p><p>attribution: cycleattrib, transitionEffect: 'resize'}</p><p>);</p><p>2. Or, you can look at the image files your browser downloads. If you put</p><p>https://a.tile.thunderforest.com/cycle/13/1325/3143.png</p><p>into a browser, it will load that one tile.</p><p>3. The pattern is pretty straight forward:</p><p>////.</p><p>In this particular case, the Tile Index pattern is the TMS style;</p><p>refer to http://www.maptiler.org/google-maps-</p><p>coordinates-tile-bounds-projection/.</p><p>4. To turn this into a layer in QGIS, open up a text editor and paste in the following</p><p>definition. This definition tells GDAL which driver to use and the server URL pattern</p><p>with z, x, and y as variables. Save the file as opencyclemap.xml:</p><p>http://c.tile.thunderforest.com/cycle/${z}/${x}/</p><p>${y}.png</p><p>-20037508.34</p><p>20037508.34</p><p>20037508.34</p><p>-20037508.34</p><p>18</p><p>1</p><p>www.it-ebooks.info</p><p>http://www.opencyclemap.org/</p><p>https://a.tile.thunderforest.com/cycle/13/1325/3143.png</p><p>http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/</p><p>http://www.maptiler.org/google-maps-coordinates-tile-bounds-projection/</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>240</p><p>1</p><p>top</p><p>EPSG:3785</p><p>256</p><p>256</p><p>3</p><p>5. You can now load the layer using the Raster dialog or the browser:</p><p>Note that there are two listings for opencyclemap.xml; only</p><p>the one with the square-shaped icon will work (that is, a raster),</p><p>as tiles are a raster format.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>241</p><p>How it works…</p><p>The XML file defines the parameters of the service; however, because XYZ-style servers don't</p><p>follow a standard, the URL pattern varies slightly for each server and the servers do not have</p><p>a GetCapabilities function that describes available layers. By telling GDAL how to handle the</p><p>URL, you are wrapping a nonstandard format into a typical GDAL layer, which QGIS can easily</p><p>be loaded as a raster.</p><p>There's more…</p><p>One additional tip when using Spherical Mercator (EPSG:3785) is that you can set a custom</p><p>list of scales (Zoom Levels) in QGIS. The following set of scales can be loaded per QGIS</p><p>project, and will change the dropdown at the bottom right. These scales match the scales</p><p>that most servers will provide, so you get the best viewing experience:</p><p>1. Go to File | Project Properties.</p><p>2. Select the General tab.</p><p>3. Check the Project Scales checkbox.</p><p>4. Load the scales.xml file that is provided:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>242</p><p>This technique is not limited to just tile services. Many other formats that GDAL works</p><p>with can be wrapped for easier usage in QGIS. This is a similar method to Virtual Raster</p><p>Tables (VRT) layers mentioned in the Creating raster overviews (pyramids) recipe in</p><p>Chapter 2, Data Management.</p><p>Lastly, you may ask why a new plugin using this method doesn't replace the OpenLayers</p><p>plugin. Such an idea has been under discussion for a while; the key sticking point is that</p><p>accessing some layers, such as Google, Bing, and so on, with this method may violate the</p><p>Terms of Service as they do not keep the Copyright, Trademark, and Logo in the correct place.</p><p>Also, caching and printing such layers may not be legal. In general, avoid using proprietary</p><p>data when possible to reduce licensing issues.</p><p>See also</p><p>f This recipe and method has actually been known and discussed in many QGIS</p><p>venues. The most frequently cited example is available at http://www.3liz.com/</p><p>blog/rldhont/index.php?post/2012/07/17/OpenStreetMap-Tiles-in-</p><p>QGIS.</p><p>f The full explanation of options for GDAL can be found at http://www.gdal.org/</p><p>frmt_wms.html.</p><p>Serving web maps with the QGIS server</p><p>QGIS and the Web is not all about consuming data, it can also be used to serve data over the</p><p>Web for others to view online or consume in other web clients (such as QGIS). Keep in mind</p><p>that setting up your own web service is not the easiest way to make a web map (refer to the</p><p>Hooking up web clients recipe in this chapter). This is, however, a great way to transition all</p><p>the hard work that you've put into a QGIS project file into something other people can see</p><p>and use.</p><p>Getting ready</p><p>For this recipe, you need a working installation of the QGIS server. This involves running a</p><p>standard web server (such as Apache or Nginx). There are many ways to set up the server, so</p><p>please see the official documentation at http://hub.qgis.org/projects/quantum-</p><p>gis/wiki/QGIS_Server_Tutorial.</p><p>Once you have the QGIS server running, then you just need a QGIS project with the configuration</p><p>outlined in this recipe.</p><p>www.it-ebooks.info</p><p>http://www.3liz.com/blog/rldhont/index.php?post/2012/07/17/OpenStreetMap-Tiles-in-QGIS</p><p>http://www.3liz.com/blog/rldhont/index.php?post/2012/07/17/OpenStreetMap-Tiles-in-QGIS</p><p>http://www.3liz.com/blog/rldhont/index.php?post/2012/07/17/OpenStreetMap-Tiles-in-QGIS</p><p>http://www.gdal.org/frmt_wms.html</p><p>http://www.gdal.org/frmt_wms.html</p><p>http://hub.qgis.org/projects/quantum-gis/wiki/QGIS_Server_Tutorial</p><p>http://hub.qgis.org/projects/quantum-gis/wiki/QGIS_Server_Tutorial</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>243</p><p>How to do it…</p><p>1. Open QGIS.</p><p>2. Load up and style some layers:</p><p>� You need at least one vector layer to offer a WFS.</p><p>� You need at least one raster layer to offer a WCS.</p><p>� WMS can be any combination of layers, you can choose to server each as an</p><p>independent layer or as a combined layer.</p><p>3. Edit the Project properties in File | Project Properties:</p><p>1. Open the OWS server tab.</p><p>2. Check the Service Capabilities box to enable GetCapabilities.</p><p>3. Fill out some of the boxes so that end users know what your server is about,</p><p>who runs it, and how to contact you:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>244</p><p>4. Now examine the WMS capabilities section:</p><p>Most of these features are optional optimizations. Pick and choose</p><p>what suits your needs.</p><p>5. Here, you can set the maximum extent that clients should expect.</p><p>6. The CRS restrictions option lets you limit what projections are allowed.</p><p>7. Exclude Layers allows you to have layers in your project that don't show up on</p><p>the web.</p><p>8. Add geometry to feature response is an optional enhancement if you are building</p><p>a web map and you want to be able to work with the actual vectors (if it is a vector</p><p>to begin with).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>245</p><p>9. GetFeatureInfo precision is about how close a user has to click to query a location. If</p><p>you have a lot of data, you probably want this number to be small; but if you have only</p><p>a few features, making this bigger makes it easier for end users.</p><p>10. Set Maximums for GetMap request if you want to reduce the load on your server</p><p>by limiting how much data a user can request at once. This is a good idea for a</p><p>public server. 2560 x 2048, as shown in the screenshot, is enough pixels for an</p><p>HD-resolution screen to be filled in a single request.</p><p>11. Next, take a look at the WFS capabilities section:</p><p>Only enable WFS if you want users to be able to request vector</p><p>data as vectors. This can be more intensive than WMS on your</p><p>bandwidth. Also, do not enable WFS-T features unless you secure</p><p>your server to only permitted users.</p><p>1. Check the Published box next to any vector layers that you want to be usable</p><p>over WFS.</p><p>2. To enable WFS-T, check the Update, Insert, and Delete checkboxes. As they</p><p>are separate, you can choose to only allow new data (Insert), only allow</p><p>edits to existing data (Update), or only allow removal of data (Delete). Insert</p><p>would be the safest option as it prevents editing or deletion of existing data.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>246</p><p>12. Finally, take a look at WCS capabilities:</p><p>This an all or none feature. Don't enable this unless you want users</p><p>to be able to download the original raster data.</p><p>13. When you are done setting options, click on the OK button.</p><p>14. Now, save the project in a place where the QGIS server has access to it.</p><p>In Apache, this is usually a folder such as /var/www/.</p><p>15. Once saved, you can test access from any OGC-compliant web client:</p><p>1. For a simple test, use a fresh QGIS project and the Add WMS dialog.</p><p>2. The GetCapabilties URL will look something like http://localhost/cgi-</p><p>bin/qgis_mapserv.fcgi?map=/usr/local/share/qgis/QGIS-</p><p>NaturalEarth-Example.qgs.</p><p>The key part of this URL that is somewhat unique to QGIS server is the</p><p>map parameter, which is followed by the full system path to the QGIS</p><p>project file.</p><p>This may seem odd, but adding your QGIS server as a WMS in QGIS is</p><p>a great way to test whether it's working.</p><p>How it works…</p><p>The QGIS server is a middleman that takes in web service requests and translates</p><p>them into</p><p>QGIS internal calls, returning the requested data or rendered images, which are delivered to</p><p>the end user via the web server.</p><p>www.it-ebooks.info</p><p>http://localhost/cgi-bin/qgis_mapserv.fcgi?map=/usr/local/share/qgis/QGIS-NaturalEarth-Example.qgs</p><p>http://localhost/cgi-bin/qgis_mapserv.fcgi?map=/usr/local/share/qgis/QGIS-NaturalEarth-Example.qgs</p><p>http://localhost/cgi-bin/qgis_mapserv.fcgi?map=/usr/local/share/qgis/QGIS-NaturalEarth-Example.qgs</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>247</p><p>There's more…</p><p>The QGIS server contains many options that allow you to control which types of service to</p><p>offer, which layers to offer over each service, and how to style these services. Alternatives to</p><p>the QGIS server include MapServer and GeoServer (refer to the Managing GeoServer from</p><p>QGIS section in this chapter).</p><p>See also</p><p>f For more details, refer to the main documentation for the QGIS server at http://</p><p>hub.qgis.org/projects/quantum-gis/wiki/QGIS_Server_Tutorial.</p><p>f Once you create a service, test it by adding your service to a QGIS project. Refer to</p><p>the previous recipes in this chapter for how to add WMS, WFS, or WCS services.</p><p>Scale-dependent rendering</p><p>While they are not specifically for web services, being able to change the styling and presence</p><p>of data based on the scale of the map can have a huge impact on the speed and readability</p><p>of web services. Unlike printed maps, web maps are viewed at multiple scales. This variation</p><p>in scales often requires different cartography to keep the map legible and usable.</p><p>Getting ready</p><p>You'll need a QGIS project, preferably one with a high data density or differing levels of</p><p>information. A good example is road data, where you have major, minor, local, and other</p><p>variants of road classification. caryStreets.shp converted from CAD in a previous</p><p>chapter is a good example.</p><p>How to do it…</p><p>1. Open QGIS and load caryStreets.shp.</p><p>2. Now, open the attribute table and look for an attribute to filter in. In</p><p>caryStreets.shp, there are several potential columns to use, such as</p><p>StreetType, Major_Road, and Main_Road.</p><p>StreetType appears to be classes, whereas the other two</p><p>columns appear to be True or False flags. Any of these are</p><p>decent candidates for filtering rules.</p><p>www.it-ebooks.info</p><p>http://hub.qgis.org/projects/quantum-gis/wiki/QGIS_Server_Tutorial</p><p>http://hub.qgis.org/projects/quantum-gis/wiki/QGIS_Server_Tutorial</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>248</p><p>3. Now, open the Properties section for the layer:</p><p>1. Switch to the Style tab to edit the symbology.</p><p>2. Change the top-right dropdown to Rule Based Rendering.</p><p>3. Create a new rule (green plus sign).</p><p>4. In the pop up dialog set Label to Major Roads and Filter to "Major_Road" = 't'.</p><p>5. (Optional) You can use the expression builder to build the filter statement and test it.</p><p>Click on the … button to open the dialog.</p><p>You could create two copies of Major with different scale ranges</p><p>so that as you zoom in, the major roads become thicker at the</p><p>same time that minor roads are enabled.</p><p>This is what your layer looks like before and after you create the first rule:</p><p>6. Now, add another rule for minor roads by filtering for "Major_Road" = 'f'.</p><p>7. This time, you're going to enable the Scale range option.</p><p>8. Set Minimum (exclusive) to 1:100,000. For any scale bigger than this, the features</p><p>will be hidden. For Maximum (inclusive), type in 1:0, which will disable the</p><p>Max filter.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>249</p><p>9. Pick a different line type and/or color for the minor roads:</p><p>10. You should now have two rules, one for major roads and one for minor roads:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>250</p><p>You don't have to open the edit rule dialog; you can directly modify</p><p>parts of the rules in the Rule Based Rendering page.</p><p>11. Go back to the map and zoom in to 1:50,000, then zoom out to 1:250,000. The</p><p>minor roads should appear and disappear as you change past the 1:100,000 scale:</p><p>How it works…</p><p>The goal with scale-dependent rendering is generally to make your map readable at many</p><p>different zoom levels. By setting the Min and Max scales for each layer or subfeatures within</p><p>a layer, you can declutter a map for readability. The rendering engine just checks the scale</p><p>against each rule before deciding what to render.</p><p>There's more…</p><p>Scale-dependent rendering can be used in several ways. This can be used to change the</p><p>styling based on zoom or hide or reveal data based on the zoom level. However, it's also</p><p>not limited to just changing styles or layers. You can also perform scale-dependent labeling,</p><p>which is part of data-driven labeling described in the Configuring data-defined labels recipe</p><p>in Chapter 10, Cartography Tips, of this book.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>251</p><p>Scale rules also work on raster layers; however, this only allows you to turn a raster on and off.</p><p>It doesn't allow you to change its appearance.</p><p>If you have a QGIS server set up from earlier in this chapter, the scaling rules should apply</p><p>to your web services (WMS and WFS).</p><p>You probably don't want to use something as complex as a street</p><p>layer via WFS in a web browser because it's almost guaranteed to</p><p>crash. Stick to pushing such complex layers as Tiles or WMS.</p><p>See also</p><p>The Rule Based Rendering has a lot of features crammed into it. However, this is not yet</p><p>a comprehensive guide to everything that it can do, so you'll need to explore and perform</p><p>Internet searches for now.</p><p>Hooking up web clients</p><p>Sometimes, the best way to share a map is to build a website with a map embedded in it.</p><p>There are many methods to accomplish this goal, ranging from a simple dump of a few layers</p><p>to a highly-interactive map, which is based on web services. There are many web clients that</p><p>will work with standard OGC services. This recipe will show you how to build a simple web map</p><p>using Leaflet—a popular JavaScript library that is used to create web maps.</p><p>Getting ready</p><p>You will need the qgis2leaf plugin and some sample data. The schools_wake.shp (Points)</p><p>and census_wake_2000.shp files make for a good example.</p><p>How to do it…</p><p>1. Install and enable the qgis2leaf plugin.</p><p>Make sure to check out the qgis2web plugin, which is a newer</p><p>variant that works similarly but has some different options.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>252</p><p>2. Load up some layers to make a map composition.</p><p>Make a copy of your layer and eliminate unnecessary columns that</p><p>you don't need to show on the web map. Reducing the size of the</p><p>attribute table will make it easier to read popups with information</p><p>and speed up web page loading.</p><p>3. Style the map as you want it to appear online.</p><p>Styling can be really tricky. Leaflet and other web map libraries don't</p><p>support 100% of the same options as QGIS. Try making a few maps,</p><p>changing settings, and re-exporting these maps a few times to</p><p>figure out how to get it the way that you want. It may not look good</p><p>in QGIS but look good in the export.</p><p>4. (Optional) Configure labels. In this example, label the School names.</p><p>Only black labels are currently supported. Though you can probably</p><p>customize the CSS and JavaScript (js) after the export if you need</p><p>labels in a different style.</p><p>5. Open the qgis2leaf plugin from its icon on the toolbar or from the Web menu:</p><p>1. Click on the GetLayers button to add the layers from your map to the</p><p>export list.</p><p>2. There are lots of options here, and they are optional. Go ahead and check</p><p>Create Legend. If you made labels, also check Export Labels and labels</p><p>on hover.</p><p>Create Cluster is a fantastic option if you have a lot of points on the</p><p>map. This will group points into a circle with a number indicating</p><p>how many points are near there. As you zoom in, they will split apart</p><p>into smaller groups, until at some zoom, all the points are in their</p><p>original spot.</p><p>3. For the frame size, you can pick a size of the page that you want the map to</p><p>take up (in pixels). However,</p><p>fullscreen works well if the map is the only thing</p><p>that you care about.</p><p>4. Go ahead and add a tile-based base layer; Stamen Terrain is an interesting</p><p>choice. Keep in mind that you can only have one of these on at a time, but</p><p>you can toggle between them.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>253</p><p>5. Pick an output folder location and fill in the remaining map information</p><p>describing how you want it to show up in the results.</p><p>6. Export the project:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>254</p><p>6. After exporting, the map should open in your web browser. If it doesn't, open your</p><p>operating system file explorer (or web browser) and navigate to the output folder.</p><p>You should see a new folder called export_year_month_day_hour_minute_</p><p>seconds (for example, export_2015_02_19_11_34_05). Inside this folder is</p><p>index.html. Open this file with a web browser to see your map:</p><p>7. Note that all the vectors are clickable, and the popup will display the attribute table</p><p>information. If you turned on labels and hover, then hovering over a point will display</p><p>the name.</p><p>How it works…</p><p>The qgis2leaf plugin converts your map into something that is compatible with the web.</p><p>Generally, this means converting vector data to the GeoJSON format and generating an</p><p>HTML page (web page) with some JavaScript to create and populate the map.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>255</p><p>Raster layers are trickier, and if you can, try to stick to using Tile or WMS services to serve</p><p>them. Refer to the next section to see how to use Tiles or WMS.</p><p>If you need host tiles locally, try using the QTiles plugin to generate them.</p><p>There's more…</p><p>The next logical step is to make the map dynamic based on a web service. To do this, you can</p><p>swap static files for web services:</p><p>1. Add a WMS layer to the map (you can use the previous recipe in this chapter on QGIS</p><p>server if you have it running). Add an external source WMS, such as the USGS NAIP</p><p>Airphoto. (Here's the GetCapabilities URL, http://isse.cr.usgs.gov/arcgis/</p><p>services/Orthoimagery/USGS_EROS_Ortho_NAIP/ImageServer/WMSServe</p><p>r?request=GetCapabilities&service=WMS).</p><p>2. Re-export with the same settings:</p><p>www.it-ebooks.info</p><p>http://isse.cr.usgs.gov/arcgis/services/Orthoimagery/USGS_EROS_Ortho_NAIP/ImageServer/WMSServer?request=GetCapabilities&service=WMS</p><p>http://isse.cr.usgs.gov/arcgis/services/Orthoimagery/USGS_EROS_Ortho_NAIP/ImageServer/WMSServer?request=GetCapabilities&service=WMS</p><p>http://isse.cr.usgs.gov/arcgis/services/Orthoimagery/USGS_EROS_Ortho_NAIP/ImageServer/WMSServer?request=GetCapabilities&service=WMS</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>256</p><p>Now that you've created the Leaflet map, if you wanted to get into JavaScript</p><p>programming, all of the code that you need is in the directory produced, either</p><p>directly in index.html or in the js folder. In particular, you can see exactly</p><p>how layers are styled and added to the map.</p><p>You don't have to use Tiles or WMS for raster layers but this is recommended. If you do want to</p><p>use a local file, be warned there is a bug currently where some exports fail unless your raster</p><p>is converted to a .jpg format image in EPSG:4326 projection.</p><p>See also</p><p>f Don't forget to look at the documentation for the Leaflet JavaScript library on how to</p><p>customize your results after the export at http://leafletjs.com/.</p><p>f qgis2web plugin aims to combine qgis2leaflet and qgis2ol3 plugins (https://</p><p>plugins.qgis.org/plugins/qgis2web/), which means it also includes export</p><p>to OpenLayers 3 that is very similar to Leaflet but uses the OpenLayers JavaScript</p><p>library. Lizmap (http://www.3liz.com/en/lizmap.html) and QGIS Web Client</p><p>(https://github.com/qgis/QGIS-Web-Client) are two more popular options</p><p>that add more elaborate prebuilt interfaces but require a little more setup.</p><p>Managing GeoServer from QGIS</p><p>QGIS does not only serve as a frontend for the QGIS server, but it can also serve as a frontend</p><p>for other similar servers. GeoServer is one of the most popular ones, and you can configure</p><p>it from QGIS, upload layers, or even edit the style of a GeoServer layer using the QGIS</p><p>symbology tools.</p><p>Getting ready</p><p>For this recipe, you will need the GeoServer Explorer plugin. This can be installed using</p><p>Plugin Manager.</p><p>You will also need a running instance of GeoServer. We will assume that you have a local</p><p>one running on port 8080, but you can replace the corresponding URL with the one of the</p><p>GeoServer instance that you have available, whether local or remote.</p><p>How to do it…</p><p>1. Open the GeoServer Explorer by navigating to Web | GeoServer | Geoserver</p><p>Explorer. The explorer will appear on the right-hand side of the QGIS window.</p><p>www.it-ebooks.info</p><p>http://leafletjs.com/</p><p>https://plugins.qgis.org/plugins/ol3/</p><p>https://plugins.qgis.org/plugins/ol3/</p><p>http://www.3liz.com/en/lizmap.html</p><p>http://www.3liz.com/en/lizmap.html</p><p>https://github.com/qgis/QGIS-Web-Client</p><p>https://github.com/qgis/QGIS-Web-Client</p><p>http://www.it-ebooks.info/</p><p>Chapter 9</p><p>257</p><p>2. Click on the GeoServer Catalogs item in the explorer tree, and then select the New</p><p>catalog button.</p><p>3. Complete the fields in the dialog that will appear to define a new catalog and</p><p>click on OK:</p><p>4. The new catalog will be added to the explorer tree, and you can now browse</p><p>its content.</p><p>5. Open the zipcodes_wake.shp layer and style it.</p><p>6. In the QGIS Layer List, drag the entry corresponding to the zipcode_wake.shp layer</p><p>and drop it on the catalog item in the GeoServer Explorer. The layer will be uploaded</p><p>and added to the default workspace of the catalog.</p><p>7. You can check whether the layer is now in the catalog by opening a web browser</p><p>and going to the GeoServer web interface at http://localhost:8080/</p><p>geoserver/web/.</p><p>How it works…</p><p>The GeoServer Suite plugin communicates with GeoServer using its REST API. By linking QGIS</p><p>with the GeoServer REST API, it allows you to easily configure many elements that, otherwise,</p><p>should be configured manually, such as the styling of layers.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>QGIS and the Web</p><p>258</p><p>There's more…</p><p>The Geoserver Explorer plugin has a lot of features to work with GeoServer. Here are some</p><p>additional ideas so that you can explore them. For more information, check out the plugin</p><p>help at http://boundlessgeo.github.io/qgis-geoserver-plugin/.</p><p>Editing a remote style</p><p>Once the layer is in the GeoServer catalog, you can edit its style without having to upload the</p><p>layer again. Just open the Styles branch in the explorer tree under the corresponding catalog</p><p>and double-click on the style to edit, or select the Edit... item in the context menu that is</p><p>shown when right-clicking on the element:</p><p>The QGIS symbology dialog will be opened, and you can edit the style in there. Once you close</p><p>the dialog, the style will be uploaded and updated in the catalog.</p><p>Support for multiple formats</p><p>The GeoServer REST API only supports shapefiles for vector layers, but you can drag and drop</p><p>any layer in any format that is supported by QGIS. This plugin will take care of converting it</p><p>before uploading, in case this is needed.</p><p>See also</p><p>Don't have a Geoserver instance, it's pretty easy to setup for testing. See http://geoserver.</p><p>org/ for details.</p><p>www.it-ebooks.info</p><p>http://boundlessgeo.github.io/qgis-geoserver-plugin/</p><p>http://geoserver.org/</p><p>http://geoserver.org/</p><p>http://www.it-ebooks.info/</p><p>259</p><p>10</p><p>Cartography Tips</p><p>In this chapter, we will cover the following recipes:</p><p>f Using Rule Based Rendering</p><p>f Handling transparencies</p><p>f Understanding the feature and layer blending modes</p><p>f Saving and loading styles</p><p>f Configuring data-defined labels</p><p>f Creating custom SVG graphics</p><p>f Making pretty graticules in any projection</p><p>f Making useful graticules in printed maps</p><p>f Creating a map series using Atlas</p><p>Introduction</p><p>Cartography has changed quite a bit in the past decade as more people transition to purely</p><p>electronic map products on a device or on the Web. While some types of visualizations are</p><p>better suited to different media, many of the underlying tools and techniques can actually be</p><p>applied across the board. This chapter covers a variety of tools that enable you, the QGIS user,</p><p>to maximize the readability and beauty of your maps.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>260</p><p>Using Rule Based Rendering</p><p>In the past, if you wanted to apply a wildly different style to more than one type of data in</p><p>the same source, the only way to do this was to duplicate or subset a layer. With Rule Based</p><p>Rendering, you now just have to create rules that are applied on-the-fly. This opens a huge</p><p>door on cartographic possibilities with different features in the same layer not only having</p><p>different colors but also different fill types, transparency, line type, and all manner of other</p><p>customizations. Extending from categorized symbology, rules also allow for mixing and</p><p>inheritance, allowing for intermediate categories or some shared properties and reducing</p><p>the amount of work to create elegant symbology.</p><p>Getting ready</p><p>Rule Based Rendering is built-in to vector symbology. So, you'll need a good complicated</p><p>vector layer to fully utilize its potential. A road layer is often a good use case, but for this</p><p>example we'll go slightly simpler with busroutesall.shp.</p><p>How to do it…</p><p>1. Load the busroutesall.shp layer.</p><p>2. Right-click on the layer name in the Layers window, select Properties, then pick</p><p>Style on the left-hand side of the new window.</p><p>3. Change the symbology drop-down type to Rule-Based.</p><p>4. Pick the attributes that you want to use to differentiate between groups of features:</p><p>1. In this case, let's edit the initial rule (double-click on the rule or the Edit icon</p><p>between + (add) and - (remove).</p><p>2. Rules can be based on attribute table values or geometry properties,</p><p>including on-the-fly calculated values. First let's style routes shorter than</p><p>2,000 map units apply here. In the Filter box type $length 2000 (don't forget to test this).</p><p>3. Pick some other symbology that differs quite a bit so that it's easy to tell</p><p>them apart (such as a different line type). Click on OK and then click on</p><p>Apply to see to the two rules in action.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>262</p><p>6. Now, things get really interesting. Let's add a subrule by either right-clicking on a rule</p><p>or by highlighting a rule and clicking on the Refine current rules dropdown:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>263</p><p>7. Select Add categories to rule:</p><p>1. In the subdialog, select Route.</p><p>2. Pick a color ramp and/or line style, click on Classify, and then</p><p>click on OK:</p><p>3. Before you click on Apply, edit the main rule and uncheck the Symbol box</p><p>(otherwise, the Rule and Sub Rules list will be additive, which can be useful</p><p>in some cases).</p><p>8. Now, when you look at the Rule list, you will see subrules under their parents.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>264</p><p>9. Finally, let's add a third top-level rule that is not based on the length:</p><p>1. Make a rule filter on the ROUTE name that contains a. The rule will look like:</p><p>"Route" LIKE '%a'.</p><p>2. Pick a line symbol that will make these routes stick out even with their</p><p>current coloring and click on Apply:</p><p>10. Play around some more; there are all sorts of things you can do, from partial</p><p>string matching to splitting by even or odd numbers ("ROUTE" % 2 = 0 is</p><p>even-numbered).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>265</p><p>11. Finally, the map looks like the following:</p><p>How it works…</p><p>Each rule is processed in the rendering order specified from top to bottom, the last rule being</p><p>drawn last and, therefore, on top. The rules are added to any existing style that is already</p><p>applied to feature. You can change the rendering order by changing the rule order or by</p><p>applying a render order. The filters work just like attribute filters in the field calculator or</p><p>the table search. All of the symbology options are available to vectors and can be applied</p><p>to one or many rules. You can group rules by scale-rendering rules too.</p><p>There's more…</p><p>There are way too many possible ways to use Rule Based Rendering than can be described</p><p>here. You can create rendering groups that inherit rules from their parent and apply their own.</p><p>Each feature given a unique ID could have a completely different look. The big improvement</p><p>over using traditional single symbol, categorized, or graduated symbology is that you don't</p><p>have to edit every possible group, and you can more easily stack rules, mixing and matching</p><p>all the original methods.</p><p>There are some catches. Not everything you do with Rule Based Rendering is possible with</p><p>web services; so, before you go too crazy, consider your output format and test your ideas</p><p>before spending too much time on this.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>266</p><p>Handling transparencies</p><p>Transparency is a lack of pigment or color, such that you can see through one feature to the</p><p>feature beneath. You can think of this as being similar to tinted or stained glass; some light is</p><p>allowed to pass through and reflect off what's inside. When used right, transparency can help</p><p>emphasize or de-emphasize features in a map composition. It can also be used to blend two</p><p>layers to look as if they are one layer.</p><p>Getting ready</p><p>This recipe demonstrates transparency for both vectors and rasters, so we'll need an example</p><p>of each. The lakes.shp and elevlid_D782_6.tif layers will work well for demonstration</p><p>purposes. Load both of these layers in a fresh project, putting lake.shp on top.</p><p>How to do it…</p><p>1. With a vector layer loaded, open Properties and the Style tab.</p><p>2. On the right-hand side of the dialog, you will see a Transparency slider at 0%</p><p>(this means 100% solid or opaque).</p><p>3. Adjust the slider to the right and apply the changes to see them in the map:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>267</p><p>Using a bold or dark color will make it easier to notice the change. You will</p><p>also notice that the Simple fill option shows the original color.</p><p>4. Now to demonstrate this on a raster, first reset the lakes back to 0%.</p><p>5. Swap the order in the Layers list so that elevlid is on top.</p><p>6. Now, open Properties of elevid and the Transparency tab.</p><p>7. The Global transparency option will change the value evenly for the whole raster.</p><p>Set it to 50% and apply it. You should now be able to see the lakes layer, which was</p><p>hidden below:</p><p>No data value is always 100% transparent no matter what Global</p><p>transparency is set to. Use this to easily eliminate values that you</p><p>don't want to show up at all.</p><p>8. (Optional) You may have noticed that below the Global transparency slider is Custom</p><p>transparency options. This will allow you to make particular values more transparent</p><p>than others. You can either assign specific values to specific transparencies, or you</p><p>can add a band to the raster (or use a multiband raster), which specifies the amount</p><p>of transparency to apply to the rest of the raster (some data formats, such as GeoTiff,</p><p>call this an Alpha Transparency band):</p><p>� Reset the global back to 0% (otherwise, this is applied in too)</p><p>� Use the green + sign to add some values</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>268</p><p>� From 100 To 125, 25% transparent</p><p>� From</p><p>125 To 150, 75% transparent</p><p>� Click Apply and notice the lower elevation lakes are harder to see:</p><p>How it works…</p><p>This is really a computer graphics thing, but the simplest explanation is that you're telling the</p><p>computer to combine a percentage of two different layers in the same location instead of the</p><p>top layer's value covering. Based on the math of the original colors and their transparency,</p><p>a blended color is calculated for each pixel on the screen.</p><p>This doesn't begin to explain all the possible variations of appearance that can be achieved</p><p>by mixing multiple layers and multiple transparencies, only tinkering can show you this.</p><p>There's more…</p><p>One classic example of transparencies is to mix hillshades and airphotos. You can place either</p><p>layer on top and then adjust the transparency to let the other show through. Generally, you</p><p>would place the hillshade underneath in this case (but either can work). The end result is a</p><p>landscape that appears to have 3D relief, but it looks like an airphoto.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>269</p><p>Another classic example is to create a mask layer with a hole cut out around the region that</p><p>you want to emphasize. You now place the mask layer on top. Before adding transparency,</p><p>it blocks everything but the hole. Then, you slowly add transparency so that you can see</p><p>surrounding regions, but they are muted and stand out less. For this technique, try a black,</p><p>gray, or white fill for the mask layer. Each will have a slightly different look.</p><p>When styling vectors, you can apply different transparencies to different features in the same</p><p>layer if you use Rule Based Rendering. Each rule can have a different transparency value and</p><p>the entire layer can have yet another transparency modifier in the Layer Rendering section.</p><p>Lastly, keep in mind that not all output formats handle transparency well. In particular, be</p><p>careful using color gradients with transparencies when exporting to PDF. Generally, PNG</p><p>handles transparency, SVG may work or at least allow to you to edit the transparency after</p><p>export, unlike image formats.</p><p>Understanding the feature and layer</p><p>blending modes</p><p>In this recipe, we will look at the different layer and feature blending modes. Using these</p><p>tools, we can achieve special rendering effects, which you may already know from other</p><p>graphics programs.</p><p>Getting ready</p><p>To follow this recipe, you just need to load stamen.png and effect.png from our sample</p><p>data. Make sure that stamen (left-hand side in the following screenshot) is the lower layer</p><p>and effect (right-hand side in the following screenshot) is the upper layer. To test the feature</p><p>blending modes, load blending.shp:</p><p>(Background maps "Watercolor" and "Toner" by Stamen Design, under CC BY 3.0. Data</p><p>by OpenStreetMap, under CC BY SA).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>270</p><p>How to do it…</p><p>Using the two raster layers, we can try the different blending modes. Of course, this works for</p><p>vector layers, as well:</p><p>1. Double-click on the effects layer to open Layer Properties.</p><p>2. You can find the blending settings by going to Layer Properties | Style | Color</p><p>Rending together with other helpful controls for Brightness, Contrast, Saturation,</p><p>and more, as shown in the next screenshot:</p><p>3. Change the Blending mode and click on Apply to see the results.</p><p>4. Similarly, for vector layers, such as our blending layer, we find the blending mode</p><p>settings by going to Layer Properties | Style | Layer rendering, as shown in</p><p>the following screenshot:</p><p>The main difference is that, for vector layers, we can control how features are blended</p><p>together, and how the result is then blended to the underlying layers using the Feature</p><p>blending and Layer blending modes, respectively. The feature blending mode is applied</p><p>on a per-feature-basis.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>271</p><p>The following screenshot shows the differences between feature and layer blending:</p><p>Feature and/or layer blending in action (from left to right): feature blending only, layer blending only,</p><p>feature and layer blending combined (background maps "Watercolor" and "Toner" by Stamen Design,</p><p>under CC BY 3.0. Data by OpenStreetMap, under CC BY SA).</p><p>The following is an explanation of the preceding screenshots:</p><p>f The leftmost image shows that Feature blending mode is set to Multiply, while Layer</p><p>blending mode is set to default, Normal. This results in a map where the vector</p><p>features are rendered on top of each other using the Multiply mode before the</p><p>whole layer is overlaid on top of the lower layer(s).</p><p>f The center image instead shows Normal Feature blending mode combined with</p><p>Multiply Layer blending mode. You can see how the features can block each other</p><p>out because they are drawn on top of each other.</p><p>f Finally, the rightmost image shows both Layer blending mode and Feature blending</p><p>modes being set to Multiply. In this combination, the Multiply rule is applied on both</p><p>the feature and the layer level and, therefore, we can see features and the underlying</p><p>background layer(s) shining through the features in the upper layer.</p><p>How it works…</p><p>Based on the selected blending mode, the pixel colors (in the RGB mode) of the lower and</p><p>upper layers are mixed as described next. For illustration and quick reference, the following</p><p>figure shows the results of all 12 blending modes (from left to right and top to bottom), except</p><p>for the Normal setting, which does not mix the colors but only uses the alpha channel of the</p><p>upper layer to blend with the layer below it:</p><p>f Lighten: The Lighten mode selects the maximum of each RGB component from</p><p>the foreground and background pixels. Be aware that the results tend to be jagged</p><p>and harsh.</p><p>f Screen: The Screen mode paints light pixels from the upper layer over the lower layer,</p><p>but it skips the dark pixels.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>272</p><p>f Dodge: The Dodge mode will brighten and saturate the lower layer based on the</p><p>lightness of the upper layer. This means that brighter colors in the upper layer cause</p><p>the saturation and brightness of the lower layer to increase. This works best if the top</p><p>pixels aren't too bright; otherwise, the effect is quite extreme.</p><p>f Addition: The Addition mode adds the pixel values of both layers. If the result</p><p>exceeds 1 (in the case of RGB), the respective areas are displayed in white.</p><p>f Darken: The Darken mode creates a result that retains the smallest RGB</p><p>components of both layers. Therefore, this is the opposite of the Lighten mode</p><p>and, just as with Lighten, the results tend to be jagged and harsh.</p><p>f Multiply: The Multiply mode multiplies the values of both layers, thus resulting</p><p>in a darker picture.</p><p>f Burn: The Burn mode causes darker colors in the upper layer to darken the lower</p><p>layer. Burn can be used to tweak and colorize underlying layers.</p><p>f Overlay: The Overlay mode combines the Multiply and Screen blending modes. As a</p><p>result, light parts become lighter and dark parts become darker.</p><p>f Soft light: The Soft light mode is very similar to Overlay, but it uses a combination</p><p>of Burn and Dodge. This is supposed to emulate shining a soft light on an image.</p><p>f Hard light: The Hard light mode is also very similar to Overlay. It is supposed to</p><p>emulate projecting a very intense light on an image.</p><p>f Difference: The Difference mode subtracts the values of the upper layer from the</p><p>lower layer (or the other way around) to always get a positive value. Blending with</p><p>black (which has an RGB value of 0,0,0) produces no change.</p><p>f Subtract: The Subtract mode subtracts the values of one layer from the other.</p><p>In the case of negative values, black is displayed:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>273</p><p>Overview of the 12 blending modes (background maps "Watercolor" and "Toner" by Stamen Design, under CC BY 3.0.</p><p>Data by OpenStreetMap, under CC BY SA): first row: Lighten, Screen, and Dodge;</p><p>second row: Addition, Darken,</p><p>and Multiply; third row: Burn, Overlay, and Soft light; fourth row: Hard light, Difference, and Subtract.</p><p>Saving and loading styles</p><p>What's better than making an awesome style for your feature layers? Being able to easily</p><p>share and reuse them. Both vector and raster styles can be saved and reused—however,</p><p>in slightly different ways.</p><p>Getting ready</p><p>For this recipe, you need two similar vector layers and a set of two similar raster layers. In the</p><p>example data that is provided, use two of the bus route shapefiles and two of the elevation</p><p>rasters (for example, elevlid_D782_6.tif).</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>274</p><p>How to do it…</p><p>First we'll start by copying and pasting styles for vector layers:</p><p>1. Load up two bus route shapefiles and two elevlid rasters.</p><p>2. The simplest method is to copy styles for vectors or rasters. Just right-click on the layer</p><p>name in the list and select Copy Style from the Style menu. Then, right-click on the</p><p>layer that you want to apply this to and select Paste Style from the Style menu. You</p><p>can only copy styles between layers of the same type (for example, Point to Point):</p><p>Try to copy and paste the style of one bus route to the second bus route using the</p><p>right-click menus:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>275</p><p>3. Now to export styles for later use, right-click on a layer and navigate to Properties |</p><p>Symbology.</p><p>4. In the bottom-right, there is a Save Style button in the Style menu. The output</p><p>choices are QGIS Layer Style File… (aka .qml) or SLD File.... Both formats are</p><p>XML text files, QGIS Layer Style... is recommended for maximum compatibility:</p><p>5. SLD is compatible with some other web map systems, but it will not capture your</p><p>QGIS style 100% except in the simplest cases (not all the same options exist in SLD).</p><p>QML is the native QGIS style file. Note the Load style option for later usage.</p><p>6. Go ahead and apply a new style to one of the bus route layers.</p><p>7. Then, save the symbology to QGIS Layer Style File (qml).</p><p>8. In the property dialog of the second bus route file, click Load Style... and pick the file</p><p>that you just saved.</p><p>You can open this file in a text editor and make customizations or,</p><p>for example, batch-find and replace values.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>276</p><p>Rasters are slightly trickier in that you can save a symbology file, but you can also save a color</p><p>table. The color table is a text file that lists raster value ranges and associated color codes.</p><p>It's a much simpler format to hand-edit than XML (QGIS Layer Style File), but does not retain</p><p>things like transparency or classification settings:</p><p>1. Go ahead and apply a new color gradient to one of the elevlid layers.</p><p>2. Save just the color table to a .txt file with the disk button (above the color table on</p><p>the right end of the button row; refer to the screenshot).</p><p>3. In the property of the second elevid file, load the color table and pick the file that you</p><p>just saved.</p><p>4. Apply the changes to your layer style:</p><p>Note that the same Style menu is available as was in the vector</p><p>properties. You can use this to save and load QGIS Layer Style File (.qml)</p><p>just as we did earlier.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>277</p><p>How it works…</p><p>Normally the style information for a layer is saved in the .qgs (if you save your project)</p><p>project file. The various export methods just package up the style information for a layer into</p><p>a separate file in a generic manner (not associated with the original data). This lets you apply</p><p>similar styles to similar data sources.</p><p>Vector symbology is stored in a special XML file that ends in the .qml extension. You can read</p><p>or edit the file if you want, and it can be produced via scripts or copied and pasted to create</p><p>mashups of multiple styles.</p><p>Raster symbology can also be stored in a .qml file. However, there's an additional option to</p><p>export the classification ranges and colors to a simple text file, one value or range of values</p><p>and one color code per line.</p><p>There's more…</p><p>The second format SLD (Style Layer Descriptors) is very common in web services. While not</p><p>all features of QGIS styling have equivalents in SLD, it's still a good starting point to share your</p><p>style across software platforms such as Mapserver or Geoserver.</p><p>Configuring data-defined labels</p><p>If there was a list of top features of QGIS, data-defined labels would be high on that list. They</p><p>offer the ease of automatic labeling with the customization of freehand labeling. You can mix</p><p>and match automatic and custom edits, storing the values in a table for later reference.</p><p>Getting ready</p><p>There are a couple of useful plugins for data-defined labeling which will add the extra attribute</p><p>fields that you need to either an existing layer or make a new layer just for labels. Download</p><p>and install Layer to labeled layer and Create labeled layer.</p><p>How to do it…</p><p>1. Open QGIS and load census_wake2000.shp.</p><p>2. Create a copy of the layer using the Save As dialog, and save the layer as census_</p><p>wake2000_label.shp. (You don't always have to do this but this process does</p><p>modify the table, so it's a good idea to make a backup.)</p><p>3. Highlight census_wake2000_label.shp in the layer list.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>278</p><p>4. Run the Layer to labeled layer plugin (Plugins | Layer to Labeled layer plugin):</p><p>1. Set Label Field to STFID.</p><p>2. Click on OK:</p><p>5. If you look at the attribute table now, you will see a whole bunch of new fields,</p><p>starting with the Lbl prefix, which are NULL:</p><p>6. Now, ensure that you have the Label toolbar open (View | Toolbars | Label):</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>279</p><p>7. Either in the layer (by navigating to Properties | Labels) or using the first button on</p><p>Label Toolbar, Layer Labeling Options, open the label management dialog.</p><p>8. Throughout the dialogs, you will see markers next to each field. A yellow one indicates</p><p>a data-defined attribute, a white marker is the same setting for all:</p><p>If you want to control additional attributes at this point, add a new</p><p>field to the layer. Then, return to this dialog and select the white</p><p>icon to pick the name of the field to use.</p><p>9. Now, you are ready to make custom edits to various labels and have the table store</p><p>the settings. Depending on the setting, there are a couple of ways to make the edits.</p><p>Note that you must toggle editing on the layer before you can change the labels:</p><p>1. You can edit the field directly in the table either by hand, or you can use the</p><p>field calculator to automate repetitive patterns (for example, give all major</p><p>roads the same Font and Color label).</p><p>2. For some attributes, such as X,Y and rotation, you can also edit by hand in</p><p>the map using the Label Toolbar option.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>280</p><p>Example: moving and rotating a label</p><p>1. Toggle editing by clicking on the following icon:</p><p>2. On the Label Toolbar menu, select the Move Label button. Now, click on a label and</p><p>drag it to a new location, releasing the mouse button when you are done moving the</p><p>label. Note that you must ensure that the X and Y fields in step 38 are set for this tool</p><p>to be usable:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>281</p><p>If you check the attribute table you will see that in the LblX and</p><p>LblY fields, the values have now been saved for the labels that</p><p>you moved.</p><p>3. Now, try the Rotate Label button. See if you can make some of the labels fit inside</p><p>their polygons using the move and rotate:</p><p>You can also use the Change Label button to edit the other properties of a specific</p><p>label that you select. This is really nice when you just need some fine-tuning.</p><p>4. Save your edits and toggle editing off to keep your changes.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>282</p><p>How it works…</p><p>The basic premise is that you keep an extra</p><p>set of attributes in a table often as additional</p><p>fields to your existing table.</p><p>You could add fields to your attribute table by hand, and assign them</p><p>to label properties. Using the Layer to Labeled Layer plugin does this</p><p>for you.</p><p>These fields if you set them are used in determining the location, size, font, color, angle, and</p><p>so on, of the label for the given row. If you don't set them, then the automatic settings from</p><p>the labeling engine are kept.</p><p>There's more…</p><p>Data-defined labels are powerful in that you can combine automated, calculated, and custom-</p><p>edited values. They are automated from the built-in labeling engine and calculated using</p><p>the field calculator to populate the data-defined fields (for example, with if statements</p><p>or calculations that are based on other attributes). Lastly, by just making these little hand</p><p>tweaks, you can fix a few not-quite labels that misbehave.</p><p>Note that you don't have to use data-defined labeling on an existing layer. You can create just</p><p>a label layer with the Create labeled layer plugin. In other software, user-defined labeling</p><p>is often called Annotation layers. QGIS also has annotation layers. These are layers where</p><p>you click to add a label to the map and then write and style it however you want. The biggest</p><p>problem is that these layers are not associated with the data that they label. You can't easily</p><p>give them to someone else, and if a label name or style changes, you have to chase down</p><p>and hand-edit every fix. In QGIS, data-defined labeling solves almost all the shortcomings of</p><p>annotation layers because it actually saves to a shapefile with all its properties as fields.</p><p>Creating custom SVG graphics</p><p>This recipe is all about making your map unique by creating custom icons, north arrows, or</p><p>even fill patterns.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>283</p><p>Getting ready</p><p>You will need a vector illustration program (for example, Inkscape or Adobe Illustrator).</p><p>Don't have one? There are several free and open source options available on</p><p>all platforms. Many people in the QGIS community use Inkscape (http://</p><p>inkscape.org), but you can also use LibreOffice Draw or OpenOffice Draw.</p><p>The most common proprietary software equivalent is Adobe Illustrator.</p><p>You will also need a text editor, such as TextEdit (Mac), Notepad, Notepad++ (Windows).</p><p>How to do it…</p><p>1. Open up your vector illustration program.</p><p>2. Set the canvas to a reasonable size to work with. Square ratios tend to work well</p><p>because the icon will eventually be used to mark points in QGIS; 100x100 pixels</p><p>is fine.</p><p>3. Draw a simple shape such as a square, circle, or star. Make sure you go most of the</p><p>way towards the edges and fill the whole page.</p><p>Remember that you will be using this drawing at sizes closer to 8-32 pixels;</p><p>it's just really annoying to work at these scales. when creating and editing</p><p>illustrations</p><p>4. Save the drawing as an .svg file.</p><p>5. Now, open the .svg file in a text editor, search and find the style line of your object,</p><p>and replace it with the following lines:</p><p>stroke-width="param(outline-width) 1"</p><p>stroke="param(outline) #000"</p><p>fill="param(fill) #FFF"</p><p>If working with a complex icon, set your line to a specific color code that is</p><p>different from all other colors in the drawing. Make a note of the color code so</p><p>that you can use it to search the .svg file in your text editor.</p><p>www.it-ebooks.info</p><p>http://inkscape.org</p><p>http://inkscape.org</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>284</p><p>The before-after scenario when this code has been incorporated is shown in the</p><p>following table:</p><p>Before After</p><p>6. Save your changes.</p><p>7. Now, start up QGIS and load a point layer.</p><p>8. Go to Properties | Style.</p><p>9. In the symbology options, there are two levels of objects that make a symbol: the</p><p>marker and then a sublevel of actual symbols that combine to make it.</p><p>10. Select the subobject, which is usually labeled Simple Marker by default.</p><p>11. Now, change the dropdown in the upper-right to SVG Marker.</p><p>12. Below the box displaying the symbol options look for the … button and select to load</p><p>an SVG from file. Use this to select the .svg file that you previously created.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>285</p><p>13. Once imported, you should be able to change the fill color of the symbol (if you</p><p>performed Step 5):</p><p>You may need to adjust the size and widths in large amounts for changes</p><p>to be apparent. Make use of the Apply button to see the changes in the</p><p>map but keep the dialog open for easy adjustment.</p><p>How it works…</p><p>The special text that you add to the .svg file is a marker or placeholder. QGIS looks for these</p><p>particular words and then utilizes them to insert symbol changes on-the-fly as the SVG is read</p><p>into the program.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>286</p><p>There's more…</p><p>While this recipe demonstrated a very simple SVG, this method applies to more complicated</p><p>symbols.</p><p>Also note that in Settings | Options | System, you can set paths to folders of SVGs so that</p><p>all of them will be available in the symbology dialogs all the time.</p><p>See also</p><p>QGIS also lets you customize fill patterns using SVG symbols. The QGIS Training Manual has</p><p>a good example of this at http://docs.qgis.org/2.8/en/docs/training_manual/</p><p>basic_map/symbology.html#hard-fa-creating-a-custom-svg-fill.</p><p>Making pretty graticules in any projection</p><p>A graticule is a set of reference lines on a map that help orient a map reader. They are often</p><p>set at, and labeled, with the coordinates. The tricky part about using graticules, however,</p><p>is projections. If you don't make them correctly, instead of smooth curves between the line</p><p>intersections, you get awkward unusual shapes (mostly straight lines). The default QGIS</p><p>graticule creator is not projection-friendly, so in this recipe, you'll see an add-on processing</p><p>algorithm that does this. This recipe is about ensuring you get nice, smooth, and properly-</p><p>labeled graticules.</p><p>Getting ready</p><p>You don't really need much for this recipe other than a bounding box and a coordinate interval</p><p>that you want to space the lines at. Usually, these will be in Latitude, Longitude WGS 84</p><p>(EPSG:4326), and decimal degrees, respectively, since the whole point of a graticule is</p><p>to add reference lines that help orient a user.</p><p>How to do it…</p><p>1. Start by downloading a Processing Toolbox algorithm specifically for this task called</p><p>Lines Graticule:</p><p>1. Open the Processing Toolbox.</p><p>2. Go to Scripts | Tools | Get scripts from on-line scripts collection:</p><p>www.it-ebooks.info</p><p>http://docs.qgis.org/2.8/en/docs/training_manual/basic_map/symbology.html#hard-fa-creating-a-custom-svg-fill</p><p>http://docs.qgis.org/2.8/en/docs/training_manual/basic_map/symbology.html#hard-fa-creating-a-custom-svg-fill</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>287</p><p>3. In the Not Installed section, check the box for the Lines Graticule algorithm.</p><p>4. Click on the OK button to install the algorithm.</p><p>Every time that you use a tool, it's good to check for updates.</p><p>You will see something like the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>288</p><p>2. Now that you've downloaded the algorithm, open it by navigating to Scripts | Vector</p><p>(it's called Lines graticule though the code is actually pygraticule.py):</p><p>3. You can fill in the parameters by hand if</p><p>you know them or use the … button to get</p><p>values from your existing project.</p><p>4. For now, you can use the defaults that will make a graticule for the whole world. The</p><p>outputs are determined by outfile and graticule. These parameters are optional, you</p><p>can choose to pick one, both, or neither. If you want a GeoJSON file, set the outfile. If</p><p>you want a shapefile, set the graticule (if you want the results to autoload afterwards,</p><p>make sure that the second output is set to temporary or a real file, just not blank).</p><p>Refer to the Help tab for details about each parameter. There are two really important</p><p>values to control the graticule:</p><p>1. The spacing value denotes how often to draw a line (when doing world-scale</p><p>maps, 20 or 30 degrees works well).</p><p>2. The density value denotes how often to put nodes:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>289</p><p>The more nodes, the smoother the curves; however, you get a bigger file</p><p>that takes longer to make. Picking the right density may require trial and</p><p>error to find the largest density before you notice the lines stop curving</p><p>smoothly for a given map scale.</p><p>5. Once you've chosen your settings, click on Run.</p><p>6. After it runs, a vector layer should get loaded with the results. This won't look all that</p><p>exciting, just straight lines making a grid.</p><p>7. The real magic is to now enable projection on-the-fly with one of the many decent</p><p>world-wide projections such as "World Robinson (EPSG:54030):</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>290</p><p>8. (Optional) If it doesn't look like the image, but instead still has straight lines that are</p><p>oddly spaced, you need to disable the QGIS rendering simplification:</p><p>1. Pick the layer from Properties | Rendering.</p><p>2. Make sure that Simplify geometry is disabled:</p><p>9. (Bonus) Generate a vector grid from Vector | Research tools. The difference looks</p><p>like the following:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>291</p><p>How it works…</p><p>Graticules are basically line layers (though sometimes they are also polygons). If you draw a</p><p>grid with nodes only at the points where two lines intersect, you can easily see how distorting</p><p>the grid will lead to blocky shapes. The key to smooth graticules is adding additional line</p><p>nodes in between the intersections (that is, increase the node density).</p><p>It's important to note that, when using projections that don't cover the whole world</p><p>(for example, polar or stereographic projections), pick bounding box values that fall</p><p>within the projection limits; otherwise, you may get errors when trying to reproject.</p><p>There's more…</p><p>The primary advantages of graticules in the main map canvas are that you can use them</p><p>as references while working in QGIS, include them in web and digital maps, and have full</p><p>control of the labels and symbology. The method used here differs from other graticule (grid)</p><p>tools in QGIS because it focuses on putting Latitude/Longitude lines with smooth curves as</p><p>references into any projection. Other grid tools focus more on making regular squares across</p><p>a map to subdivide a region.</p><p>The main advantages of the print composer method (next recipe) are its ability to make</p><p>multiple coordinate systems easily and to add tick marks around the outside edge of a map.</p><p>Tick marks are what you commonly see on navigation-oriented maps, such as USGS Topo</p><p>quads, and other printed maps.</p><p>See also</p><p>Lines graticule (aka Pygraticule) can also be used as a pure Python script; for updates and</p><p>more information, refer to https://github.com/wildintellect/pyGraticule.</p><p>To learn how to write your own processing toolbox algorithms, refer to the Writing processing</p><p>algorithms recipe in Chapter 11, Extending QGIS.</p><p>Making useful graticules in printed maps</p><p>A graticule is a set of reference lines on a map that help orient a map reader. They are</p><p>often set at and labeled with the coordinates. For traditional printed maps that are intended</p><p>for navigation and surveying tasks where you want to mark the geographic coordinates,</p><p>sometimes in multiple coordinate systems. This recipe is about adding such reference</p><p>lines to a Print Composer map.</p><p>www.it-ebooks.info</p><p>https://github.com/wildintellect/pyGraticule</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>292</p><p>Getting ready</p><p>You will need a map, typically of a small area (several miles or km across). For this recipe,</p><p>elevlid_D782_6.tif works well.</p><p>How to do it…</p><p>1. Load elevlid_D782_6.tif.</p><p>2. Turn on Projection on-the-fly by selecting UTM Zone 17 N, WGS 84 (EPSG:32617).</p><p>3. Now create New Print Composer.</p><p>4. In Print Composer, select Add New Map, and then draw a rectangle on the canvas.</p><p>5. Now that you have the map, in the dialogs on the right-hand side of the screen select</p><p>the Item Properties tab.</p><p>6. Scroll down or collapse sections until you see the Grids section.</p><p>7. Use the green plus (+) symbol to add a new grid.</p><p>8. Now, edit Interval X and Interval Y to 1,000 map units. (Make sure to tab to the next</p><p>field or click on Enter for the values to stick.):</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>293</p><p>The current map units are UTM-based, meters, which means the</p><p>lines will be 1,000 meters or 1 km apart.</p><p>9. Just below the Interval section, change Line Style and make the lines red so that</p><p>they are easier to see.</p><p>10. Now, scroll down even further to the Draw coordinates section and check the box to</p><p>enable labels for the grid lines:</p><p>11. Once this is enabled, change the top and bottom orientation to vertical, and change</p><p>the font color to red.</p><p>12. Now, to create a second grid, scroll back up to the Grids section and click the green</p><p>plus sign (+) again to add a second grid.</p><p>13. This time, change the CRS (Coordinate Reference System) to WGS 84 (EPSG:4326),</p><p>which is the most common Latitude and Longitude system that people use.</p><p>14. Make this grid be spaced 0.01 map units (that is, degrees in this case) and change</p><p>the style to blue to contrast with the other grid.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>294</p><p>15. Now, scroll down and add Draw coordinates. Also, make the top and bottom</p><p>vertical-oriented so that they avoid the first grid. You can also change the font</p><p>color to match the lines:</p><p>How it works…</p><p>Reference graticules are evenly spaced lines with marked coordinates. Based on your settings</p><p>the composer calculates the positioning of the lines from the map data coordinates. The key</p><p>to making useful graticules in the print composer is to select intervals that are often enough</p><p>to provide reference but not so often that they cover a large portion of the map. It's also</p><p>important to pick intervals that have nice rounded numbers, so that it's easy to calculate the</p><p>value half way between two lines.</p><p>There's more…</p><p>There are two ways to make grids/graticules in QGIS: the print composer for printed maps or</p><p>as a layer in QGIS for printed web maps as an internal usage.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>295</p><p>The main advantages of the print composer method are the ability to do multiple coordinate</p><p>systems easily and to add tick marks around the outside edge of a map. Tick marks are what</p><p>you commonly see on navigation oriented maps, such as USGS Topo quads.</p><p>The primary advantages of graticules in the main map canvas are that you can use them as</p><p>references while working with QGIS and have full control of the labels and symbology. Refer to</p><p>the Making pretty graticules in any projection recipe in this chapter for how to make graticules</p><p>in the main map interface.</p><p>Creating a map series using Atlas</p><p>In this recipe, we will use the Print Composer Atlas functionality to automatically create a PDF</p><p>map book with a series of maps.</p><p>Getting ready</p><p>To follow this recipe, load zipcodes_wake.shp and geology.shp from our sample data. In</p><p>the following screenshots, the zipcodes_wake layer was styled with a simple white border,</p><p>while the geology layer is styled with random colors.</p><p>How to do it…</p><p>The Print Composer Atlas feature</p><p>the sections displayed in the left-hand side, select the Metadata section. You will</p><p>see a box containing all the information corresponding to the layer data origin:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>10</p><p>Functionality provided by the GDAL library, which (mentioned earlier) acts as a provider for</p><p>raster layers, is also available in the Raster menu. This includes processing and data analysis</p><p>methods, but it also includes the information tool that is used to describe a raster data</p><p>source. You will find it by navigating to Raster | Miscellaneous | Info:</p><p>See also</p><p>f This is a more complex way to retrieve properties as you can call the tool by adjusting</p><p>the parameters with more details to get additional information. To know more, check</p><p>the gdalinfo help page at http://www.gdal.org/gdalinfo.htm .</p><p>Importing data from text files</p><p>Data can be imported from text files, providing some additional about how the geometry</p><p>information is stored in the text. This recipe shows you how to create a new points layer,</p><p>based on a text file.</p><p>www.it-ebooks.info</p><p>http://www.gdal.org/gdalinfo.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>11</p><p>How to do it…</p><p>1. Select the Add delimited text layer menu entry from the Layer menu. You will see a</p><p>dialog like the following one:</p><p>2. In the upper field, enter the path to the elev_lid792_randpts.csv file in the</p><p>sample dataset. That file contains a points layer as text.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>12</p><p>3. Once you enter the file path or select it in the file browser that can be opened by</p><p>clicking on the Browse button, the fields in the lower part of the dialog will be filled,</p><p>as shown in the following screenshot:</p><p>We are using a CSV file that has values separated by commas, so you</p><p>must select the CSV option in the Format field.</p><p>The X field and Y field drop-down lists will be populated with the fields that are</p><p>available, which are described in the first line of the text file. Select X for X field</p><p>and Y for Y field. Now, QGIS knows how to create the geometries and has enough</p><p>information to create a new layer from the text file.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>13</p><p>4. Enter a name for the layer in the Layer name field and click on OK. The layer will be</p><p>added to the QGIS project, as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>14</p><p>5. No information about the CRS is contained in the text file or entered in the</p><p>parameters dialog, so it must be added manually. In this case, the CRS used is</p><p>EPSG:3358. To set this as the CRS of the layer, right-click on the layer name and</p><p>select Set layer CRS:</p><p>6. In the CRS selection dialog, select the EPSG:3358 CRS and click on OK. The layer</p><p>now has the correct CRS.</p><p>How it works…</p><p>Data is read from the text file and processed to create geometries. All the fields in the table</p><p>(all data in a row in the text file) are also added, including the ones used to create the</p><p>geometries, as you will see by right-clicking on the layer and selecting Open attribute table,</p><p>as shown in the following screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>15</p><p>Along with the CSV file, this file may contains a CSVT file, which describes the types of the</p><p>fields. This is used by QGIS to set the appropriate type for the attributes table of the layer. If</p><p>the CSVT file is missing, as in our example's case, QGIS will try to figure out the type based on</p><p>the values for each field.</p><p>There's more…</p><p>Layers created from text files are not restricted to point files. Any geometry can be created</p><p>from the text data. However, if it is not a point, instead of selecting two columns, you must</p><p>place all the geometry information in a single one and enter a text representation of the</p><p>geometry. QGIS uses the Well-Known Text (WKT) format, which is a text markup language for</p><p>vector geometries, to describe geometries as strings. Here is an example of a very simple CSV</p><p>file with line features and two attributes:</p><p>geom,id,elevation</p><p>LINESTRING(0 1, 0 2, 1 3),1,50</p><p>LINESTRING(0 -1, 0 -2, 1 -3),2,60</p><p>LINESTRING(0 1, 0 3, 5 4),3,70</p><p>See also</p><p>f To know more about the WKT format, you can go to http://en.wikipedia.org/</p><p>wiki/Well-known_text</p><p>www.it-ebooks.info</p><p>http://en.wikipedia.org/wiki/Well-known_text</p><p>http://en.wikipedia.org/wiki/Well-known_text</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>16</p><p>Importing KML/KMZ files</p><p>KML and KMZ files are used and produced by Google Earth and are a popular format. This</p><p>recipe shows you how to open them with QGIS.</p><p>How to do it…</p><p>1. To open a KML layer, select Layer/Add vector layer.... In the dialog that opens, click</p><p>on the Browse button to open the file selector dialog. Select the Keyhole Markup</p><p>Language (KML) format and then select the file that you want to load. In the example</p><p>dataset, you can find several KML files. Select the elcontour1m.kml file. Click on</p><p>OK in the vector layer selector dialog, and the layer will be added to your project, as</p><p>shown in the following screenshot:</p><p>KMZ files can also be opened in QGIS.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>17</p><p>2. Go to Layer | Add vector layer.... In the dialog that opens, click on the Browse button</p><p>to open the file selector dialog. Select the All files option to view all the files and then</p><p>select the elcontour1m.kmz file. There is not a KMZ file type defined in QGIS, but</p><p>QGIS supports it because the underlying OGR library can read KMZ files as well.</p><p>3. Click on OK on the open layer dialog to open the selected layer.</p><p>From the layers contained in the KMZ file, you must select one of them. In this case, only a</p><p>layer is contained in the elcontour1m.kmz file, so it is loaded automatically. The layer will</p><p>be added to your QGIS project.</p><p>How it works…</p><p>KMZ files are compressed files that contain a set of layers. When you select it, the OGR library</p><p>will unzip the content of this file and then open the layers that it contains.</p><p>If just a single layer is contained, you will not see the layer selection dialog. QGIS will</p><p>automatically open the only layer in the KMZ file.</p><p>There's more…</p><p>As KMZ is not recognized as a supported format, the KMZ file will not appear in the QGIS</p><p>browser. However, the browser supports zipped files, and a KMZ file is actually a zipped file</p><p>with KML files inside it. Unzip it in a folder and then you will be able to use the QGIS Browser</p><p>to open the layers it contains.</p><p>Importing DXF/DWG files</p><p>CAD files, such as DXF and DWG files, can be opened with QGIS. This recipe shows you how to</p><p>do this.</p><p>How to do it…</p><p>1. To open a DXF layer, select Add vector layer... in the Layer menu. In the dialog that</p><p>opens, click on the Browse button to open the file selector dialog. Select the Autocad</p><p>DXF format and then the file that you want to load.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>18</p><p>2. In the example dataset, you can find several DXF files. Select the Wake_</p><p>ApproxContour_100.dxf file. Click on OK in the vector layer selector dialog and</p><p>the layer will be added to your project, as shown in the following screenshot:</p><p>How it works…</p><p>DXF files are read as normal vector layers although they do not have the same structure</p><p>as a regular vector layer as they do not allow adding arbitrary attributes to each geometry.</p><p>There's more…</p><p>The example DXF file that you opened contained just one type of geometry. DXF files can,</p><p>however, contain several of them: in this case, they cannot be added to QGIS in one layer.</p><p>When this happens, QGIS will ask you to select the type of geometry that you want to open.</p><p>In the sample dataset, you will find a file named CSS-SITE-CIV.dxf. Open it and you will</p><p>see the following dialog:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>19</p><p>Select one of the available geometries, and a layer will be added to your QGIS project.</p><p>Opening DWG files</p><p>DWG is a closed format of Autodesk. This</p><p>will create one map for each feature in the so-called</p><p>Coverage layer. In this recipe, the zipcodes layer will serve as a Coverage layer, and we will</p><p>create one map for each zipcode feature:</p><p>1. Click on the New Print Composer button or press Ctrl + P to get started. You will be</p><p>prompted to set a title for the new composer. This can be left empty if you want QGIS</p><p>to generate a title automatically.</p><p>2. Click on the Add new map button and drag open a rectangle on the composer page</p><p>to create a map item for the main map.</p><p>3. To activate the Atlas functionality, we enable the map item's Controlled by atlas</p><p>checkbox. The following screenshot shows the fully configured map's item properties.</p><p>In the Controlled by atlas section, we can select which zoom mode Atlas should use:</p><p>1. Margin around feature: This is the most flexible option, which tells Atlas to</p><p>zoom to the feature while keeping the specified margin percentage around</p><p>the feature.</p><p>2. Predefined scale (best fit): This tells Atlas to use the one predefined project</p><p>scale (configurable in Project Properties | General | Project scales) where</p><p>the feature best fits in.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>296</p><p>3. Fixed scale: This keeps the same scale for all maps of the series; the scale</p><p>is configured in the map's Main properties, that is, 100,000 in the following</p><p>screenshot:</p><p>4. Next, we add a label for the title using the Add new label button. This title label will</p><p>display the zip code polygon feature's NAME value which will be automatically updated</p><p>by Atlas for each map in the series. To achieve this, we insert the following expression</p><p>in the input field of the label item's Main properties:</p><p>[%attribute( @atlas_feature, 'NAME' ) %]</p><p>5. To finalize the Atlas configuration, we need to go to the Atlas generation tab.</p><p>There, we first have to enable the Generate an atlas checkbox. This activates</p><p>the Configuration section, where we can pick the Coverage layer and set it to</p><p>the zipcodes_wake layer, as shown in the following screenshot.</p><p>6. To preview the Atlas output, we can now click on the Preview Atlas button. This</p><p>button is only active if the Generate an atlas checkbox in the Atlas generation tab is</p><p>enabled. Once the preview mode is active, you can step through the map series using</p><p>the arrow buttons right besides the Preview Atlas button.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 10</p><p>297</p><p>7. When we are happy with the preview, we can export the map series. The output</p><p>behavior is controlled by the configuration in the Atlas generation tab's Output</p><p>section, which you can also see in the following screenshot. Atlas supports exporting</p><p>to separate image, SVG, or PDF files. Activate the Single file export when possible</p><p>option to combine all maps into one PDF and click on the Export Atlas as PDF</p><p>button, as shown in the following screenshot:</p><p>How it works…</p><p>The Atlas feature provides access to a series of variables related to the current feature. We</p><p>already used this to display the NAME value of the current feature in the title label using the</p><p>[%attribute( @atlas_feature, 'NAME' ) %] expression. Besides @atlas_feature,</p><p>you have access to the following variables:</p><p>f @atlas_feature: This is the feature ID of the current Atlas feature. This makes it</p><p>possible to use this information in rules to, for example, hide or highlight features</p><p>based on their ID.</p><p>f @atlas_geometry: This is the geometry of the current Atlas feature and can be</p><p>used in rules to, for example, only show features of other layers if their geometry</p><p>intersects the Atlas feature geometry.</p><p>f @atlas_featurenumber: This is the number of the current Atlas feature.</p><p>f @atlas_totalfeatures: This is the total number of features in the Atlas</p><p>coverage layer.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Cartography Tips</p><p>298</p><p>There's more…</p><p>Overview maps are a great way to provide context to more detailed main maps. To add</p><p>an overview map (as shown in the upper-right corner of the composition in the following</p><p>screenshot), you need to add a second map item to the composition. To turn this map item</p><p>into an overview map, go to Item properties | Overviews and click on the button with the</p><p>green plus sign. This will add an Overview 1 entry and enable the Draw "Overview 1"</p><p>overview configuration GUI:</p><p>f Map frame: The Map frame drop-down list enables us to define the main map that</p><p>should be referenced by the overview map. By default, the map items are named</p><p>Map 0, Map 1, Map 2, and so on, depending on the order they were added to the</p><p>composition. Therefore, we will select the Map 0 entry if the main map was the first</p><p>item that was added to the composition.</p><p>f Frame style: The Change … button can be used to choose a style for the overview</p><p>frame. Usually, this will be a simple fill with transparency.</p><p>f Blending mode: These are supported by overview frames, as explained in detail in</p><p>the Understanding the feature and layer blending modes recipe.</p><p>f Invert overview: Enable the Invert overview checkbox if you want to apply the</p><p>overview frame style to the areas outside the extent of the main map.</p><p>f Center on overview: Enable the Center on overview checkbox if you want the</p><p>overview map to automatically pan to center on the extent of the main map.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>299</p><p>11</p><p>Extending QGIS</p><p>In this chapter, we will cover the following topics:</p><p>f Defining custom projections</p><p>f Working near the dateline</p><p>f Working offline</p><p>f Using the QSpatiaLite plugin</p><p>f Adding plugins with Python dependencies</p><p>f Using the Python console</p><p>f Writing Processing algorithms</p><p>f Writing QGIS plugins</p><p>f Using external tools</p><p>Introduction</p><p>QGIS can do many things on its own. However, as with all software, there are limits to its</p><p>default abilities. The great news is there are many ways to extend QGIS to do even more</p><p>through built-in customization options, existing add-on plugins, creating new analysis</p><p>algorithms, creating your own plugins, and using external software that compliments QGIS.</p><p>This chapter covers just a few of the common customizations and plugins that haven't been</p><p>mentioned in other chapters, and how you can get started with making your own add-ons to</p><p>share with others.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Extending QGIS</p><p>300</p><p>Defining custom projections</p><p>Map projections stump just about everybody at some point in their GIS career, if not more</p><p>often. If you're lucky, you just stick to the common ones that are known by everyone and your</p><p>life is simple. Sometimes though, for a particular location or a custom map, you just need</p><p>something a little different that isn't in the already vast QGIS projections database. (Often,</p><p>these are also referred to as Coordinate Reference System (CRS) or Spatial Reference</p><p>System (SRS).)</p><p>I'm not going to cover what the difference is between a Projection, Projected Coordinate</p><p>System, and a Coordinate system. From a practical perspective in QGIS, you can pick the one</p><p>that matches your data or your intended output. There's lots of little caveats that come with</p><p>this, but a book or class is a much better place to get a handle on it.</p><p>Getting ready</p><p>For this recipe, we'll be using a custom graticule, a grid of lines every 10 degrees</p><p>(10d_graticule.json.geojson), and the Natural Earth 1:10 million coastline</p><p>(ne_10m_coastline.shp).</p><p>How to do it…</p><p>1. Determine what projection your data is currently in. In this case, we're starting with</p><p>EPSG:4236, which is also known as Lat/Lon WGS84.</p><p>2. Determine what projection you want to make a map in. In this example, we'll be</p><p>making an Oblique Stereographic projection centered on Ireland.</p><p>3. Search the existing QGIS projection list for a match or similar projection. If you open</p><p>the Projection dialog and type Stereographic, this is a good start.</p><p>4. If you find a similar projection and just want to customize it, highlight the proj4</p><p>string and copy the information. NAD83(CSRS) / Prince Edward</p><p>means that the specification of the format is not</p><p>available. For this reason, QGIS, like other open source applications, does not support DWG</p><p>files. To open a DWG file in QGIS, you need to convert it. Converting it to a DXF file is a good</p><p>option as this will let you open your file in QGIS without any problem. There are many tools to</p><p>do this. The Teigha converter can be found at http://opendesign.com/guestfiles/</p><p>TeighaFileConverter and is a popular and reliable option.</p><p>Another option is using the free service offered by Autodesk, called Autocad 360, which can</p><p>be found at https://www.autocad360.com/.</p><p>Opening a NetCDF file</p><p>The NetCDF data is a data format, which is designed to be used with array-oriented scientific</p><p>data, and it is frequently used for climate or ocean data, among others. This recipe shows you</p><p>how to open a NetCDF file in QGIS.</p><p>How to do it…</p><p>NetCDF files are raster files, and they can be opened using the Add raster layer menu. Select</p><p>NGMT NetCDF Grid for CDF as the file format in the file selection dialog that you will see,</p><p>and select the rx5dayETCCDI_yr_MIROC5_rcp45_r2i1p1_2006-2100.nc file from</p><p>the example dataset. Click on OK.</p><p>www.it-ebooks.info</p><p>http://opendesign.com/guestfiles/TeighaFileConverter</p><p>http://opendesign.com/guestfiles/TeighaFileConverter</p><p>https://www.autocad360.com/</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>20</p><p>How it works…</p><p>The proposed NetCDF file contains a single variable, which is opened as a regular raster layer.</p><p>There's more…</p><p>A NetCDF file can contain contain multiple layers. In this case, QGIS will prompt you to select</p><p>the one that you want to add from the ones contained in the specified file.</p><p>When only one layer is available, it is opened directly, as in the previously described example.</p><p>The NetCDF Browser plugin</p><p>Another way of opening NetCDF files is using the NetCDF Browser plugin. Select the Manage</p><p>and install plugins... menu to open the plugin manager. Go to the Not installed section</p><p>and type netcdf in the search field to filter the list of available plugins. Select the NetCDF</p><p>Browser plugin and click on Install plugin to install it. Close the plugin manager.</p><p>The plugin is now installed, and you can open it by selecting NetCDF Browser in the</p><p>Plugins menu:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>21</p><p>Select the NetCDF file in the upper field. The other fields will be updated with the content of</p><p>the selected file. Select a layer from the available ones and click on Add to add the layer to</p><p>your QGIS project.</p><p>Saving a vector layer</p><p>QGIS supports multiple formats, not just to read vector layers but to also save them.</p><p>This recipe shows you how to export a vector layer, converting it to a different format.</p><p>Getting ready</p><p>You will use the layer named poi_names_wake.shp in this recipe. Make sure that it is</p><p>loaded in your QGIS project.</p><p>How to do it…</p><p>1. Right-click on the name of the points layer in the QGIS table of contents and select</p><p>the Save as... menu. You will see the following window:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>22</p><p>2. Let's suppose that you want to use this layer to create a web map. A popular format</p><p>supported by libraries, such as Leaflet of OpenLayers 3, is the GeoJSON format.</p><p>Select GeoJSON in the format field and enter a path and filename in the Save as field.</p><p>3. In the Save as dialog, click on OK. The GeoJSON file will be created.</p><p>How it works…</p><p>The OGR library, which is used by QGIS to read and open files, is also used to write them.</p><p>Not all of the formats that are supported for reading purposes are also supported for</p><p>writing purposes.</p><p>You can export even the layers that are not originally file-based to a file, such as a layer</p><p>coming from a PostGIS database or a WFS connection. Just select the layer in the table</p><p>of contents and proceed as just explained.</p><p>There's more…</p><p>The Save as dialog allows additional configuration beyond what you have seen in the example</p><p>in this recipe.</p><p>Fine-tuning the export operation</p><p>Depending on the format that you select to export your layer, different options are available</p><p>to configure how the layer is exported.</p><p>The options are shown by clicking on the More options button. Select GeoJSON as the export</p><p>format and then display the options for that particular format. The COORDINATE PRECISION</p><p>option controls the number of decimal places to write in the output GeoJSON file. The default</p><p>precision is too high for almost all cases, and most of the time, having three or four decimal</p><p>places is more than enough. Set the precision to 4, enter a valid path and filename, and export</p><p>the layer by clicking on OK. Your points layer will now be saved in a smaller GeoJSON file. You</p><p>can open this with a text editor to verify that the coordinates are expressed with the selected</p><p>precision or compare its size with the one created without specifying a precision value.</p><p>Opening the layer after creating it</p><p>If you want to work with the layer after it is created, check the Add saved file to map box. The</p><p>output layer will be opened and added to your current QGIS project.</p><p>Saving a raster layer</p><p>Raster layers can be exported to a different file. The export process can be used to crop the</p><p>layer or perform resampling, creating a modified layer. This recipe shows you how to do this.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>23</p><p>Getting ready</p><p>Open the elev_lid792_1m layer in your QGIS project.</p><p>How to do it…</p><p>1. Right-click on the name of the raster layer in the QGIS table of contents and select</p><p>the Save as... menu. You will see the following window:</p><p>2. In the Resolution fields, replace both of them with a value of 2. The original</p><p>resolution (the size of the cell) is 1, as you saw in a previous recipe.</p><p>3. Enter an output file path in the Save as field.</p><p>4. Click on OK. The layer will be saved with a coarser resolution than the original one.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>24</p><p>How it works…</p><p>The GDAL library is used to save the file. Not all formats supported for input are also</p><p>supported for output, but the most common ones are supported for both operations.</p><p>There's more…</p><p>The layer can be exported with a reduced extent. In the QGIS canvas, zoom to a small part of</p><p>the raster layer. Then open the Save as dialog. In the Extent section, click on the Map view</p><p>extent button. The bounding coordinates of the current map view will be placed in the four</p><p>coordinate fields.</p><p>Enter a file path to save the file to and click on OK. A layer with a reduced extent covering only</p><p>the region shown in the map view will be exported.</p><p>Reprojecting a layer</p><p>Layers may be in a CRS other than the one that is best for a given task. Although QGIS</p><p>supports on-the-fly reprojection when rendering, other tasks, such as performing spatial</p><p>analysis, may require using a given CRS or having all input layers in the same one.</p><p>This recipe shows you how to reproject a vector layer.</p><p>Getting ready</p><p>Open the layer named Davis_DBO_centerline.shp from the sample dataset.</p><p>How to do it…</p><p>The Davis_DBO_centerline.shp layer uses a CRS with feet as the unit, which makes</p><p>this unsuitable for certain operations. We plan to use this layer in future recipes to calculate</p><p>routes and work in metric units, so including this in a CRS that uses them is then a much</p><p>better option:</p><p>1. Right-click on the layer name in the table of contents and select Save as....</p><p>2. Select Selected CRS in the drop-down list to specify a different output CRS.</p><p>Click on the Browse button to select a CRS. You will see the CRS selector dialog.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>25</p><p>3. You will be converting the point to the EPSG:26911 CRS. Use the filter box to find it</p><p>among the list of available CRSs and select it. Then click on OK.</p><p>4. Click on OK in the Save as dialog to create the layer. A new shapefile will be created</p><p>with the projected lines.</p><p>How it works…</p><p>Reprojecting is done by the OGR library when it saves</p><p>the file because this is one of the</p><p>options that it supports.</p><p>There's more…</p><p>Raster layers can be reprojected in a similar way:</p><p>1. In the Save as dialog, for raster layers, you can find a CRS field with a Browse button.</p><p>2. Click on it to open the CRS selector, and select the destination CRS.</p><p>3. When you click on OK, the raster layer will be exported using the selected CRS</p><p>instead of its original one.</p><p>Batch format conversion</p><p>The Save as dialog can be used to convert the format of a single layer. When several layers</p><p>have to be converted, it is a better idea to use some automation. This recipe shows you how</p><p>to easily convert an arbitrary number of layers.</p><p>Getting ready</p><p>No previous preparation is needed. Batch conversion is not performed based on open</p><p>layers but performed directly on files, so there is no need to open layers in QGIS before</p><p>converting them.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>26</p><p>How to do it…</p><p>1. Open the Processing Toolbox menu by selecting Toolbox in the Processing</p><p>menu. The Processing Toolbox menu is the main element of the QGIS Processing</p><p>framework, and it is used to call its algorithms:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>27</p><p>2. In the filter box of the Processing Toolbox menu, type save to filter the list of</p><p>available algorithms. Locate the Save selected features algorithm, right-click on</p><p>it, and select Execute as batch process. The batch processing interface will be</p><p>displayed, as shown in the following screenshot:</p><p>3. In the upper cell in the Input layer column, click on the ... button and select</p><p>Select from filesystem. A file selector dialog will appear. Select the content of the</p><p>batch_conversion folder in the dataset. It should have a total of three files. Click</p><p>on OK on the file selection dialog. The batch processing interface should now have</p><p>all these selected files, one in each row in the parameters table.</p><p>4. In the Output layer column, click on the button in the first row. A dialog for saving the</p><p>file will be opened. Select a file path in your filesystem where you want to save the</p><p>output files and type converted.geojson as the output filename. Click on OK</p><p>and a new dialog like the one shown in the following screenshot will appear:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>28</p><p>5. Select Fill with parameter values in the first field and Input layer in the second</p><p>one. Click on OK. All the rows in the table will now have an output value, which</p><p>was created using the entered filename as a prefix, followed by the name of the</p><p>input layer.</p><p>6. To avoid layers being loaded after they are created, set the first cell in the Load into</p><p>QGIS column to No. Then, double-click on the column header to automatically copy</p><p>this value to all the rows below.</p><p>7. With the table already complete, you can launch the batch conversion process by</p><p>clicking on Run. The GeoJSON files will be created in the specified paths.</p><p>How it works…</p><p>The conversion is performed by an algorithm from the QGIS Processing framework. Processing</p><p>algorithms can be run either as individual algorithms or, in this case, in a batch process.</p><p>Outputs of Processing algorithms can be created in all formats supported by QGIS. The format</p><p>is selected using the corresponding extension in the filename and, unlike in the case of saving</p><p>a single layer, does not have to be selected in a field or list. Using geojson as the extension</p><p>for your output files, you tell processing that you want to generate a file in this format.</p><p>Although the algorithm saves only the selected features of the layer, if there is no selection,</p><p>it will use all the layer features. This is the default behavior of all algorithms in processing.</p><p>As there is no selection in the layers that you have converted, all of their features will have</p><p>been used.</p><p>When converting files this way, the additional options from the Save as dialog are not</p><p>available, and the default configuration values are used.</p><p>There's more…</p><p>You can also convert vector layers with another more complex algorithm from the Processing</p><p>Toolbox menu, which allows you to enter the configuration parameters used by the underlying</p><p>OGR library that takes care of the process. It's called Export vector. Find it in the toolbox,</p><p>right-click on it, and select Execute as batch process:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>29</p><p>In this case, the output format is not controlled by the extension of the output filename as it</p><p>happens with other processing algorithms according to what has been already explained.</p><p>Batch reprojection</p><p>Layers can be reprojected in a batch operation without having to enter parameters individually</p><p>on the Save as dialog. This recipe shows you how to reproject a set of layers to a different</p><p>CRS using an algorithm from the Processing Toolbox menu. You will see how to reproject all</p><p>the files accompanying the Davis_DBO_centerline.shp file that you reprojected in the</p><p>Reprojecting a layer recipe.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>30</p><p>How to do it…</p><p>1. In the filter box of the Processing Toolbox menu, type Reproject to filter the list of</p><p>available algorithms. Locate the Reproject layer algorithm, right-click on it, and select</p><p>Execute as batch process. The batch processing interface will be shown, as follows:</p><p>2. In the upper cell of the Input layer column, click on the ... button and select Select</p><p>from filesystem. A file selector dialog will appear. Select the content of the davis</p><p>folder in the dataset and add the files to the table.</p><p>3. In the first cell in the Target CRS column, click on the ... button. A CRS selector will</p><p>appear. Select the EPSG:26911 CRS, as you did in a previous recipe when converting</p><p>a single layer. Copy the value to the rest of rows in the column by double-clicking on</p><p>the column header.</p><p>4. Set all the values in the Reprojected layer column. Select a file in the first cell, and</p><p>then use the Fill with parameter value option to automatically fill the rest of rows.</p><p>5. Once the table is complete, click on Run to reproject the layers.</p><p>How it works…</p><p>The reprojection algorithm is a part of the Processing framework, so you can select the output</p><p>format by changing the output file extension. You can use this to not only reproject a set of</p><p>input layers but to also convert their format, all in a single step.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>31</p><p>There's more…</p><p>Raster layers can also be reprojected with another algorithm from the Processing Toolbox</p><p>menu named Warp (reproject). These inputs are rather similar to the ones in the reprojection</p><p>tool for vector layers with some additional parameters that are specific to raster layers. Select</p><p>the algorithm, right-click on it, and select Execute as batch process to run it and convert a</p><p>set of raster layers.</p><p>Loading vector layers into SpatiaLite</p><p>SpatiaLite is a single file relational database that is built on top of the well-known SQLite</p><p>database. It can store many layers of various types, including nonspatial tables. Interfaces to</p><p>the format also allow the ability to run spatial queries of various kinds. It's a highly-flexible and</p><p>portable format that is great for everyday use, especially when working on standalone projects</p><p>or with only one user at a time. SpatiaLite works in a similar manner to PostGIS without the</p><p>need to configure or run a database server.</p><p>Getting ready</p><p>Pick a vector layer and load it up in QGIS. This step is optional, as you can pick the source</p><p>layer from the filesystem in a later dialog.</p><p>How to do it…</p><p>1. Create a SpatiaLite database if you don't already have one and name it cookbook.db.</p><p>The easiest way to do this is with the Browser tab, as shown in the following</p><p>screenshot:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>32</p><p>2. Then, pick one of the following methods to import your data. The first option is faster,</p><p>but the second option gives</p><p>you more control over the import settings:</p><p>� Import method 1—the fast method</p><p>1. In the QGIS Browser tab, find the layer that you want to copy to</p><p>the database.</p><p>2. Drag and drop this layer on the Spatialite DB entry.</p><p>If you have a lot of files listed, this will be quite difficult as the browser</p><p>doesn't scroll during the drag operation. You can optionally open a</p><p>second browser window and drag the layer across. Also, note that this</p><p>defaults to multi-type geometry. If you need to control the options, use</p><p>the next method.</p><p>� Import method 2—the standard method</p><p>1. Open DB Manager from the Database menu.</p><p>2. Expand the Spatialite item to list your databases. Expand the database</p><p>that you want to connect to.</p><p>3. Click on the following import layer icon:</p><p>4. A dialog will pop up, providing you with import options.</p><p>SQL databases are usually case insensitive, so you can use</p><p>all lower case characters. Also, never use spaces or special</p><p>characters in table names; this can just lead to headaches later.</p><p>An occasional underscore is okay.</p><p>5. Select the layer to import from the drop-down list.</p><p>6. Fill in a name for the new table.</p><p>7. In most cases, the only thing left to do is check the Create spatial</p><p>index checkbox.</p><p>8. If this works, great. Now, you can load the layer to the map and verify</p><p>that it's identical to the input.</p><p>This method is more similar to traditional database import and very</p><p>similar to the PostGIS recipe next in this chapter.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>33</p><p>How it works…</p><p>QGIS converts your geometry to a format that is compatible with SpatiaLite and inserts it,</p><p>along with the attribute table. Afterwards, it updates the metadata tables in SpatiaLite to</p><p>register the geometry column and build the spatial index on it. These two postprocesses make</p><p>the database table appear as a spatial layer to QGIS and speed up the loading of data from</p><p>the table when panning and zooming.</p><p>There's more…</p><p>The import dialog contained a few other features that are often useful. You can reproject</p><p>data as part of the import process if you want, or you can specify the projection if QGIS didn't</p><p>detect it properly. You can also name the geometry column something different than the</p><p>default, geom; for example, utmz10n83 (this is normally not recommended). You can specify</p><p>the character encoding of the text in the event that it's not handled correctly.</p><p>You can even use the dialog to append data to an existing table; for example, you have</p><p>multiple counties with the same data structure that come as two separate files, but you</p><p>want them all in one layer.</p><p>If, for some reason, the layer didn't import the way that you want, delete it and redo the</p><p>import. If you delete layers, make sure to learn how to vacuum the database to recover</p><p>the now empty space in the file and shrink its total size (this is not automatic).</p><p>Look for the Vacuum option as a button in many graphical tools. If you</p><p>don't see it, no worries, just run the SQL, VACUUM;.</p><p>What happens if this fails? Databases can be really picky sometimes. Here are some common</p><p>issues and solutions:</p><p>f It could be character encoding (accents, non-Latin languages), which requires that</p><p>you specify the encoding.</p><p>f It could be picky about mixing multilayers with regular layers. Multilayers is when you</p><p>have several separate geometries that are part of one record. For example, Hawaii is</p><p>actually many islands. So, if you only have one row representing Hawaii, you need to</p><p>cram all the island polygons into one geometry field. However, if you mix this with North</p><p>Dakota, which is just a polygon, the import will fail. If you have this problem, you'll need</p><p>to perform the import on the command-line using ogr2ogr and its newish feature, -nlt</p><p>PROMOTE_TO_MULTI, which converts all single items to multi-items to fix this.</p><p>f Depending on your original source, you may have a mix of points, lines, and polygons.</p><p>You'll either need to convert this to a Geometry Collection, or you need to split each</p><p>type of geometry into a separate layer. Geometry Collections are currently poorly-</p><p>supported in many GIS viewers, so this is only recommended for advanced users.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>34</p><p>See also</p><p>If you need more advanced settings or can't get the QGIS tool to work, you may need to use</p><p>the QspatiaLite Plugin (install this with Manage Python Plugins under the Plugins menu), the</p><p>spatialite-gui (download this from https://www.gaia-gis.it/fossil/spatialite_</p><p>gui/index) application, or the ogr2ogr command line (this comes with QGIS, which is part</p><p>of OSGeo4w shell on Windows, or the terminal on Mac or Linux).</p><p>Loading vector layers into PostGIS</p><p>PostGIS is the spatial add-on to the popular PostgreSQL database. It's a server-style database</p><p>with authentication, permissions, schemas, and handling of simultaneous users. When</p><p>you want to store large amounts of vector data and query them efficiently, especially in a</p><p>multicomputer networked environment, consider PostGIS. This works fine for small data too,</p><p>but many users find its configuration too much work when SpatiaLite may be better suited.</p><p>Getting ready</p><p>Pick a vector layer and load it in QGIS. You will also need to have a working copy of Postgres/</p><p>PostGIS running, a PostGIS database created, and an account that allows table creation.</p><p>BostonGIS maintains a decent tutorial on installation for Windows, and</p><p>getting a PostGIS set up for everyone. You can find this at http://</p><p>www.bostongis.com/?content_name=postgis_tut01#316.</p><p>You should configure QGIS to be aware of your database and its connection parameters by</p><p>creating a new database item in the PostGIS load dialog or by right-clicking on PostGIS in the</p><p>Browser tab and selecting New Connection:</p><p>You can find more information about PostGIS at http://docs.qgis.org/2.8/en/docs/</p><p>user_manual/working_with_vector/supported_data.html#postgis-layers.</p><p>How to do it…</p><p>Now that you can connect to a PostGIS database, you are ready to try importing data:</p><p>1. Open DB Manager from the Database menu.</p><p>2. Expand the PostGIS item to list your databases. Expand the database that you want</p><p>to connect to, and you should be prompted to authenticate (if you haven't saved your</p><p>password in the settings).</p><p>3. Expand the list and select the Public schema.</p><p>www.it-ebooks.info</p><p>https://www.gaia-gis.it/fossil/spatialite_gui/index</p><p>https://www.gaia-gis.it/fossil/spatialite_gui/index</p><p>http://www.bostongis.com/?content_name=postgis_tut01#316</p><p>http://www.bostongis.com/?content_name=postgis_tut01#316</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/supported_data.html#postgis-layers</p><p>http://docs.qgis.org/2.8/en/docs/user_manual/working_with_vector/supported_data.html#postgis-layers</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>35</p><p>In general, unless you are performing advanced work and understand how</p><p>Postgres schemas work, place your layers in the Public schema. This is the</p><p>default that everyone expects.</p><p>4. Click on the following import layer icon:</p><p>5. A dialog will pop up, providing you with import options.</p><p>SQL databases are usually case insensitive, so you can use all lowercase.</p><p>Also, never use spaces or special characters in table names; this can just</p><p>lead to headaches later. An occasional underscore is okay.</p><p>6. Select the layer to import from the drop-down list.</p><p>7. Fill in a name for the new table.</p><p>8. Check whether schema is set to public.</p><p>9. In most cases, the only thing left to do is check the Create spatial index checkbox:</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Input and Output</p><p>36</p><p>How it works…</p><p>QGIS converts your geometries to a format that is compatible with PostGIS, and inserts it,</p><p>along with importing the attributes. Afterwards, it updates the metadata views in PostGIS</p><p>to register the geometry column and build the spatial index on it. These two post-processes</p><p>make the database table appear as a spatial layer to QGIS and speed up the loading of data</p><p>from the table when panning</p><p>and zooming.</p><p>There's more…</p><p>The options presented in the dialog are not all the options that are available. If you need</p><p>more control or advanced options present, you'll likely be looking at the command-line tools:</p><p>shp2pgsql (a graphical plugin for pgadmin3 is available on some platforms) and ogr2ogr.</p><p>The shp2pgsql tool generally only handles shapefiles. If you have other formats, ogr2ogr can</p><p>handle everything that QGIS is capable of loading. You can also use these tools to develop</p><p>batch import scripts.</p><p>To import large or complicated CSV or text files, you sometimes will need to use the pgadmin3</p><p>or psql command-line interface to Postgres.</p><p>Need even more control? Then, consider scripting. OGR and Postgres both have very capable</p><p>Python libraries.</p><p>Another option is using the OpenGeo Suite plugin, which has some additional options, such as</p><p>allowing importing multiple layers into a single table or into one table per layer. To learn more</p><p>about this, including how to install it, refer to http://qgis.boundlessgeo.com/static/</p><p>docs/intro.html.</p><p>What happens if this fails? Databases can be really picky sometimes:</p><p>f It could be character encoding (accents, non-Latin languages), which requires</p><p>specifying the encoding.</p><p>f It could be picky about mixing multilayers with regular layers. Multilayers is when you</p><p>have several separate geometries that are part of one record. For example, Hawaii</p><p>is actually many islands. So, if you only have one row representing Hawaii, you need</p><p>to cram all the island polygons into one geometry field. However, if you mix this with</p><p>North Dakota that is just a polygon, the import will fail. If you have this problem, you'll</p><p>need to perform the import on the command-line using ogr2ogr and its new feature,</p><p>-nlt PROMOTE_TO_MULTI, which converts all single items to multi-items, to fix this.</p><p>f Depending on your original source, you may have a mix of points, lines, and polygons.</p><p>You'll either need to convert this to a Geometry Collection, or you need to split each</p><p>type of geometry into a separate layer. Geometry Collections are currently poorly</p><p>supported in many GIS viewers, so this is only recommended for advanced users.</p><p>www.it-ebooks.info</p><p>http://qgis.boundlessgeo.com/static/docs/intro.html</p><p>http://qgis.boundlessgeo.com/static/docs/intro.html</p><p>http://www.it-ebooks.info/</p><p>Chapter 1</p><p>37</p><p>See also</p><p>For more information on PostGIS installation and setup, refer to http://postgis.net/</p><p>install.</p><p>For a more in-depth text on using PostGIS, there are many books available, including Packt</p><p>Publishing's PostGIS Cookbook.</p><p>www.it-ebooks.info</p><p>http://postgis.net/install</p><p>http://postgis.net/install</p><p>http://www.it-ebooks.info/</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>39</p><p>2</p><p>Data Management</p><p>In this chapter, we will cover the following recipes:</p><p>f Joining layer data</p><p>f Cleaning up the attribute table</p><p>f Configuring relations</p><p>f Joining tables in databases</p><p>f Creating views in SpatiaLite</p><p>f Creating views in PostGIS</p><p>f Creating spatial indexes</p><p>f Georeferencing rasters</p><p>f Georeferencing vector layers</p><p>f Creating raster overviews (pyramids)</p><p>f Building virtual rasters (catalogs)</p><p>Introduction</p><p>One of the reasons to use QGIS is its many features that enable management and analysis</p><p>preparation of spatial data in a visual manner. This chapter focuses on common operations</p><p>that users need to perform to get data ready for other uses, such as analysis, cartography, or</p><p>input into other programs.</p><p>In this chapter, you will find recipes to manage vector as well as raster data. These recipes</p><p>cover the handling of data from both file and database sources.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>40</p><p>Joining layer data</p><p>We often get data in different formats and information spread over multiple files. Therefore, one</p><p>important skill to know is how to join attribute data from different layers. Joining data is a way to</p><p>combine data from multiple tables based on common values, such as IDs or categories.</p><p>This exercise shows you how to use the join functionality in Layer Properties to join geographic</p><p>census tract data to tabular population data and how to save the results to a new file.</p><p>Getting ready</p><p>To follow this exercise, load the census tracts in census_wake2000.shp using Add Vector</p><p>Layer (you can also drag and drop the shapefile from the file browser to QGIS) and population</p><p>data in census_wake2000_pop.csv using Add Delimited Text Layer.</p><p>You can also load the .csv text file using Add Vector Layer, but this will load</p><p>all data as text columns because the .csv file does not come with a .csvt</p><p>file to specify data types. Instead, the Add Delimited Text Layer tool will scan</p><p>the data and determine the most suitable data type for each column.</p><p>How to do it…</p><p>To join two layers, there has to be a column with values/IDs that both layers have in common.</p><p>If we check the attribute tables of the two layers that we just loaded, we will see that both</p><p>have the STFID field in common. So, to join the population data to the census tracts, use the</p><p>following steps:</p><p>1. Open the Layer Properties option of the census_wake2000 layer (for example,</p><p>by double-clicking on the layer name in the Layers list) and go to Joins.</p><p>2. To set up a new join action, press the green + button in the lower-left corner of</p><p>the dialog.</p><p>3. The following screenshot shows the Add vector join dialog, which allows you to</p><p>configure the join by selecting Join layer, which you want to use to join the census</p><p>tracts and the columns containing the common values/IDs (Join field and Target field):</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Chapter 2</p><p>41</p><p>If you want to change a join, you just need to select the join</p><p>definition from the list and then press the edit button with the</p><p>pencil icon, which you find below the list. This will reopen the</p><p>join definition dialog, and you can make your changes.</p><p>4. When you press OK, the join definition will be added to the list of joins, as shown in</p><p>the following screenshot.</p><p>5. To verify that you set up the join correctly, close Layer Properties and open attribute</p><p>table to see whether the population columns have been added and are filled with data.</p><p>How it works…</p><p>Joins can be used to join vector layers and tabular layers from many different file and database</p><p>sources, including (but not limited to) Shapefiles, PostGIS, CSV, Excel sheets, and more.</p><p>When two layers are joined, the attributes of Join layer are appended to the original layer's</p><p>attribute table. If you want, you can use the Choose which fields are joined option to select</p><p>which of the fields from the population layer should be joined to the census tracts. Otherwise,</p><p>by default, all fields will be added. The number of features in the original layer is not changed.</p><p>Whenever there is a match between the values in the join and the target field, the new</p><p>attribute values will be filled; otherwise, there will be NULL values in the new columns.</p><p>www.it-ebooks.info</p><p>http://www.it-ebooks.info/</p><p>Data Management</p><p>42</p><p>By default, the names of the new columns are constructed from join layer name with</p><p>underscore followed by join layer column name. For example, the STATE column of</p><p>census_wake2000_pop becomes census_wake2000_pop_STATE. You can change this</p><p>default behavior by enabling the Custom field name prefix option, as shown in the previous</p><p>screenshot. With these settings, the STATE column becomes pop_STATE, which is</p><p>considerably shorter and, thus, easier to handle.</p><p>There's more…</p><p>The join that you've created now only exists in memory. None of the original files have been</p><p>altered. However, it's possible to create a new file from the joined layers. To do this, just use</p><p>Save as … from the Layer menu or Context menu. You can choose between a variety of data</p><p>formats, including the ESRI shapefile, Mapinfo MIF, or GML.</p><p>Shapefiles are a very common choice as they are still the de facto standard GIS data</p><p>exchange format, but if you are familiar with GIS data formats, you will have noticed</p>
  • gabsan01c
  • Lista de Exercícios 3 - Resumo
  • [LOG]C2U2S25A1
  • [LOG]C2U2S25A2
  • [LOG]C2U2S25A3
  • [LOG]C2U2S25A4
  • Análise de Sistemas Orientada a Objetos
  • Entrega e Implantação Contínua
  • DD076-CP-CO-Por_v1r0 (1)
  • 4e109b40-8651-df-929e-9ead876189a5
  • Questionário Desenvolvimento de Software
  • Desenvolvimento de Software: Práticas Ágeis
  • Não faz parte dos propósitos da Gerência de Configuração. A Redução de Custos. B Melhoria de Desempenho. C Redução no número de fornecedores. D Aum...
  • uma empresa de desenvolvimento de softwere está enfrentando um desafio com a entrega de seus projetos, Muitas vezes os prazos não são cumpridos, os...
  • Qual das alternativas não corresponde às técnicas utilizadas na modelagem de processos? a. UML. b. IDEF. c. SSP. d. EPC. e. BPMN. 0...
  • A atividade de Engenharia de Sistemas tem por objetivo entender as necessidades de negócio do cliente e estabelecer requisitos do ponto de vista si...
  • A linguagem de programação Python possui várias estruturas de dados que devem representar um conjunto de informações manipuladas pelo programa em e...
  • Para a publicação de aplicativos Android, qual dos seguintes passos NÃO é mencionado como parte do processo de preparação para o lançamento de um a...
  • Sobre os atalhos úteis, assinale a alternativa CORRETA:A) O Android não é personalizável porque se trata de um sistema de código aberto disponív...
  • Sobre a solução de problemas no desenvolvimento android classifique V para a(s) sentença(s) verdadeira(s) e F para a(s) falsa(s). (V) Para evitar p...
  • Sobre a arquitetura da aplicação Android, classifique V para a(s) sentença(s) verdadeira(s) e F para a(s) falsa(s). (V) Uma arquitetura de aplicati...
  • Analise o desafio a seguir: Você promoveu o Mauro, excelente profissional de vendas, a gerente distrital, comandando diretamente 12 profissionais...
  • Na Videoaula 2, vimos o exemplo de um modelo simplificado do processo de espelhamento de imagens, como na figura a seguir: Imagem de uma foto rep...
  • Questão 07 (PS CONCURSOS) Ao se tratar de avaliação, podemos aqui citar a avaliação mediadora a qual se desenvolve em benefício do educando e acont...
  • O desenvolvimento de software ágil revolucionou a indústria da tecnologia ao enfatizar valores como colaboração adaptabilidade e entrega contínua d...
  • AV3 sistemas operacionais- Mario Monteiro
  • aula_ti_segunda

Perguntas dessa disciplina

Grátis

What is the conclusion and future perspectives for the use of geotechnologies?a) Geotechnologies are fundamental tools for the management and anal...

Grátis

What are some examples of data mining tools?RapidMinerKNIMEWekaPythonRI - RapidMiner is a predictive analytics platform that offers a graphic...
What are some examples of data mining tools?RapidMinerKNIMEWekaPythonRI - RapidMiner is a predictive analytics platform that offers a graphic...
What are the main topics covered in chapters 1, 6, and 7 of the Corporate Finance textbook? a. Chapter 1: Financial challenges and functions of f...
Qual o nome da linguagem de escrita de métricas que é utilizada pelo Power BI Desktop? a. ADX (Automation Data Expressions)b. ADX (Analysis Data...
QGIS 2 Cookbook  Become a QGIS power user and master QGIS data management, visualization, and spatial analysis techniques - Engenharia de Software (2024)
Top Articles
Latest Posts
Recommended Articles
Article information

Author: Van Hayes

Last Updated:

Views: 5571

Rating: 4.6 / 5 (46 voted)

Reviews: 93% of readers found this page helpful

Author information

Name: Van Hayes

Birthday: 1994-06-07

Address: 2004 Kling Rapid, New Destiny, MT 64658-2367

Phone: +512425013758

Job: National Farming Director

Hobby: Reading, Polo, Genealogy, amateur radio, Scouting, Stand-up comedy, Cryptography

Introduction: My name is Van Hayes, I am a thankful, friendly, smiling, calm, powerful, fine, enthusiastic person who loves writing and wants to share my knowledge and understanding with you.