An OGC API to geospatial data

This presentation available at https://pygeoapi.io/presentations/default

Table of contents

Geospatial API evolution
REST/JSON/OpenAPI/Swagger
OGC activity
Say hello to pygeoapi
Core capabilities
Deploying
Production instances
Roadmap
Support

Geospatial API evolution

1990s

Spirit of XML-RPC/CORBA
SOAP/WSDL/UDDI
Service Oriented Architecture (SOA)
Strong concept of RDBMS as the backend
OGC WMS (1999)

2000s

Web 2.0
JavaScript/AJAX/Google Maps
Slippy maps, tiles
OGC WFS (2002), WCS (2003), WPS (2005), CSW (2007)

Realities of OGC web services

Key Value Parameters (KVP) or XML encodings
Requests using HTTP GET, HTTP POST and SOAP
Responses as XML (GML)
Error handling as custom ExceptionReport within an HTTP 200 response
Exhaustive specifications/requirements

Sample response from an OGC WFS 1.0/2.0 service

More realities

Complex machinery/architectures (e.g. ldproxy) for services to be crawlable
Challenging for web developers to implement
Challenging for mainstream web integration (search indexing)
Geonovum testbed showed OGC world can easily be made crawlable, and crafted LDProxy

REST/JSON/OpenAPI/ Swagger

REST

REpresentational State Transfer (REST)
HTTP verbs (GET/PUT/POST/DELETE)
HTTP codes (200, 201, 404, etc.)
Uses URI to identify resources
Content negotiation (media types)
Stateless

JSON

JavaScript Object Notation
Easy to write and understand
Very popular among web developers
JSON is first class in RESTful web services
No rigid standards (for better or worse)

OpenAPI

Specification on how to describe a REST API
Defines endpoints, request parameters, response
Programming language agnostic
Formerly Swagger

Swagger

OpenAPI client tooling
Robust, web developer focused
Automated documentation, testing, code generation

OGC activity

https://ogcapi.ogc.org

W3C Spatial Data on the Web Best Practices (2017)

Recommendations on formats, identifiers, access, licensing, provenance
Being webby
https://www.w3.org/TR/sdw-bp

Design patterns

Being webby (humans, search engines)
Developer friendly
Lightweight specification development
Removing HTTP use as a tunnel
- ~~/ows?request=GetFeature&typename=roads&featureid=5~~
- /api/collections/roads/items/5
Modular specification development
- Core and extensions

Timeline

2017: W3C Spatial Data on the Web Best Practices
2017: OGC API Whitepaper
2018: WFS3 Hackathon, Weather on the Web API Hackathon
2019-2020: OGC API Hackathons
OGC API Roadmap

Future OGC standards

OGC API - Common (OWS Common)
OGC API - Features (WFS)
OGC API - Coverages (WCS)
OGC API - Maps (WMS)
OGC API - Tiles (WMTS)
OGC API - Processes (WPS)
OGC API - Records (CSW)
OGC API - Styles (SE/SLD)

OGC activity on GitHub

webby
Specification development
Issue tracking/discussion/collaboration
Standards written in AsciiDoc (!)
Standards released as HTML (!) or traditional PDF/etc.
Gitter chat

Say hello to pygeoapi

A Geopython project
Created by Tom Kralidis in 2018 on Valentine's Day with ❤️
OSGeo Community Project

Project Overview

Geospatial data API framework
OGC Compliant (already!)
OSGeo Community Project
International team (Canada, Netherlands, Greece, Italy, New Zealand)
- Numerous core contributors
- 21000 upstream contributors (dependencies)

Technical Overview

Publishing (leaves data pristine/alone)
- Fits to the data (not the inverse)
Leverages powerful ecosystem of Python packages

Technical Overview

Core abstract API; web framework agnostic (default Flask)
YAML configuration (metadata, dataset connections
Automated OpenAPI document generation (data binding)
Robust plugin framework (data connections, formats, processing)
Easy deployment (pip install, Docker, conda, UbuntuGIS)
Minimal dependencies

Feature Data Providers


Elasticsearch	CSV	Spatialite*	GeoJSON*

* also via GDAL-OGR Provider

Feature Data Providers - GDAL/OGR

Via OGR-Python bindings
Unleashes ~100 Vector Formats! (WFS 1/2, GeoPackage, Shapefile, GeoJSON, PostGIS, ...)
Paging, reprojection, spatial/attribute query
Unlock existing WFS 1/2s!

Coverage Data Providers


xarray	rasterio

NetCDF, Cloud Optimized GeoTIFF (COG)

Tile Data Providers


minio	ZXY directory tree

MapBox Vector Tiles via tippecanoe

schema.org

Lower the barrier to data search
HTML encoding is annotated as schema.org/Dataset to facilitate search engines
Google dataset search locates pygeoapi datasets

Linked Data - JSON-LD

JSON-LD provided as additional representation
Default out of the box functionality for most pygeoapi resources / endpoints
Additional configuration

Core Capabilities

OpenAPI

OGC API - Features

OGC API - Coverages

OGC API - Tiles

OGC API - Processes

SpatioTemporal Asset Catalog (STAC)

Deploying

Packaging

Numerous deployment options

PyPI
UbuntuGIS
Docker
Conda
FreeBSD
Coming to OSGeoLive

Docker overview

Docker

pygeoapi Docker Images on DockerHub
Images create Linux containers running content
Use Docker Images to test locally or deploy in server

Docker options

Custom config via Docker Volume Mapping
Run with Docker Compose
Subpath running via Flask SCRIPT_NAME environment variable
Examples

Scaling pygeoapi

Docker scale

Cloud clustering with high availability

Production instances

Canada

Meteorological Service of Canada GeoMet platform

OGC web services platform of Canadian weather, climate and water data
Realtime weather
Numerical weather prediction
Hydrometric and Climate archives

MSC pygeoapi implementation

GLOSIS - Global Soil Information

Global soil information system under development by FAO and ISRIC
Using pygeoapi as an external API providing OGC API - Features
Testbed for JSON-LD for soil data distribution
https://glosis.isric.org/ogc

Roadmap

Roadmap

More OGC APIs: Maps, Records, Styles
OGC API Improvements: Processes management
More data providers
Advanced filters (CQL)
Transactions
Content negotiation (e.g. Response as GeoPackage, GML,...)
OSGeoLive

OGC API - Records

Initial implementation currently in development with specification
Development team part of OGC API - Records SWG
Plan to use pycsw backend (PostgreSQL-based)
Users will be able to update a single backend and have both CSW and OGC API - Records support
Dependency strategy to be determined (e.g. pygeoapi may become pycsw dependency to support OGC API - Records)

Django app

Django application project

django_app folder next to flask_app.py and starlette_app.py
urls.py with routes
views.py with the binding of pygeoapi core API into the process of Request/Response
settings.py with the pygeoapi configuration path

Add --django option to the cli command to serve the usual Django server
Make the Django app reusable
Challenges:

Use DRF serializers
Inject generated OpenAPI document into drf-spectacular
Inject Jinja2 templates or use Django templates natively

https://github.com/geopython/pygeoapi/issues/586

GeoNode integration

Next generation GeoNode API GNIP-51
New v2 GeoNode REST API has been merged into master: GNIP-79
Discussion needed to determine how GeoNode could use pygeoapi as a backend:
- Metadata (Resource base model -- similar to pycsw GeoNode integration)
- Direct Elasticsearch integration
- For OGC API Features, Maps, Styles, Tiles, Coverages (similar to GeoServer and QGIS Server)
- How would auth work in this case?

Support

Need support for WFS3.0???

GeoCat can help you

Links

Home: https://pygeoapi.io
Docs: https://docs.pygeoapi.io
Demo: https://demo.pygeoapi.io
GitHub: https://github.com/geopython/pygeoapi
Docker: https://hub.docker.com/r/geopython/pygeoapi
Chat: https://gitter.im/geopython/pygeoapi
Mail: https://lists.osgeo.org/mailman/listinfo/pygeoapi

Thank you!

@tomkralidis @JMendesDeJesus @normanbarker @perrygeo @justb4 @pvgenuchten @tzotsos @alpha-beta-soup @francbartoli