osgearth_conv

osgearth_conv is a command-line tool for converting geospatial datasets into tiled formats optimized for fast loading, streaming, and portability.

Motivation

osgEarth can load many different data formats for imagery and elevation. Internally, osgEarth chops geospatial into an optimized format – a pyramid of multi-resolution tiles that enables you to view the data at any level of detail without requiring the entire dataset to be resident in memory.

If we do this optimization offline, osgEarth can skip this step and get right to the business of rendering. This means:

  • Faster load times. Pre-optimized datasets are ready to render without any runtime processing.

  • Streaming. A pre-optimized dataset may be exported as small, independent tiles that are easily streamed from a standard web server.

  • Portability. Large datasets may be tiled and stored in a single database file, combining the benefits of fast loading AND portability.

osgearth_conv does this. The tool will read, subdivide, and tile input datasets for better load-time performance and various deployment options.

Examples

Here are some common usage examples. Later we will describe the various options in detail.

Convert a local GeoTIFF into a TMS repository of JPEG tiles:

osgearth_conv --in driver gdalimage       // read using the GDAL imagery driver
              --in url myinputfile.tif    // location of the GeoTIFF to read
              --out driver tmsimage       // export to a TMS imagery repository
              --out url output/tms.xml    // location of the TMS repository metadata file
              --out format jpg            // export each tile in JPEG format

Convert a local DTED elevation datafile into a local MBTiles database:

osgearth_conv --in driver gdalelevation       // read elevation data using GDAL
              --in url mydem.dt1              // location of the input file to read (DTED1)
              --in vdatum egm96               // tell the GDAL driver that DTED uses the EGM96 vertical dataum
              --out driver mbtileselevation   // write elevation data to an MBTiles database
              --out filename mydem.mbtiles    // location of the MBTiles database to write
              --out format tiff               // internal file format to use (must be tiff for elevation data)

Convert an online WMS (Web Map Service) repository to an offline MBTiles file:

osgearth_conv --in driver wmsimage
              --in url https://basemap.nationalmap.gov/arcgis/services/USGSImageryOnly/MapServer/WMSServer
              --in layers 0
              --in format jpeg
              --in srs EPSG:4326
              --out driver mbtilesimage
              --out filename ypgImagery.mbtiles
              --out format jpg
              --min-level 6
              --max-level 13
              --extents 33.020325 -114.604442 33.557478 -114.258021

General usage

The usage template for osgearth_conv looks like this:

osgearth_conv --in driver [input driver]
              --in [input property] [value] ...
              --out driver [output driver]
              --out [output property] [value] ...
              [general properties] ...

–in driver designates the osgEarth layer type of the input data. Supported input drivers are:

  • GDALImage / GDALElevation - Read from any GDAL-supported local image file (like a GeoTIFF)

  • WMSImage - Read imagery from an OGC Web Map Service endpoint

  • ArcGISServerImage - Read imagery data from an ESRI ArcGIS server layer

–out driver designates to target format for optimized tiled data. Supported output drivers are:

  • TMSImage / TMSElevation - Create a TileMapService repository, useful for streaming data over the web with only a standard web server (like Apache)

  • MBTilesImage / MBTilesElevation - MapBox Tiles container format, this is a local SQLite database containing tiled geodata. These are useful when you want to put the tiled dataset in a single file for portability. However, you cannot stream from an MBTiles file without additional server software.

PLEASE BE SURE to specify an “image” driver for imagery data, and an “elevation” driver for heightfield data. They are not interchangable!

Inputs

GDALImage / GDALElevation

GDAL is a toolkit for reading many geospatial data formats. The most common one is GeoTIFF.

Property Description
--in url path URL (file location) of the input data (e.g., a GeoTIFF file)
--in vdatum string Optional vertical datum; use this if you need to tell GDAL which vertical datum the input data is using (e.g, egm96)

Example: Convert a local GeoTIFF into an MBTiles database in PNG format (which supports transparency):

osgearth_conv --in driver gdalimage
              --in url mpdata.tif
              --out driver mbtilesimage
              --out filename mapdata.mbtiles
              --out format png

WMSImage

This driver reads from an OGC Web Map Service endpoint.

Property Description
--in url string HTTP (or HTTPS) endpoint of the WMS service. Always required.
--in layers string WMS layers for which to generate an image; corresponds to the WMS LAYERS parameter. This is usually required.
--in srs string Spatial reference system of the data to retrieve. Examples are wgs84 or EPSG:4326. Usually the server will have a default, but you can override that here.
--in format string File format to return. Usually this will be a mime-type like image/jpeg or image/png; the supported formats will be available in the WMS capatilibies.
--in wms_version string If the service requires a specific WMS protocol version, use this to send it. Usually you can omit this.
--in style string WMS STYLE parameter, if the service requires one. Usually you can omit this.
--in transparent true Request images with an alpha channel, if supported. Make sure if you use this that you also request a format that support transparency (e.g. --in format png).

Example:

osgearth_conv --in driver wmsimage
              --in url https://basemap.nationalmap.gov/arcgis/services/USGSImageryOnly/MapServer/WMSServer
              --in layers 0
              --in format image/jpeg
              --in srs EPSG:4326
              --out driver mbtilesimage
              --out filename output.mbtiles
              --out format jpg
              --min-level 6
              --max-level 13
              --extents 33.020325 -114.604442 33.557478 -114.258021

Notes:

  • We use a min-level and max-level to limit to zoom levels of the output data.

  • The output format of jpg is used since we do not expect to need transparency support.

ArcGISServerImage

Property Description
--in url string HTTP (or HTTPS) endpoint of the ArcGIS service. Always required.
--in token string Authentication token (may be required)
--in layers string Layers designator that selects which image to download. This is usually required.
--in format string File format to request. There will be a default, but you can use this to override it. Values like jpeg or png are common.

Example:

osgearth_conv --in driver arcgisserverimage
              --in url https://basemap.nationalmap.gov/arcgis/rest/services/USGSImageryOnly/MapServer
              --out driver mbtilesimage
              --out filename usgsImagery6.mbtiles
              --out format jpg
              --min-level 0
              --max-level 6
              --extents -90 -180 90 180
              --profile global-geodetic

Notes:

  • min_level and max_level properties limit the scope of the conversion

  • extents in this case is requesting full world extents

  • A global-geodetic output profile will reproject the input data from Mercator to Geographic (lat/long) for the best osgEarth load performance.

Outputs

TMSImage / TMSElevation

A TMS (TileMapService) repository is a simple hierarchical collection of files. Each tile in the pyramid is stored in its own file. This is a good format to use if you want to stream your dataset from a simple web server (like Apache).

Property Description
--out url path Location of the metadata file that contains the repository's geospatial information. The repository will be stored relative to the location of this XML file.
--out format string Format for each individual tile file. This should be jpg for RGB imagery, png for RGBA imagery, or tiff for elevation data.

Example (transform a local GDAL GeoTIFF imagery file into a TMS repository):

osgearth_conv --in driver gdalimage
              --in url myinputfile.tif
              --out driver tmsimage
              --out url output/tms.xml
              --out format jpg

In this example, the repository will be saved in a folder called output. You can transfer this folder directly to a web server to stream it over the web, or you can access it locally.

MBTilesImage / MBTilesElevation

MapBox Tiles stores a geospatial tileset in a SQLite database. MBTiles gives you the optimizations of a pre-tiled dataset along with the convenience and portability of a single file.

Property Description
--out url path Location of the SQLite MBTiles database file. It is common practice (but not required) to give this an .mbtiles extension.
--out format string Format for each individual tile file. This should be jpg or png for imagery, and must be tiff for elevation data.

Example:

osgearth_conv --in driver gdalelevation
              --in url mydem.dt1
              --in vdatum egm96
              --out driver mbtileselevation
              --out filename mydem.mbtiles
              --out format tiff

Common Properties

These properties apply to all osgearth_conv operations.

Property Description
--min-level integer Lowest zoom level for which to create tiles. Default is 0.
--max-level integer Highest zoom level for which to create tiles. The default value is computed automatically based on the input.
--extents minlat minlong maxlat maxlong Geographic extents (in degrees) for which to generate tiles.
--profile value Forces the output to a specific tiling profile. By default the output will match the geospatial profile of the input data (for example, if the input if spherical mercator, the output will be tiled in a spherical mercator profile). Use this to force something different -- typically global-geodetic for data that will run in a whole-earth osgEarth map.
--no-overwrite By default, running osgearth_conv will regenerate and overwrite any existing tiles that were output on a previous run. Use this option to disable that and ignore output tiles that already exist.
--threads integer Using multiple threads can speed up the tile generation process in some cases. A good value to try is 4.

© Copyright 2022 Pelican Mapping