Skip to content

Commit

Permalink
Merge pull request #24 from nasa-gibs/gdal-docs
Browse files Browse the repository at this point in the history 
Updated doc for GDAL scripts to include WMTS driver examples and merged TiledWMS info
  • Loading branch information
@jtroberts
jtroberts authored May 6, 2024
2 parents a22d933 + c399028 commit 7178ed9
Showing 1 changed file with 105 additions and 6 deletions.
111 changes: 105 additions & 6 deletions docs/map-library-usage.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -19,7 +19,15 @@ Please see our ["GIBS Web Examples" GitHub area](https://github.com/nasa-gibs/gi

## GDAL

The Geospatial Data Abstraction Library ([GDAL](https://gdal.org/){:target="_blank"}) can be used as a basis to generate imagery from custom scripts. Its WMS driver supports several internal 'minidrivers' that allow access to different web mapping services. Each of these services may support a different set of options in the Service block. Documentation for these minidrivers can be found in the [GDAL WMS documentation area](https://gdal.org/drivers/raster/wms.html){:target="_blank"}. Two of these minidrivers in particular can be used by users to download GIBS imagery programmatically. They are the Tile Map Specification (TMS) and the OnEarth Tiled WMS (TiledWMS) minidrivers. Examples for both of these minidrivers are included below.
The Geospatial Data Abstraction Library ([GDAL](https://gdal.org/){:target="_blank"}) can be used as a basis to generate imagery from custom scripts. GDAL is a well known geospatial data access library which is part of most GIS applications. GDAL can read (and sometimes write) geospatial raster and vector data in many formats, the format specific operations being handled by a GDAL driver.

The GDAL WMS driver can access many of the known tiled and non-tiled protocols. GDAL WMS has a common part that handles interfacing with GDAL, making the requests and receiving the responses, decoding the received data as needed and a few other things. Since using tiles and multiple resolutions (known as a pyramid) is well supported in GDAL and widespread in the GIS domain, the core of the WMS driver is based on this model, and tile requests are made even when accessing a dynamic service that could provide its own subsetting and resampling.

The WMS driver supports several internal "minidrivers" that allow access to different web mapping services. Each of these services may support a different set of options in the Service block. Documentation for these minidrivers can be found in the [GDAL WMS documentation area](https://gdal.org/drivers/raster/wms.html){:target="_blank"}. Two of these minidrivers in particular can be used by users to download GIBS imagery programmatically. They are the Tile Map Specification (TMS) and the OnEarth Tiled WMS (TiledWMS) minidrivers. Examples for both of these minidrivers are included below.

The WMS driver is also used by many other GDAL drivers, for example it is used by the [GDAL WMTS driver](https://gdal.org/drivers/raster/wmts.html){:target="_blank"}. Accessing GIBS imagery the WMTS driver is detailed further below.



### Requirements

Expand All @@ -45,7 +53,24 @@ This section provides basic examples of both the TiledWMS and WMS GDAL drivers.
```

#### TiledWMS Driver
The TiledWMS GDAL minidriver relies on a simple XML configuration block (see example below) from the user, pulling all other information from the GIBS Tiled Web Map Service ([TWMS](../access-basics/#tiled-web-map-service-twms)) "Tile Service" document at runtime.
TiledWMS adds a GetTileService call to the OGC WMS, the XML document returned is what contains all the information needed to configure and connect to any dataset. The GetTileService request for datasets in EPSG:4326 projection is https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService
This URL can be used by gdalinfo to get a list of available datasets with their `TiledGroupName`:
``` shell
gdalinfo "https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService"
```

To see information about a single dataset, the `gdalinfo` command may be invoked with the associated `SUBDATASET` ID of the `TiledGroupName` for the dataset by using the `-sd` flag. For example:
``` shell
gdalinfo -sd 479 "https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService"
```

To filter for specific datasets with `gdalinfo`, the `-oo TiledGroupName=` option may be used:
``` shell
gdalinfo -oo TiledGroupName="MODIS Aqua" "https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService"
```


The TiledWMS GDAL minidriver supports a simple XML configuration block (see example below) from the user, pulling all other information from the GIBS Tiled Web Map Service ([TWMS](../access-basics/#tiled-web-map-service-twms)) "Tile Service" document at runtime.

``` xml
<GDAL_WMS>
Expand All @@ -61,13 +86,15 @@ In the above XML block, the following values may be changed to meet your needs:

* **Server Url** - Set the *ServerUrl* to a value of `{Endpoint Root}/twms.cgi?`, where the endpoint root is defined in [this](../access-basics/#service-endpoints_2) table.
* e.g. `https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?`
* **Tiled Group Name** - When accessing visualizations through TWMS, the *TiledGroupName* value is utilized instead of the layers identifier. The *TiledGroupName* can be generated by replacing all underscores in a visualization's *Identifier* with spaces and appending " tileset". For example:
* **Tiled Group Name** - When accessing visualizations through TWMS, the *TiledGroupName* value is utilized instead of the layers identifier. The *TiledGroupName* can also be generated by replacing all underscores in a visualization's *Identifier* from GetCapabilities with spaces and appending " tileset". For example:
* **Identifier** - MODIS_Aqua_CorrectedReflectance_TrueColor
* **Tiled Group Name** - MODIS Aqua CorrectedReflectance TrueColor tileset
* **Time** - Insert the date (e.g. 2013-08-21) or datetime (e.g. 2013-08-21T00:00:00Z) you are requesting.
* **Time** - Insert the date (e.g. 2013-08-21) or datetime (e.g. 2013-08-21T00:00:00Z) you are requesting. If no time is provided, you can add an option to your GDAL command to specify the value, e.g., `-oo Change=time:2020-02-05`

The following examples demonstrate how to invoke the GDAL TiledWMS driver.
##### #1 - Configuration File Input


The following examples demonstrate how to invoke the GDAL TiledWMS driver for imagery requests.
##### #1 - "TiledWMS" Driver Configuration File Input
In this example, the following XML file is created and saved to your local file system. Then the `gdal_translate` command shown below is run with your desired area of interest and output dimensions.

``` xml
Expand All @@ -79,17 +106,44 @@ In this example, the following XML file is created and saved to your local file
</Service>
</GDAL_WMS>
```
Let's save the XML as a file named `GIBS_Aqua_MODIS_true.xml`

```
gdal_translate -of GTiff -outsize 1200 1000 -projwin -105 42 -93 32 GIBS_Aqua_MODIS_true.xml GreatPlainsSmoke1.tif
```
The XML file can be opened in any GDAL based GIS software. However, in ArcGIS Pro, TileWMS is not recognized without some extra configuration steps, but it can be dragged and dropped into a map and it will work. Alternatively, we can change the extension into something supported, like .tif for example.

This file shows one of the main advantages of the TiledWMS over other GDAL WMS minidriver protocols. Only the minimum amount of information is stored in the handle file. No fiddling with bounding boxes, tile sizes, number of levels is required. Getting the detail correct becomes the responsibility of the source server. All that is needed is the server URL, the name of the tiled group and optionally a set of parameter changes supported by the tiled group.

##### #2 - "TiledWMS" Driver Command Line Input
This example invokes gdal_translate with the content of the TileWMS local service description XML file embedded as a command line argument. This approach is useful for automated scripting to download various layers, dates, etc. To generate the same image as the previous example, run the following:

```
gdal_translate -of GTiff -outsize 1200 1000 -projwin -105 42 -93 32 '<GDAL_WMS><Service name="TiledWMS"><ServerUrl>https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?</ServerUrl><TiledGroupName>MODIS Aqua CorrectedReflectance TrueColor tileset</TiledGroupName><Change key="${time}">2013-08-21</Change></Service></GDAL_WMS>' GreatPlainsSmoke2.tif
```
##### #3 - "TiledWMS" Driver Configuration File for All Datasets
The TiledGroupName can also be fed as an open option. Create the following XML file:
``` xml
<GDAL_WMS>
<Service name="TiledWMS">
<ServerUrl>https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?</ServerUrl>
</Service>
</GDAL_WMS>
```
Let's save the XML as a file named `GIBS_epsg4326.xml`. Now we can run `gdal_translate` and use the `-oo` options to specify the dataset and time.
``` shell
gdal_translate -of JPEG -outsize 2560 1280 -oo TiledGroupName="MODIS Aqua CorrectedReflectance TrueColor tileset" -oo Change=time:2020-02-05 GIBS_epsg4326.xml MODIS_A_Feb_05_2020.jpg
```

##### #4 - Generate "TiledWMS" Configuration Files for Specified Datasets
gdal_translate has a `-sds` option where each subdataset of the input is copied, in sequence. This can be used to generate multiple handle files in a single command. We can use the open options to restrict which gets generated. For example, this generates all the handle files for patterns that contain the word "infrared":
``` shell
gdal_translate -of WMS -sds -oo TiledGroupName="infrared" -oo Change=time:2019-10-21 "https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService" Infrared.tWMS
```
Additionally, a `StoreConfiguration` open option can be applied to store the GetTileService response locally to speed up the requests:
``` shell
gdal_translate -of WMS -sds -oo StoreConfiguration=yes -oo TiledGroupName="infrared" -oo Change=time:2019-10-21 "https://gibs.earthdata.nasa.gov/twms/epsg4326/best/twms.cgi?request=GetTileService" Infrared.tWMS
```

<hr style="border:1px solid gray"> </hr>

Expand Down Expand Up @@ -188,6 +242,51 @@ gdal_translate -of GTiff -outsize 1200 1000 -projwin -105 42 -93 32 '<GDAL_WMS><

<hr style="border:1px solid gray"> </hr>

#### "WMTS Driver"
A WMTS driver was added to GDAL in version 2.1 and greater. It can be used with the GIBS GetCapabilities to get a list of available datasets:
``` shell
gdalinfo "https://gibs.earthdata.nasa.gov/wmts/epsg4326/best/wmts.cgi?SERVICE=WMTS&request=GetCapabilities"
```
or information about a single dataset using the `WMTS:` prefix:
``` shell
gdalinfo "WMTS:https://gibs.earthdata.nasa.gov/wmts/epsg4326/best/wmts.cgi?SERVICE=WMTS&request=GetCapabilities,layer=MODIS_Terra_CorrectedReflectance_TrueColor"
```

**Note the drawback of using WMTS is that the `time` parameter is not supported. This means users are limited to only the default `time` for any given layer.**

##### #1 - Configuration File Input
Create a local service description XML file and invoke gdal_translate. In this example, GIBS_Terra_MODIS_true.xml is used to generate a true color JPEG image from Terra MODIS of the entire world for the current day. The contents of GIBS_Terra_MODIS_true.xml would be:

``` xml
<GDAL_WMTS>
<GetCapabilitiesUrl>https://gibs.earthdata.nasa.gov/wmts/epsg4326/best/wmts.cgi?SERVICE=WMTS&amp;request=GetCapabilities</GetCapabilitiesUrl>
<Layer>MODIS_Terra_CorrectedReflectance_TrueColor</Layer>
<Style>default</Style>
<TileMatrixSet>250m</TileMatrixSet>
<DataWindow>
<UpperLeftX>-180</UpperLeftX>
<UpperLeftY>90</UpperLeftY>
<LowerRightX>180</LowerRightX>
<LowerRightY>-90</LowerRightY>
</DataWindow>
<BandsCount>3</BandsCount>
<DataType>Byte</DataType>
<Cache />
</GDAL_WMTS>
```
```
gdal_translate -of JPEG -outsize 2560 1280 GIBS_Terra_MODIS_true.xml GIBS_Terra_MODIS_true.jpeg
```

##### #2 - "WMTS" Driver Command Line Input
Invoke gdal_translate with the content of the local service description XML file embedded into the command. To generate the same image as above:
```
gdal_translate -of JPEG -outsize 2560 1280 "WMTS:https://gibs.earthdata.nasa.gov/wmts/epsg4326/best/wmts.cgi?SERVICE=WMTS&request=GetCapabilities,layer=MODIS_Terra_CorrectedReflectance_TrueColor" MODIS_Terra_default.jpg
```
Note that the WMTS driver defaults to 4 bands, which may result in discoloration if the incorrect number of bands is specified. There is currently not a way to specify the number of bands from the command line with the WMTS driver, which may result in discoloration with this example.

<hr style="border:1px solid gray"> </hr>

### Advanced Usage
#### TMS Driver Configuration File Input w/ Transparent PNG

Expand Down

0 comments on commit 7178ed9

Please sign in to comment.