Get DEM

Take a bounding box and output a GeoTIFF DEM.

Introduction

This is a thin wrapper around sardem: https://github.com/scottstanie/sardem

The wrapper is designed for use with the MAAP project; it is meant to exercise the MAAP processing pipeline.

The source DEM is hardcoded to be the Copernicus DEM, which is fetched from the AWS Open Data registry. See: https://registry.opendata.aws/copernicus-dem/

The code will fetch the necessary DEM tiles, stitch them together with GDAL, and create a single geotiff DEM in the out_dir directory, named dem.tif.

If the --compute flag is included, it will open the generated dem.tif file and do compute-intensive, multi-core linear algebra computations on that DEM raster. There are no changes made to the dem.tif; this command is simply for benchmarking compute. These computations use NumPy's linear algebra module, which uses all available CPU cores.

Example command line calls:

# bounding box: [left bottom right top]
python get_dem.py \
    --bbox -118.06817 34.22169 -118.05801 34.22822 \
    --out_dir output

# add --compute to have the compute node perform intense, multi-core computations
python get_dem.py \
    --bbox -118.06817 34.22169 -118.05801 34.22822 \
    --compute \
    --out_dir output

The three test bbox's

Let's use these three bounding boxes for development and to compare between platforms:

1. "Mount Wilson"

   • --bbox -118.06817 34.22169 -118.05801 34.22822
   • Very small (24 x 37 pixels)
   • Should take ~5-8 seconds to complete the algorithm¹

2. "California and Nevada"

   • --bbox -124.81360 32.44506 -113.75989 42.24498
   • 35280 x 39793 pixels
   • With 8 cores on NASA DPS, takes 9-10 min to fetch+stitch DEM, and ~13-14 min for computations¹
   • Warning: Please be mindful of memory usage

3. Italy

   • --bbox 6.26868 36.00380 18.57179 47.28139
   • 40599 x 44291 pixels
   • With 8 cores on NASA DPS, takes 9-10 min to fetch+stitch DEM, and ~23-25 min for computations¹
   • Warning: Please be mindful of memory usage

Comparing ESA and NASA Outputs

In addition to being able to compare performance between the ESA and NASA runtime systems, we want to make sure that given the same inputs, on both systems, an algorithm will produce the same output(s). Before deploying this algorithm to either system, we can locally emulate running the algorithm and compare the outputs produced by running the following:

make compare-outputs

This will do the following:

1. Build the ESA Docker container.
2. Run the algorithm in the container, emulating the ESA platform, using a pre-defined bounding box (see the BBOX value in Makefile).
3. Run the algorithm emulating the NASA platform (with the same BBOX value).
4. Compare the checksum (cksum) values of the dem.tif generated by each.
5. Exit successfully if the checksums match, otherwise exit erroneously.

The file generated by the ESA emulation will appear in output/esa/dem.tif, and the one generated by the NASA emulation will appear in output/dem.tif.

At the end of the command output, you should see something like the following, if all goes well, where the numbers on the left are the checksum values, and the numbers on the right are the file sizes (bytes):

...
4020122921 50576766 output/esa/dem.tif
4020122921 50576766 output/dem.tif
outputs are the same

If you wish to override the BBOX value used for the emulations, you may set the BBOX environment variable as desired. For example (note the quotes around the coordinates, as well as the inclusion of the clean target to force regenerating the files):

BBOX="-118.06817 34.22169 -118.05801 34.22822" make clean compare-outputs

Footnotes

1. Time estimates are for timings internal to the algorithm; they do not include DPS packaging time, etc. ↩ ↩² ↩³