Possibility to specify opening options

[goergen95]

Hi,

This is an awesome package and I am looking forward to explore it in more detail!
I faced a blocker to use vapour_layer_info() on a data source that requires some opening options (-oo) to be
read correctly by GDAL. I think that also applies to other vapour_*() functions, if there isn't a way to specify those options that I have overlooked?

Please find a reprex below where you can see that sf::gdal_utils() allows for an option argument that is then passed to GDAL. Would be great to see something similar here.

library(vapour)
library(sf)
#> Linking to GEOS 3.11.1, GDAL 3.8.2, PROJ 9.1.1; sf_use_s2() is TRUE

ds <- "/vsizip/vsicurl/https://ucdp.uu.se/downloads/ged/ged231-csv.zip"
vapour_layer_info(ds)
#> problem in get extent
#> $dsn
#> [1] "/vsizip/vsicurl/https://ucdp.uu.se/downloads/ged/ged231-csv.zip"
#> 
#> $driver
#> [1] "CSV"
#> 
#> $layer
#> [1] "GEDEvent_v23_1"
#> 
#> $layer_names
#> [1] "GEDEvent_v23_1"
#> 
#> ..... (output truncated)
#> 
#> $count
#> [1] 316818
#> 
#> $extent
#> [1] 0 0 0 0
#> 
#> $projection
#> $projection$Wkt
#> NULL
#> 
#> $projection$Proj4
#> NULL
#> 
#> $projection$EPSG
#> NULL
gdal_utils("vectorinfo", ds,
           options = c("-json", "-oo", "GEOM_POSSIBLE_NAMES=geom_wkt"))
#> {
#>   "description":"/vsizip/vsicurl/https://ucdp.uu.se/downloads/ged/ged231-csv.zip",
#>   "driverShortName":"CSV",
#>   "driverLongName":"Comma Separated Value (.csv)",
#>   "layers":[
#>     {
#>       "name":"GEDEvent_v23_1",
#>       "metadata":{
#>       },
#>       "geometryFields":[
#>         {
#>           "name":"geom_wkt",
#>           "type":"Geometry",
#>           "nullable":true,
#>           "extent":[
#>             -117.3,
#>             -37.813611000000002,
#>             155.896681,
#>             68.979169999999996
#>           ],
#>           "coordinateSystem":null
#>         }
#>       ],
#>       "featureCount":316818,
#>
#>   ..... (output truncated)
#>
#>   "metadata":{
#>   },
#>   "domains":{
#>   },
#>   "relationships":{
#>   }
#> }

^{Created on 2024-01-13 with reprex v2.0.2}

[mdsumner]

agreed! the vapour_ functions mostly predate my ability to code strings (!!😅) in C++ and so you see options in the new warper-based read gdal_raster_ functions but rarely elsewhere.

I am still trying to learn to wrap the entire api up and then not improving what we have here already. Very interested in any feedback and I will try to implement this request soon 🙏

[goergen95]

Great, I will try to get a better understanding of vapour and maybe I'll be able to contribute PRs further down the line. But I am just getting my feet wet with C++, so let's see how this turns out.. 😆

A quick question: are you aiming exclusively on raster data models with vapour for future developments or is vector data still in scope?

[mdsumner]

definitely still in scope, ideally I want an actual api package, like the "osgeo" suite of osgeo.gdal, osgeo.ogr, osgeo.osr, osgeo.gdal_array, etc. - but I'm not able to get it to work (mostly I muck up the live pointers), there's a lot of code to write and a lot can be automated but I'm not able yet (look at paleolimbot/libproj and paleolimbot/geos for where I'd like to go), but also rgdal did a lot of this already but got sidelined by more successful projects (it's very similar in Python, rasterio and fiona mask the power in the osgeo modules and a lot of downstream effort goes into things that should be fixed in the libs).

[goergen95]

I had a look at the automation side of things. GDAL's python bindings are generated using SWIG. SWIG also supports generating R bindings, though it seems that it needs some improvements (see e.g. swig/swig#854). Still could be worth exploring how far does one get with the current implementation and communicate feature requests to SWIG?

[mdsumner]

I agree, and maybe better R swig tools will help - but fwiw Even Rouault recommended not using SWIG, he said

I'm not sure SWIG bindings are the way to go. They tend to produce
non-idiomatic bindings for any language, which people hate using

fwiw, the best examples are rgdal2 (apparently Tim used swig to start but didn't record how and he abandoned it while sf took off), rgdal (has a lot of good open-pointer handling with base R api but presented via S4, which i like fwiw), gdalraster (uses Rcpp modules), gdalcubes (I don't know much), and then there's geos and libproj, both base R api and using vctrs but all done in cpp, and of course terra with Rcpp modules all internal, and sf all Rcpp. There's a handful of others but very light use I think

[RichardScottOZ]

definitely still in scope, ideally I want an actual api package, like the "osgeo" suite of osgeo.gdal, osgeo.ogr, osgeo.osr, osgeo.gdal_array, etc. - but I'm not able to get it to work (mostly I muck up the live pointers), there's a lot of code to write and a lot can be automated but I'm not able yet (look at paleolimbot/libproj and paleolimbot/geos for where I'd like to go), but also rgdal did a lot of this already but got sidelined by more successful projects (it's very similar in Python, rasterio and fiona mask the power in the osgeo modules and a lot of downstream effort goes into things that should be fixed in the libs).

-- yes, fixing the libs being harder, technically!

Also, Michael, I noticed you are currently lucky 13! Nice work! https://github.com/gayanvoice/top-github-users/blob/main/markdown/public_contributions/australia.md