City parameters could be refactored to separate configuration files for use across different code/projects

[carlhiggs]

In some of the matsim-melbourne network code (and related work that uses this ), there are parameter settings referring to data paths and other aspects for specific regions that could change in the future. If that occurred, or new regions were wanting to be modelled, the code would have to be edited.

It could be useful to extract these city-specific conguration parameters to external configuration files. This would have several benefits:

• reduce need to rewrite code
• reduce code length and complexity
• could re-use single source of city configuration parameters across multiple files or projects

The following excerpt of the NetworkGenerate.R code contains around 100 lines devoted to city-specific parameters in the makeNetwork() function:

network/NetworkGenerator.R

Lines 2 to 111 in 6711eda

	# city = "Bendigo"
	# city = "Melbourne"
	# outputSubdirectory = "generated_network"
	# Parameters --------------------------------------------------------------
	# CITY PARAMETERS
	# City parameters to be set
	# • region: if 'downloadOsm=T', file delineating the boundary of the area for
	# which Osm extract is to be downloaded (assumed to be in sqlite format
	# with a single layer)
	# • outputCrs: desired coordinate system for network
	# • osmGpkg: location where downloaded OSM extract for region is to be stored
	# (if 'downloadOsm=T') and/or read from (if 'processOsm=T')
	# • unconfiguredSqlite: location where processed OSM file is to be stored
	# (if 'networkFromOsm=T') or read from (if 'networkFromOsm=F')
	# • cropAreaPoly: if 'crop2TestArea=T' cropArea location from
	# https://github.com/JamesChevalier/cities/tree/master/australia/victoria
	# (only supported for Victoria at this stage)
	# • demFile: if 'addElevation=T', digital elevation model raster file
	# • ndviFile: if 'addNDVI=T', raster file with NDVI values
	# • gtfs_feed: if 'addGtfs=T' or 'addDestinationLayer=T, zip file containing
	# GTFS data (and, if 'addGtfs=T', also set analysis date in GTFS section)
	if (city == "Bendigo") {
	region = "./data/greater_bendigo.sqlite"
	outputCrs = 7899
	osmGpkg = "./output/bendigo_osm.gpkg"
	unconfiguredSqlite = "./output/bendigo_network_unconfigured.sqlite"
	cropAreaPoly = "" # must set 'crop2Area=F'
	demFile = "./data/dem_bendigo.tif"
	ndviFile = "./data/NDVI_Bendigo_2023.tif"
	gtfs_feed = "./data/gtfs.zip"
	} else if (city == "Melbourne") {
	region = "./data/greater_melbourne.sqlite"
	outputCrs = 7899
	osmGpkg = "./output/melbourne_osm.gpkg"
	unconfiguredSqlite = "./output/melbourne_network_unconfigured.sqlite"
	cropAreaPoly = "city-of-melbourne_victoria"
	demFile = "./data/dem_melbourne.tif"
	ndviFile = "./data/NDVI_Melbourne_2023.tif"
	gtfs_feed = "./data/gtfs.zip"
	} else {
	echo(paste("City parameters for", city, "have not been set; unable to proceed\n"))
	return()
	}
	# REGION BUFFER DISTANCE
	# Distance to buffer region when getting osm extract, destinations or gtfs routes
	regionBufferDist=10000
	# DOWNLOAD OSM EXTRACT
	# A flag for whether to download osm extract for the region (if not, and if
	# network needs to be processed, then must already have osmGpkg file)
	downloadOsm=T
	retainDownload=F # Whether to retain downloaded file after region extracted
	# NETWORK FROM OSM
	# A flag for whether to build unconfigured network from osm extract (if not,
	# must already have unconfigured sqlite)
	networkFromOsm=T
	saveUnconfigured=T
	# SIMPLIFICATION
	shortLinkLength=20
	minDangleLinkLengh=500
	crop2Area=F
	# DENSIFICATION
	densificationMaxLength=500
	densifyBikeways=T
	# CAPACITY ADJUSTMENT
	# A flag for whether to multiply capacity of links shorter than 100m by 2 or not
	# In some cases such as when building network for simulation of small samples (e.g. <1%) it might be desired
	adjustCapacity=F
	# ELEVATION
	# A flag for whether to add elevation or not
	addElevation=T
	ElevationMultiplier=1
	# DESTINATIONS
	# A flag for whether to add a destinations layer (drawn from OSM, and GTFS for PT) or not
	addDestinationLayer=T
	# NDVI
	# A flag for whether to add NDVI or not
	addNDVI=T
	# Buffer distance for finding average NDVI for links
	ndviBuffDist=30
	# GTFS
	# A flag for whether to add a network based on GTFS or not
	addGtfs=T
	# Select an analysis date, eg a midweek day that's not a public or school holiday
	analysis_date=as.Date("2023-11-15","%Y-%m-%d")
	onroadBus=T # whether to route buses on roads (rather than create separate pseudo links)
	# Outputs
	# outputSubdirectory=format(Sys.time(),"%d%b%y_%H%M") # date_hour, eg. "17Aug21_1308"
	if(exists("outputSubdirectory")){
	outputSubdirectory=outputSubdirectory
	} else {outputSubdirectory="generated_network"}
	writeXml=F
	writeShp=F
	writeSqlite=T

I would propose that these lines could be removed, and instead of asking for the following arguments

makeNetwork<-function(city, outputSubdirectory = "generated_network")

instead, the function could work as follows:

makeNetwork<-function(region_configuration_file)

where region is the path to a region configuration file, like `./regions/Bendigo.yml'.

Then, the parameters in this file could be loaded in .R or .sh scripts by calling a sub-function, accepting command line arguments

  # Load configuration
  commandArgs <- function(...) region_configuration_file
  source('load_configuration.R')

I've made a demonstration of how these changes could work on a branch of a related project that uses this code, demonstrating the loading of city parameters from configuration files into R global environment or Shell environments.
https://github.com/jibeproject/networkMelbourne/compare/refactor-configuration

This contains an example region files using the current parameters from the code for Bendigo.

If of use, I could make a similar branch here, and you could see if these changes would be useful!