-
Notifications
You must be signed in to change notification settings - Fork 4
GSIP 106 ResourceStore API
The JDBC config module is a great way to share configuration, except it missed out on configuration file management.
This proposal is to introduce a file management APIallowing JDBC config store configuration files as blobs in the database. File access will be streaming (as often as we can) and unpacked into the data directory as a last resort.
Jody Garnett
This proposal is for GeoServer 2.6-beta.
Choose one of:
- Under Discussion
- In Progress
- Completed
This proposal completes the JDBC Config module, allowing geoserver to configuration to be responsible for configuration files that have been outside of our XStream configuration set-up used for core geoserver.
- SLD support files such as icons and fonts
- Extensions such as monitor that rely on configuration files
- It may help community modules such as app-schema that rely on the file system (but specific testing is outside of scope).
This proposal introduces a ResourceStore API as an alternative to File access. Resources can be accessed directly as an input stream or output stream. Or they can be unpacked into the data directory for use as a file.
- Initial ResourceStore API, integration into core classes (GeoServerResourceLoader,GeoServerDataDirectory,GeoServerSecurityManager).
- Addition of file locking
- Addition of events (allowing update of “file watcher code”)
- Transition of GeoServer codebase module by module
Design docs and migration planning:
This proposal touches on a the entire codebase, please discuss any suggestions on the email list so everyone can take part.
Ideas:
- Q: Add file locking? A: Good idea added to proposal.
- Q: Is it worth scanning SLD files to determine icons used? A: Easier to access directory (and thus grab all directory contents).
- Q: How to ignore some resources - choice of opt-in (module lists the files to managed) or opt out (list files like the geowebcache tmp that should be ignored). A: JDBCConfig currently hard codes a small list of files.
Out of scope:
- It may be possible to construct a wicket UI to review “managed files” which will be of benefit even (even if jdbcconfig is not being used)
Initial proposal:
- geoserver-devel discussion: GSIP 106 - Managed File API
- IanS: Consider implementing Atomic-File-Write (use file lock, write to separate file and rename into place)
- aaime: Look at file watcher / file lock code to track changes in the data directory
- aaime: Good implementation in GWC LockManager (which allows other non NIO lock based implementations to be plugged in).
- Ben: Q: Can we generate a list of associated files (example mapping files which reference other mapping files). A: Use resource.dir () to grab all the files in a directory
- aaime: Q: going to create some support/replacement for property file watcher? A: Yes events are scheduled for phase 2.
- aaime: The watchers ensure we do not check the filesystem too often, similar approach will be needed to ensure we do not check the database 100 times in a second.
Feedback on pull request 531] ResourceStore:
- Can resource be null? ResourceStore should return a Resource, throw IllegalArgumentException when faced with an invalid path.
- Are the null checks needed on Resources methods? Based on the above no!
- Resources.EMPTY is confusing. Replaced with ResourceStore.EMPTY
- Variable names: hard to at a glance if a res variable is Resource or ResourceInfo.
| local variable | Type | Note |
|---|---|---|
| resource | Resource | |
| directory | Resource | |
| location | String | File location (used in GeoServerResourceLoader) |
| path | String | Resource path |
Feedback on pull request 538] Lock:
- lack of GWC header? Andrea clarified code created in 2012
- user interface for changing lock strategy? done
Feedback on pull request 562] ResourceListener:
- aaime: simplify Content and Content.Read
Feedback on pull request 568] Migrate GeoserverDataDirectory:
- kevin: Make GeoServerExtensions as easy to use as GeoserverDataDirectory
- GeoServerExtensionsHelper discussion
- (fixed) GEOS–6461 GeoServerPersister stray namespace and workspace files
State here any backwards compatibility issues.
- data directory: this proposal does not modify the data directory
- file access: community modules that do not update to the new API may break if used on a node that has not unpacked the appropriate file.
Regressions noted during [Resource API Transition Plan]:
- Simplification: GeoServerResourceLoader search locations removed (as it was only used once)
- Regression: GeoServerJ2eeRoleService access WEB-INF/web.xml no longer uses GeoServerResourceLoader.find (“web.xml”) due to removal of search locations
- Simplification: Relative path used to unpack test resources no longer used directly as target filename (example: styles/../temperature.sld now unpack to styles/temperature.sld)
- GeoServerDataDirectory findDataFile and findDataDir behaved in an inconsistent fashion depending on the presence of the “data” folder. These methods now operate in a consistent fashion.
- Alessio Fabiani: +1
- Andrea Aime: +1
- Ben Caradoc Davies: +1
- Christian Mueller:
- Gabriel Roldan: +1
- Jody Garnett: +1
- Jukka Rahkonen:
- Justin Deoliveira: +1
- Phil Scadden: +1
- Simone Giannecchini: +0