Add support for YAML and JSON loaders

[lcodes]

Given that a properties file is not nearly as convenient to edit as a YAML or JSON file, support to load configuration in these formats would definitely be a good thing.

In my case, it would make the choice of owner to manage application settings a no-brainer.

[lviggiano]

I do agree that YAML would be nice, and I take into account that there are some users interested in this feature. Unfortunately, with current OWNER internal design, configuration is kept into a java.util.Properties object. It is possible to offer some basic yaml support, but not a real full support of the yaml specification, because I would need to map a YAML file into a Properties object (as per the implementation I have now).
So, as first step I'll try to implement some basic YAML support. Then, I'll need to change the OWNER internal data structure to fully support YAML and other more complex formats.

This feature request is related to #34 #14 and #2.

Thanks for your feedback. I will seriously think about YAML support as one of the high priorities for future enhancements.

[lcodes]

Basic YAML support would be a fantastic starting point, however, our needs would also require the support of nested structures, including nested mappings.

On a past Java project I used dots to split property names back into a path identifying the value in the config tree and a semicolon to split the name into hash-map-name:key. Would this also work for OWNER? It would effectively lift the properties support to be as full-featured as json/yaml are (at the cost of generating more verbose .properties files).

Our use case for nested properties is data reuse: when loading the configuration for the "core" plugin we can merge the "global", "service" and "core" values into the final configuration value that is then injected into the corresponding plugin at startup. All of these configurations also contain nested values (for example the hostname of the http server would be the nested key "server.http.host" in any of the top-level keys used to construct the runtime config object. For convenience, all of these plugins are configured from a single deployment/hostname-specific file (which also inherit from deployment/hostname-agnostic files).

This enables for configuration patterns as flexible and powerful as prototype inheritance found in JavaScript or Lua.

[lviggiano]

@ddude wrote:

On a past Java project I used dots to split property names back into a path identifying the value in the config tree and a semicolon to split the name into hash-map-name:key. Would this also work for OWNER?

I don't get it. Can you provide an example?

You mean:

author=\
       name : Dante Alighieri, \
       book : Divine Comedy, \
       birth_year: 1265, death_year: 1321\
       ;\
       name : Alessandro Manzoni, \
       book : The Betrothed, \
       birth_year: 1785, \
       death_year: 1873

Notice that the above is a valid Properties file for Java.

Parsing the above property is not supported by OWNER by default, but it is trivial for a user to implement as client code:

import org.aeonbits.owner.Config;
import org.aeonbits.owner.ConfigFactory;
import org.aeonbits.owner.Converter;

import java.lang.reflect.Method;
import java.util.LinkedHashMap;
import java.util.Map;

/**
 * @author Luigi R. Viggiano
 */
public class MapPropertyExample {
    interface MyConfig extends Config {
        @Separator(";")
        @DefaultValue("name : Dante Alighieri,    book : Divine Comedy, birth_year: 1265, death_year: 1321;" +
                      "name : Alessandro Manzoni, book : The Betrothed, birth_year: 1785, death_year: 1873")
        @ConverterClass(MapPropertyConverter.class)
        Map<String, String>[] authors();
    }

    public static class MapPropertyConverter implements Converter<Map<String,String>> {
        public Map<String, String> convert(Method method, String input) {
            Map<String, String> result = new LinkedHashMap<String, String>();
            String[] chunks = input.split(",", -1);
            for (String chunk : chunks) {
                String[] entry = chunk.split(":", -1);
                String key = entry[0].trim();
                String value = entry[1].trim();
                result.put(key, value);
            }
            return result;
        }
    }

    public static void main(String[] args) {
        MyConfig cfg = ConfigFactory.create(MyConfig.class);
        Map<String, String>[] authors = cfg.authors();
        for (Map<String,String> author : authors) {
            for (Map.Entry<String, String> entry : author.entrySet())
                System.out.printf("%s:\t%s\n", entry.getKey(), entry.getValue());
            System.out.println();
        }
    }
}

...or, since ":" is also a valid separator for the Properties class, we can use Properties objects (that extend from Map) to parse subproperties ...

import org.aeonbits.owner.Config;
import org.aeonbits.owner.ConfigFactory;
import org.aeonbits.owner.Converter;

import java.io.IOException;
import java.io.StringReader;
import java.lang.reflect.Method;
import java.util.Map;
import java.util.Properties;

/**
 * @author Luigi R. Viggiano
 */
public class MapPropertyExample {
    interface MyConfig extends Config {
        @Separator(";")
        @DefaultValue("name : Dante Alighieri,    book : Divine Comedy, birth_year: 1265, death_year: 1321;" +
                      "name : Alessandro Manzoni, book : The Betrothed, birth_year: 1785, death_year: 1873")
        @ConverterClass(MapPropertyConverter.class)
        Map<Object, Object>[] authors();
    }

    public static class MapPropertyConverter implements Converter<Map<Object,Object>> {
        public Map<Object, Object> convert(Method method, String input) {
            try {
                Properties props = new Properties();
                props.load(new StringReader(input.replaceAll(",", "\n")));
                return props;
            } catch (IOException e) {
                throw new RuntimeException("This shouldn't happen since we are reading from a String...", e);
            }
        }
    }

    public static void main(String[] args) {
        MyConfig cfg = ConfigFactory.create(MyConfig.class);
        Map<Object, Object>[] authors = cfg.authors();
        for (Map<Object, Object> author : authors) {
            for (Map.Entry<Object, Object> entry : author.entrySet())
                System.out.printf("%s:\t%s\n", entry.getKey(), entry.getValue());
            System.out.println();
        }
    }
}

By the way, the design of OWNER library is ready to allow more file format (anything that can be mapped to a Properties object, that -unfortunately- at the moment is the internal structure used to keep the internal configuration data; but that may change in future). Have a look at the package org.aeonbits.owner.loaders and the class LoaderManager to see how Properties are loaded from different file types.

P.S. There is also a pull request that I have received recently that is proposing some properties mapping into Map object, but I doubt that is what you are asking about. Anyway I will review and eventually merge that feature in the next days.

[lcodes]

I meant properties keys like this:

book.author;0;name = Dante Alighieri
book.author;0;book = Divine Comedy
book.author;0;birth_year = 1265
book.author;0;death_year = 1321

When loading the properties file I would iterate through the keys and reconstruct the structured data. Of course, it quickly leads to complex keys:

book;0;author;0;address.country = Canada

On another project I only needed one or two mappings for the entire project's configuration, so I chose another route:

fully.qualified.mapping.name = key1:value1
fully.qualified.mapping.name = key2:value2
fully.qualified.mapping.name = key3:value3

These are both hacks to fit types not originally supported by properties files, however. The resulting properties files aren't easily maintained.

If I had to redo the system I would drop the distinction between ; and . and use reflection under the hood to detect whether to parse the value into a field or into a hash-map.

[lviggiano]

I think, there are good frameworks to deserialize json and yaml into java objects. But this task is beyond the initial purpose for this library, anyway looks an interesting feature to add.

Thinking a second time, java Properties objects does support binding java objects to property names, so it is actually possible to deserialize json or yaml into a java.util.Properties object. Still it is not trivial, so it will take time.

[FredDeschenes]

If anyone is interested, I forked the lib and made a basic YAML implementation here : https://github.com/FrimaStudio/owner/tree/yaml

This unfortunately breaks some features (XML loading, saving properties to a file), but hey it "kinda works"™.

I will not create a pull request (unless Luigi wants me too) since this is HIGHLY untested (YAML works fine, not so sure about '.properties' files), breaks core functionality and adds an external dependency. It could give you a good starting point regarding the move from a "java.utils.Properties" backing structure (I used a simple Map<String, Object>) and implementing a JSON loader in it would be dead simple.

[lviggiano]

@FredDeschenes It's for sure an interesting work. I fetched your branch and I'll have a look as soon as I can.

Thanks!

[lviggiano]

Its good to have a look at your work, so I can see which problems you
encountered and what was your solutions.
The principles that are driving me in the development of this library are:

• guarantee backward compatibility
• keeping things reliable, and keeping bugs probability low (having code
  tested)
• no dependencies on 3rd party libraries (or having optional dependencies)
• not being so strict (but also not so lenient) about above principles.

But any work is appreciated on this library, and I appreciate all help and
feedback.
Lately I've been very busy with work and life, but I hope to soon put my
hands heavily on this project.

L.
Il 19/feb/2014 18:01 "FredDeschenes" [email protected] ha scritto:

If anyone is interested, I forked the lib and made a basic YAML
implementation here : https://github.com/FrimaStudio/owner/tree/yaml

This unfortunately breaks some features (XML loading, saving properties to
a file), but hey it "kinda works"™.

I will not create a pull request (unless Luigi wants me too) since this is
HIGHLY untested (YAML works fine, not so sure about '.properties' files),
breaks core functionality and adds an external dependency. It could give
you a good starting point regarding the move from a "java.utils.Properties"
backing structure (I used a simple Map) and implementing a JSON loader in
it would be dead simple.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/53#issuecomment-35521287
.

[rocketraman]

Since this issue talks about YAML and JSON, I'd also like to mention TOML: https://github.com/mojombo/toml. One quote from the TOML wiki:

But why? Because we need a decent human-readable format that unambiguously maps to a hash table and the YAML spec is like 80 pages long and gives me rage. No, JSON doesn't count. You know why.

[ketan]

I've submitted a #144 to add some JSON support. I'm sure it should be straight forward to port it to YAML, in case someone is interested.

[lcodes]

Cleaning up old issues

[joshualibrarian]

I know this is a very old and closed thread, but I really wish your library did support yaml files, even with nice POJO binding, could be driven by Jackson, and as I find myself about to have to do it myself without all your nice bells and whistles, I wonder did that redesign ever happen? Prolly not, but just sayin'. ;)