-
Notifications
You must be signed in to change notification settings - Fork 32
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support for structs in properties #101
Comments
Hi, I like this idea, and it would simplify some things that we do at MindStand internally. So far we've stuck relatively close in features to the Neo4j Java OGM, but there's nothing stopping this project from branching out and implementing interesting features like this. I will say that the approach of flattening a struct into a properties map is a lot more elegant then our initial idea, for this exact problem, which was to serialize the offending field to json and back. One drawback I see is that more complex structs that involve sub-maps or slices would lead to all sorts of interesting edge cases. This solution would also have to have limits so that super nested structs aren't allowed. |
Hi! Thank you for the interest. I've created an awful routine to recursively flatten structs, maps, array/slices and primitive types. func flattenStruct(strct interface{}, prefix string, remove string) (item map[string]interface{}) {
item = make(map[string]interface{})
structValue := reflect.ValueOf(strct)
kind := structValue.Kind()
// Return the real type immediately if pointer or interface
if kind == reflect.Ptr || kind == reflect.Interface {
structValue = reflect.Indirect(structValue)
kind = structValue.Kind()
}
switch kind {
case reflect.Array, reflect.Slice:
for i := 0; i < structValue.Len(); i++ {
for k, v := range flattenStruct(structValue.Index(i).Interface(), prefix, remove) {
k = removeDuplicates(k, prefix, remove)
item[k+prefix+strconv.Itoa(i)] = v
}
}
case reflect.Map:
for _, mapKey := range structValue.MapKeys() {
mapValue := structValue.MapIndex(mapKey)
cleanKey := strings.Replace(mapKey.String(), ":", strings.Repeat(prefix, 2), -1) // TODO: doc
cleanKey = strings.Replace(cleanKey, "-", strings.Repeat(prefix, 3), -1) // TODO: doc
for k, v := range flattenStruct(mapValue.Interface(), prefix, remove) {
val := fmt.Sprintf("%v", v)
if k == "string" {
item[cleanKey] = val
} else {
k = removeDuplicates(k+prefix+cleanKey, prefix, remove)
item[k] = val
}
}
}
case reflect.String:
item[reflect.TypeOf(strct).Name()] = structValue.String()
case reflect.Struct:
for structIterator := 0; structIterator < structValue.NumField(); structIterator++ {
field := structValue.Field(structIterator)
// Only exported fields
if field.CanInterface() {
key := structValue.Type().Field(structIterator).Name
value := field.Interface()
switch valueCasted := value.(type) {
case string, bool, time.Time, int, int32:
item[key] = valueCasted
case *string:
item[key] = aws.ToString(valueCasted)
case *time.Time:
if valueCasted != nil {
item[key] = *valueCasted
}
case *int32:
if valueCasted != nil {
item[key] = *valueCasted
}
case *bool:
item[key] = aws.ToBool(valueCasted)
default:
// Another struct, recursively return the fields
for k, v := range flattenStruct(value, prefix, remove) {
k = removeDuplicates(key+prefix+k, prefix, remove)
item[k] = v
}
}
}
}
}
return
} this is ugly but it's working to map AWS API JSON structs output into neo4j node properties...may be this could be a starting point |
That's a great solution, the resulting data should still be perfectly queriable from neo4j without having to resort to something like using APOC. There are some interesting edge cases at play, like how we should handle loops (and for that matter if we have two references to the same struct, do we duplicate the data in the resulting structure). If we limit the property structs to a certain depth, how should we handle structs that exceed the depth? Should we error out, or truncate the output, or should we check if a struct is valid when the decorators are being parsed? Definitely looking for feedback here (@erictg 👀). Would you be interested in implementing this idea? We can definitely provide you with pointers on the encoder/decoder as well as the struct decorator. |
I'd like to give it try if you can point me where to put the code and lead me on how the data is flowing in the library!
I think it's the user's responsibility to pass/create non-duplicated fields in the struct.
is it really a problem here? The only caveat that I see it's the computational time and eventually recursive structs mistakenly created by the user (i.e. a field that reference the struct itself, I don't if it's even possible). I'm open to discuss all of the above points :) |
Hey @notdodo, I apologize for not responding sooner! I've been super busy. ApproachesSo off the top of my head I can think of 2 approaches you could take to serializing this.
I would personally chose option 2. Tbh the more I think about this the more I think option 1 isn't practical. Areas of code to be updated to support this1. Decorator logic
2. Save logic
3. Decode logic
In my opinion, the hardest part for this will probably be the decode logic, since you would have to assemble up rather the recursively build out the fields. I would also recommend trying to store as much information about the whole object tree in the I definitely think this is worth doing, feel free to reach out here for help. I will be much more responsive on this going forward. Additionally thank you for your support in gogm and willingness to implement a cool feature like this :) |
In the coming weeks i'll be adding more test and comment coverage, which should definitely help. If you come across code that doesn't make sense, feel free to ask what it does! |
Hi @erictg! I took another approach: since all structures in Golang can be marshalled to a JSON string I've created a library that given a JSON input string it returns a JSON string with all flatten fields. The library is here: https://github.com/notdodo/goflat I think that this library (or just copy/paste the code) could be integrated here to do what we want to achieve. |
@notdodo Is this lib something that you would use in conjunction with gogm, or would you have to edit the gogm logic itself for this? Im running into this same thing and wondering how to do this: type IndexTemplate struct {
gogm.BaseNode
Name string `gogm:"name=Name" json:"index_pattern"`
Spec struct {
Mappings struct {
Meta struct {
Description string `gogm:"name=description" json:"description"`
} `json:"_meta"`
} `json:"mappings"`
Retention string `gogm:"name=Retention" json:"Retention"`
PolicyName string `gogm:"name=PolicyName" json:"PolicyName"`
ComponentTemplateName string `gogm:"name=ComponentTemplateName" json:"ComponentTemplateName"`
Environment string `gogm:"name=Environment" json:"Environment"`
TotalDocCount int `gogm:"name=TotalDocCount" json:"TotalDocCount"`
TotalIndexCount int `gogm:"name=TotalIndexCount" json:"TotalIndexCount"`
TotalShardCount int `gogm:"name=TotalShardCount" json:"TotalShardCount"`
} `gogm:"name=Spec;properties" json:"spec"`
ComponentTemplate *ComponentTemplate `gogm:"direction=incoming;relationship=MANAGES" json:"-"`
Policy *LifecyclePolicy `gogm:"direction=incoming;relationship=MANAGES" json:"-"`
} |
Feature Request:
Add support for this kind of properties definition:
It should be useful to store custom defined structs inside the node properties; as a side note a possible way to achieve this could be to flatten the struct into something like
TestStruct_A
,TestStruct_B
,TestStruct_C_D_0
.Context
This feature could be used to directly upload complex struct into the node (i.e. coming from a JSON response/data).
Alternatives
I don't it is possible without manually rewriting the node structure with the required fields since the library only support primitive types.
Would you be interested in implementing this feature?
Yes
The text was updated successfully, but these errors were encountered: