Registry federation

[afeliachi]

As it has been discussed the registry federation works only with namespace forwarding and delegated register. The Registry federation as described in here doesn't work as intended.

Having the functions described in this part of the wiki must be enough for our needs.
Do you need further information to see how this could be added

[der]

At the risk of repeating information that might be captured elsewhere ... could we have some details on the use case and the critical behaviour you need?

That part of Principles and Concepts is not that precisely specified and caused some confusion [1] at the time which is one reason it has not been implemented. In the design, as documented there, the federation option was the same as the namespace forwarding with the exception of the support for federated search. Does that match your requirements, is federated search the key function you need?

[1] Maybe "confusion" is the wrong term. The text is relatively clear but some users hoped, from the name, more functionality than described there. Such as somehow wrapping the federated register within the look-and-feel and navigation structure of the parent registry.

[afeliachi]

Thanks Dave

[1] Maybe "confusion" is the wrong term. The text is relatively clear but some users hoped, from the name, more functionality than described there. Such as somehow wrapping the federated register within the look-and-feel and navigation structure of the parent registry.

This is not what we understood :)

... the federation option was the same as the namespace forwarding with the exception of the support for federated search. Does that match your requirements, is federated search the key function you need?

That's quite the behaviour we understood & expected. In addition we expected also Sparql query to be federated too (but I know this part can be inherently handled by Sparql query federation).

While thinking about the different possible implementations and their consequences. we are asking ourselves about the number of requests that the registries of federated registers would get every time federated-search query is performed.
For example if in BRGM we have 150 registers but only 100 are declared as federated registers on national registry. Does that mean that every time a search is performed on the national registry level, that would generate 100 search request toward our BRGM registry, or do you imagine one request for them all? How do you foresee the implementation?

Thanks

[der]

The cost of federated search was another reason this was never implemented :)

It would, by default, indeed mean you'd be hit by 100 separate search requests for each top level request. Essentially the search in the federating registry would need to check for all federated registers that fall under the requested search point and then send a search-in-register request to all of them (somewhat in parallel via some thread pool executor).

When the feature was first proposed we discussed various caching schemes to make it usable but none of them were workable.

I guess in principle, in your case, then the federating register could be clever and decide when there were "enough" requests to be made to the same registry (but different registers) that it was worth merging requests into a single global search and then filtering the results against target registers. Among the challenges would be knowing what the top level registry is since people can currently deploy with or without a root segment. The federation declaration would have to be extended to include the associated registry service (or it's search endpoint).

That would reduce the load on the federated registry service(s) and so hopefully improve the responsiveness of search in the federating registry a little. At the cost of more complex implementation.

Separately, regarding Sparql queries, then I would not be keen on federating those. While technically possible that would be a whole major piece of implementation beyond what's already supported for Sparql I was very reluctant to expose Sparql queries in the first place due the complexity of the registry data model and the scope for accidental denial of service. Exposing large fanout federated queries would just compound that.