Provide option to take a Redfish snapshot for validation

[dchanman]

Uncovered by @JordanCHW: When validating multiple profiles, the current implementation does ValidateURITree for each profile.

Redfish-Interop-Validator/RedfishInteropValidator.py

Lines 196 to 215 in 2ed5dd2

	for profile in all_profiles:
	if 'single' in pmode:
	success, _, resultsNew, _, _ = validateSingleURI(ppath, profile, 'Target', expectedJson=jsonData)
	elif 'tree' in pmode:
	success, _, resultsNew, _, _ = validateURITree(ppath, profile, 'Target', expectedJson=jsonData, check_oem=args.oemcheck)
	else:
	success, _, resultsNew, _, _ = validateURITree('/redfish/v1/', profile, 'ServiceRoot', expectedJson=jsonData, check_oem=args.oemcheck)
	profileName = profile.get('ProfileName')
	if results is None:
	results = resultsNew
	else:
	for item in resultsNew:
	for x in resultsNew[item]['messages']:
	x.name = profileName + ' -- ' + x.name
	if item in results:
	innerCounts = results[item]['counts']
	innerCounts.update(resultsNew[item]['counts'])
	results[item]['messages'].extend(resultsNew[item]['messages'])
	else:
	results[item] = resultsNew[item]

This can slow down the test significantly given that for N profiles, we do N treecrawls to validate all profiles.

Can the implementation instead take a snapshot of the redfish tree prior to ValidateURITree and use the same snapshot for validating all profiles?

[jautor]

That is certainly possible and should not affect the outcome of a profile validation, so having the tree "cached" locally would certainly speed up processing. And since we expect (and encourage) users to build profiles on top of existing ones, this use case is going to become more frequent.
We can probably leverage the code from the Mockup Creator to store a cache copy for processing.

[tomasg2012]

Redfish-Interop-Validator/traverseInterop.py

Line 155 in 2ed5dd2

@lru_cache(maxsize=128)

I was under the impression that this decorator would cache the last calls of this function for later, but maybe this behavior isn't checking out with changes to the program. If it is, it should be caching the responses for each resource based on the URI provided.

edit: It's very possible that the cache limit is too low for many services, especially since you can easily have 128 log entries and the cache will remove the earlier calls, defeating the purpose of the cache. If you are able to test bumping this up to a much higher number, it might work at expense of memory allocation.

Also wondering if stacking the profiles together before validation is more accurate to the spec than validating each profile individually.