Psych DisalowedClass MiqAeEngine::StateVarHash error while retiring an Openstack Instance

[agrare]

[----] I, [2024-07-17T12:30:20.546139#109140:9ad8]  INFO -- evm: MIQ(MiqQueue#deliver) Message id: [10931], Delivering...
[----] I, [2024-07-17T12:30:20.569415#109140:9ad8]  INFO -- evm: Delivering {"request"=>"vm_retire", "VmOrTemplate::vm"=>185} for object [VmRetireTask.2] with state [CheckRemovedFromProvider] to Automate
[----] I, [2024-07-17T12:30:20.582462#109140:9ad8]  INFO -- evm: MIQ(VmRetireTask#after_ae_delivery) ae_result="error"
[----] E, [2024-07-17T12:30:20.625368#109140:9ad8] ERROR -- evm: MIQ(MiqQueue#deliver) Message id: [10931], Error: [Tried to load unspecified class: MiqAeEngine::StateVarHash]
[----] E, [2024-07-17T12:30:20.625477#109140:9ad8] ERROR -- evm: [Psych::DisallowedClass]: Tried to load unspecified class: MiqAeEngine::StateVarHash  Method:[block (2 levels) in <class:LogProxy>]
[----] E, [2024-07-17T12:30:20.625518#109140:9ad8] ERROR -- evm: /home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/class_loader.rb:99:in `find'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/class_loader.rb:28:in `load'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/to_ruby.rb:426:in `resolve_class'
/home/agrare/src/manageiq/manageiq/config/initializers/yaml_autoloader.rb:15:in `resolve_class'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/to_ruby.rb:215:in `visit_Psych_Nodes_Mapping'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/visitor.rb:30:in `visit'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/visitor.rb:6:in `accept'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/to_ruby.rb:35:in `accept'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/to_ruby.rb:320:in `visit_Psych_Nodes_Document'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/visitor.rb:30:in `visit'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/visitor.rb:6:in `accept'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych/visitors/to_ruby.rb:35:in `accept'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych.rb:334:in `safe_load'
/home/agrare/src/manageiq/manageiq/lib/extensions/yaml_load_aliases.rb:7:in `safe_load'
/home/agrare/.gem/ruby/3.1.4/gems/psych-4.0.6/lib/psych.rb:369:in `load'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:166:in `instantiate'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:54:in `instantiate_with_user'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:48:in `block in instantiate'
/home/agrare/src/manageiq/manageiq/app/models/user.rb:383:in `with_user'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb:48:in `instantiate'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine.rb:343:in `resolve_automation_object'
/home/agrare/.gem/ruby/3.1.4/bundler/gems/manageiq-automation_engine-770768456550/lib/miq_automation_engine/engine/miq_ae_engine.rb:106:in `deliver'
/home/agrare/src/manageiq/manageiq/app/models/miq_queue.rb:517:in `block in dispatch_method'

[jrafanie]

First thought is automation engine should have a list of allowed classes and pass it to safe_load here:

manageiq-automation_engine/lib/miq_automation_engine/engine/miq_ae_engine/miq_ae_workspace_runtime.rb

Line 166 in 7707684

@persist_state_hash.merge!(YAML.load(ae_state_data))

Or we need to add this class to the default list:
https://github.com/ManageIQ/manageiq/blob/31ce59bae092487710e925a3a8f78fa622d782dc/lib/yaml_permitted_classes.rb#L2-L14

I don't know if it makes sense to pass it at runtime.rb at that point.

[agrare]

Yeah we either need to allow loading that class or better yet find where it is being saved as an object and serialize/deserialize it properly which I think would be better

Here is what the payload looks like, :ae_state_data=>"--- !ruby/object:MiqAeEngine::StateVarHash\nbinary_blob_id: 53\n" seems to just be a binary_blob_id

Full payload:

[----] I, [2024-07-17T12:29:19.696379#109140:9ad8]  INFO -- evm: Requeuing {:object_type=>"VmRetireTask", :object_id=>2, :attrs=>{"request"=>"vm_retire", "VmOrTemplate::vm"=>185}, :instance_name=>"AUTOMATION", :user_id=>1, :miq_group_id=>2, :tenant_id=>1, :miq_task_id=>nil, :state=>"CheckRemovedFromProvider", :ae_fsm_started=>nil, :ae_state_started=>"2024-07-17 16:29:18 UTC", :ae_state_retries=>1, :ae_state_data=>"--- !ruby/object:MiqAeEngine::StateVarHash\nbinary_blob_id: 53\n", :ae_state_previous=>"---\n\"/ManageIQ/Cloud/VM/Retirement/StateMachines/VMRetirement/Default\":\n  ae_state: CheckRemovedFromProvider\n  ae_state_retries: 1\n  ae_state_started: 2024-07-17 16:29:18 UTC\n  ae_state_max_retries: 100\n"} for object [VmRetireTask.2] with state [CheckRemovedFromProvider] to Automate for delivery in [60] seconds

[jrafanie]

related... #538. fixing the code to not serialize it would be good. when can we delete automate?

[jrafanie]

FYI, #554 is a generic workaround for the loading of aliases. We probably need to treat allowed classes similarly and add it to allowed list directly on the call or centrally as described in #552 (comment). If we knew where these were coming from, it would be better to not serialize these objects in the first place.

[miq-bot]

This issue has been automatically marked as stale because it has not been updated for at least 3 months.

If you can still reproduce this issue on the current release or on master, please reply with all of the information you have about it in order to keep the issue open.