Need a way to enable high condor debugging on the Frontend

[StevenCTimm]

Is your feature request related to a problem? Please describe.
The frontend dunegpfrontend01.fnal.gov is having a difficult problem trying to debug authentication with a remote condor_schedd.
We need to get _CONDOR_TOOL_DEBUG=D_SECURITY:2 D_FULLDEBUG set and get the debug output in the logs to figure this out. This is doubly confusing because as user "frontend" from the command line I can do condor_q against this schedd
and it works but the frontend using the python bindings can't do it.

Describe the solution you'd like
Add a configuration knob in the frontend.xml to enable debugging output if necessary.

Describe alternatives you've considered
The frontend doesn't respect the files in /etc/condor/config.d so we can't do it that way.

Info (please complete the following information):

• Priority: Critical--DUNE global pool can't function until we get this in place (and then find and fix the auth error)
• Stakeholders: Concerned stakeholder(s) DUNE
• Components: Frontend

Additional context
The following patch works in the DE:
[root@hepcsvc03 publishers]# diff publisher.py publisher.py.1.7.4
91,96d90
< # S. Timm 10/27/22 add debugging
< htcondor.set_subsystem("TOOL")
< htcondor.param['TOOL_DEBUG'] = 'D_SECURITY:2 D_FULLDEBUG'
< htcondor.param['TOOL_LOG'] = '/tmp/tool.log'
< htcondor.enable_log()
< htcondor.enable_debug()

Those same things will have to be passed in the frontend to the routine that calls the binding but the structure of that code squire different.

[mambelli]

@StevenCTimm There are a couple of commands in the aliases defined in the dev-tools that run condor_q and condor_advertise using the configuration used by the Frontend and do that with debug flags enabled (feccq, fecca)

alias feccq='su -c "CONDOR_CONFIG=/var/lib/gwms-frontend/vofrontend/frontend.condor_config _CONDOR_TOOL_DEBUG=D_FULLDEBUG,D_SECURITY condor_q -debug -global -allusers" -s /bin/bash - frontend'
alias fecca='su -c "CONDOR_CONFIG=/var/lib/gwms-frontend/vofrontend/frontend.condor_config _CONDOR_TOOL_DEBUG=D_FULLDEBUG,D_SECURITY condor_advertise -debug" -s /bin/bash - frontend'

Did you try using these? Would they do what you need?

[StevenCTimm]

Hi Marco—yes those condor commands would do, but it would be nice that you could turn it on in a running frontend.. the main reason is for debugging the authentication issues that always happen wheh you have to periodically authenticate to a new factory. Same should be the case for the decision engine...there I learned how to do it myself. Steve

…

________________________________ From: Marco Mambelli ***@***.***> Sent: Monday, October 14, 2024 11:39 AM To: glideinWMS/glideinwms ***@***.***> Cc: Steven C Timm ***@***.***>; Mention ***@***.***> Subject: Re: [glideinWMS/glideinwms] Need a way to enable high condor debugging on the Frontend (Issue #297) [EXTERNAL] – This message is from an external sender @StevenCTimm<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_StevenCTimm&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=xw_luy8TvJUqblkrOnU03PVo0py7uc3biCkckQ_ALFpsM-k84syeLA2Z3or8DsLj&s=x0kToQFRKmpe0UMklT7bFD3gm-INTTT4jROMcyh8vJ0&e=> There are a couple of commands in the aliases defined in the dev-tools<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_glideinWMS_dev-2Dtools&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=xw_luy8TvJUqblkrOnU03PVo0py7uc3biCkckQ_ALFpsM-k84syeLA2Z3or8DsLj&s=rigq63l8l6_9v_oLChP01Suw28yR1dJUsdYsylH44pA&e=> that run condor_q and condor_advertise using the configuration used by the Frontend and do that with debug flags enabled (feccq, fecca) alias feccq='su -c "CONDOR_CONFIG=/var/lib/gwms-frontend/vofrontend/frontend.condor_config _CONDOR_TOOL_DEBUG=D_FULLDEBUG,D_SECURITY condor_q -debug -global -allusers" -s /bin/bash - frontend' alias fecca='su -c "CONDOR_CONFIG=/var/lib/gwms-frontend/vofrontend/frontend.condor_config _CONDOR_TOOL_DEBUG=D_FULLDEBUG,D_SECURITY condor_advertise -debug" -s /bin/bash - frontend' Did you try using these? Would they do what you need? — Reply to this email directly, view it on GitHub<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_glideinWMS_glideinwms_issues_297-23issuecomment-2D2411751930&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=xw_luy8TvJUqblkrOnU03PVo0py7uc3biCkckQ_ALFpsM-k84syeLA2Z3or8DsLj&s=nxR0UD3CCOBZmtMyOtOtXgjwJtNwAw7vtXV1S04970o&e=>, or unsubscribe<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_notifications_unsubscribe-2Dauth_AGG4SOB75S2BS7U4MFSJQXLZ3PXSRAVCNFSM6AAAAABP5L7YH6VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDIMJRG42TCOJTGA&d=DwMFaQ&c=gRgGjJ3BkIsb5y6s49QqsA&r=10BCTK25QMgkMYibLRbpYg&m=xw_luy8TvJUqblkrOnU03PVo0py7uc3biCkckQ_ALFpsM-k84syeLA2Z3or8DsLj&s=fQ6lko_B2ZH2NVzPp2_xRj91AwZDHXNpNyiVGfxi2oA&e=>. You are receiving this because you were mentioned.Message ID: ***@***.***>