improve progress display accuracy

[lsloan]

Problem

After running new code that handles and ignores errors, one course out of 812 could not have its LTI tools changed. When running the app using the same parameters, there should be only two tool changes for the one course that need to be done. However, the tqdm progress bar says otherwise…

tool-migration_main  | 2023-10-17 13:49:19,471 | INFO | main:113 | Starting migration…
tool-migration_main  | 2023-10-17 13:49:19,472 | INFO | main:120 | Setting environment from file "/app/env".
tool-migration_main  | 2023-10-17 13:49:19,474 | INFO | main:129 | LOG_LEVEL: 20
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:136 | HTTP_LOG_LEVEL: 'INFO'
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:149 | API_URL: "https://umich.beta.instructure.com"
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:152 | API_KEY: *REDACTED*
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:155 | ACCOUNT_ID: (112)
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:159 | ENROLLMENT_TERM_IDS_CSV: [1]
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:162 | SOURCE_TOOL_ID: (37063)
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:165 | TARGET_TOOL_ID: (15593)
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:168 | WH_HOST: 'udw.prod.unizin.org'
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:171 | WH_PORT: '5439'
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:174 | WH_NAME: 'dbx14by8tn'
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:177 | WH_USER: 'duknnize8s2c'
tool-migration_main  | 2023-10-17 13:49:19,475 | INFO | main:180 | WH_PASSWORD: *REDACTED*
tool-migration_main  | 2023-10-17 13:49:19,476 | INFO | main:190 | Warehouse connection is configured, so it will be used for some data fetching…
tool-migration_main  | 2023-10-17 13:49:21,338 | INFO | main:75 | Number of tools found in account 112: 62
tool-migration_main  | 2023-10-17 13:49:24,595 | INFO | utils:60 | WarehouseAccountManager.get_courses_in_terms took 3.2557480335235596 seconds to complete.
tool-migration_main  | 2023-10-17 13:49:24,595 | INFO | main:85 | Number of courses found in account 112 for terms [1]: 812
tool-migration_main  | 2023-10-17 13:49:24,595 | INFO | main:90 | Source tool: ExternalTool(id=37063, name='NameCoach Roster')
tool-migration_main  | 2023-10-17 13:49:24,595 | INFO | main:91 | Target tool: ExternalTool(id=15593, name='Gradescope')
 60%|██████    | 490/812 [00:16<00:09, 34.97it/s]2023-10-17 13:49:40,649 | WARNING | api:86 | HTTP 422: PUT https://umich.beta.instructure.com/api/v1/courses/40857/tabs/context_external_tool_15593?hidden=false&position=44; response: '{"errors":[{"field":"conclude_at","message":"End date cannot be before start date","error_code":null}]}'
 63%|██████▎   | 509/812 [00:16<00:08, 36.10it/s]2023-10-17 13:49:41,266 | WARNING | api:86 | HTTP 422: PUT https://umich.beta.instructure.com/api/v1/courses/40857/tabs/context_external_tool_37063?hidden=true; response: '{"errors":[{"field":"conclude_at","message":"End date cannot be before start date","error_code":null}]}'
831it [00:25, 33.07it/s]
tool-migration_main  | 2023-10-17 13:49:49,739 | INFO | utils:60 | main took 30.15569496154785 seconds to complete.
tool-migration_main  | 2023-10-17 13:49:49,740 | INFO | main:217 | Migration complete.
tool-migration_main exited with code 0

Observations:

1. Each operation, LTI tool hide or show, makes movement in the progress indicator. Therefore, if there were 812 courses to handle, the denominator of the progress completed should be 2x that, 1624.
2. In this particular run, only one course needs changes, so the denominator should be 2.
3. Where is the number for the numerator coming from?

Solutions/Questions

1. Multiply the number of courses by two to get the denominator if there will always be two operations per course.
2. If there is only one course to be changed, can the others be eliminated to get the denominator down?
3. Is the numerator really based on the number of the operations to be performed? In the example above, was there really a list of 812 (or 1624) operations to be done, but because most of them no longer apply, they are rapidly passed by?