[FIX]: Unnecessary Logs, Preventing System Inefficiencies and Inefficient Retries While Populating Dashboard Database

[JiyaGupta-cs]

Issue Overview

When running the rake populate:dev task, the system was encountering a ActiveRecord::RecordNotUnique error due to duplicate entries in the campaigns_courses table. This error led to the creation of excessive logs, making it difficult to analyze system behavior and troubleshoot issues efficiently.

Additionally, if a duplicate record is encountered for one entry during the execution of the populate_dashboard task, the process halts further attempts to populate subsequent entries. This causes the database to remain partially populated, failing to insert other valid records.

The presence of a single pre-existing record in the database could arise if the user terminates the populate:dev command prematurely. This interruption might happen because the process takes too long to fetch and insert data, leading to an incomplete population of demo records.

Closes #6079

Proposed Solution

To address the issues with the populate:dev task and ensure a smoother data population process, the following solutions were implemented:

1. Handle Duplicate Records Gracefully:
   Adjusted the logic to check for existing entries before creating new records. This prevents duplicate records from causing the process to fail with ActiveRecord::RecordNotUnique errors.

2. Continue Processing After Errors:
   Introduced exception handling to ensure that encountering an error (e.g., a duplicate record) does not stop the task. Instead, the task skips the problematic entry and continues processing subsequent entries, ensuring the database is populated with all valid records.

3. Avoid Redundant Associations:
   Added checks to avoid re-associating courses that already exist in a campaign, eliminating unnecessary database operations and log noise.

4. Improved Logging and Error Reporting:
   Enhanced logging to provide clear, actionable error messages when issues arise, improving the debugging experience.

Changes Made

1. Duplicate Record Prevention:

• Introduced a conditional check in the make_copy_of function to verify the existence of records (e.g., courses, users, wikis) before attempting to create them.

2. Graceful Error Handling:

• Wrapped key operations in begin-rescue blocks to catch and log errors like ActiveRecord::RecordInvalid and other unexpected issues.
• Skips problematic entries instead of halting the entire process, ensuring subsequent entries are processed.

3. Optimized Campaign Association:

• Before associating a course with a campaign, the code checks if the course is already part of the campaign, avoiding redundant operations.

4. Logging Improvements:
   Added detailed error and progress messages using Rails.logger and puts to provide better visibility into the task execution.

5. Database Population Flow:

• Ensured all related data, such as users, assignments, and wikis, are fetched and created efficiently.
• Avoided re-adding the home wiki when processing related wikis.

Screenshots -

Before -

After -