Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Iris: Enhance student support with proactive assistance #9558

Merged
merged 76 commits into from
Dec 10, 2024

Conversation

kaancayli
Copy link
Contributor

@kaancayli kaancayli commented Oct 21, 2024

⚠️ Contains Database Migration - Deploy on TS3

Checklist

General

Server

  • Important: I implemented the changes with a very good performance and prevented too many (unnecessary) and too complex database calls.
  • I strictly followed the principle of data economy for all database calls.
  • I strictly followed the server coding and design guidelines.
  • I added multiple integration tests (Spring) related to the features (with a high test coverage).
  • I documented the Java code using JavaDoc style.

Client

  • Important: I implemented the changes with a very good performance, prevented too many (unnecessary) REST calls and made sure the UI is responsive, even with large data (e.g. using paging).
  • I strictly followed the principle of data economy for all client-server REST calls.
  • I strictly followed the client coding and design guidelines.
  • Following the theming guidelines, I specified colors only in the theming variable files and checked that the changes look consistent in both the light and the dark theme.
  • I added multiple integration tests (Jest) related to the features (with a high test coverage), while following the test guidelines.
  • I documented the TypeScript code using JSDoc style.
  • I added multiple screenshots/screencasts of my UI changes.
  • I translated all newly inserted strings into English and German.

Motivation and Context

Implements the feature proposal described in #8800

Description

  • Added a new proactivity feature to Iris
  • Added PyrisEventService that dispatches the detected events to their handler methods
  • Added eventVariant to pipeline execution parameters
  • Defined 3 event types Iris can react to (jol, build_failed, progress_stalled)
  • Build failed event gets triggered when the a submission fails to build
  • Progress stalled event is triggered, when the student's score is not increasing after at least 3 subsequent submissions
  • Added proactivity settings in Iris chat settings for the new proactivity feature
  • Added event payloads to pipeline execution DTOs

Steps for Testing

Prerequisites:

  • 1 Instructor
  • 1 Student
  • 1 Programming Exercise with Iris Enabled
  • Iris terms and conditions are already accepted
  • Monitor student submission build failures under Iris Chat Settings Enabled
  • Monitor exercise performance progress under Iris Chat Settings Enabled
  • Exercise Chat: Implement native function calling agent Pyris#154 merged or deployed on pyris-test
  1. Log in to Artemis

  2. Navigate to the programming exercise with Iris enabled

  3. Make sure the proactive event options under Iris Chat Settings are enabled in all levels:

    • Global
    • Course
    • Exercise
  4. Testing events:

    1. Build failed event:

      • Make a programming exercise submission with a failing build
      • Verify that Iris reacted and sent a proactive message related to the error
    2. Progress stalled event:

      • Make 3 consequent programming exercise submissions
      • Make sure the scores after each submission are not increasing (e.g., you can submit the initial repository 3 times)
      • Verify that Iris reacted and sent a proactive message related to the error

Testserver States

Note

These badges show the state of the test servers.
Green = Currently available, Red = Currently locked
Click on the badges to get to the test servers.







Review Progress

Code Review

  • Code Review 1
  • Code Review 2

Manual Tests

  • Test 1
  • Test 2

Test Coverage

Client

Class/File Line Coverage Confirmation (assert/expect)
iris-sub-settings.model.ts 100%
exercise-chatbot-button.component.ts 87.3%
iris-chat.service.ts 89.04%
iris-common-sub-settings-update.component.ts 88.88%

Server

Class/File Line Coverage Confirmation (assert/expect)
IrisCourseSettings.java 100%
IrisExerciseSettings.java 94%
IrisGlobalSettings.java 100%
IrisSettings.java 100%
IrisSubSettings.java 100%
IrisSubSettingsType.java 100%
IrisCombinedSettingsDTO.java 100%
IrisSettingsRepository.java 100%
IrisCompetencyGenerationService.java 82%
PyrisConnectorService.java 79%
PyrisEventService.java 100%
PyrisPipelineService.java 80%
PyrisEventDTO.java 100%
PyrisCourseChatPipelineExecutionDTO.java 100%
NewResultEvent.java 100%
PyrisEvent.java 100%
IrisCourseChatSessionService.java 68%
IrisExerciseChatSessionService.java 91%
IrisTextExerciseChatSessionService.java 86%
IrisSettingsService.java 91%
IrisSubSettingsService.java 91%
IrisExerciseChatSessionResource.java 91%

Screenshots

Proactivity Settings

Screenshot 2024-11-21 at 23 41 00
Screenshot 2024-11-21 at 23 31 23

Proactive responses

ScreenRecording2024-11-21at23 37 59-ezgif com-video-to-gif-converter

Summary by CodeRabbit

Release Notes

  • New Features

    • Introduced new proactive event management settings for chat, allowing users to enable or disable events based on their preferences.
    • Added support for new event types, including BUILD_FAILED and PROGRESS_STALLED, enhancing the system's responsiveness to user interactions.
    • Enhanced the chat interface with a message bubble feature that previews new messages.
    • Added new methods for managing submissions and event notifications, improving integration with the Pyris event system.
    • Implemented new exception classes for better error handling related to event processing.
    • Added a new method for retrieving submissions with results, improving data access.
    • Introduced a new PyrisEventDTO for encapsulating event data along with its type.
    • Added functionality for managing proactive settings directly within the UI for chat sub-settings.
  • Bug Fixes

    • Improved error handling for unsupported events and refined logging for better traceability.
  • Documentation

    • Updated documentation to reflect the new event types and proactive settings.
  • Tests

    • Expanded test coverage for new event management features and ensured robust handling of edge cases in event processing.
    • Added integration tests for the Pyris event system to validate behavior under various scenarios.

@github-actions github-actions bot added tests server Pull requests that update Java code. (Added Automatically!) client Pull requests that update TypeScript code. (Added Automatically!) database Pull requests that update the database. (Added Automatically!). Require a CRITICAL deployment. atlas Pull requests that affect the corresponding module core Pull requests that affect the corresponding module exercise Pull requests that affect the corresponding module iris Pull requests that affect the corresponding module programming Pull requests that affect the corresponding module labels Oct 21, 2024
Copy link

There hasn't been any activity on this pull request recently. Therefore, this pull request has been automatically marked as stale and will be closed if no further activity occurs within seven days. Thank you for your contributions.

@github-actions github-actions bot added the stale label Oct 29, 2024
@kaancayli kaancayli force-pushed the feature/iris/event-service branch from 1052ebc to 41dc8dd Compare November 4, 2024 20:31
@kaancayli kaancayli force-pushed the feature/iris/event-service branch from 41dc8dd to 8f0912f Compare November 4, 2024 20:34
@github-actions github-actions bot removed the stale label Nov 5, 2024
sebastianloose
sebastianloose previously approved these changes Dec 5, 2024
Copy link
Contributor

@sebastianloose sebastianloose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reapprove 👍

@krusche krusche added this to the 7.7.5 milestone Dec 5, 2024
@krusche krusche changed the title Iris: Proactive Event System Iris: Enhance student support with proactive assistance Dec 5, 2024
yassinsws
yassinsws previously approved these changes Dec 8, 2024
Copy link
Contributor

@yassinsws yassinsws left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

reapprove

Copy link

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🧹 Outside diff range and nitpick comments (1)
src/test/java/de/tum/cit/aet/artemis/iris/settings/IrisSettingsIntegrationTest.java (1)

434-464: Consider extracting helper methods for better readability.

The test method effectively verifies the combination of disabled events but could benefit from helper methods to improve readability and reusability.

Consider extracting helper methods:

private IrisCourseSettings createCourseSettingsWithDisabledEvent(Course course, IrisEventType eventType) {
    var settings = new IrisCourseSettings();
    settings.setCourse(course);
    settings.setIrisChatSettings(new IrisChatSubSettings());
    settings.getIrisChatSettings().setEnabled(true);
    settings.getIrisChatSettings().setSelectedVariant(null);
    settings.getIrisChatSettings().setDisabledProactiveEvents(new TreeSet<>(Set.of(eventType.name().toLowerCase())));
    return settings;
}
📜 Review details

Configuration used: .coderabbit.yaml
Review profile: CHILL

📥 Commits

Reviewing files that changed from the base of the PR and between b9955aa and 6a36edb.

⛔ Files ignored due to path filters (1)
  • src/main/resources/config/liquibase/master.xml is excluded by !**/*.xml
📒 Files selected for processing (12)
  • src/main/java/de/tum/cit/aet/artemis/iris/service/session/IrisCourseChatSessionService.java (2 hunks)
  • src/main/java/de/tum/cit/aet/artemis/iris/service/settings/IrisSettingsService.java (6 hunks)
  • src/main/java/de/tum/cit/aet/artemis/iris/service/settings/IrisSubSettingsService.java (5 hunks)
  • src/main/webapp/app/entities/iris/settings/iris-sub-settings.model.ts (1 hunks)
  • src/main/webapp/app/iris/settings/iris-settings-update/iris-common-sub-settings-update/iris-common-sub-settings-update.component.ts (7 hunks)
  • src/main/webapp/i18n/de/iris.json (2 hunks)
  • src/main/webapp/i18n/en/iris.json (2 hunks)
  • src/test/java/de/tum/cit/aet/artemis/iris/AbstractIrisIntegrationTest.java (1 hunks)
  • src/test/java/de/tum/cit/aet/artemis/iris/settings/IrisSettingsIntegrationTest.java (5 hunks)
  • src/test/javascript/spec/component/iris/settings/iris-course-settings-update.component.spec.ts (3 hunks)
  • src/test/javascript/spec/component/iris/settings/iris-global-settings-update.component.spec.ts (1 hunks)
  • src/test/javascript/spec/component/iris/settings/mock-settings.ts (2 hunks)
🚧 Files skipped from review as they are similar to previous changes (9)
  • src/test/java/de/tum/cit/aet/artemis/iris/AbstractIrisIntegrationTest.java
  • src/main/webapp/app/entities/iris/settings/iris-sub-settings.model.ts
  • src/test/javascript/spec/component/iris/settings/mock-settings.ts
  • src/test/javascript/spec/component/iris/settings/iris-global-settings-update.component.spec.ts
  • src/test/javascript/spec/component/iris/settings/iris-course-settings-update.component.spec.ts
  • src/main/webapp/i18n/en/iris.json
  • src/main/webapp/i18n/de/iris.json
  • src/main/java/de/tum/cit/aet/artemis/iris/service/settings/IrisSubSettingsService.java
  • src/main/webapp/app/iris/settings/iris-settings-update/iris-common-sub-settings-update/iris-common-sub-settings-update.component.ts
🧰 Additional context used
📓 Path-based instructions (3)
src/test/java/de/tum/cit/aet/artemis/iris/settings/IrisSettingsIntegrationTest.java (1)

Pattern src/test/java/**/*.java: test_naming: descriptive; test_size: small_specific; fixed_data: true; junit5_features: true; assert_use: assertThat; assert_specificity: true; archunit_use: enforce_package_rules; db_query_count_tests: track_performance; util_service_factory_pattern: true; avoid_db_access: true; mock_strategy: static_mocks; context_restart_minimize: true

src/main/java/de/tum/cit/aet/artemis/iris/service/session/IrisCourseChatSessionService.java (1)

Pattern src/main/java/**/*.java: naming:CamelCase; principles:{single_responsibility,small_methods,no_duplication}; db:{perf_queries,datetime_not_timestamp}; rest:{stateless,singleton,delegate_logic,http_only,minimal_dtos}; dtos:{java_records,no_entities,min_data,single_resp}; di:constructor_injection; kiss:simple_code; file_handling:os_indep_paths; practices:{least_access,avoid_transactions,code_reuse,static_member_ref,prefer_primitives}; sql:{param_annotation,uppercase,avoid_subqueries};java:avoid_star_imports

src/main/java/de/tum/cit/aet/artemis/iris/service/settings/IrisSettingsService.java (1)

Pattern src/main/java/**/*.java: naming:CamelCase; principles:{single_responsibility,small_methods,no_duplication}; db:{perf_queries,datetime_not_timestamp}; rest:{stateless,singleton,delegate_logic,http_only,minimal_dtos}; dtos:{java_records,no_entities,min_data,single_resp}; di:constructor_injection; kiss:simple_code; file_handling:os_indep_paths; practices:{least_access,avoid_transactions,code_reuse,static_member_ref,prefer_primitives}; sql:{param_annotation,uppercase,avoid_subqueries};java:avoid_star_imports

🔇 Additional comments (9)
src/test/java/de/tum/cit/aet/artemis/iris/settings/IrisSettingsIntegrationTest.java (3)

149-152: LGTM! Correctly handles collection order independence.

The modification properly accounts for the order-independent nature of the disabledProactiveEvents collection in the assertion.


445-445: Verify case sensitivity handling in the system.

The use of toLowerCase() on enum names suggests potential case sensitivity issues in the system. This could lead to inconsistencies if the case handling is not standardized across all components.

Also applies to: 457-457

✅ Verification successful

Based on the search results, I can now generate the final response:

Case sensitivity is consistently handled throughout the codebase

The codebase shows a standardized approach to case handling for event types:

  • The enum IrisEventType defines the events in uppercase (BUILD_FAILED, PROGRESS_STALLED)
  • All comparisons consistently use toLowerCase() when storing/checking disabled events
  • The frontend (TypeScript) and backend (Java) maintain consistent casing in their interfaces
  • API endpoints and mock tests demonstrate consistent lowercase usage for event parameters
🏁 Scripts executed

The following scripts were executed for the analysis:

Script:

#!/bin/bash
# Description: Check for inconsistent case handling of event types
# Test: Search for different case variations of event type handling

# Look for event type case handling
rg -A 2 "IrisEventType\." 

# Look for direct string usage of event types
rg -i "progress_stalled|build_failed"

Length of output: 13573


252-270: 🛠️ Refactor suggestion

Use IrisEventType enum instead of string literals.

The test correctly verifies the functionality but uses string literals for event types. Using enum constants would make the code more maintainable and type-safe.

Apply this diff to use enum constants:

-        loadedSettings1.getIrisChatSettings().setDisabledProactiveEvents(new TreeSet<>(Set.of("PROGRESS_STALLED")));
+        loadedSettings1.getIrisChatSettings().setDisabledProactiveEvents(new TreeSet<>(Set.of(IrisEventType.PROGRESS_STALLED.name())));

-        assertThat(updatedSettings.getIrisChatSettings().getDisabledProactiveEvents()).containsExactly("PROGRESS_STALLED");
+        assertThat(updatedSettings.getIrisChatSettings().getDisabledProactiveEvents()).containsExactly(IrisEventType.PROGRESS_STALLED.name());

Likely invalid or redundant comment.

src/main/java/de/tum/cit/aet/artemis/iris/service/session/IrisCourseChatSessionService.java (1)

118-120: 🛠️ Refactor suggestion

Consider using a marker interface instead of Object for type safety

Using Object as a parameter type loses type safety and could lead to runtime errors. This could be problematic when handling different event types (build_failed, progress_stalled, jol).

The previous suggestion to create a marker interface is still valid:

public interface IrisEventPayload {}

private void requestAndHandleResponse(IrisCourseChatSession session, String variant, IrisEventPayload payload) {
    var chatSession = (IrisCourseChatSession) irisSessionRepository.findByIdWithMessagesAndContents(session.getId());
    pyrisPipelineService.executeCourseChatPipeline(variant, chatSession, payload);
}
src/main/java/de/tum/cit/aet/artemis/iris/service/settings/IrisSettingsService.java (5)

44-44: Import Statement Added

The import of IrisEventType is necessary for handling the new event types and appears correctly added.


88-92: Parameter Renaming Enhances Readability

Renaming the parameter from event to ignoredEvent in the execute method clarifies that the parameter is not used within the method, improving code readability.


487-493: ⚠️ Potential issue

Incorrect Feature Check: Should Verify PROACTIVITY Instead of CHAT

In the method isActivatedForElseThrow(IrisEventType type, Course course), the code checks if the CHAT feature is enabled. Since this method pertains to events related to proactivity, it would be more appropriate to check if the PROACTIVITY feature is enabled.

Apply this diff to correct the feature check:

 public void isActivatedForElseThrow(IrisEventType type, Course course) {
-    isEnabledForElseThrow(IrisSubSettingsType.CHAT, course);
+    isEnabledForElseThrow(IrisSubSettingsType.PROACTIVITY, course);

     if (!isActivatedFor(type, course)) {
         throw new AccessForbiddenAlertException("The Iris " + type.name() + " event is disabled for this course.", "Iris",
             "iris." + type.name().toLowerCase() + "Disabled");
     }
 }

503-509: ⚠️ Potential issue

Incorrect Feature Check in Exercise Method: Should Verify PROACTIVITY Instead of CHAT

Similarly, in the method isActivatedForElseThrow(IrisEventType type, Exercise exercise), the code checks if the CHAT feature is enabled. It should check the PROACTIVITY feature to align with the event type.

Apply this diff to correct the feature check:

 public void isActivatedForElseThrow(IrisEventType type, Exercise exercise) {
-    isEnabledForElseThrow(IrisSubSettingsType.CHAT, exercise);
+    isEnabledForElseThrow(IrisSubSettingsType.PROACTIVITY, exercise);

     if (!isActivatedFor(type, exercise)) {
         throw new AccessForbiddenAlertException("The Iris " + type.name() + " event is disabled for this exercise.", "Iris",
             "iris." + type.name().toLowerCase() + "Disabled");
     }
 }

759-779: 🛠️ Refactor suggestion

Simplify Event Validation Logic to Reduce Code Duplication

The method isEventEnabledInSettings contains duplicate logic within the switch cases for PROGRESS_STALLED and BUILD_FAILED. To improve readability and maintainability, consider refactoring the code to eliminate duplication.

Apply this diff to refactor the method:

 private boolean isEventEnabledInSettings(IrisCombinedSettingsDTO settings, IrisEventType type) {
+    if (type == IrisEventType.PROGRESS_STALLED || type == IrisEventType.BUILD_FAILED) {
+        var disabledEvents = settings.irisChatSettings().disabledProactiveEvents();
+        return disabledEvents == null || !disabledEvents.contains(type.name().toLowerCase());
+    }
+    throw new IllegalStateException("Unexpected value: " + type); // TODO: Add JOL event, once Course Chat Settings are implemented
-    return switch (type) {
-        case PROGRESS_STALLED -> {
-            if (settings.irisChatSettings().disabledProactiveEvents() != null) {
-                yield !settings.irisChatSettings().disabledProactiveEvents().contains(IrisEventType.PROGRESS_STALLED.name().toLowerCase());
-            }
-            else {
-                yield true;
-            }
-        }
-        case BUILD_FAILED -> {
-            if (settings.irisChatSettings().disabledProactiveEvents() != null) {
-                yield !settings.irisChatSettings().disabledProactiveEvents().contains(IrisEventType.BUILD_FAILED.name().toLowerCase());
-            }
-            else {
-                yield true;
-            }
-        }
-        default -> throw new IllegalStateException("Unexpected value: " + type); // TODO: Add JOL event, once Course Chat Settings are implemented
-    };
 }

Also, there's a TODO comment in the default case regarding adding the JOL event. Would you like me to create a GitHub issue to track the implementation of the JOL event?

Likely invalid or redundant comment.

@github-actions github-actions bot added the communication Pull requests that affect the corresponding module label Dec 10, 2024
Copy link
Contributor

@sebastianloose sebastianloose left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Reapprove

Copy link

@isabellagessl isabellagessl left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Tested on Test Server 3 look good :)

@bassner bassner merged commit cae4fb5 into develop Dec 10, 2024
31 of 35 checks passed
@bassner bassner deleted the feature/iris/event-service branch December 10, 2024 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
atlas Pull requests that affect the corresponding module client Pull requests that update TypeScript code. (Added Automatically!) communication Pull requests that affect the corresponding module core Pull requests that affect the corresponding module database Pull requests that update the database. (Added Automatically!). Require a CRITICAL deployment. exercise Pull requests that affect the corresponding module iris Pull requests that affect the corresponding module programming Pull requests that affect the corresponding module ready to merge server Pull requests that update Java code. (Added Automatically!) tests too-long-open !!! This is an antipattern, we should aim for small PRs that are only open for a short time !!!
Projects
Status: Merged
Status: Done
Status: Done
Development

Successfully merging this pull request may close these issues.

10 participants