Merge pull request #1 from bderenzi/bdr/web-ui

Bdr/web UI

.gitignore

@@ -0,0 +1,2 @@
+.DS_Store
+.python-version

README.md

@@ -5,7 +5,7 @@ https://github.com/dimagi/outlier-detect-workflow
 
 A command-line tool to generate customized outlier detection reports from [CommCare HQ](https://www.commcarehq.org). A prerequisite of this tool is that the user has deployed a CommCare application and performed some data collection.
 
-Installation & Quick Start
+Installation and Quick Start
 --------------------------
 
 0\. Install [Docker](https://docs.docker.com/engine/install/).
@@ -16,13 +16,15 @@ Installation & Quick Start
 $ git clone https://github.com/dimagi/outlier-detect-workflow.git
 ```
 
-2\. Generate a Data Export Tool config file using the [CommCare HQ Form Data Export](https://confluence.dimagi.com/display/commcarepublic/Form+Data+Export). First, create a new form export. In the "Export Settings", click "Generate a Data Export Tool config file" and change the "Export Name" and "Sheet Name" to "outlier_data_export".  For the fields, you must select "username", "completed_time" and the fields being analyzed for outlier detection.
+### Run Instructions - Command Line Tool
+
+1\. Generate a Data Export Tool config file using the [CommCare HQ Form Data Export](https://confluence.dimagi.com/display/commcarepublic/Form+Data+Export). First, create a new form export. In the "Export Settings", click "Generate a Data Export Tool config file" and change the "Export Name" and "Sheet Name" to "outlier_data_export".  For the fields, you must select "username", "completed_time" and the fields being analyzed for outlier detection.
 
 Save the `outlier_data_export-DET.xlsx` file.
 
-3\. Edit the `config.yaml` file and save it. Modify the `source_form_outlier_questions` to match the "Field" columns from the CommCare Data Export Tool Excel Query File for the outlier questions only. Update the `activity_outlier_startdate` and `activity_outlier_enddate` to indicate the time period on which to run the outlier detection algorithm.
+2\. Edit the `config.yaml` file and save it. Modify the `source_form_outlier_questions` to match the "Field" columns from the CommCare Data Export Tool Excel Query File for the outlier questions only. Update the `activity_outlier_startdate` and `activity_outlier_enddate` to indicate the time period on which to run the outlier detection algorithm.
 
-4\. Run the CommCare Form Outlier Detection Workflow tool. This action will export data from your CommCare HQ project space, run the outlier detection algorithm and upload the outlier results back into CommCare HQ.
+3\. Run the CommCare Form Outlier Detection Workflow tool. This action will export data from your CommCare HQ project space, run the outlier detection algorithm and upload the outlier results back into CommCare HQ.
 
 ```
 ./run.sh <config.yaml filepath> <outlier_data_export-DET.xlsx filepath>
@@ -38,6 +40,24 @@ You can optionally pass a .env file (see ".env_template") to this command:
 
 To protect your credentials, it is best to keep the .env file outside this repo or add it to your .gitignore file.
 
+### Run Instructions - User Interface
+
+0\. Install node js and n.
+
+1\. From the web-ui directory, install needed packages: ```npm install```
+
+2.\ Build the application: ```npx build```
+
+3.\ Run the application using the following command from the web-ui directory: ```python app.py```. Open the user interface at http://127.0.0.1:5000.
+
+4.\ Configure the algorithm run. Enter your CommCare credentials and the date range for the outlier detection.
+
+5\. Generate a Data Export Tool config file using the [CommCare HQ Form Data Export](https://confluence.dimagi.com/display/commcarepublic/Form+Data+Export). First, create a new form export. In the "Export Settings", click "Generate a Data Export Tool config file" and change the "Export Name" and "Sheet Name" to "outlier_data_export".  For the fields, you must select "username", "completed_time" and the fields being analyzed for outlier detection.
+
+Save the `outlier_data_export-DET.xlsx` file. Upload the file to the user interface.
+
+6\. Run the tool.
+
 Optional: CommCare Outlier Detection Reporting
 --------------------------
 Additionally, you can visualize the outlier reseults using [CommCare HQ's reporting](https://dimagi.atlassian.net/wiki/spaces/commcarepublic/pages/2198241457/Native+Reporting+in+CommCare) feature. Instructions for generating a report for the outlier detection algorith run is given in `Outlier Detection CommCare HQ Reporting.pdf`.

commcare_data_export_outlierdetect/main.py

@@ -33,7 +33,9 @@ def restructure_outlier_output(output_dict):
             user_question_results = {'username':username,'questionid': questionid, 'score':user_outlier_results['score'],
                             'p_value':user_outlier_results['p_value'], 
                             'expected_freq':str(expected_freq),
-                            'observed_freq':str(user_outlier_results['observed_freq']), 'name':case_name, 'owner_id': OWNER_ID, 'case_name': case_name}
+                            'observed_freq':str(user_outlier_results['observed_freq']),
+                            'date_range' : f"{activity_outlier_startdate} to {activity_outlier_enddate}",
+                            'name':case_name, 'owner_id': OWNER_ID, 'case_name': case_name}
                             #'expected_freq':str(user_outlier_results['expected_freq']),
             res.append(user_question_results)
     res_df = pd.DataFrame.from_records(res)

docker-compose.yml

@@ -17,52 +17,14 @@ services:
   commcare_data_export_outlierdetect:
     container_name: commcare_data_export_outlierdetect
     build: commcare_data_export_outlierdetect/
-    environment:
-      - CC_HQ
-      - CC_USER
-      - CC_APIKEY
-      - CC_AUTH_MODE
-      - CC_PROJECT
-      - CC_PASSWORD
-      - CC_OWNERID
+    env_file:
+      - ./commcare_data_export_outlierdetect/.env
     networks:
       - outlier_network
     depends_on:
       postgres:
         condition: service_healthy
 
-  # commcare_data_export:
-  #   container_name: commcare_data_export
-  #   build: commcare_data_export/
-  #   environment:
-  #     - CC_HQ
-  #     - CC_USER
-  #     - CC_APIKEY
-  #     - CC_AUTH_MODE
-  #     - CC_PROJECT
-  #   networks:
-  #     - outlier_network
-  #   depends_on:
-  #     postgres:
-  #       condition: service_healthy
-
-  # outlierdetect:
-  #   container_name: outlierdetect
-  #   build: outlierdetect/
-  #   environment:
-  #     - CC_HQ
-  #     - CC_USER
-  #     - CC_PROJECT
-  #     - CC_PASSWORD
-  #     - CC_OWNERID
-  #   networks:
-  #     - outlier_network
-  #   depends_on:
-  #     commcare_data_export:
-  #       condition: service_completed_successfully
-  #     postgres:
-  #       condition: service_healthy
-
 networks:
   outlier_network:
     driver: bridge

package.json

@@ -0,0 +1,5 @@
+{
+  "dependencies": {
+    "react-dropzone": "^14.2.3"
+  }
+}

run.sh

@@ -6,24 +6,27 @@ if [ "$#" -lt 2 ]; then
     exit 1
 fi
 
+# Get the directory of the current script
+script_dir=$(dirname "$0")
+
 # Validate and set file paths
 config_file="$1"
 outlier_data_file="$2"
 env_file="$3"
 
-#copy files
-cp "$config_file" commcare_data_export_outlierdetect/config.yaml
-cp "$outlier_data_file" commcare_data_export_outlierdetect/outlier_data_export-DET.xlsx
+# Copy files
+cp "$config_file" "$script_dir/commcare_data_export_outlierdetect/config.yaml"
+cp "$outlier_data_file" "$script_dir/commcare_data_export_outlierdetect/outlier_data_export-DET.xlsx"
 
 # If an .env file is provided, source it to set environment variables
 if [ -n "$env_file" ]; then
     source "$env_file"
 fi
 
 # Proceed with the docker-compose command
-docker-compose up --build --abort-on-container-exit
-docker compose down --volumes
+(cd "$script_dir" && docker-compose up --build --abort-on-container-exit)
+(cd "$script_dir" && docker-compose down --volumes)
 
 # Clean up by deleting the copied files
-rm -f commcare_data_export_outlierdetect/config.yaml
-rm -f commcare_data_export_outlierdetect/outlier_data_export-DET.xlsx
+rm -f "$script_dir/commcare_data_export_outlierdetect/config.yaml"
+rm -f "$script_dir/commcare_data_export_outlierdetect/outlier_data_export-DET.xlsx"

web-ui/.eslintrc.json

@@ -0,0 +1,3 @@
+{
+  "extends": "next/core-web-vitals"
+}

web-ui/.gitignore

@@ -0,0 +1,36 @@
+# See https://help.github.com/articles/ignoring-files/ for more about ignoring files.
+
+# dependencies
+/node_modules
+/.pnp
+.pnp.js
+.yarn/install-state.gz
+
+# testing
+/coverage
+
+# next.js
+/.next/
+/out/
+
+# production
+/build
+
+# misc
+.DS_Store
+*.pem
+
+# debug
+npm-debug.log*
+yarn-debug.log*
+yarn-error.log*
+
+# local env files
+.env*.local
+
+# vercel
+.vercel
+
+# typescript
+*.tsbuildinfo
+next-env.d.ts

web-ui/README.md

@@ -0,0 +1,36 @@
+This is a [Next.js](https://nextjs.org/) project bootstrapped with [`create-next-app`](https://github.com/vercel/next.js/tree/canary/packages/create-next-app).
+
+## Getting Started
+
+First, run the development server:
+
+```bash
+npm run dev
+# or
+yarn dev
+# or
+pnpm dev
+# or
+bun dev
+```
+
+Open [http://localhost:3000](http://localhost:3000) with your browser to see the result.
+
+You can start editing the page by modifying `app/page.tsx`. The page auto-updates as you edit the file.
+
+This project uses [`next/font`](https://nextjs.org/docs/basic-features/font-optimization) to automatically optimize and load Inter, a custom Google Font.
+
+## Learn More
+
+To learn more about Next.js, take a look at the following resources:
+
+- [Next.js Documentation](https://nextjs.org/docs) - learn about Next.js features and API.
+- [Learn Next.js](https://nextjs.org/learn) - an interactive Next.js tutorial.
+
+You can check out [the Next.js GitHub repository](https://github.com/vercel/next.js/) - your feedback and contributions are welcome!
+
+## Deploy on Vercel
+
+The easiest way to deploy your Next.js app is to use the [Vercel Platform](https://vercel.com/new?utm_medium=default-template&filter=next.js&utm_source=create-next-app&utm_campaign=create-next-app-readme) from the creators of Next.js.
+
+Check out our [Next.js deployment documentation](https://nextjs.org/docs/deployment) for more details.