Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

report on repo #56

Merged
merged 13 commits into from
Apr 17, 2024
Merged

report on repo #56

Show file tree
Hide file tree
Changes from 5 commits
Commits
Show all changes
13 commits
Select commit Hold shift + click to select a range
8bd36ed
report on repo
abhicodes369 Mar 8, 2024
6d210a0
created wrapper column
abhicodes369 Mar 10, 2024
58a423e
Delete local.env.example
abhicodes369 Mar 10, 2024
6c1b907
Delete .gitignore
abhicodes369 Mar 10, 2024
321fa55
Adding recovered .gitignore file
abhicodes369 Mar 10, 2024
cf05312
added a new column in the csv file . please tell me if there are any …
abhicodes369 Mar 15, 2024
baf5d4a
added a new column in the csv file please tell me if there are any mo…
abhicodes369 Mar 15, 2024
d7df0ca
added a new column to csv and modified python file
abhicodes369 Mar 15, 2024
992147d
Refactor column names in repostatus.py and combined_report.html
abhicodes369 Apr 11, 2024
6ab39b5
Add zenodo-get, sphinx, and sphinx_rtd_theme to requirements.txt
abhicodes369 Apr 16, 2024
06167e7
Merge branch 'main' into created_repostatus/statistics
abhicodes369 Apr 16, 2024
4a148b8
wrap the html into a function
abhicodes369 Apr 16, 2024
e17a3e7
Merge branch 'created_repostatus/statistics' of https://github.com/ab…
abhicodes369 Apr 16, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion .gitignore
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -15,4 +15,4 @@ bvals.txt
.cache
nosetests.xml
coverage.xml
*.pyc
*.pyc
3 changes: 2 additions & 1 deletion requirements.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,4 +9,5 @@ matplotlib
scienceplots
cvxpy
pytest
tqdm
tqdm
pandas
64 changes: 64 additions & 0 deletions utilities/repostatus.py
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,64 @@
import os
abhicodes369 marked this conversation as resolved.
Show resolved Hide resolved
import pandas as pd
import json

# directory of the current script
script_dir = os.path.dirname(os.path.abspath(__file__))
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think by the convention you're been following, this should be SCRIPT_DIR.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be problematic to add a column called 'algorithms' to the data frame? it would be easily understandable. Is it a bad idea?

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What would that add? The algorithms from the json are the ones running the automated testing. That's beyond the scope of the csv file. Is there some other algorithm you want to add?


# path to the repository
REPO_DIR = os.path.dirname(script_dir)

CODE_CONTRIBUTIONS_FILE = os.path.join(REPO_DIR, "doc", "code_contributions_record.csv")
ALGORITHMS_FILE = os.path.join(REPO_DIR, "tests", "IVIMmodels", "unit_tests", "algorithms.json")
SOURCE_FOLDER = os.path.join(REPO_DIR, "src", "original")
WRAPPED_FOLDER = os.path.join(REPO_DIR, "src", "standardized")

# Read the CSV file
df = pd.read_csv(CODE_CONTRIBUTIONS_FILE)

unique_subfolders = df['subfolder'].unique().tolist()

# Read the JSON file
with open(ALGORITHMS_FILE, 'r') as f:
algorithms_data = json.load(f)

# list of all algorithms from the JSON file
all_algorithms = algorithms_data['algorithms']

# Check if both code_contributions_file matches with source folder
for subfolder in unique_subfolders:
subfolder_path = os.path.join(SOURCE_FOLDER, subfolder)
if not os.path.exists(subfolder_path):
print(f"Warning: Subfolder '{subfolder}' does not exist in the source folder.")

# Add column 'Tested' to the DataFrame if it starts with that of subfolder
df['Tested'] = df.apply(lambda row: 'Yes' if any(algorithm.startswith(row['subfolder'].split('_')[0]) for algorithm in all_algorithms) else 'No', axis=1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm just looking at this logic and it seems like it's just matching everything before the first _. That's not specific because that's usually the author's initials and they can have several algorithms.

Looking at it a little more, this could become fancier, but maybe we actually want this column in our csv file. So another approach would be to just manually go through and find the matches and add a "Wrapped" column to the csv file that contains what you'd see in the algorithms.json.

Then that would reduce what is needed in here to just having a sanity check that the wrapped folders match the csv. And it would make checking for tested much easier because it would be a simple string match.

How does that sound?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do i need to manually update the code_contribution_csv file or write a script to read algorithms from algorithms.json and write 'wrapped' in csv file and automatically update whenever there is change in algorithms. json file

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add the column manually. That column is not the same as what's in algorithm.json. That column would be the wrapped name, so if an algorithm is wrapped it should have an entry there. The algorithm.json is what is being tested. In theory everything wrapped should be tested but it does need to be added and confirmed it doesn't fail. So they're not the same thing.


# Parse files in the WRAPPED_FOLDER
wrapped_algorithms = []
for root, dirs, files in os.walk(WRAPPED_FOLDER):
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This could just be added to the csv, see comment above.

for file in files:
if file.endswith('.py'):
file_path = os.path.join(root, file)
with open(file_path, 'r') as f:
content = f.read()
for algorithm in all_algorithms:
if algorithm in content:
wrapped_algorithms.append(algorithm)

# Add a column 'Wrapped' to the DataFrame
df['Wrapped'] = df.apply(lambda row: 'Yes' if any(algorithm.startswith(row['subfolder'].split('_')[0]) for algorithm in wrapped_algorithms) else 'No', axis=1)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This as well, see above.


# Select the desired columns
df_selected = df[['Technique', 'subfolder', 'Authors', 'Tested', 'Wrapped']]
df_selected.columns = ['Technique', 'Subfolder', 'Contributors', 'Tested', 'Wrapped']

# Convert the DataFrame to HTML
html_string = df_selected.to_html(index=False)

# Save the HTML to a file
with open(os.path.join(REPO_DIR, 'combined_report.html'), 'w') as f:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think os.path.join(REPO_DIR, website, 'combined_report.html')

f.write(html_string)

# Printing message that report has been successfully generated
print("Combined HTML report generated successfully.")
146 changes: 146 additions & 0 deletions website/combined_report.html
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,146 @@
<table border="1" class="dataframe">
<thead>
<tr style="text-align: right;">
<th>Technique</th>
<th>Subfolder</th>
<th>Contributors</th>
<th>Tested</th>
abhicodes369 marked this conversation as resolved.
Show resolved Hide resolved
<th>Wrapped</th>
</tr>
</thead>
<tbody>
<tr>
<td>IVIM</td>
<td>OGC_AmsterdamUMC</td>
<td>Oliver Gurney-Champion</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>OGC_AmsterdamUMC</td>
<td>Oliver Gurney-Champion</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>Tri-exponential</td>
<td>OGC_AmsterdamUMC</td>
<td>Oliver Gurney-Champion</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>Tri-exponential</td>
<td>OGC_AmsterdamUMC</td>
<td>Oliver Gurney-Champion</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>OGC_AmsterdamUMC</td>
<td>Oliver Gurney-Champion/Sebastiano Barbieri</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>PvH_KB_NKI</td>
<td>Petra van Houdt/Stefan Zijlema/Koen Baas</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>PV_MUMC</td>
<td>Paulien Voorter</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Farooq et al. Modified by Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Fadnavis et al. Modified by Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Modified by Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>IAR_LundUniversity</td>
<td>Modified by Ivan A. Rashid</td>
<td>Yes</td>
<td>Yes</td>
</tr>
<tr>
<td>IVIM</td>
<td>OJ_GU</td>
<td>Oscar Jalnefjord</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td>IVIM</td>
<td>OJ_GU</td>
<td>Oscar Jalnefjord</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td>IVIM</td>
<td>OJ_GU</td>
<td>Oscar Jalnefjord</td>
<td>No</td>
<td>No</td>
</tr>
<tr>
<td>IVIM</td>
<td>ETP_SRI</td>
<td>Eric Peterson</td>
<td>Yes</td>
<td>Yes</td>
</tr>
</tbody>
</table>