Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add functions to spatially visualise individual and aggregate criteria for location selection #650

Closed
wants to merge 39 commits into from

Conversation

Rosejoycrocker
Copy link
Collaborator

Closes #620

Adds viz.decision_matrices to visualise specified criteria and aggregate criteria spatially from a decision matrix, aggregate score vector and result set.

Adds analysis.decision_matrices which calculates decision matrices for seeding and fogging from a result set and outputs these as NamedDimsArrays labelled using the criteria weightings names.

Some changes to criteria weights names have also been added for consistency and ease of labelling the decision matrices as NamedDimsArray. These changes will make addressing #624 easier.

The weighting for ingoing and outgoing connectivity for fogging has been split into two separate weightings fog_in_connectivity and fog_out_connectivity for consistency with seeding. The weights coral_cover_low and coral_cover_high have been renamed seed_coral_cover_low and fog_coral_cover_high for consistency with other weights names. The weights ordering in CriteriaWeights has been reordered so that criteria_params extracts the weights names in the correct order to label the decision matrices.

@Zapiano
Copy link
Collaborator

Zapiano commented Jan 16, 2024

@Rosejoycrocker when I save the file ext\AvizExt\viz\location_selection.jl, Julia Formatter changes a lot of things. Is your Julia Formatter working?

@Rosejoycrocker
Copy link
Collaborator Author

@Rosejoycrocker when I save the file ext\AvizExt\viz\location_selection.jl, Julia Formatter changes a lot of things. Is your Julia Formatter working?

Hi @Zapiano, I recently changed laptops and had to set up my formatter again. I thought I finally had it working but obviously not, I'll look into it again.

Copy link
Collaborator

@Zapiano Zapiano left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did an initial review but it's too hard to review without the autoformatted fixed....

# Arguments
- `g` : Figure GridPosition or GridLayout
- `rs` : Result set
- `S` : A normalised decision matrix calculated using decison.decision_matrices
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is this name S being used in other places as well? Because it is a very opaque name. Why not call it decision_matrix?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

SE and SH are used for the seeding and shading matrices respectively, but I'll change it to decision_matrix for clarity

axis_opts::Dict = Dict(),
)
if length(rs.site_data.site_id) != size(S, 1)
error("Only unfiltered decision matrices can be plotted.")
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Maybe it's just because I don't have context, but what do you mean by "unfiltered decision matrices" ?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The decision matrices can be filtered based on thresholds for different criteria (like heat stress and wave stress) but this means they have less sites than the whole site set, which means when you go to plot them the geodata (created from the result set) doesn't match the number of sites when plotting using map. Alternatively, filtered sites could be set as 0.0 or something, but they'd have to be added back into the decision matrix because these are automatically filtered in create_decision_matrix.

opts[:colorbar_limits] = get(opts, :colorbar_limits, (0.0, 1.0))

n_criteria::Int64 = length(criteria)
n_rows = n_cols = ceil(Int64, sqrt(n_criteria + 1))
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think you should consider using _calc_gridsize instead. Not only it is the standard BUT is has a max_cols param to prevent each map from being too small in case we have too many criteria. Since you're plotting all criteria plus one aggregate, you can call _calc_gridsize(n_criteria + 1) and you should be fine. (:

s_col = 1

for row in 1:n_rows, col in 1:n_cols
if step > n_criteria
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use _calc_gridsize you wont need this if anymore.

axis_opts = axis_opts_temp
)

step += 1
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If you use _calc_gridsize you won't need this step anymore.

rs.dhw_stats[RCP](; stat = "mean"), (:dhw_scenario)
)

TP_data = rs.connectivity_data[RCP] # connectivity matrix
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if this is the connectivity matrix, what does TP stand for?

rs.dhw_stats[RCP](; stat = "mean"), (:dhw_scenario)
)

TP_data = rs.connectivity_data[RCP] # connectivity matrix
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If I understand correctly, you're only using this TP_data multiplied by site_k_area, so you could do that here instead and not when calling the function. It would increase readability.

Copy link
Collaborator Author

@Rosejoycrocker Rosejoycrocker Jan 17, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

TP_base is what the connectivity matrix is called in the connectivity calcs, the TP stands for transfer probability :

- `TP_base` : Base transfer probability matrix to create Directed Graph from.
- `area_weighted_TP` : Transfer probability matrix weighted by location `k` area
- `cover` : Total relative coral cover at location
- `TP_cache` : Cache matrix of same size as TP_base to hold intermediate values
# Returns

but you're right it's not a very transparent name. I'll change it to conn_data which is also used in other places in ADRIA.


TP_data = rs.connectivity_data[RCP] # connectivity matrix
connectivity_data = connectivity_strength(
TP_data .* site_k_area(rs), vec(loc_coral_cover), TP_data
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

isn't loc_coral_cover a vector already? why do you need this vec here?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good point, not sure why this is here


# Criteria for strongest larval sources to priority locations
priority_source_criteria = ADRIA.decision.priority_predecessor_criteria(
strong_pred, vec(rs.sim_constants["priority_sites"]), length(site_ids)
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

since you are using vec(rs.sim_constants["priority_sites"]) in more than one place, you could extract it to a variable.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only used once, the other usages is vec(rs.sim_constants["priority_zones"])


return (in_conn=C1, out_conn=C2, strongest_predecessor=strong_pred)
function connectivity_strength(
TP_base::AbstractMatrix{<:Union{Float32, Float64}}
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
TP_base::AbstractMatrix{<:Union{Float32, Float64}}
TP_base::AbstractMatrix{<:Real}

Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I recommend keeping the union: allows the compiler to specialise for just two types (Float32 and Float64) compared to having to support all possible Reals

@Rosejoycrocker
Copy link
Collaborator Author

@Rosejoycrocker when I save the file ext\AvizExt\viz\location_selection.jl, Julia Formatter changes a lot of things. Is your Julia Formatter working?

Hi @Zapiano, I recently changed laptops and had to set up my formatter again. I thought I finally had it working but obviously not, I'll look into it again.

Hi @Zapiano, I think I have the formatter working now (ended up doing a full uninstall and reinstall of VS code), but let me know if some things look odd. I've addressed your comments and also changed some things suggested by @ConnectedSystems, such as making viz.decision_matrices a method of viz.map .

@Rosejoycrocker Rosejoycrocker force-pushed the mcda-visualisations branch 3 times, most recently from ef520b8 to c0254be Compare January 24, 2024 00:49
Add docs for `store_conn` and add detail to docs for `store_env_summary`.
…culation into separate functions to avoid repetition
…rid of spatial heatmaps

Label dimensions with full site id to avoid errors with `viz.map`
…ded figure object

Also fix how `axis_opts` is defined to avoid overwriting inputs

Add documentation for `decision_matrices`
Change `decision.decision_matrices` to give both intervention matrices as output

Add in plotting of scores in `viz.decision_matrices()` and change to square plotting grid
Add title to aggregate score plot
…_sites` in criteria constuctors

( as these are their types in the result set)
+ formatting
Add decison matrix creation from resultset to analysis as new `decision.jl" file

Remove `decision_matrices` from "location_selection.jl"
…quired

Due to changes in how locations are deemed as having sufficient space
… fogging

This allows ease of extraction for decision matrix labelling
Plus, update criteria weights docs

Add use of separate ingoing and outgoing connectivity weights for fogging to mcda
…at no filtering occurs when plotting seed decision matrix

Update example and test scenarios with new parameters and parameter names
…usted

So all plots in `viz.decision_matrices` have the same colorbar range. This requires removing `color_range`which was an equivalent variable, previously set to `(0.0,maximum(data))`

Add use of colorbar_limits ro `viz.decision_matrices`
Remove `s_row_`s_col` statement as no longer necessary

Remove unnecessary catch for the case when one figure is plotted (the least that can be plotted is 2)
+ fix missing variables in doc string
Revert "Remove unnecessary `vec()`"

This reverts commit 591daf3.

Rename `TP_data` `conn_data` and remove unnecessary `vec()`
…ction and `mcda_func` for scores calculation

Change output to Dict of decision matrices and scores for each intervention
…functions as no longer needed inside decision module
Move `decision_matrices` to `decision` module

Move `decision_matrices` into decision module

Move around file compilation order to allow this in `ADRIA.jl`
…readable criteria names for labels

Plus add criteria names as optional inputs so less than total set of criteria can be plotted
@Rosejoycrocker Rosejoycrocker force-pushed the mcda-visualisations branch 2 times, most recently from 3c593a6 to 3f3a10f Compare January 24, 2024 01:52
@ConnectedSystems
Copy link
Collaborator

Bump! Where did we end up on this @Zapiano ?

@Zapiano
Copy link
Collaborator

Zapiano commented Sep 10, 2024

@Rosejoycrocker this needs a rebase before I review again. I'm so sorry it took me so long...

I strongly suggest you don't simply rebase it though, because

  1. the example csv was changed so you better generate a new one.
  2. there seem to be a lot of formatting changes and recently Arlo updated the formatting for the entire codebase
  3. you have almost 40 commits but what you actually want to change seems to be way less than what this is actually changing.

I think it might be more productive and avoid mistakes to just create a tmp-mcda-visualisations branch from main, manually bring the relevant changes to that and reset this branch to the tmp one. 😢 (I know it is annoying but I honestly think this will be faster and easier to do and that's what I would do probably).

Let me know if you have any questions (or if you hate me now)

@Rosejoycrocker
Copy link
Collaborator Author

@Rosejoycrocker this needs a rebase before I review again. I'm so sorry it took me so long...

I strongly suggest you don't simply rebase it though, because

  1. the example csv was changed so you better generate a new one.
  2. there seem to be a lot of formatting changes and recently Arlo updated the formatting for the entire codebase
  3. you have almost 40 commits but what you actually want to change seems to be way less than what this is actually changing.

I think it might be more productive and avoid mistakes to just create a tmp-mcda-visualisations branch from main, manually bring the relevant changes to that and reset this branch to the tmp one. 😢 (I know it is annoying but I honestly think this will be faster and easier to do and that's what I would do probably).

Let me know if you have any questions (or if you hate me now)

@Zapiano honestly I forgot about this because it's been 8 months since I responded to your review and naturally much has changed in Main since then. I agree the best approach is probably to start a new branch although I won't be able to give time to this until October. Of course I don't hate you!

@Rosejoycrocker
Copy link
Collaborator Author

#893 now addresses this

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
3 participants