content_marked_missing incorrect about missing status

[gcyoung1]

ncanda@pipeline-back[pipeline_back_1]:/fs/ncanda-share/log/make_all_inventories/reports/7y_visit_arm_1/content_marked_missing$ cat limesurvey_ssaga_youth.csv 
study_id,redcap_event_name,dag,non_nan_count,exclude,missing,complete,form_name,status
A-00082-F-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
A-00082-F-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
C-70183-F-1,7y_visit_arm_1,duke,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00012-F-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00014-F-2,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00014-F-2,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00064-F-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00068-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00081-F-0,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00090-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00093-M-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00093-M-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00096-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00109-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00115-M-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00116-M-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00139-F-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00143-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00156-F-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00158-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00159-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00159-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00165-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00196-M-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00197-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00197-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00215-F-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00223-F-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00232-M-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00233-M-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00236-F-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00274-M-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00278-M-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00280-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00280-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00296-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00299-M-1,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00332-M-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00333-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00336-M-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00336-M-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00338-F-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00339-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00344-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00358-M-5,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00361-M-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00361-M-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00366-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00387-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00392-F-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00403-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00403-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00408-F-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00413-M-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00427-M-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00443-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00443-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00447-F-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00466-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00466-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00490-F-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00510-F-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00520-F-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00523-F-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00523-F-3,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00526-M-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00541-M-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00541-M-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00547-M-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00548-M-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00560-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00563-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00570-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00570-F-4,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00579-M-0,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00593-M-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00638-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00638-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00639-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00794-M-0,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00816-M-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00816-M-2,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00842-F-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00859-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00859-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00880-M-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00923-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00925-M-6,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00937-F-8,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00966-M-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00982-M-5,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-00994-F-1,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01009-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01062-F-9,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01080-F-2,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01106-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01210-F-7,7y_visit_arm_1,ucsd,5,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01212-F-2,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT
E-01212-F-2,7y_visit_arm_1,ucsd,4,0,1,2,limesurvey_ssaga_youth,PRESENT

However, when I go to e.g. A-00082-F-3's limesurvey_ssaga_youth, it's not marked missing.

[gcyoung1]

Also the case (though possibly unrelated) for the following:

ncanda@pipeline-back[pipeline_back_1]:/fs/ncanda-share/log/make_all_inventories/report_by_site$ for site in `ls`;do echo $site;for event in 7y_visit_arm_1;do echo $event;for form in content_marked_missing; do echo $form;cat $site/$event/${form}.csv;done;done;done
duke
7y_visit_arm_1
content_marked_missing
study_id,redcap_event_name,dag,non_nan_count,exclude,missing,complete,form_name,status
C-70183-F-1,7y_visit_arm_1,duke,5,0,1,2,limesurvey_ssaga_youth,PRESENT

From the response from Gloria about this form:

Hi Griffin,
I'm a little confused about this case. Mattie once told me that we should mark parts 2-4 as skipped per protocol when the participant only has an abbreviated SSAGA. I see in your sheet that it's the SSAGA, but which part of the SSAGA is it saying that we marked as missing when we have data for it?

Then from Simon:

I think we discovered this was a bug in our inventory script, and no action is required.

[gcyoung1]

This is happening on limesurvey ssagas which have multiple parts of which only some are missing. Time to look at make_all_inventories to see how the global missingness is being set.
The missingness of a form is set to true if any subform is marked missing. This obviously doesn't comport with 'content_marked_missing', since the content could be on a different subform.
Can we split it into subforms? Or maybe we just want to mark a form not missing if any form is not missing?

If we were to change the missingness/excludedness categories to be the min of the subforms instead of the max of the subforms, then:
empty_marked_present would begin applying to forms which have at least one subforms marked present but that have no data on them.. It would also start applying to forms which have only one form marked excluded. This could potentially start raising some false positive e.g. if sites only mark one form missing or excluded, but I don't think it's an issue because ideally sites will do this for all subforms without data.
content_marked_missing would apply to less forms. Specifically, forms which have at least one present/not excluded subform will be considered present/not excluded and thus will not be included in this report. This will create false negatives in which a subform has data on it but it's marked missing. For this to be an issue, the content would have to exist on a subform which had been marked missing/not excluded. Is that even possible? I don't think so, since the subform collapses when you mark missingness. Unsure for excludedness.
less_content_than_max not affected
empty_unmarked only checks for null missingness, so we only have to worry about excludedness. This will apply to more forms (since we're filtering out excluded forms)--specifically, those which have at least one subform marked not excluded. Could genuinely increase false positives if sites don't mark every form excluded on an empty form (but are those really false positives?). Should be fine.
content_unmarked only checks nullness, unaffected.
content_not_complete doesn't check excludedness or missinges, unaffected
missing_not_complete applies to fewer forms now. Specifically, forms which have at least one present subform will now not be included in this report. Completeness on forms with multiuple subforms is currently set to the completeness status of the last subform. So forms whose last subform is not complete but which have at least one present subform will now not be flagged. This will genuinely introduce false negatives. Is it a problematic amount? I'm unsure.
excluded_with_content will flag fewer forms. Specifically, forms which have at least one non-excluded form will no longer be included in this report, even if they have content on them. Does this report ever even arise legitimately? Is it possible to have content on an excluded subform? It is possible on a whole form.

@kipohl is it possible for excluded subforms to have data on them? If not I think this proposal is fine.

[kipohl]

@gcyoung1 sorry too much text - can you give me a link to a specific record in redcap and how you want to have it behave - what specific record is Gloria confused about ?

[gcyoung1]

@kipohl this is the record Gloria is talking about, which is being flagged in content_marked_missing because, although the content it has is on subform 1 (which is not marked missing), there are other subforms which are marked missing. The report logic considers a form missing if any subform is marked missing.
A similar thing is happening for the cases in the first post in this thread.
My proposed fix is the have the reports consider forms missing only if every subform is marked missing. I broke down the implications of this fix for each subform above. The only important one is this one:
missing_not_complete applies to fewer forms now. Specifically, forms which have at least one present subform will now not be included in this report. Completeness on forms with multiuple subforms is currently set to the completeness status of the last subform. So forms whose last subform is not complete but which have at least one present subform will now not be flagged. This will genuinely introduce false negatives. Is it a problematic amount? I'm unsure

[kipohl]

@gcyoung1 lets talk about it at 2pm - I do not see the problem / I do not understand what the variable "global missing" is referring to

[kipohl]

approach makes sense - just list those where all subforms are missing