Model ignores specified bounds and priors

[lauraglobig]

Hi All,

I'm running a hierarchical DDM, but keep running into the issue of negative a,z,t,(and v_trial_type values that go beyond bounds if specified).

I've tried the following: 1) specifying priors; 2) explicit bounds ; 3) both. I also notice this issue for a_1|subj_idx values and found no working way to set priors for these. I might be missing something obvious but would be grateful for any help.

I am including the code that contains both prior and bounds specifications below:

`stimcoded_ddm_model = hssm.HSSM(
data=data,
model="ddm",

include=[
    {
        "name": "v",
        "formula": "v ~ 1 + trial_type + (1|subj_idx)+ (trial_type|subj_idx)",
        "prior": {
        "Intercept": {"name": "Uniform", "lower": -3, "upper":3},
        "trial_type": {"name": "Uniform", "lower": -1.0, "upper": 1.0},
    },
         "link": "identity",
    },
    {
        "name": "a",
        "formula": "a ~ 1 + (1|subj_idx)",
        "prior": {
        "Intercept": {"name": "Uniform", "lower": 0.01, "upper":3},
    },
        "link": "identity",
        "bounds": (0.0, np.inf) 
    },
     {
         "name": "t",
         "formula": "t ~ 1 + (1|subj_idx)",
         "prior": {
         "Intercept": {"name": "Uniform", "lower": 0, "upper":3, "initval":0.1},
         },
         "link": "identity",
         "bounds": (0, np.inf)
     },
    {
        "name": "z",
        "formula": "z ~ 1 + (1|subj_idx)",
        "prior": {
        "Intercept": {"name": "Uniform", "lower": 0, "upper":1},
        },
        "link": "identity",
        "bounds": (0, 1)
    },
],

)
`

[frankmj]

Hi Laura, hm this is quite strange that the parameters go negative. Did it work for you with a single subject first and give sensible results? Two other things to try for now before the developers can dig in to this: - add the following line to your model description, e.g. in the line before the include statement loglik_kind="approx_differentiable", this is because there are some known issues with the analytical likelihood in complex models that are usually resolved when using the neural net approximations, and it would be good for us to know if it works better when you use that. - try adding p_outlier = 0, to the model description : the default use of lapses may interact in complex models in ways we have not yet looked at. Michael Michael J Frank, PhD | Edgar L. Marston Professor Director, Carney Center for Computational Brain Science <https://www.brown.edu/carney/ccbs> Laboratory of Neural Computation and Cognition <https://www.lnccbrown.com/> Brown University website <http://ski.clps.brown.edu>

…

On Wed, Aug 9, 2023 at 2:11 PM Laura K. Globig ***@***.***> wrote: Hi All, I'm running a hierarchical DDM, but keep running into the issue of negative a,z,t,(and v_trial_type values that go beyond bounds if specified). I've tried the following: 1) specifying priors; 2) explicit bounds ; 3) both. I also notice this issue for a_1|subj_idx values and found no working way to set priors for these. I might be missing something obvious but would be grateful for any help. I am including the code that contains both prior and bounds specifications below: `stimcoded_ddm_model = hssm.HSSM( data=data, model="ddm", include=[ { "name": "v", "formula": "v ~ 1 + trial_type + (1|subj_idx)+ (trial_type|subj_idx)", "prior": { "Intercept": {"name": "Uniform", "lower": -3, "upper":3}, "trial_type": {"name": "Uniform", "lower": -1.0, "upper": 1.0}, }, "link": "identity", }, { "name": "a", "formula": "a ~ 1 + (1|subj_idx)", "prior": { "Intercept": {"name": "Uniform", "lower": 0.01, "upper":3}, }, "link": "identity", "bounds": (0.0, np.inf) }, { "name": "t", "formula": "t ~ 1 + (1|subj_idx)", "prior": { "Intercept": {"name": "Uniform", "lower": 0, "upper":3, "initval":0.1}, }, "link": "identity", "bounds": (0, np.inf) }, { "name": "z", "formula": "z ~ 1 + (1|subj_idx)", "prior": { "Intercept": {"name": "Uniform", "lower": 0, "upper":1}, }, "link": "identity", "bounds": (0, 1) }, ], ) ` — Reply to this email directly, view it on GitHub <#263>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7TFHAIBSUPNXX6JDPY4TXUPHGXANCNFSM6AAAAAA3KL7TQI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>

[lauraglobig]

Hi Michael,
Thanks for the suggestions:

1. p_outlier doesn't seem to make a difference.
2. I also tried loglik_kind="approx_differentiable" and so long as I specify priors and I get the group-level parameters to be positive - however individual-level values are still mostly negative. If I don't specify priors and just leave default a,t,z are negative. Oddly, specifying bounds doesn't really seem to have an effect.
3. I also just fit a simple model to an individual subject as follows:
   simple_ddm_model = hssm.HSSM(data=data)
   This works great - so am I right in thinking this is primarily a hierarchical issue? Is there a way to specify bounds/priors for the individual-level parameters that I am missing?
   Best,
   Laura

[frankmj]

Thanks Laura, its possible that you would also need to constrain the priors on the standard deviation at the group level. But really the params shouldn't go negative so the team will look at this. Not sure why bounds don't have effect. For the simple indiv subject model - I would want to see a version where you still allow for trialtype v ~ 1 + trial_type in the model just without subj_idx. also you could do that for a subset of hte data that has just one subject, or apply it to the whole dataset but it would assume there was one subject..

…

On Thu, Aug 10, 2023 at 3:02 PM Laura K. Globig ***@***.***> wrote: Hi Michael, Thanks for the suggestions: 1. p_outlier doesn't seem to make a difference. 2. I also tried loglik_kind="approx_differentiable" and so long as I specify priors and I get the group-level parameters to be positive - however individual-level values are still mostly negative. If I don't specify priors and just leave default a,t,z are negative. Oddly, specifying bounds doesn't really seem to have an effect. 3. I also just fit a simple model to an individual subject as follows: simple_ddm_model = hssm.HSSM(data=data) This works great - so am I right in thinking this is primarily a hierarchical issue? Is there a way to specify bounds/priors for the individual-level parameters that I am missing? Best, Laura — Reply to this email directly, view it on GitHub <#263 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7TFAH3JKQJZUVJNWBULLXUUV4VANCNFSM6AAAAAA3KL7TQI> . You are receiving this because you commented.Message ID: ***@***.***>

[lauraglobig]

Hi Michael,
Thank you! I've just run it with trial_type without subj_idx and again the results look exactly as they should.
For reference the hierarchical model produces negative values regardless of whether it's with or without trial_type.
I attempted to specify the std yesterday but couldn't find a way to make it work. Could you tell me how to do it? Sorry if it's really obvious. So far I have tried the following:
For a:
"sigma": {"name": "Uniform", "lower": 0, "upper":3}
"Intercept_sigma": {"name": "Uniform", "lower": 0, "upper":3}
"a_1|subj_idx ": {"name": "Uniform", "lower": 0, "upper":3}

Best
Laura

[frankmj]

Hi Laura, thanks, this and some other threads reported as issues make it clear that something is askew with the more complex hierarchical models and their bounds. The team is working on it and we'll get back to you - unfortunately two critical people are away on summer vacation (and me too next week), so it might be a couple weeks. That's why this is still an alpha version for now, but you are helping us get there :^) Michael

…

On Thu, Aug 10, 2023 at 3:35 PM Laura K. Globig ***@***.***> wrote: Hi Michael, Thank you! I've just run it with trial_type without subj_idx and again the results look exactly as they should. For reference the hierarchical model produces negative values regardless of whether it's with or without trial_type. I attempted to specify the std yesterday but couldn't find a way to make it work. Could you tell me how to do it? Sorry if it's really obvious. So far I have tried the following: For a: "sigma": {"name": "Uniform", "lower": 0, "upper":3} "Intercept_sigma": {"name": "Uniform", "lower": 0, "upper":3} "a_1|subj_idx ": {"name": "Uniform", "lower": 0, "upper":3} Best Laura — Reply to this email directly, view it on GitHub <#263 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7TFBDY6TNHC6RSYA5U63XUUZYPANCNFSM6AAAAAA3KL7TQI> . You are receiving this because you commented.Message ID: ***@***.***>

[lauraglobig]

Hi Michael
Thanks for the information. Enjoy your vacation! I look forward to the updates!
Best,
Laura

[AlexanderFengler]

Hi Laura,

thank you for bringing this to our attention, we are back from vacation and looking into this now.
The culprit is most likely the way in which variables are initialized at the beginning of sampling.

Best,
Alex

[lauraglobig]

Thank you!

[AlexanderFengler]

initial value settings should be fixed, this will be part of the next release, currently sitting in main branch (you can use pip install git+https://github.com/lnccbrown/HSSM@main)

[XiaoyuZeng]

Hi Laura,
I also encounter a problem related to prior setting.
In my model, multiple parameters are regression targets, but I don't know how to set priors when I don't have any.
you mentioned you tried just ignoring the priors.
did that work for you?

Best,
Xiaoyu

[AlexanderFengler]

Is your problem related to running into illegal parameter values as well?

[XiaoyuZeng]

many thanks for your reply! no, my problem relate to totally ignoring the priors. then I decided to apply the 'safe' priors provided in the latest hssm.

Best,
Xiaoyu