How to do "Significance Testing" on the posterior distribution in HSSM?

[JoeSu112]

Hi all,
I'm now using HSSM to fit my experiment data into hierarchical drift-diffusion model (HDDM).
Here is my model:

HDDM = hssm.HSSM(
    data=data,
    model="ddm",
    loglik_kind="approx_differentiable",
    include=[
        {
            "name": "v",
            "prior": {
                "Intercept": {"name": "Normal", "mu": 0, "sigma": 3},
                "month_delayed": {"name": "Normal", "mu": 0, "sigma": 3},
                "pay_diff": {"name": "Normal", "mu": 0, "sigma": 3},
                "time_advantage": {"name": "Normal", "mu": 0, "sigma": 3},
            },
            "formula": "v ~ 1 + (month_delayed|treatment_will) + (pay_diff|treatment_will) + (time_advantage|treatment_will)",
            "link": "identity",
        },
      {
            "name": "a",
            "formula": "a ~ 0 + (1|subject) + (1|treatment_will)",
            "link": "log", #identity
        },
        {
            "name": "z",
            "formula": "z ~ 0 + (1|subject) + (1|treatment_will)",
            "link": "logit", #identity
        },
    ],
)

After sampling, I can obtain the posterior distribution of each parameter with az.plot_posterior().
Additionally, I want to know whether the distribution is different between control group and treatment group (e.g. pay_diff|treatment_will[0] v.s. pay_diff|treatment_will[1]). I know that in the HDDM package, there is a method to test:

print("P(v_pay_diff_1  > v_pay_diff_0) = ", (v_pay_diff_1.trace() > v_pay_diff_0.trace()).mean())

However, in HSSM, I can't find the same method to test the difference. I am wondering if there is any function in HSSM or other methods that can help me to test. Thanks for any help!!!!

[JoeSu112]

Hi all,
I think you might not notice this problem. Could someone give me some advices or solutions?
Thanks for any help!!!

[JamesWeiChen]

Dear Michael @frankmj ,

I have the same problem. Any help would be great!
Thanks~

Best,

James

[frankmj]

Hi both, sorry for the delay on this one. We will add tutorials on significance testing and other things soon. But for HSSM uses Arviz for accessing and plotting posteriors, and so anything you can do with Arviz you can do with HSSM. Our main tutorial shows how you can access the HDI for any parameter which gives you credible intervals. But if you are interested in comparing the traces for two posteriors, and other related examples, see below for how to do so in Arviz (especially the section on Group Comparison which has some code for plotting differences between parameters). https://towardsdatascience.com/hands-on-bayesian-statistics-with-python-pymc3-arviz-499db9a59501

…

On Tue, Nov 21, 2023 at 9:54 AM James ***@***.***> wrote: Dear all, I have the same problem. Any help would be great! Thanks~ Best, James — Reply to this email directly, view it on GitHub <#314 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAG7TFB2FWELSU57K32QJ5DYFS6DBAVCNFSM6AAAAAA67UP7JCVHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM3TMMZRGUZTI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>