issue with constraint_logpdf(auxdata, pars)

[IzaV10]

Hi,

I have some problem using constraint_logpdf(auxdata, pars). I setup the example like this:

inits = inModel_SM.config.suggested_init() inits[ inModel_SM.config.poi_index ] = 1.0 exp_data_model=inModel_SM.expected_data( inits ) auxdata_sm=inModel_SM.expected_auxdata( inits ) inModel_SM.logpdf(inits, exp_data_model)) inModel_SM.constraint_logpdf(auxdata=auxdata_sm,pars=inits) 

While inModel_SM.logpdf(inits, exp_data_model)) works fine I get the following error when calling constraint_logpdf(auxdata, pars)

--------------------------------------------------------------------------- TypeError Traceback (most recent call last) <ipython-input-45-af486baff749> in <module> ----> 1 inModel_SM.constraint_logpdf(auxdata=auxdata_sm,pars=inits) /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/pyhf/pdf.py in constraint_logpdf(self, auxdata, pars) 667 668 """ --> 669 return self.make_pdf(pars)[1].log_prob(auxdata) 670 671 def mainlogpdf(self, maindata, pars): /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/pyhf/pdf.py in make_pdf(self, pars) 700 if mainpdf: 701 pdfobjs.append(mainpdf) --> 702 constraintpdf = self.constraint_model.make_pdf(pars) 703 if constraintpdf: 704 pdfobjs.append(constraintpdf) /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/pyhf/pdf.py in make_pdf(self, pars) 376 pdfobjs = [] 377 --> 378 gaussian_pdf = self.constraints_gaussian.make_pdf(pars) 379 if gaussian_pdf: 380 pdfobjs.append(gaussian_pdf) /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/pyhf/constraints.py in make_pdf(self, pars) 105 flat_pars = tensorlib.reshape(pars, (-1,)) 106 --> 107 normal_means = tensorlib.gather(flat_pars, self.access_field) 108 109 # pdf pars are done, now get data and compute /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/pyhf/tensor/jax_backend.py in gather(self, tensor, indices) 190 191 def gather(self, tensor, indices): --> 192 return tensor[indices] 193 194 def boolean_mask(self, tensor, mask): /cvmfs/sft.cern.ch/lcg/views/LCG_100/x86_64-centos7-gcc8-opt/lib/python3.8/site-packages/jax/interpreters/xla.py in _forward_method(attrname, self, fun, *args) 1008 1009 def _forward_method(attrname, self, fun, *args): -> 1010 return fun(getattr(self, attrname), *args) 1011 _forward_to_value = partial(_forward_method, "_value") 1012 TypeError: only integer scalar arrays can be converted to a scalar index

Thanks for your help!

Cheers,
Iza

[matthewfeickert]

@IzaV10 Sorry I missed this — just seeing it now. The problem is that pyhf.pdf.Model.constraint_logpdf requires tensor objects and your inits is a list. I understand your confusion as to why logpdf works but constraint_logpdf doesn't and that's our fault for making sure that some parts of the API will work even with non-tensors when we should strictly enforce tensors everywhere.

The good news is that this is very simple to fix: just wrap your inits in pyhf.tensorlib.astensor.

As a full example of this, if we take the "Hello World" example from the README and modify it a bit

# discussion_1608.py  import pyhf pyhf.set_backend("jax") model = pyhf.simplemodels.uncorrelated_background( signal=[12.0, 11.0], bkg=[50.0, 52.0], bkg_uncertainty=[3.0, 7.0] ) data = [51, 48] + model.config.auxdata init_params = model.config.suggested_init() init_params[model.config.poi_index] = 1.0 init_params = pyhf.tensorlib.astensor(init_params) expected_data = model.expected_data(init_params) expected_auxdata = model.expected_auxdata(init_params) print(f"model.logpdf: {model.logpdf(init_params, expected_data)}") print( f"model.constraint_logpdf: {model.constraint_logpdf(expected_auxdata, pars=init_params)}" )

we get

$ python discussion_1608.py  model.logpdf: [-12.63410988] model.constraint_logpdf: -6.658431444518868

The above is done with pyhf v0.6.3, and from your output I can see that you're using the version of pyhf from what looks like LCV view LCG_100 on CVMFS which we don't recommend as it is an old version of pyhf (we don't do the distribution of pyhf of CVMFS)

[feickert@login ~]$ lsetup "views LCG_100 x86_64-centos7-gcc8-opt" ************************************************************************ Requested: views ...   Setting up views LCG_100:x86_64-centos7-gcc8-opt ...  >>>>>>>>>>>>>>>>>>>>>>>>> Information for user <<<<<<<<<<<<<<<<<<<<<<<<< ************************************************************************ [feickert@login ~]$ pyhf --version pyhf, version 0.5.4

Though in this case it doesn't matter as the API for pyhf.pdf.Model.constraint_logpdf in v0.5.4 is the same. 👍

Hope that helps.

[kratsg]

@IzaV10 if this fixes it, please mark it as the answer for your discussion. As Matthew pointed out, we have not perfected our API everywhere so there are rough edges that could be smoothed out.