[Userinfo endpoint] Exception if token was released with unspecified/unknow scope

[peppelinux]

A better approach that patches b76f28f

Error

ERROR:oidc_op.views:argument of type 'NoneType' is not iterable
Traceback (most recent call last):
  File "/home/wert/DEV3/OIDC-Project/django-oidc-op/example/oidc_op/views.py", line 175, in service_endpoint
    args = endpoint.process_request(req_args, **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/oidc/userinfo.py", line 124, in process_request
    info = collect_user_info(self.endpoint_context, session)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/userinfo.py", line 173, in collect_user_info
    if "sub" in userinfo_claims:
TypeError: argument of type 'NoneType' is not iterable
[2020-09-02 13:32:25,006] Internal Server Error: /userinfo [(ERROR) django.request.log_response:228]
ERROR:django.request:Internal Server Error: /userinfo

How to reproduce

Configure a RP in OP with some custom/unknow scope in allowed_scopes.

Result after this PR

A specific Exception will be raised with a eloquent error message.

[2020-09-02 14:19:59,043] authn_info: {'token': 'eyJhbGciOiJFUzI1NiIsImtpZCI6IlQwZGZTM1ZVYUcxS1ZubG9VVTQwUXpJMlMyMHpjSHBRYlMxdGIzZ3hZVWhCYzNGaFZWTlpTbWhMTUEifQ.eyJzaWQiOiAiYzk4MDQ2MmUwMjU0YzI2MDY4YjI4NGI3ZGM5YWIxMTk3NzdlM2JjZWI0NDM1ZWRiZDdjYjA4YTQiLCAidHR5cGUiOiAiVCIsICJzdWIiOiAiMDc2ZWNjYTk0ZmU0NTQ2N2I0NDM1ZDhlZWFkMjE4OGFkMzc3MWUxMGZmNjcyY2UxOTMwYzA0YWE4NjI0MTgxYyIsICJpc3MiOiAiaHR0cHM6Ly8xMjcuMC4wLjE6ODAwMCIsICJpYXQiOiAxNTk5MDU2Mzk4LCAiZXhwIjogMTU5OTA1OTk5OCwgImF1ZCI6IFsiMVVVbDZjd05pZ21qIiwgImh0dHBzOi8vMTI3LjAuMC4xOjgwMDAiXX0.Mlq3p7mmefGF_IR5XETdnn51HzDzX-Ln6vJLgYfUBMcX98ckB8UPQX4xY90dKRRkxjtksdQlCloLQc93ZB9fNQ', 'method': 'bearer_header', 'client_id': '1UUl6cwNigmj'} [(DEBUG) oidcendpoint.endpoint.client_authentication:299]
[2020-09-02 14:19:59,172] Client 1UUl6cwNigmj doesn't have any claims belonging to one or more scopes. [(WARNING) oidcendpoint.userinfo.collect_user_info:161]
[2020-09-02 14:19:59,172] Some additional scopes doesn't have any claims. [(ERROR) oidc_op.views.service_endpoint:181]
Traceback (most recent call last):
  File "/home/wert/DEV3/OIDC-Project/django-oidc-op/example/oidc_op/views.py", line 175, in service_endpoint
    args = endpoint.process_request(req_args, **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/oidc/userinfo.py", line 124, in process_request
    info = collect_user_info(self.endpoint_context, session)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/userinfo.py", line 162, in collect_user_info
    raise ImproperlyConfigured("Some additional scopes doesn't have any claims.")
oidcendpoint.exception.ImproperlyConfigured: Some additional scopes doesn't have any claims.
ERROR:oidc_op.views:Some additional scopes doesn't have any claims.
Traceback (most recent call last):
  File "/home/wert/DEV3/OIDC-Project/django-oidc-op/example/oidc_op/views.py", line 175, in service_endpoint
    args = endpoint.process_request(req_args, **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/oidc/userinfo.py", line 124, in process_request
    info = collect_user_info(self.endpoint_context, session)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcendpoint/userinfo.py", line 162, in collect_user_info
    raise ImproperlyConfigured("Some additional scopes doesn't have any claims.")
oidcendpoint.exception.ImproperlyConfigured: Some additional scopes doesn't have any claims.
[2020-09-02 14:19:59,173] Internal Server Error: /userinfo [(ERROR) django.request.log_response:228]
ERROR:django.request:Internal Server Error: /userinfo

[rohe]

Give me an example of what an OP configuration with such a scope would look like .
I've tried it but can't replicate the error.

[peppelinux]

Yes Sir,

Simplest as possible, just put "ciao" in allowed_scopes (OP side, regarding a specific RP).
That's RP side:

Client registration done...
Connecting to Authorization url:
 {
  "url": "https://127.0.0.1:8000/authorization?redirect_uri=https%3A%2F%2F127.0.0.1%3A8099%2Fauthz_cb%2Fdjango_oidc_op&scope=openid+profile+email+address+phone&response_type=code&nonce=SwW1nQOIn75fuwQHdXk7tDvK&state=09FqGxrDjWkv9JLDlPDHJWfowSQelj4O&code_challenge=_eljnSs1r1zi3elhPg8KBiBOdjpzC7VTKnSVQjyE778&code_challenge_method=S256&client_id=1UUl6cwNigmj",
  "state": "09FqGxrDjWkv9JLDlPDHJWfowSQelj4O"
}

Starting connection to Authorization URL


The Authorization endpoint returns a HTML authentication form with a token
 {
  "token": "eyJhbGciOiJSUzI1NiIsImtpZCI6ImJXdG9SekV4VXkxak9GVXlSV2hwZUdkbFREWlBaME55TW1ka05ERlFaakJSUzJreVQwaExVazVJUVEifQ.eyJhdXRobl9jbGFzc19yZWYiOiAib2lkY2VuZHBvaW50LnVzZXJfYXV0aG4uYXV0aG5fY29udGV4dC5JTlRFUk5FVFBST1RPQ09MUEFTU1dPUkQiLCAicXVlcnkiOiAicmVkaXJlY3RfdXJpPWh0dHBzJTNBJTJGJTJGMTI3LjAuMC4xJTNBODA5OSUyRmF1dGh6X2NiJTJGZGphbmdvX29pZGNfb3Amc2NvcGU9b3BlbmlkK3Byb2ZpbGUrZW1haWwrYWRkcmVzcytwaG9uZSZyZXNwb25zZV90eXBlPWNvZGUmbm9uY2U9U3dXMW5RT0luNzVmdXdRSGRYazd0RHZLJnN0YXRlPTA5RnFHeHJEaldrdjlKTERsUERISldmb3dTUWVsajRPJmNvZGVfY2hhbGxlbmdlPV9lbGpuU3MxcjF6aTNlbGhQZzhLQmlCT2RqcHpDN1ZUS25TVlFqeUU3NzgmY29kZV9jaGFsbGVuZ2VfbWV0aG9kPVMyNTYmY2xpZW50X2lkPTFVVWw2Y3dOaWdtaiIsICJyZXR1cm5fdXJpIjogImh0dHBzOi8vMTI3LjAuMC4xOjgwOTkvYXV0aHpfY2IvZGphbmdvX29pZGNfb3AiLCAiaXNzIjogImh0dHBzOi8vMTI3LjAuMC4xOjgwMDAiLCAiaWF0IjogMTU5OTQ3MDE2NX0.HVANDQDsif69cAU_gOdG_kvrgCb4sBH5NbSScXvfZPq9lhNQGO9dHFbYYv8z9TmeDto-NzuMm269X1aL56W0Rgop6Xcj-p_7ushhbM0lt8m0aAbQ7C1oXAYbh2JL5-Br-sK4oHBCgJY-FTcsgFNlCc1oaZfL4ExTd-T1OoG4GoD0bG8WBDta2gzprt30jueOS1zga6niFYdi66v9jAjLDNztWG1ph5drqX0SbrU89TN995v8Dz2xKWVx5qpS_uFKOWPy4rhTe5n3i8PgLelEJ5b3hxMC5mO4ml_LRXEbWcW7X43uhDY-7mwNroH1UjXBdxN7Fx6gbFOatykA4SkOQw",
  "url": "verify/oidc_user_login/"
}


Authorization code content: 
 {
  "authn_class_ref": "oidcendpoint.user_authn.authn_context.INTERNETPROTOCOLPASSWORD",
  "query": "redirect_uri=https%3A%2F%2F127.0.0.1%3A8099%2Fauthz_cb%2Fdjango_oidc_op&scope=openid+profile+email+address+phone&response_type=code&nonce=SwW1nQOIn75fuwQHdXk7tDvK&state=09FqGxrDjWkv9JLDlPDHJWfowSQelj4O&code_challenge=_eljnSs1r1zi3elhPg8KBiBOdjpzC7VTKnSVQjyE778&code_challenge_method=S256&client_id=1UUl6cwNigmj",
  "return_uri": "https://127.0.0.1:8099/authz_cb/django_oidc_op",
  "iss": "https://127.0.0.1:8000",
  "iat": 1599470165
}


The Authorization returns a HttpRedirect (302) to https://127.0.0.1:8099/authz_cb/django_oidc_op?state=09FqGxrDjWkv9JLDlPDHJWfowSQelj4O&scope=openid+profile+email+address+phone&code=Z0FBQUFBQmZWZnBWd1l4WHhfOWtpMm1YQ1prbGtiMC1ORGJTaWpDaVBWd1J1MnBWbzdKTFR5a2dlQnhqd3luUHNsakQzN0VST1ZZMWxuXzNyVi1xcXpjRG1Va242c2ptZjlMcWVQb0RpZktOd09ENlFsYkx5NEE2RXdQclo2MXp4U2RwWEtYam1mbVl4TXBTWWlUc1JJN3cxU1l5LXZSSnRxa3JFeUtudGpJQzBienc4WGprTU9INmNCSndBdnZMX3pVYTZOc3B1LUlFYUdfdzZucnVIUzNMeU13UDJuUjFvYTg1aUlCNFlYNjJjaFo3S2V1eFpKcz0%3D&session_state=676381bcfb76fecd4aa997c735aeaf994b163599bb00f9a7835566cae97d29b3.ZfBneuHRV1XRI7ev&iss=https%3A%2F%2F127.0.0.1%3A8000&client_id=1UUl6cwNigmj
 

Going to finalize the session ...
 {
  "state": "09FqGxrDjWkv9JLDlPDHJWfowSQelj4O",
  "scope": "openid profile email address phone",
  "code": "Z0FBQUFBQmZWZnBWd1l4WHhfOWtpMm1YQ1prbGtiMC1ORGJTaWpDaVBWd1J1MnBWbzdKTFR5a2dlQnhqd3luUHNsakQzN0VST1ZZMWxuXzNyVi1xcXpjRG1Va242c2ptZjlMcWVQb0RpZktOd09ENlFsYkx5NEE2RXdQclo2MXp4U2RwWEtYam1mbVl4TXBTWWlUc1JJN3cxU1l5LXZSSnRxa3JFeUtudGpJQzBienc4WGprTU9INmNCSndBdnZMX3pVYTZOc3B1LUlFYUdfdzZucnVIUzNMeU13UDJuUjFvYTg1aUlCNFlYNjJjaFo3S2V1eFpKcz0=",
  "session_state": "676381bcfb76fecd4aa997c735aeaf994b163599bb00f9a7835566cae97d29b3.ZfBneuHRV1XRI7ev",
  "iss": "https://127.0.0.1:8000",
  "client_id": "1UUl6cwNigmj"
}
(500) Error
Traceback (most recent call last):
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/oauth2/__init__.py", line 172, in service_request
    response = service.get_response_ext(url, method, body, response_body_type, headers,
AttributeError: 'UserInfo' object has no attribute 'get_response_ext'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "snippets/rp_handler.py", line 173, in <module>
    run_rp_session(rph, issuer_id, args.u, args.p)
  File "snippets/rp_handler.py", line 124, in run_rp_session
    result = rph.finalize(request_args['iss'], request_args)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/__init__.py", line 843, in finalize
    access_token=token['access_token'])
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/__init__.py", line 690, in get_user_info
    request_args=request_args, **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/oauth2/__init__.py", line 108, in do_request
    state=_state, **_info)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/oauth2/__init__.py", line 176, in service_request
    **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/oauth2/__init__.py", line 148, in get_response
    response_body_type, **kwargs)
  File "/home/wert/DEV3/OIDC-Project/env/lib/python3.7/site-packages/oidcrp/oauth2/__init__.py", line 237, in parse_request_response
    raise ParseError("ERROR: Something went wrong: %s" % reqresp.text)

this PR would only log the error saying to admins that theresn't any claims belonging to "ciao" scope.
I can give you more informations.

My configurations (as usual) are here, RP side:
https://github.com/peppelinux/django-oidc-op/blob/master/example/data/oidc_rp/conf.django.yaml

OP side:
https://github.com/peppelinux/django-oidc-op/blob/master/example/example/oidc_op.conf.yaml

Mind that I use RDBMS definitions for client reg and allowed_scopes to them, so that's my UI:

and allowed_scopes is here, as they would be a list :)
https://github.com/peppelinux/django-oidc-op/blob/da3d3085cbc3354141e8d8bf19cfbcd975218c5b/oidc_op/models.py#L103

[rohe]

Ah, this opens another can of worms.
I've assumed you would have to define all 'locally defined' scopes using add_on/custom_scopes before you can use them to defined client specific allowed_scopes.
Am I correct in that you've used a new scope id in allowed_scopes without adding it to add_on/custom_scopes ?

[peppelinux]

I've just checked it, I can say YES, it's a global configuration error that I'm facing in this PR and what you're are telling us is formally correct.

The goal of this PR is to handle this misconfiguration with a human readable error/exception that could indicate to the admin that theresn't at least a scope with a valid claim to be released, so the userinfo endpoint it self raises the exception and this latter should be handled in the http view that handle the requests.

So that, this is just for handling an exception and nothing more in terms of feature, design and cool stuffs.

This PR would be also referenced to #70 because, a day, we'll have a global configuration validator to prevent this kind of errors. At this moment just an exception handling in a specific endpoint, if you agree