Endpoints 2.0.0b7 - Firebase, Service Account tokens

[lhmatt]

Spent a good few hours trying to get non-default Service Accounts and Firebase tokens to work with the endpoints framework. I wanted to feedback some issues, some work-arounds, and some general pointers for anyone else trying to do the same thing.

After following the quickstart guides on the endpoint docs I was able to get the tokens validating on the endpoint proxy. However, when the tokens reached the endpoints framework things started to go wrong -- the tokens failed to validate.

It seems a little odd that the framework re-validates the tokens, though I'm sure there is a reason why.
The endpoint proxy passes the X-Endpoint-API-UserInfo header, so perhaps we could just check that instead?

Regardless, after some tinkering I managed to get the endpoint framework to play nicely with Firebase tokens and custom Service Accounts. These are hacks, but they serve as a proof of concept.
Although it took a long time to figure out, there weren't actually that many modifications to make. All of the changes are in endpoints/user_id_token.py.

For reference, here is the api decorator I am using:

main.py

...

firebase_issuer = endpoints.Issuer(
    issuer='https://securetoken.google.com/[YOUR-PROJECT-ID]',
    jwks_uri='https://www.googleapis.com/service_accounts/v1/metadata/x509/[email protected]'
)

service_account_issuer = endpoints.Issuer(
    issuer='[SERVICE-ACCOUNT-NAME]@[YOUR-PROJECT-ID].iam.gserviceaccount.com',
    jwks_uri='https://www.googleapis.com/service_accounts/v1/metadata/x509/[SERVICE-ACCOUNT-NAME]@[YOUR-PROJECT-ID].iam.gserviceaccount.com'
)

@endpoints.api(name='echo',
               version='v1',
               allowed_client_ids=[[SERVICE-ACCOUNT-CLIENT-ID], 'firebase_auth'],
               audiences={
                   'google_jwt': ['this.is.a.test'],
                   'firebase': ['[YOUR-PROJECT-ID]'],
               },
               issuers={
                   'google_jwt': service_account_issuer,
                   'firebase': firebase_issuer,
               })
...

This is basically the echo example from the quickstart, modified to support Firebase tokens and non-default Service Accounts.

General Comments:

• The use of try-except with top-level Exceptions makes debugging incredibly frustrating because it swallows some important details about why the exception occurred. I can see why it is used in the code, but I think it could be handled better. As it stands you need a lot of debug logging throughout the code in order to work out why a token fails to validate.

• The jwks_uri for each issuer needs to be the raw variant in order to work properly with the endpoints framework. However, the endpoints proxy only accepts the x509 variant. I therefore had to override the uris in the endpoint framework. This is not mentioned in the endpoint docs, as far as I can tell.

• When using Firebase tokens, aud (audience) is your project ID. This doesn't appear to be mentioned in the endpoint docs.

• Service accounts must supply their client_id as the azp field in order for the validation to pass.

_verify_parsed_token

Custom Security Issuers

By default the code checks _ISSUERS only. This does not account for custom security directives in the api e.g. Firebase. I modified the issuer check to include custom values from the api security definition.

...
def _verify_parsed_token(parsed_token, audiences, allowed_client_ids):
  # Verify the issuer.
  iss = parsed_token.get('iss')
  if iss not in _ISSUERS and iss not in ['[SERVICE-ACCOUNT-NAME]@[YOUR-PROJECT-ID].iam.gserviceaccount.com', 'https://securetoken.google.com/[YOUR-PROJECT-ID]']:
    logging.warning('Issuer was not valid: %s', parsed_token.get('iss'))
    return False
...

Audiences

By default the code does not handle audiences being a dict (provider => audiences). I added a loop to iterate through each of the providers' audience values and check for a match. A potentially better solution could be to combine all of the provider audiences into a single set on init.

    ...
# Check audiences.
  aud = parsed_token.get('aud')
  logging.debug(u'AUD: {}'.format(aud))
  if not aud:
    logging.warning('No aud field in token')
    return False
  # Special handling if aud == cid.  This occurs with iOS and browsers.
  # As long as audience == client_id and cid is allowed, we need to accept
  # the audience for compatibility.
  cid = parsed_token.get('azp')

  valid_audience = False
  if aud == cid:
    valid_audience = True
  elif isinstance(audiences, dict):
    for provider, provider_audiences in audiences.iteritems():
      if aud in provider_audiences:
        valid_audience = True
        break
  elif aud in audiences:
    valid_audience = True

  if not valid_audience:
    logging.warning('Audience not allowed: %s', aud)
    return False
    ...

Firebase tokens

Firebase auth tokens do not include a cid value which results in a Client ID is not allowed message. To circumvent this, I added a check on see if iss is equal to https://securetoken.google.com/[YOUR-PROJECT-ID]. If it does then assign a cid of firebase_auth (see the main.py code above for the allowed_client_ids definition)

...
  if not cid and iss == 'https://securetoken.google.com/[YOUR-PROJECT-ID]':
    cid = 'firebase_auth'
...

_verify_signed_jwt_with_certs

By default the code only checks _DEFAULT_CERT_URI, which obviously causes Firebase, and Service Account, tokens to fail to validate. To remedy this, I modified the code to retrieve the aud (audience) value from token_body and used this to lookup the correct cert_uri for each token type.

  ...
  token_aud = parsed.get('aud')

  if token_aud == 'this.is.a.test':
    logging.debug(u'Service Account token found based on audience: {}'.format(token_aud))
    cert_uri = 'https://www.googleapis.com/service_accounts/v1/metadata/raw/[SERVICE-ACCOUNT-NAME]@[YOUR-PROJECT-ID].iam.gserviceaccount.com'
  elif token_aud == '[YOUR-PROJECT-ID]':
    logging.debug(u'Firebase token found based on audience: {}'.format(token_aud))
    cert_uri = 'https://www.googleapis.com/service_accounts/v1/metadata/raw/[email protected]'
  else:
    logging.debug(u'Could not determine token type based on audience: {}'.format(token_aud))
    cert_uri = _DEFAULT_CERT_URI
  ...

Complete Function Reference

endpoints/user_id_token.py

def _verify_parsed_token(parsed_token, audiences, allowed_client_ids):
  """Verify a parsed user ID token.

  Args:
    parsed_token: The parsed token information.
    audiences: The allowed audiences.
    allowed_client_ids: The allowed client IDs.

  Returns:
    True if the token is verified, False otherwise.
  """
  # Verify the issuer.
  iss = parsed_token.get('iss')
  if iss not in _ISSUERS and iss not in ['[email protected]', 'https://securetoken.google.com/lh-endpoint-test']:
    logging.warning('Issuer was not valid: %s', parsed_token.get('iss'))
    return False

  # Check audiences.
  aud = parsed_token.get('aud')
  logging.debug(u'AUD: {}'.format(aud))
  if not aud:
    logging.warning('No aud field in token')
    return False
  # Special handling if aud == cid.  This occurs with iOS and browsers.
  # As long as audience == client_id and cid is allowed, we need to accept
  # the audience for compatibility.
  cid = parsed_token.get('azp')

  valid_audience = False
  if aud == cid:
    valid_audience = True
  elif isinstance(audiences, dict):
    for provider, provider_audiences in audiences.iteritems():
      if aud in provider_audiences:
        valid_audience = True
        break
  elif aud in audiences:
    valid_audience = True

  if not valid_audience:
    logging.warning('Audience not allowed: %s', aud)
    return False

  if not cid and iss == 'https://securetoken.google.com/lh-endpoint-test':
    cid = 'firebase_auth'

  # Check allowed client IDs.
  if list(allowed_client_ids) == SKIP_CLIENT_ID_CHECK:
    logging.warning('Client ID check can\'t be skipped for ID tokens.  '
                    'Id_token cannot be verified.')
    return False
  elif not cid or cid not in allowed_client_ids:
    logging.warning('Client ID is not allowed: %s', cid)
    return False

  if 'email' not in parsed_token:
    return False

  return True

def _verify_signed_jwt_with_certs(
    jwt, time_now, cache,
    cert_uri=_DEFAULT_CERT_URI):
  """Verify a JWT against public certs.

  See http://self-issued.info/docs/draft-jones-json-web-token.html.

  The PyCrypto library included with Google App Engine is severely limited and
  so you have to use it very carefully to verify JWT signatures. The first
  issue is that the library can't read X.509 files, so we make a call to a
  special URI that has the public cert in modulus/exponent form in JSON.

  The second issue is that the RSA.verify method doesn't work, at least for
  how the JWT tokens are signed, so we have to manually verify the signature
  of the JWT, which means hashing the signed part of the JWT and comparing
  that to the signature that's been encrypted with the public key.

  Args:
    jwt: string, A JWT.
    time_now: The current time, as a long (eg. long(time.time())).
    cache: Cache to use (eg. the memcache module).
    cert_uri: string, URI to get cert modulus and exponent in JSON format.

  Returns:
    dict, The deserialized JSON payload in the JWT.

  Raises:
    _AppIdentityError: if any checks are failed.
  """

  segments = jwt.split('.')

  if len(segments) != 3:
    # Note that anywhere we print the jwt or its json body, we need to use
    # %r instead of %s, so that non-printable characters are escaped safely.
    raise _AppIdentityError('Token is not an id_token (Wrong number of '
                            'segments)')
  signed = '%s.%s' % (segments[0], segments[1])

  signature = _urlsafe_b64decode(segments[2])

  # pycrypto only deals in integers, so we have to convert the string of bytes
  # into a long.
  lsignature = long(signature.encode('hex'), 16)

  # Verify expected header.
  header_body = _urlsafe_b64decode(segments[0])
  try:
    header = json.loads(header_body)
  except:
    raise _AppIdentityError("Can't parse header")
  if header.get('alg') != 'RS256':
    raise _AppIdentityError('Unexpected encryption algorithm: %r' %
                            header.get('alg'))

  # Parse token.
  json_body = _urlsafe_b64decode(segments[1])
  try:
    parsed = json.loads(json_body)
  except:
    raise _AppIdentityError("Can't parse token body")
  logging.debug(u'JWT Token Body: {}'.format(json_body))

  token_aud = parsed.get('aud')

  if token_aud == 'this.is.a.test':
    logging.debug(u'Service Account token found based on audience: {}'.format(token_aud))
    cert_uri = 'https://www.googleapis.com/service_accounts/v1/metadata/raw/[email protected]'
  elif token_aud == 'lh-endpoint-test':
    logging.debug(u'Firebase token found based on audience: {}'.format(token_aud))
    cert_uri = 'https://www.googleapis.com/service_accounts/v1/metadata/raw/[email protected]'
  else:
    logging.debug(u'Could not determine token type based on audience: {}'.format(token_aud))
    cert_uri = _DEFAULT_CERT_URI

  certs = _get_cached_certs(cert_uri, cache)
  if certs is None:
    raise _AppIdentityError(
        'Unable to retrieve certs needed to verify the signed JWT')
  logging.debug(u'Retrieved certs: {}'.format(certs))

  # Verify that we were able to load the Crypto libraries, before we try
  # to use them.
  if not _CRYPTO_LOADED:
    raise _AppIdentityError('Unable to load pycrypto library.  Can\'t verify '
                            'id_token signature.  See http://www.pycrypto.org '
                            'for more information on pycrypto.')

  # SHA256 hash of the already 'signed' segment from the JWT. Since a SHA256
  # hash, will always have length 64.
  local_hash = SHA256.new(signed).hexdigest()

  # Check signature.
  verified = False
  for keyvalue in certs['keyvalues']:
    try:
      modulus = _b64_to_long(keyvalue['modulus'])
      exponent = _b64_to_long(keyvalue['exponent'])
      key = RSA.construct((modulus, exponent))

      # Encrypt, and convert to a hex string.
      hexsig = '%064x' % key.encrypt(lsignature, '')[0]
      # Make sure we have only last 64 base64 chars
      hexsig = hexsig[-64:]

      # Check the signature on 'signed' by encrypting 'signature' with the
      # public key and confirming the result matches the SHA256 hash of
      # 'signed'.
      verified = (hexsig == local_hash)
      if verified:
        break
    except Exception, e:  # pylint: disable=broad-except
      # Log the exception for debugging purpose.
      logging.debug(
          'Signature verification error: %s; continuing with the next cert.', e)
      continue
  if not verified:
    raise _AppIdentityError('Invalid token signature')

  # Check creation timestamp.
  iat = parsed.get('iat')
  if iat is None:
    raise _AppIdentityError('No iat field in token')
  earliest = iat - _CLOCK_SKEW_SECS

  # Check expiration timestamp.
  exp = parsed.get('exp')
  if exp is None:
    raise _AppIdentityError('No exp field in token')
  if exp >= time_now + _MAX_TOKEN_LIFETIME_SECS:
    raise _AppIdentityError('exp field too far in future')
  latest = exp + _CLOCK_SKEW_SECS

  if time_now < earliest:
    raise _AppIdentityError('Token used too early, %d < %d' %
                            (time_now, earliest))
  if time_now > latest:
    raise _AppIdentityError('Token used too late, %d > %d' %
                            (time_now, latest))

  return parsed

[Tungamirai]

Hi I was trying to do the same thing for days now trying , to get endpoints and firebase authentication working together, is your workaround still the best way to do it ? or did google fix this and make it less confusing ?

[lhmatt]

Hey @Tungamirai. I haven't checked out the latest version but, it doesn't look like the users_id_token.py file has been modified in the repository. I'm going to assume this is still a problem.

[Tungamirai]

@lhmatt thanks for the reply , are they any solid examples/tutorials form start to end that use endpoints , with firebase authentication or any full repositories or complete projects that show this working ?

I just started to learn app-engine and endpoints and it just seems very frustrating in general, was expecting it to be better documented with solid examples, are problems like this common with endpoints ? , I dont understand why google hasn't just made a solid example that actually works

[lhmatt]

No worries, @Tungamirai. I have not found a full example with firebase auth, hence my post above. The closest I found was in the docs: https://cloud.google.com/endpoints/docs/frameworks/python/authenticating-users but I couldn't get it to work during the beta. Definitely give it a go if you can.

This version of endpoints has just come out of beta so it's understandable that the docs may be missing some information. The best thing you can do is submit feedback (there is a button the the bottom of each page on the docs) to help them improve the information available.
Failing that, there is almost always someone on StackOverflow that has experienced the same problems as you!

[Tungamirai]

Thanks @lhmatt , will be keeping my eyes open for a documentation update , the link you found is the same one i also found but also couldn't get it to work, I hope the docs are updated soon

Thanks again

[Synthoova]

@lhmatt Thanks so much for your post. It allowed me to finally get Firebase authentication working.

Google devs.....any word on when we can see the issues with Firebase authentication repaired in endpoints?

[tekulvw]

Thank you so much! I've been banging my head against the wall for days trying to get this working and just now found this post. I ended up using google.auth.id_token.verify_firebase_token() to verify the token on one service and spawning a new microservice running pyrebase just to get user information. I really hope this gets fixed quickly.

[sxhan]

@lhmatt Thank you so much for this. I spent a good part of my day trying to get my app to work, looking through the endpoints source code and thinking that theres no way this Firebase thing should work as described in the guides...

Are you planning on making a pull request for this at all? If not, I was thinking of writing up something based on your PoC.

[lhmatt]

No worries, @sxhan. I've abandoned my efforts with this framework so feel free to create your own pull request.

After quite a bit of investigation, I've moved on to using Endpoints via ESP on App Engine flexible. I know not everyone can/wants to move to the flexible environment but it works for what I need to do. You trade higher running costs for developer productivity - with GAE flexible you can use the existing OpenAPI (swagger) tools to generate server stubs and client libraries, and you're not reliant on Google to build features into this framework.
Note also that you can use both standard and flexible environments in the same project.

[Al77056]

The example given for 3rd party issuer is like the following:

@endpoints.api(
        audiences={'auth0': ['aud-1.auth0.com', 'aud-2.auth0.com']},
        issuers={'auth0': endpoints.Issuer('https://test.auth0.com',
                                           'https://test.auth0.com/.well-known/jwks.json')})

And version 2.0.5 works with the custom issuer I am using.

[sxhan]

@lhmatt Thanks for the suggestion! I was ready to do that yesterday before I found your post, but your reply is making me consider going that route again. Will probably play with this framework a bit more.

@Al77056, thanks for the note. Unfortunately I can't get my Firebase Authentication to work without the changes lhmatt mentioned despite being on 2.0.5 . Are you testing your app locally or in the cloud? I see the following comment from a recent commit:

+  If `endpoints_management.control.wsgi.AuthenticationMiddleware` is enabled,
 +  this returns the user info decoded by the middleware. Otherwise, if the
 +  current request uses an id_token, this validates and parses the token against
 +  the info in the current request handler and returns the user.  Or, for an
 +  Oauth token, this call validates the token against the tokeninfo endpoint and
 +  oauth.get_current_user with the scopes provided in the method's decorator.

Which looks promising. Looks like AuthenticationMiddleware is supposed to be doing the token verifications and overriding the _ENDPOINTS_USER_INFO environment variable, which causes users_id_token.get_current_user(). to skip all the verification logic. The question is... how do I enable that?

[Al77056]

@sxhan, I am testing in the cloud. Before changes made in 2.0.5, Endpoints framework was not validating JWT signed by Google service account. 2.0.5 fixes that.

[tangiel]

@sxhan Could you provide sample logs of what happens when you try to use Firebase auth? 2.0.5 should indeed fix it.

[gianni-di-noia]

@tangiel probably this screeshot of logs can clarifay.
I'm using GAE standard, the GAE users API for internal use and firebase auth to authenticate users requests.
As you can see in the screenshot, endpoints.get_current_user() returns the GAE users obj (line 1), the same obj of google.appengine.api.users.get_current_user().
But reading the docs, it should returns the google.api.auth.user_info obj (line 2) which contains the firebase user_id (line 3).

[tangiel]

Ah, so the issue isn't really the type of the user object returned, rather that it doesn't contain the id. Correct?

[gianni-di-noia]

I don't want to create confusion, I just start from the snippet example in the documentation https://cloud.google.com/endpoints/docs/frameworks/python/authenticating-users#authenticating_with_firebase_auth
where user = endpoints.get_current_user() return the something related with firebase and not with GAE users API.

[sxhan]

Looks like I'm hijacking the thread a bit. Apologies in advance for that.

@tangiel, my problem is different than that of @presveva. My issue was that I kept getting ERROR 2017-05-16 20:26:38,633 users_id_token.py:362] Token info endpoint returned status 400: Invalid Value whenever I tried to make a request to the endpoint.

I now realize that it may be because I was testing my endpoints application in the local development server (via dev_appserver.py). I tried deploying to the cloud following the instructions here and then here and was able to get token validation working and the user's email address returned via endpoints.get_current_user().

However, I am unable to get it working with the local server. I see that the validation code path for local server is different than for when its deployed (_set_bearer_user_vars_local() vs _set_bearer_user_vars()), and that local development server usage was not mentioned anywhere in the docs, so maybe I am trying to use a non-existent feature. Your feedback is appreciated!

Following the example Al77056 provided, this is how I'm setting up the endpoints decorator:

firebase_issuer = endpoints.Issuer(
    issuer='https://securetoken.google.com/tutorial-165519',
    jwks_uri='https://www.googleapis.com/service_accounts/v1/metadata/x509/[email protected]')
@endpoints.api(name='echo', version='v1',
               issuers={"firebase": firebase_issuer},
               audiences={"firebase": [PROJECT_ID]},
               allowed_client_ids=["firebase_auth"])

Edit: looks like its a known issue, as reported in #55

[sllegendre]

@sxhan Did you make it work by appling the changes in 'endpoints/user_id_token.py' as decribed by Ihmatt or was that no longer neccessary for you?

[sxhan]

@sllegendre, following lhmatt's suggestions made it work, but I still have trouble without those changes. I ended up writing a wrapper around endpoints.get_current_user() to make development easier. Something like this...

def get_current_user():
    """Returns the endpoint authenticated user object"""
    class FakeUser:
        def email(self):
            return "[email protected]"

    if (os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/') or
        os.getenv('SERVER_SOFTWARE', '').startswith('Development/1.0 (testbed)')): 
        user = endpoints.get_current_user()
        if not user:
            raise endpoints.UnauthorizedException('Authorization required')
    else:
        user = FakeUser()
    return user

This made it easy for me to test all my endpoints locally, although it won't help you debug any configuration issues with firebase auth.

The call to os.getenv('SERVER_SOFTWARE', '').startswith('Google App Engine/') to check for development vs. production environment is suggested in the GAE docs, so that should be safe to do. Just be careful if you mess around with those env vars.

[tmst]

@sxhan I was wondering how to abstract this for use in unit tests. There's also the users.get_current_user method, so the FakeUser may not be required.

[ankitpatidar1]

@lhmatt thanks ,my project working with firebase and endpoint.but my issue is when I add file from Google drive with help of picker library of JavaScript then I try to store these document .that time I get 401(not authenticate ) error