[ISSUE] Documentation for creating catalog and adding members

[cpc-ruffin]

Description
I am trying to create a group with an azure service principal and add that service principal as a member of the group. I had to dig through the documentation to find the iam.ComplexValue() syntax to pass into the members parameter of a.groups.create(). I am able to pass in my own credentials into the members variables and it works as expected but I cannot figure out how to pass in the service principal credentials. I have tried passing in the client_id and the databricks acccount name into service_principal_name but neither add the service principal into this group. Am I missing something? Or is there a better way to do this?

Reproduction
a.groups.create(
display_name = <group_name>,
id=None,
members=[
iam.ComplexValue(display=service_principal_name, primary=None, type=None,value=spID),
iam.ComplexValue(display='', primary=None, type=None, value='<account_ID')
]
)

I am not sure if this is the place to post this but could not find anywhere else.

Update

After some testing, the group and members seem to be created fine, but the members of the groups do not have the correct permission. Below is a recreation of the issue.

1. Created catalog with account1
   CREATE CATALOG sdk_catalog

2. Used Databricks sdk account client to create a new group with the below code
   for group in a.groups.list():
   groups[group.display_name] = group

de_group = groups["Data Engineers"].display_name
de_group_id = groups["Data Engineers"].id
print(de_group, de_group_id)

group_name = "SDK Test Group"

a.groups.create(
display_name=group_name,
id=None,
members=[
iam.ComplexValue(display=de_group, primary=None, type=None,value=de_group_id)
]
)

3. Using account1 ran the below command
   ALTER CATALOG sdk_catalog OWNER TO SDK Test Group;

4. Tried to delete catalog with account2 (which is apart of “Data Engineers” group) and got the below error.

Disclaimer

While testing there were times when using this method did work. Some of the times members would have the correct permissions and other times they would not. We could not narrow down when/how this issue would come about.

Workaround

We also found that using .create() without specifying the members and then using .update() to add members did correctly give permissions to the members of the group.

[mgyucht]

Thank you for filing this report @cpc-ruffin! We do need to improve the documentation for how to use this API, to be honest. I'll work with the upstream IAM team to see if they can help improve the documentation for this API. We also have #400 which may help you better navigate the docs and find the data structures you're looking for.

[cpc-ruffin]

Thank you for filing this report @cpc-ruffin! We do need to improve the documentation for how to use this API, to be honest. I'll work with the upstream IAM team to see if they can help improve the documentation for this API. We also have #400 which may help you better navigate the docs and find the data structures you're looking for.

Thanks for the response @mgyucht. The method above did correctly add the service principal to the group but it did not have the correct permissions I was assigning it. We did some further testing with service principals/groups/permissions (with/without the sdk) and we believe Databricks has an internal issue with service principals and groups. There seems to be no functionality issues with the adding service principals to a group through the sdk (I don't think).

On the other hand, I do agree that the documentation needs to be improved. I shouldn't have to dive into the repo to find the syntax on how to add members into a newly created group. This feature is not the only time I ran into documentation issues with the sdk, but I am glad you guys are working on it. Other than that, the sdk as been super useful and I appreciate the work you guys are doing!

(Also, small UI error also with the below page. You cannot scroll when the navigation goes below your viewable screen. https://databricks-sdk-py.readthedocs.io/en/latest/workspace/index.html )

[mgyucht]

Thanks for the tip on the docsite navigation. We're tracking this here: #415, which follows an upstream issue in our Sphinx documentation generator: sphinx-doc/alabaster#212.