Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

New example doc format #1008

Merged
merged 6 commits into from
Jul 19, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@

Documentation
~~~~~~~~~~~~~

- Added a new experimental "Updated Examples" section which rewrites and reorders
many examples to aid in discovery. (:pr:`NUMBER`)
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

Creating a Guest Collection
===========================

TODO

.. literalinclude:: user_guest_collection.py
:caption: ``user_guest_collection.py`` [:download:`download <user_guest_collection.py>`]
:language: python
Original file line number Diff line number Diff line change
@@ -0,0 +1,45 @@
import globus_sdk
from globus_sdk.experimental.globus_app import UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
kurtmckee marked this conversation as resolved.
Show resolved Hide resolved
USER_APP = UserApp("my-simple-transfer", client_id=NATIVE_CLIENT_ID)

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
ENDPOINT_HOSTNAME = "https://b7a4f1.75bc.data.globus.org"
MAPPED_COLLECTION_ID = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"


def main():
gcs_client = globus_sdk.GCSClient(ENDPOINT_HOSTNAME, app=USER_APP)

# Comment out this line if the mapped collection is high assurance
attach_data_access_scope(gcs_client, MAPPED_COLLECTION_ID)

collection_request = globus_sdk.GuestCollectionDocument(
public=True,
collection_base_path="/",
display_name="example_guest_collection",
mapped_collection_id=MAPPED_COLLECTION_ID,
)

collection = gcs_client.create_collection(collection_request)
print(f"Created guest collection. Collection ID: {collection['id']}")


def attach_data_access_scope(gcs_client, collection_id):
"""Compose and attach a ``data_access`` scope for the supplied collection"""
endpoint_scopes = gcs_client.get_gcs_endpoint_scopes(gcs_client.endpoint_client_id)
collection_scopes = gcs_client.get_gcs_collection_scopes(collection_id)

manage_collections = globus_sdk.Scope(endpoint_scopes.manage_collections)
data_access = globus_sdk.Scope(collection_scopes.data_access, optional=True)

manage_collections.add_dependency(data_access)

gcs_client.add_app_scope(manage_collections)


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Identifying Entity Type
=======================

TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@

Endpoints and Collections
=========================

.. toctree::
:maxdepth: 1

identifying_entity_type
create_guest_collection/index
5 changes: 5 additions & 0 deletions docs/experimental/examples/flows/create.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Creating a Flow
===============

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/flows/delete.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Deleting a Flow
===============

TODO
11 changes: 11 additions & 0 deletions docs/experimental/examples/flows/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@

Flows
=====

.. toctree::
:maxdepth: 1

create
list
run
delete
5 changes: 5 additions & 0 deletions docs/experimental/examples/flows/list.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Listing Flows
=============

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/flows/run.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Running a Flow
==============

TODO
20 changes: 20 additions & 0 deletions docs/experimental/examples/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,20 @@

Updated Examples
================

This experimental doc restructures the existing `Globus SDK Examples <../../examples>`_
derek-globus marked this conversation as resolved.
Show resolved Hide resolved
section by:

* Grouping examples by common topic
* Consolidating & updating examples into a more uniform active format
* Updating references to leverage the latest SDK constructs (notably GlobusApp)


.. toctree::
:maxdepth: 2

transferring_data/index
endpoints_and_collections/index
flows/index
projects/index
oauth2/index
5 changes: 5 additions & 0 deletions docs/experimental/examples/oauth2/authorizers.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Using Authorizers
=================

TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Customizing Token Storage
=========================

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/oauth2/globus_app.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Using a Globus App
==================

TODO
12 changes: 12 additions & 0 deletions docs/experimental/examples/oauth2/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@

OAuth2 at Globus
================

.. toctree::
:maxdepth: 1

globus_app
authorizers
login_flows
customizing_token_storage
three_legged_oauth
5 changes: 5 additions & 0 deletions docs/experimental/examples/oauth2/login_flows.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Running Login Flows
===================

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/oauth2/three_legged_oauth.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Performing Three Legged OAuth in Flask
derek-globus marked this conversation as resolved.
Show resolved Hide resolved
========================================

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/projects/create.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Creating a Project
==================

TODO
5 changes: 5 additions & 0 deletions docs/experimental/examples/projects/delete.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Deleting a Project
==================

TODO
10 changes: 10 additions & 0 deletions docs/experimental/examples/projects/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,10 @@

Auth Projects
=============

.. toctree::
:maxdepth: 1

create
list
delete
5 changes: 5 additions & 0 deletions docs/experimental/examples/projects/list.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Listing Projects
================

TODO
11 changes: 11 additions & 0 deletions docs/experimental/examples/transferring_data/index.rst
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@

Transferring Data
=================

.. toctree::
:maxdepth: 1

submit_transfer/index
schedule_transfer/index
task_deadlines
recursive_ls
Comment on lines +8 to +11
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm guessing that submit_transfer/ and schedule_transfer/ are expected to have example files that could stand to be grouped in the same subdirectory, but is there an expectation that task_deadlines and recursive_ls make more sense to deviate from that?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think each should be using /index pages, but it doesn't need to happen this minute.
We moved several of these to dirs to be able to make their example scripts separate files which are available for download. I would like to employ that structure everywhere, but we can partially do it as we work through these examples and fill them out.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would like to employ that structure everywhere, but we can partially do it as we work through these examples and fill them out.

That was basically my thought.

  • Certain examples will have their own file structure linking to scripts.
  • Certain examples will be a single file with inline code references.

I'm not sure which camp each example will fall into but as I fill them in, I'll be editing them into the more accurate one.

Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Recursively Listing a Filesystem
================================

TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Scheduling a Transfer
=====================

TODO
Original file line number Diff line number Diff line change
@@ -0,0 +1,54 @@

Initiating a Transfer
=====================

Moving data within the Globus Ecosystem can be performed by submitting a `Transfer Task`
to the Globus Transfer service.

The below example demonstrates how to submit that task using a ``TransferClient``.

.. note::
The below example is for moving data between two publicly hosted "tutorial"
collections.
You should replace these collection ids and path values with your own collection
ids and paths.

.. note::
Some collections require you to attach a "data_access" scope to your transfer
request. You should evaluate whether this is necessary for both your source and
destination collections and omit the ``transfer_client.add_app_data_access_scope``
calls as needed.

A collection requires "data_access" if it is (1) a mapped collection and (2) is
not high assurance.

If you're still not able to determine, an alternative reactive script is included
below.

.. literalinclude:: submit_transfer.py
:caption: ``submit_transfer.py`` [:download:`download <submit_transfer.py>`]
:language: python


Reactively solving ConsentRequired errors
derek-globus marked this conversation as resolved.
Show resolved Hide resolved
------------------------------------------

Some collections require you to attach a "data_access" scope to your transfer
request.

This can be typically be done proactively: if you know which collections you or your
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I would not characterize either case as typical. In my experience, our users break into three basic groups:

  • those who are writing tools against fixed collections (both ends fixed)
  • those who are writing tools against one fixed collection (one end fixed, one end open)
  • those who are writing generic tools (both ends open)

And the latter two groups are more similar than the first.

If anything, knowing both of your collection IDs ahead of time is the exception, not the rule.

Do I object to the ordering used in this doc, or the overall presentation? Eh. Definitely not strongly, but this note jumps out at me and makes me feel that something is slightly amiss. I'm not sure, but I think that we should actually have a preamble that says:

  • if you know one or both of your collection IDs, do it this way
  • if you don't know the collection IDs, do it that way

with intra-document links.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hmm, I can take another stab at this doc with the preamble you suggest.

Copy link
Contributor Author

@derek-globus derek-globus Jul 18, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

users will interacting with, you can evaluate whether they require this scope (they are
non-High Assurance mapped collections), and attach it through the client if so.

Sometimes however you do not know in advance the full range of collections that your
derek-globus marked this conversation as resolved.
Show resolved Hide resolved
script will be interacting with. This script demonstrates how to reactively solve the
``ConsentRequired`` errors which are raised when a collection requires a "data_access"
scope but the client did not attach one.

.. note::
Given that this script is reactive, it can involve two user login interactions
kurtmckee marked this conversation as resolved.
Show resolved Hide resolved
where the above script only involves one.

.. literalinclude:: submit_transfer_reactive_consent_required.py
:caption: ``submit_transfer_reactive_consent_required.py`` [:download:`download <submit_transfer_reactive_consent_required.py>`]
:language: python
Original file line number Diff line number Diff line change
@@ -0,0 +1,38 @@
import globus_sdk
from globus_sdk.experimental.globus_app import UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
USER_APP = UserApp("my-simple-transfer", client_id=NATIVE_CLIENT_ID)

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"


def main():
transfer_client = globus_sdk.TransferClient(app=USER_APP)

# Comment out each of these lines if the referenced collection is either
# (1) A guest collection or (2) high assurance.
transfer_client.add_app_data_access_scope(SRC_COLLECTION)
transfer_client.add_app_data_access_scope(DST_COLLECTION)

transfer_request = globus_sdk.TransferData(
source_endpoint=SRC_COLLECTION,
destination_endpoint=DST_COLLECTION,
)
transfer_request.add_item(SRC_PATH, DST_PATH)

task = transfer_client.submit_transfer(transfer_request)
print(f"Submitted transfer. Task ID: {task['task_id']}.")


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
import globus_sdk
from globus_sdk.experimental.globus_app import UserApp

# Tutorial Client ID - <replace this with your own client>
NATIVE_CLIENT_ID = "61338d24-54d5-408f-a10d-66c06b59f6d2"
USER_APP = UserApp("my-simple-transfer", client_id=NATIVE_CLIENT_ID)

# Globus Tutorial Collection 1
# https://app.globus.org/file-manager/collections/6c54cade-bde5-45c1-bdea-f4bd71dba2cc
SRC_COLLECTION = "6c54cade-bde5-45c1-bdea-f4bd71dba2cc"
SRC_PATH = "/share/godata/file1.txt"

# Globus Tutorial Collection 2
# https://app.globus.org/file-manager/collections/31ce9ba0-176d-45a5-add3-f37d233ba47d
DST_COLLECTION = "31ce9ba0-176d-45a5-add3-f37d233ba47d"
DST_PATH = "/~/example-transfer-script-destination.txt"


def main():
transfer_client = globus_sdk.TransferClient(app=USER_APP)

transfer_request = globus_sdk.TransferData(
source_endpoint=SRC_COLLECTION,
destination_endpoint=DST_COLLECTION,
)
transfer_request.add_item(SRC_PATH, DST_PATH)

try:
task = transfer_client.submit_transfer(transfer_request)
except globus_sdk.TransferAPIError as err:
if not err.info.consent_required:
raise

print("Additional consent required.")
transfer_client.add_app_scope(err.info.consent_required.required_scopes)

task = transfer_client.submit_transfer(transfer_request)
print(f"Submitted transfer. Task ID: {task['task_id']}.")


if __name__ == "__main__":
main()
Original file line number Diff line number Diff line change
@@ -0,0 +1,5 @@

Setting Task Deadlines
======================

TODO
2 changes: 2 additions & 0 deletions docs/index.rst
Original file line number Diff line number Diff line change
Expand Up @@ -45,6 +45,8 @@ Table of Contents

testing/index
experimental/index
experimental/examples/index


.. toctree::
:caption: Additional Info
Expand Down
Loading