Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update "Direct DAAC S3 Bucket Access" Notebook #402

Open
1 task
smk0033 opened this issue Jun 4, 2024 · 3 comments
Open
1 task

Update "Direct DAAC S3 Bucket Access" Notebook #402

smk0033 opened this issue Jun 4, 2024 · 3 comments
Assignees
Labels
documentation Improvements or additions to documentation

Comments

@smk0033
Copy link
Contributor

smk0033 commented Jun 4, 2024

Description

Need to update Direct DAAC S3 Bucket Access (BETA) Notebook per this ticket: https://github.com/NASA-IMPACT/active-maap-sprint/issues/884

Acceptance Criteria/Tasks

  • update notebook (update roles)
@smk0033 smk0033 added the documentation Improvements or additions to documentation label Jun 4, 2024
@smk0033 smk0033 self-assigned this Jun 11, 2024
@smk0033
Copy link
Contributor Author

smk0033 commented Jun 18, 2024

Unassigning myself in case someone else can pick it up before the end of the sprint. If not, I'll work on it whenever I get back next week

@smk0033 smk0033 removed their assignment Jun 18, 2024
@smk0033 smk0033 self-assigned this Jul 8, 2024
@wildintellect
Copy link
Collaborator

@smk0033 @chuckwondo please post a public example in this ticket. Only our team can read the internal MAAP document. PS: EIS Fire needs this info ASAP for MAAP-Project/Community#1032

@wildintellect
Copy link
Collaborator

Working through this with @chuckwondo we realized that it works in DPS, but the ADE requires 1 time manual step on the part of users to configure the AWS config profile first.
MAAP-Project/Community#1001

%%bash

mkdir -p /projects/.aws && cat >> ~/.aws/config <<CONFIG
[profile maap-data-reader] 
region = us-west-2 
role_arn = <redacted>/maap-data-reader 
credential_source = Ec2InstanceMetadata 
CONFIG
#python 
import xarray as xr
import fsspec

sample_file = 's3://lp-prod-protected/VJ114IMG.002/VJ114IMG.A2024198.0942.002.2024198150736/VJ114IMG.A2024198.0942.002.2024198150736.nc'
s3_fsspec = fsspec.filesystem("s3", profile="maap-data-reader")
test = xr.open_dataset(s3_fsspec.open(sample_file), engine="h5netcdf", phony_dims='sort')

Example relates to MAAP-Project/Community#1032 and MAAP-Project/Community#1031

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
documentation Improvements or additions to documentation
Projects
None yet
Development

No branches or pull requests

3 participants