Skip to content

Latest commit

 

History

History
44 lines (30 loc) · 3.1 KB

README.md

File metadata and controls

44 lines (30 loc) · 3.1 KB
Raw

OCI Function Example to Invoke a Data Science Notebook

Introduction

This contains one OCI function to execute a sample Data Science notebook using OCI Data Science.

There are various properties parameterized for demonstration purposes, more could be done such as shape and subnet, but for a publicly available ipynb this is all that is needed;

  • noteBook the Python notebook URL to use, for example you can use the sample provided here to test https://raw.githubusercontent.com/tensorflow/docs/master/site/en/tutorials/customization/basics.ipynb
  • jobName provide the job name for the OCI Data Science job that will be launched
  • logGroupId the log group Ocid to be used.
  • projectId the OCI Data Science project Ocid - create a project to be used
  • compartmentId the compartment Ocid where the job will be created
  • outputFolder the output folder in OCI to write the results ie. oci://bucketName@namespace/objectFolderName

The function is based on the Python example in [https://accelerated-data-science.readthedocs.io/en/latest/user_guide/jobs/run_notebook.html#tensorflow-example] (TensorFlow example) the OCI Data Science accelerated data science documentation.

Permissions

Example permissions to test from functions and from OCI Data Integration.

Resource principal for testing from OCI Functions for example;

  • allow any-user to manage data-science-family in compartment YOURCOMPARTMENT where ALL {request.principal.type='fnfunc'}
  • allow any-user to manage log-groups in compartment YOURCOMPARTMENT where ALL {request.principal.type='fnfunc'}
  • allow any-user to manage log-content in compartment YOURCOMPARTMENT where ALL {request.principal.type='fnfunc'}

Resource principal for testing from Workspaces for example;

  • allow any-user to manage data-science-family in compartment YOURCOMPARTMENT where ALL {request.principal.type='disworkspace',request.principal.id='ocid1.disworkspace.oc1.iad....'}
  • allow any-user to manage log-groups in compartment YOURCOMPARTMENT where ALL {request.principal.type='disworkspace',request.principal.id='ocid1.disworkspace.oc1.iad....'}
  • allow any-user to manage log-content in compartment YOURCOMPARTMENT where ALL {request.principal.type='disworkspace',request.principal.id='ocid1.disworkspace.oc1.iad....'}

Function Deployment

Follow the regular function deployment pattern. I will not go through this here, there are tutorials on Functions here;

[https://docs.oracle.com/en-us/iaas/Content/Functions/Tasks/functionsquickstartguidestop.htm] (OCI Functions Quickstart)

Sample Execution

echo '{"jobName":"My Job", "logGroupId":"ocid1.loggroup.oc1.iad....", "compartmentId":"ocid1.compartment.oc1.....", "projectId":"ocid1.datascienceproject.oc1.iad.....",  "noteBook":"https://raw.githubusercontent.com/tensorflow/docs/master/site/en/tutorials/customization/basics.ipynb", "outputFolder":"oci://bucket@namespace/helloworld"}' | fn invoke yourapp notebook

Orchestrating from OCI Data Integration

See [https://blogs.oracle.com/dataintegration/post/oci-rest-task-collection-for-oci-data-integration] (Invoking Data Science via REST Tasks)