Skip to content

Commit

Permalink
create connection with wrong settings
Browse files Browse the repository at this point in the history
  • Loading branch information
elliotzh committed Oct 10, 2023
1 parent cc9e20b commit f91e6d8
Show file tree
Hide file tree
Showing 6 changed files with 43 additions and 10 deletions.
11 changes: 11 additions & 0 deletions .github/workflows/bootstrapping-resources.yml
Original file line number Diff line number Diff line change
Expand Up @@ -162,6 +162,17 @@ jobs:
working-directory: cli/jobs/pipelines-with-components/basics/1b_e2e_registered_components
timeout-minutes: 30
continue-on-error: true
- name: create asset for cli/jobs/pipelines-with-components/pipeline_job_with_flow_as_component
if: ${{ always() }}
run: |
source "${{ github.workspace }}/infra/bootstrapping/sdk_helpers.sh";
source "${{ github.workspace }}/infra/bootstrapping/init_environment.sh";
set -x
pip install promptflow
pf connection create -f connection.yaml --set api_key="${{ github.workspace }}" --set api_base="${{ github.workspace }}"
working-directory: cli/jobs/pipelines-with-components/basics/pipeline_job_with_flow_as_component
timeout-minutes: 30
continue-on-error: true
- name: setup env for cli/jobs/pipelines-with-components/rai_pipeline_adult_analyse/
if: ${{ always() }}
run: |
Expand Down
Original file line number Diff line number Diff line change
@@ -1,8 +1,11 @@
This is a dummy pipeline job with anonymous reference of a flow as a component. Flow directory is copied from [sample in promptflow repository](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/web-classification).

Note that `.promptflow/flow.tools.json` in the flow directory is required to use a flow as a component. Usually you can use `pf flow validate` or `pf run validate` to generate it.
Prerequirements:
1. `.promptflow/flow.tools.json` in the flow directory is required to use a flow as a component. Usually you can use `pf flow validate` or `pf run validate` to generate it.
2. You should either update connection name in `flow.dag.yaml` or update `connection.yaml` with your own api information and use `pf connection create --file connection.yaml` to create a workspace connection.
3. You need to either edit the compute cluster in `pipeline.yml` or create a compute cluster named `cpu-cluster` in your workspace.

After that, You need to edit the compute cluster in the defaults section and run `az ml job create --file pipeline.yml` to submit the pipeline job.
After that, you can run `az ml job create --file pipeline.yml` to submit the pipeline job.

References:
- [microsoft/promptflow: Build high-quality LLM apps](https://github.com/microsoft/promptflow)
Expand Down
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
$schema: https://azuremlschemas.azureedge.net/promptflow/latest/AzureOpenAIConnection.schema.json
name: azure_open_ai_connection
type: azure_open_ai # snake case
api_key: "<to-be-replaced>"
api_base: "aoai-api-endpoint"
api_type: "azure"
api_version: "2023-07-01-preview"
2 changes: 0 additions & 2 deletions sdk/python/jobs/pipelines/1l_flow_in_pipeline/README.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,5 @@
This is a dummy pipeline job with anonymous reference of a flow as a component. This example has reused the flow in corresponding CLI example, which is copied from [sample in promptflow repository](https://github.com/microsoft/promptflow/tree/main/examples/flows/standard/web-classification). Please check [this path](../../../../../cli/jobs/pipelines-with-components/pipeline_job_with_flow_as_component/) for dependent resources.

Note that `.promptflow/flow.tools.json` in the flow directory is required to use a flow as a component. Usually you can use `pf flow validate` or `pf run validate` to generate it.

References:
- [microsoft/promptflow: Build high-quality LLM apps](https://github.com/microsoft/promptflow)
- [Reference - Prompt flow docuentation](https://microsoft.github.io/promptflow/reference/index.html)
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"# Use Flow as Component in Pipeline\n",
"# Use Flow as Component in Pipeline Job\n",
"\n",
"**Requirements** - In order to benefit from this tutorial, you will need:\n",
"- A basic understanding of Machine Learning\n",
Expand Down Expand Up @@ -79,7 +79,9 @@
"source": [
"## 1.3 Get a handle to the workspace\n",
"\n",
"We use config file to connect to a workspace. The Azure ML workspace should be configured with computer cluster. [Check this notebook for configure a workspace](../../configuration.ipynb)"
"We use config file to connect to a workspace. The Azure ML workspace should be configured with computer cluster. [Check this notebook for configure a workspace](../../configuration.ipynb).\n",
"\n",
"As we'll use flow with connection, the workspace should also be configured with a workspace connection. [Check this notebook for configure a connection](../../../resources/connections/connections.ipynb)"
]
},
{
Expand All @@ -92,8 +94,13 @@
"ml_client = MLClient.from_config(credential=credential)\n",
"\n",
"# Retrieve an already attached Azure Machine Learning Compute.\n",
"cluster_name = \"gpu-cluster\"\n",
"print(ml_client.compute.get(cluster_name))"
"cluster_name = \"cpu-cluster\"\n",
"print(ml_client.compute.get(cluster_name))\n",
"\n",
"# note that you'll need to install package promptflow to check connections, but it's not required if you want to submit the pipeline job only\n",
"from promptflow import PFClient\n",
"pf = PFClient(ml_client)\n",
"print(pf.connections.get(\"azure_open_ai_connection\"))"
]
},
{
Expand All @@ -102,9 +109,12 @@
"source": [
"# 2. Load flow as component\n",
"\n",
"We suppose that there has already been a flow authored with Promptflow SDK/CLI/portal. Then we can load its flow dag yaml as a component like regular component specs.\n",
"We suppose that there has already been a flow authored with Promptflow SDK/CLI/portal and `.promptflow/flow.tools.json` is already generated in the flow directory:\n",
"\n",
"- For more information about how to author a flow, please check [official doc site](https://microsoft.github.io/promptflow/).\n",
"- If `.promptflow/flow.tools.json` is not generated yet or is not up-to-date, you may use `pf flow validate` or `pf run validate` to generate it - [reference doc](https://microsoft.github.io/promptflow/reference/pf-command-reference.html).\n",
"\n",
"Here we reuse [the flow definition yaml in CLI examples](../../../../../cli/jobs/pipelines-with-components/pipeline_job_with_flow_as_component/web_classification/flow.dag.yaml)."
"Then we can load its flow dag yaml as a component like regular component specs. Here we reused [the flow definition yaml in CLI examples](../../../../../cli/jobs/pipelines-with-components/pipeline_job_with_flow_as_component/web_classification/flow.dag.yaml)."
]
},
{
Expand Down
4 changes: 4 additions & 0 deletions sdk/python/setup.sh
Original file line number Diff line number Diff line change
Expand Up @@ -20,4 +20,8 @@ pip install azure-ai-ml
# https://docsupport.blob.core.windows.net/ml-sample-submissions/1905732/azure_ai_ml-1.0.0-py3-none-any.whl
# </az_ml_sdk_test_install>

# <connection_manage_install>
pip install promptflow
# </connection_manage_install>

pip list

0 comments on commit f91e6d8

Please sign in to comment.