Sensor Agent Doc Integration Example

[Future-Outlier]

TL;DR

Enable users to use the sensor agent.
Senso Index: https://flyte--1180.org.readthedocs.build/projects/cookbook/en/1180/auto_examples/sensor/index.html
How to use sensor?: https://flyte--1180.org.readthedocs.build/projects/cookbook/en/1180/auto_examples/sensor/sensor.html

Tracking issue

#1173
flyteorg/flyte#4195
flyteorg/flyte#4235
flyteorg/flyte#3936

Screenshots

[pingsutw]

why do we need two sensor folder. /sensor/sensor/...? can we update the file structure here, update it to

- sensor
| _ file_sensor_example.py
| _ http_sensor_example.py
| _ etc...

[neverett]

I think all the other examples are like this, and have a README and Dockerfile in the top-level directory, and an __init__.py and example (or examples) in the enclosed directory. This PR appears to be missing a Dockerfile.

[neverett]

@Future-Outlier I think you will need to create some missing files so your example matches the directory structure shown in the contribution guide. Specifically, I think you need to add a Dockerfile, requirements.txt, and requirements.in file.

@pingsutw does that seem right to you?

[pingsutw]

@neverett mind taking a look when you get a chance, thanks

[neverett]

I think the information looks good, and I suggested some ways to reorganize the content so it flows a bit better. Let me know if you have any questions!

[neverett]

I recommend "First" instead of "To begin"

[neverett]

I recommend changing this to "Next, create a FileSensor task" and moving the next code block directly under this sentence.

[neverett]

I recommend changing this to "To use the FileSensor created in the previous step, you must specify the path parameter. In the sandbox, you can use the S3 path" and moving the last code block directly below it.

[neverett]

I recommend moving this section the very bottom and putting it in a note, like so:

# % [markdown]
# ::{note}
# We have already set the minio credentials in the agent by default. If you test the sandbox example locally, you will need to set the AWS credentials in your environment variables.
# ```{prompt} bash
# export FLYTE_AWS_ENDPOINT="http://localhost:30002"
# export FLYTE_AWS_ACCESS_KEY_ID="minio"
# export FLYTE_AWS_SECRET_ACCESS_KEY="miniostorage"
# :::
# %%

[neverett]

Note that you may have to tweak the formatting to get the code snippet to appear correctly in the note -- I just started, and am not entirely familiar with how these docs are formatted.

[Future-Outlier]

https://flyte--1180.org.readthedocs.build/projects/cookbook/en/1180/auto_examples/sensor/file_sensor_example.html

[Future-Outlier]

@neverett Please review again! Thanks really much!

[neverett]

@Future-Outlier the documentation looks good to me! I'll leave it to @pingsutw for final approval.

[pingsutw]

LGTM, thank you @Future-Outlier @neverett

[Future-Outlier]

@neverett Can you help us merge this PR?
Also, can you help us remove the request changed?
This can help us auto-merge this PR, thank you really much!

[neverett]

@Future-Outlier I just approved this PR, so it should be able to be merged now!

[Future-Outlier]

@neverett Do you know how to solve the sphinx timeout error?
I think the error is not related to the sensor doc.
Thanks a lot!