Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AWS SUPPORT] Add Amazon OpenSearch support #758

Open
dawidcx3 opened this issue Jul 10, 2024 · 0 comments
Open

[AWS SUPPORT] Add Amazon OpenSearch support #758

dawidcx3 opened this issue Jul 10, 2024 · 0 comments

Comments

@dawidcx3
Copy link

Currently application uses Elasticsearch service with cloud_id and api_key requirement as here: https://elasticsearch-py.readthedocs.io/en/v8.14.0/api/elasticsearch.html.
Current connection can be found here: https://github.com/i-dot-ai/redbox/blob/main/redbox-core/redbox/models/settings.py#L211

AWS does not support Elasticsearch anymore and it uses Amazon OpenSearch instead. We prepared and tested a code that creates a connection with Amazon OpenSearch on AWS platform:

from opensearchpy import OpenSearch, AWSV4SignerAuth, RequestsHttpConnection
import boto3

credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, "eu-west-2")

host = 'vpc-redbox-dev-tggskxn42wlh7czdzdqvuuolfy.eu-west-2.es.amazonaws.com'
port = 443

client = OpenSearch(
  hosts = [{'host': host, 'port': port}],
  http_compress = True,
  use_ssl = True,
  http_auth = auth,
  verify_certs = True,
  connection_class = RequestsHttpConnection
)

info = client.info()
print(f"Welcome to {info['version']['distribution']} {info['version']['number']}!")

More details can be found in OpenSearch documentation: https://opensearch.org/docs/latest/clients/python-low-level/

Our recommended way of authentication is to use IAM Role prepared from infrastructure side. To use it is required to add the following lines:

credentials = boto3.Session().get_credentials()
auth = AWSV4SignerAuth(credentials, "eu-west-2")

Environment variables to set hostname and port are also required as well.

@dawidcx3 dawidcx3 changed the title [AWS] Add Amazon OpenSearch support [AWS SUPPORT] Add Amazon OpenSearch support Jul 11, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant