Docs Ubuntu Upgrade

[ghost]

This PR updates the docs server to Ubuntu 22.04 #381.
It also resolves ES SSL configuration issues #396 and #387, I have achieved this by moving ES behind Apache.

Questions

1. From what I could see we are using one ES index ocdsindex_en? if we are using more I will need to reconfigure the proxy configuration as it is limiting access to one index.
2. What is the best way to populate ElasticSearch?
3. We need to update future docs deployments for the new ES URL (https://standard.open-contracting.org/ocdsindex_en/_search?size=100)

Go live plan

• Merge and deploy
• Copy /home/ocds-docs/web directory

rsync -avz ocp07:/home/ocds-docs/web/ /home/ocds-docs/web/

• Copy ElasticSearch data

# Delete data on ocp19 before syncing.
rsync -avz ocp07:/var/lib/elasticsearch/ /var/lib/elasticsearch/

• Update site search.js for new URL

find /home/ocds-docs/web/ -name search.js -exec sed -i "s/standard.open-contracting.org:9200/standard.open-contracting.org\/search/g" {} \+

• Update DNS record for standard.open-contracting.org

Site Specific changes - With James

• Commit change of URL to standard_theme repository
• Update each documentation sub-site to the latest standard_profile_template (if we are upgrading ES)
• Push changes, including to deployment branches (1.0, 1.1, etc.)

[jpmckinney]

From what I could see we are using one ES index ocdsindex_en? if we are using more I will need to reconfigure the proxy configuration as it is limiting access to one index.

There is one per language (ocdsindex_es, etc.).

What is the best way to populate ElasticSearch?

I think I've used https://github.com/elasticsearch-dump/elasticsearch-dump in the past.

We need to update future docs deployments for the new ES URL (https://standard.open-contracting.org/ocdsindex_en/_search?size=100)

Does that mean search will not work until docs are re-deployed? I guess this could involve #229

[ghost]

@jpmckinney Did you want to take this opportunity to update Elasticsearch to version 8?

[jpmckinney]

@jpmckinney Did you want to take this opportunity to update Elasticsearch to version 8?

I suppose we should. Support for 7 ends in August https://endoflife.date/elasticsearch

[ghost]

From what I could see we are using one ES index ocdsindex_en? if we are using more I will need to reconfigure the proxy configuration as it is limiting access to one index.

There is one per language (ocdsindex_es, etc.).

I have modified Elasticsearch and Apache to host the search under /search/, now all ES endpoints work as expected. :)
@jpmckinney , please can you check this works with ocds-index and any other tools.

You can test the new hosting by setting the following in your hosts file:

178.79.135.174  standard.open-contracting.org

What is the best way to populate ElasticSearch?

I have found a simple solution using rsync and recorded it in my original comment.
We may need to add other steps if we are upgrading ES along side this.

We need to update future docs deployments for the new ES URL (https://standard.open-contracting.org/ocdsindex_en/_search?size=100)

Does that mean search will not work until docs are re-deployed? I guess this could involve #229

I have put a find-replace command in my original comment to update search.js file as part of the migration. I believe this covers everything?
I'd like to leave issue #229 out of scope to help keep this PR concise.

[jpmckinney]

please can you check this works with ocds-index and any other tools.

The current version of ocds-index will work as-is.

I've authored a new version that works with Elasticsearch 8. Are we upgrading Elasticsearch as part of this PR?

I have put a find-replace command in my original comment to update search.js file as part of the migration

This is not enough, because the next time we generate the documentation, we will get the port again.

The steps would be for me to:

• Commit change of URL to standard_theme repository
• Update each documentation sub-site to the latest standard_profile_template (if we are upgrading ES)
• Push changes, including to deployment branches (1.0, 1.1, etc.)

[ghost]

Are we upgrading Elasticsearch as part of this PR?

I will test upgrading ES, if it is simple enough then let's add it into this PR.

[ghost]

@jpmckinney I think this PR is ready merge from my side :)
We will need to coordinate deploying so you can make the site changes after the new server goes live.