Skip to content

Commit

Permalink
DOCKER-442 fix solr backup numberToLive
Browse files Browse the repository at this point in the history
* DOCKER-442 fix solr backup numberToLive

* DOCKER-442 add time to live integration test
  • Loading branch information
hechmi-dammak-xenit authored Nov 28, 2023
1 parent 151713e commit df7937b
Show file tree
Hide file tree
Showing 17 changed files with 341 additions and 194 deletions.
File renamed without changes.
41 changes: 29 additions & 12 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,10 +3,11 @@ on:
push:
workflow_dispatch:
env:
ORG_GRADLE_PROJECT_alfresco_nexus_username: ${{ secrets.ALFRESCO_NEXUS_USERNAME }}
ORG_GRADLE_PROJECT_alfresco_nexus_password: ${{ secrets.ALFRESCO_NEXUS_PASSWORD }}
GRADLE_OPTS: >-
-Dorg.gradle.project.org.alfresco.maven.nexus.username=${{ secrets.ALFRESCO_NEXUS_USERNAME }}
-Dorg.gradle.project.org.alfresco.maven.nexus.password=${{ secrets.ALFRESCO_NEXUS_PASSWORD }}
jobs:
integration-tests:
test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
Expand All @@ -15,24 +16,41 @@ jobs:
- uses: actions/setup-java@v1
with:
java-version: 11
- name: Login to Docker
run: |
echo "${{ secrets.CLOUDSMITH_APIKEY }}" | docker login private.docker.xenit.eu --username "${{ secrets.CLOUDSMITH_USER }}" --password-stdin
- name: Test
- name: Integration test
uses: gradle/gradle-build-action@v2
with:
cache-read-only: false
arguments: |
integration-tests:solr6:integrationTest
arguments: test -x :integration-tests:test
- name: Upload Test Artifact
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: test-result
path: /home/runner/work/**/build/reports
retention-days: 2
integration-test:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2
with:
fetch-depth: 0
- uses: actions/setup-java@v1
with:
java-version: 11
- name: Integration test
uses: gradle/gradle-build-action@v2
with:
cache-read-only: false
arguments: integration-tests:test
- name: Upload Test Artifact
if: success() || failure()
uses: actions/upload-artifact@v3
with:
name: integration-test-result
path: /home/runner/work/**/build/reports
retention-days: 2
publish:
needs: [ integration-tests ]
needs: [ test, integration-test ]
runs-on: ubuntu-latest
if: ${{ startsWith(github.ref, 'refs/tags/v') }}
steps:
Expand All @@ -53,5 +71,4 @@ jobs:
ORG_GRADLE_PROJECT_sonatype_password: ${{ secrets.SONATYPE_S01_PASSWORD }}
with:
cache-read-only: false
arguments: |
publish -PsigningKeyId=CDE3528F -i
arguments: publish -PsigningKeyId=CDE3528F -i
8 changes: 8 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,14 @@
title: Changelog - Solr Backup

# Alfresco Backup Changelog
## v0.0.9 - 28-11-2023

* DOCKER-442 fix solr backup numberToLive

## v0.0.8 - 20-11-2023

* DOCKER-441 improve solr backup documentation and add new env

## v0.0.5 - 3-11-2023

* OUPDAUNTLE-54 drop aws keys and use integrated env variables
Expand Down
42 changes: 33 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -25,20 +25,44 @@ Integration tests follow the same line:
* trigger a restore /solr/alfresco/replication?command=restore&repository=s3&location=s3:///
* check if the restore was successful in a certain timeout (3 minutes) by following the output of
/solr/alfresco/replication?command=restorestatus
## Setup

you need to put the solr.xml file under /opt/alfresco-search-services/solrhome/
```
<?xml version='1.0' encoding='UTF-8'?>
<solr>
<str name="adminHandler">${adminHandler:org.alfresco.solr.AlfrescoCoreAdminHandler}</str>
<backup>
<repository name="s3" class="eu.xenit.solr.backup.s3.S3BackupRepository" default="false">
<str name="s3.bucket.name">${S3_BUCKET_NAME:}</str>
<str name="s3.endpoint">${S3_ENDPOINT:http://s3.eu-central-1.amazonaws.com}</str>
<str name="s3.region">${S3_REGION:eu-central-1}</str>
<str name="s3.access.key">${S3_ACCESS_KEY:}</str>
<str name="s3.secret.key">${S3_SECRET_KEY:}</str>
<str name="s3.proxy.host">${S3_PROXY_HOST:}</str>
<int name="s3.proxy.port">${S3_PROXY_PORT:0}</int>
<bool name="s3.path.style.access.enabled">${S3_PATH_STYLE_ACCESS_ENABLED:false}</bool>
</repository>
</backup>
</solr>
```

and specify the Environment or Java variables

## Variables

all of these variable can be set as environment variable or as a system property so that it is substituted in solr.xml

| Environment variable | Java system property | Default | required |
|----------------------|----------------------|--------------------------------------|----------|
| S3_ENDPOINT | -DS3_ENDPOINT | http://s3.eu-central-1.amazonaws.com | false |
| S3_BUCKET_NAME | -DS3_BUCKET_NAME | | true |
| S3_REGION | -DS3_REGION | eu-central-1 | false |
| S3_ACCESS_KEY | -DS3_ACCESS_KEY | | false |
| S3_SECRET_KEY | -DS3_SECRET_KEY | | false |
| S3_PROXY_HOST | -DS3_PROXY_HOST | | false |
| S3_PROXY_PORT | -DS3_PROXY_PORT | | false |
| Environment variable | Java system property | Default | required |
|------------------------------|--------------------------------|--------------------------------------|----------|
| S3_ENDPOINT | -DS3_ENDPOINT | http://s3.eu-central-1.amazonaws.com | false |
| S3_BUCKET_NAME | -DS3_BUCKET_NAME | | true |
| S3_REGION | -DS3_REGION | eu-central-1 | false |
| S3_ACCESS_KEY | -DS3_ACCESS_KEY | | false |
| S3_SECRET_KEY | -DS3_SECRET_KEY | | false |
| S3_PROXY_HOST | -DS3_PROXY_HOST | | false |
| S3_PROXY_PORT | -DS3_PROXY_PORT | | false |
| S3_PATH_STYLE_ACCESS_ENABLED | -DS3_PATH_STYLE_ACCESS_ENABLED | false | false |

## Testing against DataCore Swarm docker

Expand Down
11 changes: 0 additions & 11 deletions build.gradle
Original file line number Diff line number Diff line change
Expand Up @@ -3,22 +3,11 @@ plugins {
id 'eu.xenit.docker-compose' version '5.4.0' apply false
}

def copyPropertyValueIfExists(sourcePropertyName, targetPropertyName) {
if (project.hasProperty(sourcePropertyName)) {
project.ext[targetPropertyName] = project.property(sourcePropertyName)
}
}

subprojects {
apply plugin: 'java'
def baseVersion = System.getenv("TAG_VERSION") ?: 'v0.0.6'
version = baseVersion[1..baseVersion.length() - 1]

// It is not possible to set properties with a dot via GitHub Actions env variables, therefore we introduce support
// for a non-dotted-equivalent
copyPropertyValueIfExists('alfresco_nexus_username', 'org.alfresco.maven.nexus.username')
copyPropertyValueIfExists('alfresco_nexus_password', 'org.alfresco.maven.nexus.password')

repositories {
mavenCentral()
maven {
Expand Down
8 changes: 8 additions & 0 deletions gradle.properties
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
junitJupiterVersion=5.4.2
mockitoVersion=2.27.0
solrVersion=6.6.5
assVersion=2.0.8.2
amazonVersion=1.12.32
jaxBVersion=2.3.2
restAssuredVersion=4.0.0
awaitablityVersion=4.2.0
102 changes: 44 additions & 58 deletions integration-tests/build.gradle
Original file line number Diff line number Diff line change
@@ -1,74 +1,60 @@
plugins {
id 'java'
id 'idea'
apply plugin: 'java'
apply plugin: 'idea'
apply plugin: 'eu.xenit.docker'
apply plugin: 'eu.xenit.docker-compose.auto'

ext {
solrVersion = '6.6.5'
assVersion = '2.0.6'
solrBaseImage = 'docker.io/xenit/alfresco-solr6-xenit:2.0.6'
alfrescoimage = 'docker.io/xenit/alfresco-repository-community:7.3.0'
flavor = 'solr6'
}

dependencies {
testImplementation group: 'io.rest-assured', name: 'rest-assured', version: '3.0.1'
testImplementation group: 'io.rest-assured', name: 'json-path', version: '3.0.1'
testImplementation group: 'io.rest-assured', name: 'rest-assured-common', version: '3.0.1'
testImplementation "org.junit.jupiter:junit-jupiter-engine:5.4.2"
testImplementation "org.junit.jupiter:junit-jupiter-params:5.4.2"

testRuntimeOnly group: 'org.glassfish.jaxb', name: 'jaxb-runtime', version: '2.3.2'
}
description = "Solr ${flavor} with backup"

test {
enabled = false
useJUnitPlatform()
testLogging {
events "passed", "skipped", "failed"
}
configurations {
backupJar
}

subprojects {
apply plugin: 'java'
apply plugin: 'eu.xenit.docker'
apply plugin: 'eu.xenit.docker-compose.auto'
apply from: "${project.projectDir}/overload.gradle"

description = "Solr ${flavor} with backup"

configurations {
amazonSdkCore
amazonSdkS3
backupJar
}

dependencies {
amazonSdkCore group: 'com.amazonaws', name: 'aws-java-sdk-core', version: '1.12.32', ext: 'jar'
amazonSdkS3 group: 'com.amazonaws', name: 'aws-java-sdk-s3', version: '1.12.32', ext: 'jar'
dependencies {
backupJar project(path: ":solr-backup")

backupJar project(path: ":solr-backup")
}
testImplementation "io.rest-assured:rest-assured:${restAssuredVersion}"
testImplementation "io.rest-assured:json-path:${restAssuredVersion}"
testImplementation "io.rest-assured:rest-assured-common:${restAssuredVersion}"
testImplementation "org.junit.jupiter:junit-jupiter-engine:${junitJupiterVersion}"
testImplementation "org.junit.jupiter:junit-jupiter-params:${junitJupiterVersion}"
testImplementation "org.awaitility:awaitility:${awaitablityVersion}"

testImplementation platform("com.amazonaws:aws-java-sdk-bom:${amazonVersion}")

task integrationTest(type: Test, group: "verification") {
useJUnitPlatform()
testClassesDirs = project.parent.sourceSets.test.output.classesDirs
classpath = project.parent.sourceSets.test.runtimeClasspath
outputs.upToDateWhen { false }
}
testImplementation('com.amazonaws:aws-java-sdk-core')
testImplementation('com.amazonaws:aws-java-sdk-s3')
testImplementation("com.amazonaws:aws-java-sdk-sts")

testRuntimeOnly "org.glassfish.jaxb:jaxb-runtime:${jaxBVersion}"
}
test {
useJUnitPlatform()
}

createDockerFile {
from "${solrBaseImage}"
createDockerFile {
from "${solrBaseImage}"

dependsOn(configurations.backupJar)
dependsOn(configurations.backupJar)

if (flavor == "solr6") {
smartCopy "${project.parent.projectDir}/src/test/resources/solr.xml", "/opt/alfresco-search-services/solrhome/solr.xml"
smartCopy configurations.backupJar, "/opt/alfresco-search-services/solrhome/lib/"
smartCopy configurations.amazonSdkCore.singleFile, "/opt/alfresco-search-services/solrhome/lib/"
smartCopy configurations.amazonSdkS3.singleFile, "/opt/alfresco-search-services/solrhome/lib/"
}
if (flavor == "solr6") {
smartCopy "${project.projectDir}/src/test/resources/solr.xml", "/opt/alfresco-search-services/solrhome/solr.xml"
smartCopy configurations.backupJar, "/opt/alfresco-search-services/solrhome/lib/"
}
}

dockerCompose {
environment.put 'ALFRESCO_IMAGE', project.alfrescoimage
useComposeFiles = [
"${project.parent.projectDir}/src/test/resources/compose/docker-compose.yml"
]
isRequiredBy(project.tasks.integrationTest)
}
dockerCompose {
environment.put 'ALFRESCO_IMAGE', project.alfrescoimage
useComposeFiles = [
"${project.projectDir}/src/test/resources/compose/docker-compose.yml"
]
isRequiredBy(project.tasks.test)
}
7 changes: 0 additions & 7 deletions integration-tests/solr6/overload.gradle

This file was deleted.

Loading

0 comments on commit df7937b

Please sign in to comment.