Skip to content

Commit

Permalink
documented
Browse files Browse the repository at this point in the history
  • Loading branch information
kurzum committed Sep 6, 2019
1 parent 1cfa09c commit d667cc4
Showing 1 changed file with 18 additions and 9 deletions.
27 changes: 18 additions & 9 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -16,13 +16,13 @@ This contribution by TIB is a great push towards incentivizing Open Data and est
## Downloading the wikimedia dumps
TODO

## Running the extraction
## Update and Run the extraction
TODO

## Deploy on Databus
## Deploy MARVIN on Databus
TODO

## Run Databus-Derive (clone and parse)
## [Manual] Run Databus-Derive (clone and parse)
On the respective server there is a user marvin-fetch, that has access to `/data/derive` containing the pom.xml of https://github.com/dbpedia/databus-maven-plugin/tree/master/dbpedia

```
Expand All @@ -37,28 +37,37 @@ SELECT distinct (?derive) WHERE {
BIND (CONCAT("<version>",?artifact,"/${databus.deriveversion}</version>") as ?derive)
}
order by asc(?derive)
```


```
#######
# This is still manual, will be a cronjob soon
#######
su marvin-fetch
tmux a -t derive

WHAT=mappings
NEWVERSION=2019.08.30
# prepare
cd /data/derive/databus-maven-plugin/dbpedia/$WHAT
git pull
mvn versions:set -DnewVersion=$NEWVERSION
# run
mvn -T 23 databus-derive:clone -Ddatabus.deriveversion=$NEWVERSION
mvn databus-derive:clone -Ddatabus.deriveversion=$NEWVERSION
```

## Move data to download server (internal)
## [Manual] pull data to downloads.dbpedia.org server
run marvin-fetch.sh script in databus/dbpedia folder

```
cd /media/bigone/25TB/releases/databus-maven-plugin/dbpedia
./marvin-fetch.sh wikidata 2019.08.01
```

## Deploy official files

```
cd /media/bigone/25TB/releases/databus-maven-plugin/dbpedia/mappings
mvn clean
mvn validate
mvn -T 8 deploy
```

0 comments on commit d667cc4

Please sign in to comment.