Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Integrate variation-commons into eva-pipeline #150

Open
wants to merge 8 commits into
base: master
Choose a base branch
from
Open

Integrate variation-commons into eva-pipeline #150

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
56e7f0e
Replace Annotation Metadata
Mar 24, 2019
1474e08
Replace HgvsMongo with import from variation-commons-mongodb
Mar 24, 2019
2c861ef
Erase Xref and import XrefMongo instead
Mar 24, 2019
4f58bf1
Replace VariantAt with VariantAtMongo from commons-mongodb
Mar 24, 2019
daca45a
Make eva-pipeline use variation-commons' VariantAtMongo and VariantSo…
Mar 25, 2019
be45575
Use Annotation, ConsequenceType, and Score from variation-commons
Mar 25, 2019
5ebacd5
Make eva-pipeline use Simplified Annotation of variation-commons
Mar 28, 2019
c5e6bff
Minor README fix to correct markdown for Variant Brownser URL
Mar 28, 2019
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ The European Variation Archive pipeline processes Variant Call Format (VCF) file
* Variant annotation: consequence type, SIFT and Polyphen scores, etc
* Statistics: allele and genotype counts and frequencies

You can find a more detailed description of these operations in the [project wiki](https://github.com/EBIvariation/eva-pipeline/wiki/Jobs). Please visit the [EVA website](http://www.ebi.ac.uk/eva/?Variant Browser) to see a public service depending on this pipeline, and the [EVA web services repository](https://github.com/EBIvariation/eva-ws) for more information on the API.
You can find a more detailed description of these operations in the [project wiki](https://github.com/EBIvariation/eva-pipeline/wiki/Jobs). Please visit the [EVA website](http://www.ebi.ac.uk/eva/?Variant%20Browser) to see a public service depending on this pipeline, and the [EVA web services repository](https://github.com/EBIvariation/eva-ws) for more information on the API.

The pipeline automatically tracks the job status, and avoids waste of computation by resuming a job in the exact point where it failed; successful steps already executed are automatically skipped.

Expand Down
10 changes: 10 additions & 0 deletions pom.xml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,16 @@
</exclusion>
</exclusions>
</dependency>
Copy link
Member

@sundarvenkata-EBI sundarvenkata-EBI Apr 1, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Any reason why the DependencyManagement tag (referring to variation-commons) isn't included?

Copy link
Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I did not include the DependencyManagement tag because it results in test errors if I do so The test errors say they cannot find the configurations. This might possibly be because eva-pipeline's POM already has a parent POM - for reference: Spring Boot Dependency Management with a Custom Parent.

Also, Travis CLI build passes without the tag. So the other reason was to keep the eva-pipeline POM lean.

To test if I could add dependencyManagement, I added the following to eva-pipeline's POM, which results in test errors (because it is unable to find Spring configurations).
I'm new to Maven so I'm not 100% sure about my reasons. So please find the code used to include the tag:

    <dependencyManagement>
        <dependencies>
            <dependency>
                <groupId>uk.ac.ebi.eva</groupId>
                <artifactId>variation-commons</artifactId>
                <version>0.7-SNAPSHOT</version>
                <type>pom</type>
                <scope>import</scope>
            </dependency>
        </dependencies>
    </dependencyManagement>

Errors received were like so:

java.lang.NoSuchMethodError: org.springframework.util.ClassUtils.getQualifiedMethodName(Ljava/lang/reflect/Method;Ljava/lang/Class;)Ljava/lang/String;
        at org.springframework.transaction.interceptor.TransactionAspectSupport.methodIdentification(TransactionAspectSupport.java:396)

Copy link
Author

@108krohan 108krohan Apr 2, 2019 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi, please have a look at this comment thread.

My current understanding (something I haven't tried yet) is to include dependencies via a singular variation-commons.

So for example, only the following tag should suffice:

<dependency>
    <groupid>uk.ac.eba.eva</groupid>
    <artifactid>variation-commons</artifactid>
    <version>0.7-SNAPSHOT</version>
</dependency>

<dependency>
<groupId>uk.ac.ebi.eva</groupId>
<artifactId>variation-commons-mongodb</artifactId>
<version>0.7-SNAPSHOT</version>
</dependency>
<dependency>
<groupId>uk.ac.ebi.eva</groupId>
<artifactId>variation-commons-core</artifactId>
<version>0.7-SNAPSHOT</version>
</dependency>
</dependencies>

<build>
Expand Down
4 changes: 2 additions & 2 deletions src/main/java/uk/ac/ebi/eva/commons/models/data/Variant.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
package uk.ac.ebi.eva.commons.models.data;

import org.opencb.commons.utils.CryptoUtils;
import uk.ac.ebi.eva.commons.models.mongo.entity.Annotation;
import uk.ac.ebi.eva.commons.mongodb.entities.AnnotationMongo;

import java.util.HashMap;
import java.util.HashSet;
Expand Down Expand Up @@ -120,7 +120,7 @@ public enum VariantType {
/**
* Annotations of the genomic variation.
*/
private Set<Annotation> annotations;
private Set<AnnotationMongo> annotations;


public Variant() {
Expand Down
57 changes: 0 additions & 57 deletions src/main/java/uk/ac/ebi/eva/commons/models/metadata/AnnotationMetadata.java
View file
Open in desktop

This file was deleted.

203 changes: 0 additions & 203 deletions src/main/java/uk/ac/ebi/eva/commons/models/mongo/entity/Annotation.java
View file
Open in desktop

This file was deleted.

14 changes: 7 additions & 7 deletions src/main/java/uk/ac/ebi/eva/commons/models/mongo/entity/VariantDocument.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -21,10 +21,10 @@
import org.springframework.data.mongodb.core.mapping.Document;
import org.springframework.data.mongodb.core.mapping.Field;
import uk.ac.ebi.eva.commons.models.data.Variant;
import uk.ac.ebi.eva.commons.models.mongo.entity.subdocuments.HgvsMongo;
import uk.ac.ebi.eva.commons.mongodb.entities.subdocuments.HgvsMongo;
import uk.ac.ebi.eva.commons.models.mongo.entity.subdocuments.VariantAnnotation;
import uk.ac.ebi.eva.commons.models.mongo.entity.subdocuments.VariantAt;
import uk.ac.ebi.eva.commons.models.mongo.entity.subdocuments.VariantSourceEntryMongo;
import uk.ac.ebi.eva.commons.mongodb.entities.subdocuments.VariantAtMongo;
import uk.ac.ebi.eva.commons.mongodb.entities.subdocuments.VariantSourceEntryMongo;
import uk.ac.ebi.eva.commons.models.mongo.entity.subdocuments.VariantStatsMongo;

import java.util.HashSet;
Expand Down Expand Up @@ -92,7 +92,7 @@ public class VariantDocument {
private String alternate;

@Field(AT_FIELD)
private VariantAt at;
private VariantAtMongo at;

@Field(HGVS_FIELD)
private Set<HgvsMongo> hgvs;
Expand Down Expand Up @@ -184,13 +184,13 @@ public static String buildVariantId(String chromosome, int start, String referen
return builder.toString();
}

public static VariantAt generateAtField(String chromosome, int start) {
public static VariantAtMongo generateAtField(String chromosome, int start) {
int smallChunkId = start / VariantMongoDBWriter.CHUNK_SIZE_SMALL;
int bigChunkId = start / VariantMongoDBWriter.CHUNK_SIZE_BIG;
String chunkSmall = chromosome + "_" + smallChunkId + "_" + ONE_THOUSAND_STRING;
String chunkBig = chromosome + "_" + bigChunkId + "_" + TEN_THOUSAND_STRING;

return new VariantAt(chunkSmall, chunkBig);
return new VariantAtMongo(chunkSmall, chunkBig);
}

public static Set<HgvsMongo> createHgvsMongo(Map<String, Set<String>> hgvs) {
Expand Down Expand Up @@ -236,7 +236,7 @@ public String getAlternate() {
return alternate;
}

public VariantAt getAt() {
public VariantAtMongo getAt() {
return at;
}

Expand Down
Loading