change the partial members of genomix-data, genomix-hyracks for adapting ray-style #60
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
added 9 commits
November 9, 2013 17:10
…/ray-genomix # By Jake Biesinger # Via Jake Biesinger * commit '1b7005d5a2f185c29ea677f4246a4889d805fa36': remove garbage GAGETEST.fasta fix scaffolding bug where incomingMsg details weren't included explicitly set the length to 0 at the beginning of BFS fix BubbleMerge bug where an incorrect major/minor to bubble dir was used use null for default values in all pregelix messages stopgap fix for pathmerge problem introduced by #51 in commit 447146c Conflicts: genomix/genomix-pregelix/src/main/java/edu/uci/ics/genomix/pregelix/io/message/BubbleMergeMessage.java genomix/genomix-pregelix/src/main/java/edu/uci/ics/genomix/pregelix/io/message/PathMergeMessage.java
| outputEdge += outputNode + " -> " + e.getKey().toString() + "[color = \"" + getColor(et) | ||
| + "\" label =\"" + et + ": " + e.getValue() + "\"]\n"; | ||
| outputEdge += outputNode + " -> " + e.toString() + "[color = \"" + getColor(et) | ||
| + "\" label =\"" + et + "\"]\n"; |
There was a problem hiding this comment.
@anbangx since there are no readids on the edges in this version, we'll have to reconsider what DIRECTED_GRAPH_WITH_ALLDETAILS now means. Perhaps it should print the internalKmer instead of the key?
There was a problem hiding this comment.
@jakebiesinger WITH_ALLDETAILS includes all the other fields inside the node, this key is only for edge connection.
|
I want to see those test cases back in place and I think we're probably going to need an optimized representation for these kmer lists (like a sorted list backed by a byte array for all the kmers). Also, I think the mate0 vs mate1 isn't the right way to think about this... Should be this vs other and single-end reads should work without any change to the interface. |
added 3 commits
November 13, 2013 10:33
# By buyingyi (16) and others # Via [email protected] (8) and others * genomix/fullstack_genomix: (41 commits) update to new pregelix aggregator interface fix application lifecyle mgmt in hyracks nc fix the pinned page issue during a node failure Make sure the validty bit in the metadata page is flushed to disk when marking a component to be valid. NodeControllers clean up appEntryPoints on shutdown (2nd try) fix an issue found by Sattam Fix for issue 127. minor fix for heartbeat state population support multiple user-defined global aggregators reverted the change of removing adjacent exchange operators updated hivestrix test case for running aggregation fix Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed. Fixed the incorrect exchange merging introduced by the previous commit; updated the IntroHashPartitionMergeExchange rule to handle the hash-merge-exchange operator. Fixed a bug on omitted order by columns when added an exchange operator to enforce the group-by property. Fixed a bug on unclosed running aggregation runtime; fixed an issue on two adjacent exchange operators (connectors) when duplicate sort operator is removed. revert a minor change revert a minor change fix IIndexAccessor interface, add a boolean exclusiveMode parameter for the createSearchCursor method fix file write race condition Revert changes to InlineVariablesRule. ...
…ng nothing about null members
|
Hey @Nan-Zhang the KMP stuff is cool but really belongs in a different branch. It's a totally separate feature! Could you fix up this branch based on the comments here and submit a separate PR for the KMP stuff? |
|
Also as a general comment for everyone, your commits should rarely cause syntax errors. Syntax errors will cause any future |
…ing/ray-genomix-refactor Conflicts: genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/pathmerge/BasicPathMergeVertex.java
…wbiesing/ray-genomix-refactor Conflicts: genomix/genomix- data/src/main/java/edu/uci/ics/genomix/config/GenomixJobConf.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/minicluster/GenerateGraphViz.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/type/EDGETYPE.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/type/EdgeMap.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/type/Node.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/type/ReadHeadSet.java genomix/genomix- data/src/main/java/edu/uci/ics/genomix/type/VKmer.java genomix/genomix- data/src/test/java/edu/uci/ics/genomix/type/EdgeMapTest.java genomix/genomix- data/src/test/java/edu/uci/ics/genomix/type/NodeTest.java genomix/genomix- data/src/test/java/edu/uci/ics/genomix/type/ReadIdSetTest.java genomix/genomix- data/src/test/java/edu/uci/ics/genomix/type/VKmerTest.java genomix/genomix- hadoop/src/main/java/edu/uci/ics/genomix/hadoop/contrailgraphbuilding/GenomixReducer.java genomix/genomix- hadoop/src/main/java/edu/uci/ics/genomix/hadoop/graph/GraphStatistics.java genomix/genomix- hyracks/src/main/java/edu/uci/ics/genomix/hyracks/graph/dataflow/AggregateKmerAggregateFactory.java genomix/genomix- hyracks/src/main/java/edu/uci/ics/genomix/hyracks/graph/dataflow/ReadsKeyValueParserFactory.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/DeBruijnGraphCleanVertex.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/bubblemerge/SimpleBubbleMergeVertex.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/pathmerge/BasicPathMergeVertex.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/splitrepeat/SplitRepeatVertex.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/operator/unrolltandemrepeat/UnrollTandemRepeat.java genomix/genomix- pregelix/src/main/java/edu/uci/ics/genomix/pregelix/testhelper/BubbleAddVertex.java
…factor Conflicts: genomix/genomix- pregelix/src/test/java/edu/uci/ics/genomix/pregelix/jobgen/JobGenerator.java
Refactor on top of Nan's changes to remove syntax errors
|
This PR was superseded by #71, which has now been merged in. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
@jakebiesinger @JavierJia @anbangx please give some feedback. I have changed some partial structure in genomix-data, and genomix-hyracks for adapting to ray-style. tests are not been completed but in works.