Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Some snapshots from doing testing before merging revision01 to master #20

Open
claczny opened this issue Dec 10, 2014 · 4 comments
Open

Some snapshots from doing testing before merging revision01 to master #20

claczny opened this issue Dec 10, 2014 · 4 comments
Labels
wontfix

Comments

@claczny
Copy link
Owner

claczny commented Dec 10, 2014

Below will be some snapshots/ideas that I found to be important to note down before the merge.
@claczny
Copy link
Owner Author

claczny commented Dec 10, 2014

From Windows console output:

C:\Users\Jennifer>"C:\Program Files (x86)\Java\jre7\bin\java.exe" -jar -Xmx1g C:
\Users\Jennifer\Desktop\VizBin\VizBin-dist.jar
2014-12-10 10:51:04,320 DEBUG [main] (MainFrame.java:61) - Init of Main applicat
ion frame
2014-12-10 10:51:28,750 DEBUG [AWT-EventQueue-0] (ProcessInput.java:61) - Init o
f ProcessInput
2014-12-10 10:51:28,765 DEBUG [Thread-3] (ProcessInput.java:115) - C:\Users\Jenn
ifer\AppData\Local\Temp\
2014-12-10 10:51:29,062 DEBUG [Thread-3] (ProcessInput.java:124) - Loading data
from file.
Contig length treshold: 1000
2014-12-10 10:51:29,062 DEBUG [Thread-3] (ProcessInput.java:91) - Loading fasta
file: C:\Users\Jennifer\Desktop\VizBin\37A\37A\37A.concatenated.fa
2014-12-10 10:51:34,740 DEBUG [Thread-3] (ProcessInput.java:91) - DataSet loaded
 (13042 sequences)
2014-12-10 10:51:34,740 DEBUG [Thread-3] (ProcessInput.java:91) - Creating kmers
 (k=5, merge = true)
2014-12-10 10:51:34,756 DEBUG [Thread-3] (DataSetUtils.java:65) - Creating usedV
al for k=5
2014-12-10 10:51:41,604 DEBUG [Thread-3] (ProcessInput.java:91) - Normalizing ve
ctors...
2014-12-10 10:51:41,869 DEBUG [Thread-3] (ProcessInput.java:91) - Clr normalizat
ion...
2014-12-10 10:51:42,805 DEBUG [Thread-3] (ProcessInput.java:91) - Running PCA...
 (Mtj)
Dez 10, 2014 10:51:46 AM com.github.fommil.netlib.LAPACK <clinit>
Warnung: Failed to load implementation from: com.github.fommil.netlib.NativeSyst
emLAPACK
Dez 10, 2014 10:51:47 AM com.github.fommil.jni.JniLoader liberalLoad
Information: successfully loaded C:\Users\Jennifer\AppData\Local\Temp\jniloader2
365430519185301729netlib-native_ref-win-i686.dll
2014-12-10 10:52:21,587 DEBUG [Thread-3] (DataSetUtils.java:249) - DONE: Compute
d the new basis.
2014-12-10 10:52:42,085 DEBUG [Thread-3] (DataSetUtils.java:256) - DONE: Project
ed from sample to eigen space.
2014-12-10 10:52:42,085 DEBUG [Thread-3] (ProcessInput.java:91) - Running T-SNE.
..
2014-12-10 10:52:45,049 DEBUG [Thread-3] (DataSetUtils.java:264) - Running comma
nd: "C:\Users\Jennifer\.vizbin\pbh_tsne.exe" in directory: C:\Users\Jennifer\App
Data\Local\Temp\map7054369329860871950

Number of threads: 1
Seed: 0
2014-12-10 10:52:47,093 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Read
the 13042 x 50 data matrix successfully!
2014-12-10 10:52:47,093 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Runni
ng BH_TSNE with 1 threads and seed = 0
2014-12-10 10:52:47,093 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Using
 no_dims = 2, perplexity = 30.000000, theta = 0.500000

@claczny
Copy link
Owner Author

claczny commented Dec 10, 2014

From Yohan's console output (OSX Yosemite10.10.1 / Retina):

2014-12-10 10:43:28,928 DEBUG [main] (MainFrame.java:61) - Init of Main application frame
2014-12-10 10:43:43,777 DEBUG [AWT-EventQueue-0] (ProcessInput.java:61) - Init of ProcessInput
2014-12-10 10:43:43,780 DEBUG [Thread-1] (ProcessInput.java:115) - /var/folders/gn/p47pzlx16v7bnwnwv4b10rv9skqz7l/T/
2014-12-10 10:43:43,794 DEBUG [Thread-1] (ProcessInput.java:124) - Loading data from file.
Contig length treshold: 1000
2014-12-10 10:43:43,794 DEBUG [Thread-1] (ProcessInput.java:91) - Loading fasta file: /Users/yohan.jarosz/Downloads/37A/37A.concatenated.fa
2014-12-10 10:43:44,664 DEBUG [Thread-1] (ProcessInput.java:91) - DataSet loaded (13042 sequences)
2014-12-10 10:43:44,665 DEBUG [Thread-1] (ProcessInput.java:91) - Creating kmers (k=5, merge = true)
2014-12-10 10:43:44,666 DEBUG [Thread-1] (DataSetUtils.java:65) - Creating usedVal for k=5
2014-12-10 10:43:47,529 DEBUG [Thread-1] (ProcessInput.java:91) - Normalizing vectors...
2014-12-10 10:43:47,597 DEBUG [Thread-1] (ProcessInput.java:91) - Clr normalization...
2014-12-10 10:43:48,247 DEBUG [Thread-1] (ProcessInput.java:91) - Running PCA... (Mtj)
Dec 10, 2014 10:43:48 AM com.github.fommil.jni.JniLoader liberalLoad
INFO: successfully loaded /var/folders/gn/p47pzlx16v7bnwnwv4b10rv9skqz7l/T/jniloader3997748089661723248netlib-native_system-osx-x86_64.jnilib
2014-12-10 10:43:49,150 DEBUG [Thread-1] (DataSetUtils.java:249) - DONE: Computed the new basis.
2014-12-10 10:43:49,629 DEBUG [Thread-1] (DataSetUtils.java:256) - DONE: Projected from sample to eigen space.
2014-12-10 10:43:49,630 DEBUG [Thread-1] (ProcessInput.java:91) - Running T-SNE...
2014-12-10 10:43:49,965 DEBUG [Thread-1] (DataSetUtils.java:264) - Running command: "/Users/yohan.jarosz/.vizbin/pbh_tsne_osx" in directory: /var/folders/gn/p47pzlx16v7bnwnwv4b10rv9skqz7l/T/map2098942954547252279
Number of threads: 1
Seed: 0
2014-12-10 10:43:50,377 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Read the 13042 x 50 data matrix successfully!
2014-12-10 10:43:50,379 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Running BH_TSNE with 1 threads and seed = 0
2014-12-10 10:43:50,380 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Using no_dims = 2, perplexity = 30.000000, theta = 0.500000
2014-12-10 10:43:50,380 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Computing input similarities...
2014-12-10 10:43:50,459 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Building tree...
2014-12-10 10:43:50,459 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE:  - point 0 of 13042
2014-12-10 10:43:52,338 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE:  - point 10000 of 13042
2014-12-10 10:43:53,175 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Done in 2.79 seconds (sparsity = 0.010317)!
2014-12-10 10:43:53,175 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Learning embedding...
2014-12-10 10:43:55,891 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 50: error is 98.882329 (50 iterations in 0.00 seconds)
2014-12-10 10:43:58,439 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 100: error is 87.618302 (50 iterations in 0.00 seconds)
2014-12-10 10:44:00,705 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 150: error is 81.454664 (50 iterations in 0.00 seconds)
2014-12-10 10:44:02,919 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 200: error is 80.281685 (50 iterations in 0.00 seconds)
2014-12-10 10:44:05,062 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 250: error is 4.155104 (50 iterations in 0.00 seconds)
2014-12-10 10:44:07,134 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 300: error is 3.178465 (50 iterations in 0.00 seconds)
2014-12-10 10:44:09,252 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 350: error is 2.854119 (50 iterations in 0.00 seconds)
2014-12-10 10:44:11,337 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 400: error is 2.672712 (50 iterations in 0.00 seconds)

-> MTJ is working at maximum performance, as expected for a Mac.

@claczny
Copy link
Owner Author

claczny commented Dec 10, 2014

From Ubuntu-14.10VM:

test@test-VirtualBox:~/VizBin$ java -jar VizBin-dist.jar 
2014-12-10 11:08:56,816 DEBUG [main] (MainFrame.java:61) - Init of Main application frame
2014-12-10 11:09:44,795 DEBUG [AWT-EventQueue-0] (ProcessInput.java:61) - Init of ProcessInput
2014-12-10 11:09:44,800 DEBUG [Thread-0] (ProcessInput.java:115) - /tmp
2014-12-10 11:09:44,835 DEBUG [Thread-0] (ProcessInput.java:124) - Loading data from file.
Contig length treshold: 1000
2014-12-10 11:09:44,835 DEBUG [Thread-0] (ProcessInput.java:91) - Loading fasta file: /home/test/Downloads/37A/37A/37A.concatenated.fa
2014-12-10 11:09:46,574 DEBUG [Thread-0] (ProcessInput.java:91) - DataSet loaded (13042 sequences)
2014-12-10 11:09:46,575 DEBUG [Thread-0] (ProcessInput.java:91) - Creating kmers (k=5, merge = true)
2014-12-10 11:09:46,579 DEBUG [Thread-0] (DataSetUtils.java:65) - Creating usedVal for k=5
2014-12-10 11:09:50,596 DEBUG [Thread-0] (ProcessInput.java:91) - Normalizing vectors...
2014-12-10 11:09:50,707 DEBUG [Thread-0] (ProcessInput.java:91) - Clr normalization...
2014-12-10 11:09:51,061 DEBUG [Thread-0] (ProcessInput.java:91) - Running PCA... (Mtj)
Dec 10, 2014 11:09:52 AM com.github.fommil.jni.JniLoader liberalLoad
INFO: successfully loaded /tmp/jniloader9144771906803982692netlib-native_system-linux-x86_64.so
2014-12-10 11:09:56,600 DEBUG [Thread-0] (DataSetUtils.java:249) - DONE: Computed the new basis.
2014-12-10 11:09:58,336 DEBUG [Thread-0] (DataSetUtils.java:256) - DONE: Projected from sample to eigen space.
2014-12-10 11:09:58,337 DEBUG [Thread-0] (ProcessInput.java:91) - Running T-SNE...
2014-12-10 11:09:59,068 DEBUG [Thread-0] (DataSetUtils.java:264) - Running command: "/home/test/.vizbin/pbh_tsne" in directory: /tmp/map6211623356938677046
Number of threads: 2
Seed: 0
2014-12-10 11:09:59,415 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Read the 13042 x 50 data matrix successfully!
2014-12-10 11:09:59,421 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Running BH_TSNE with 2 threads and seed = 0
2014-12-10 11:09:59,422 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Using no_dims = 2, perplexity = 30.000000, theta = 0.500000 
2014-12-10 11:09:59,422 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Computing input similarities...
2014-12-10 11:09:59,557 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Building tree...
2014-12-10 11:09:59,557 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE:  - point 0 of 13042
2014-12-10 11:10:01,859 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE:  - point 10000 of 13042
2014-12-10 11:10:02,907 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Done in 3.48 seconds (sparsity = 0.010317)!
2014-12-10 11:10:02,907 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Learning embedding...
2014-12-10 11:10:05,780 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 50: error is 98.882329 (50 iterations in 0.00 seconds)
2014-12-10 11:10:08,680 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 100: error is 87.618302 (50 iterations in 0.00 seconds)
2014-12-10 11:10:11,312 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 150: error is 81.455067 (50 iterations in 0.00 seconds)
2014-12-10 11:10:13,889 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 200: error is 80.282551 (50 iterations in 0.00 seconds)
2014-12-10 11:10:16,425 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 250: error is 4.155243 (50 iterations in 0.00 seconds)
2014-12-10 11:10:18,812 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 300: error is 3.178847 (50 iterations in 0.00 seconds)
2014-12-10 11:10:21,273 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 350: error is 2.854269 (50 iterations in 0.00 seconds)
2014-12-10 11:10:23,672 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 400: error is 2.672754 (50 iterations in 0.00 seconds)
2014-12-10 11:10:26,154 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 450: error is 2.554703 (50 iterations in 0.00 seconds)
2014-12-10 11:10:28,864 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 500: error is 2.470680 (50 iterations in 0.00 seconds)
2014-12-10 11:10:31,203 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 550: error is 2.407209 (50 iterations in 0.00 seconds)
2014-12-10 11:10:33,592 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 600: error is 2.357605 (50 iterations in 0.00 seconds)
2014-12-10 11:10:35,978 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 650: error is 2.317692 (50 iterations in 0.00 seconds)
2014-12-10 11:10:38,360 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 700: error is 2.285571 (50 iterations in 0.00 seconds)
2014-12-10 11:10:40,739 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 750: error is 2.259456 (50 iterations in 0.00 seconds)
2014-12-10 11:10:43,100 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 800: error is 2.238512 (50 iterations in 0.00 seconds)
2014-12-10 11:10:45,503 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 850: error is 2.222015 (50 iterations in 0.00 seconds)
2014-12-10 11:10:47,846 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 900: error is 2.209019 (50 iterations in 0.00 seconds)
2014-12-10 11:10:50,261 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 950: error is 2.199357 (50 iterations in 0.00 seconds)
2014-12-10 11:10:52,644 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Iteration 999: error is 2.192653 (50 iterations in 0.00 seconds)
2014-12-10 11:10:52,645 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Fitting performed in 0.00 seconds.
2014-12-10 11:10:52,664 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: Wrote the 13042 x 2 data matrix successfully!
2014-12-10 11:10:52,664 DEBUG [TSNERunner] (DataSetUtils.java:300) - TSNE: 
2014-12-10 11:10:52,686 DEBUG [Thread-0] (ProcessInput.java:91) - Points created.
2014-12-10 11:10:52,989 DEBUG [Thread-0] (ProcessInput.java:91) - Creating png files....
2014-12-10 11:10:55,109 DEBUG [Thread-0] (ProcessInput.java:91) - Done.

-> MTJ is working at maximum performance, as expected for a Linux with the necessary packages installed (i.e., BLAS and LAPACK).

@claczny
Copy link
Owner Author

claczny commented Dec 10, 2014

Average error between the points file returned by Windows and the one under Mac OS X

cedric.laczny@CLMBP:~/Documents/phd/projects/BINNING/publication/VizBin_-_Application_Note/revision_01/data/examples/37A $ paste -d"," 37A.concatenated.points /Users/cedric.laczny/Downloads/37A_Windows_points.txt | awk -F"," '{sum += sqrt(($1-$3)*($1-$3))} END {print sum}'
4999.66
cedric.laczny@CLMBP:~/Documents/phd/projects/BINNING/publication/VizBin_-_Application_Note/revision_01/data/examples/37A $ paste -d"," 37A.concatenated.points /Users/cedric.laczny/Downloads/37A_Windows_points.txt | awk -F"," '{sum += sqrt(($1-$3)*($1-$3))} END {print sum/NR}'
0.383351
cedric.laczny@CLMBP:~/Documents/phd/projects/BINNING/publication/VizBin_-_Application_Note/revision_01/data/examples/37A $ paste -d"," 37A.concatenated.points /Users/cedric.laczny/Downloads/37A_Windows_points.txt | awk -F"," '{sum += sqrt(($2-$4)*($2-$4))} END {print sum/NR}'
0.647647
cedric.laczny@CLMBP:~/Documents/phd/projects/BINNING/publication/VizBin_-_Application_Note/revision_01/data/examples/37A $ head 37A.concatenated.points /Users/cedric.laczny/Downloads/37A_Windows_points.txt
==> 37A.concatenated.points <==
15.148317,-10.563461
15.546716,-10.248951
15.815515,-10.100727
14.651723,-10.931276
15.734991,-10.211598
16.043028,-9.728845
15.835106,-10.058832
15.644402,-10.170950
15.822641,-10.046160
15.184782,-10.477033

==> /Users/cedric.laczny/Downloads/37A_Windows_points.txt <==
15.084483,-10.184800
15.497696,-9.890971
15.773701,-9.769374
14.572603,-10.524224
15.691000,-9.870220
16.049392,-9.413775
15.798445,-9.729780
15.601806,-9.814748
15.783813,-9.717228
15.123478,-10.100430
cedric.laczny@CLMBP:~/Documents/phd/projects/BINNING/publication/VizBin_-_Application_Note/revision_01/data/examples/37A $

Seems to be related mainly to differences in precision etc. (really hard to avoid and would not add any information as it is anyways a locally optimal solution, albeit a very good one.)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@claczny