Maximum upload size exceeded

[avivdm]

When trying to upload a large (~100 MB) count data file, shiny reports that the maximum upload size is exceeded.

[nephantes]

We have a limitation about file size. I might suggest you change that limitation in the code.

1. So you can clone the source to your machine with;

   git clone https://github.com/UMMS-Biocore/debrowser.git

2. Increase the memory from 30 to 100 on line 16 in the R/startShiny.R file

   options( shiny.maxRequestSize = 30 * 1024 ^ 2, warn = -1,
   shiny.sanitize.errors = TRUE)

3. Install it using the command below;
   R CMD INSTALL debrowser

You should be all set. But if you have many samples, the names might not fit to the plots well. I have tried it with up to 40 samples but more than that, some plots might not behave well.
Let me know if you have any question or difficulty to make the change.

[tomasflyo]

Thank you for the help.
After completing steps 1. 2. and 3. above, how would you initiate the DEBrowser itself?
Returning to the console and using the previous commands library(debrowser) and startDEBrowser() does not work

[nephantes]

All the messages are looking right.
Are you running it in RStudio or R?
If not please run R and
run the commands below;

library(debrowser)
startDEBrowser()

If you get an error; please send the error.

[tomasflyo]

That did the trick! Thanks!
Now I am getting a different error message, but I assume its something relating to the construct of the Count Data File I am using?

[nephantes]

Yes, there is a problem in your count data file. It should be filled with all numeric values. If there is a missing value or a string rather than a count value, you will get this error.

[tomasflyo]

It should be filled with all numeric values. If there is a missing value or a string rather than a count value, you will get this error.

I do not think this is the case. I have checked and re-checked my dataframe. No missing values or strings in there.

[nephantes]

If you send me the first 100 lines of the file. I can look at it, if there is a problem about the file. Sometimes, when the file is generated in microsoft environment, because of the enter characters at the end of the lines, they are not suitable to use in R. That might be the case.

[tomasflyo]

Attaching the Count Data File (tsv format)
I have been using python for data prep and cleanup.

Count_Data_File_100.txt

Thanks again

[avivdm]

Hi Alper, Many thanks for your kind help! My name is Aviv, I will try and work this thing out with instead of Thomas, which is a lab member of mine. So far DEBrowser fails on these two files (attached) after upload - it freezes and generates no output. They were generated in GNU/Linux environment. Thanks, Aviv.

…

________________________________ From: tomasflyo <[email protected]> Sent: Sunday, January 13, 2019 10:24:34 AM To: UMMS-Biocore/debrowser Cc: Aviv De-Morgan; Author Subject: Re: [UMMS-Biocore/debrowser] Maximum upload size exceeded (#196) Attaching the Count Data File (tsv format) I have been using python for data prep and cleanup. Count_Data_File_100.txt<https://github.com/UMMS-Biocore/debrowser/files/2752914/Count_Data_File_100.txt> Thanks again — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#196 (comment)>, or mute the thread<https://github.com/notifications/unsubscribe-auth/Ah_6-Nom7l9DGcW8EZKUVgMKwgWZOFq_ks5vCu1CgaJpZM4Z5NmE>.

[tomasflyo]

Dear Alper,

Short update.
There seem to be a couple of distinct issues working in tandem:

1. I have successfully uploaded a 15000 line Count Data File (35MB). That excludes the possibility of this being a format issue.
   
   When attempting to upload a larger file, e.g. 18000 lines (41MB), the DEBrowser shuts down and I get the following message in the Console:
   
   I have not yet tried to pinpoint if this is an issue of line count or byte size.

2. When I try to go ahead with the 15000 line file (that uploaded successfully) and upload the respective metadata file as well, I get the following error msg in the DEBrowser:
   
   and in the console:
   
   After first uploading the 15000 Count Data File, I noticed that the uploaded sample ID's received an 'X' as a prefix and that the '-' dashes in their names were replaced with '.' periods, so I tried doing the same in the metadata file, but this produced the same error messages as above.
   (tsv format)
   Metadata File2.txt

Error in data.frame: arguments imply differing number of rows: 654, 657

So it seems that for the Metadata file it is a formatting issue. I found some info on this error message, but I am still working on it.
Perhaps this may help to pinpoint the issue.

Thanks again

[nephantes]

Could you remove "-" in the sample names in your count file and try?

[nephantes]

These are duplicated genes in this data table.
So the gene names has to be unique in a count table. Otherwise, it cannot load the table.

5502 LIMS3 16 117 13
3372 NPIPA7 62 68 92
1896 PNRC2 147 498 68
6462 SCARNA17 14 34 37
2055 SNORA11 5 13 1
5053 SNORA31 4 6 3
5077 SNORA31 40 18 23
6230 SNORA40 58 38 17
5045 SNORA66 21 39 58
2098 SNORD19 6 18 10
4053 snoU13 8 14 14
4055 snoU13 66 41 72
4056 snoU13 10 2 9
4058 snoU13 5 9 33
4060 snoU13 18 9 7
4062 snoU13 69 38 58
4066 snoU13 16 10 6
4067 snoU13 33 33 57
6296 U3 23 12 387
6900 U3 13 19 10
765 Y_RNA 29 52 51
768 Y_RNA 24 14 17
770 Y_RNA 11 25 13
773 Y_RNA 17 5 12
774 Y_RNA 38 20 47
782 Y_RNA 16 36 22
788 Y_RNA 6 7 14
1193 Y_RNA 6 11 20
1204 Y_RNA 16 2 9
1214 Y_RNA 1 11 6
1216 Y_RNA 48 0 0
1221 Y_RNA 9 14 15
1411 Y_RNA 34 1 1
4065 Y_RNA 61 76 107
5041 Y_RNA 22 0 14
5048 Y_RNA 55 373 229
5069 Y_RNA 66 77 162
5073 Y_RNA 7 16 21
5074 Y_RNA 25 17 25
3830 ZNF26 2 21 15
1969 ZNF84 22 22 20

[baranaldemir]

We have a limitation about file size. I might suggest you change that limitation in the code.

1. So you can clone the source to your machine with;
   git clone https://github.com/UMMS-Biocore/debrowser.git
2. Increase the memory from 30 to 100 on line 16 in the R/startShiny.R file
   options( shiny.maxRequestSize = 30 * 1024 ^ 2, warn = -1,
   shiny.sanitize.errors = TRUE)
3. Install it using the command below;
   R CMD INSTALL debrowser

You should be all set. But if you have many samples, the names might not fit to the plots well. I have tried it with up to 40 samples but more than that, some plots might not behave well.
Let me know if you have any question or difficulty to make the change.

I cloned the file under R\win-library\4.1 and did the 2nd step. But When I do R CMD INSTALL debrowser I get an error like this:
What am I missing here?