Merge pull request #1 from shukriadams/dev

merge dev to master

.github/workflows/build.yml

@@ -0,0 +1,15 @@
+name: builds 
+on: 
+  push:
+    tags:
+      - '*'
+jobs:
+  build:
+    runs-on: ubuntu-latest
+    steps:
+      - uses: actions/checkout@v1
+      - name: builds
+        run: cd build && sh ./build.sh --ci --dockerpush
+        env:
+            DOCKER_PASS: ${{ secrets.DOCKER_PASS }}
+            DOCKER_USER: ${{ secrets.DOCKER_USER }}

.gitignore

@@ -27,6 +27,7 @@ src/.scss-linters
 /docker/.images
 /src/client/bundle
 /src/yarn-error.log
+/src/server/reference/music
 /src/client/lib-dev
 /src/__logs
 /docker/node/.deployKeys
@@ -43,3 +44,5 @@ build/node/.stage.tar.gz
 build/logs
 build/data
 src/data
+src/.cache-require-paths.json
+!src/client/app/localLib/**

README.md

@@ -16,6 +16,13 @@ Tuna lets you stream your music to any modern browser. It is open source and sel
 Tuna uses Oauth to access to your music on your storage platform. Both Dropbox and Nextcloud allow you to set up apps which can grant access, and you'll need to set one up first. Fortunately the process is relatively straightforward. 
 
 - [Dropbox](https://www.dropbox.com/developers/apps/create)
+
+#### NextCloud
+
+When setting up your Oauth app in Nextcloud, you'll need to specify a callback URL. Use
+
+    https://yourtunaurl.com/v1/oauth/nextcloud
+
 - [Nextcloud](later)
 
 ### Server 
@@ -41,9 +48,10 @@ The following docker-compose script will set up everything you need to run a Tun
             depends_on:
                 - mongo
             volumes:
-                - ./tuna:/usr/app/data/:rw
+                - ./tuna:/usr/tuna/data/:rw
             environment:
                 mongoConnectionString: "mongodb://admin:yourPasswordHere@mongo:27017"
+                siteUrl: "https://yourtunaurl.com"
 
                 # Allowed values are : nextcloud|dropbox
                 musicSource : nextcloud 
@@ -62,6 +70,11 @@ The following docker-compose script will set up everything you need to run a Tun
 
 Change "yourPasswordHere" to something better. Note that this setup isn't ideal for security as it passwords are stored in clear text, and you're connecting to Mongo as root, but it's "good enough" to get started.
 
+Before starting you should create the local tuna folder and set its permission
+
+    mkdir tuna
+    chown 1000 -R tuna
+
 ### Indexing music
 
 Tuna needs to know what music you've got in your Dropbox or NextCloud drive. Download the Tuna Indexer app, install and point it to your Dropbox or NextCloud folder and let it do its thing.
@@ -76,6 +89,19 @@ If you lose your password you can set a new one from the command line with
 
 When you log in, you'll be prompted to give access to either Dropbox or Nextcloud, follow the on-screen instructions. 
 
+## Advanced 
+
+### Nginx and Socket.io
+
+This app makes entensive use of websockets, if you're hosting your app behind Nginx, you might have problems with this. Try adding the following to your Nginx config
+
+    location / {
+        proxy_pass http://localhost:YOUR-PORT-HERE; # add your own app port hehre
+        proxy_http_version 1.1;
+        proxy_set_header Upgrade $http_upgrade;
+        proxy_set_header Connection "upgrade";
+        proxy_set_header Host $host;
+    }
 
 ## License
 

build/build.sh

@@ -24,22 +24,26 @@ fi
 mkdir -p .clone &&    
 
 if [ $CI -eq 1 ]; then
+    # if running on CI system, copy everything from src to .clone folder
     cp -R ./../src .clone/ 
 else
+    echo "Copying a bunch of stuff, this will likely take a while ...."
+    # copy everything from src but exclude 
     if [ $JSPM -eq 1 ]; then
-        rsync -r --exclude=node_modules --exclude=client/lib --exclude=.* ./../src .clone 
+        # exclude node modules, jspm lib and all hidden files in project root
+        rsync -v -r --exclude=node_modules --exclude=client/lib --exclude=.* ./../src .clone 
     else
-        rsync -r --exclude=node_modules --exclude=.* ./../src .clone 
+        rsync -v -r --exclude=node_modules --exclude=.* ./../src .clone 
     fi
 fi
 
 # build 1: build frontend CSS/JS, and leaves it behind in .clone/src/dist folder. This build will npm install dev modules, which we
 # want to delete
-docker run -it -v $(pwd)/.clone:/tmp/tuna shukriadams/node10build:0.0.3 sh -c "cd /tmp/tuna/src && sh ./build-frontend.sh --setup" &&
+docker run -v $(pwd)/.clone:/tmp/tuna shukriadams/node10build:0.0.3 sh -c "cd /tmp/tuna/src && sh ./build-frontend.sh --setup" &&
 sudo rm -rf .clone/src/node_modules &&
 
 # build 2: run a second container install, this one npm intalls "production" modules, these we want
-docker run -it -v $(pwd)/.clone:/tmp/tuna shukriadams/node10build:0.0.3 sh -c "cd /tmp/tuna/src && yarn --no-bin-links --ignore-engines --production" &&
+docker run -v $(pwd)/.clone:/tmp/tuna shukriadams/node10build:0.0.3 sh -c "cd /tmp/tuna/src && yarn --no-bin-links --ignore-engines --production" &&
 
 # combine artifacts from steps 1 and 2 and zip them
 rm -rf .stage &&
@@ -48,9 +52,12 @@ mkdir -p .stage/public &&
 cp -R .clone/src/dist/client .stage &&
 cp -R .clone/src/node_modules .stage &&
 cp -R .clone/src/server .stage &&
+cp -R .clone/src/scripts .stage &&
 cp -R .clone/src/public/css .stage/public &&
+cp -R .clone/src/client/media .stage/public &&
 cp .clone/src/index.js .stage &&
 cp .clone/src/package.json .stage &&
+rm -rf .stage/server/reference &&
 rm -f node/.stage.tar.gz &&
 tar -czvf node/.stage.tar.gz .stage &&
 

build/docker-compose-test.yml

@@ -2,7 +2,7 @@
 # with dev resources
 version: "2"
 services:
-    mongo:
+    mongotest:
         image: mongo:3.6.18
         container_name: mongo-test
         restart: unless-stopped
@@ -17,7 +17,7 @@ services:
         container_name: tuna-test
         restart: unless-stopped
         depends_on:
-            - mongo
+            - mongotest
         volumes:
             - ./logs:/usr/tuna/__logs/:rw
             - ./data:/usr/tuna/data
@@ -27,6 +27,7 @@ services:
             # connecting as root is only guaranteed method, mongo is pure garbage
             mongoConnectionString: "mongodb://admin:secret@mongo:27017"
             mongoDBName: "tuna"
+            musicSourceSandboxMode: "true"
         ports:
         - "58002:48004"
 

docs/development.md

@@ -1,49 +1,67 @@
-# Developer docs
+# Tuna Developer docs
 
 This a work in progress.
 
 ## Setup
 
-This project is configured to work in Vagrant. After cloning the project
+This project is configured to work in Vagrant. After cloning the project start vagrant
 
-    # start vagrant
-    cd /vagrant
     vagrant up
     vagrant ssh
 
-    # start local mongo
-    cd /vagrant/build
-    docker-compose -f docker-compose-dev.yml up -d 
+You should be in the folder /vagrant/src, which is where you'll spend most of your time when developing on Tuna. _The path in the rest of this document are based on the assumption  you're in this folder._
+
+ Install NodeJS packages with the following
 
-    cd /vagrant/src
     yarn --no-bin-links 
     npm rebuild node-sass --no-bin-links
     jspm install -y
 
-You should always use yarn to install node modules, installing with npm fails.
+You should always use yarn to install node modules, installing with npm will usually fail, likely because of a combination of this project's NodeJS version and some of the older packages used.
+
+## Settings
+
+All Tuna settings are environment variables. You can edit settings directly in _/src/server/helpers/settings.js_, but a better way is to create a settings override file @ _/src/.env_. This file is not committed to source control, and contains name=value pairs. You should add at least the following values to it
+
+    musicSourceSandboxMode=true
+
+This puts Tuna in sandbox mode.
+
+### Sandbox Mode
 
-### Run
+Tuna can be run in sandbox mode, whereby it simulates Dropbox, Nextcloud and any other service it interacts with. Use this to develop and test locally without having to interact with those platforms. This is particularly useful for services like developer-unfriendly Nextcloud, which offers no convenient way to get dev auth tokens. 
+
+When you import music in sandbox mode, Tuna will import song listings from the /src/server/reference/.tuna.xml file - in live mode, this file would normally be in the root of your Dropbox or NextCloud folder.
+
+To stream music in sandbox box, add any mp3 songs to /src/server/reference/music. Random songs will be streamed back.
+
+#### Run
 
 The easiest way to to run the project is
 
-    cd /src
     node index
 
-To start the dev environment with debugging enabled, sass watching, server restarting etc, run
+To start the dev environment with debugging enabled, live compile etc run
 
     cd /src
     npm run dev
 
+If you want to force the app to wait until your debugger is attached use
+
+    npm run dev -- --brk
+    npm run dev -- --break
+
 View at http://localhost:48004
 
-Visual Studio Code debug config is committed with project source, so if you start with the above you can debug immediately.
+A Visual Studio Code debug config for with Vagrant support is committed with project source, so if you start with the above you can debug immediately.
 
 ### Mongo 
 
 Part of the dev tools includes a Mongo-express admin interface already running at http://localhost:49002
+
 If you want to connect with your own mongo client, the dev Mongo database is exposed via http://localhost:49001
 
-### Debug Chrome
+### Debug with Chrome
 
 - on the host system, get the debug url with http://127.0.0.1:9222/json
 - Post url in chrome, add break points and you're off
@@ -52,16 +70,15 @@ If you want to connect with your own mongo client, the dev Mongo database is exp
 
 ### Frontend only 
 
-    cd /src
-    ./build-frontend.sh
+    sh ./build-frontend.sh
 
 This is useful for testing Jspm & Sass bundling etc.
 
 ### Local docker build
 
 To build a docker container image of your local code
 
-    cd /build
+    cd /vagrant/build
     sh ./build.sh
 
 To test your container build locally
@@ -72,3 +89,26 @@ Open in browser from host machine
 
     http://localhost:58002
 
+
+## Tests
+
+To run all tests use
+
+    npm test
+
+To run a single test use
+
+    npm test "test name"
+
+where "test name" can be either a group of tests as defined in 
+
+    mocha("group name") 
+
+at the top of a test file, or an individual test's name in 
+
+    it("test name")
+
+To debug a single test use
+
+    npm run testdebug "test name" 
+

docs/nextcloud.md

@@ -16,9 +16,9 @@ If you're running on localhost and want to test your integration on a live Nextc
 
             https://{NEXTCLOUDHOST}/index.php/apps/oauth2/authorize?response_type=code&client_id={NEXTCLOUD_CLIENT_ID}&state=somethingrandom&redirect_uri=https://{DEFINITELY_BOGUS_URL}
 
-    Change {NEXTCLOUDHOST}, {NEXTCLOUD_CLIENT_ID} AND {DEFINITELY_BOGUS_URL} - the latter should be a URL you know doesn't exist. Paste the resulting URL into a browser address bar to start the OAUTH flow. Authorize the request in Nextcloud - this will attempt to redirect to the bogus url and stall there. Copy the "code" querystring value that should now be in your browser address bar
+    Change {NEXTCLOUDHOST}, {NEXTCLOUD_CLIENT_ID} AND {DEFINITELY_BOGUS_URL} - the latter should be a URL you know doesn't exist and won't be redirected to a 404 or other page. Paste the resulting URL into a browser address bar to start the OAUTH flow. Authorize the request in Nextcloud - this will attempt to redirect to the bogus url and stall there. Copy the "code" querystring value that should now be in your browser address bar
 
-    4. Open a console window in your Tuna source code 
+    4. Make sure your local Tuna server is already running, and that it is not running sandbox mode. Open a console window in your Tuna source code 
 
         cd /src/scripts
         node nextcloud-codeToToken --code {YOUR-CODE-HERE}

src/build-frontend.sh

@@ -40,8 +40,9 @@ node ./build/build-all-sass && minify ./public/css/style.css > ./public/css/styl
 # build and minify JS - this requires cd'ing into jspm relative folder. 
 # note that --minify option for jspm fails with ES6, we use uglifyJS instead
 cd client/app/
-jspm bundle-sfx main/main.js ../../dist/client/app/bundle.js
-cp config.js ../../dist/client/app/config.js
+jspm bundle-sfx main/main.js ../../dist/client/app/app.js
+# bundle large js files with bash for simplicity, appending newline between them
+awk 1 ./../lib/system.js ../../dist/client/app/app.js config.js > ../../dist/client/app/bundle.js
 cd -
 
 # wait for bundle to appear, it can take its time
@@ -51,5 +52,4 @@ do
   sleep 2
 done
 
-uglifyjs -c -v -o ./dist/client/app/bundle.min.js -- ./dist/client/app/bundle.js
-uglifyjs -c -v -o ./dist/client/app/config.min.js -- ./dist/client/app/config.js
+uglifyjs -c -v -o ./dist/client/app/bundle.min.js -- ./dist/client/app/bundle.js