updated by DAT-Core

.gitignore

@@ -1 +1,2 @@
 _site/
+.DS_Store

_data/sidebars/sparc_sidebar.yml

@@ -9,7 +9,6 @@ entries:
   - title: Overview
     output: web, pdf
     folderitems:
-
     - title: Information
       url: /introduction.html
       output: web, pdf
@@ -19,9 +18,21 @@ entries:
   - title: Data submission
     output: web, pdf
     folderitems:
-    - title: DAT-Core user accounts
+    - title: User accounts
       url: /user_accounts.html
       output: web, pdf  
     - title: Submitting data
       url: /submit_data.html
       output: web, pdf
+  - title: DAT-Core Information
+    output: web, pdf
+    folderitems:
+    - title: DAT-Core information
+      url: /dat_information.html
+      output: web, pdf
+    - title: DAT-Core roadmap
+      url: /dat_roadmap.html
+      output: web, pdf
+
+
+

pages/dat_core/dat_core_roadmap.md

@@ -0,0 +1,49 @@
+---
+title: "DAT-Core Road Map"
+keywords: documentation, github
+sidebar: sparc_sidebar
+permalink: dat_roadmap.html
+summary: This page documents the SPARC Milestones for the DAT-Core
+folder: general
+---
+
+## Summary
+This document outlines the SPARC milestones for the DAT-Core as outlined in the SPARC Notice of Award (NOA). It provides an outline of the functionality that will be developed for the DAT-Core platform as part of the SPARC effort. This document will be updated when changes to milestones occur. 
+
+### Q1: October-December 2018
+
+1. **Model, and dataset templates:** DAT-Core will implement functionality to create meta-data templates and use those templates to define a metadata structure for a dataset. 
+2. **Support for linking model properties to other records:** The DAT-Core will provide the ability to define metadata models that contain properties that point to other records. This expands exisiting functionality to create relationships between records.
+3. **Support for enumerated values, and multiple values in properties:** The DAT-Core will provide functionality to define properties in metadata models where users can only select from predefined values ('a dropdown menu'). This will expand functionality to standardize datasets.
+4. **Support for uploading large files:** Based on user feedback, there is a need to upload very large files that take longer than 1 hour to upload per file. The DAT-Core will provide functionality to support this in a secure and performent way.
+5. **Documentation for workflow, and platform:** The DAT-Core will provide extensive documentation around the data submission workflow and documentation of its platform.
+
+### Q2: January-March 2019
+
+1. **Expanding user permissions on datasets:** The DAT-Core will expand functionality for roles on datasets to support the SPARC data workflow.
+2. **Expanded functionality for search over metadata:** The DAT-Core will expand functionality in their platform to search over metadata within datasets. 
+3. **Support for dataset status:** Adding support for a status flag on datasets indicating the stage of curation it is in (draft, in-review, finalized)
+4. **Support for N-dimensional imaging data:** The DAT-Core will expand functionality for 3D clinical imaging, and N-Dimensional microscopy imaging. 
+5. **Improved tracking of SPARC data-use:** The DAT-Core will improve funcitonality to track how much data is contributed by SPARC investigators.
+
+### Q3: April-June 2019
+
+1. **Support for extracting header information from files:** DAT-Core will expand functionality to extract useful information from headers of files that are uploaded. 
+2. **Launch of the Open Data Library:** The Open Data Library is a separate website where anyone can find and request access to public datasets shared on the DAT-Core. It is a webapplication that allows users of the DAT-Core platform to share their data publicly. Public datasets will receive a global DOI.
+3. **Support for copyright and License information for public datasets:** Adding functionality for users to provide License and Copyright information to their datasets in preparation for make the data publicly available.
+4. **Support for viewing Microsoft documents (if feasible):** The DAT-Core will implement a mechanism to better visualize Microsoft Office documents.
+5. **Support for versioning of metadata:** The DAT-Core will provide an MVP for versioning of metadata on the platform.
+6. **Deliver a MVP for the sparc data portal (data.sparc.science):** The DAT-Core, in collaboration with the other cores, will provide a separate web-application to visualize the integrated SPARC dataset. 
+
+### Q4: July-September 2019
+
+1. **Support for hierarchical ontologies:** The DAT-Core will expand functionality to link to ontologies and support for hierarchical ontologies. 
+2. **Support for merging metadata from multiple datasets:** The DAT-Core will provide functionality to merge metadata between datasets.
+3. **Expanded support for the Open Data Library:** DAT-Core will implement functionality to allow investigators outside the SPARC initiative, without a subscription to the DAT-Core platform, to access/download data from the Open Data Library in a sustainable way (cost for downloading data will be passed ot the downloader).
+
+
+### SPARC Year 3
+- **Focus on analysis:** The ability to run analysis on the cloud.
+- **Focus on federated datasets:** The ability to store data remotely.
+- **Focus on integration:** Integration with other platforms within, and beyond the SPARC DRC.
+

pages/dat_core/dat_core_user_accounts.md

@@ -1,5 +1,5 @@
 ---
-title: "DAT-Core User Accounts"
+title: "User Accounts"
 keywords: documentation, github
 sidebar: sparc_sidebar
 permalink: user_accounts.html

pages/dat_core/information.md

@@ -0,0 +1,24 @@
+---
+title: "DAT-Core Information"
+keywords: documentation, github
+sidebar: sparc_sidebar
+permalink: dat_information.html
+summary: This page provides information about the SPARC DAT-Core
+folder: general
+---
+
+## Introduction
+The SPARC Data Core (DAT-Core) is implemented by Blackfynn Inc. [Blackfynn](https://www.blackfynn.com) develops cloud based solutions for data management and analysis for the Neurosciences. As part of the SPARC OT award, Blackfynn will develop functionality in their platform to support the SPARC data workflow. The platform is developed with sustainability in mind assuring that the DAT-Core platform will continue to be viable beyond the funding horizon of the SPARC program, and beyond the scope of the SPARC initiative. 
+
+## Contact the DAT-Core
+There are a number of ways to get in contact with the DAT-Core:
+
+1. For all support questions about the platform, feedback, or bug reports, please use the "Get Help" button after logging into the platform to send the DAT-Core a message (see below). This is the best method to reach the DAT-Core development team, and will be prioritized within Blackfynn to guarantee a quick response. {% include image.html file="dat_core_help.png" alt="Get Help" max-width="200" %}
+
+2. For more general questions about the SPARC effort, and the role of the DAT-Core, please contact Leonardo Guercio or Joost Wagenaar by email or Slack (using the SPARC Slack account).
+
+3. For other general questions about Blackfynn, use the contact form on the [Blackfynn website](https://www.blackfynn.com).
+
+## Using Blackfynn outside of SPARC
+SPARC investigators are allowed to use the Blackfynn platform to organize their own scientific data outside of the SPARC effort within their individual accounts. The individual accounts are limited to match the Blackfynn Academic subscription model ([Blackfynn Academia](https://www.blackfynn/academia)). Contact Blackfynn if you have any questions about Blackfynn Academia.
+

pages/data_submission/dat_core_user_accounts.md

@@ -0,0 +1,30 @@
+---
+title: "User Accounts"
+keywords: documentation, github
+sidebar: sparc_sidebar
+permalink: user_accounts.html
+summary: This pages outlines the workflow for submitting data to the SPARC DAT-Core. 
+folder: general
+---
+
+## User accounts on the DAT-Core
+
+All SPARC investigators have access to the DAT-Core platform ([Blackfynn](https://app.blackfynn.io)). SPARC Investigators will have access to two accounts on the Blackfynn platform. 
+
+1. **A private account for their lab:** This account is available to the individual SPARC investigators and is independent of the SPARC requirements. Investigators can use this account for their data management needs within, and beyond the SPARC program. This account is private and not managed by the SPARC Consortium and has the same restrictions as a standard [Blackfynn academic subscription](https://www.blackfynn.com/academia)
+
+2. **The SPARC Consortium account:** This account will be used for all data submissions to the SPARC consortium. This account will enforce strict data models and data submission processes. 
+
+The remainder of the SPARC documenentation will reference the **SPARC Consortium organization** only. 
+
+### The SPARC Consortium account on the DAT-Core
+
+Since the SPARC Consortium account will host all SPARC data (including embargoed data), access to to this organization will be restricted until users sign a non-disclosure agreement (NDA). User accounts to this organization will be managed by the DAT-CORE. By signing the NDA, SPARC awardees agree not to unilaterally publish or disseminate data made available from other labs and not to use data for commercial purposes during the embargo period. Once the users have signed the NDA, they will be granted access to the SPARC Consortium account. 
+
+### SPARC Teams on the DAT-Core
+
+Once users are accepted into the SPARC Consortium account, each investigator group can create a team. Initially, teams will reflect the individual laboratories, but it is possible to create any number of teams comprised of SPARC investigators (i.e. users with access to the SPARC Consortium Account). Teams will be used to set data access rights and roles for submitted datasets.
+
+## SPARC Slack Organization
+
+All SPARC investigators can join the SPARC [Slack Account](https://www.slack.com). SPARC uses Slack to foster collaboration and communication between investigators within the SPARC program. If you are interested to join the Slack account, send a message to Leonardo Guercio from the DAT-Core ([email protected]).

pages/dat_core/submit_data.md → pages/data_submission/submit_data.md

@@ -3,7 +3,7 @@ title: "SPARC Data submission workflow"
 keywords: documentation, github
 sidebar: sparc_sidebar
 permalink: submit_data.html
-summary: (DRAFT) This pages outlines the workflow for submitting data to the SPARC DAT-Core. 
+summary: This pages outlines the workflow for submitting data to the SPARC DAT-Core. 
 folder: general
 ---
 
@@ -61,6 +61,8 @@ Who owns the data: The investigator that submitted the data or the person who is
 - Publish the dataset to the Open Data Library to make the dataset findable for anybody.
 
 
-##  Data on the platform
+##  Schematic representation of SPARC Data on the DAT-Core
+
+The figure below is a schematic representation of the workflow described above. It highlights how data is generated by individual investigators, is curated by the curation team, and shared as an embargoed dataset with the SPARC consortium. It shows how the data is made available to the public over time, and how the integrated dataset intersects with the individual datasets created by the investigators.
 
 {% include image.html file="dat_core_diagram.png" alt="workflow" %}

pages/general/introduction.md

@@ -1,18 +1,15 @@
 ---
-title: SPARC DRC Contact information
+title: The SPARC program
 keywords: SPARC 
 toc: false
-summary: "(DRAFT) How to reach the various components of the Data Resource Center"
+summary: "Introduction to the docs.sparce.science documentation site."
 sidebar: sparc_sidebar
 permalink: introduction.html
 folder: general
 ---
 
-### DAT-Core (Data Core):
-[email protected] or through the Blackfynn web application for technical support
-
-### SIM-Core (Simulation Core):
-
-
-### MAP-Core (Mapping Core):
+## Introduction
+This documentation provides information relevant to SPARC investigators and anyone interested in understanding the data workflow and policies of the SPARC project. The information on this site is managed by the SPARC Data Resource Center (DRC) and content might change significantly over time. 
 
+## Contributing to the documentation
+This documentation is created by the SPARC community and provides information related to the various aspects of the SPARC program. The sourcecode for this documentation site is available on Github: [https://github.com/Blackfynn/docs.sparc.science](https://github.com/Blackfynn/docs.sparc.science) and anyone can generate a pull request to make changes to this documentation site. More information here.