Merge pull request #101 from leewujung/master

Update model module in handling user-specified Sv filename and path

README.rst

@@ -22,8 +22,8 @@ with increasing data volume - by providing open-source tools as entry points for
 scientists to make discovery using these new data.
 
 
-Installation and Usage
-----------------------
+Installation
+------------
 
 Echopype currently supports file conversion and computation of data produced by:
 
@@ -36,17 +36,32 @@ computations are performed.
 The data processing routines include calibration (instrument-specific), noise
 removal, and mean volume backscattering strength (MVBS) calculation.
 
-Echopype can be installed from PyPI or through conda:
+Echopype can be installed from PyPI:
 
 .. code-block:: console
 
-    # PyPI
-    $ pip install echopype
+   $ pip install echopype
+
 
-    # conda
-    $ conda install -c conda-forge echopype
+or through conda:
+
+.. code-block:: console
 
-Check out the `echopype documentation`_ for more details on installation and usage!
+   $ conda install -c conda-forge echopype
+
+
+When creating an conda environment to work with echopype,
+use the supplied ``environment.yml`` or do
+
+.. code-block:: console
+
+   $ conda create -c conda-forge -n echopype python=3.8 --file requirements.txt
+
+
+Usage
+-----
+
+Check out the `echopype documentation`_ for more details on installation and usage.
 
 Watch the `echopype talk`_  at SciPy 2019 for background, discussions and a quick demo!
 
@@ -57,25 +72,19 @@ Watch the `echopype talk`_  at SciPy 2019 for background, discussions and a quic
 Contributors
 ------------
 
-`Wu-Jung Lee <http://leewujung.github.io>`_ (@leewujung),
-`Kavin Nguyen <https://github.com/ngkavin>`_ (@ngkavin) and
-`Paul Robinson <https://github.com/prarobinson/>`_ (@prarobinson)
+`Wu-Jung Lee <http://leewujung.github.io>`_ (@leewujung) and
+`Kavin Nguyen <https://github.com/ngkavin>`_ (@ngkavin)
 are primary developers of this project.
 `Valentina Staneva <https://escience.washington.edu/people/valentina-staneva/>`_ (@valentina-s)
-provides consultation and also contributes to development.
-
-Other contributors include:
-`Frederic Cyr <https://github.com/cyrf0006>`_ (@cyrf0006),
-`Sven Gastauer <https://www.researchgate.net/profile/Sven_Gastauer>`_ (@SvenGastauer),
-`Marian Peña <https://www.researchgate.net/profile/Marian_Pena2>`_ (@marianpena),
-`Mark Langhirt <https://www.linkedin.com/in/mark-langhirt-7b33ba80>`_ (@bnwkeys),
-`Erin LaBrecque <https://www.linkedin.com/in/erin-labrecque/>`_ (@erinann),
-`Emma Ozanich <https://www.linkedin.com/in/emma-reeves-ozanich-b8671938/>`_ (@emma-ozanich),
-`Aaron Marburg <http://apl.uw.edu/people/profile.php?last_name=Marburg&first_name=Aaron>`_ (@amarburg)
+provides consultation and also contributes to development. Other contributors are listed `here <echopype documentation>`_.
 
 We thank Dave Billenness of ASL Environmental Sciences for
 providing the AZFP Matlab Toolbox as reference for our
 development of AZFP support in echopype.
+We also thank `Rick Towler <https://github.com/rhtowler>`_
+of the Alaska Fisheries Science Center
+for providing low-level file parsing routines for
+Simrad EK60 and EK80 echosounders.
 
 
 License

docs/source/data-format.rst

@@ -20,45 +20,33 @@ data analysis modules.
 Interoperable netCDF files
 ---------------------------
 
-echopype use a modified form of the `ICES SONAR-netCDF4 convention`_ as the
-interoperable data format to which all data can be converted to.
-We made critical modifications to the data storage structure so that
-the underlying computation can take full advantage of the power of
+Echopype follows the `ICES SONAR-netCDF4 convention`_ when possible
+to create an interoperable data format to which all data are converted to.
+We made modifications to the file structure in the convention so that
+the computation can take full advantage of the power of
 xarray in manipulating labelled multi-dimensional arrays.
 See `Modifications to SONAR-netCDF4`_ for details of this modification.
 
-We also consider switching to use `zarr <https://zarr.readthedocs.io/en/stable/>`_
-in the future for cloud-optimized data storage and access.
+We are also experimenting with using `zarr <https://zarr.readthedocs.io/en/stable/>`_
+for cloud-optimized data storage and access.
+It is possible to convert raw data files to .zarr files using the current
+version of echopype, but computational support for this format has not
+been thoroughly tested.
 
 .. _ICES SONAR-netCDF4 convention:
    http://www.ices.dk/sites/pub/Publication%20Reports/Cooperative%20Research%20Report%20(CRR)/CRR341.pdf
 
 
-Supported file types
---------------------
-
-echopype currently supports conversion from
-
-- ``.raw`` files generated by Simrad's EK60 echosounder
-- ``.01A`` files generated by ASL Environmental Sciences' AZFP echosounder
-
-Support for conversion from other sonar data files will be added in future release.
-File types included in the near-term plan are:
-
-- ``.raw`` files from Simrad's EK80 broadband echosounder
-- *raw beam* data from common Acoustic Doppler Current Profilers (ADCPs)
-
-We also consider support for multibeam sonar data in the future.
-
-
 Modifications to SONAR-netCDF4
 ------------------------------
-echopype is designed to handle multi-dimensional labelled data sets
-efficiently. Therefore, we store backscatter data (the echoes) from
+Echopype is designed to handle multi-dimensional labelled data sets
+efficiently, using ``xarray`` under the hood.
+Therefore, we store backscatter data (the echoes) from
 different frequency channels in a multi-dimensional array under a
-single ``Beam`` group within a netCDF file. Because of this change,
-all frequency-dependent parameters, such as absorption coefficients,
-are also stored as an array with a frequency coordinate.
+single ``Beam`` group within a netCDF file.
+Because of this change, all frequency-dependent parameters,
+such as absorption coefficients, sample intervals, etc.,
+are stored as an array with a frequency coordinate.
 
 .. For example:
     .. code-block:: python
@@ -73,21 +61,26 @@ are also stored as an array with a frequency coordinate.
         Out[2]:
 
 This is different from the SONAR-netCDF4 convention, in which data
-and parameters from different channels are stored in different beam
-groups under the ``Sonar`` group.
+and parameters from different frequency channels are stored in different
+beam groups under the ``Sonar`` group.
 In the convention this was designed to accommodate potential differences
-in the number of bins along range.
-However, it is more convenient to store and index data directly by the
+in the number of bins along range, or when there is a change of the
+temporal length of data collection in the middle of a file.
+
+However, it is more convenient to store and slice data directly by the
 time, range, and frequency/beam direction coordinates (see ``pandas``
 and ``xarray`` documentation for more info about coordinates and
-dimensions).
-echopype handles this by filling in the multi-dimensional array with
-``NaN`` when the number of bins along range is different across
-different channels.
+dimensions) when the data are stored in a cubic form.
+To accommodate this change, in the above two cases, echopype
+
+- handles the uneven number of data samples along range by filling in
+  ``NaN`` for the shorter channels, and
+- splits the raw data file into multiple files when there is a change of
+  the temporal length of data collection along range in the middle of a file.
 
 In addition to computational efficiency, another advantage of
-echopype's approach in restructuring the netCDF format is the enhanced
-readability of code, which make data analysis computations more
+echopype's approach in restructuring the netCDF format is to enhance
+the code readability and make data analysis computations more
 tractable. For example, to extract data from a particular frequency,
 users can simply do the following without worrying about the numerical
 sequence of the index of the selected frequency:

docs/source/index.rst

@@ -33,14 +33,15 @@ scientists to make discovery using these new data.
 Contributors
 ------------
 
-`Wu-Jung Lee <http://leewujung.github.io>`_ (@leewujung)
-and `Kavin Nguyen <https://github.com/ngkavin>`_ (@ngkavin)
-are the primary developers of this project.
+`Wu-Jung Lee <http://leewujung.github.io>`_ (@leewujung) and
+`Kavin Nguyen <https://github.com/ngkavin>`_ (@ngkavin)
+are primary developers of this project.
 `Valentina Staneva <https://escience.washington.edu/people/valentina-staneva/>`_ (@valentina-s)
-provides consultation and also contributes to the development.
+provides consultation and also contributes to development.
 
 Other contributors include:
 `Frederic Cyr <https://github.com/cyrf0006>`_ (@cyrf0006),
+`Paul Robinson <https://github.com/prarobinson/>`_ (@prarobinson),
 `Sven Gastauer <https://www.researchgate.net/profile/Sven_Gastauer>`_ (@SvenGastauer),
 `Marian Peña <https://www.researchgate.net/profile/Marian_Pena2>`_ (@marianpena),
 `Mark Langhirt <https://www.linkedin.com/in/mark-langhirt-7b33ba80>`_ (@bnwkeys),
@@ -51,6 +52,10 @@ Other contributors include:
 We thank Dave Billenness of ASL Environmental Sciences for
 providing the AZFP Matlab Toolbox as reference for our
 development of AZFP support in echopype.
+We also thank `Rick Towler <https://github.com/rhtowler>`_
+of the Alaska Fisheries Science Center
+for providing low-level file parsing routines for
+Simrad EK60 and EK80 echosounders.
 
 
 License

docs/source/resources.rst

@@ -8,18 +8,13 @@ as well as the programming language in which it is written.
 - `EchoView <https://www.echoview.com/>`_: GUI-based proprietary software,
   current go-to tool for echosounder data analysis
 
-  - `EchoView R <https://github.com/AustralianAntarcticDivision/EchoviewR>`_:
-    interfacing R and EchoView,
-    related `paper <https://www.frontiersin.org/articles/10.3389/
-    fmars.2015.00015/full>`_
+- `EchoView R <https://github.com/AustralianAntarcticDivision/EchoviewR>`_:
+  interfacing R and EchoView, related `paper <https://www.frontiersin.org/articles/10.3389/fmars.2015.00015/full>`_
 
 - `Echogram <https://CRAN.R-project.org/package=echogram>`_: an R package for
   reading data in the `HAC <http://www.ices.dk/sites/pub/Publication%20Reports/
   Cooperative%20Research%20Report%20%28CRR%29/crr278/crr278.pdf>`_ format
 
-- `PyEchoLab <https://github.com/CI-CMG/pyEcholab>`_:
-  an open-source Python package based solely on the NumPy library
-
 - `ESP3 <https://bitbucket.org/echoanalysis/esp3/overview>`_:
   a Matlab toolbox for sonar data analysis.
 
@@ -28,7 +23,10 @@ as well as the programming language in which it is written.
 - `MATECHO <https://usermanual.wiki/Pdf/MatechoUserManual18052017.963673607.pdf>`_:
   requires MATLAB and Movies3D
 
-  - `Movies3D <https://forge.ifremer.fr/plugins/mediawiki/wiki/movies3d/index.php/Accueil>`_:
-    reads and writes HAC files and is capable of joining EK60 and ME70 data
+- `Movies3D <https://forge.ifremer.fr/plugins/mediawiki/wiki/movies3d/index.php/Accueil>`_:
+  reads and writes HAC files and is capable of joining EK60 and ME70 data
+
+- `PyEchoLab <https://github.com/CI-CMG/pyEcholab>`_:
+  an open-source Python package based solely on the NumPy library
 
 - `Sonar4 and Sonar5-Pro <http://folk.uio.no/hbalk/sonar4_5/index.htm>`_