diff --git a/README.md b/README.md
index 7a21c62..b3144cf 100644
--- a/README.md
+++ b/README.md
@@ -1,75 +1,101 @@
-# echoflow
+## Echoflow: Streamlined Data Pipeline Orchestration
 
-**NOTE: This project is currently under heavy development,
-and has not been tested or deployed on the cloud**
+Welcome to **Echoflow**! Echoflow is a powerful data pipeline orchestration tool designed to simplify and enhance the execution of data processing tasks. Leveraging the capabilities of [Prefect 2.0](https://www.prefect.io/) and YAML configuration files, Echoflow caters to the needs of scientific research and data analysis. It provides an efficient way to define, configure, and execute complex data processing workflows.
 
-Sonar conversion pipeline tool with echopype.
-This tool allows for users to quickly setup sonar data processing pipelines,
-and deploy it locally or on the cloud. It uses [Prefect 2.0](https://www.prefect.io/),
-a data workflow orchestration tool to run these flows on various platforms.
+Echoflow integrates with **echopype**, a renowned package for sonar data analysis, to provide a versatile solution for researchers, analysts, and engineers. With Echoflow, users can seamlessly process and analyze sonar data using a modular and user-friendly approach.
 
-## Development
 
-To develop the code, simply install the package in a python virtual environment in editable mode.
+# Getting Started with Echoflow
+
+This guide will walk you through the initial steps to set up and run your Echoflow pipelines.
+
+## 1. Create a Virtual Environment
+
+To keep your Echoflow environment isolated, it's recommended to create a virtual environment using Conda or Python's built-in `venv` module. Here's an example using Conda:
+
+```bash
+conda create --name echoflow-env
+conda activate echoflow-env
+```
+
+Or, using Python's venv:
+
+```bash
+python -m venv echoflow-env
+source echoflow-env/bin/activate  # On Windows, use `echoflow-env\Scripts\activate`
+```
+
+## 2. Clone the Project
+Now that you have a virtual environment set up, you can clone the Echoflow project repository to your local machine using the following command:
+
+```bash
+git clone <repository_url>
+```
+
+## 3. Install the Package
+Navigate to the project directory you've just cloned and install the Echoflow package. The -e flag is crucial as it enables editable mode, which is especially helpful during development and testing. Now, take a moment and let the echoflow do its thing while you enjoy your coffee.
+
+```bash
+cd <project_directory>
+pip install -e .
+```
+
+## 4. Echoflow and Prefect Initialization
+
+To kickstart your journey with Echoflow and Prefect, follow these simple initialization steps:
+
+### 4.1 Initializing Echoflow
+Begin by initializing Echoflow with the following command:
 
 ```bash
-pip install -e .[all]
+echoflow init
 ```
+This command sets up the groundwork for your Echoflow environment, preparing it for seamless usage.
 
-This will install all of the dependencies that the package need.
-
-**Check out the [Hake Flow Demo](./notebooks/HakeFlowDemo.ipynb) notebook to get started.**
-
-## Package structure
-
-All of the code lives in a directory called [echoflow](./echoflow/).
-
-Under that directory, there are currently 4 main subdirectory:
-
-- [settings](./echoflow/settings/): This is where pipeline configurations object models are found,
-as well as a home for any package configurations.
-  - [models](./echoflow/settings/models/): This sub-directory to `settings` contains [pydantic](https://docs.pydantic.dev/) models to validate the configuration file specified by the user.
-  This can look like below in [YAML](https://yaml.org/) format.
-
-    ```yaml
-    name: Bell_M._Shimada-SH1707-EK60
-    sonar_model: EK60
-    raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6})
-    args:
-      urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw
-      # Set default parameter values as found in urlpath
-      parameters:
-        ship_name: Bell_M._Shimada
-        survey_name: SH1707
-        sonar_model: EK60
-      storage_options:
-        anon: true
-    transect:
-        # Transect file spec
-        # can be either single or multiple files
-        file: ./hake_transects_2017.zip
-    output:
-      urlpath: ./combined_files
-      overwrite: true
-    ```
-
-    This yaml file turns into a `MainConfig` object that looks like:
-
-    ```python
-    MainConfig(name='Bell_M._Shimada-SH1707-EK60', sonar_model='EK60', raw_regex='(.*)-?D(?P<date>\\w{1,8})-T(?P<time>\\w{1,6})', args=Args(urlpath='s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw', parameters={'ship_name': 'Bell_M._Shimada', 'survey_name': 'SH1707', 'sonar_model': 'EK60'}, storage_options={'anon': True}, transect=Transect(file='./hake_transects_2017.zip', storage_options={})), output=Output(urlpath='./combined_files', storage_options={}, overwrite=True), echopype=None)
-    ```
-
-- [stages](./echoflow/stages/): Within this directory lives the code for various stages within the sonar data processing pipeline, which is currently sketched out and discussed [here](https://github.com/uw-echospace/data-processing-levels/blob/main/discussion_2022-07-12.md).
-- [subflows](./echoflow/subflows/): Subflows contains flows that support processing level flows.
-Essentially this is the individual smaller flows that need to run within a data processing level.
-  - Currently, each subflow is a directory that contains the following python files:
-    - `flows.py`: Code regarding to flow lives here
-    - `tasks.py`: Code regarding to task lives here
-    - `utils.py`: Code that is used for utility functions lives here
-    - `__init__.py`: File to import flow so the subflow directory can become a module and flow to be easily imported.
-- [tests](./echoflow/tests/): Tests code lives in this directory.
-
-For more details about prefect, go to their extensive [documentation](https://docs.prefect.io/).
+### 4.2 Initializing Prefect
+For Prefect, initialization involves a few extra steps, including secure authentication. Enter the following command to initiate the Prefect authentication process:
+
+- If you have a Prefect Cloud account, provide your Prefect API key to securely link your account. Type your API key when prompted and press Enter.
+
+```bash
+prefect cloud login
+```
+
+- If you don't have a Prefect Cloud account yet, you can use local prefect account. This is especially useful for those who are just starting out and want to explore Prefect without an account.
+
+```bash
+prefect profiles create echoflow-local
+```
+
+The initialization process will ensure that both Echoflow and Prefect are properly set up and ready for you to dive into your cloud-based workflows.
+
+## 5. Configure Blocks
+Echoflow utilizes the concept of [blocks](./docs/configuration/blocks.md) which are secure containers for storing credentials and sensitive data. If you're running the entire flow locally, feel free to bypass this step.To set up your cloud credentials, configure blocks according to your cloud provider. For detailed instructions, refer to the [Blocks Configuration Guide](./docs/configuration/blocks.md#creating-credential-blocks).
+
+## 6. Edit the Pipeline Configuration
+Open the [pipeline.yaml](./docs/configuration/pipeline.md) file. This YAML configuration file defines the processes you want to execute as part of your pipeline. Customize it by adding the necessary stages and functions from echopype that you wish to run.
+
+## 7. Define Data Sources and Destinations
+Customize the [datastore.yaml](./docs/configuration/datastore.md) file to define the source and destination for your pipeline's data. This is where Echoflow will fetch and store data as it executes the pipeline.
+
+## 8. Execute the Pipeline
+You're now ready to execute your Echoflow pipeline! Use the echoflow_start function, which is a central piece of Echoflow, to kick off your pipeline. Import this function from Echoflow and provide the paths or URLs of the configuration files. You can also pass additional options or storage options as needed. Here's an example:
+
+Customize the paths, block name, storage type, and options based on your requirements.
+
+
+```python
+from echoflow import echoflow_start, StorageType, load_block
+
+dataset_config = # url or path of datastore.yaml
+pipeline_config = # url or path of pipeline.yaml
+logfile_config = # url or path of logging.yaml (Optional)
+
+aws = load_block(name="<block_name>", type=<StorageType>)
+
+options = {"storage_options_override": False} # Enabling this assigns the block for universal use, avoiding the need for repetitive configurations when employing a single credential block throughout the application.
+data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, logging_config=logfile_config, storage_options=aws, options=options)
+```
 
 ## License
 
diff --git a/deployment/deploy_echoflow_worker.sh b/deployment/deploy_echoflow_worker.sh
new file mode 100644
index 0000000..efbc433
--- /dev/null
+++ b/deployment/deploy_echoflow_worker.sh
@@ -0,0 +1,56 @@
+#!/bin/bash
+
+# Step 1: Create a Python Virtual Environment
+python3 -m venv $HOME/env/echoflow-prod
+source $HOME/env/echoflow-prod/bin/activate
+
+# Step 2: Clone the Echoflow Repository
+cd $HOME/
+git clone https://github.com/OSOceanAcoustics/echoflow.git
+cd $HOME/echoflow
+
+# Step 3: Checkout the Dev Branch and Update (Optional) - Skip if using Prod/main branch
+git checkout dev
+git pull origin dev
+
+# Step 4: Install the Echoflow Project in Editable Mode
+pip install -e .
+
+# Step 5: Log in to Prefect Cloud and Set Your API Key - Change to step 5b if using prefect locally
+echo "Enter Prefect API key: "
+read prefectKey
+prefect cloud login -k $prefectKey
+
+# Step 5b: Setup prefect locally
+# prefect profile create echoflow-local
+
+# Step 6: Set Up the Prefect Worker as a Systemd Service
+echo "Enter Work Pool Name: "
+read workPool
+cd /etc/systemd/system
+
+# Create and edit the prefect-worker.service file
+sudo cat <<EOL > prefect-worker.service
+[Unit]
+Description=Prefect-Worker
+
+[Service]
+User=$(whoami)
+WorkingDirectory=$HOME/echoflow
+ExecStart=$(which prefect) agent start --pool $workPool
+Restart=always
+
+[Install]
+WantedBy=multi-user.target
+EOL
+
+# Step 7: Restart to to make systemd aware of the new service
+sudo systemctl daemon-reload
+
+# Optionally, enable the service to start at boot
+sudo systemctl enable prefect-worker.service
+
+# Step 8: Start the Prefect Worker Service
+sudo systemctl start prefect-worker.service
+
+echo "Setup completed. The Echoflow worker is now running. Send tasks to $workPool using Prefect UI or CLI."
\ No newline at end of file
diff --git a/docs/configuration/blocks.md b/docs/configuration/blocks.md
new file mode 100644
index 0000000..b471f77
--- /dev/null
+++ b/docs/configuration/blocks.md
@@ -0,0 +1,105 @@
+# Echoflow Configuration and Credential Blocks
+
+Echoflow leverages the concept of "blocks" from Prefect, which serve as containers for storing various types of data, including credentials and sensitive information. Currently, Echoflow supports two types of blocks: Azure Cosmos DB Credentials Block and AWS Credentials Block. These blocks allow you to securely store sensitive data while benefiting from Prefect's robust integration capabilities.
+
+For a deeper understanding of blocks, you can refer to the [Prefect documentation](https://docs.prefect.io/2.11.5/concepts/blocks/).
+
+## Types of Blocks in Echoflow
+
+In the context of Echoflow, there are two main categories of blocks:
+
+### 1. Echoflow Configuration Blocks
+
+These blocks serve as repositories for references to credential blocks, as well as repositories for the various Prefect profiles that have been established using Echoflow's functions.
+
+### 2. Credential Blocks
+
+Credential blocks store sensitive information, such as authentication keys and tokens, securely. Echoflow integrates with Prefect's capabilities to ensure that sensitive data is protected.
+
+## Creating Credential Blocks
+
+Credential blocks can be conveniently created using an `.ini` file. By leveraging Prefect's integration, Echoflow ensures that the credentials stored in these blocks are handled securely. To create a credential block, you can follow these steps:
+
+1. Open the `credentials.ini` file, which is located under the `.echoflow` directory in your home directory.
+```bash
+# Terminal command
+cd ~/.echoflow
+```
+2. Place the necessary credential information within the `credentials.ini` file.
+```bash
+# Terminal command
+nano credentials.ini # Or use any of your favourite editors
+```
+3. Store the updated `.ini` file in the `.echoflow` directory, which resides in your home directory.
+4. Utilize [echoflow load-credentials](../../echoflow/stages/subflows/echoflow.py#load_credential_configuration) command to generate a new credential block, leveraging the content from the `.ini` file. 
+```bash 
+echoflow load-credentials
+```
+5. Add the name of the block in pipeline or datastore yaml configuration files under `storage_options` section with the appropriate storage type (refer [StorageType](../../echoflow/config/models/datastore.py#StorageType)).
+
+```yaml
+# Example
+storage_options:
+  block_name: echoflow-aws-credentials  # Name of the block containing credentials
+  type: AWS  # Specify the storage type using StorageType enum
+```
+
+By providing the block name and storage type, ensure that the correct block is used for storage operations, and maintain clarity regarding the chosen storage type.
+
+Once a credential block is created, it can be managed through the Prefect Dashboard. Additionally, if needed, you can use the `echoflow load-credentials` command with the `--sync` argument to ensure your blocks stay up-to-date with any changes made in the Prefect UI. This ensures that your configurations remain accurate and aligned across the application. **It is highly recommended to create new blocks whenever possible, as modifying existing blocks can lead to data loss or conflicts.** 
+
+## Considerations When Using `echoflow load-credentials`
+
+When utilizing the `echoflow load-credentials` command, be aware of the following considerations:
+
+- **Overwriting Values**: When using `echoflow load-credentials`, all the values from the `.ini` file will be written to the credential block, potentially overwriting existing values. Exercise caution when using this command to prevent unintentional data loss.
+- **Creating New Blocks**: To maintain data integrity and security, it's advised to create new blocks rather than modifying existing ones. If editing an existing block becomes necessary, it should be done through the Prefect Dashboard.
+- **Sync Argument**: The `--sync` argument is available in the `echoflow load-credentials` command. When set, this option syncs the credential block updates with the Prefect UI. This feature facilitates the seamless management of blocks through the dashboard, enhancing collaboration and control over credentials.
+
+By adhering to these guidelines, you can ensure the secure management of sensitive information while effectively configuring and utilizing Echoflow within your projects.
+
+
+# Configuration File Explanation: credentials.ini
+
+This Markdown file explains the structure and contents of the `credentials.ini` configuration file.
+
+## AWS Section
+
+The `[AWS]` section contains configuration settings related to AWS credentials.
+
+- `aws_access_key_id`: Your AWS access key.
+- `aws_secret_access_key`: Your AWS secret access key.
+- `aws_session_token`: AWS session token (optional).
+- `region_name`: AWS region name.
+- `name`: Name of the AWS credentials configuration.
+- `active`: Indicates if the AWS credentials are active (True/False).
+- `options`: Additional options for AWS configuration.
+
+## AzureCosmos Section
+
+The `[AZCosmos]` section contains configuration settings related to Azure Cosmos DB credentials.
+
+- `name`: Name of the Azure Cosmos DB credentials configuration.
+- `connection_string`: Azure Cosmos DB connection string.
+- `active`: Indicates if the Azure Cosmos DB credentials are active (True/False).
+- `options`: Additional options for Azure Cosmos DB configuration.
+
+Example:
+
+```ini
+[AWS]
+aws_key = my-access-key
+aws_secret = my-secret-key
+token = my-session-token
+region = us-west-1
+name = my-aws-credentials
+active = True
+option_key = option_value
+
+[AZCosmos]
+name = my-az-cosmos-credentials
+connection_string = my-connection-string
+active = True
+option_key = option_value
+```
+
diff --git a/docs/configuration/datastore.md b/docs/configuration/datastore.md
new file mode 100644
index 0000000..c078278
--- /dev/null
+++ b/docs/configuration/datastore.md
@@ -0,0 +1,59 @@
+# Echoflow Run Configuration Documentation
+
+This document provides detailed explanations for the keys used in the provided YAML configuration used to define an Echoflow run.
+
+## Run Details
+
+- `name`: This key specifies the name of the Echoflow run. It is used to identify and label the execution of the Echoflow process.
+- `sonar_model`: This key indicates the model of the sonar used for data collection during the run.
+- `raw_regex`: This key indicates the regex to be used while parsing the source directory to match the files to be processed.
+
+## Input Arguments
+
+- `urlpath`: This key defines the source data URL pattern for accessing raw data files. The pattern can contain placeholders that will be dynamically replaced during execution.
+- `parameters`: This section holds parameters used in the source data URL. These parameters dynamically replace placeholders in the URL path.
+- `storage_options`: This section defines storage options for accessing source data. It may include settings to anonymize access to the data.
+- `transect`: This section provides information about the transect data, including the URL of the transect file and storage options.
+- `json_export`: When set to true, this key indicates that raw JSON metadata of files should be exported for processing.
+- `raw_json_path`: This key defines the path where the raw JSON metadata will be stored. It can be used to skip parsing files in the source directory and instead fetch files from this JSON.
+
+## Output Arguments
+
+- `urlpath`: This key defines the destination data URL where processed data will be stored.
+- `overwrite`: When set to true, this key specifies that the data should overwrite any existing data in the output directory.
+- `storage_options`: This section defines storage options for the destination data, which may include details such as the block name and type.
+
+## Notes
+
+- The provided configuration serves as a structured setup for executing an Echoflow run, allowing customization through the specified keys.
+- Dynamic placeholders like `ship_name`, `survey_name`, and `sonar_model` are replaced with actual values based on the context.
+
+Example:
+
+```yaml
+name: Bell_M._Shimada-SH1707-EK60 # Name of the Echoflow Run
+sonar_model: EK60 # Sonar Model
+raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6}) # Regex to parse the filenames
+args: # Input arguments
+  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw # Source data URL
+  parameters: # Source data URL parameters
+    ship_name: Bell_M._Shimada
+    survey_name: SH1707
+    sonar_model: EK60
+  storage_options: # Source data storage options
+    anon: true
+  transect: # Source data transect information
+    file: ./x0007_fileset.txt       # Transect file URL. Accepts .zip or .txt file
+    storage_options: # Transect file storage options
+      block_name: echoflow-aws-credentials # Block name. For more information on Blocks refer blocks.md
+      type: AWS # Block type 
+    default_transect_num: 1 # Set when not using a file to pass transect information
+  json_export: true # Export raw json metadata of files to be processed
+  raw_json_path: s3://echoflow-workground/combined_files/raw_json # Path to store the raw json metadata. Can also work to skip the process of parsing the files at source directory and fetch files present in this json instead.
+output: # Output arguments
+  urlpath: s3://echoflow-workground/combined_files_dask # Destination data URL parameters
+  overwrite: true # Flag to overwrite the data if present in the output directory
+  storage_options: # Destination data storage options
+    block_name: echoflow-aws-credentials
+    type: AWS
+```
diff --git a/docs/configuration/pipeline.md b/docs/configuration/pipeline.md
new file mode 100644
index 0000000..9919333
--- /dev/null
+++ b/docs/configuration/pipeline.md
@@ -0,0 +1,103 @@
+# Data Processing Pipeline Configuration Documentation
+
+This documentation outlines the YAML configuration used to define and manage the data processing pipeline. The configuration governs the execution of a series of stages, each responsible for specific processing tasks.
+
+## Active Recipe
+
+The `active_recipe` parameter specifies the recipe to be executed on the input data. This configuration influences the workflow by determining the specific actions taken within the pipeline.
+
+## Dask Cluster Configuration
+
+- **`use_local_dask`**: When set to `true`, initiates a local Dask cluster with three workers by default. This distributed computing framework enhances processing efficiency.
+- **`scheduler_address`**: Option to Specify the address for the Dask scheduler. Use this or `use_local_dask` to control cluster creation. For more precise control, refer to `prefect_config` to set `DaskTaskRunner`.
+
+## Pipeline Configuration
+
+The pipeline is composed of a sequence of stages, each with its own distinct attributes.
+
+### Stage Configuration
+
+- **`recipe_name`**: The name of the recipe associated with this stage.
+- **`stages`**: A list of stages the data passes through within the pipeline.
+
+### Stage Attributes
+
+Each stage contains:
+
+- **`name`**: The function to execute within this stage.
+- **`module`**: The module in which the function is located.
+- **`options`**: Configurable settings specific to the Echoflow functionality.
+
+#### Options
+
+- **`save_raw_file`**: When `true`, saves the downloaded raw file to the output directory.
+- **`use_raw_offline`**: Skips the download process, utilizing the raw file present in the output directory. Missing files are downloaded.
+- **`use_offline`**: Skips the current process if Zarr files exist in the output directory.
+- **`out_path`**: Configures the output directory for the current process.
+
+### Prefect Configuration
+
+The `prefect_config` section configures Prefect-related settings for the flow. Refer https://docs.prefect.io/2.11.5/concepts/flows/#flow-settings for all the options available for configuration.
+
+- **`retries`**: Determines the number of retries the flow attempts before transitioning to a failure state.
+- **`task_runner`**: Sets the task runner configuration for this specific stage. E.g., `DaskTaskRunner` with a designated address.
+- **`persist_result`**: When `true`, persists Prefect results. Useful for advanced Prefect configurations.
+- **`result_storage`**: Specifies the location and serializer for storing results.
+
+### External Parameters
+
+- **`external_params`**: This section allows configuring external parameters relevant to the function. Currently supports only primitive types.
+- Parameters such specific to the current process can be defined here.
+
+## Example Stages
+
+Here's an overview of some of the pipeline stages:
+
+1. **`echoflow_open_raw`**: Executes the `open_raw` function from the `echoflow.stages.subflows.open_raw` module. Allows various options and Prefect configurations.
+2. **`echoflow_combine_echodata`**: Utilizes the `combine_echodata` function from the designated module, with relevant options and configurations.
+3. **`echoflow_compute_SV`**: Executes the `compute_SV` function, supporting offline mode.
+4. **`echoflow_compute_MVBS`**: Executes the `compute_MVBS` function, supporting offline mode. External parameters like `range_meter_bin` and `ping_time_bin` can be set here.
+
+Example:
+
+```yaml
+active_recipe: standard # Specify the recipe to execute on input data
+use_local_dask: true # Spin up a local dask cluster of 3 workers
+scheduler_address: tcp://127.0.0.1:61918 # Specify scheduler address or use_local_dask to control cluster creation. For more granular control, under prefect_config, use DaskTaskRunner(address=<scheduler_address>)
+pipeline: # List of pipeline configurations; only the active_recipe will be executed.
+- recipe_name: standard # Name of the recipe
+  stages: # list of processes to execute on the data
+  - name: echoflow_open_raw # Name of the function
+    module: echoflow.stages.subflows.open_raw # Module of the function
+    options: # Echoflow configuration options
+      save_raw_file: true # Save the downloaded raw file to output directory. Refer <link> for more information on how to configure output directory.
+      use_raw_offline: true # Skip the download process and take the raw file present in the output directory. Note: Missing files will be downloaded in the output directory.
+      use_offline: true # Skip this process if zarr files are already present in the output directory.
+      out_path: ./temp_files # Configure the output directory for this process 
+    prefect_config: # Configure any prefect related settings for a flow. For an exhaustive list of configurations refer <https://docs.prefect.io/2.11.5/concepts/flows/#flow-settings>. Task based configurations are optimized and handled by echoflow 
+      retries: 3 # Number of retries before failing the flow
+      task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487) # Configure Runner setting for this specific stage
+      persist_result: true # Persist the prefect results. Note: By default the output will be stored in the output directory, this option should only be used if dealing with advanced prefect configuration and integration
+      result_storage: LocalFileSystem(basepath=my-results) # Location and type of serializer to be used for storing the result
+    external_params: # External parameters relevant to the function can be configured using below. Currently only primitive types are supported under this configuration
+      sonar_model: EK60
+      xml_path: s3//
+  - name: echoflow_combine_echodata
+    module: echoflow.stages.subflows.combine_echodata
+    options:
+      use_offline: true
+    prefect_config:
+      retries: 0
+      task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487)
+  - name: echoflow_compute_SV
+    module: echoflow.stages.subflows.compute_SV
+    options:
+      use_offline: true
+  - name: echoflow_compute_MVBS
+    module: echoflow.stages.subflows.compute_MVBS
+    options:
+      use_offline: true
+    external_params:
+      range_meter_bin: 20 
+      ping_time_bin: 20S
+```
\ No newline at end of file
diff --git a/docs/setup.md b/docs/setup.md
new file mode 100644
index 0000000..b99bdbc
--- /dev/null
+++ b/docs/setup.md
@@ -0,0 +1,39 @@
+# Echoflow
+
+## Example notebook : [FlowDemo](FlowDemo.ipynb)
+
+## Pre-requisite:
+### Files :
+1. [pipeline.yaml](/echoflow/config/pipeline.yaml)
+2. [datastore.yaml](/echoflow/config/datastore.yaml)
+3. logging_config.yaml (optional; still in progress for setting up external logging)
+### Dataset :
+- Hake.zip (contains two transects: x0007_fileset and x00010_fileset)
+- Transects contain files from Summer 2017 for Summer 2017 for 
+    - survey ship : Bell_M._Shimada
+    - survey name : SH1707
+    - sonar model : EK60
+
+## The workflow if executed with current configuration will execute the following steps:
+
+### 1. Trigger pipeline :
+- Parses the YAML configuration into a dictionary and performs basic validation checks.
+### 2. Initialization Flow :
+- Parses the datastore YAML to fetch all the raw files defined under the URL path corresponding to the specified transect.
+- Sequentially executes the functions defined in pipeline.yaml for the collected list of files.
+### 3. Open Raw :
+- Processes a list of raw file URLs concurrently, performing the following actions for each file:
+    * Downloads the raw file to the designated output path as specified in pipeline.yaml or creates a temporary folder if preferred.
+    * Converts the raw file to the Zarr format and stores it in the output folder specified in datastore.yaml.
+    * Deletes the temporary Zarr and raw files based on the defined retention settings.
+    * Returns the echodata object.
+ 
+### 4. Combine Echodata :
+- Combines a list of echodata objects based on the transect defined in datastore.yaml.
+- Returns the combined echodata object.
+### 5. Compute Sv :
+- Calculates Sv by processing a list of echodata objects, one per transect.
+- Returns an Xarray Dataset.
+### 6. Compute MVBS :
+- Computes MVBS using the Sv values obtained from the previous step.
+- Utilizes external binning parameters specified in pipeline.yaml.
diff --git a/echoflow/__init__.py b/echoflow/__init__.py
index 1fb1194..07446c6 100644
--- a/echoflow/__init__.py
+++ b/echoflow/__init__.py
@@ -1,5 +1,15 @@
 from pkg_resources import DistributionNotFound, get_distribution
 
+from . import echoflow_cli
+from .models.datastore import StorageType
+from .stages.echoflow import (echoflow_config_AWS,
+                                       echoflow_config_AZ_cosmos,
+                                       echoflow_create_prefect_profile,
+                                       echoflow_start, get_active_profile,
+                                       load_profile)
+from .utils.config_utils import extract_fs, glob_url, load_block
+from .utils.file_utils import get_ed_list, get_last_run_output, get_zarr_list
+
 try:
     VERSION = get_distribution(__name__).version
 except DistributionNotFound:  # pragma: no cover
@@ -12,3 +22,20 @@
             "use the PyPI ones."
         )
 __version__ = VERSION
+
+__all__ = [
+    "echoflow_start",
+    "echoflow_create_prefect_profile",
+    "load_profile",
+    "echoflow_config_AWS",
+    "echoflow_config_AZ_cosmos",
+    "get_active_profile",
+    "StorageType",
+    "load_block",
+    "echoflow_cli",
+    "extract_fs", 
+    "glob_url",
+    "get_last_run_output",
+    "get_ed_list", 
+    "get_zarr_list"
+]
diff --git a/echoflow/settings/__init__.py b/echoflow/aspects/__init__.py
similarity index 100%
rename from echoflow/settings/__init__.py
rename to echoflow/aspects/__init__.py
diff --git a/echoflow/aspects/echoflow_aspect.py b/echoflow/aspects/echoflow_aspect.py
new file mode 100644
index 0000000..7c084a6
--- /dev/null
+++ b/echoflow/aspects/echoflow_aspect.py
@@ -0,0 +1,101 @@
+"""
+Module containing a decorator for logging and aspect-oriented programming in echoflow.
+
+This module provides a decorator called `echoflow` which is used for logging and
+aspect-oriented programming in the echoflow framework.
+
+Functions:
+    echoflow(processing_stage: str = "DEFAULT", type: str = "TASK"):
+        A decorator used to log function entry and exit, as well as modify arguments
+        based on the execution context.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import functools
+import logging
+
+from echoflow.utils.rest_utils import get_last_run_history
+
+from .singleton_echoflow import Singleton_Echoflow
+
+
+def echoflow(processing_stage: str = "DEFAULT", type: str = "TASK"):
+    """
+    Decorator for logging and aspect-oriented programming in the echoflow framework.
+
+    This decorator is used to log the entry and exit of a decorated function, as well as
+    modify the function's arguments based on the execution context. It supports two types
+    of execution: "TASK" and "FLOW". For a "FLOW" execution, if a processing stage other
+    than the default is provided, it fetches the history of the last run for that stage
+    and modifies the arguments accordingly.
+
+    Args:
+        processing_stage (str, optional): The processing stage identifier. Defaults to "DEFAULT".
+        type (str, optional): The type of execution, either "TASK" or "FLOW". Defaults to "TASK".
+
+    Returns:
+        function: The decorated function.
+
+    Example:
+        @echoflow(processing_stage="StageA", type="FLOW")
+        def my_function(arg1, arg2):
+            # Function code here
+            pass
+    """
+    def decorator(func=None):
+        def before_function_call(
+            gea: Singleton_Echoflow, type: str, processing_stage: str, *args, **kwargs
+        ):
+            # mod_args = []
+            # if type == "FLOW" and processing_stage != "DEFAULT":
+            #     # run_history = get_last_run_history(name=processing_stage)
+            #     mod_args = [arg for arg in args]
+            # else:
+            #     mod_args = [arg for arg in args]
+                
+            if type != "TASK":
+                gea.log(
+                    msg=f"Entering with memory at {gea.log_memory_usage()}: ",
+                    extra={"mod_name": func.__module__,
+                        "func_name": func.__name__},
+                    level=logging.DEBUG,
+                )
+            
+            if type == "FLOW" and processing_stage!= "DEFAULT":
+                prev_stage = args[-1]
+                stage = args[-2]
+                if prev_stage is not None:
+                    possible_functions = gea.get_possible_next_functions(prev_stage.name)
+                    if stage.name not in possible_functions:
+                        raise ValueError(stage.name, " cannot be executed after ", prev_stage.name, ". Please consider configuring rules if this validation is wrong.")
+
+
+        def after_function_call(gea: Singleton_Echoflow, *args, **kwargs):
+            if type != "TASK":
+                gea.log(
+                    msg=f"Exiting with memory at {gea.log_memory_usage()}: ",
+                    extra={"mod_name": func.__module__,
+                        "func_name": func.__name__},
+                    level=logging.DEBUG,
+                )
+
+        @functools.wraps(func)
+        def wrapper(*args, **kwargs):
+            gea = Singleton_Echoflow.get_instance()
+            try:
+                before_function_call(
+                    gea, type, processing_stage, *args, **kwargs)
+                result = func(*args, **kwargs)
+                after_function_call(gea, *args, **kwargs)
+                return result
+            except Exception as e:
+                if type == "TASK":
+                    return {'error': True, 'error_desc': e}
+                else:
+                    raise e
+
+        return wrapper
+
+    return decorator
diff --git a/echoflow/aspects/singleton_echoflow.py b/echoflow/aspects/singleton_echoflow.py
new file mode 100644
index 0000000..fcf39e6
--- /dev/null
+++ b/echoflow/aspects/singleton_echoflow.py
@@ -0,0 +1,236 @@
+"""
+Module containing the Singleton_Echoflow class and utility methods.
+
+This module defines the `Singleton_Echoflow` class, which implements a Singleton
+pattern for managing echoflow-related configurations, logging, and database operations.
+It also provides utility methods for logging, managing processes, inserting log data,
+and logging memory usage.
+
+Classes:
+    Singleton_Echoflow: Singleton class for managing echoflow configurations and operations.
+
+Functions:
+    - get_instance(self) -> "Singleton_Echoflow": Get the instance of Singleton_Echoflow.
+    - get_logger(self) -> "logging.Logger": Get the logger instance.
+    - logger_init(self, log_file: Union[Dict[str, str], str]) -> None:
+        Initialize the logger based on the provided log configuration.
+    - setup_echoflow_db(self): Setup the echoflow database and return a DB_Log instance.
+    - log(self, msg, level, extra): Log a message at the specified log levels.
+    - add_new_process(self, process: Process, name: str): Add a new process to the DB_Log instance.
+    - insert_log_data(self): Insert or update log data in the database.
+    - log_memory_usage(self): Get the memory usage of the current process.
+    - get_possible_next_functions(self, function_name): Retrieve a list of functions that can potentially be executed next
+    - load(self): Load predefined rules from a file and establish dependency relationships
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import logging
+import os
+from datetime import datetime
+from typing import Dict, Union
+
+import psutil
+import yaml
+
+from echoflow.models.datastore import Dataset
+from echoflow.models.db_log_model import DB_Log, Log_Data, Process
+from echoflow.models.pipeline import Recipe
+from echoflow.rule_engine.dependency_engine import DependencyEngine
+from echoflow.utils.databse_utils import (create_log_table, get_connection,
+                                          get_last_log,
+                                          insert_log_data_by_conn,
+                                          update_log_data_by_conn)
+
+
+class Singleton_Echoflow:
+    _instance: "Singleton_Echoflow" = None
+
+    rengine: DependencyEngine = DependencyEngine()
+    pipeline: Recipe
+    dataset: Dataset
+    db_log: DB_Log
+    logger: logging.Logger = None
+    log_level: int  = 0
+
+    def __new__(
+        cls,
+        log_file: Union[Dict[str, str], str] = None,
+        pipeline: Recipe = None,
+        dataset: Dataset = None 
+    ) -> "Singleton_Echoflow":
+        
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+            if log_file is not None:
+                cls._instance.logger = cls._instance.logger_init(log_file)
+                cls._instance.log_level = cls._instance.logger.level
+                
+        cls._instance.pipeline = pipeline
+        cls._instance.dataset = dataset
+        # cls._instance.db_log = cls._instance.setup_echoflow_db()
+        cls._instance.rengine = cls._instance.load()
+        return cls._instance
+
+    @classmethod
+    def get_instance(self) -> "Singleton_Echoflow":
+        """
+        Get the instance of the Singleton_Echoflow class.
+
+        Returns:
+            Singleton_Echoflow: The Singleton_Echoflow instance.
+        """
+        return self._instance
+    
+    @classmethod
+    def get_logger(self) -> "logging.Logger":
+        """
+        Get the logger instance.
+
+        Returns:
+            logging.Logger: The logger instance.
+        """
+        return self.logger
+
+    def logger_init(self, log_file: Union[Dict[str, str], str]) -> None:
+        """
+        Initialize the logger based on the provided log configuration.
+
+        Args:
+            log_file (Union[Dict[str, str], str]): The log configuration file or dictionary.
+        """
+        if type(log_file) == str:
+            with open(log_file, "r") as file:
+                logging_config_dict = yaml.safe_load(file.read())
+                logging.config.dictConfig(logging_config_dict)
+        else:
+            logging.config.dictConfig(log_file)
+        return logging.getLogger("echoflow")
+
+    def setup_echoflow_db(self):
+        """
+        Setup the echoflow database and return a DB_Log instance.
+
+        Returns:
+            DB_Log: The initialized DB_Log instance.
+        """
+        last_log = DB_Log()
+        try:
+            conn = get_connection(self.pipeline.database_path)
+            create_log_table(conn=conn)
+
+            if self.pipeline.use_previous_recipe == True:
+                last_log = get_last_log(conn)
+
+            last_log.start_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
+            last_log.run_id = None
+
+            return last_log
+        except Exception as e:
+            print("Failed to create Database with below error")
+            print(e)
+
+    def log(self, msg, level, extra):
+        """
+        Log a message at the specified log levels.
+
+        Args:
+            msg: The message to be logged.
+            level: The log level.
+            extra: Extra information to include in the log record.
+        """
+        if self.logger is not None:
+            self.logger.log(level=logging.DEBUG, msg=msg, extra=extra)
+            self.logger.log(level=logging.ERROR, msg=msg, extra=extra)
+            self.logger.log(level=logging.INFO, msg=msg, extra=extra)
+            self.logger.log(level=logging.WARNING, msg=msg, extra=extra)
+
+    def add_new_process(self, process: Process, name: str):
+        """
+        Add a new process to the DB_Log instance.
+
+        Args:
+            process (Process): The process to be added.
+            name (str): The name of the process.
+        """
+        process.end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
+        if name not in self.db_log.data:
+            self.db_log.data[name] = Log_Data(name=name)
+        self.db_log.data[name].process_stack.append(process)
+
+    def insert_log_data(self):
+        """
+        Insert or update log data in the database.
+
+        Returns:
+            int: The inserted or updated log entry ID.
+        """
+        self.db_log.end_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
+        try:
+            conn = get_connection(self._instance.pipeline.database_path)
+            if self.db_log.run_id is None:
+                return insert_log_data_by_conn(conn, log=self.db_log)
+            else:
+                update_log_data_by_conn(conn, log=self.db_log)
+                return self.db_log.run_id
+        except Exception as e:
+            print(e)
+        finally:
+            conn.close()
+        
+
+    def log_memory_usage(self):
+        """
+        Get the memory usage of the current process.
+
+        Returns:
+            int: The memory usage in bytes.
+        """
+        process = psutil.Process()
+        mem = process.memory_info().rss
+        return mem
+    
+    def get_possible_next_functions(self, function_name: str):
+        """
+        Get Possible Next Functions
+
+        Retrieve a list of functions that can potentially be executed next, according
+        to the defined rules and dependencies.
+
+        Args:
+            function_name (str): The name of the function that has completed.
+
+        Returns:
+            list: A list of functions that can be executed next.
+
+        Example:
+            # Get possible next functions after executing "data_download"
+            next_functions = executor.get_possible_next_functions(function_name="data_download")
+        """
+        return self.rengine.get_possible_next_functions(function_name)
+        
+    def load(self):
+        """
+        Load Rules and Dependencies
+
+        Load predefined rules from a file and establish dependency relationships
+        between functions using the DependencyEngine.
+
+        Returns:
+            DependencyEngine: The initialized DependencyEngine with loaded rules.
+
+        Example:
+            # Load rules from file and establish dependencies
+            loaded_rengine = executor.load()
+        """
+        rengine: DependencyEngine = DependencyEngine()
+        home_directory = os.path.expanduser("~")
+        config_directory = os.path.join(home_directory, ".echoflow")
+        rules_file_path = os.path.join(config_directory, "echoflow_rules.txt")
+
+        with open(rules_file_path, 'r') as file:
+            for line in file:
+                target, dependent = line.strip().split(':')
+                rengine.add_dependency(target_function=target, dependent_function=dependent)
+        return rengine
diff --git a/echoflow/config/__init__.py b/echoflow/config/__init__.py
new file mode 100644
index 0000000..0650744
--- /dev/null
+++ b/echoflow/config/__init__.py
@@ -0,0 +1 @@
+from . import models
diff --git a/echoflow/config/datastore.yaml b/echoflow/config/datastore.yaml
new file mode 100644
index 0000000..4ba95be
--- /dev/null
+++ b/echoflow/config/datastore.yaml
@@ -0,0 +1,26 @@
+name: Bell_M._Shimada-SH1707-EK60 # Name of the Echoflow Run
+sonar_model: EK60 # Sonar Model
+raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6}) # Regex to parse the filenames
+args: # Input arguments
+  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw # Source data URL
+  parameters: # Source data URL parameters
+    ship_name: Bell_M._Shimada
+    survey_name: SH1707
+    sonar_model: EK60
+  storage_options: # Source data storage options
+    anon: true
+  transect: # Source data transect information
+    file: ./x0007_fileset.txt # Transect file URL. Accepts .zip or .txt file. File name Must follow r"x(?P<transect_num>\d+)" regex or set default_transect_num to process without grouping. To process all the files at the source, remove the transect section from this yaml.
+    storage_options: # Transect file storage options
+      block_name: echoflow-aws-credentials # Block name. For more information on Blocks refer blocks.md
+      type: AWS # Block type 
+  default_transect_num: 1 # Set when not using a file to pass transect information
+  json_export: true # Export raw json metadata of files to be processed
+  raw_json_path: s3://echoflow-workground/combined_files/raw_json # Path to store the raw json metadata. Can also work to skip the process of parsing the files at source directory and fetch files present in this json instead.
+output: # Output arguments
+  urlpath: s3://echoflow-workground/combined_files_dask # Destination data URL parameters
+  overwrite: true # Flag to overwrite the data if present in the output directory
+  retention: false # Deletes all the data stored while executing the pipeline
+  storage_options: # Destination data storage options
+    block_name: echoflow-aws-credentials
+    type: AWS
diff --git a/echoflow/config/logging_config.yaml b/echoflow/config/logging_config.yaml
new file mode 100644
index 0000000..079c09f
--- /dev/null
+++ b/echoflow/config/logging_config.yaml
@@ -0,0 +1,43 @@
+# version: 1
+# handlers:
+#   file:
+#     class: logging.FileHandler
+#     level: DEBUG
+#     formatter: simpleFormatter
+#     filename: demo.log
+#     mode: a  # 'a' for append, 'w' for overwrite
+# formatters:
+#   simpleFormatter:
+#     format: '%(asctime)s - %(name)s - %(levelname)s - %(message)s'
+# root:
+#   level: DEBUG
+#   handlers: [file]
+
+version: 1
+disable_existing_loggers: False
+formatters:
+  json:
+    format: "[%(asctime)s] %(process)d %(levelname)s %(mod_name)s:%(func_name)s:%(lineno)s - %(message)s"
+  plaintext:
+    format: "[%(asctime)s] %(process)d %(levelname)s %(mod_name)s:%(func_name)s:%(lineno)s - %(message)s"
+handlers:
+  logfile:
+    class: logging.handlers.RotatingFileHandler
+    formatter: plaintext
+    level: DEBUG
+    filename: echoflow.log
+    maxBytes: 1000000
+    backupCount: 3
+  # echoflow_watchtower:
+  #   class: watchtower.CloudWatchLogHandler
+  #   formatter: json
+  #   level: DEBUG
+  #   log_group_name: echoflow_logs
+  #   log_stream_name: echoflow_stream
+  #   send_interval: 10
+  #   create_log_group: False
+loggers:
+  echoflow:
+    level: DEBUG
+    propagate: False
+    handlers: [logfile]
diff --git a/echoflow/config/pipeline.yaml b/echoflow/config/pipeline.yaml
new file mode 100644
index 0000000..d9e67c6
--- /dev/null
+++ b/echoflow/config/pipeline.yaml
@@ -0,0 +1,83 @@
+active_recipe: standard # Specify the recipe to execute on input data
+use_local_dask: true # Spin up a local dask cluster of 3 workers if n_workers is not used
+n_workers: 4 # Number of workers for local dask cluster
+scheduler_address: tcp://127.0.0.1:61918 # Specify scheduler address or use_local_dask to control cluster creation. For more granular control, under prefect_config, use DaskTaskRunner(address=<scheduler_address>)
+pipeline: # List of pipeline configurations; only the active_recipe will be executed.
+- recipe_name: standard # Name of the recipe
+  stages: # list of processes to execute on the data
+  - name: echoflow_open_raw # Name of the function
+    module: echoflow.stages.subflows.open_raw # Module of the function
+    options: # Echoflow configuration options
+      save_raw_file: true # Save the downloaded raw file to output directory. Refer <link> for more information on how to configure output directory.
+      use_raw_offline: true # Skip the download process and take the raw file present in the output directory. Note: Missing files will be downloaded in the output directory.
+      use_offline: true # Skip this process if zarr files are already present in the output directory.
+      out_path: ./temp_files # Configure the output directory for this process 
+    prefect_config: # Configure any prefect related settings for a flow. For an exhaustive list of configurations refer <https://docs.prefect.io/2.11.5/concepts/flows/#flow-settings>. Task based configurations are optimized and handled by echoflow 
+      retries: 3 # Number of retries before failing the flow
+      task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487) # Configure Runner setting for this specific stage
+      persist_result: true # Persist the prefect results. Note: By default the output will be stored in the output directory, this option should only be used if dealing with advanced prefect configuration and integration
+      result_storage: LocalFileSystem(basepath=my-results) # Location and type of serializer to be used for storing the result
+    external_params: # External parameters relevant to the function can be configured using below. Currently only primitive types are supported under this configuration
+      sonar_model: EK60
+      xml_path: s3//
+  - name: echoflow_combine_echodata
+    module: echoflow.stages.subflows.combine_echodata
+    options:
+      use_offline: true
+    prefect_config:
+      retries: 0
+      task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487)
+  - name: echoflow_compute_SV
+    module: echoflow.stages.subflows.compute_SV
+    options:
+      use_offline: true
+  - name: echoflow_compute_MVBS
+    module: echoflow.stages.subflows.compute_MVBS
+    options:
+      use_offline: true
+    external_params:
+      range_meter_bin: 20 
+      ping_time_bin: 20S
+- recipe_name: target_strength
+  stages:
+  - name: echoflow_open_raw
+    module: echoflow.stages.subflows.open_raw
+    external_params:
+      sonar_model: EK60
+      xml_path: s3//
+    options:
+      save_raw_file: true
+      use_raw_offline: true
+      use_offline: true
+      # out_path: ./temp_files
+    prefect_config:
+      retries: 0
+      # task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487)
+      # persist_result: true
+      # result_storage: LocalFileSystem(basepath=my-results)
+  - name: echoflow_compute_TS
+    module: echoflow.stages.subflows.compute_TS
+    options:
+      use_offline: true
+- recipe_name: invalid
+  stages:
+  - name: echoflow_open_raw
+    module: echoflow.stages.subflows.open_raw
+    external_params:
+      sonar_model: EK60
+      xml_path: s3//
+    options:
+      save_raw_file: true
+      use_raw_offline: true
+      use_offline: true
+      # out_path: ./temp_files
+    prefect_config:
+      retries: 0
+      # task_runner: DaskTaskRunner(address=tcp://127.0.0.1:59487)
+      # persist_result: true
+      # result_storage: LocalFileSystem(basepath=my-results)
+  - name: echoflow_compute_MVBS
+    module: echoflow.stages.subflows.compute_MVBS
+    options:
+      use_offline: true
+
diff --git a/echoflow/echoflow_cli.py b/echoflow/echoflow_cli.py
new file mode 100644
index 0000000..8b1fe6d
--- /dev/null
+++ b/echoflow/echoflow_cli.py
@@ -0,0 +1,162 @@
+"""
+Echoflow CLI
+
+This module provides a command-line interface (CLI) for managing Echoflow credentials and configurations.
+You can use this CLI to generate, load, and manage credential blocks stored in the `credentials.ini` file.
+
+Usage:
+- To load credentials and create credential blocks:
+  echoflow load-credentials [--sync]
+- To initialize an empty `credentials.ini` file:
+  echoflow init
+
+Subcommands:
+- `load-credentials`: Load credentials from the `.ini` file and create corresponding credential blocks.
+  Options:
+  --sync: If provided, syncs blocks updated using the Prefect UI.
+
+- `init`: Initializes an empty `.ini` file for Echoflow credentials configuration.
+
+Example:
+- Load credentials and sync blocks:
+  echoflow load-credentials --sync
+
+- Initialize an empty `.ini` file:
+  echoflow init
+
+Note:
+- Use the appropriate subcommand to perform desired actions related to Echoflow credentials.
+- When using `load-credentials`, ensure that the `credentials.ini` file is correctly formatted with appropriate sections and keys.
+- Supported section names are "AWS" and "AZCosmos".
+"""
+
+
+import argparse
+import os
+
+from echoflow.stages.echoflow import load_credential_configuration
+
+
+def generate_ini_file():
+    """
+    Generate the credentials.ini file and configuration files for Echoflow.
+
+    This function creates a directory named `.echoflow` in the user's home directory
+    if it doesn't exist already. It then generates the `credentials.ini` file within
+    this directory, which serves as the configuration file for Echoflow credentials.
+
+    Additionally, the function configures pre-defined rules by generating an `echoflow_rules.txt`
+    file containing rule definitions that dictate the workflow execution sequence in Echoflow.
+
+    Example:
+        Calling this function will create a `.echoflow` directory in the user's home
+        directory and generate the `credentials.ini` and `echoflow_rules.txt` files within it.
+        The credentials.ini file holds Echoflow configuration details, while the echoflow_rules.txt
+        file defines the workflow execution sequence.
+
+    Rules Format:
+        The `echoflow_rules.txt` file contains lines with rule definitions. Each rule specifies
+        the execution sequence of two workflow components, separated by a colon. For example:
+        "echoflow_open_raw:echoflow_compute_SV" means `echoflow_open_raw` should be executed
+        before `echoflow_compute_SV`.
+
+    Note:
+        If the `.echoflow` directory or `credentials.ini` file already exists, this function will
+        skip their creation.
+
+    """
+
+    # Get the user's home directory and expand ~
+    home_directory = os.path.expanduser("~")
+
+    # Create the directory if it doesn't exist
+    config_directory = os.path.join(home_directory, ".echoflow")
+    os.makedirs(config_directory, exist_ok=True)
+
+    # Write the .ini file
+    ini_file_path = os.path.join(config_directory, "credentials.ini")
+    
+    if not os.path.exists(ini_file_path):
+        with open(ini_file_path, "w") as config_file:
+            config_file.write("# Credential configuration file for Echoflow\n")
+        print("Successfully created credentials.ini file under .echoflow directory.")
+    else:
+        print("credentials.ini file already exists. Skipping creation.")
+
+    print("Configuring pre-defined rules...")
+    rules_path = os.path.join(config_directory, "echoflow_rules.txt")
+    rules_file = open(rules_path, "w")
+    rules = [
+    "echoflow_open_raw:echoflow_compute_SV",
+    "echoflow_open_raw:echoflow_combine_echodata",
+    "echoflow_open_raw:echoflow_compute_TS",
+    "echoflow_combine_echodata:echoflow_compute_SV",
+    "echoflow_compute_SV:echoflow_compute_MVBS"
+    ]
+    for r in rules:
+        rules_file.write(r+"\n")
+    rules_file.close()
+
+    print("Initilization complete")
+
+def main():
+    """
+    Main entry point of the Echoflow CLI script.
+
+    This function provides subcommands for generating and managing Echoflow credentials configuration:
+
+    Subcommands:
+    - `load-credentials`: Load credentials from a configuration file and create corresponding credential blocks.
+      Options:
+      --sync: If provided, syncs blocks updated using the Prefect UI.
+
+    - `init`: Initializes an empty `.ini` file for Echoflow credentials configuration.
+
+    Example:
+    To load credentials and sync:
+    ```
+    echoflow load-credentials --sync
+    ```
+
+    To initialize an empty `.ini` file:
+    ```
+    echoflow init
+    ```
+
+    Note:
+    - Use the appropriate subcommand to perform desired actions related to Echoflow credentials.
+    - When using `load-credentials`, ensure that the `credentials.ini` file is correctly formatted with appropriate sections and keys.
+    - Supported section names are "AWS" and "AZCosmos".
+    - Any unrecognized sections will be reported.
+    """
+    parser = argparse.ArgumentParser(description="Echoflow")
+
+    subparsers = parser.add_subparsers(title="Echoflow Commands", dest="command")
+
+    load_parser = subparsers.add_parser("load-credentials", help="Load credentials from .ini file")
+    load_parser.add_argument('--sync', action='store_true', help="If provided, syncs blocks updated using Prefect Dashoard.")
+
+    run_parser = subparsers.add_parser("init", help="Initializes an empty .ini file")
+
+    args = parser.parse_args()
+    if args.command is None:
+        print("No command provided. Use 'load-credentials' or 'init'.")
+    else:
+        if args.command == "load-credentials":
+            if args.sync:
+                print("Syncing with Prefect Dashboard")
+                print("Creating credential blocks from ~/.echoflow/credentials.ini")
+                load_credential_configuration(sync=True)
+            else:
+                print("To sync with Prefect Dashboard use `--sync` option")
+                print("Creating credential blocks from ~/.echoflow/credentials.ini")
+                load_credential_configuration(sync=False)
+            print()
+        elif args.command == "init":
+            print("Initializing Echoflow")
+            generate_ini_file()
+        else:
+            print("Unknown Command")
+
+if __name__ == "__main__":
+    main()
\ No newline at end of file
diff --git a/notebooks/combined_files/.gitkeep b/echoflow/models/__init__.py
similarity index 100%
rename from notebooks/combined_files/.gitkeep
rename to echoflow/models/__init__.py
diff --git a/echoflow/models/datastore.py b/echoflow/models/datastore.py
new file mode 100644
index 0000000..c99781c
--- /dev/null
+++ b/echoflow/models/datastore.py
@@ -0,0 +1,158 @@
+"""
+Echoflow Configuration Models
+
+This module defines Pydantic models that represent different aspects of the Echoflow configuration.
+
+Classes:
+    StorageType (Enum): An enumeration representing different storage types.
+    StorageOptions (BaseModel): Model for defining storage options.
+    Parameters (BaseModel): Model for defining parameters.
+    Transect (BaseModel): Model for defining transect options.
+    Args (BaseModel): Model for defining input arguments.
+    Output (BaseModel): Model for defining output options.
+    Dataset (BaseModel): Model for defining dataset configuration.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+from enum import Enum
+from typing import Any, Dict, Optional
+
+import jinja2
+from pydantic import BaseModel
+
+
+class StorageType(Enum):
+    """
+    Enumeration for storage types.
+
+    Attributes:
+        AWS: Amazon Web Services storage type.
+        AZ: Azure storage type.
+        GCP: Google Cloud Platform storage type.
+    """
+    AWS = "AWS"
+    AZCosmos = "AZCosmos"
+    GCP = "GCP"
+    ECHOFLOW = "ECHOFLOW"
+
+
+class StorageOptions(BaseModel):
+    """
+    Model for defining storage options.
+
+    Attributes:
+        type (Optional[StorageType]): The type of storage.
+        block_name (Optional[str]): The name of the block.
+        anon (Optional[bool]): Whether to use anonymous access. Default is False.
+    """
+    type: Optional[StorageType]
+    block_name: Optional[str]
+    anon: Optional[bool] = False
+
+
+class Parameters(BaseModel):
+    """
+    Model for defining parameters.
+
+    Attributes:
+        ship_name (str): The name of the ship.
+        survey_name (str): The name of the survey.
+        sonar_model (str): The model of the sonar.
+    """
+    ship_name: str
+    survey_name: str
+    sonar_model: str
+
+
+class Transect(BaseModel):
+    """
+    Model for defining transect options.
+
+    Attributes:
+        file (Optional[str]): The file associated with the transect.
+        storage_options (Optional[StorageOptions]): Storage options for the transect.
+        storage_options_dict (Optional[Dict[str, Any]]): Additional storage options as a dictionary.
+        default_transect_num (int): Default transect number.
+    """
+    file: Optional[str]
+    storage_options: Optional[StorageOptions] = None
+    storage_options_dict: Optional[Dict[str, Any]] = {}  
+
+
+class Args(BaseModel):
+    """
+    Model for defining input arguments.
+
+    Attributes:
+        urlpath (str): The URL path.
+        parameters (Parameters): Parameters for rendering.
+        storage_options (Optional[StorageOptions]): Storage options for the arguments.
+        storage_options_dict (Optional[Dict[str, Any]]): Additional storage options as a dictionary.
+        transect (Optional[Transect]): Transect options.
+        zarr_store (Optional[str]): Zarr store information.
+        json_export (Optional[bool]): Whether to export in JSON format. Default is False.
+        raw_json_path (Optional[str]): Path for raw JSON data.
+    """
+    urlpath: str
+    parameters: Parameters
+    storage_options: Optional[StorageOptions] = None
+    storage_options_dict: Optional[Dict[str, Any]] = {}
+    transect: Optional[Transect]
+    default_transect_num: int = None
+    zarr_store: Optional[str]
+    json_export: Optional[bool] = False
+    raw_json_path: Optional[str] = None
+
+    @property
+    def rendered_path(self):
+        """
+        Rendered URL path from input parameters.
+
+        Returns:
+            str: Rendered URL path.
+        """
+        if self.parameters is not None:
+            env = jinja2.Environment()
+            template = env.from_string(self.urlpath)
+            return template.render(self.parameters)
+        return self.urlpath
+
+
+class Output(BaseModel):
+    """
+    Model for defining output options.
+
+    Attributes:
+        urlpath (str): The URL path for output.
+        retention (bool): Retains output at each stage. Works as central flag for all the processes but does not trump the `save_offline` flag.
+        overwrite (Optional[bool]): Whether to overwrite existing data. Default is True.
+        storage_options (Optional[StorageOptions]): Storage options for the output.
+        storage_options_dict (Optional[Dict[str, Any]]): Additional storage options as a dictionary.
+    """
+    urlpath: str
+    retention: bool = True 
+    overwrite: Optional[bool] = True
+    storage_options: Optional[StorageOptions] = None
+    storage_options_dict: Optional[Dict[str, Any]] = {}
+
+
+class Dataset(BaseModel):
+    """
+    Model for defining dataset configuration.
+
+    Attributes:
+        name (str): The name of the dataset.
+        sonar_model (str): The model of the sonar.
+        raw_regex (str): Regular expression for matching raw files.
+        args (Args): Input arguments for the dataset.
+        output (Output): Output options for the dataset.
+        passing_params (Optional[Dict[str, Any]]): Additional passing parameters.
+    """
+    name: str
+    sonar_model: str
+    raw_regex: str
+    args: Args
+    output: Output
+    passing_params: Optional[Dict[str, Any]]
diff --git a/echoflow/models/db_log_model.py b/echoflow/models/db_log_model.py
new file mode 100644
index 0000000..86f509d
--- /dev/null
+++ b/echoflow/models/db_log_model.py
@@ -0,0 +1,69 @@
+"""
+Logging Models for Echoflow
+
+This module defines Pydantic models for managing logging and process data in Echoflow.
+
+Classes:
+    Process (BaseModel): Model for representing a process.
+    Log_Data (BaseModel): Model for representing log data.
+    DB_Log (BaseModel): Model for managing database log.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+
+from datetime import datetime
+from typing import Dict, List, Optional
+
+from pydantic import BaseModel
+
+
+class Process(BaseModel):
+    """
+    Model for representing a process.
+
+    Attributes:
+        name (str): The name of the process.
+        start_time (str): The start time of the process. Default is the current datetime.
+        end_time (Optional[str]): The end time of the process.
+        status (bool): The status of the process. Default is False.
+        error (Optional[str]): The error message associated with the process, if any.
+    """
+    name: str
+    start_time: str = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
+    end_time: Optional[str]
+    status: bool = False
+    error: Optional[str]
+
+
+class Log_Data(BaseModel):
+    """
+    Model for representing log data.
+
+    Attributes:
+        name (str): The name of the log data.
+        process_stack (Optional[List[Process]]): A list of associated processes.
+    """
+    name: str
+    process_stack: Optional[List[Process]] = []
+
+
+class DB_Log(BaseModel):
+    """
+    Model for managing database log.
+
+    Attributes:
+        run_id (Optional[int]): The run ID of the log.
+        start_time (Optional[str]): The start time of the log. Default is the current datetime.
+        end_time (Optional[str]): The end time of the log.
+        data (Optional[Dict[str, Log_Data]]): A dictionary of log data.
+        status (str): The status of the log. Default is True.
+        error (Optional[str]): The error message associated with the log, if any.
+    """
+    run_id: Optional[int]
+    start_time: Optional[str] = datetime.now().strftime("%Y-%m-%d %H:%M:%S.%f")
+    end_time: Optional[str]
+    data: Optional[Dict[str, Log_Data]] = {}
+    status: str = True
+    error: Optional[str]
\ No newline at end of file
diff --git a/echoflow/models/echoflow_config.py b/echoflow/models/echoflow_config.py
new file mode 100644
index 0000000..192596f
--- /dev/null
+++ b/echoflow/models/echoflow_config.py
@@ -0,0 +1,91 @@
+"""
+Module containing configuration classes related to Echoflow and Prefect integration.
+
+This module defines the following configuration classes:
+    - EchoflowPrefectConfig: Configuration for integrating Echoflow with Prefect.
+    - BaseConfig: Base configuration for Echoflow.
+    - EchoflowConfig: Configuration for the Echoflow library.
+
+Classes:
+    - EchoflowPrefectConfig(Block): Configuration for integrating Echoflow with Prefect.
+    - BaseConfig(Block): Base configuration for Echoflow.
+    - EchoflowConfig(Block): Configuration for the Echoflow library.
+
+Functions:
+    No functions are defined in this module.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+from typing import Any, Dict, List, Optional
+
+from prefect.blocks.core import Block
+from pydantic import SecretStr
+
+from .datastore import StorageType
+
+
+class EchoflowPrefectConfig(Block):
+    """
+    Configuration class for integrating Echoflow with Prefect.
+
+    Attributes:
+        prefect_api_key (Optional[SecretStr]): Prefect API key.
+        prefect_account_id (Optional[SecretStr]): Prefect account ID.
+        prefect_workspace_id (Optional[SecretStr]): Prefect workspace ID.
+        profile_name (str): Profile name for local Prefect setup.
+
+    Methods:
+        get_api_url(self): Get the API URL based on configuration.
+    """
+    prefect_api_key: Optional[SecretStr] = None
+    prefect_account_id: Optional[SecretStr] = None
+    prefect_workspace_id: Optional[SecretStr] = None
+    profile_name: str = None
+
+    def get_api_url(self):
+        """
+        Get the API URL based on configuration.
+
+        Returns:
+            str: The API URL.
+        """
+        if self.prefect_api_key is not None:
+            return f"https://api.prefect.cloud/api/accounts/{ self.prefect_account_id.get_secret_value() }/workspaces/{ self.prefect_workspace_id.get_secret_value() }" 
+        elif self.profile_name == "echoflow_prefect_local":
+            return "127.0.0.1:4200"
+
+class BaseConfig(Block):
+    """
+    Base configuration class for Echoflow.
+
+    Attributes:
+        name (str): The name of the configuration.
+        type (StorageType): The type of the configuration.
+        active (Optional[bool]): Whether the configuration is active.
+        options (Optional[Dict[str, Any]]): Additional configuration options.
+
+    Methods:
+        No methods are defined in this class.
+    """
+    name: str
+    type: StorageType
+    active: Optional[bool] = False
+    options: Optional[Dict[str, Any]] = {}
+
+class EchoflowConfig(Block):
+    """
+    Configuration class for the Echoflow library.
+
+    Attributes:
+        active (Optional[str]): The active configuration.
+        prefect_configs (Optional[List[str]]): List of Prefect configuration names.
+        blocks (Optional[List[BaseConfig]]): List of BaseConfig instances.
+
+    Methods:
+        No methods are defined in this class.
+    """
+    active: Optional[str] = None
+    prefect_configs : Optional[List[str]]
+    blocks: Optional[List[BaseConfig]] = []
\ No newline at end of file
diff --git a/echoflow/models/output_model.py b/echoflow/models/output_model.py
new file mode 100644
index 0000000..bc2fcfc
--- /dev/null
+++ b/echoflow/models/output_model.py
@@ -0,0 +1,27 @@
+"""
+Output Model for Echoflow
+
+This module defines a Pydantic model for representing output data in Echoflow.
+
+Classes:
+    Output (BaseModel): Model for representing output data.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+from typing import Any, Dict
+
+from pydantic import BaseModel
+
+
+class Output(BaseModel):
+    """
+    Model for representing output data.
+
+    Attributes:
+        data (Any): The output data.
+        passing_params (Dict[str, Any]): A dictionary of passing parameters. Default is an empty dictionary.
+    """
+    data: Any
+    passing_params: Dict[str, Any] = {}
\ No newline at end of file
diff --git a/echoflow/models/pipeline.py b/echoflow/models/pipeline.py
new file mode 100644
index 0000000..7b0df3e
--- /dev/null
+++ b/echoflow/models/pipeline.py
@@ -0,0 +1,71 @@
+"""
+Pipeline Configuration Models for Echoflow
+
+This module defines Pydantic models for representing pipeline configurations in Echoflow.
+
+Classes:
+    Stage (BaseModel): Model for representing pipeline stages.
+    Pipeline (BaseModel): Model for representing pipelines.
+    Recipe (BaseModel): Model for representing recipes.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+from typing import Any, Dict, List, Optional
+
+from pydantic import BaseModel
+
+
+class Stage(BaseModel):
+    """
+    Model for representing pipeline stages.
+
+    Attributes:
+        name (str): The name of the stage.
+        module (Optional[str]): The module associated with the stage.
+        external_params (Optional[Dict[str, Any]]): External parameters for the stage.
+        options (Optional[Dict[str, Any]]): Options for the stage.
+        prefect_config (Optional[Dict[str, Any]]): Prefect configuration for the stage.
+    """
+    name: str
+    module: Optional[str]
+    external_params: Optional[Dict[str, Any]]
+    options: Optional[Dict[str, Any]]
+    prefect_config: Optional[Dict[str, Any]]
+
+class Pipeline(BaseModel):
+    """
+    Model for representing a pipeline.
+
+    Attributes:
+        recipe_name (str): The name of the recipe.
+        generate_graph (Optional[bool]): Flag to indicate whether to generate a graph. Default is False.
+        stages (List[Stage]): List of stages in the pipeline.
+    """
+    recipe_name: str
+    generate_graph: Optional[bool] = False
+    stages: List[Stage]
+
+class Recipe(BaseModel):
+    """
+    Model for representing a recipe.
+
+    Attributes:
+        active_recipe (str): The active recipe name.
+        use_local_dask (bool): Flag to indicate whether to use local Dask. Default is False.
+        n_workers (int): Number of workers to spin up for local cluster. Default is 3 
+        scheduler_address (str): The scheduler address. Default is None.
+        use_previous_recipe (Optional[bool]): Flag to indicate whether to use a previous recipe. Default is False.
+        database_path (Optional[str]): The path to the database. Default is an empty string.
+        pipeline (List[Pipeline]): List of pipelines in the recipe.
+    """
+    active_recipe: str
+    use_local_dask: bool = False
+    n_workers: int = 3
+    scheduler_address: str = None
+    use_previous_recipe: Optional[bool] = False
+    database_path: Optional[str] = ''
+    pipeline: List[Pipeline]
+
+    
diff --git a/echoflow/rule_engine/__init__.py b/echoflow/rule_engine/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/echoflow/rule_engine/dependency_engine.py b/echoflow/rule_engine/dependency_engine.py
new file mode 100644
index 0000000..8397783
--- /dev/null
+++ b/echoflow/rule_engine/dependency_engine.py
@@ -0,0 +1,72 @@
+class DependencyEngine:
+    """
+    Dependency Engine for Managing Function Dependencies
+
+    This class represents a dependency engine that helps manage dependencies between
+    functions. It allows adding dependencies between target functions and dependent
+    functions. The engine then provides a mechanism to retrieve the functions that can
+    potentially be executed next, given a current function's completion.
+
+    Usage:
+        Initialize an instance of the DependencyEngine class. Use the `add_dependency`
+        method to establish dependencies between functions. The `get_possible_next_functions`
+        method can be used to retrieve the functions that can be executed next, based on
+        the completion of a given function.
+
+    Example:
+        # Initialize the DependencyEngine
+        engine = DependencyEngine()
+
+        # Add dependencies
+        engine.add_dependency(target_function="download_data", dependent_function="preprocess_data")
+        engine.add_dependency(target_function="preprocess_data", dependent_function="analyze_data")
+
+        # Retrieve possible next functions after "download_data"
+        next_functions = engine.get_possible_next_functions(current_function="download_data")
+    """
+    def __init__(self):
+        self.dependencies = {}
+
+    def add_dependency(self, target_function, dependent_function):
+        """
+        Add a Dependency between Functions
+
+        This method adds a dependency relationship between a target function and a
+        dependent function. The dependent function is expected to be executed after
+        the successful completion of the target function.
+
+        Args:
+            target_function (str): The name of the target function.
+            dependent_function (str): The name of the dependent function.
+
+        Example:
+            # Add a dependency relationship
+            engine.add_dependency(target_function="data_download", dependent_function="data_preprocess")
+        """
+        if target_function not in self.dependencies:
+            self.dependencies[target_function] = []
+        self.dependencies[target_function].append(dependent_function)
+
+    def get_possible_next_functions(self, current_function):
+        """
+        Get Possible Next Functions
+
+        Retrieve a list of functions that can potentially be executed next, given the
+        completion of a current function.
+
+        Args:
+            current_function (str): The name of the function that has completed.
+
+        Returns:
+            list: A list of functions that can be executed next.
+
+        Example:
+            # Get possible next functions after "data_download"
+            next_functions = engine.get_possible_next_functions(current_function="data_download")
+        """
+        if current_function in self.dependencies:
+            return self.dependencies[current_function]
+        else:
+            return []
+    
+    
\ No newline at end of file
diff --git a/echoflow/rule_engine/echoflow_rules.txt b/echoflow/rule_engine/echoflow_rules.txt
new file mode 100644
index 0000000..184fdb5
--- /dev/null
+++ b/echoflow/rule_engine/echoflow_rules.txt
@@ -0,0 +1,5 @@
+echoflow_open_raw:echoflow_compute_SV
+echoflow_open_raw:echoflow_combine_echodata
+echoflow_open_raw:echoflow_compute_TS
+echoflow_combine_echodata:echoflow_compute_SV
+echoflow_compute_SV:echoflow_compute_MVBS
\ No newline at end of file
diff --git a/echoflow/settings/models/__init__.py b/echoflow/settings/models/__init__.py
deleted file mode 100644
index c76193a..0000000
--- a/echoflow/settings/models/__init__.py
+++ /dev/null
@@ -1,5 +0,0 @@
-from .raw_config import RawConfig
-from .echopype_config import EchopypeConfig
-from .main_config import MainConfig
-
-__all__ = ["MainConfig", "RawConfig", "EchopypeConfig"]
diff --git a/echoflow/settings/models/echopype_config/__init__.py b/echoflow/settings/models/echopype_config/__init__.py
deleted file mode 100644
index 4df2ecf..0000000
--- a/echoflow/settings/models/echopype_config/__init__.py
+++ /dev/null
@@ -1,9 +0,0 @@
-from typing import Optional
-
-from pydantic import BaseModel
-
-from .calibrate import Calibrate
-
-
-class EchopypeConfig(BaseModel):
-    calibrate: Optional[Calibrate]
diff --git a/echoflow/settings/models/echopype_config/calibrate.py b/echoflow/settings/models/echopype_config/calibrate.py
deleted file mode 100644
index 02fe7f6..0000000
--- a/echoflow/settings/models/echopype_config/calibrate.py
+++ /dev/null
@@ -1,14 +0,0 @@
-from typing import Dict, Any, Optional, Literal
-from pydantic import BaseModel
-
-
-class ComputeCal(BaseModel):
-    env_params: Optional[Dict[Any, Any]]
-    cal_params: Optional[Dict[Any, Any]]
-    waveform_mode: Optional[Literal["CW", "BB"]]
-    encode_mode: Optional[Literal["complex", "power"]]
-
-
-class Calibrate(BaseModel):
-    compute_Sv: Optional[ComputeCal]
-    compute_TS: Optional[ComputeCal]
diff --git a/echoflow/settings/models/main_config.py b/echoflow/settings/models/main_config.py
deleted file mode 100644
index 093ff4c..0000000
--- a/echoflow/settings/models/main_config.py
+++ /dev/null
@@ -1,8 +0,0 @@
-from typing import Optional
-
-from .raw_config import RawConfig
-from .echopype_config import EchopypeConfig
-
-
-class MainConfig(RawConfig):
-    echopype: Optional[EchopypeConfig]
diff --git a/echoflow/settings/models/raw_config.py b/echoflow/settings/models/raw_config.py
deleted file mode 100644
index a2f38ef..0000000
--- a/echoflow/settings/models/raw_config.py
+++ /dev/null
@@ -1,111 +0,0 @@
-"""raw_config.py
-
-Modules for the raw configuration models
-used within the pipeline to pass information
-between L0 to L1
-"""
-from typing import Any, Dict, Optional, Union, List
-
-import jinja2
-from pydantic import BaseModel, validator
-
-from ...utils import get_glob_parameters
-
-
-class Output(BaseModel):
-    """
-    Model for `output` field in raw configuration
-    """
-
-    urlpath: str
-    storage_options: Dict[str, Any] = {}
-    overwrite: bool = False
-
-
-class Transect(BaseModel):
-    """
-    Model for `transect` field in `args` section.
-    """
-
-    file: Union[List[str], str]
-    storage_options: Dict[str, Any] = {}
-
-    @validator("file")
-    def file_has_one_minimum(cls, v):
-        """Validator to check if file has at least one value"""
-        if isinstance(v, list):
-            if len(v) >= 1:
-                return v
-        elif v != "":
-            return v
-        raise ValueError("Please provide at least one value for file")
-
-    @validator("file")
-    def file_must_be_zip_or_text(cls, v):
-        """Validator to check that file is either .zip or .txt"""
-
-        valid_extensions = ('.zip', '.txt')
-        if isinstance(v, list):
-            if all(f.endswith(valid_extensions) for f in v):
-                return v
-        elif v.endswith(valid_extensions):
-            return v
-
-        raise ValueError(
-            "Found invalid file format. Only .zip and .txt are acceptable."
-        )
-
-
-class Args(BaseModel):
-    """
-    Model for `args` field in raw configuration
-    """
-
-    urlpath: str
-    parameters: Optional[Dict[str, Any]]
-    storage_options: Dict[str, Any] = {}
-    transect: Optional[Transect] = None
-
-    class Config:
-        validate_assignment = True
-        underscore_attrs_are_private = True
-
-    @property
-    def rendered_path(self):
-        """Rendered url path from inputs of parameters"""
-        if self.parameters is not None:
-            env = jinja2.Environment()
-            template = env.from_string(self.urlpath)
-            return template.render(**self.parameters)
-        return self.urlpath
-
-    @validator("parameters", pre=True, always=True)
-    def check_parameters(cls, glob_params, values):
-        """Check parameters to ensure that they're in `urlpath` glob string."""
-        param_values = get_glob_parameters(values.get("urlpath"))
-        if len(param_values) > 0 and glob_params is None:
-            raise ValueError(
-                "Parameters found in `urlpath` but not defined in `parameters` setting.",  # noqa
-            )
-        missing_keys = []
-        for p in param_values:
-            if p not in glob_params:
-                missing_keys.append(p)
-
-        if len(missing_keys) > 0:
-            raise ValueError(
-                f"{', '.join(missing_keys)} is/are missing from parameters."
-            )
-        return glob_params
-
-
-class RawConfig(BaseModel):
-    """
-    Model for raw configurations from yaml
-    """
-
-    name: str
-    sonar_model: str
-    raw_regex: str
-    args: Args
-    output: Output
diff --git a/echoflow/stages/echoflow.py b/echoflow/stages/echoflow.py
new file mode 100644
index 0000000..70172ed
--- /dev/null
+++ b/echoflow/stages/echoflow.py
@@ -0,0 +1,567 @@
+"""
+Echoflow Utility Functions
+
+This module provides utility functions for configuring Prefect profiles, checking internet connectivity,
+and managing Echoflow configurations.
+
+Classes:
+    None
+
+Functions:
+    check_internet_connection(host="8.8.8.8", port=53, timeout=5)
+    echoflow_create_prefect_profile(name: str, api_key: str = None, workspace_id: str = None, account_id: str = None, set_active: bool = True)
+    load_profile(name: str)
+    get_active_profile()
+    echoflow_start(dataset_config: Union[Dict[str, Any], str], pipeline_config: Union[Dict[str, Any], str], logging_config: Union[Dict[str, Any], str] = {}, storage_options: Union[Dict[str, Any], Block] = None, options: Optional[Dict[str, Any]] = {})
+    update_prefect_config(prefect_api_key: Optional[str] = None, prefect_account_id: Optional[str] = None, prefect_workspace_id: Optional[str] = None, profile_name: str = None, active: bool = True)
+    update_base_config(name: str, b_type: str, active: bool = False, options: Dict[str, Any] = {})
+    echoflow_config_AWS(aws_key: str, aws_secret: str, token: str = None, name: str = "echoflow-aws-credentials", region: str = None, options: Dict[str, Any] = {}, active: bool = False)
+    echoflow_config_AZ_cosmos(name: str = "echoflow-az-credentials", connection_string: str = None, options: Dict[str, Any] = {}, active: bool = False)
+    load_credential_configuration(): Loads credentials from an `.ini` configuration file and creates corresponding credential blocks.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+
+import asyncio
+import configparser
+import json
+import os
+import socket
+from enum import Enum
+from pathlib import Path
+from typing import Any, Coroutine, Dict, List, Optional, Union
+
+import toml
+from prefect.blocks.core import Block
+from prefect_aws import AwsCredentials
+from prefect_azure import AzureCosmosDbCredentials
+from pydantic import SecretStr
+
+from echoflow.models.datastore import StorageType
+from echoflow.models.echoflow_config import (BaseConfig, EchoflowConfig,
+                                             EchoflowPrefectConfig)
+from echoflow.utils.config_utils import get_storage_options, load_block
+
+from echoflow.stages.echoflow_trigger import echoflow_trigger
+
+
+def check_internet_connection(host="8.8.8.8", port=53, timeout=5):
+    """
+    Check if there is an active internet connection.
+
+    Args:
+        host (str, optional): IP address or hostname to check the connection to. Defaults to "8.8.8.8".
+        port (int, optional): Port number to check the connection on. Defaults to 53 (DNS port).
+        timeout (int, optional): Timeout for the connection check in seconds. Defaults to 5.
+
+    Returns:
+        bool: True if the connection is successful, False otherwise.
+
+    Example:
+        # Check if internet connection is available
+        internet_available = check_internet_connection()
+        print("Internet connection available:", internet_available)
+    """
+    try:
+        # Set a timeout for the connection check
+        socket.setdefaulttimeout(timeout)
+        # Create a socket and connect to the host and port
+        socket.socket(socket.AF_INET, socket.SOCK_STREAM).connect((host, port))
+        return True
+    except Exception as e:
+        return False
+
+
+def echoflow_create_prefect_profile(
+    name: str,
+    api_key: str = None,
+    workspace_id: str = None,
+    account_id: str = None,
+    set_active: bool = True,
+):
+    """
+    Create or update a Prefect profile in the local configuration file.
+
+    Args:
+        name (str): Name of the profile.
+        api_key (str, optional): Prefect API key. Defaults to None.
+        workspace_id (str, optional): Prefect workspace ID. Defaults to None.
+        account_id (str, optional): Prefect account ID. Defaults to None.
+        set_active (bool, optional): Set the profile as active. Defaults to True.
+
+    Example:
+        # Create a new Prefect profile
+        echoflow_create_prefect_profile(
+            name="my-profile",
+            api_key="my-api-key",
+            workspace_id="my-workspace-id",
+            account_id="my-account-id",
+            set_active=True,
+        )
+    """
+    config_path = os.path.expanduser("~/.prefect/profiles.toml")
+    with open(config_path, "r") as f:
+        config = toml.load(f)
+
+    # Update the active profile if specified
+    if set_active:
+        config["active"] = name
+
+    profiles = config["profiles"]
+    if api_key is not None and workspace_id is not None and account_id is not None:
+        profiles[name] = {
+            "PREFECT_API_KEY": api_key,
+            "PREFECT_API_URL": f"https://api.prefect.cloud/api/accounts/{ account_id }/workspaces/{ workspace_id }",
+        }
+    else:
+        profiles[name] = {}
+
+    # Save the updated configuration file
+    with open(config_path, "w") as f:
+        toml.dump(config, f)
+
+    # Does not update if switching from cloud to local or vice-versa, but updates the old profile which is active. This is the default behaviour of Prefect.
+    update_prefect_config(
+        prefect_api_key=api_key,
+        prefect_workspace_id=workspace_id,
+        prefect_account_id=account_id,
+        profile_name=name,
+    )
+
+
+def load_profile(name: str):
+    """
+    Load an existing Prefect profile by setting it as the active profile.
+
+    Args:
+        name (str): Name of the profile to load.
+
+    Example:
+        # Load an existing Prefect profile
+        load_profile("my-profile")
+    """
+
+    # Load the existing Prefect configuration file
+    config_path = os.path.expanduser("~/.prefect/profiles.toml")
+    with open(config_path, "r") as f:
+        config = toml.load(f)
+
+    # Check if the specified profile exists
+    if config.get("profiles").get(name) is None:
+        raise ValueError(
+            "No such profile exists. Please try creating profile with this name")
+
+    # Set the specified profile as the active profile
+    config["active"] = name
+
+    # Save the updated configuration file
+    with open(config_path, "w") as f:
+        toml.dump(config, f)
+
+
+def get_active_profile():
+    """
+    Get the configuration of the active Prefect profile.
+
+    Returns:
+        dict: Configuration of the active profile.
+
+    Example:
+        # Get the configuration of the active Prefect profile
+        active_profile = get_active_profile()
+        print("Active profile configuration:", active_profile)
+    """
+    # Load the existing Prefect configuration file
+    config_path = os.path.expanduser("~/.prefect/profiles.toml")
+    with open(config_path, "r") as f:
+        config = toml.load(f)
+
+    profiles = config["profiles"]
+
+    # Find the active profile and return its configuration
+    for p in profiles.keys():
+        if p == config["active"]:
+            return profiles[p]
+
+    raise ValueError("No profile found.")
+
+
+def echoflow_start(
+    dataset_config: Union[Dict[str, Any], str, Path],
+    pipeline_config: Union[Dict[str, Any], str, Path],
+    logging_config: Union[Dict[str, Any], str, Path] = None,
+    storage_options: Union[Dict[str, Any], Block] = None,
+    options: Optional[Dict[str, Any]] = {},
+    json_data_path: Union[str, Path] = None
+):
+    """
+    Start an Echoflow pipeline execution.
+
+    Args:
+        dataset_config (Union[Dict[str, Any], str, Path]): Configuration for the dataset to be processed.
+        pipeline_config (Union[Dict[str, Any], str, Path]): Configuration for the pipeline to be executed.
+        logging_config (Union[Dict[str, Any], str, Path], optional): Configuration for logging. Defaults to None.
+        storage_options (Union[Dict[str, Any], Block], optional): Storage options for accessing data. Defaults to None.
+        options (Optional[Dict[str, Any]], optional): Additional options. Defaults to {}.
+
+    Returns:
+        Any: Result of the pipeline execution.
+
+    Example:
+        # Define configuration files and options
+        dataset_config = "dataset_config.yaml"
+        pipeline_config = "pipeline_config.yaml"
+        logging_config = {}
+        storage_options = {"key": "value"}
+        options = {"storage_options_override": True}
+
+        # Start the Echoflow pipeline
+        result = echoflow_start(
+            dataset_config=dataset_config,
+            pipeline_config=pipeline_config,
+            logging_config=logging_config,
+            storage_options=storage_options,
+            options=options
+        )
+        print("Pipeline execution result:", result)
+    """
+
+    # Try loading the Prefect config block
+    try:
+        echoflow_config = load_block(
+            name="echoflow-config", type=StorageType.ECHOFLOW)
+    except ValueError as e:
+        print("No Prefect Cloud Configuration found. Creating Prefect Local named 'echoflow-local'. Please add your prefect cloud ")
+        # Add local profile to echoflow config but keep default as active since user might configure using Prefect setup
+        echoflow_create_prefect_profile(
+            name="echoflow-local", set_active=False)
+
+    # Check if program can connect to the Internet.
+    if check_internet_connection() == False:
+        active_profile = get_active_profile()
+        if active_profile["PREFECT_API_KEY"] is not None:
+            raise ValueError(
+                "Please connect to internet or consider switching to a local prefect environment. This can be done by calling load_profile(name_of_local_prefect_profile or 'echoflow-local' if no prefect profile was created) method."
+            )
+        else:
+            print("Using a local prefect environment. To go back to your cloud workspace call load_profile(<name>) with <name> of your cloud profile.")
+
+    if isinstance(storage_options, Block):
+        storage_options = get_storage_options(storage_options=storage_options)
+    # Call the actual pipeline
+    return echoflow_trigger(
+        dataset_config=dataset_config,
+        pipeline_config=pipeline_config,
+        logging_config=logging_config,
+        storage_options=storage_options,
+        options=options,
+        json_data_path=json_data_path
+    )
+
+
+def update_prefect_config(
+    prefect_api_key: Optional[str] = None,
+    prefect_account_id: Optional[str] = None,
+    prefect_workspace_id: Optional[str] = None,
+    profile_name: str = None,
+    active: bool = True,
+):
+    """
+    Update or create a Prefect configuration in the echoflow configuration block.
+
+    Args:
+        prefect_api_key (Optional[str], optional): Prefect API key. Defaults to None.
+        prefect_account_id (Optional[str], optional): Prefect account ID. Defaults to None.
+        prefect_workspace_id (Optional[str], optional): Prefect workspace ID. Defaults to None.
+        profile_name (str, optional): Name of the profile. Defaults to None.
+        active (bool, optional): Set the profile as active. Defaults to True.
+
+    Returns:
+        Any: Updated EchoflowConfig instance.
+
+    Example:
+        # Update or create a Prefect configuration with API key and workspace ID
+        update_prefect_config(
+            prefect_api_key="my-api-key",
+            prefect_workspace_id="my-workspace-id",
+            profile_name="my-profile",
+            active=True
+        )
+    """
+    profiles: List[str] = []
+    prefect_config = EchoflowPrefectConfig(
+        prefect_account_id=prefect_account_id,
+        prefect_workspace_id=prefect_workspace_id,
+        prefect_api_key=prefect_api_key,
+        profile_name=profile_name,
+    )
+
+    uuid = prefect_config.save(name=profile_name, overwrite=True)
+    if isinstance(uuid, Coroutine):
+        uuid = asyncio.run(uuid)
+
+    active_profile: str = None
+    if active:
+        active_profile = profile_name
+    profiles.append(profile_name)
+
+    try:
+        current_config = EchoflowConfig.load("echoflow-config", validate=False)
+        if isinstance(current_config, Coroutine):
+            current_config = asyncio.run(current_config)
+        if current_config.prefect_configs is not None:
+
+            if active_profile is None:
+                active_profile = current_config.active
+
+            profiles = current_config.prefect_configs
+
+            for p in profiles:
+                if p == profile_name:
+                    profiles.remove(p)
+            profiles.append(profile_name)
+
+        ecfg = EchoflowConfig(active=active_profile, prefect_configs=profiles, blocks=current_config.blocks).save(
+            "echoflow-config", overwrite=True
+        )
+        if isinstance(ecfg, Coroutine):
+            ecfg = asyncio.run(ecfg)
+    except ValueError as e:
+        ecfg = EchoflowConfig(active=active_profile, prefect_configs=profiles, blocks=[]).save(
+            "echoflow-config", overwrite=True
+        )
+        if isinstance(ecfg, Coroutine):
+            ecfg = asyncio.run(ecfg)
+    return ecfg
+
+
+def update_base_config(name: str, b_type: StorageType, active: bool = False, options: Dict[str, Any] = {}):
+    """
+    Update or create a base configuration in the echoflow configuration block.
+
+    Args:
+        name (str): Name of the configuration.
+        b_type (StorageType): Type of the configuration.
+        active (bool, optional): Set the configuration as active. Defaults to False.
+        options (Dict[str, Any], optional): Additional options. Defaults to {}.
+
+    Returns:
+        Any: Updated EchoflowConfig instance.
+
+    Example:
+        # Update or create a base configuration
+        update_base_config(
+            name="my-aws-config",
+            b_type="AWS",
+            active=True,
+            options={"option_key": "option_value"}
+        )
+    """
+    aws_base = BaseConfig(name=name, type=b_type,
+                          active=active, options=options)
+    ecfg: Any = None
+    try:
+        blocks: List[BaseConfig] = []
+        current_config = EchoflowConfig.load("echoflow-config", validate=False)
+        if isinstance(current_config, Coroutine):
+            current_config = asyncio.run(current_config)
+
+        if current_config.blocks is not None:
+            blocks = current_config.blocks
+            for b in blocks:
+                if b.name == name:
+                    blocks.remove(b)
+        blocks.append(aws_base)
+        ecfg = EchoflowConfig(
+            prefect_configs=current_config.prefect_configs, blocks=blocks
+        ).save("echoflow-config", overwrite=True)
+        if isinstance(ecfg, Coroutine):
+            ecfg = asyncio.run(ecfg)
+    except ValueError as e:
+        ecfg = EchoflowConfig(active=None, prefect_configs=[], blocks=[aws_base]).save(
+            "echoflow-config", overwrite=True
+        )
+        if isinstance(ecfg, Coroutine):
+            ecfg = asyncio.run(ecfg)
+    return ecfg
+
+
+def echoflow_config_AWS(
+    aws_access_key_id: str,
+    aws_secret_access_key: str,
+    aws_session_token: str = None,
+    name: str = "echoflow-aws-credentials",
+    region_name: str = None,
+    options: Union[str, Dict[str, Any]] = {},
+    active: bool = False,
+    **kwargs
+):
+    """
+    Configure AWS credentials in the echoflow configuration block.
+
+    Args:
+        aws_access_key_id (str): AWS access key.
+        aws_secret_access_key (str): AWS secret access key.
+        aws_session_token (str, optional): AWS session token. Defaults to None.
+        name (str, optional): Name of the configuration. Defaults to "echoflow-aws-credentials".
+        region_name (str, optional): AWS region name. Defaults to None.
+        options (str, Dict[str, Any], optional): Additional options. Defaults to {}.
+        active (bool, optional): Set the configuration as active. Defaults to False.
+
+    Example:
+        # Configure AWS credentials
+        echoflow_config_AWS(
+            aws_key="my-access-key",
+            aws_secret="my-secret-key",
+            token="my-session-token",
+            region="us-west-1",
+            name="my-aws-credentials",
+            active=True,
+            options={"option_key": "option_value"}
+        )
+    """
+    coro = AwsCredentials(
+        aws_access_key_id=aws_access_key_id,
+        aws_secret_access_key=aws_secret_access_key,
+        aws_session_token=aws_session_token,
+        region_name=region_name,
+    ).save(name, overwrite=True)
+    if isinstance(coro, Coroutine):
+        coro = asyncio.run(coro)
+    if isinstance(options, str):
+        options = json.loads(options)
+    update_base_config(name=name, active=active,
+                       options=options, b_type=StorageType.AWS)
+
+
+def echoflow_config_AZ_cosmos(
+    name: str = "echoflow-az-credentials",
+    connection_string: str = None,
+    options: Union[str, Dict[str, Any]] = {},
+    active: bool = False,
+    **kwargs
+):
+    """
+    Configure Azure Cosmos DB credentials in the local configuration file.
+
+    Args:
+        name (str, optional): Name of the configuration. Defaults to "echoflow-az-credentials".
+        connection_string (str): Azure Cosmos DB connection string.
+        options (str, Dict[str, Any], optional): Additional options. Defaults to {}.
+        active (bool, optional): Set the configuration as active. Defaults to False.
+
+    Raises:
+        ValueError: If connection string is empty.
+
+    Example:
+        # Configure Azure Cosmos DB credentials
+        echoflow_config_AZ_cosmos(
+            name="my-az-cosmos-credentials",
+            connection_string="my-connection-string",
+            active=True,
+            options={"option_key": "option_value"}
+        )
+    """
+    if connection_string is None:
+        raise ValueError("Connection string cannot be empty.")
+    coro = AzureCosmosDbCredentials(
+        connection_string=connection_string
+    ).save(name, overwrite=True)
+    if isinstance(coro, Coroutine):
+        coro = asyncio.run(coro)
+    if isinstance(options, str):
+        options = json.loads(options)
+    update_base_config(name=name, active=active,
+                       options=options, b_type=StorageType.AZCosmos)
+
+
+def load_credential_configuration(sync: bool = False):
+    """
+    Load credentials from an `.ini` configuration file and create corresponding credential blocks.
+
+    This function reads the `credentials.ini` file from the `.echoflow` directory located in the user's home directory.
+    It parses the contents of the `.ini` file and creates credential blocks using Echoflow's `echoflow_config_AWS`
+    and `echoflow_config_AZ_cosmos` functions. Each section in the `.ini` file corresponds to a different type of
+    credential block, such as AWS and Azure Cosmos DB.
+
+    Args:
+        sync (bool): If True, syncs blocks updated using Prefect UI. Defaults to False.
+
+    Example:
+        If `credentials.ini` contains the following:
+
+        [AWS]
+        aws_key = my-access-key
+        aws_secret = my-secret-key
+        region = us-west-1
+
+        [AZCosmos]
+        name = my-az-cosmos-credentials
+        connection_string = my-connection-string
+
+        Calling `load_credential_configuration()` will create an AWS credential block with the provided credentials
+        and an Azure Cosmos DB credential block with the specified connection string.
+
+    Note:
+        - Ensure the `credentials.ini` file is correctly formatted with appropriate sections and keys.
+        - Supported section names are "AWS" and "AZCosmos".
+        - Any unrecognized sections will be reported.
+        - If `sync` is True, the function will sync blocks with the Prefect UI.
+
+    Raises:
+        FileNotFoundError: If the `credentials.ini` file is not found.
+        ValueError: If no Echoflow configuration is found when `sync` is True.
+    """
+    config = configparser.ConfigParser()
+    home_directory = os.path.expanduser("~")
+
+    # Create the directory if it doesn't exist
+    config_directory = os.path.join(home_directory, ".echoflow")
+    os.makedirs(config_directory, exist_ok=True)
+
+    # Write the .ini file
+    ini_file_path = os.path.join(config_directory, "credentials.ini")
+    config.read(ini_file_path)
+
+    if sync:
+        current_config: EchoflowConfig = None
+        try:
+            current_config = EchoflowConfig.load(
+                "echoflow-config", validate=False)
+            if isinstance(current_config, Coroutine):
+                current_config = asyncio.run(current_config)
+            if current_config is not None:
+
+                for base in current_config.blocks:
+
+                    block = load_block(base.name, base.type)
+                    block_dict = dict(block)
+                    block_dict['name'] = base.name
+                    block_dict['active'] = base.active
+                    block_dict['options'] = json.dumps(base.options)
+                    block_dict['provider'] = base.type
+                    converted_dict = {}
+                    for key, value in block_dict.items():
+                        if isinstance(value, SecretStr):
+                            value = value.get_secret_value()
+                        if isinstance(value, Enum):
+                            value = value.value
+                        converted_dict[key] = str(value)
+                    config[base.name] = converted_dict
+                with open(ini_file_path, "w") as config_file:
+                    config.write(config_file)
+        except ValueError:
+            raise ("No Echoflow configuration found.")
+    for section in config.sections():
+        provider = config.get(section, 'provider')
+        data_dict = dict(config[section])
+        data_dict['name'] = section
+        data_dict.pop('provider')
+        if provider == "AWS":
+            echoflow_config_AWS(**data_dict)
+        elif provider == "AZCosmos":
+            echoflow_config_AZ_cosmos(**data_dict)
+        else:
+            print(f"Unknown section: {provider}")
diff --git a/echoflow/stages/echoflow_trigger.py b/echoflow/stages/echoflow_trigger.py
new file mode 100644
index 0000000..9c1dad6
--- /dev/null
+++ b/echoflow/stages/echoflow_trigger.py
@@ -0,0 +1,166 @@
+"""
+Echoflow Trigger Flow
+
+This module defines the Prefect Flow that acts as a trigger to initialize and execute the processing pipeline.
+
+Functions:
+    echoflow_trigger(
+        dataset_config: Union[Dict, str],
+        pipeline_config: Union[Dict, str],
+        logging_config: Union[Dict, str] = {},
+        storage_options: Optional[Dict] = {}
+    )
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+from pathlib import Path
+from typing import Any, Dict, Optional, Union
+
+from prefect import flow
+from prefect.blocks.core import Block
+from prefect.task_runners import SequentialTaskRunner
+
+from echoflow.aspects.singleton_echoflow import Singleton_Echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.pipeline import Recipe
+from echoflow.utils.config_utils import (check_config, extract_config,
+                                         get_storage_options, load_block)
+
+from .subflows.initialization_flow import init_flow
+
+
+@flow(name="Echoflow-Trigger", task_runner=SequentialTaskRunner())
+def echoflow_trigger(
+    dataset_config: Union[dict, str, Path],
+    pipeline_config: Union[dict, str, Path],
+    logging_config: Union[dict, str, Path] = None,
+    storage_options: Optional[dict] = None,
+    options: Optional[dict] = {},
+    json_data_path: Union[str, Path] = None
+):
+    """
+    Trigger the initialization and execution of the processing pipeline.
+
+    Args:
+        dataset_config (Union[dict, str, Path]): Configuration for the dataset to be 
+        processed.
+        pipeline_config (Union[dict, str, Path]): Configuration for the processing 
+        pipeline.
+        logging_config (Union[dict, str, Path], optional): Configuration for logging. 
+        Defaults to None.
+        storage_options (Optional[dict], optional): Takes block_name and type, similar to yaml. 
+        Defaults to {}.
+        options: Optional[dict]: Set of options for centralized configuration. 
+        Defaults to {}
+        json_data_path (str, Path): Takes external json data path to process the files. 
+        Defaults to None
+
+    Returns:
+        Any: Output data from the pipeline.
+
+    Example:
+        # Define dataset, pipeline, and logging configurations
+        data_config = ...
+        pipeline_config = ...
+        logging_config = ...
+
+        # Trigger the pipeline
+        pipeline_output = echoflow_trigger(
+            dataset_config=data_config,
+            pipeline_config=pipeline_config,
+            logging_config=logging_config
+        )
+        print("Pipeline output:", pipeline_output)
+    """
+    print(storage_options)
+    if storage_options is not None:
+        # Check if storage_options is a Block (fsspec storage) and convert it to a dictionary
+        if isinstance(storage_options, dict) and storage_options.get("block_name") is not None:
+            block = load_block(
+                name=storage_options.get("block_name"), type=storage_options.get("type"))
+            storage_options = get_storage_options(block)
+        
+    else:
+        storage_options = {}
+    
+    if isinstance(dataset_config, Path):
+        dataset_config = str(dataset_config)
+    if isinstance(logging_config, Path):
+        logging_config = str(logging_config)
+    if isinstance(pipeline_config, Path):
+        pipeline_config = str(pipeline_config)
+    if isinstance(json_data_path, Path):
+        json_data_path = str(json_data_path)
+
+    if isinstance(dataset_config, str):
+        if not dataset_config.endswith((".yaml", ".yml")):
+            raise ValueError("Configuration file must be a YAML!")
+        dataset_config_dict = extract_config(dataset_config, storage_options)
+    elif isinstance(dataset_config, dict):
+        dataset_config_dict = dataset_config
+
+    if isinstance(pipeline_config, str):
+        if not pipeline_config.endswith((".yaml", ".yml")):
+            raise ValueError("Configuration file must be a YAML!")
+        pipeline_config_dict = extract_config(pipeline_config, storage_options)
+    elif isinstance(pipeline_config, dict):
+        pipeline_config_dict = pipeline_config
+
+    if isinstance(logging_config, str):
+        if not logging_config.endswith((".yaml", ".yml")):
+            raise ValueError("Configuration file must be a YAML!")
+        logging_config_dict = extract_config(logging_config, storage_options)
+    else:
+        logging_config_dict = logging_config
+
+    print(dataset_config_dict)
+    print(pipeline_config_dict)
+    # Do any config checks on config dicts
+    # Should be done in pydantic class
+    check_config(dataset_config_dict, pipeline_config_dict)
+    pipeline = Recipe(**pipeline_config_dict)
+    dataset = Dataset(**dataset_config_dict)
+
+    
+
+    if options.get('storage_options_override') is not None and options['storage_options_override'] is False:
+        storage_options = {}
+    if not storage_options:
+        if dataset.output.storage_options is not None:
+            if dataset.output.storage_options.anon is False:
+                block = load_block(
+                    name=dataset.output.storage_options.block_name,
+                    type=dataset.output.storage_options.type)
+                dataset.output.storage_options_dict = get_storage_options(
+                    block)
+            else:
+                dataset.output.storage_options_dict = {
+                    "anon": dataset.output.storage_options.anon}
+
+        if dataset.args.storage_options is not None:
+            if dataset.args.storage_options.anon is False:
+                block = load_block(
+                    name=dataset.args.storage_options.block_name, type=dataset.args.storage_options.type)
+                dataset.args.storage_options_dict = get_storage_options(block)
+            else:
+                dataset.args.storage_options_dict = {
+                    "anon": dataset.args.storage_options.anon}
+        if dataset.args.transect is not None:
+            if dataset.args.transect.storage_options is not None:
+                if dataset.args.transect.storage_options.anon is False:
+                    block = load_block(
+                        name=dataset.args.transect.storage_options.block_name, type=dataset.args.transect.storage_options.type)
+                    dataset.args.transect.storage_options_dict = get_storage_options(
+                        block)
+                else:
+                    dataset.args.transect.storage_options_dict = {
+                        "anon": dataset.args.transect.storage_options.anon}
+    else:
+        dataset.output.storage_options_dict = storage_options
+        dataset.args.storage_options_dict = storage_options
+        dataset.args.transect.storage_options_dict = storage_options
+
+    Singleton_Echoflow(log_file=logging_config_dict,
+                       pipeline=pipeline, dataset=dataset)
+    return init_flow(config=dataset, pipeline=pipeline, json_data_path=json_data_path)
diff --git a/echoflow/stages/level0.py b/echoflow/stages/level0.py
deleted file mode 100644
index da5f64d..0000000
--- a/echoflow/stages/level0.py
+++ /dev/null
@@ -1,61 +0,0 @@
-from typing import Any, Dict, Union
-
-from prefect import flow
-
-from ..subflows.convert import conversion_pipeline
-from ..subflows.fetch import find_raw_pipeline
-from .utils import init_configuration
-
-
-@flow(name="00-fetch-and-convert")
-def fetch_and_convert(
-    config: Union[Dict[str, Any], str],
-    parameters: Dict[Any, Any] = {},
-    storage_options: Dict[Any, Any] = {},
-    export: bool = False,
-    export_path: str = "",
-    export_storage_options: Dict[Any, Any] = {},
-):
-    """
-    Level 0 processing for fetching and converting raw sonar data
-
-    Parameters
-    ----------
-    config : dict or str
-        Path string to configuration file or dictionary
-        containing configuration values
-    parameters : dict
-        Input parameters for creating the full url path.
-        *These inputs will overwrite the parameters,
-        within the config.*
-    storage_options : dict
-        Storage options for reading config file if specified
-    export : bool
-        Flag to export raw paths for individual raw files
-        to a raw urls JSON file
-    export_path : str
-        Path to store raw urls JSON file if export is True
-    export_storage_options: dict
-        Storage options for destination to store paths file
-
-    Notes
-    -----
-    DO NOT use Dask Task Runner for this flow.
-    Dask is used underneath within echopype.
-    """
-    new_config = init_configuration(
-        config=config, parameters=parameters, storage_options=storage_options
-    )
-    raw_dicts = find_raw_pipeline(
-        config=new_config,
-        export=export,
-        export_path=export_path,
-        export_storage_options=export_storage_options,
-    )
-
-    return conversion_pipeline(
-        config=new_config,
-        raw_dicts=raw_dicts,
-        raw_json=export_path,
-        raw_json_storage_options=export_storage_options,
-    )
diff --git a/echoflow/stages/level1.py b/echoflow/stages/level1.py
deleted file mode 100644
index 0f97188..0000000
--- a/echoflow/stages/level1.py
+++ /dev/null
@@ -1,43 +0,0 @@
-from typing import Any, Dict, Union, List, Literal
-
-from prefect import flow
-
-from ..subflows.calibrate import calibrate_pipeline
-from .utils import init_configuration
-
-
-@flow(name="01-open-and-calibrate")
-def open_and_calibrate(
-    config: Union[Dict[str, Any], str],
-    file_paths: List[str] = [],
-    storage_options: Dict[str, Any] = {},
-    cal_type: Literal["Sv", "TS"] = "Sv",
-    open_kwargs: Dict[str, Any] = {},
-):
-    """
-    Level 1 processing for reading converted raw file and perform calibration
-
-    Parameters
-    ----------
-    config : dict or str
-        Path string to configuration file or dictionary
-        containing configuration values
-    file_paths : list
-        List of file paths to converted raw files to be calibrated
-    storage_options : dict
-        Storage options for reading the converted raw file(s)
-    cal_type : {"Sv", "TS"}, optional
-        Calibration type to use. Default is "Sv"
-    open_kwargs : dict
-        Keyword arguments to pass into xarray's `open_dataset` function
-    """
-    new_config = init_configuration(
-        config=config, storage_options=storage_options
-    )
-
-    return calibrate_pipeline(
-        config=new_config,
-        file_paths=file_paths,
-        cal_type=cal_type,
-        open_kwargs=open_kwargs,
-    )
diff --git a/echoflow/stages/subflows/__init__.py b/echoflow/stages/subflows/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/echoflow/stages/subflows/combine_echodata.py b/echoflow/stages/subflows/combine_echodata.py
new file mode 100644
index 0000000..e3329b5
--- /dev/null
+++ b/echoflow/stages/subflows/combine_echodata.py
@@ -0,0 +1,162 @@
+"""
+Echoflow Combine Echodata Stage
+
+This module defines a Prefect Flow and associated tasks for the Echoflow Combine Echodata stage.
+The stage involves combining echodata files into a single zarr file, organized by transects.
+
+Classes:
+    None
+
+Functions:
+    echoflow_combine_echodata(config: Dataset, stage: Stage, data: List[Output])
+    process_combine_echodata(config: Dataset, stage: Stage, out_data: Union[List[Dict], List[Output]], working_dir: str)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import os
+from typing import Any, Dict, List, Optional, Union
+
+from echopype import combine_echodata, echodata, open_converted
+from prefect import flow, task
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+from echoflow.utils.file_utils import (get_ed_list, get_output,
+                                       get_working_dir, isFile,
+                                       process_output_transects)
+
+
+@flow
+@echoflow(processing_stage="Combine-Echodata", type="FLOW")
+def echoflow_combine_echodata(
+    config: Dataset, stage: Stage, prev_stage: Optional[Stage]
+):
+    """
+    Combine echodata files into a single zarr file organized by transects.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        prev_stage (Stage): Configuration for the previous processing stage.
+
+    Returns:
+        List[Output]: List of combined outputs organized by transects.
+
+    Example:
+        # Define configuration and data
+        dataset_config = ...
+        pipeline_stage = ...
+        processed_data = ...
+
+        # Execute the Echoflow Combine Echodata stage
+        combined_outputs = echoflow_combine_echodata(
+            config=dataset_config,
+            stage=pipeline_stage,
+            data=processed_data
+        )
+        print("Combined outputs:", combined_outputs)
+    """
+    futures = []
+    outputs: List[Output] = []
+
+    data: List[Output] = get_output()
+
+    working_dir = get_working_dir(config=config, stage=stage)
+
+    if type(data) == list:
+        if type(data[0].data) == list:
+            for output_obj in data:
+                transect = str(output_obj.data[0].get("transect")) + ".zarr"
+                process_combine_echodata_wo = process_combine_echodata.with_options(
+                    name=transect, task_run_name=transect, retries=3
+                )
+                future = process_combine_echodata_wo.submit(
+                    config=config, stage=stage, out_data=output_obj.data, working_dir=working_dir
+                )
+                futures.append(future)
+        else:
+            future = process_combine_echodata.submit(
+                config=config, stage=stage, out_data=data, working_dir=working_dir)
+            futures.append(future)
+
+        ed_list = [f.result() for f in futures]
+        outputs = process_output_transects(name=stage.name, config=config, ed_list=ed_list)
+    return outputs
+
+
+@task
+@echoflow()
+def process_combine_echodata(
+    config: Dataset,
+    stage: Stage,
+    out_data: Union[List[Dict], List[Output]],
+    working_dir: str
+):
+    """
+    Process and combine echodata files into a single zarr file.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        out_data (Union[List[Dict], List[Output]]): List of processed outputs to be combined.
+        working_dir (str): Working directory for processing.
+
+    Returns:
+        Output: Combined output information.
+
+    Example:
+        # Define configuration, processed data, and working directory
+        dataset_config = ...
+        pipeline_stage = ...
+        processed_outputs = ...
+        working_directory = ...
+
+        # Process and combine echodata
+        combined_output = process_combine_echodata(
+            config=dataset_config,
+            stage=pipeline_stage,
+            out_data=processed_outputs,
+            working_dir=working_directory
+        )
+        print("Combined output:", combined_output)
+    """
+    ed_list = []
+    if type(out_data) == list and type(out_data[0]) == dict:
+        out_zarr = os.path.join(working_dir, str(out_data[0].get(
+            "transect")), str(out_data[0].get("transect")) + ".zarr")
+        if stage.options.get("use_offline") == False or isFile(out_zarr, config.output.storage_options_dict) == False:
+            ed_list = get_ed_list.fn(
+                config=config, stage=stage, transect_data=out_data)
+            ceds = combine_echodata(echodata_list=ed_list)
+            ceds.to_zarr(
+                save_path=out_zarr,
+                overwrite=True,
+                output_storage_options=dict(
+                    config.output.storage_options_dict),
+                compute=False
+            )
+            del ceds
+        return {'out_path': out_zarr, 'transect': out_data[0].get(
+            "transect"), 'file_name': str(out_data[0].get("transect")) + ".zarr", 'error': False}
+    else:
+        out_zarr = os.path.join(
+            working_dir, "default", + "Default_Transect.zarr")
+        if stage.options.get("use_offline") == False or isFile(out_zarr) == False:
+            for output_obj in out_data:
+                ed_list.extend(get_ed_list.fn(
+                    config=config, stage=stage, transect_data=output_obj))
+            ceds = combine_echodata(echodata_list=ed_list)
+            ceds.to_zarr(
+                save_path=out_zarr,
+                overwrite=True,
+                output_storage_options=dict(
+                    config.output.storage_options_dict),
+                compute=False
+            )
+            del ceds
+        return {'out_path': out_zarr, 'transect': "Default_Transect",
+                        'file_name': 'Default_Transect.zarr', 'error': False}
diff --git a/echoflow/stages/subflows/compute_MVBS.py b/echoflow/stages/subflows/compute_MVBS.py
new file mode 100644
index 0000000..459616c
--- /dev/null
+++ b/echoflow/stages/subflows/compute_MVBS.py
@@ -0,0 +1,146 @@
+"""
+Echoflow Compute MVBS Stage
+
+This module defines a Prefect Flow and associated tasks for the Echoflow Compute MVBS stage.
+The stage involves computing the MVBS (Mean Volume Backscattering Strength) from echodata.
+
+Classes:
+    None
+
+Functions:
+    echoflow_compute_MVBS(config: Dataset, stage: Stage, data: Union[str, List[Output]])
+    process_compute_MVBS(config: Dataset, stage: Stage, out_data: Union[List[Dict], List[Output]], working_dir: str)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import os
+from typing import Dict, List, Optional, Union
+
+import echopype as ep
+from prefect import flow, task
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+from echoflow.utils.file_utils import (get_output, get_working_dir,
+                                       get_zarr_list, isFile,
+                                       process_output_transects)
+
+
+@flow
+@echoflow(processing_stage="Compute-MVBS", type="FLOW")
+def echoflow_compute_MVBS(
+        config: Dataset, stage: Stage, prev_stage: Optional[Stage]
+):
+    """
+    Compute Mean Volume Backscattering Strength (MVBS) from echodata.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        prev_stage (Stage): Configuration for the previous processing stage.
+
+    Returns:
+        List[Output]: List of computed MVBS outputs.
+
+    Example:
+        # Define configuration and data
+        dataset_config = ...
+        pipeline_stage = ...
+        echodata_outputs = ...
+
+        # Execute the Echoflow Compute MVBS stage
+        computed_mvbs_outputs = echoflow_compute_MVBS(
+            config=dataset_config,
+            stage=pipeline_stage,
+            data=echodata_outputs
+        )
+        print("Computed MVBS outputs:", computed_mvbs_outputs)
+    """
+    data: Union[str, List[Output]] = get_output()
+    outputs: List[Output] = []
+    futures = []
+    working_dir = get_working_dir(config=config, stage=stage)
+
+    if type(data) == list:
+        if type(data[0].data) == list:
+            for output_data in data:
+                transect_list = output_data.data
+                for ed in transect_list:
+                    transect = str(ed.get("out_path")).split(".")[0] + ".MVBS"
+                    process_compute_MVBS_wo = process_compute_MVBS.with_options(
+                        name=transect, task_run_name=transect, retries=3
+                    )
+                    future = process_compute_MVBS_wo.submit(
+                        config=config, stage=stage, out_data=ed, working_dir=working_dir
+                    )
+                    futures.append(future)
+        else:
+            for output_data in data:
+                future = process_compute_MVBS.submit(
+                    config=config, stage=stage, out_data=output_data, working_dir=working_dir
+                )
+                futures.append(future)
+
+        ed_list = [f.result() for f in futures]
+        outputs = process_output_transects(name=stage.name, config=config, ed_list=ed_list)
+    return outputs
+
+
+@task
+@echoflow()
+def process_compute_MVBS(
+    config: Dataset, stage: Stage, out_data: Union[Dict, Output], working_dir: str
+):
+    """
+    Process and compute Mean Volume Backscattering Strength (MVBS) from xarray dataset.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        out_data (Union[Dict, Output]): Processed outputs (xr.Dataset) to compute MVBS.
+        working_dir (str): Working directory for processing.
+
+    Returns:
+        Output: Computed MVBS output information.
+
+    Example:
+        # Define configuration, processed data, and working directory
+        dataset_config = ...
+        pipeline_stage = ...
+        processed_outputs = ...
+        working_directory = ...
+
+        # Process and compute MVBS
+        computed_mvbs_output = process_compute_MVBS(
+            config=dataset_config,
+            stage=pipeline_stage,
+            out_data=processed_outputs,
+            working_dir=working_directory
+        )
+        print("Computed MVBS output:", computed_mvbs_output)
+    """
+
+    if type(out_data) == dict:
+        file_name = str(out_data.get("file_name")).split(".")[0] + "_MVBS.zarr"
+        transect = str(out_data.get("transect"))
+        
+    else:
+        file_name = str(out_data.data.get("file_name")).split(".")[0] + "_MVBS.zarr"
+        transect = str(out_data.data.get("transect"))
+        
+    out_zarr = os.path.join(working_dir, transect, file_name)
+    if stage.options.get("use_offline") == False or isFile(out_zarr, config.output.storage_options_dict) == False:
+        ed_list = get_zarr_list.fn(transect_data=out_data, storage_options=config.output.storage_options_dict)
+        xr_d_mvbs = ep.commongrid.compute_MVBS(
+                    ds_Sv=ed_list[0],
+                    range_meter_bin=stage.external_params.get(
+                        "range_meter_bin"),
+                    ping_time_bin=stage.external_params.get("ping_time_bin")
+                )
+        xr_d_mvbs.to_zarr(store=out_zarr, mode="w", consolidated=True,
+                        storage_options=config.output.storage_options_dict)
+    return {'out_path': out_zarr, 'transect': transect, 'file_name': file_name, 'error': False}
diff --git a/echoflow/stages/subflows/compute_SV.py b/echoflow/stages/subflows/compute_SV.py
new file mode 100644
index 0000000..27ae13f
--- /dev/null
+++ b/echoflow/stages/subflows/compute_SV.py
@@ -0,0 +1,143 @@
+"""
+Echoflow Compute SV Stage
+
+This module defines a Prefect Flow and associated tasks for the Echoflow Compute SV stage.
+The stage involves computing the Sv (volume backscattering strength) from echodata.
+
+Classes:
+    None
+
+Functions:
+    echoflow_compute_SV(config: Dataset, stage: Stage, data: List[Output])
+    process_compute_SV(config: Dataset, stage: Stage, out_data: Union[List[Dict], List[Output]], working_dir: str)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import os
+from typing import Any, Dict, List, Optional, Union
+
+import echopype as ep
+from prefect import flow, task
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+from echoflow.utils.file_utils import (get_ed_list, get_output,
+                                       get_working_dir, isFile,
+                                       process_output_transects)
+
+
+@flow
+@echoflow(processing_stage="Compute-SV", type="FLOW")
+def echoflow_compute_SV(config: Dataset, stage: Stage, prev_stage: Optional[Stage]):
+    """
+    Compute volume backscattering strength (Sv) from echodata.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        prev_stage (Stage): Configuration for the previous processing stage.
+
+    Returns:
+        List[Output]: List of computed Sv outputs.
+
+    Example:
+        # Define configuration and data
+        dataset_config = ...
+        pipeline_stage = ...
+        echodata_outputs = ...
+
+        # Execute the Echoflow Compute SV stage
+        computed_sv_outputs = echoflow_compute_SV(
+            config=dataset_config,
+            stage=pipeline_stage,
+            data=echodata_outputs
+        )
+        print("Computed Sv outputs:", computed_sv_outputs)
+    """
+    data: List[Output] = get_output()
+    outputs: List[Output] = []
+    futures = []
+    working_dir = get_working_dir(config=config, stage=stage)
+    # [Output(data=[{'out_path': '/Users/soham/Desktop/EchoWorkSpace/echoflow/notebooks/combined_files/echoflow_combine_echodata/7.zarr', 'transect': 7}], passing_params={}), Output(data=[{'out_path': '/Users/soham/Desktop/EchoWorkSpace/echoflow/notebooks/combined_files/echoflow_combine_echodata/10.zarr', 'transect': 10}], passing_params={})]
+
+    if type(data) == list:
+        if type(data[0].data) == list:
+            for output_data in data:
+                transect_list = output_data.data
+                for ed in transect_list:
+                    transect = str(ed.get("out_path")).split(".")[0] + ".SV"
+                    process_compute_SV_wo = process_compute_SV.with_options(
+                        name=transect, task_run_name=transect, retries=3
+                    )
+                    future = process_compute_SV_wo.submit(
+                        config=config, stage=stage, out_data=ed, working_dir=working_dir
+                    )
+                    futures.append(future)
+        else:
+            for output_data in data:
+                future = process_compute_SV.submit(
+                    config=config, stage=stage, out_data=output_data, working_dir=working_dir
+                )
+                futures.append(future)
+
+        ed_list = [f.result() for f in futures]
+        outputs = process_output_transects(name=stage.name, config=config, ed_list=ed_list)
+
+    return outputs
+
+
+
+@task
+@echoflow()
+def process_compute_SV(
+    config: Dataset, stage: Stage, out_data: Union[Dict, Output], working_dir: str
+):
+    """
+    Process and compute volume backscattering strength (Sv) from echodata.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        out_data (Union[Dict, Output]):  Processed echodata output.
+        working_dir (str): Working directory for processing.
+
+    Returns:
+        Output: Computed Sv output information.
+
+    Example:
+        # Define configuration, processed data, and working directory
+        dataset_config = ...
+        pipeline_stage = ...
+        processed_outputs = ...
+        working_directory = ...
+
+        # Process and compute Sv
+        computed_sv_output = process_compute_SV(
+            config=dataset_config,
+            stage=pipeline_stage,
+            out_data=processed_outputs,
+            working_dir=working_directory
+        )
+        print("Computed Sv output:", computed_sv_output)
+    """
+    
+    if type(out_data) == dict:
+        file_name = str(out_data.get("file_name")).split(".")[0] + "_SV.zarr"
+        transect = str(out_data.get("transect"))
+        
+    else:
+        file_name = str(out_data.data.get("file_name")).split(".")[0] + "_SV.zarr"
+        transect = str(out_data.data.get("transect"))
+        
+    out_zarr = os.path.join(working_dir, transect, file_name)
+    if stage.options.get("use_offline") == False or isFile(out_zarr, config.output.storage_options_dict) == False:
+        ed_list = get_ed_list.fn(
+                config=config, stage=stage, transect_data=out_data)
+        xr_d_sv = ep.calibrate.compute_Sv(echodata=ed_list[0])
+        xr_d_sv.to_zarr(store=out_zarr, mode="w", consolidated=True,
+                        storage_options=config.output.storage_options_dict)
+    return {'out_path': out_zarr, 'transect': transect, 'file_name': file_name, 'error': False}
diff --git a/echoflow/stages/subflows/compute_TS.py b/echoflow/stages/subflows/compute_TS.py
new file mode 100644
index 0000000..a4f66c0
--- /dev/null
+++ b/echoflow/stages/subflows/compute_TS.py
@@ -0,0 +1,143 @@
+"""
+Echoflow Compute TS Stage
+
+This module defines a Prefect Flow and associated tasks for the Echoflow Compute TS stage.
+The stage involves computing the TS (Target strength) from echodata.
+
+Classes:
+    None
+
+Functions:
+    echoflow_compute_TS(config: Dataset, stage: Stage, data: List[Output])
+    process_compute_TS(config: Dataset, stage: Stage, out_data: Union[List[Dict], List[Output]], working_dir: str)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import os
+from typing import Any, Dict, List, Optional, Union
+
+import echopype as ep
+from prefect import flow, task
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+from echoflow.utils.file_utils import (get_ed_list, get_output,
+                                       get_working_dir, isFile,
+                                       process_output_transects)
+
+
+@flow
+@echoflow(processing_stage="Compute-TS", type="FLOW")
+def echoflow_compute_TS(config: Dataset, stage: Stage, prev_stage: Optional[Stage]):
+    """
+    Compute Target strength (TS) from echodata.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        prev_stage (Stage): Configuration for the previous processing stage.
+
+    Returns:
+        List[Output]: List of computed TS outputs.
+
+    Example:
+        # Define configuration and data
+        dataset_config = ...
+        pipeline_stage = ...
+        echodata_outputs = ...
+
+        # Execute the Echoflow Compute TS stage
+        computed_ts_outputs = echoflow_compute_TS(
+            config=dataset_config,
+            stage=pipeline_stage,
+            data=echodata_outputs
+        )
+        print("Computed TS outputs:", computed_ts_outputs)
+    """
+    data: List[Output] = get_output()
+
+    outputs: List[Output] = []
+    futures = []
+    working_dir = get_working_dir(config=config, stage=stage)
+    # [Output(data=[{'out_path': '/Users/soham/Desktop/EchoWorkSpace/echoflow/notebooks/combined_files/echoflow_combine_echodata/7.zarr', 'transect': 7}], passing_params={}), Output(data=[{'out_path': '/Users/soham/Desktop/EchoWorkSpace/echoflow/notebooks/combined_files/echoflow_combine_echodata/10.zarr', 'transect': 10}], passing_params={})]
+
+    if type(data) == list:
+        if type(data[0].data) == list:
+            for output_data in data:
+                transect_list = output_data.data
+                for ed in transect_list:
+                    transect = str(ed.get("out_path")).split(".")[0] + ".TS"
+                    process_compute_TS_wo = process_compute_TS.with_options(
+                        name=transect, task_run_name=transect, retries=3
+                    )
+                    future = process_compute_TS_wo.submit(
+                        config=config, stage=stage, out_data=ed, working_dir=working_dir
+                    )
+                    futures.append(future)
+        else:
+            for output_data in data:
+                future = process_compute_TS.submit(
+                    config=config, stage=stage, out_data=output_data, working_dir=working_dir
+                )
+                futures.append(future)
+
+        ed_list = [f.result() for f in futures]
+        outputs = process_output_transects(name=stage.name, config=config, ed_list=ed_list)
+    return outputs
+
+
+
+@task
+@echoflow()
+def process_compute_TS(
+    config: Dataset, stage: Stage, out_data: Union[Dict, Output], working_dir: str
+):
+    """
+    Process and compute volume backscattering strength (TS) from echodata.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        out_data (Union[Dict, Output]):  Processed echodata output.
+        working_dir (str): Working directory for processing.
+
+    Returns:
+        Output: Computed TS output information.
+
+    Example:
+        # Define configuration, processed data, and working directory
+        dataset_config = ...
+        pipeline_stage = ...
+        processed_outputs = ...
+        working_directory = ...
+
+        # Process and compute TS
+        computed_TS_output = process_compute_TS(
+            config=dataset_config,
+            stage=pipeline_stage,
+            out_data=processed_outputs,
+            working_dir=working_directory
+        )
+        print("Computed TS output:", computed_TS_output)
+    """
+    
+    if type(out_data) == dict:
+        file_name = str(out_data.get("file_name")).split(".")[0] + "_TS.zarr"
+        transect = str(out_data.get("transect"))
+        
+    else:
+        file_name = str(out_data.data.get("file_name")).split(".")[0] + "_TS.zarr"
+        transect = str(out_data.data.get("transect"))
+        
+    out_zarr = os.path.join(working_dir, transect, file_name)
+    if stage.options.get("use_offline") == False or isFile(out_zarr, config.output.storage_options_dict) == False:
+        ed_list = get_ed_list.fn(
+                config=config, stage=stage, transect_data=out_data)
+        xr_d_ts = ep.calibrate.compute_TS(echodata=ed_list[0])
+        xr_d_ts.to_zarr(store=out_zarr, mode="w", consolidated=True,
+                        storage_options=config.output.storage_options_dict)
+    return {'out_path': out_zarr, 'transect': transect, 'file_name': file_name, 'error': False}
diff --git a/echoflow/stages/subflows/initialization_flow.py b/echoflow/stages/subflows/initialization_flow.py
new file mode 100644
index 0000000..f082c7c
--- /dev/null
+++ b/echoflow/stages/subflows/initialization_flow.py
@@ -0,0 +1,140 @@
+"""
+Main Prefect Flow Initialization
+
+This module defines the main Prefect Flow for initializing and executing the processing pipeline.
+It includes tasks for extracting paths for raw data files, executing processing stages, and managing Dask clusters.
+
+Functions:
+    init_flow(pipeline: Recipe, dataset: Dataset)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import json
+import logging
+from typing import Any, Dict
+
+import dask
+from distributed import Client, LocalCluster
+from prefect import flow
+from prefect.filesystems import *
+from prefect.task_runners import SequentialTaskRunner
+from prefect_dask import DaskTaskRunner
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.pipeline import Recipe
+from echoflow.utils.config_utils import (club_raw_files,
+                                         get_prefect_config_dict,
+                                         glob_all_files, parse_raw_paths)
+from echoflow.utils.file_utils import (cleanup, extract_fs,
+                                       get_last_run_output, store_json_output)
+from echoflow.utils.function_utils import dynamic_function_call
+
+
+@flow(name="Init-Flow", task_runner=SequentialTaskRunner())
+@echoflow(type="FLOW")
+def init_flow(
+        pipeline: Recipe,
+        config: Dataset,
+        json_data_path: str = None
+):
+    """
+    Initialize and execute the processing pipeline using Prefect.
+
+    Args:
+        pipeline (Recipe): Configuration for the processing pipeline.
+        dataset (Dataset): Configuration for the dataset to be processed.
+
+    Returns:
+        Any: Output data from the pipeline.
+
+    Example:
+        # Define pipeline and dataset configurations
+        processing_pipeline = ...
+        data_dataset = ...
+
+        # Initialize and execute the processing flow
+        pipeline_output = init_flow(
+            pipeline=processing_pipeline,
+            dataset=data_dataset
+        )
+        print("Pipeline output:", pipeline_output)
+    """
+    prefect_config_dict = {}
+    file_dicts = []
+
+    if json_data_path is None:  
+        if config.args.raw_json_path is None:  
+            total_files = glob_all_files(config=config)
+            file_dicts = parse_raw_paths(all_raw_files=total_files, config=config)
+        data = club_raw_files(
+            config=config,
+            raw_dicts=file_dicts,
+            raw_url_file=config.args.raw_json_path,
+            json_storage_options=config.output.storage_options_dict
+        )
+    else:
+        file_system = extract_fs(
+            json_data_path, storage_options=config.args.storage_options_dict
+        )
+        with file_system.open(json_data_path) as f:
+            data = json.load(f)
+
+    store_json_output(data, config=config, name=config.name)
+
+    process_list = pipeline.pipeline
+    client: Client = None
+    process = None
+    for process in process_list:
+        if process.recipe_name == pipeline.active_recipe:
+            active_pipeline = process 
+    
+    prev_stage = None
+    for stage in active_pipeline.stages:
+        function = dynamic_function_call(stage.module, stage.name)
+        prefect_config_dict = get_prefect_config_dict(
+            stage, pipeline, prefect_config_dict)
+
+        if pipeline.scheduler_address is not None and pipeline.use_local_dask == False:
+            if client is None:
+                client = Client(pipeline.scheduler_address)
+            prefect_config_dict["task_runner"] = DaskTaskRunner(
+                address=client.scheduler.address)
+            print(client)
+        elif pipeline.use_local_dask == True and prefect_config_dict is not None and prefect_config_dict.get("task_runner") is None:
+            if client is None:
+                cluster = LocalCluster(n_workers=pipeline.n_workers)
+                client = Client(cluster.scheduler_address)
+            prefect_config_dict["task_runner"] = DaskTaskRunner(
+                address=client.scheduler.address)
+            print(client)
+
+        function = function.with_options(**prefect_config_dict)
+        print("-"*50)
+        print("\nExecuting stage : ", stage)
+        output = function(config, stage, prev_stage)
+        print("\nCompleted stage", stage)
+        print("-"*50)
+
+        store_json_output(data=output, name=stage.name+"_output", config=config)
+
+        if prev_stage is not None:
+            if config.output.retention == False:
+                if (prev_stage.options.get("save_offline") is None or prev_stage.options.get("save_offline") == False):
+                    cleanup(config, prev_stage, data)
+            else:
+                if (prev_stage.options.get("save_offline") is not None and prev_stage.options.get("save_offline") == False):
+                    cleanup(config, prev_stage, data)
+        
+        prev_stage = stage
+
+    # Close the local cluster but not the cluster hosted.
+    if pipeline.scheduler_address is None and pipeline.use_local_dask == True:
+        client.close()
+        print("Local Client has been closed")
+    
+    output = get_last_run_output(data=output, storage_options=config.output.storage_options_dict)
+
+    return output
diff --git a/echoflow/stages/subflows/open_raw.py b/echoflow/stages/subflows/open_raw.py
new file mode 100644
index 0000000..0b928f7
--- /dev/null
+++ b/echoflow/stages/subflows/open_raw.py
@@ -0,0 +1,144 @@
+"""
+Echoflow Open Raw Stage
+
+This module defines a Prefect Flow and associated tasks for the Echoflow Open Raw stage.
+The stage involves processing raw sonar data files, converting them to zarr format, and
+organizing the processed data based on transects.
+
+Classes:
+    None
+
+Functions:
+    echoflow_open_raw(config: Dataset, stage: Stage, data: Union[str, List[List[Dict[str, Any]]]])
+    process_raw(raw, working_dir: str, config: Dataset, stage: Stage)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import os
+from pathlib import Path
+from typing import Any, Dict, List, Optional, Union
+
+from echopype import open_raw
+from prefect import flow, task
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+from echoflow.utils.file_utils import (download_temp_file, get_output,
+                                       get_working_dir, isFile,
+                                       process_output_transects)
+
+
+@flow
+@echoflow(processing_stage="Open-Raw", type="FLOW")
+def echoflow_open_raw(config: Dataset, stage: Stage, prev_stage: Optional[Stage]):
+    """
+    Process raw sonar data files and convert them to zarr format.
+
+    Args:
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+        prev_stage (Stage): Configuration for the previous processing stage.
+
+    Returns:
+        List[Output]: List of processed outputs organized based on transects.
+
+    Example:
+        # Define configuration and data
+        dataset_config = ...
+        pipeline_stage = ...
+        raw_data = ...
+
+        # Execute the Echoflow Open Raw stage
+        processed_outputs = echoflow_open_raw(
+            config=dataset_config,
+            stage=pipeline_stage,
+            data=raw_data
+        )
+        print("Processed outputs:", processed_outputs)
+    """
+
+    data: Union[str, List[List[Dict[str, Any]]]] = get_output("Raw")
+
+    working_dir = get_working_dir(stage=stage, config=config)
+
+    ed_list = []
+    futures = []
+    outputs: List[Output] = []
+
+    # Dealing with single file
+    if type(data) == str or type(data) == Path:
+        ed = process_raw(data, working_dir, config, stage)
+        output = Output()
+        output.data = ed
+        outputs.append(output)
+    else:
+        for raw_dicts in data:
+            for raw in raw_dicts:
+                new_processed_raw = process_raw.with_options(
+                    task_run_name=raw.get("file_path"), name=raw.get("file_path"), retries=3
+                )
+                future = new_processed_raw.submit(
+                    raw, working_dir, config, stage)
+                futures.append(future)
+
+        ed_list = [f.result() for f in futures]
+
+        outputs = process_output_transects(name=stage.name, config=config, ed_list=ed_list)
+    return outputs
+
+
+@task()
+@echoflow()
+def process_raw(raw, working_dir: str, config: Dataset, stage: Stage):
+    """
+    Process a single raw sonar data file.
+
+    Args:
+        raw (str): Path to the raw data file.
+        working_dir (str): Working directory for processing.
+        config (Dataset): Configuration for the dataset being processed.
+        stage (Stage): Configuration for the current processing stage.
+
+    Returns:
+        Dict[str, Any]: Processed output information.
+
+    Example:
+        # Define raw data, working directory, and configurations
+        raw_file = "path/to/raw_data.raw"
+        working_directory = "path/to/working_dir"
+        dataset_config = ...
+        pipeline_stage = ...
+
+        # Process the raw data
+        processed_output = process_raw(
+            raw=raw_file,
+            working_dir=working_directory,
+            config=dataset_config,
+            stage=pipeline_stage
+        )
+        print("Processed output:", processed_output)
+    """
+    temp_file = download_temp_file(raw, working_dir, stage, config)
+    local_file = temp_file.get("local_path")
+    local_file_name = os.path.basename(temp_file.get("local_path"))
+    out_zarr = os.path.join(working_dir, str(
+        raw.get("transect_num")), local_file_name.replace(".raw", ".zarr"))
+    if stage.options.get("use_offline") == False or isFile(out_zarr, config.output.storage_options_dict) == False:
+        ed = open_raw(raw_file=local_file, sonar_model=raw.get(
+            "instrument"), storage_options=config.output.storage_options_dict)
+        ed.to_zarr(
+            save_path=str(out_zarr),
+            overwrite=True,
+            output_storage_options=config.output.storage_options_dict,
+            compute=False
+        )
+        del ed
+
+        if stage.options.get("save_raw_file") == False:
+            local_file.unlink()
+
+    return {'out_path': out_zarr, 'transect': raw.get("transect_num"), 'file_name': local_file_name, 'error': False}
diff --git a/echoflow/stages/utils.py b/echoflow/stages/utils.py
deleted file mode 100644
index 110dbce..0000000
--- a/echoflow/stages/utils.py
+++ /dev/null
@@ -1,95 +0,0 @@
-from typing import Any, Dict, Union
-
-from prefect import task, get_run_logger
-
-import yaml
-from ..subflows.utils import extract_fs
-from ..settings.models import MainConfig
-
-
-def check_and_extract_config(
-    config: Union[Dict[str, Any], str], storage_options: Dict[str, Any] = {}
-) -> Dict[str, Any]:
-    """
-    Read config file and extract to a dictionary
-
-    Parameters
-    ----------
-    config : dict or str
-        Path string to configuration file or dictionary
-        containing configuration values
-    storage_options : dict
-        Storage options for reading config file if specified
-
-    Returns
-    -------
-    dict
-        Configuration dictionary for pipeline
-    """
-    if isinstance(config, str):
-        if not config.endswith(('.yaml', '.yml')):
-            raise ValueError("Configuration file must be a YAML!")
-        file_system = extract_fs(config, storage_options)
-        with file_system.open(config, 'rb') as yaml_file:
-            return yaml.safe_load(yaml_file.read())
-    return config
-
-
-def setup_config(
-    input_config: Dict[Any, Any], parameters: Dict[Any, Any]
-) -> MainConfig:
-    """
-    Validate configuration dictionary,
-    and updating url parameters for harvest
-
-    Parameters
-    ----------
-    input_config : dict
-        Pipeline configuration
-    parameters : dict
-        Parameters input to overwrite the parameters,
-        within the input_config
-
-    Returns
-    -------
-    MainConfig
-        Updated configuration object
-    """
-    logger = get_run_logger()
-    logger.info("Validating configurations ...")
-    config = MainConfig(**input_config)
-    config.args.parameters.update(parameters)
-    logger.info("Configurations validation completed.")
-    return config
-
-
-@task
-def init_configuration(
-    config: Union[Dict[str, Any], str],
-    parameters: Dict[Any, Any] = {},
-    storage_options: Dict[str, Any] = {},
-) -> MainConfig:
-    """
-    Task to perform configuration extraction, intialization and validation
-
-    Parameters
-    ----------
-    config : dict or str
-        Path string to configuration file or dictionary
-        containing configuration values
-    parameters : dict
-        Input parameters for creating the full url path.
-        *These inputs will overwrite the parameters,
-        within the config.*
-    storage_options : dict
-        Storage options for reading config file if specified
-
-    Returns
-    -------
-    MainConfig
-        Updated configuration object
-    """
-    config = check_and_extract_config(
-        config=config, storage_options=storage_options
-    )
-    return setup_config(input_config=config, parameters=parameters)
diff --git a/echoflow/subflows/calibrate/__init__.py b/echoflow/subflows/calibrate/__init__.py
deleted file mode 100644
index 8801993..0000000
--- a/echoflow/subflows/calibrate/__init__.py
+++ /dev/null
@@ -1,4 +0,0 @@
-"""Calibrate subflow module"""
-from .flows import calibrate_pipeline
-
-__all__ = ["calibrate_pipeline"]
diff --git a/echoflow/subflows/calibrate/flows.py b/echoflow/subflows/calibrate/flows.py
deleted file mode 100644
index fd2551d..0000000
--- a/echoflow/subflows/calibrate/flows.py
+++ /dev/null
@@ -1,47 +0,0 @@
-from typing import List, Literal, Dict, Any
-
-from prefect import flow
-
-from ...settings.models import MainConfig
-from .tasks import open_and_calibrate
-
-
-@flow
-def calibrate_pipeline(
-    config: MainConfig,
-    file_paths: List[str],
-    cal_type: Literal["Sv", "TS"] = "Sv",
-    open_kwargs: Dict[str, Any] = {},
-) -> List[str]:
-    """
-    Calibration pipeline for converted raw files.
-
-    Parameters
-    ----------
-    config : object
-        Pipeline configuration object
-    file_paths : list
-        List of file paths to converted raw files to be calibrated
-    cal_type : {"Sv", "TS"}, optional
-        Calibration type to use. Default is "Sv"
-    open_kwargs : dict
-        Keyword arguments to pass into xarray's `open_dataset` function
-
-    Returns
-    -------
-    List of path string to converted files
-
-    Notes
-    -----
-    Don't run this pipeline with Dask Task Runners
-    """
-    futures = []
-    for urlpath in file_paths:
-        future = open_and_calibrate.submit(
-            config=config,
-            file_path=urlpath,
-            cal_type=cal_type,
-            open_kwargs=open_kwargs,
-        )
-        futures.append(future)
-    return [f.result() for f in futures]
diff --git a/echoflow/subflows/calibrate/tasks.py b/echoflow/subflows/calibrate/tasks.py
deleted file mode 100644
index 16d3f1e..0000000
--- a/echoflow/subflows/calibrate/tasks.py
+++ /dev/null
@@ -1,60 +0,0 @@
-from typing import Dict, Any, Literal
-
-from prefect import task
-
-from echopype import open_converted
-from echopype.calibrate import compute_Sv, compute_TS
-import xarray as xr
-
-from ...settings.models import MainConfig
-
-
-@task
-def open_and_calibrate(
-    config: MainConfig,
-    file_path: str,
-    cal_type: Literal["Sv", "TS"] = "Sv",
-    open_kwargs: Dict[str, Any] = {},
-) -> xr.Dataset:
-    """
-    Task for opening converted file, and calibrating
-
-    Parameters
-    ----------
-    config : object
-        Pipeline configuration object
-    file_paths : list
-        List of file paths to converted raw files to be calibrated
-    cal_type : {"Sv", "TS"}, optional
-        Calibration type to use. Default is "Sv"
-    open_kwargs : dict
-        Keyword arguments to pass into xarray's `open_dataset` function
-
-    Returns
-    -------
-    xr.Dataset
-        Xarray dataset of the calibrated data
-    """
-    storage_options = config.output.storage_options
-    calibrate_config = (
-        config.echopype.calibrate if config.echopype is not None else None
-    )
-    echodata = open_converted(
-        converted_raw_path=file_path,
-        storage_options=storage_options,
-        **open_kwargs
-    )
-    if cal_type == "Sv":
-        calibrate = compute_Sv
-        kwargs = (
-            calibrate_config.compute_Sv if calibrate_config is not None else {}
-        )
-    elif cal_type == "TS":
-        calibrate = compute_TS
-        kwargs = (
-            calibrate_config.compute_TS if calibrate_config is not None else {}
-        )
-    else:
-        raise ValueError("cal_type must be Sv or TS")
-
-    return calibrate(echodata=echodata, **kwargs)
diff --git a/echoflow/subflows/convert/__init__.py b/echoflow/subflows/convert/__init__.py
deleted file mode 100644
index 7427c6d..0000000
--- a/echoflow/subflows/convert/__init__.py
+++ /dev/null
@@ -1,4 +0,0 @@
-"""Convert subflow module"""
-from .flows import conversion_pipeline
-
-__all__ = ["conversion_pipeline"]
diff --git a/echoflow/subflows/convert/flows.py b/echoflow/subflows/convert/flows.py
deleted file mode 100644
index 0fdcbfa..0000000
--- a/echoflow/subflows/convert/flows.py
+++ /dev/null
@@ -1,57 +0,0 @@
-from typing import Any, Dict, List, Optional
-
-from prefect import flow
-
-from ...settings.models import MainConfig
-from .tasks import data_convert, get_client, parse_raw_json
-
-
-@flow
-def conversion_pipeline(
-    config: MainConfig,
-    raw_dicts: List[Dict[str, Any]] = [],
-    raw_json: Optional[str] = None,
-    raw_json_storage_options: Dict[str, Any] = {},
-    client=None,
-):
-    """
-    Conversion pipeline for raw echosounder data.
-    The results will be converted as weekly or transect files.
-
-    Parameters
-    ----------
-    config : object
-        Pipeline configuration object
-    raw_dicts : list
-        List of raw url dictionary
-    raw_json : str, optional
-        The path to raw urls json file
-    raw_json_storage_options: dict
-        Storage options for reading raw urls file path
-    client : dask.distributed.Client, optional
-        The dask client to use for `echopype.combine_echodata`
-
-    Returns
-    -------
-    List of path string to converted files
-
-    Notes
-    -----
-    Don't run this pipeline with Dask Task Runners
-    """
-    all_files = parse_raw_json(
-        config=config,
-        raw_dicts=raw_dicts,
-        raw_url_file=raw_json,
-        json_storage_options=raw_json_storage_options,
-    )
-    futures = []
-    client = get_client(client)
-    for raw_dicts in all_files:
-        future = data_convert.submit(
-            raw_dicts=raw_dicts,
-            client=client,
-            config=config,
-        )
-        futures.append(future)
-    return [f.result() for f in futures]
diff --git a/echoflow/subflows/convert/tasks.py b/echoflow/subflows/convert/tasks.py
deleted file mode 100644
index 3e5856c..0000000
--- a/echoflow/subflows/convert/tasks.py
+++ /dev/null
@@ -1,158 +0,0 @@
-import itertools as it
-import json
-from typing import Any, Dict, List, Optional
-
-from dask import delayed
-import dask.distributed
-from prefect import task
-
-from ...settings.models import RawConfig
-from ..utils import extract_fs
-from .utils import (
-    combine_data,
-    download_temp_file,
-    get_output_file_path,
-    make_temp_folder,
-    open_and_save,
-)
-
-
-@task
-def get_client(client: dask.distributed.Client = None):
-    """
-    Task to get dask client if doesn't exist,
-    otherwise use the client passed into argument
-
-    Parameters
-    ----------
-    client : dask.distributed.Client, optional
-        Dask distributed client object to use
-
-    Returns
-    -------
-    dask.distributed.Client
-    """
-    if client is None:
-        return dask.distributed.Client()
-    return client
-
-
-@task
-def data_convert(
-    raw_dicts: List[Dict[str, Any]],
-    config: RawConfig,
-    client,
-):
-    """
-    Task for running the data conversion on a list of raw url
-    dictionaries.
-
-    Parameters
-    ----------
-    raw_dicts : list
-        The list of raw url dictionary
-    config : dict
-        Pipeline configuration file
-    client : dask.distributed.Client, optional
-        The dask client to use for `echopype.combine_echodata`
-
-    Returns
-    -------
-    String path to the combined echodata file
-    """
-    zarr_path = get_output_file_path(raw_dicts=raw_dicts, config=config)
-
-    # TODO: Allow for specifying output path
-    temp_raw_dir = make_temp_folder()
-    ed_tasks = []
-    for raw in raw_dicts:
-        raw = delayed(download_temp_file)(raw, temp_raw_dir=temp_raw_dir)
-        ed = delayed(open_and_save)(raw, config=config)
-        ed_tasks.append(ed)
-    ed_futures = client.compute(ed_tasks)
-    ed_list = client.gather(ed_futures)
-    return combine_data(ed_list, zarr_path, client, config)
-
-
-@task
-def parse_raw_json(
-    config: Dict[Any, Any] = [],
-    raw_dicts: List[Dict[str, Any]] = [],
-    raw_url_file: Optional[str] = None,
-    json_storage_options: Dict[Any, Any] = {},
-) -> List[List[Dict[str, Any]]]:
-    """
-    Task to parse raw urls json files and splits them into
-    weekly list by utilizing julian days.
-
-    This assumes the following raw url dictionary
-
-    ```
-    {'instrument': 'EK60',
-    'file_path': 'https://example.com/some-file.raw',
-    'month': 1,
-    'year': 2017,
-    'jday': 1,
-    'datetime': '2017-01-01T00:00:00'}
-    ```
-
-    Parameters
-    ----------
-    config : dict
-        Pipeline configuration file
-    raw_dicts : list, optional
-        List of raw url dictionary
-    raw_url_file : str, optional
-        Raw urls file path string
-    json_storage_options : dict
-        Storage options for reading raw urls file path
-
-    Returns
-    -------
-    List of list of raw urls string,
-    broken up to 7 julian days each chunk
-    """
-    if len(raw_dicts) == 0:
-        if raw_url_file is None:
-            raise ValueError("Must have raw_dicts or raw_url_file present.")
-        file_system = extract_fs(
-            raw_url_file, storage_options=json_storage_options
-        )
-        with file_system.open(raw_url_file) as f:
-            raw_dicts = json.load(f)
-
-    if config.args.transect is not None:
-        # Transect, split by transect spec
-        raw_dct = {}
-        for r in raw_dicts:
-            transect_num = r['transect_num']
-            if transect_num not in raw_dct:
-                raw_dct[transect_num] = []
-            raw_dct[transect_num].append(r)
-
-        all_files = [
-            sorted(raw_list, key=lambda a: a['datetime'])
-            for raw_list in raw_dct.values()
-        ]
-    else:
-        # Number of days for a week chunk
-        n = 7
-
-        all_jdays = sorted({r.get("jday") for r in raw_dicts})
-        split_days = [
-            all_jdays[i : i + n] for i in range(0, len(all_jdays), n)
-        ]  # noqa
-
-        day_dict = {}
-        for r in raw_dicts:
-            mint = r.get("jday")
-            if mint not in day_dict:
-                day_dict[mint] = []
-            day_dict[mint].append(r)
-
-        all_files = []
-        for week in split_days:
-            files = list(it.chain.from_iterable([day_dict[d] for d in week]))
-            all_files.append(files)
-
-    return all_files
diff --git a/echoflow/subflows/convert/utils.py b/echoflow/subflows/convert/utils.py
deleted file mode 100644
index b48b83b..0000000
--- a/echoflow/subflows/convert/utils.py
+++ /dev/null
@@ -1,132 +0,0 @@
-import os
-from pathlib import Path
-
-import echopype as ep
-from dateutil import parser
-
-from echoflow.settings.models.raw_config import RawConfig
-
-from ..utils import extract_fs
-
-ep.verbose()
-
-
-def make_temp_folder():
-    """
-    Make temporary echopype folder locally at
-    `./temp_echopype_output/raw_temp_files`
-    """
-    temp_raw_dir = Path("temp_echopype_output/raw_temp_files")
-    temp_raw_dir.mkdir(exist_ok=True, parents=True)
-    return temp_raw_dir
-
-
-def get_output_file_path(raw_dicts, config):
-    """
-    Get the output file path based on the raw file dictionary
-    datetime key. It will grab the first file and use that as the
-    file name in a form of 'DYYYYmmdd-THHMMSS.zarr'.
-
-    This method will also grab value from the config.output.urlpath
-    for where the file should go.
-
-    Parameters
-    ----------
-    raw_dicts : list
-        The list of raw url dictionary. Must have 'datetime' key!
-    config : RawConfig
-        The pydantic RawConfig object that contains all the configurations
-        needed.
-
-    Returns
-    -------
-    The full path string to the zarr file.
-    """
-    if len(raw_dicts) < 1:
-        raise ValueError("There must be at least one raw file dictionary!")
-
-    first_file = raw_dicts[0]
-    datetime_obj = parser.parse(first_file.get("datetime"))
-    if config.args.transect is None:
-        # Only use datetime
-        out_fname = datetime_obj.strftime("D%Y%m%d-T%H%M%S.zarr")
-    else:
-        # Use transect number
-        transect_num = first_file.get("transect_num", None)
-        date_name = datetime_obj.strftime("D%Y%m%d-T%H%M%S")
-        out_fname = f"x{transect_num:04}-{date_name}.zarr"
-    return "/".join([config.output.urlpath, out_fname])
-
-
-def download_temp_file(raw, temp_raw_dir):
-    """
-    Download temporary raw file
-    """
-    urlpath = raw.get("file_path")
-    file_system = extract_fs(urlpath)
-    fname = os.path.basename(urlpath)
-    out_path = temp_raw_dir / fname
-    with file_system.open(urlpath, 'rb') as source_file:
-        with open(out_path, mode="wb") as f:
-            f.write(source_file.read())
-    raw.update({"local_path": out_path})
-    return raw
-
-
-def open_and_save(raw, config: RawConfig, **kwargs):
-    """
-    Open, convert, and save raw file
-    """
-    local_file = Path(raw.get("local_path"))
-    out_zarr = "/".join(
-        [config.output.urlpath, local_file.name.replace(".raw", ".zarr")]
-    )
-    ed = ep.open_raw(
-        raw_file=local_file,
-        sonar_model=raw.get("instrument"),
-        offload_to_zarr=True,
-        **kwargs
-    )
-    ed.to_zarr(
-        save_path=str(out_zarr),
-        overwrite=True,
-        output_storage_options=config.output.storage_options,
-    )
-
-    # Delete temp zarr
-    del ed
-    # Delete temp raw
-    local_file.unlink()
-
-    return ep.open_converted(
-        converted_raw_path=str(out_zarr),
-        storage_options=config.output.storage_options,
-    )
-
-
-def clean_up_files(ed_list, config):
-    """Clean up converted raw files"""
-    for ed in ed_list:
-        path = str(ed.converted_raw_path)
-        file_system = extract_fs(
-            path, storage_options=config.output.storage_options
-        )
-        file_system.delete(path, recursive=True)
-        del ed
-
-
-def combine_data(ed_list, zarr_path, client, config: RawConfig, **kwargs):
-    """Combine echodata object and clean up the individual files"""
-    combined_ed = ep.combine_echodata(
-        echodatas=list(ed_list),
-        zarr_path=zarr_path,
-        client=client,
-        overwrite=config.output.overwrite,
-        storage_options=config.output.storage_options,
-        **kwargs
-    )
-
-    # Clean up objects
-    del combined_ed
-    clean_up_files(ed_list, config)
-    return zarr_path
diff --git a/echoflow/subflows/fetch/__init__.py b/echoflow/subflows/fetch/__init__.py
deleted file mode 100644
index 26624a3..0000000
--- a/echoflow/subflows/fetch/__init__.py
+++ /dev/null
@@ -1,4 +0,0 @@
-"""Fetch subflow module"""
-from .flows import find_raw_pipeline
-
-__all__ = ["find_raw_pipeline"]
diff --git a/echoflow/subflows/fetch/flows.py b/echoflow/subflows/fetch/flows.py
deleted file mode 100644
index e0eb7ee..0000000
--- a/echoflow/subflows/fetch/flows.py
+++ /dev/null
@@ -1,45 +0,0 @@
-from typing import Dict, Any, Tuple, List
-from prefect import flow
-
-from .tasks import (
-    export_raw_dicts,
-    glob_all_files,
-    parse_raw_paths,
-)
-from ...settings.models import MainConfig
-
-
-@flow
-def find_raw_pipeline(
-    config: MainConfig,
-    export: bool = False,
-    export_path: str = "",
-    export_storage_options: Dict[Any, Any] = {},
-) -> Tuple[List[Dict[Any, Any]], Dict[Any, Any]]:
-    """
-    Pipeline for finding raw files from a given url location
-
-    Parameters
-    ----------
-    config : object
-        Pipeline configuration object
-    export : bool
-        Flag to export raw paths for individual raw files
-        to a raw urls JSON file
-    export_path : str
-        Path to store raw urls JSON file if export is True
-    export_storage_options: dict
-        Storage options for destination to store paths file
-
-    Returns
-    -------
-    Tuple
-        Raw urls dictionary and an updated configuration dictionary
-
-    """
-    total_files = glob_all_files(config)
-    file_dicts = parse_raw_paths(total_files, config)
-
-    if export:
-        export_raw_dicts(file_dicts, export_path, export_storage_options)
-    return file_dicts
diff --git a/echoflow/subflows/fetch/tasks.py b/echoflow/subflows/fetch/tasks.py
deleted file mode 100644
index e127e26..0000000
--- a/echoflow/subflows/fetch/tasks.py
+++ /dev/null
@@ -1,141 +0,0 @@
-from typing import Dict, Any, List
-import itertools as it
-import os
-
-import fsspec
-from prefect import get_run_logger, task
-
-from ..utils import glob_url
-from .utils import parse_file_path, extract_transect_files
-
-
-@task
-def glob_all_files(config: Dict[Any, Any]) -> List[str]:
-    """
-    Task for fetch individual file urls from a source path,
-    defined in config dictionary
-
-    Parameters
-    ----------
-    config : dict
-        Pipeline configuration
-
-    Returns
-    -------
-    list
-        List of raw url paths string
-    """
-    logger = get_run_logger()
-    logger.info("Fetching raw file paths ...")
-    total_files = []
-    data_path = config.args.rendered_path
-    storage_options = config.args.storage_options
-    logger.info(f"File pattern: {data_path}.")
-    if data_path is not None:
-        if isinstance(data_path, list):
-            for path in data_path:
-                all_files = glob_url(path, **storage_options)
-                total_files.append(all_files)
-            total_files = list(it.chain.from_iterable(total_files))
-        else:
-            total_files = glob_url(data_path, **storage_options)
-    logger.info(f"There are {len(total_files)} raw files.")
-    return total_files
-
-
-@task
-def parse_raw_paths(
-    all_raw_files: List[str], config: Dict[Any, Any]
-) -> List[Dict[Any, Any]]:
-    """
-    Task for parsing raw url paths,
-    extracting info from it,
-    and creating a file dictionary
-
-    Parameters
-    ----------
-    all_raw_files : list
-        List of raw url paths string
-    config : dict
-        Pipeline configuration
-
-    Returns
-    -------
-    list
-        List of raw url paths dictionary
-    """
-    logger = get_run_logger()
-    logger.info("Parsing file paths into dictionary ...")
-    sonar_model = config.sonar_model
-    fname_pattern = config.raw_regex
-    transect_dict = {}
-    if config.args.transect is not None:
-        # When transect info is available, extract it
-        logger.info("Transect file provided. Extracting ...")
-        file_input = config.args.transect.file
-        storage_options = config.args.transect.storage_options
-        if isinstance(file_input, str):
-            filename = os.path.basename(file_input)
-            _, ext = os.path.splitext(filename)
-            transect_dict = extract_transect_files(
-                ext.strip('.'), file_input, storage_options
-            )
-        else:
-            transect_dict = {}
-            for f in file_input:
-                filename = os.path.basename(f)
-                _, ext = os.path.splitext(filename)
-                result = extract_transect_files(
-                    ext.strip('.'), f, storage_options
-                )
-                transect_dict.update(result)
-
-    raw_file_dicts = []
-    for raw_file in all_raw_files:
-        # get transect info from the transect_dict above
-        transect = transect_dict.get(os.path.basename(raw_file), {})
-        transect_num = transect.get('num', None)
-        if (config.args.transect is None) or (
-            config.args.transect is not None and transect_num is not None
-        ):
-            # Only adds to the list if not transect
-            # if it's a transect, ensure it has a transect number
-            raw_file_dicts.append(
-                dict(
-                    instrument=sonar_model,
-                    file_path=raw_file,
-                    transect_num=transect_num,
-                    **parse_file_path(raw_file, fname_pattern),
-                )
-            )
-    logger.info("Finished creating list of raw file dicts.")
-    return raw_file_dicts
-
-
-@task
-def export_raw_dicts(
-    raw_dicts: List[Dict[Any, Any]],
-    export_path: str,
-    export_storage_options: Dict[Any, Any] = {},
-):
-    """
-    Task for exporting the raw url paths dictionary to a json file
-
-    Parameters
-    ----------
-    raw_dicts : list
-        List of raw url paths dictionary
-    export_path : str
-        Full path to JSON file for saving
-    export_storage_options : dict
-        Storage options for destination to store paths file
-
-    Returns
-    -------
-    None
-    """
-    import json
-
-    json_str = json.dumps(raw_dicts)
-    with fsspec.open(export_path, mode="wt", **export_storage_options) as f:
-        f.write(json_str)
diff --git a/echoflow/subflows/fetch/utils.py b/echoflow/subflows/fetch/utils.py
deleted file mode 100644
index 8405f37..0000000
--- a/echoflow/subflows/fetch/utils.py
+++ /dev/null
@@ -1,141 +0,0 @@
-from dateutil import parser
-import os
-import re
-from typing import Dict, Any, Literal, List
-from zipfile import ZipFile
-
-import fsspec
-
-from ..utils import extract_fs
-
-TRANSECT_FILE_REGEX = r"x(?P<transect_num>\d+)"
-
-
-def parse_file_path(raw_file: str, fname_pattern: str) -> Dict[str, Any]:
-    """
-    Parses file path to get at the datetime
-
-    Parameters
-    ----------
-    raw_file : str
-        Raw file url string
-    fname_pattern : str
-        Regex pattern string for date extraction from file
-
-    Returns
-    -------
-    dict
-        Raw file url dictionary that contains parsed dates
-    """
-    matcher = re.compile(fname_pattern)
-    file_match = matcher.search(raw_file)
-    match_dict = file_match.groupdict()
-    file_datetime = None
-    if "date" in match_dict and "time" in match_dict:
-        datetime_obj = parser.parse(
-            f"{file_match['date']}{file_match['time']}"
-        )  # noqa
-        file_datetime = datetime_obj.isoformat()
-        jday = datetime_obj.timetuple().tm_yday
-        match_dict.pop("date")
-        match_dict.pop("time")
-        match_dict.setdefault("month", datetime_obj.month)
-        match_dict.setdefault("year", datetime_obj.year)
-        match_dict.setdefault("jday", jday)
-
-    match_dict.setdefault("datetime", file_datetime)
-    return dict(**match_dict)
-
-
-def _extract_from_zip(
-    file_system, file_path: str
-) -> Dict[str, Dict[str, Any]]:
-    transect_dict = {}
-    """Extracts raw files from transect file zip format"""
-    with file_system.open(file_path, 'rb') as f:
-        with ZipFile(f) as zf:
-            zip_infos = zf.infolist()
-            for zi in zip_infos:
-                m = re.match(TRANSECT_FILE_REGEX, zi.filename)
-                transect_num = int(m.groupdict()['transect_num'])
-                if zi.is_dir():
-                    raise ValueError(
-                        "Directory found in zip file. This is not allowed!"
-                    )
-                with zf.open(zi.filename) as txtfile:
-                    file_list = txtfile.read().decode('utf-8').split('\r\n')
-                    for rawfile in file_list:
-                        transect_dict.setdefault(
-                            rawfile,
-                            {"filename": zi.filename, "num": transect_num},
-                        )
-    return transect_dict
-
-
-def _extract_from_text(
-    file_system, file_path: str
-) -> Dict[str, Dict[str, Any]]:
-    """Extracts raw files from transect file text format"""
-    filename = os.path.basename(file_path)
-    m = re.match(TRANSECT_FILE_REGEX, filename)
-    transect_num = int(m.groupdict()['transect_num'])
-
-    transect_dict = {}
-
-    with file_system.open(file_path) as txtfile:
-        file_list = txtfile.read().decode('utf-8').split('\r\n')
-        for rawfile in file_list:
-            transect_dict.setdefault(
-                rawfile,
-                {"filename": filename, "num": transect_num},
-            )
-
-    return transect_dict
-
-
-def extract_transect_files(
-    file_format: Literal['txt', 'zip'],
-    file_path: str,
-    storage_options: Dict[str, Any] = {},
-) -> Dict[str, Dict[str, Any]]:
-    """
-    Extracts raw file names and transect number from transect file(s)
-
-    Parameters
-    ----------
-    file_format : str
-        Transect file format. Options are 'txt' or 'zip' only.
-    file_path : str
-        The full path to the transect file.
-    storage_options : dict
-        Storage options for transect file access.
-
-    Returns
-    -------
-    dict
-        Raw file name dictionary with the original transect file name
-        and transect number. For example:
-
-        ```
-        {
-            'Summer2017-D20170627-T171516.raw': {
-                'filename': 'x0007_fileset.txt',
-                'num': 7
-            },
-            'Summer2017-D20170627-T174838.raw': {
-                'filename': 'x0007_fileset.txt',
-                'num': 7
-            }
-        }
-        ```
-
-    """
-    file_system = extract_fs(file_path, storage_options=storage_options)
-    if file_format == "zip":
-        return _extract_from_zip(file_system, file_path)
-    elif file_format == "txt":
-        return _extract_from_text(file_system, file_path)
-    else:
-        raise ValueError(
-            f"Invalid file format: {file_format}. Only 'txt' or 'zip' are valid"
-        )
diff --git a/echoflow/subflows/utils.py b/echoflow/subflows/utils.py
deleted file mode 100644
index 0c20742..0000000
--- a/echoflow/subflows/utils.py
+++ /dev/null
@@ -1,62 +0,0 @@
-from typing import Any, Dict, Union, Tuple, List
-from urllib.parse import urlparse
-
-import fsspec
-
-
-def glob_url(path: str, **storage_options: Dict[str, Any]) -> List[str]:
-    """
-    Glob files based on given path string,
-    using fsspec.filesystem.glob
-
-    Parameters
-    ----------
-    path : str
-        The url path string glob pattern
-    **storage_options : dict, optional
-        Extra arguments to pass to the filesystem class,
-        to allow permission to glob
-
-    Returns
-    -------
-    List of paths from glob
-    """
-    file_system, scheme = extract_fs(
-        path, storage_options, include_scheme=True
-    )
-    all_files = [
-        # Add scheme back into file urlpath for fs like s3
-        # since it seem to get lost
-        f if f.startswith(scheme) else f"{scheme}://{f}"
-        for f in file_system.glob(path)
-    ]
-
-    return all_files
-
-
-def extract_fs(
-    url: str,
-    storage_options: Dict[Any, Any] = {},
-    include_scheme: bool = False,
-) -> Union[Tuple[Any, str], Any]:
-    """
-    Extract the fsspec file system from a url path.
-
-    Parameters
-    ----------
-    url : str
-        The url path string to be parsed and file system extracted
-    storage_options : dict
-        Additional keywords to pass to the filesystem class
-    include_scheme : bool
-        Flag to include scheme in the output of the function
-
-    Returns
-    -------
-    Filesystem for given protocol and arguments
-    """
-    parsed_path = urlparse(url)
-    file_system = fsspec.filesystem(parsed_path.scheme, **storage_options)
-    if include_scheme:
-        return file_system, parsed_path.scheme
-    return file_system
diff --git a/echoflow/tests/AWS/AWSEchoflowDemo.ipynb b/echoflow/tests/AWS/AWSEchoflowDemo.ipynb
new file mode 100644
index 0000000..36fcffe
--- /dev/null
+++ b/echoflow/tests/AWS/AWSEchoflowDemo.ipynb
@@ -0,0 +1,535 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {
+    "jp-MarkdownHeadingCollapsed": true
+   },
+   "source": [
+    "# Getting Started with Echoflow on AWS: Processing EK60 Data to compute MVBS\n",
+    "\n",
+    "\n",
+    "Welcome to the Echoflow AWS Deployment Guide notebook! This notebook will walk you through the steps to deploy an Echoflow pipeline on Amazon Web Services (AWS). Echoflow is a powerful tool for acoustics data processing and analysis, and AWS provides a scalable and reliable cloud platform for running your workflows.\n",
+    "\n",
+    "In this notebook, you will learn how to set up an EC2 instance, install Echoflow, configure AWS S3 storage for your processed data, and initiate your Echoflow pipeline using Prefect. By following these steps, you'll be able to seamlessly run your acoustics data processing workflows in the cloud.\n",
+    "\n",
+    "Let's get started!\n",
+    "\n",
+    "## Prerequisites\n",
+    "\n",
+    "Before you begin, make sure you have the following prerequisites in place:\n",
+    "\n",
+    "- An AWS account with necessary permissions to create EC2 instances and S3 buckets.\n",
+    "- A private key (.pem) file for SSH access to your EC2 instance.\n",
+    "- Basic familiarity with the command-line interface (CLI).\n",
+    "\n",
+    "## Notebook Setup\n",
+    "\n",
+    "In this notebook, we'll go through the following steps:\n",
+    "\n",
+    "1. Create an EC2 Instance: Set up an AWS EC2 instance to run your Echoflow pipeline.\n",
+    "2. Connect to EC2 using SSH: Establish a secure connection to your EC2 instance using SSH.\n",
+    "3. Install Echoflow: Create a virtual environment, clone the Echoflow repository, and install the package.\n",
+    "4. Initialize Echoflow and Prefect: Configure your Echoflow and Prefect environments.\n",
+    "5. Create an S3 Bucket: Set up an AWS S3 bucket to store your processed data.\n",
+    "6. Store AWS Credentials: Store your AWS credentials securely for access.\n",
+    "7. Create Credential Blocks: Set up credential blocks for secure access within Prefect.\n",
+    "8. Open Jupyter Notebook: Launch Jupyter Notebook to execute your Echoflow pipeline.\n",
+    "9. Setting Up: Setting up source S3.\n",
+    "10. Getting Data: Get data from the source S3.\n",
+    "11. Preparing Files: Preparing files for processing.\n",
+    "12. Processing with echoflow: Execute the Pipeline.\n",
+    "\n",
+    "Now, let's dive into each step to deploy your Echoflow pipeline on AWS!\n",
+    "\n",
+    "---\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Step 1: Create an EC2 Instance\n",
+    "Refer to the [AWS EC2] (https://docs.aws.amazon.com/efs/latest/ug/gs-step-one-create-ec2-resources.html) documentation for a step-by-step guide on creating an EC2 instance. Make sure you have the private key (.pem) file for SSH access.\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 2: Connect to EC2 using SSH\n",
+    "Connect to your EC2 instance using SSH. You can follow the instructions in the [AWS SSH] (https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/connect-linux-inst-ssh.html) documentation. You'll need the private key (.pem) file.\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 3: Install Echoflow\n",
+    "\n",
+    "### Step 3.1: Make a virtual environment\n",
+    "\n",
+    "To keep your Echoflow environment isolated, it's recommended to create a virtual environment using Conda or Python's built-in venv module. Here's an example using Conda:\n",
+    "\n",
+    "```bash\n",
+    "conda create --name echoflow-env\n",
+    "conda activate echoflow-env\n",
+    "```\n",
+    "\n",
+    "Or, using Python's venv:\n",
+    "\n",
+    "```bash\n",
+    "python -m venv echoflow-env\n",
+    "source echoflow-env/bin/activate  # On Windows, use `echoflow-env\\Scripts\\activate`\n",
+    "```\n",
+    "### Step 3.2: Clone the Repository\n",
+    "Now that you have a virtual environment set up, you can clone the Echoflow project repository to your local machine using the following command:\n",
+    "\n",
+    "```bash\n",
+    "git clone <echoflow_repo>\n",
+    "```\n",
+    "\n",
+    "### Step 3.3: Install the Package\n",
+    "Navigate to the project directory you've just cloned and install the Echoflow package. The -e flag is crucial as it enables editable mode, which is especially helpful during development and testing. Now, take a moment and let the echoflow do its thing while you enjoy your coffee.\n",
+    "\n",
+    "```bash\n",
+    "cd <project_directory>\n",
+    "pip install -e .\n",
+    "```\n",
+    "\n",
+    "---\n",
+    "\n",
+    "### Step 4: Initialize Echoflow and Prefect\n",
+    "\n",
+    "To kickstart your journey with Echoflow and Prefect, follow these simple initialization steps:\n",
+    "\n",
+    "##### 4.1 Initializing Echoflow\n",
+    "Begin by initializing Echoflow with the following command:\n",
+    "\n",
+    "```bash\n",
+    "echoflow init\n",
+    "```\n",
+    "\n",
+    "This command sets up the groundwork for your Echoflow environment, preparing it for seamless usage.\n",
+    "\n",
+    "##### 4.2 Initializing Prefect\n",
+    "For Prefect, initialization involves a few extra steps, including secure authentication. Enter the following command to initiate the Prefect authentication process:\n",
+    "\n",
+    "- If you have a Prefect Cloud account, provide your Prefect API key to securely link your account. Type your API key when prompted and press Enter.\n",
+    "\n",
+    "```bash\n",
+    "prefect cloud login\n",
+    "```\n",
+    "\n",
+    "- If you don't have a Prefect Cloud account yet, you can use local prefect account. This is especially useful for those who are just starting out and want to explore Prefect without an account.\n",
+    "\n",
+    "```bash\n",
+    "prefect profiles create echoflow-local\n",
+    "```\n",
+    "\n",
+    "\n",
+    "The initialization process will ensure that both Echoflow and Prefect are properly set up and ready for you to dive into your cloud-based workflows.\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 5: Create a S3 bucket to store the output\n",
+    "\n",
+    "Create an S3 bucket to store the processed output. Refer to the [AWS S3] (https://docs.aws.amazon.com/quickstarts/latest/s3backup/step-1-create-bucket.html) documentation for guidance. Add the S3 URI to datastore.yaml in the same directory as this notebook under the urlpath key in the output section:\n",
+    "\n",
+    "```yaml\n",
+    "# ...rest of the cofiguration\n",
+    "output: # Output arguments\n",
+    "  urlpath: <YOUR_S3_URI> # Destination data URL parameters\n",
+    "  overwrite: true \n",
+    "  storage_options: \n",
+    "    block_name: echoflow-aws-credentials\n",
+    "    type: AWS\n",
+    "```\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 6: Store AWS Credentials\n",
+    "\n",
+    "Edit the ~/.echoflow/credentials.ini file and add your AWS Key and Secret.\n",
+    "\n",
+    "```bash\n",
+    "nano ~/.echoflow/credentials.ini\n",
+    "\n",
+    "# add the following and save:\n",
+    "[echoflow-aws-credentials]\n",
+    "aws_access_key_id=my-aws-key\n",
+    "aws_secret_access_key=my-aws-secret\n",
+    "provider=AWS\n",
+    "```\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 7: Create Credential blocks\n",
+    "\n",
+    "Once you have stored the credentials in the ini file, call the below command to create a block securedly stored in your prefect account. For more about blocks refer [Blocks] (https://github.com/OSOceanAcoustics/echoflow/blob/dev/docs/configuration/blocks.md). \n",
+    "\n",
+    "```bash\n",
+    "echoflow load-credentials\n",
+    "```\n",
+    "\n",
+    "---\n",
+    "\n",
+    "# Step 8: Jupyter Notebook\n",
+    "Open Jupyter Notebook using terminal in the same activated environment \n",
+    "\n",
+    "```bash\n",
+    "jupyter notebook\n",
+    "```\n",
+    "\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Step 9: Setting Up\n",
+    "We begin by importing the required libraries and specifying the paths for the dataset and pipeline configuration files. These files contain the necessary information for data processing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pathlib import Path\n",
+    "from echoflow import echoflow_start, StorageType, glob_url\n",
+    "\n",
+    "dataset_config = Path(\"./datastore.yaml\").resolve()\n",
+    "pipeline_config = Path(\"./pipeline.yaml\").resolve()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Step 9.1\n",
+    "### Pipeline Configuration: Mean Volume Backscattering Strength\n",
+    "In this section, we will provide you with the pipeline configuration that we'll be using for our MVBS processing. The configuration is presented in YAML format, which is a structured and human-readable way to define settings for data processing.\n",
+    "\n",
+    "Here's the configuration we'll be using:\n",
+    "\n",
+    "```yaml\n",
+    "active_recipe: standard \n",
+    "use_local_dask: true\n",
+    "n_workers: 3\n",
+    "pipeline:\n",
+    "- recipe_name: standard \n",
+    "  stages: \n",
+    "  - name: echoflow_open_raw \n",
+    "    module: echoflow.stages.subflows.open_raw \n",
+    "    options: \n",
+    "      save_raw_file: true\n",
+    "      use_raw_offline: true \n",
+    "      use_offline: true \n",
+    "  - name: echoflow_combine_echodata\n",
+    "    module: echoflow.stages.subflows.combine_echodata\n",
+    "    options:\n",
+    "      use_offline: true\n",
+    "  - name: echoflow_compute_SV\n",
+    "    module: echoflow.stages.subflows.compute_SV\n",
+    "    options:\n",
+    "      use_offline: true\n",
+    "  - name: echoflow_compute_MVBS\n",
+    "    module: echoflow.stages.subflows.compute_MVBS\n",
+    "    options:\n",
+    "      use_offline: true\n",
+    "    external_params:\n",
+    "      range_meter_bin: 20 \n",
+    "      ping_time_bin: 20S\n",
+    "\n",
+    "```\n",
+    "    \n",
+    "Let's break down the components of this configuration:\n",
+    "\n",
+    "- **active_recipe**: Specifies the recipe to be used for processing, which is set as \"standard\" in this case.\n",
+    "\n",
+    "- **use_local_dask**: This flag indicates that we'll be utilizing a local Dask Cluster for parallel processing.\n",
+    "\n",
+    "- **n_workers**: Determines the number of worker processes in the Dask Cluster. Here, we're using 3 workers for efficient parallelization.\n",
+    "\n",
+    "- **pipeline**: This section defines the sequence of stages to execute. In this example, we're following the \"standard\" recipe, which comprises four stages.\n",
+    "\n",
+    "    - **echoflow_open_raw**: This stage utilizes the `open_raw` subflow module to open raw data files. It includes options such as saving raw files, using raw data in offline mode, and utilizing offline data.\n",
+    "    \n",
+    "    - **echoflow_combine_echodata**: This stage employs the `combine_echodata` subflow module to combine echodatas based on transect. It includes an option to use offline data.\n",
+    "    \n",
+    "    - **compute_SV**: This stage employs the `compute_SV` subflow module to compute Backscattering Strength. It includes an option to use offline data.\n",
+    "    \n",
+    "    - **compute_MVBS**: This stage employs the `compute_MVBS` subflow module to calculate MVBS. It includes an option to use offline data.\n",
+    "\n",
+    "**Note**: For a more comprehensive understanding of each option and its functionality, you can refer to the [Pipeline documentation](https://github.com/OSOceanAcoustics/echoflow/blob/dev/docs/configuration/pipeline.md).\n",
+    "\n",
+    "Keep in mind that in this example, we'll be setting up a local Dask Cluster with 3 workers for parallel processing. This configuration will enable us to efficiently process our data for MVBS analysis. To turn it off, toggle `use_local_dask` to false.\n",
+    "\n",
+    "Feel free to explore and modify the configuration to understand better."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 9.2\n",
+    "### Datastore Configuration: Organizing Data for Processing\n",
+    "\n",
+    "In this section, we'll delve into the configuration that defines how the data will be organized and managed for processing. This configuration is provided in YAML format and plays a crucial role in structuring data inputs and outputs.\n",
+    "\n",
+    "Here's the detailed breakdown of the configuration:\n",
+    "\n",
+    "```yaml\n",
+    "name: Bell_M._Shimada-SH1707-EK60\n",
+    "sonar_model: EK60 \n",
+    "raw_regex: (.*)-?D(?P<date>\\w{1,8})-T(?P<time>\\w{1,6}) \n",
+    "args: \n",
+    "  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw\n",
+    "  parameters:\n",
+    "    ship_name: Bell_M._Shimada\n",
+    "    survey_name: SH1707\n",
+    "    sonar_model: EK60\n",
+    "  storage_options:\n",
+    "    anon: true\n",
+    "  transect:\n",
+    "    file: ./EK60_SH1707_Shimada.txt\n",
+    "    default_transect_num: 2017\n",
+    "  json_export: true \n",
+    "output: \n",
+    "  urlpath: <YOUR-S3-BUCKET>\n",
+    "  retention: false\n",
+    "  overwrite: true\n",
+    "  storage_options: \n",
+    "    block_name: echoflow-aws-credentials\n",
+    "    type: AWS\n",
+    "```\n",
+    "\n",
+    "Let's delve into the individual components of the configuration presented here:\n",
+    "\n",
+    "- **name**: Specifies a descriptive name for the configuration, aiding in identifying its purpose.\n",
+    "\n",
+    "- **sonar_model**: Indicates the type of sonar being utilized, which in this case is \"EK60\".\n",
+    "\n",
+    "- **raw_regex**: Defines a regular expression pattern for extracting date and time information from raw data file names.\n",
+    "\n",
+    "- **args**: This section provides crucial arguments for structuring data inputs:\n",
+    "\n",
+    "  - **urlpath**: Defines the URL pattern to access the raw data files stored on a remote server. The placeholders `{{ ship_name }}`, `{{ survey_name }}`, and `{{ sonar_model }}` are dynamically replaced with the specified values.\n",
+    "\n",
+    "  - **storage_options**: Sets storage options, such as anonymous access (`anon: true`), for retrieving the data.\n",
+    "\n",
+    "  - **transect**: Specifies a file (`EK60_SH1707_Shimada.txt`) containing the list of files to process, along with default transect information.\n",
+    "\n",
+    "  - **json_export**: Enables JSON metadata export.\n",
+    "\n",
+    "- **output**: This section configures the output settings for processed data:\n",
+    "\n",
+    "  - **urlpath**: Determines the output directory (`<YOUR-S3-BUCKET>`) where the processed data will be stored.\n",
+    "\n",
+    "  - **retention**: Disables data retention, indicating that only MVBS data will be stored in this case.\n",
+    "\n",
+    "  - **overwrite**: Allows data overwriting if the data already exists.\n",
+    "\n",
+    "**Note**: \n",
+    "- For a more comprehensive understanding of each option and its functionality, you can refer to the [Datast documentation](https://github.com/OSOceanAcoustics/echoflow/blob/dev/docs/configuration/datastore.md).\n",
+    "- The pipeline will store MVBS Strength output under `<YOUR-S3-BUCKET>`. As the retention is set to false, only MVBS Strength files will be stored. To specify files for processing, create a list of file names and store it in `EK60_SH1707_Shimada.txt`, which should be placed under the transect directory.\n",
+    "\n",
+    "This configuration facilitates efficient data organization and management for the processing pipeline. Feel free to tailor it to your specific data and processing requirements.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 10: Getting Data\n",
+    "Next, we'll use the glob_url function to retrieve a list of URLs matching a specific pattern. In this case, we're targeting raw EK60 data files from the SH1707 survey."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "all_files = glob_url(\"s3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/*.raw\", {'anon':True})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 11: Preparing Files\n",
+    "We'll now extract the file names from the URLs and create a file listing for the transect. This will help us organize and work with the data effectively."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "files = []\n",
+    "for file in all_files:\n",
+    "    f = file.split(\".r\")[0]\n",
+    "    files.append(f.split(\"/\")[-1])\n",
+    "\n",
+    "transect = open('EK60_SH1707_Shimada.txt','w')\n",
+    "i = 0\n",
+    "for f in files:\n",
+    "    if i == 5:\n",
+    "        break\n",
+    "    transect.write(f+\".raw\\n\")\n",
+    "    i = i + 1\n",
+    "transect.close()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 12: Processing with echoflow\n",
+    "Now, we're ready to kick off the data processing using echoflow. We'll provide the dataset and pipeline configurations, along with additional options."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2023-08-30 16:01:16,445 - distributed.diskutils - INFO - Found stale lock file and directory '/tmp/dask-worker-space/worker-7jjfqrq8', purging\n"
+     ]
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<Client: 'tcp://127.0.0.1:35413' processes=3 threads=6, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params=None options={'save_raw_file': True, 'use_raw_offline': True, 'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "\n",
+      "Completed stage name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params=None options={'save_raw_file': True, 'use_raw_offline': True, 'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "<Client: 'tcp://127.0.0.1:35413' processes=3 threads=6, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_combine_echodata' module='echoflow.stages.subflows.combine_echodata' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "Cleaning :  s3://echoflow-workground/combined_files_dask/echoflow_open_raw\n",
+      "s3://echoflow-workground/combined_files_dask/echoflow_open_raw\n",
+      "Failed to cleanup s3://echoflow-workground/combined_files_dask/echoflow_open_raw\n",
+      "\n",
+      "Completed stage name='echoflow_combine_echodata' module='echoflow.stages.subflows.combine_echodata' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "<Client: 'tcp://127.0.0.1:35413' processes=3 threads=6, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_compute_SV' module='echoflow.stages.subflows.compute_SV' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "Cleaning :  s3://echoflow-workground/combined_files_dask/echoflow_combine_echodata\n",
+      "s3://echoflow-workground/combined_files_dask/echoflow_combine_echodata\n",
+      "Failed to cleanup s3://echoflow-workground/combined_files_dask/echoflow_combine_echodata\n",
+      "\n",
+      "Completed stage name='echoflow_compute_SV' module='echoflow.stages.subflows.compute_SV' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "<Client: 'tcp://127.0.0.1:35413' processes=3 threads=6, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_compute_MVBS' module='echoflow.stages.subflows.compute_MVBS' external_params={'range_meter_bin': 20, 'ping_time_bin': '20S'} options={'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "Cleaning :  s3://echoflow-workground/combined_files_dask/echoflow_compute_SV\n",
+      "Cleanup complete\n",
+      "\n",
+      "Completed stage name='echoflow_compute_MVBS' module='echoflow.stages.subflows.compute_MVBS' external_params={'range_meter_bin': 20, 'ping_time_bin': '20S'} options={'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "Local Client has been closed\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/tmp/ipykernel_7087/3922359326.py:2: RuntimeWarning: coroutine 'Block.load' was never awaited\n",
+      "  data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, options=options)\n",
+      "RuntimeWarning: Enable tracemalloc to get the object allocation traceback\n"
+     ]
+    }
+   ],
+   "source": [
+    "options = {\"storage_options_override\": False}\n",
+    "data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, options=options)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 5: Results\n",
+    "Finally, let's take a look at the first entry from the processed data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[[<xarray.Dataset>\n",
+       "  Dimensions:            (channel: 3, ping_time: 456, echo_range: 38)\n",
+       "  Coordinates:\n",
+       "    * channel            (channel) <U37 'GPT  18 kHz 009072058c8d 1-1 ES18-11' ...\n",
+       "    * echo_range         (echo_range) float64 0.0 20.0 40.0 ... 700.0 720.0 740.0\n",
+       "    * ping_time          (ping_time) datetime64[ns] 2017-06-15T19:02:00 ... 201...\n",
+       "  Data variables:\n",
+       "      Sv                 (channel, ping_time, echo_range) float64 dask.array<chunksize=(2, 456, 38), meta=np.ndarray>\n",
+       "      frequency_nominal  (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "  Attributes:\n",
+       "      processing_function:          commongrid.compute_MVBS\n",
+       "      processing_software_name:     echopype\n",
+       "      processing_software_version:  0.7.2.dev51+gb45c942\n",
+       "      processing_time:              2023-08-30T15:54:41Z]]"
+      ]
+     },
+     "execution_count": 5,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/echoflow/tests/AWS/datastore.yaml b/echoflow/tests/AWS/datastore.yaml
new file mode 100644
index 0000000..35a1856
--- /dev/null
+++ b/echoflow/tests/AWS/datastore.yaml
@@ -0,0 +1,22 @@
+name: Bell_M._Shimada-SH1707-EK60
+sonar_model: EK60 
+raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6}) 
+args: 
+  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw
+  parameters:
+    ship_name: Bell_M._Shimada
+    survey_name: SH1707
+    sonar_model: EK60
+  storage_options:
+    anon: true
+  transect:
+    file: ./EK60_SH1707_Shimada.txt
+    default_transect_num: 2017
+  json_export: true 
+output: # Output arguments
+  urlpath: <YOUR-S3-BUCKET>
+  retention: false
+  overwrite: true 
+  storage_options: 
+    block_name: echoflow-aws-credentials
+    type: AWS
\ No newline at end of file
diff --git a/echoflow/tests/AWS/pipeline.yaml b/echoflow/tests/AWS/pipeline.yaml
new file mode 100644
index 0000000..975f3a5
--- /dev/null
+++ b/echoflow/tests/AWS/pipeline.yaml
@@ -0,0 +1,27 @@
+active_recipe: standard 
+use_local_dask: true
+n_workers: 3
+pipeline:
+- recipe_name: standard 
+  stages: 
+  - name: echoflow_open_raw 
+    module: echoflow.stages.subflows.open_raw 
+    options: 
+      save_raw_file: true
+      use_raw_offline: true 
+      use_offline: true 
+  - name: echoflow_combine_echodata
+    module: echoflow.stages.subflows.combine_echodata
+    options:
+      use_offline: true
+  - name: echoflow_compute_SV
+    module: echoflow.stages.subflows.compute_SV
+    options:
+      use_offline: true
+  - name: echoflow_compute_MVBS
+    module: echoflow.stages.subflows.compute_MVBS
+    options:
+      use_offline: true
+    external_params:
+      range_meter_bin: 20 
+      ping_time_bin: 20S
diff --git a/echoflow/tests/offline/LocalEchoflowDemo.ipynb b/echoflow/tests/offline/LocalEchoflowDemo.ipynb
new file mode 100644
index 0000000..d790045
--- /dev/null
+++ b/echoflow/tests/offline/LocalEchoflowDemo.ipynb
@@ -0,0 +1,1737 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Getting Started with Echoflow: Processing EK60 Data to Extract Target Strength\n",
+    "\n",
+    "Welcome to this beginner-friendly notebook that guides you through the process of using `echoflow` to process EK60 data from the SH1707 survey. `echoflow` is a powerful tool for ocean acoustics data processing. In this notebook, we'll walk through the following steps:\n",
+    "\n",
+    "1. **Setting Up**: Importing necessary libraries and setting up the configuration paths for the dataset and pipeline.\n",
+    "\n",
+    "2. **Getting Data**: Using `glob_url` to retrieve a list of URLs matching the specified pattern of raw EK60 data files.\n",
+    "\n",
+    "3. **Preparing Files**: Extracting file names from URLs and creating a file listing for the transect.\n",
+    "\n",
+    "4. **Processing with echoflow**: Starting the echoflow processing using the specified configurations.\n",
+    "\n",
+    "5. **Results**: Displaying the first entry from the processed data.\n",
+    "\n",
+    "Let's get started!"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "# Step 0: Install Echoflow\n",
+    "\n",
+    "Before we proceed with the instructions on using `echoflow`, let's ensure that we have properly set up and prepared our system with echoflow for usage. Feel free to skip this step if you have echoflow already set up.\n",
+    "\n",
+    "### Step 0.1: Make a virtual environment\n",
+    "\n",
+    "To keep your Echoflow environment isolated, it's recommended to create a virtual environment using Conda or Python's built-in venv module. Here's an example using Conda:\n",
+    "\n",
+    "```bash\n",
+    "conda create --name echoflow-env\n",
+    "conda activate echoflow-env\n",
+    "```\n",
+    "\n",
+    "Or, using Python's venv:\n",
+    "\n",
+    "```bash\n",
+    "python -m venv echoflow-env\n",
+    "source echoflow-env/bin/activate  # On Windows, use `echoflow-env\\Scripts\\activate`\n",
+    "```\n",
+    "### Step 0.2: Clone the Repository\n",
+    "Now that you have a virtual environment set up, you can clone the Echoflow project repository to your local machine using the following command:\n",
+    "\n",
+    "```bash\n",
+    "git clone <echoflow_repo>\n",
+    "```\n",
+    "\n",
+    "### Step 0.3: Install the Package\n",
+    "Navigate to the project directory you've just cloned and install the Echoflow package. The -e flag is crucial as it enables editable mode, which is especially helpful during development and testing. Now, take a moment and let the echoflow do its thing while you enjoy your coffee.\n",
+    "\n",
+    "```bash\n",
+    "cd <project_directory>\n",
+    "pip install -e .\n",
+    "```\n",
+    "\n",
+    "---\n",
+    "\n",
+    "### Step 0.4: Initialize Echoflow and Prefect\n",
+    "\n",
+    "To kickstart your journey with Echoflow and Prefect, follow these simple initialization steps:\n",
+    "\n",
+    "##### 0.4.1 Initializing Echoflow\n",
+    "Begin by initializing Echoflow with the following command:\n",
+    "\n",
+    "```bash\n",
+    "echoflow init\n",
+    "```\n",
+    "\n",
+    "This command sets up the groundwork for your Echoflow environment, preparing it for seamless usage.\n",
+    "\n",
+    "##### 0.4.2 Initializing Prefect\n",
+    "For Prefect, initialization involves a few extra steps, including secure authentication. Enter the following command to initiate the Prefect authentication process:\n",
+    "\n",
+    "- If you have a Prefect Cloud account, provide your Prefect API key to securely link your account. Type your API key when prompted and press Enter.\n",
+    "\n",
+    "```bash\n",
+    "prefect cloud login\n",
+    "```\n",
+    "\n",
+    "- If you don't have a Prefect Cloud account yet, you can use local prefect account. This is especially useful for those who are just starting out and want to explore Prefect without an account.\n",
+    "\n",
+    "```bash\n",
+    "prefect profiles create echoflow-local\n",
+    "```\n",
+    "\n",
+    "\n",
+    "The initialization process will ensure that both Echoflow and Prefect are properly set up and ready for you to dive into your cloud-based workflows.\n",
+    "\n",
+    "---"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1: Setting Up\n",
+    "\n",
+    "We begin by importing the required libraries and specifying the paths for the dataset and pipeline configuration files. These files contain the necessary information for data processing."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "from pathlib import Path\n",
+    "from echoflow import echoflow_start, glob_url\n",
+    "\n",
+    "dataset_config = Path(\"./datastore.yaml\").resolve()\n",
+    "pipeline_config = Path(\"./pipeline.yaml\").resolve()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1.1\n",
+    "### Pipeline Configuration: Target Strength Processing\n",
+    "In this section, we will provide you with the pipeline configuration that we'll be using for our target strength processing. The configuration is presented in YAML format, which is a structured and human-readable way to define settings for data processing.\n",
+    "\n",
+    "Here's the configuration we'll be using:\n",
+    "\n",
+    "```yaml\n",
+    "active_recipe: target_strength \n",
+    "use_local_dask: true\n",
+    "n_workers: 5\n",
+    "pipeline:\n",
+    "- recipe_name: target_strength\n",
+    "  stages:\n",
+    "  - name: echoflow_open_raw\n",
+    "    module: echoflow.stages.subflows.open_raw\n",
+    "    options:\n",
+    "      save_raw_file: true\n",
+    "      use_raw_offline: true\n",
+    "      use_offline: true\n",
+    "  - name: echoflow_compute_TS\n",
+    "    module: echoflow.stages.subflows.compute_TS\n",
+    "    options:\n",
+    "      use_offline: true\n",
+    "```\n",
+    "\n",
+    "Let's break down the components of this configuration:\n",
+    "\n",
+    "- **active_recipe**: Specifies the recipe to be used for processing, which is set as \"target_strength\" in this case.\n",
+    "\n",
+    "- **use_local_dask**: This flag indicates that we'll be utilizing a local Dask Cluster for parallel processing.\n",
+    "\n",
+    "- **n_workers**: Determines the number of worker processes in the Dask Cluster. Here, we're using 5 workers for efficient parallelization.\n",
+    "\n",
+    "- **pipeline**: This section defines the sequence of stages to execute. In this example, we're following the \"target_strength\" recipe, which comprises two stages.\n",
+    "\n",
+    "- **echoflow_open_raw**: This stage utilizes the `open_raw` subflow module to open raw data files. It includes options such as saving raw files, using raw data in offline mode, and utilizing offline data.\n",
+    "\n",
+    "- **echoflow_compute_TS**: This stage employs the `compute_TS` subflow module to compute target strength. It includes an option to use offline data.\n",
+    "\n",
+    "**Note**: For a more comprehensive understanding of each option and its functionality, you can refer to the [Pipeline documentation](https://github.com/OSOceanAcoustics/echoflow/blob/dev/docs/configuration/pipeline.md).\n",
+    "\n",
+    "Keep in mind that in this example, we'll be setting up a local Dask Cluster with 5 workers for parallel processing. This configuration will enable us to efficiently process our data for target strength analysis. To turn it off, toggle `use_local_dask` to false.\n",
+    "\n",
+    "Feel free to explore and modify the configuration to understand better."
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 1.2\n",
+    "### Datastore Configuration: Organizing Data for Processing\n",
+    "\n",
+    "In this section, we'll delve into the configuration that defines how the data will be organized and managed for processing. This configuration is provided in YAML format and plays a crucial role in structuring data inputs and outputs.\n",
+    "\n",
+    "Here's the detailed breakdown of the configuration:\n",
+    "\n",
+    "```yaml\n",
+    "name: Bell_M._Shimada-SH1707-EK60\n",
+    "sonar_model: EK60 \n",
+    "raw_regex: (.*)-?D(?P<date>\\w{1,8})-T(?P<time>\\w{1,6}) \n",
+    "args: \n",
+    "  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw\n",
+    "  parameters:\n",
+    "    ship_name: Bell_M._Shimada\n",
+    "    survey_name: SH1707\n",
+    "    sonar_model: EK60\n",
+    "  storage_options:\n",
+    "    anon: true\n",
+    "  transect:\n",
+    "    file: ./EK60_SH1707_Shimada.txt\n",
+    "    default_transect_num: 2017\n",
+    "  json_export: true \n",
+    "output: \n",
+    "  urlpath: ./echoflow-output\n",
+    "  retention: false\n",
+    "  overwrite: true\n",
+    "```\n",
+    "\n",
+    "Let's delve into the individual components of the configuration presented here:\n",
+    "\n",
+    "- **name**: Specifies a descriptive name for the configuration, aiding in identifying its purpose.\n",
+    "\n",
+    "- **sonar_model**: Indicates the type of sonar being utilized, which in this case is \"EK60\".\n",
+    "\n",
+    "- **raw_regex**: Defines a regular expression pattern for extracting date and time information from raw data file names.\n",
+    "\n",
+    "- **args**: This section provides crucial arguments for structuring data inputs:\n",
+    "\n",
+    "  - **urlpath**: Defines the URL pattern to access the raw data files stored on a remote server. The placeholders `{{ ship_name }}`, `{{ survey_name }}`, and `{{ sonar_model }}` are dynamically replaced with the specified values.\n",
+    "\n",
+    "  - **storage_options**: Sets storage options, such as anonymous access (`anon: true`), for retrieving the data.\n",
+    "\n",
+    "  - **transect**: Specifies a file (`EK60_SH1707_Shimada.txt`) containing the list of files to process, along with default transect information.\n",
+    "\n",
+    "  - **json_export**: Enables JSON metadata export.\n",
+    "\n",
+    "- **output**: This section configures the output settings for processed data:\n",
+    "\n",
+    "  - **urlpath**: Determines the output directory (`./echoflow-output`) where the processed data will be stored.\n",
+    "\n",
+    "  - **retention**: Disables data retention, indicating that only Target Strength data will be stored in this case.\n",
+    "\n",
+    "  - **overwrite**: Allows data overwriting if the data already exists.\n",
+    "\n",
+    "**Note**: \n",
+    "- For a more comprehensive understanding of each option and its functionality, you can refer to the [Datast documentation](https://github.com/OSOceanAcoustics/echoflow/blob/dev/docs/configuration/datastore.md).\n",
+    "- The pipeline will store Target Strength output under `./echoflow-output`. As the retention is set to false, only Target Strength files will be stored. To specify files for processing, create a list of file names and store it in `EK60_SH1707_Shimada.txt`, which should be placed under the transect directory.\n",
+    "\n",
+    "This configuration facilitates efficient data organization and management for the processing pipeline. Feel free to tailor it to your specific data and processing requirements.\n"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 2: Getting Data\n",
+    "Next, we'll use the glob_url function to retrieve a list of URLs matching a specific pattern. In this case, we're targeting raw EK60 data files from the SH1707 survey."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "all_files = glob_url(\"s3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/*.raw\", {'anon':True})"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 3: Preparing Files\n",
+    "We'll now extract the file names from the URLs and create a file listing for the transect. This will help us organize and work with the data effectively."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "files = []\n",
+    "for file in all_files:\n",
+    "    f = file.split(\".r\")[0]\n",
+    "    files.append(f.split(\"/\")[-1])\n",
+    "\n",
+    "transect = open('EK60_SH1707_Shimada.txt','w')\n",
+    "i = 0\n",
+    "for f in files:\n",
+    "    if i == 100:\n",
+    "        break\n",
+    "    transect.write(f+\".raw\\n\")\n",
+    "    i = i + 1\n",
+    "transect.close()"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 4: Processing with echoflow\n",
+    "Now, we're ready to kick off the data processing using echoflow. We'll provide the dataset and pipeline configurations, along with additional options."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<Client: 'tcp://127.0.0.1:43583' processes=5 threads=5, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params=None options={'save_raw_file': True, 'use_raw_offline': True, 'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "\n",
+      "Completed stage name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params=None options={'save_raw_file': True, 'use_raw_offline': True, 'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "<Client: 'tcp://127.0.0.1:43583' processes=5 threads=5, memory=15.61 GiB>\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_compute_TS' module='echoflow.stages.subflows.compute_TS' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n",
+      "Cleaning :  /home/ubuntu/echoflow/echoflow/echoflow/tests/offline/echoflow-output/echoflow_open_raw\n",
+      "\n",
+      "Completed stage name='echoflow_compute_TS' module='echoflow.stages.subflows.compute_TS' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "--------------------------------------------------\n",
+      "Local Client has been closed\n"
+     ]
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/tmp/ipykernel_103521/3922359326.py:2: RuntimeWarning: coroutine 'Block.load' was never awaited\n",
+      "  data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, options=options)\n",
+      "RuntimeWarning: Enable tracemalloc to get the object allocation traceback\n"
+     ]
+    }
+   ],
+   "source": [
+    "options = {\"storage_options_override\": False}\n",
+    "data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, options=options)"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "## Step 5: Results\n",
+    "Finally, let's take a look at the first entry from the processed data."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<div><svg style=\"position: absolute; width: 0; height: 0; overflow: hidden\">\n",
+       "<defs>\n",
+       "<symbol id=\"icon-database\" viewBox=\"0 0 32 32\">\n",
+       "<path d=\"M16 0c-8.837 0-16 2.239-16 5v4c0 2.761 7.163 5 16 5s16-2.239 16-5v-4c0-2.761-7.163-5-16-5z\"></path>\n",
+       "<path d=\"M16 17c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
+       "<path d=\"M16 26c-8.837 0-16-2.239-16-5v6c0 2.761 7.163 5 16 5s16-2.239 16-5v-6c0 2.761-7.163 5-16 5z\"></path>\n",
+       "</symbol>\n",
+       "<symbol id=\"icon-file-text2\" viewBox=\"0 0 32 32\">\n",
+       "<path d=\"M28.681 7.159c-0.694-0.947-1.662-2.053-2.724-3.116s-2.169-2.030-3.116-2.724c-1.612-1.182-2.393-1.319-2.841-1.319h-15.5c-1.378 0-2.5 1.121-2.5 2.5v27c0 1.378 1.122 2.5 2.5 2.5h23c1.378 0 2.5-1.122 2.5-2.5v-19.5c0-0.448-0.137-1.23-1.319-2.841zM24.543 5.457c0.959 0.959 1.712 1.825 2.268 2.543h-4.811v-4.811c0.718 0.556 1.584 1.309 2.543 2.268zM28 29.5c0 0.271-0.229 0.5-0.5 0.5h-23c-0.271 0-0.5-0.229-0.5-0.5v-27c0-0.271 0.229-0.5 0.5-0.5 0 0 15.499-0 15.5 0v7c0 0.552 0.448 1 1 1h7v19.5z\"></path>\n",
+       "<path d=\"M23 26h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
+       "<path d=\"M23 22h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
+       "<path d=\"M23 18h-14c-0.552 0-1-0.448-1-1s0.448-1 1-1h14c0.552 0 1 0.448 1 1s-0.448 1-1 1z\"></path>\n",
+       "</symbol>\n",
+       "</defs>\n",
+       "</svg>\n",
+       "<style>/* CSS stylesheet for displaying xarray objects in jupyterlab.\n",
+       " *\n",
+       " */\n",
+       "\n",
+       ":root {\n",
+       "  --xr-font-color0: var(--jp-content-font-color0, rgba(0, 0, 0, 1));\n",
+       "  --xr-font-color2: var(--jp-content-font-color2, rgba(0, 0, 0, 0.54));\n",
+       "  --xr-font-color3: var(--jp-content-font-color3, rgba(0, 0, 0, 0.38));\n",
+       "  --xr-border-color: var(--jp-border-color2, #e0e0e0);\n",
+       "  --xr-disabled-color: var(--jp-layout-color3, #bdbdbd);\n",
+       "  --xr-background-color: var(--jp-layout-color0, white);\n",
+       "  --xr-background-color-row-even: var(--jp-layout-color1, white);\n",
+       "  --xr-background-color-row-odd: var(--jp-layout-color2, #eeeeee);\n",
+       "}\n",
+       "\n",
+       "html[theme=dark],\n",
+       "body[data-theme=dark],\n",
+       "body.vscode-dark {\n",
+       "  --xr-font-color0: rgba(255, 255, 255, 1);\n",
+       "  --xr-font-color2: rgba(255, 255, 255, 0.54);\n",
+       "  --xr-font-color3: rgba(255, 255, 255, 0.38);\n",
+       "  --xr-border-color: #1F1F1F;\n",
+       "  --xr-disabled-color: #515151;\n",
+       "  --xr-background-color: #111111;\n",
+       "  --xr-background-color-row-even: #111111;\n",
+       "  --xr-background-color-row-odd: #313131;\n",
+       "}\n",
+       "\n",
+       ".xr-wrap {\n",
+       "  display: block !important;\n",
+       "  min-width: 300px;\n",
+       "  max-width: 700px;\n",
+       "}\n",
+       "\n",
+       ".xr-text-repr-fallback {\n",
+       "  /* fallback to plain text repr when CSS is not injected (untrusted notebook) */\n",
+       "  display: none;\n",
+       "}\n",
+       "\n",
+       ".xr-header {\n",
+       "  padding-top: 6px;\n",
+       "  padding-bottom: 6px;\n",
+       "  margin-bottom: 4px;\n",
+       "  border-bottom: solid 1px var(--xr-border-color);\n",
+       "}\n",
+       "\n",
+       ".xr-header > div,\n",
+       ".xr-header > ul {\n",
+       "  display: inline;\n",
+       "  margin-top: 0;\n",
+       "  margin-bottom: 0;\n",
+       "}\n",
+       "\n",
+       ".xr-obj-type,\n",
+       ".xr-array-name {\n",
+       "  margin-left: 2px;\n",
+       "  margin-right: 10px;\n",
+       "}\n",
+       "\n",
+       ".xr-obj-type {\n",
+       "  color: var(--xr-font-color2);\n",
+       "}\n",
+       "\n",
+       ".xr-sections {\n",
+       "  padding-left: 0 !important;\n",
+       "  display: grid;\n",
+       "  grid-template-columns: 150px auto auto 1fr 20px 20px;\n",
+       "}\n",
+       "\n",
+       ".xr-section-item {\n",
+       "  display: contents;\n",
+       "}\n",
+       "\n",
+       ".xr-section-item input {\n",
+       "  display: none;\n",
+       "}\n",
+       "\n",
+       ".xr-section-item input + label {\n",
+       "  color: var(--xr-disabled-color);\n",
+       "}\n",
+       "\n",
+       ".xr-section-item input:enabled + label {\n",
+       "  cursor: pointer;\n",
+       "  color: var(--xr-font-color2);\n",
+       "}\n",
+       "\n",
+       ".xr-section-item input:enabled + label:hover {\n",
+       "  color: var(--xr-font-color0);\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary {\n",
+       "  grid-column: 1;\n",
+       "  color: var(--xr-font-color2);\n",
+       "  font-weight: 500;\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary > span {\n",
+       "  display: inline-block;\n",
+       "  padding-left: 0.5em;\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in:disabled + label {\n",
+       "  color: var(--xr-font-color2);\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in + label:before {\n",
+       "  display: inline-block;\n",
+       "  content: '►';\n",
+       "  font-size: 11px;\n",
+       "  width: 15px;\n",
+       "  text-align: center;\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in:disabled + label:before {\n",
+       "  color: var(--xr-disabled-color);\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in:checked + label:before {\n",
+       "  content: '▼';\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in:checked + label > span {\n",
+       "  display: none;\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary,\n",
+       ".xr-section-inline-details {\n",
+       "  padding-top: 4px;\n",
+       "  padding-bottom: 4px;\n",
+       "}\n",
+       "\n",
+       ".xr-section-inline-details {\n",
+       "  grid-column: 2 / -1;\n",
+       "}\n",
+       "\n",
+       ".xr-section-details {\n",
+       "  display: none;\n",
+       "  grid-column: 1 / -1;\n",
+       "  margin-bottom: 5px;\n",
+       "}\n",
+       "\n",
+       ".xr-section-summary-in:checked ~ .xr-section-details {\n",
+       "  display: contents;\n",
+       "}\n",
+       "\n",
+       ".xr-array-wrap {\n",
+       "  grid-column: 1 / -1;\n",
+       "  display: grid;\n",
+       "  grid-template-columns: 20px auto;\n",
+       "}\n",
+       "\n",
+       ".xr-array-wrap > label {\n",
+       "  grid-column: 1;\n",
+       "  vertical-align: top;\n",
+       "}\n",
+       "\n",
+       ".xr-preview {\n",
+       "  color: var(--xr-font-color3);\n",
+       "}\n",
+       "\n",
+       ".xr-array-preview,\n",
+       ".xr-array-data {\n",
+       "  padding: 0 5px !important;\n",
+       "  grid-column: 2;\n",
+       "}\n",
+       "\n",
+       ".xr-array-data,\n",
+       ".xr-array-in:checked ~ .xr-array-preview {\n",
+       "  display: none;\n",
+       "}\n",
+       "\n",
+       ".xr-array-in:checked ~ .xr-array-data,\n",
+       ".xr-array-preview {\n",
+       "  display: inline-block;\n",
+       "}\n",
+       "\n",
+       ".xr-dim-list {\n",
+       "  display: inline-block !important;\n",
+       "  list-style: none;\n",
+       "  padding: 0 !important;\n",
+       "  margin: 0;\n",
+       "}\n",
+       "\n",
+       ".xr-dim-list li {\n",
+       "  display: inline-block;\n",
+       "  padding: 0;\n",
+       "  margin: 0;\n",
+       "}\n",
+       "\n",
+       ".xr-dim-list:before {\n",
+       "  content: '(';\n",
+       "}\n",
+       "\n",
+       ".xr-dim-list:after {\n",
+       "  content: ')';\n",
+       "}\n",
+       "\n",
+       ".xr-dim-list li:not(:last-child):after {\n",
+       "  content: ',';\n",
+       "  padding-right: 5px;\n",
+       "}\n",
+       "\n",
+       ".xr-has-index {\n",
+       "  font-weight: bold;\n",
+       "}\n",
+       "\n",
+       ".xr-var-list,\n",
+       ".xr-var-item {\n",
+       "  display: contents;\n",
+       "}\n",
+       "\n",
+       ".xr-var-item > div,\n",
+       ".xr-var-item label,\n",
+       ".xr-var-item > .xr-var-name span {\n",
+       "  background-color: var(--xr-background-color-row-even);\n",
+       "  margin-bottom: 0;\n",
+       "}\n",
+       "\n",
+       ".xr-var-item > .xr-var-name:hover span {\n",
+       "  padding-right: 5px;\n",
+       "}\n",
+       "\n",
+       ".xr-var-list > li:nth-child(odd) > div,\n",
+       ".xr-var-list > li:nth-child(odd) > label,\n",
+       ".xr-var-list > li:nth-child(odd) > .xr-var-name span {\n",
+       "  background-color: var(--xr-background-color-row-odd);\n",
+       "}\n",
+       "\n",
+       ".xr-var-name {\n",
+       "  grid-column: 1;\n",
+       "}\n",
+       "\n",
+       ".xr-var-dims {\n",
+       "  grid-column: 2;\n",
+       "}\n",
+       "\n",
+       ".xr-var-dtype {\n",
+       "  grid-column: 3;\n",
+       "  text-align: right;\n",
+       "  color: var(--xr-font-color2);\n",
+       "}\n",
+       "\n",
+       ".xr-var-preview {\n",
+       "  grid-column: 4;\n",
+       "}\n",
+       "\n",
+       ".xr-index-preview {\n",
+       "  grid-column: 2 / 5;\n",
+       "  color: var(--xr-font-color2);\n",
+       "}\n",
+       "\n",
+       ".xr-var-name,\n",
+       ".xr-var-dims,\n",
+       ".xr-var-dtype,\n",
+       ".xr-preview,\n",
+       ".xr-attrs dt {\n",
+       "  white-space: nowrap;\n",
+       "  overflow: hidden;\n",
+       "  text-overflow: ellipsis;\n",
+       "  padding-right: 10px;\n",
+       "}\n",
+       "\n",
+       ".xr-var-name:hover,\n",
+       ".xr-var-dims:hover,\n",
+       ".xr-var-dtype:hover,\n",
+       ".xr-attrs dt:hover {\n",
+       "  overflow: visible;\n",
+       "  width: auto;\n",
+       "  z-index: 1;\n",
+       "}\n",
+       "\n",
+       ".xr-var-attrs,\n",
+       ".xr-var-data,\n",
+       ".xr-index-data {\n",
+       "  display: none;\n",
+       "  background-color: var(--xr-background-color) !important;\n",
+       "  padding-bottom: 5px !important;\n",
+       "}\n",
+       "\n",
+       ".xr-var-attrs-in:checked ~ .xr-var-attrs,\n",
+       ".xr-var-data-in:checked ~ .xr-var-data,\n",
+       ".xr-index-data-in:checked ~ .xr-index-data {\n",
+       "  display: block;\n",
+       "}\n",
+       "\n",
+       ".xr-var-data > table {\n",
+       "  float: right;\n",
+       "}\n",
+       "\n",
+       ".xr-var-name span,\n",
+       ".xr-var-data,\n",
+       ".xr-index-name div,\n",
+       ".xr-index-data,\n",
+       ".xr-attrs {\n",
+       "  padding-left: 25px !important;\n",
+       "}\n",
+       "\n",
+       ".xr-attrs,\n",
+       ".xr-var-attrs,\n",
+       ".xr-var-data,\n",
+       ".xr-index-data {\n",
+       "  grid-column: 1 / -1;\n",
+       "}\n",
+       "\n",
+       "dl.xr-attrs {\n",
+       "  padding: 0;\n",
+       "  margin: 0;\n",
+       "  display: grid;\n",
+       "  grid-template-columns: 125px auto;\n",
+       "}\n",
+       "\n",
+       ".xr-attrs dt,\n",
+       ".xr-attrs dd {\n",
+       "  padding: 0;\n",
+       "  margin: 0;\n",
+       "  float: left;\n",
+       "  padding-right: 10px;\n",
+       "  width: auto;\n",
+       "}\n",
+       "\n",
+       ".xr-attrs dt {\n",
+       "  font-weight: normal;\n",
+       "  grid-column: 1;\n",
+       "}\n",
+       "\n",
+       ".xr-attrs dt:hover span {\n",
+       "  display: inline-block;\n",
+       "  background: var(--xr-background-color);\n",
+       "  padding-right: 10px;\n",
+       "}\n",
+       "\n",
+       ".xr-attrs dd {\n",
+       "  grid-column: 2;\n",
+       "  white-space: pre-wrap;\n",
+       "  word-break: break-all;\n",
+       "}\n",
+       "\n",
+       ".xr-icon-database,\n",
+       ".xr-icon-file-text2,\n",
+       ".xr-no-icon {\n",
+       "  display: inline-block;\n",
+       "  vertical-align: middle;\n",
+       "  width: 1em;\n",
+       "  height: 1.5em !important;\n",
+       "  stroke-width: 0;\n",
+       "  stroke: currentColor;\n",
+       "  fill: currentColor;\n",
+       "}\n",
+       "</style><pre class='xr-text-repr-fallback'>&lt;xarray.Dataset&gt;\n",
+       "Dimensions:                        (channel: 3, ping_time: 19,\n",
+       "                                    range_sample: 3888, filenames: 1)\n",
+       "Coordinates:\n",
+       "  * channel                        (channel) &lt;U37 &#x27;GPT  18 kHz 009072058c8d 1...\n",
+       "  * filenames                      (filenames) int64 0\n",
+       "  * ping_time                      (ping_time) datetime64[ns] 2017-06-15T19:0...\n",
+       "  * range_sample                   (range_sample) int64 0 1 2 ... 3885 3886 3887\n",
+       "Data variables: (12/16)\n",
+       "    TS                             (channel, ping_time, range_sample) float64 dask.array&lt;chunksize=(2, 10, 1944), meta=np.ndarray&gt;\n",
+       "    angle_offset_alongship         (channel) float64 dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;\n",
+       "    angle_offset_athwartship       (channel) float64 dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;\n",
+       "    angle_sensitivity_alongship    (channel) float64 dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;\n",
+       "    angle_sensitivity_athwartship  (channel) float64 dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;\n",
+       "    beamwidth_alongship            (channel) float64 dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;\n",
+       "    ...                             ...\n",
+       "    gain_correction                (ping_time, channel) float64 dask.array&lt;chunksize=(19, 3), meta=np.ndarray&gt;\n",
+       "    sa_correction                  (ping_time, channel) float64 dask.array&lt;chunksize=(19, 3), meta=np.ndarray&gt;\n",
+       "    sound_absorption               (channel, ping_time) float64 dask.array&lt;chunksize=(3, 19), meta=np.ndarray&gt;\n",
+       "    sound_speed                    (channel, ping_time) float64 dask.array&lt;chunksize=(3, 19), meta=np.ndarray&gt;\n",
+       "    source_filenames               (filenames) &lt;U126 dask.array&lt;chunksize=(1,), meta=np.ndarray&gt;\n",
+       "    water_level                    float64 ...\n",
+       "Attributes:\n",
+       "    processing_function:          calibrate.compute_TS\n",
+       "    processing_software_name:     echopype\n",
+       "    processing_software_version:  0.7.2.dev51+gb45c942\n",
+       "    processing_time:              2023-08-29T19:38:55Z</pre><div class='xr-wrap' style='display:none'><div class='xr-header'><div class='xr-obj-type'>xarray.Dataset</div></div><ul class='xr-sections'><li class='xr-section-item'><input id='section-c3c8008d-3751-475a-99cb-a70fd0a1bf88' class='xr-section-summary-in' type='checkbox' disabled ><label for='section-c3c8008d-3751-475a-99cb-a70fd0a1bf88' class='xr-section-summary'  title='Expand/collapse section'>Dimensions:</label><div class='xr-section-inline-details'><ul class='xr-dim-list'><li><span class='xr-has-index'>channel</span>: 3</li><li><span class='xr-has-index'>ping_time</span>: 19</li><li><span class='xr-has-index'>range_sample</span>: 3888</li><li><span class='xr-has-index'>filenames</span>: 1</li></ul></div><div class='xr-section-details'></div></li><li class='xr-section-item'><input id='section-f7d1679a-cfaa-4645-9707-e7665d2195ec' class='xr-section-summary-in' type='checkbox'  checked><label for='section-f7d1679a-cfaa-4645-9707-e7665d2195ec' class='xr-section-summary' >Coordinates: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>channel</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>&lt;U37</div><div class='xr-var-preview xr-preview'>&#x27;GPT  18 kHz 009072058c8d 1-1 ES...</div><input id='attrs-575c44e3-970d-40ad-9fea-1606b1dbe41f' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-575c44e3-970d-40ad-9fea-1606b1dbe41f' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-75905c0c-5a8b-4d10-93b0-af840c931111' class='xr-var-data-in' type='checkbox'><label for='data-75905c0c-5a8b-4d10-93b0-af840c931111' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Vendor channel ID</dd></dl></div><div class='xr-var-data'><pre>array([&#x27;GPT  18 kHz 009072058c8d 1-1 ES18-11&#x27;,\n",
+       "       &#x27;GPT  38 kHz 009072058146 2-1 ES38B&#x27;,\n",
+       "       &#x27;GPT 120 kHz 00907205a6d0 4-1 ES120-7C&#x27;], dtype=&#x27;&lt;U37&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>filenames</span></div><div class='xr-var-dims'>(filenames)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0</div><input id='attrs-4c622053-df85-4128-901d-adfbccb1d72d' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-4c622053-df85-4128-901d-adfbccb1d72d' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-d3dc9f55-e5fe-4401-b234-5b3fb10b6b45' class='xr-var-data-in' type='checkbox'><label for='data-d3dc9f55-e5fe-4401-b234-5b3fb10b6b45' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Index for data and metadata source filenames</dd></dl></div><div class='xr-var-data'><pre>array([0])</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>ping_time</span></div><div class='xr-var-dims'>(ping_time)</div><div class='xr-var-dtype'>datetime64[ns]</div><div class='xr-var-preview xr-preview'>2017-06-15T19:02:14.206856192 .....</div><input id='attrs-d3ba0b04-2ceb-4ff5-92bb-41df33e62fd7' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-d3ba0b04-2ceb-4ff5-92bb-41df33e62fd7' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-faecbc63-8762-4d16-adc9-22524e2d9cbb' class='xr-var-data-in' type='checkbox'><label for='data-faecbc63-8762-4d16-adc9-22524e2d9cbb' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><pre>array([&#x27;2017-06-15T19:02:14.206856192&#x27;, &#x27;2017-06-15T19:02:16.784002048&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:19.356149760&#x27;, &#x27;2017-06-15T19:02:21.927298048&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:24.495442944&#x27;, &#x27;2017-06-15T19:02:27.073590784&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:29.644739072&#x27;, &#x27;2017-06-15T19:02:32.213883904&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:34.794033152&#x27;, &#x27;2017-06-15T19:02:37.374179840&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:39.951328256&#x27;, &#x27;2017-06-15T19:02:42.524474880&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:45.094622208&#x27;, &#x27;2017-06-15T19:02:47.672770048&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:50.241916928&#x27;, &#x27;2017-06-15T19:02:52.814064128&#x27;,\n",
+       "       &#x27;2017-06-15T19:02:55.394345984&#x27;, &#x27;2017-06-15T19:02:57.974493184&#x27;,\n",
+       "       &#x27;2017-06-15T19:03:00.543640064&#x27;], dtype=&#x27;datetime64[ns]&#x27;)</pre></div></li><li class='xr-var-item'><div class='xr-var-name'><span class='xr-has-index'>range_sample</span></div><div class='xr-var-dims'>(range_sample)</div><div class='xr-var-dtype'>int64</div><div class='xr-var-preview xr-preview'>0 1 2 3 4 ... 3884 3885 3886 3887</div><input id='attrs-602fc0f7-cef9-43eb-8946-aa4387983f04' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-602fc0f7-cef9-43eb-8946-aa4387983f04' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-f30d2187-e8f3-49fc-adbc-9ec8f5eb35f5' class='xr-var-data-in' type='checkbox'><label for='data-f30d2187-e8f3-49fc-adbc-9ec8f5eb35f5' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Along-range sample number, base 0</dd></dl></div><div class='xr-var-data'><pre>array([   0,    1,    2, ..., 3885, 3886, 3887])</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-3e96ea0b-7223-4890-9a14-c4d0b211b4fc' class='xr-section-summary-in' type='checkbox'  ><label for='section-3e96ea0b-7223-4890-9a14-c4d0b211b4fc' class='xr-section-summary' >Data variables: <span>(16)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-var-name'><span>TS</span></div><div class='xr-var-dims'>(channel, ping_time, range_sample)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(2, 10, 1944), meta=np.ndarray&gt;</div><input id='attrs-0d42ed02-d812-4e00-8ea5-8798f726745f' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-0d42ed02-d812-4e00-8ea5-8798f726745f' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-69d68080-1370-45f4-baa0-09a83c5cd26f' class='xr-var-data-in' type='checkbox'><label for='data-69d68080-1370-45f4-baa0-09a83c5cd26f' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>actual_range :</span></dt><dd>[-131.4, 65.9]</dd><dt><span>long_name :</span></dt><dd>Target strength (TS re 1 m^2)</dd><dt><span>units :</span></dt><dd>dB</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 1.69 MiB </td>\n",
+       "                        <td> 303.75 kiB </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3, 19, 3888) </td>\n",
+       "                        <td> (2, 10, 1944) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 8 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"194\" height=\"90\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"24\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"10\" y1=\"13\" x2=\"24\" y2=\"28\" />\n",
+       "  <line x1=\"10\" y1=\"25\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"25\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"19\" y1=\"9\" x2=\"19\" y2=\"35\" />\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"10.0,0.0 24.9485979497544,14.948597949754403 24.9485979497544,40.36121446433689 10.0,25.412616514582485\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"130\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"19\" y1=\"9\" x2=\"139\" y2=\"9\" />\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"24\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"70\" y1=\"0\" x2=\"84\" y2=\"14\" />\n",
+       "  <line x1=\"130\" y1=\"0\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"10.0,0.0 130.0,0.0 144.9485979497544,14.948597949754403 24.9485979497544,14.948597949754403\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"24\" y1=\"28\" x2=\"144\" y2=\"28\" />\n",
+       "  <line x1=\"24\" y1=\"40\" x2=\"144\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"84\" y1=\"14\" x2=\"84\" y2=\"40\" />\n",
+       "  <line x1=\"144\" y1=\"14\" x2=\"144\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"24.9485979497544,14.948597949754403 144.9485979497544,14.948597949754403 144.9485979497544,40.36121446433689 24.9485979497544,40.36121446433689\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"84.948598\" y=\"60.361214\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3888</text>\n",
+       "  <text x=\"164.948598\" y=\"27.654906\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,164.948598,27.654906)\">19</text>\n",
+       "  <text x=\"7.474299\" y=\"52.886915\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(45,7.474299,52.886915)\">3</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>angle_offset_alongship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-10e23ab2-9665-4295-ad38-816a90c20487' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-10e23ab2-9665-4295-ad38-816a90c20487' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-e50fae00-f2d3-4232-91b3-645f2aa33e95' class='xr-var-data-in' type='checkbox'><label for='data-e50fae00-f2d3-4232-91b3-645f2aa33e95' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders. The alongship angle corresponds to the minor angle in SONAR-netCDF4 vers 2. </dd><dt><span>long_name :</span></dt><dd>electrical alongship angle offset of the transducer</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>angle_offset_athwartship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-9eb79066-603a-4248-bc63-a0d86e6610cd' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-9eb79066-603a-4248-bc63-a0d86e6610cd' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-95128957-cb7c-4d36-912b-d40fb12cf0f7' class='xr-var-data-in' type='checkbox'><label for='data-95128957-cb7c-4d36-912b-d40fb12cf0f7' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders. The athwartship angle corresponds to the major angle in SONAR-netCDF4 vers 2. </dd><dt><span>long_name :</span></dt><dd>electrical athwartship angle offset of the transducer</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>angle_sensitivity_alongship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-563d1c5e-f9e5-4b3c-865c-0fa9bfec8252' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-563d1c5e-f9e5-4b3c-865c-0fa9bfec8252' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-bd74b1aa-afb6-4e13-bbda-64a4a84631f4' class='xr-var-data-in' type='checkbox'><label for='data-bd74b1aa-afb6-4e13-bbda-64a4a84631f4' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders. The alongship angle corresponds to the minor angle in SONAR-netCDF4 vers 2. </dd><dt><span>long_name :</span></dt><dd>alongship angle sensitivity of the transducer</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>angle_sensitivity_athwartship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-f0910ef7-0b07-4bff-8f9c-4cd9917af987' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-f0910ef7-0b07-4bff-8f9c-4cd9917af987' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-1219a487-e2d5-4eac-b284-67d3813315a8' class='xr-var-data-in' type='checkbox'><label for='data-1219a487-e2d5-4eac-b284-67d3813315a8' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders. The athwartship angle corresponds to the major angle in SONAR-netCDF4 vers 2. </dd><dt><span>long_name :</span></dt><dd>athwartship angle sensitivity of the transducer</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>beamwidth_alongship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-f115f3d7-9c44-4daa-ae89-53e0f8aa9706' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-f115f3d7-9c44-4daa-ae89-53e0f8aa9706' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-2c7075da-3ce6-4b49-98cb-b9bc84d5910c' class='xr-var-data-in' type='checkbox'><label for='data-2c7075da-3ce6-4b49-98cb-b9bc84d5910c' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders to avoid potential confusion with convention definitions. The alongship angle corresponds to the minor angle in SONAR-netCDF4 vers 2. The convention defines one-way transmit or receive beamwidth (beamwidth_receive_minor and beamwidth_transmit_minor), but Simrad echosounders record two-way beamwidth in the data.</dd><dt><span>long_name :</span></dt><dd>Half power two-way beam width along alongship axis of beam</dd><dt><span>units :</span></dt><dd>arc_degree</dd><dt><span>valid_range :</span></dt><dd>[0.0, 360.0]</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>beamwidth_athwartship</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-a9240292-1751-4868-85d6-eaf7164ce42c' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-a9240292-1751-4868-85d6-eaf7164ce42c' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-948ef537-d549-4e6f-8218-70314395bc4b' class='xr-var-data-in' type='checkbox'><label for='data-948ef537-d549-4e6f-8218-70314395bc4b' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>comment :</span></dt><dd>Introduced in echopype for Simrad echosounders to avoid potential confusion with convention definitions. The athwartship angle corresponds to the major angle in SONAR-netCDF4 vers 2. The convention defines one-way transmit or receive beamwidth (beamwidth_receive_major and beamwidth_transmit_major), but Simrad echosounders record two-way beamwidth in the data.</dd><dt><span>long_name :</span></dt><dd>Half power two-way beam width along athwartship axis of beam</dd><dt><span>units :</span></dt><dd>arc_degree</dd><dt><span>valid_range :</span></dt><dd>[0.0, 360.0]</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>echo_range</span></div><div class='xr-var-dims'>(channel, ping_time, range_sample)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(2, 10, 1944), meta=np.ndarray&gt;</div><input id='attrs-61d3b5e0-4730-4750-b3ce-91caf57f7dfd' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-61d3b5e0-4730-4750-b3ce-91caf57f7dfd' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-c261e30c-5462-461d-b85c-3784bece91cd' class='xr-var-data-in' type='checkbox'><label for='data-c261e30c-5462-461d-b85c-3784bece91cd' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Range distance</dd><dt><span>units :</span></dt><dd>m</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 1.69 MiB </td>\n",
+       "                        <td> 303.75 kiB </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3, 19, 3888) </td>\n",
+       "                        <td> (2, 10, 1944) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 8 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"194\" height=\"90\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"24\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"10\" y1=\"13\" x2=\"24\" y2=\"28\" />\n",
+       "  <line x1=\"10\" y1=\"25\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"10\" y2=\"25\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"19\" y1=\"9\" x2=\"19\" y2=\"35\" />\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"10.0,0.0 24.9485979497544,14.948597949754403 24.9485979497544,40.36121446433689 10.0,25.412616514582485\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"130\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"19\" y1=\"9\" x2=\"139\" y2=\"9\" />\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"10\" y1=\"0\" x2=\"24\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"70\" y1=\"0\" x2=\"84\" y2=\"14\" />\n",
+       "  <line x1=\"130\" y1=\"0\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"10.0,0.0 130.0,0.0 144.9485979497544,14.948597949754403 24.9485979497544,14.948597949754403\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"144\" y2=\"14\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"24\" y1=\"28\" x2=\"144\" y2=\"28\" />\n",
+       "  <line x1=\"24\" y1=\"40\" x2=\"144\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"24\" y1=\"14\" x2=\"24\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"84\" y1=\"14\" x2=\"84\" y2=\"40\" />\n",
+       "  <line x1=\"144\" y1=\"14\" x2=\"144\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"24.9485979497544,14.948597949754403 144.9485979497544,14.948597949754403 144.9485979497544,40.36121446433689 24.9485979497544,40.36121446433689\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"84.948598\" y=\"60.361214\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3888</text>\n",
+       "  <text x=\"164.948598\" y=\"27.654906\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,164.948598,27.654906)\">19</text>\n",
+       "  <text x=\"7.474299\" y=\"52.886915\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(45,7.474299,52.886915)\">3</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>equivalent_beam_angle</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-4fcbd454-4185-421e-b711-70ac0f89db3e' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-4fcbd454-4185-421e-b711-70ac0f89db3e' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-5a43d6a6-101a-4e20-9cae-91a0335f114c' class='xr-var-data-in' type='checkbox'><label for='data-5a43d6a6-101a-4e20-9cae-91a0335f114c' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Equivalent beam angle</dd><dt><span>units :</span></dt><dd>sr</dd><dt><span>valid_range :</span></dt><dd>[0.0, 12.566370614359172]</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>frequency_nominal</span></div><div class='xr-var-dims'>(channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3,), meta=np.ndarray&gt;</div><input id='attrs-9a712c8d-8633-4f35-bf75-ff756b59c2bb' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-9a712c8d-8633-4f35-bf75-ff756b59c2bb' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-10da5729-3af3-4ed4-b283-a85eeb4ae0fd' class='xr-var-data-in' type='checkbox'><label for='data-10da5729-3af3-4ed4-b283-a85eeb4ae0fd' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Transducer frequency</dd><dt><span>standard_name :</span></dt><dd>sound_frequency</dd><dt><span>units :</span></dt><dd>Hz</dd><dt><span>valid_min :</span></dt><dd>0.0</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 24 B </td>\n",
+       "                        <td> 24 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3,) </td>\n",
+       "                        <td> (3,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"93\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"43\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"43\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,43.88583450710766 0.0,43.88583450710766\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"63.885835\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"140.000000\" y=\"21.942917\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,21.942917)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>gain_correction</span></div><div class='xr-var-dims'>(ping_time, channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(19, 3), meta=np.ndarray&gt;</div><input id='attrs-5d752645-2bb5-43e1-870b-1428226aa823' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-5d752645-2bb5-43e1-870b-1428226aa823' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-8223f1ba-3203-453d-a135-da5ac419e935' class='xr-var-data-in' type='checkbox'><label for='data-8223f1ba-3203-453d-a135-da5ac419e935' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 456 B </td>\n",
+       "                        <td> 456 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (19, 3) </td>\n",
+       "                        <td> (19, 3) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"90\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"40\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"120\" x2=\"40\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"40\" y1=\"0\" x2=\"40\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 40.95137597507779,0.0 40.95137597507779,120.0 0.0,120.0\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"20.475688\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"60.951376\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,60.951376,60.000000)\">19</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>sa_correction</span></div><div class='xr-var-dims'>(ping_time, channel)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(19, 3), meta=np.ndarray&gt;</div><input id='attrs-28124778-908c-463c-b3da-86639517def1' class='xr-var-attrs-in' type='checkbox' disabled><label for='attrs-28124778-908c-463c-b3da-86639517def1' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-82084616-edf0-4dec-a971-78af864209ec' class='xr-var-data-in' type='checkbox'><label for='data-82084616-edf0-4dec-a971-78af864209ec' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 456 B </td>\n",
+       "                        <td> 456 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (19, 3) </td>\n",
+       "                        <td> (19, 3) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"90\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"40\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"120\" x2=\"40\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"40\" y1=\"0\" x2=\"40\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 40.95137597507779,0.0 40.95137597507779,120.0 0.0,120.0\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"20.475688\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >3</text>\n",
+       "  <text x=\"60.951376\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,60.951376,60.000000)\">19</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>sound_absorption</span></div><div class='xr-var-dims'>(channel, ping_time)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3, 19), meta=np.ndarray&gt;</div><input id='attrs-cb8231a1-d66d-4eb8-accb-cd65b33e7837' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-cb8231a1-d66d-4eb8-accb-cd65b33e7837' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-668919fd-f580-415c-9e1e-55a8355edada' class='xr-var-data-in' type='checkbox'><label for='data-668919fd-f580-415c-9e1e-55a8355edada' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Indicative acoustic absorption</dd><dt><span>units :</span></dt><dd>dB/m</dd><dt><span>valid_min :</span></dt><dd>0.0</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 456 B </td>\n",
+       "                        <td> 456 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3, 19) </td>\n",
+       "                        <td> (3, 19) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"90\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"40\" x2=\"120\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,40.95137597507779 0.0,40.95137597507779\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"60.951376\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >19</text>\n",
+       "  <text x=\"140.000000\" y=\"20.475688\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,20.475688)\">3</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>sound_speed</span></div><div class='xr-var-dims'>(channel, ping_time)</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(3, 19), meta=np.ndarray&gt;</div><input id='attrs-3d15d804-bf56-44c5-a351-d6befb4d5cf8' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-3d15d804-bf56-44c5-a351-d6befb4d5cf8' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-e3bf967f-f028-4a7c-bb97-9962116bc336' class='xr-var-data-in' type='checkbox'><label for='data-e3bf967f-f028-4a7c-bb97-9962116bc336' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Indicative sound speed</dd><dt><span>standard_name :</span></dt><dd>speed_of_sound_in_sea_water</dd><dt><span>units :</span></dt><dd>m/s</dd><dt><span>valid_min :</span></dt><dd>0.0</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 456 B </td>\n",
+       "                        <td> 456 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (3, 19) </td>\n",
+       "                        <td> (3, 19) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> float64 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"90\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"40\" x2=\"120\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"40\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,40.95137597507779 0.0,40.95137597507779\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"60.951376\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >19</text>\n",
+       "  <text x=\"140.000000\" y=\"20.475688\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,20.475688)\">3</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>source_filenames</span></div><div class='xr-var-dims'>(filenames)</div><div class='xr-var-dtype'>&lt;U126</div><div class='xr-var-preview xr-preview'>dask.array&lt;chunksize=(1,), meta=np.ndarray&gt;</div><input id='attrs-70db7fae-e4fb-41cf-96c8-5f0b5be6bbf9' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-70db7fae-e4fb-41cf-96c8-5f0b5be6bbf9' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-066713c9-6ff0-4670-b53f-2ce659eeaa70' class='xr-var-data-in' type='checkbox'><label for='data-066713c9-6ff0-4670-b53f-2ce659eeaa70' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Source filenames</dd></dl></div><div class='xr-var-data'><table>\n",
+       "    <tr>\n",
+       "        <td>\n",
+       "            <table style=\"border-collapse: collapse;\">\n",
+       "                <thead>\n",
+       "                    <tr>\n",
+       "                        <td> </td>\n",
+       "                        <th> Array </th>\n",
+       "                        <th> Chunk </th>\n",
+       "                    </tr>\n",
+       "                </thead>\n",
+       "                <tbody>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Bytes </th>\n",
+       "                        <td> 504 B </td>\n",
+       "                        <td> 504 B </td>\n",
+       "                    </tr>\n",
+       "                    \n",
+       "                    <tr>\n",
+       "                        <th> Shape </th>\n",
+       "                        <td> (1,) </td>\n",
+       "                        <td> (1,) </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Dask graph </th>\n",
+       "                        <td colspan=\"2\"> 1 chunks in 2 graph layers </td>\n",
+       "                    </tr>\n",
+       "                    <tr>\n",
+       "                        <th> Data type </th>\n",
+       "                        <td colspan=\"2\"> <U126 numpy.ndarray </td>\n",
+       "                    </tr>\n",
+       "                </tbody>\n",
+       "            </table>\n",
+       "        </td>\n",
+       "        <td>\n",
+       "        <svg width=\"170\" height=\"170\" style=\"stroke:rgb(0,0,0);stroke-width:1\" >\n",
+       "\n",
+       "  <!-- Horizontal lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"120\" y2=\"0\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"0\" y1=\"120\" x2=\"120\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Vertical lines -->\n",
+       "  <line x1=\"0\" y1=\"0\" x2=\"0\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "  <line x1=\"120\" y1=\"0\" x2=\"120\" y2=\"120\" style=\"stroke-width:2\" />\n",
+       "\n",
+       "  <!-- Colored Rectangle -->\n",
+       "  <polygon points=\"0.0,0.0 120.0,0.0 120.0,120.0 0.0,120.0\" style=\"fill:#ECB172A0;stroke-width:0\"/>\n",
+       "\n",
+       "  <!-- Text -->\n",
+       "  <text x=\"60.000000\" y=\"140.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" >1</text>\n",
+       "  <text x=\"140.000000\" y=\"60.000000\" font-size=\"1.0rem\" font-weight=\"100\" text-anchor=\"middle\" transform=\"rotate(0,140.000000,60.000000)\">1</text>\n",
+       "</svg>\n",
+       "        </td>\n",
+       "    </tr>\n",
+       "</table></div></li><li class='xr-var-item'><div class='xr-var-name'><span>water_level</span></div><div class='xr-var-dims'>()</div><div class='xr-var-dtype'>float64</div><div class='xr-var-preview xr-preview'>...</div><input id='attrs-0fd443b4-34f8-494c-99f4-ea5ceec85cd0' class='xr-var-attrs-in' type='checkbox' ><label for='attrs-0fd443b4-34f8-494c-99f4-ea5ceec85cd0' title='Show/Hide attributes'><svg class='icon xr-icon-file-text2'><use xlink:href='#icon-file-text2'></use></svg></label><input id='data-004036a4-00e0-46f5-9b26-b5992245f099' class='xr-var-data-in' type='checkbox'><label for='data-004036a4-00e0-46f5-9b26-b5992245f099' title='Show/Hide data repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-var-attrs'><dl class='xr-attrs'><dt><span>long_name :</span></dt><dd>Distance from the platform coordinate system origin to the nominal water level along the z-axis</dd><dt><span>units :</span></dt><dd>m</dd></dl></div><div class='xr-var-data'><pre>[1 values with dtype=float64]</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-1a0b178b-35f0-4144-bdba-e16ba9b0e4d3' class='xr-section-summary-in' type='checkbox'  ><label for='section-1a0b178b-35f0-4144-bdba-e16ba9b0e4d3' class='xr-section-summary' >Indexes: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><ul class='xr-var-list'><li class='xr-var-item'><div class='xr-index-name'><div>channel</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-56842546-d6c7-4e7d-bdb2-72f75ee59194' class='xr-index-data-in' type='checkbox'/><label for='index-56842546-d6c7-4e7d-bdb2-72f75ee59194' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Index([&#x27;GPT  18 kHz 009072058c8d 1-1 ES18-11&#x27;,\n",
+       "       &#x27;GPT  38 kHz 009072058146 2-1 ES38B&#x27;,\n",
+       "       &#x27;GPT 120 kHz 00907205a6d0 4-1 ES120-7C&#x27;],\n",
+       "      dtype=&#x27;object&#x27;, name=&#x27;channel&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>filenames</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-4ff1b268-a95a-4853-bb76-8f7463659a49' class='xr-index-data-in' type='checkbox'/><label for='index-4ff1b268-a95a-4853-bb76-8f7463659a49' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Int64Index([0], dtype=&#x27;int64&#x27;, name=&#x27;filenames&#x27;))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>ping_time</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-54867842-cad7-4f8f-9305-be94c56d0a29' class='xr-index-data-in' type='checkbox'/><label for='index-54867842-cad7-4f8f-9305-be94c56d0a29' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(DatetimeIndex([&#x27;2017-06-15 19:02:14.206856192&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:16.784002048&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:19.356149760&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:21.927298048&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:24.495442944&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:27.073590784&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:29.644739072&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:32.213883904&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:34.794033152&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:37.374179840&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:39.951328256&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:42.524474880&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:45.094622208&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:47.672770048&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:50.241916928&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:52.814064128&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:55.394345984&#x27;,\n",
+       "               &#x27;2017-06-15 19:02:57.974493184&#x27;,\n",
+       "               &#x27;2017-06-15 19:03:00.543640064&#x27;],\n",
+       "              dtype=&#x27;datetime64[ns]&#x27;, name=&#x27;ping_time&#x27;, freq=None))</pre></div></li><li class='xr-var-item'><div class='xr-index-name'><div>range_sample</div></div><div class='xr-index-preview'>PandasIndex</div><div></div><input id='index-ffe7aac0-24fb-41e8-b7ad-02e64d5a7731' class='xr-index-data-in' type='checkbox'/><label for='index-ffe7aac0-24fb-41e8-b7ad-02e64d5a7731' title='Show/Hide index repr'><svg class='icon xr-icon-database'><use xlink:href='#icon-database'></use></svg></label><div class='xr-index-data'><pre>PandasIndex(Int64Index([   0,    1,    2,    3,    4,    5,    6,    7,    8,    9,\n",
+       "            ...\n",
+       "            3878, 3879, 3880, 3881, 3882, 3883, 3884, 3885, 3886, 3887],\n",
+       "           dtype=&#x27;int64&#x27;, name=&#x27;range_sample&#x27;, length=3888))</pre></div></li></ul></div></li><li class='xr-section-item'><input id='section-9e475aaa-6887-4b11-9dd6-7a0f6ec1c72c' class='xr-section-summary-in' type='checkbox'  checked><label for='section-9e475aaa-6887-4b11-9dd6-7a0f6ec1c72c' class='xr-section-summary' >Attributes: <span>(4)</span></label><div class='xr-section-inline-details'></div><div class='xr-section-details'><dl class='xr-attrs'><dt><span>processing_function :</span></dt><dd>calibrate.compute_TS</dd><dt><span>processing_software_name :</span></dt><dd>echopype</dd><dt><span>processing_software_version :</span></dt><dd>0.7.2.dev51+gb45c942</dd><dt><span>processing_time :</span></dt><dd>2023-08-29T19:38:55Z</dd></dl></div></li></ul></div></div>"
+      ],
+      "text/plain": [
+       "<xarray.Dataset>\n",
+       "Dimensions:                        (channel: 3, ping_time: 19,\n",
+       "                                    range_sample: 3888, filenames: 1)\n",
+       "Coordinates:\n",
+       "  * channel                        (channel) <U37 'GPT  18 kHz 009072058c8d 1...\n",
+       "  * filenames                      (filenames) int64 0\n",
+       "  * ping_time                      (ping_time) datetime64[ns] 2017-06-15T19:0...\n",
+       "  * range_sample                   (range_sample) int64 0 1 2 ... 3885 3886 3887\n",
+       "Data variables: (12/16)\n",
+       "    TS                             (channel, ping_time, range_sample) float64 dask.array<chunksize=(2, 10, 1944), meta=np.ndarray>\n",
+       "    angle_offset_alongship         (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "    angle_offset_athwartship       (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "    angle_sensitivity_alongship    (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "    angle_sensitivity_athwartship  (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "    beamwidth_alongship            (channel) float64 dask.array<chunksize=(3,), meta=np.ndarray>\n",
+       "    ...                             ...\n",
+       "    gain_correction                (ping_time, channel) float64 dask.array<chunksize=(19, 3), meta=np.ndarray>\n",
+       "    sa_correction                  (ping_time, channel) float64 dask.array<chunksize=(19, 3), meta=np.ndarray>\n",
+       "    sound_absorption               (channel, ping_time) float64 dask.array<chunksize=(3, 19), meta=np.ndarray>\n",
+       "    sound_speed                    (channel, ping_time) float64 dask.array<chunksize=(3, 19), meta=np.ndarray>\n",
+       "    source_filenames               (filenames) <U126 dask.array<chunksize=(1,), meta=np.ndarray>\n",
+       "    water_level                    float64 ...\n",
+       "Attributes:\n",
+       "    processing_function:          calibrate.compute_TS\n",
+       "    processing_software_name:     echopype\n",
+       "    processing_software_version:  0.7.2.dev51+gb45c942\n",
+       "    processing_time:              2023-08-29T19:38:55Z"
+      ]
+     },
+     "execution_count": 10,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "data[0][0]"
+   ]
+  },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "**Congratulations!** You've successfully processed EK60 data using echoflow. This notebook provides a simplified overview, and you can explore the capabilities of echoflow for more advanced processing tasks.\n",
+    "\n",
+    "Feel free to modify the parameters, paths, and configurations as needed to adapt to your data and requirements."
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.9.16"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/echoflow/tests/offline/datastore.yaml b/echoflow/tests/offline/datastore.yaml
new file mode 100644
index 0000000..e20c050
--- /dev/null
+++ b/echoflow/tests/offline/datastore.yaml
@@ -0,0 +1,19 @@
+name: Bell_M._Shimada-SH1707-EK60
+sonar_model: EK60 
+raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6}) 
+args: 
+  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw
+  parameters:
+    ship_name: Bell_M._Shimada
+    survey_name: SH1707
+    sonar_model: EK60
+  storage_options:
+    anon: true
+  transect:
+    file: ./EK60_SH1707_Shimada.txt
+    default_transect_num: 2017
+  json_export: true 
+output: 
+  urlpath: ./echoflow-output
+  retention: false
+  overwrite: true
diff --git a/echoflow/tests/offline/pipeline.yaml b/echoflow/tests/offline/pipeline.yaml
new file mode 100644
index 0000000..cea24cc
--- /dev/null
+++ b/echoflow/tests/offline/pipeline.yaml
@@ -0,0 +1,16 @@
+active_recipe: target_strength 
+use_local_dask: true
+n_workers: 5
+pipeline:
+- recipe_name: target_strength
+  stages:
+  - name: echoflow_open_raw
+    module: echoflow.stages.subflows.open_raw
+    options:
+      save_raw_file: true
+      use_raw_offline: true
+      use_offline: true
+  - name: echoflow_compute_TS
+    module: echoflow.stages.subflows.compute_TS
+    options:
+      use_offline: true
\ No newline at end of file
diff --git a/echoflow/utils.py b/echoflow/utils.py
deleted file mode 100644
index e3945d0..0000000
--- a/echoflow/utils.py
+++ /dev/null
@@ -1,23 +0,0 @@
-import re
-from typing import Set
-
-PARAM_PATTERN = r"({{[^\n\r/]+}})+"
-
-
-def get_glob_parameters(glob_str: str) -> Set[str]:
-    """
-    Extract the variables from glob pattern string based on jinja2
-    templating
-
-    Parameters
-    ----------
-    glob_str : str
-        Glob string to be extracted parameters from
-        {{ test }} results in {'test'}
-    Returns
-    -------
-    Set
-        Set of parameters string
-    """
-    param_values = re.findall(PARAM_PATTERN, glob_str)
-    return {pstr.strip("{{").strip("}}").strip(" ") for pstr in param_values}
diff --git a/echoflow/utils/__init__.py b/echoflow/utils/__init__.py
new file mode 100644
index 0000000..e69de29
diff --git a/echoflow/utils/config_utils.py b/echoflow/utils/config_utils.py
new file mode 100644
index 0000000..ed253b7
--- /dev/null
+++ b/echoflow/utils/config_utils.py
@@ -0,0 +1,542 @@
+"""
+This module contains utility functions for various tasks, such as configuration extraction, file processing, and more.
+
+Functions:
+    extract_config(
+        config: Union[Dict[str, Any], str], storage_options: Dict[str, Any] = {}
+    ) -> Dict[str, Any]
+    check_config(dataset_config: Dict[str, Any], pipeline_config: Dict[str, Any])
+    glob_url(path: str, storage_options: Dict[str, Any]={}) -> List[str]
+    extract_transect_files(
+        file_format: Literal["txt", "zip"],
+        file_path: str,
+        storage_options: Dict[str, Any] = {},
+    ) -> Dict[str, Dict[str, Any]]
+    parse_file_path(raw_file: str, fname_pattern: str) -> Dict[str, Any]
+    get_prefect_config_dict(stage: Stage, pipeline: Recipe, prefect_config_dict: Dict[str, Any])
+    glob_all_files(config: Dataset) -> List[str]
+    parse_raw_paths(all_raw_files: List[str], config: Dataset) -> List[Dict[Any, Any]]
+    club_raw_files(
+        config: Dataset,
+        raw_dicts: List[Dict[str, Any]] = [],
+        raw_url_file: Optional[str] = None,
+        json_storage_options: StorageOptions = None
+    ) -> List[List[Dict[str, Any]]]
+    get_storage_options(storage_options: Block = None) -> Dict[str, Any]
+    load_block(name: str = None, type: StorageType =  None)
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import asyncio
+import itertools as it
+import json
+import os
+import re
+from typing import Any, Coroutine, Dict, List, Literal, Optional, Union
+from zipfile import ZipFile
+from echoflow.models.echoflow_config import EchoflowConfig
+
+import nest_asyncio
+import yaml
+from dateutil import parser
+from prefect import task
+from prefect.filesystems import *
+from prefect.task_runners import *
+from prefect_aws import AwsCredentials
+from prefect_azure import AzureCosmosDbCredentials
+
+from echoflow.aspects.echoflow_aspect import echoflow
+from echoflow.models.datastore import Dataset, StorageOptions, StorageType
+from echoflow.models.pipeline import Recipe, Stage
+from echoflow.utils.file_utils import extract_fs, make_temp_folder
+
+TRANSECT_FILE_REGEX = r"x(?P<transect_num>\d+)"
+nest_asyncio.apply()
+
+
+@task
+def extract_config(
+    config: Union[Dict[str, Any], str], storage_options: Dict[str, Any] = {}
+) -> Dict[str, Any]:
+    """
+    Extracts configuration from YAML file.
+
+    Args:
+        config (Union[Dict[str, Any], str]): Configuration data or path to YAML file.
+        storage_options (Dict[str, Any], optional): Storage options for file access. Defaults to {}.
+
+    Returns:
+        Dict[str, Any]: Extracted configuration.
+
+    Example:
+        config_dict = extract_config("config.yaml")
+    """
+    file_system = extract_fs(config, storage_options)
+    with file_system.open(config, "rb") as yaml_file:
+        return yaml.safe_load(yaml_file.read())
+
+
+@task
+def check_config(dataset_config: Dict[str, Any], pipeline_config: Dict[str, Any]):
+    """
+    Checks if the provided pipeline configuration is valid.
+
+    Args:
+        dataset_config (Dict[str, Any]): Dataset configuration.
+        pipeline_config (Dict[str, Any]): Pipeline configuration.
+
+    Raises:
+        ValueError: If active recipe name is missing in pipeline configuration,
+                    or if active recipe name is not found in the recipe list.
+
+    Example:
+        dataset_config = {...}
+        pipeline_config = {...}
+        check_config(dataset_config, pipeline_config)
+    """
+    if not "active_recipe" in pipeline_config:
+        raise ValueError(
+            "Pipeline Configuration must have active recipe name!")
+    else:
+        recipe_name = pipeline_config["active_recipe"]
+        recipe_list = pipeline_config["pipeline"]
+        recipe_names = []
+        for recipe in recipe_list:
+            recipe_names.append(recipe["recipe_name"])
+        if not recipe_name in recipe_names:
+            raise ValueError(
+                "Active recipe name not found in the recipe list!")
+
+
+def glob_url(path: str, storage_options: Dict[str, Any] = {}) -> List[str]:
+    """
+    Glob files based on the given path string using fsspec.filesystem.glob.
+
+    Args:
+        path (str): The URL path string glob pattern.
+        storage_options (Dict[str, Any], optional): Extra arguments for the filesystem class.
+                                                     Defaults to {}.
+
+    Returns:
+        List[str]: List of paths from the glob.
+
+    Example:
+        files = glob_url("s3://my-bucket/data/*.txt", {"anon": True})
+    """
+
+    file_system, scheme = extract_fs(
+        path, storage_options, include_scheme=True)
+    all_files = [f if f.startswith(
+        scheme) else f"{scheme}://{f}" for f in file_system.glob(path)]
+    return all_files
+
+
+def extract_transect_files(
+    file_format: Literal["txt", "zip"],
+    file_path: str,
+    storage_options: Dict[str, Any] = {},
+    default_transect: int = 0
+) -> Dict[str, Dict[str, Any]]:
+    """
+    Extracts raw file names and transect numbers from transect file(s).
+
+    Args:
+        file_format (Literal["txt", "zip"]): Transect file format (txt or zip).
+        file_path (str): The full path to the transect file.
+        storage_options (dict): Storage options for transect file access.
+        default_transect: int = Default transect in case of no transect specified. Defaults to 0
+
+    Returns:
+        Dict[str, Dict[str, Any]]: Raw file name dictionary with transect information.
+
+    Example:
+        file_info = extract_transect_files("zip", "s3://my-bucket/transects.zip", {"anon": True})
+    """
+    file_system = extract_fs(file_path, storage_options=storage_options)
+
+    if file_format == "zip":
+        return _extract_from_zip(file_system, file_path)
+    elif file_format == "txt":
+        return _extract_from_text(file_system, file_path, default_transect)
+    else:
+        raise ValueError(
+            f"Invalid file format: {file_format}. Only 'txt' or 'zip' are valid")
+
+
+def _extract_from_zip(file_system, file_path: str) -> Dict[str, Dict[str, Any]]:
+    """
+    Extracts raw files from transect file zip format.
+
+    Parameters:
+        file_system: The filesystem where the zip file is located.
+        file_path (str): The path to the transect zip file.
+
+    Returns:
+        Dict[str, Dict[str, Any]]: A dictionary containing extracted raw file information.
+
+    Raises:
+        ValueError: If a directory is found in the zip file.
+
+    Example:
+        file_system = ...
+        file_path = 'path/to/transect_file.zip'
+        extracted_data = _extract_from_zip(file_system, file_path)
+    """
+    transect_dict = {}
+    with file_system.open(file_path, "rb") as f:
+        with ZipFile(f) as zf:
+            zip_infos = zf.infolist()
+            for zi in zip_infos:
+                m = re.match(TRANSECT_FILE_REGEX, zi.filename)
+                transect_num = int(m.groupdict()["transect_num"])
+                if zi.is_dir():
+                    raise ValueError(
+                        "Directory found in zip file. This is not allowed!")
+                with zf.open(zi.filename) as txtfile:
+                    file_list = txtfile.read().decode("utf-8").split("\r\n")
+                    for rawfile in file_list:
+                        transect_dict.setdefault(
+                            rawfile,
+                            {"filename": zi.filename, "num": transect_num},
+                        )
+    return transect_dict
+
+
+def _extract_from_text(file_system, file_path: str, default_transect: int = 0) -> Dict[str, Dict[str, Any]]:
+    """
+    Extracts raw files from transect file text format.
+
+    Parameters:
+        file_system: The filesystem where the text file is located.
+        file_path (str): The path to the transect text file.
+        default_transect: int = Default transect in case of no transect specified. Defaults to 0
+
+
+    Returns:
+        Dict[str, Dict[str, Any]]: A dictionary containing extracted raw file information.
+
+    Example:
+        file_system = ...
+        file_path = 'path/to/transect_file.txt'
+        extracted_data = _extract_from_text(file_system, file_path)
+    """
+    filename = os.path.basename(file_path)
+    m = re.match(TRANSECT_FILE_REGEX, filename)
+    if m is None:
+        transect_num = default_transect
+    else:
+        transect_num = int(m.groupdict()["transect_num"])
+
+    transect_dict = {}
+
+    with file_system.open(file_path) as txtfile:
+        file_list = txtfile.read().decode("utf-8").splitlines()
+        for rawfile in file_list:
+            transect_dict.setdefault(
+                rawfile,
+                {"filename": filename, "num": transect_num},
+            )
+
+    return transect_dict
+
+
+def parse_file_path(raw_file: str, fname_pattern: str) -> Dict[str, Any]:
+    """
+    Parses file path to extract datetime information.
+
+    Parameters:
+        raw_file (str): The raw file URL.
+        fname_pattern (str): The regex pattern for date extraction.
+
+    Returns:
+        Dict[str, Any]: A dictionary containing parsed date and time information.
+
+    Example:
+        raw_file = 'https://example.com/file_2023_0815.txt'
+        fname_pattern = r'file_(?P<year>\d{4})_(?P<month>\d{2})(?P<day>\d{2})'
+        parsed_data = parse_file_path(raw_file, fname_pattern)
+    """
+    matcher = re.compile(fname_pattern)
+    file_match = matcher.search(raw_file)
+    match_dict = file_match.groupdict()
+    file_datetime = None
+    if "date" in match_dict and "time" in match_dict:
+        datetime_obj = parser.parse(f"{file_match['date']}{file_match['time']}")  # noqa
+        file_datetime = datetime_obj.isoformat()
+        jday = datetime_obj.timetuple().tm_yday
+        match_dict.pop("date")
+        match_dict.pop("time")
+        match_dict.setdefault("month", datetime_obj.month)
+        match_dict.setdefault("year", datetime_obj.year)
+        match_dict.setdefault("jday", jday)
+
+    match_dict.setdefault("datetime", file_datetime)
+    return dict(**match_dict)
+
+
+@task
+def get_prefect_config_dict(stage: Stage, pipeline: Recipe, prefect_config_dict: Dict[str, Any]):
+    """
+    Gets the updated Prefect configuration dictionary.
+
+    Parameters:
+        stage (Stage): The Prefect stage.
+        pipeline (Recipe): The Prefect pipeline.
+        prefect_config_dict (Dict[str, Any]): The Prefect configuration dictionary.
+
+    Returns:
+        Dict[str, Any]: The updated Prefect configuration dictionary.
+
+    Example:
+        stage = ...
+        pipeline = ...
+        prefect_config_dict = {...}
+        updated_config = get_prefect_config_dict(stage, pipeline, prefect_config_dict)
+    """
+    prefect_config: Dict[str, Any] = stage.prefect_config
+    updated_config = {}
+    if (stage.prefect_config is not None):
+        for key, value in prefect_config.items():
+            if type(value) == str and '(' in value and ')' in value:
+                class_name, params_str = value.split('(', 1)
+                params_str = params_str.rstrip(')')
+                parameters = {}
+                if params_str is not None and params_str != '':
+                    for param in params_str.split(','):
+                        param_key, param_value = param.split('=')
+                        parameters[param_key.strip()] = param_value.strip()
+                class_object = globals()[class_name](**parameters)
+                updated_config[key] = class_object
+            else:
+                updated_config[key] = value
+    return updated_config
+
+
+@task
+@echoflow(processing_stage="Configuration", type="CONFIG_TASK")
+def glob_all_files(config: Dataset) -> List[str]:
+    """
+    Fetches individual file URLs from a source path in the Dataset configuration.
+
+    Parameters:
+        config (Dataset): The Dataset configuration.
+
+    Returns:
+        List[str]: A list of raw URL paths.
+
+    Example:
+        dataset_config = ...
+        raw_urls = glob_all_files(dataset_config)
+    """
+    total_files = []
+    data_path = config.args.rendered_path
+    storage_options = config.args.storage_options_dict
+    if data_path is not None:
+        if isinstance(data_path, list):
+            for path in data_path:
+                all_files = glob_url(path, dict(storage_options))
+                total_files.append(all_files)
+            total_files = list(it.chain.from_iterable(total_files))
+        else:
+            total_files = glob_url(data_path, dict(storage_options))
+
+    return total_files
+
+
+@task
+@echoflow(processing_stage="Configuration", type="CONFIG_TASK")
+def parse_raw_paths(all_raw_files: List[str], config: Dataset) -> List[Dict[Any, Any]]:
+    """
+    Parses raw URL paths, extracts information, and creates a file dictionary.
+
+    Parameters:
+        all_raw_files (List[str]): List of raw URL paths.
+        config (Dataset): The Pipeline configuration.
+
+    Returns:
+        List[Dict[Any, Any]]: List of dictionaries containing parsed raw URL information.
+
+    Example:
+        all_raw_files = ['https://example.com/file1.txt', 'https://example.com/file2.txt']
+        dataset_config = ...
+        parsed_data = parse_raw_paths(all_raw_files, dataset_config)
+    """
+    sonar_model = config.sonar_model
+    fname_pattern = config.raw_regex
+    transect_dict = {}
+
+    if config.args.transect is not None and config.args.transect.file is not None:
+        # When transect info is available, extract it
+        file_input = config.args.transect.file
+        storage_options = config.args.transect.storage_options_dict
+        default_transect = config.args.default_transect_num
+        if isinstance(file_input, str):
+            filename = os.path.basename(file_input)
+            _, ext = os.path.splitext(filename)
+            transect_dict = extract_transect_files(
+                ext.strip("."), file_input, storage_options, default_transect)
+        else:
+            transect_dict = {}
+            for f in file_input:
+                filename = os.path.basename(f)
+                _, ext = os.path.splitext(filename)
+                result = extract_transect_files(
+                    ext.strip("."), f, storage_options, default_transect)
+                transect_dict.update(result)
+
+    raw_file_dicts = []
+    for raw_file in all_raw_files:
+        # get transect info from the transect_dict above
+        transect = transect_dict.get(os.path.basename(raw_file), {})
+        transect_num = transect.get(
+        "num", config.args.default_transect_num)
+        if (config.args.transect is None) or (
+            config.args.transect is not None and transect_num is not None and bool(transect)
+        ):
+            # Only adds to the list if not transect
+            # if it's a transect, ensure it has a transect number
+            raw_file_dicts.append(
+                dict(
+                    instrument=sonar_model,
+                    file_path=raw_file,
+                    transect_num=transect_num,
+                    **parse_file_path(raw_file, fname_pattern),
+                )
+            )
+    return raw_file_dicts
+
+
+@task
+@echoflow(processing_stage="Configuration", type="CONFIG_TASK")
+def club_raw_files(
+    config: Dataset,
+    raw_dicts: List[Dict[str, Any]] = [],
+    raw_url_file: Optional[str] = None,
+    json_storage_options: StorageOptions = None
+) -> List[List[Dict[str, Any]]]:
+    """
+    Parses raw URLs, splits them into weekly lists using Julian days.
+
+    Parameters:
+        config (Dataset): The Pipeline configuration.
+        raw_dicts (List[Dict[str, Any]]): List of raw URL dictionaries.
+        raw_url_file (Optional[str]): Path to raw URLs JSON file.
+        json_storage_options (StorageOptions): Storage options for reading raw URLs.
+
+    Returns:
+        List[List[Dict[str, Any]]]: List of lists of raw URL dictionaries grouped by week.
+
+    Example:
+        dataset_config = ...
+        raw_dicts = [...]
+        raw_url_file = 'path/to/raw_urls.json'
+        json_storage_options = ...
+        grouped_raw_data = club_raw_files(dataset_config, raw_dicts, raw_url_file, json_storage_options)
+    """
+
+    if len(raw_dicts) == 0:
+        if raw_url_file is None:
+            raise ValueError("Must have raw_dicts or raw_json_path present.")
+        file_system = extract_fs(
+            raw_url_file, storage_options=json_storage_options
+        )
+        with file_system.open(raw_url_file) as f:
+            raw_dicts = json.load(f)
+
+    if config.args.transect is not None:
+        # Transect, split by transect spec
+        raw_dct = {}
+        for r in raw_dicts:
+            transect_num = r['transect_num']
+            if transect_num not in raw_dct:
+                raw_dct[transect_num] = []
+            raw_dct[transect_num].append(r)
+
+        all_files = [
+            sorted(raw_list, key=lambda a: a['datetime'])
+            for raw_list in raw_dct.values()
+        ]
+    else:
+        # Number of days for a week chunk
+        n = 7
+
+        all_jdays = sorted({r.get("jday") for r in raw_dicts})
+        split_days = [
+            all_jdays[i: i + n] for i in range(0, len(all_jdays), n)
+        ]  # noqa
+
+        day_dict = {}
+        for r in raw_dicts:
+            mint = r.get("jday")
+            if mint not in day_dict:
+                day_dict[mint] = []
+            day_dict[mint].append(r)
+
+        all_files = []
+        for week in split_days:
+            files = list(it.chain.from_iterable([day_dict[d] for d in week]))
+            all_files.append(files)
+    return all_files
+
+
+def get_storage_options(storage_options: Block = None) -> Dict[str, Any]:
+    """
+    Get storage options from a Block.
+
+    Parameters:
+        storage_options (Block, optional): A block containing storage options.
+
+    Returns:
+        Dict[str, Any]: Dictionary containing storage options.
+
+    Example:
+        aws_credentials = AwsCredentials(...)
+        storage_opts = get_storage_options(aws_credentials)
+    """
+    storage_options_dict: Dict[str, Any] = {}
+    if storage_options is not None:
+        if isinstance(storage_options, AwsCredentials):
+            storage_options_dict["key"] = storage_options.aws_access_key_id
+            storage_options_dict["secret"] = storage_options.aws_secret_access_key.get_secret_value(
+            )
+            if storage_options.aws_session_token:
+                storage_options_dict["token"] = storage_options.aws_session_token
+
+    return storage_options_dict
+
+
+def load_block(name: str = None, type: StorageType = None):
+    """
+    Load a block of a specific type by name.
+
+    Parameters:
+        name (str, optional): The name of the block to load.
+        type (StorageType, optional): The type of the block to load.
+
+    Returns:
+        block: The loaded block.
+
+    Raises:
+        ValueError: If name or type is not provided.
+
+    Example:
+        loaded_aws_credentials = load_block(name="my-aws-creds", type=StorageType.AWS)
+    """
+    if name is None or type is None:
+        raise ValueError("Cannot load block without name")
+
+    if type == StorageType.AWS or type == StorageType.AWS.value:
+        coro = AwsCredentials.load(name=name)
+    elif type == StorageType.AZCosmos or type == StorageType.AZCosmos.value:
+        coro = AzureCosmosDbCredentials.load(name=name)
+    elif type == StorageType.ECHOFLOW or type == StorageType.ECHOFLOW.value:
+        coro = EchoflowConfig.load(name=name)
+
+    if isinstance(coro, Coroutine):
+        block = nest_asyncio.asyncio.run(coro)
+    else:
+        block = coro
+    return block
+    
\ No newline at end of file
diff --git a/echoflow/utils/databse_utils.py b/echoflow/utils/databse_utils.py
new file mode 100644
index 0000000..cb1c03d
--- /dev/null
+++ b/echoflow/utils/databse_utils.py
@@ -0,0 +1,361 @@
+"""
+This module contains utility functions for working with SQLite databases and managing log data.
+
+The functions in this module provide an interface for creating, updating, and retrieving log data
+in an SQLite database. The module also includes functions to manage database connections and perform
+SQL queries.
+
+Functions:
+    get_connection(path: str='') -> sqlite3.Connection:
+        Gets a SQLite database connection.
+
+    execute_sql(conn: sqlite3.Connection, query: str) -> Any:
+        Executes an SQL query on the provided connection.
+
+    create_table(conn: sqlite3.Connection, table_name: str, columns: List[str]):
+        Creates a table in the database.
+
+    create_log_table(conn: sqlite3.Connection):
+        Creates a 'log' table in the database.
+
+    insert_log_data_by_conn(conn: sqlite3.Connection, log: DB_Log) -> int:
+        Inserts log data into the 'log' table using the provided connection.
+
+    insert_log_data_by_path(path: str, log: DB_Log) -> int:
+        Inserts log data into the 'log' table using the database path.
+
+    convert_to_serializable_dict(obj: Any) -> Union[BaseModel, dict, List[dict], Any]:
+        Converts objects to a serializable dictionary format.
+
+    update_log_data_by_conn(conn: sqlite3.Connection, log: DB_Log):
+        Updates log data in the 'log' table using the provided connection.
+
+    update_log_data_by_path(path: str, log: DB_Log):
+        Updates log data in the 'log' table using the database path.
+
+    parse_all_log_data(conn: sqlite3.Connection) -> List[DB_Log]:
+        Parses and retrieves all log data from the 'log' table.
+
+    get_last_log(conn: sqlite3.Connection) -> DB_Log:
+        Retrieves the most recent log entry from the 'log' table.
+
+    parse_log(row: Tuple) -> DB_Log:
+        Parses a row of log data and creates a DB_Log object.
+
+Classes:
+    DB_Log:
+        Data class representing a log entry.
+
+    Log_Data:
+        Data class representing detailed log information.
+
+Notes:
+    - The module assumes the existence of a database with the 'log' table for storing log data.
+    - The functions provided here can be used to interact with the log database for various applications.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+
+import json
+import os
+import sqlite3
+import threading
+from datetime import datetime
+from pathlib import Path
+
+from pydantic import BaseModel
+
+from echoflow.models.db_log_model import DB_Log, Log_Data
+
+db_connections = threading.local()
+
+
+def get_connection(path: str = '') -> sqlite3.Connection:
+    """
+    Gets a SQLite database connection.
+
+    Args:
+        path (str): The path to the database file.
+
+    Returns:
+        sqlite3.Connection: A connection to the SQLite database.
+
+    Example:
+        db_connection = get_connection('/path/to/db')
+    """
+    if not hasattr(db_connections, 'db_connection'):
+        if os.path.isdir(path) == False:
+            dir = Path(path)
+            dir.mkdir(exist_ok=True, parents=True)
+            os.chmod(dir, 0o777)
+        db_connections.db_connection = sqlite3.connect(path+'/echoflow.db')
+    return db_connections.db_connection
+
+
+def execute_sql(conn, query):
+    """
+    Executes an SQL query on the provided connection.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+        query (str): The SQL query to execute.
+
+    Returns:
+        Any: The result of the query.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        query_result = execute_sql(connection, 'SELECT * FROM table_name')
+    """
+    cursor = conn.cursor()
+    cursor.execute(query)
+    result = cursor.fetchall()
+    conn.commit()
+    cursor.close()
+    return result
+
+
+def create_table(conn, table_name, columns):
+    """
+    Creates a table in the database.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+        table_name (str): The name of the table to create.
+        columns (List[str]): List of column definitions.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        create_table(connection, 'table_name', ['column1 INT', 'column2 TEXT'])
+    """
+    query = f"CREATE TABLE IF NOT EXISTS {table_name} ({', '.join(columns)})"
+    execute_sql(conn, query)
+
+
+def create_log_table(conn):
+    """
+    Creates a 'log' table in the database.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        create_log_table(connection)
+    """
+    create_table_query = '''
+    CREATE TABLE IF NOT EXISTS log (
+        run_id INTEGER PRIMARY KEY AUTOINCREMENT,
+        start_time TEXT,
+        end_time TEXT,
+        data TEXT,
+        status TEXT,
+        error TEXT
+    );
+    '''
+    execute_sql(conn=conn, query=create_table_query)
+
+
+def insert_log_data_by_conn(conn: sqlite3.Connection, log: DB_Log):
+    """
+    Inserts log data into the 'log' table using the provided connection.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+        log (DB_Log): The log data to insert.
+
+    Returns:
+        int: The ID of the inserted log.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        log_data = DB_Log(...)
+        log_id = insert_log_data_by_conn(connection, log_data)
+    """
+
+    insert_query = '''
+    INSERT INTO log (run_id, start_time, end_time, data, status, error)
+    VALUES (?, ?, ?, ?, ?, ?);
+    '''
+    data_json = json.dumps(convert_to_serializable_dict(log.data))
+    res = conn.execute(insert_query, (log.run_id, log.start_time,
+                       log.end_time, data_json, log.status, log.error))
+    id = res.lastrowid
+    conn.commit()
+    return id
+
+
+def insert_log_data_by_path(path: str, log: DB_Log):
+    """
+    Inserts log data into the 'log' table using the database path.
+
+    Args:
+        path (str): The path to the database.
+        log (DB_Log): The log data to insert.
+
+    Returns:
+        int: The ID of the inserted log.
+
+    Example:
+        log_data = DB_Log(...)
+        log_id = insert_log_data_by_path('/path/to/db', log_data)
+    """
+    conn = get_connection(path=path)
+    return insert_log_data_by_conn(conn, log)
+
+
+def convert_to_serializable_dict(obj):
+    """
+    Converts objects to a serializable dictionary format.
+
+    Args:
+        obj (Any): The object to convert.
+
+    Returns:
+        Union[BaseModel, dict, List[dict], Any]: The serialized dictionary.
+
+    Example:
+        serialized_data = convert_to_serializable_dict(some_object)
+    """
+    if isinstance(obj, BaseModel):
+        return obj.dict()
+    elif isinstance(obj, dict):
+        return {str(key): convert_to_serializable_dict(value) for key, value in obj.items()}
+    elif isinstance(obj, list):
+        return [convert_to_serializable_dict(item) for item in obj]
+    else:
+        return obj
+
+
+def update_log_data_by_conn(conn: sqlite3.Connection, log: DB_Log):
+    """
+    Updates log data in the 'log' table using the provided connection.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+        log (DB_Log): The updated log data.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        updated_log_data = DB_Log(...)
+        update_log_data_by_conn(connection, updated_log_data)
+    """
+    update_query = '''
+    UPDATE log
+    SET start_time = ?, end_time = ?, data = ?, status = ?, error =?
+    WHERE run_id = ?;
+    '''
+    data_json = json.dumps(convert_to_serializable_dict(log.data))
+    conn.execute(
+        update_query,
+        (log.start_time, log.end_time, data_json,
+         log.status, log.error, log.run_id),
+    )
+    conn.commit()
+
+
+def update_log_data_by_path(path: str, log: DB_Log):
+    """
+    Updates log data in the 'log' table using the database path.
+
+    Args:
+        path (str): The path to the database.
+        log (DB_Log): The updated log data.
+
+    Example:
+        updated_log_data = DB_Log(...)
+        update_log_data_by_path('/path/to/db', updated_log_data)
+    """
+    conn = get_connection(path=path)
+    return update_log_data_by_conn(conn, log)
+
+
+def parse_all_log_data(conn):
+    """
+    Parses and retrieves all log data from the 'log' table.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+
+    Returns:
+        List[DB_Log]: List of parsed log objects.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        logs = parse_all_log_data(connection)
+    """
+    select_query = '''
+    SELECT run_id, start_time, end_time, data, status, error
+    FROM log;
+    '''
+
+    rows = execute_sql(conn, select_query)
+
+    print(rows)
+
+    logs = []
+    for row in rows:
+        log_obj = parse_log(row)
+        logs.append(log_obj)
+
+    return logs
+
+
+def get_last_log(conn):
+    """
+    Retrieves the most recent log entry from the 'log' table.
+
+    Args:
+        conn (sqlite3.Connection): The database connection.
+
+    Returns:
+        DB_Log: The parsed log object.
+
+    Example:
+        connection = get_connection('/path/to/db')
+        last_log_entry = get_last_log(connection)
+    """
+    select_query = '''
+    SELECT run_id, start_time, end_time, data, status, error
+    FROM log
+    ORDER BY id DESC
+    LIMIT 1;
+    '''
+
+    last_log = DB_Log()
+    row = execute_sql(conn, select_query)
+
+    if row:
+        last_log = parse_log(row)
+
+    return last_log
+
+
+def parse_log(row):
+    """
+    Parses a row of log data and creates a DB_Log object.
+
+    Args:
+        row (Tuple): The row of log data.
+
+    Returns:
+        DB_Log: The parsed log object.
+
+    Example:
+        log_row = (1, '2023-08-15', '2023-08-16', {...}, 'success', None)
+        log_obj = parse_log(log_row)
+    """
+    run_id, start_time, end_time, data_json, status, error = row
+    data_dict = json.loads(data_json)
+
+    log_data_dict = {}
+    for key, value in data_dict.items():
+        log_data_obj = Log_Data(**value)
+        log_data_dict[key] = log_data_obj
+
+    log_obj = DB_Log(run_id=run_id, start_time=start_time,
+                     end_time=end_time, data=log_data_dict, status=status, error=error)
+
+    return log_obj
diff --git a/echoflow/utils/file_utils.py b/echoflow/utils/file_utils.py
new file mode 100644
index 0000000..50fcda0
--- /dev/null
+++ b/echoflow/utils/file_utils.py
@@ -0,0 +1,502 @@
+"""
+This module provides utility functions related to working with files, directories, and data paths.
+
+Module Functions:
+    download_temp_file(raw: Dict[str, Any], working_dir: str, stage: Stage, config: Dataset) -> Dict[str, Any]:
+        Downloads a temporary raw file from a URL path.
+
+    extract_fs(url: str, storage_options: Dict[Any, Any] = {}, include_scheme: bool = False) -> Union[Tuple[Any, str], Any]:
+        Extracts the fsspec file system from a URL path.
+
+    make_temp_folder(folder_name: str, storage_options: Dict[str, Any]) -> str:
+        Creates a temporary folder locally or remotely using fsspec.
+
+    get_output_file_path(raw_dicts: List[Dict[str, Any]], config: Dataset) -> str:
+        Generates the output file path based on the raw file dictionary's datetime key.
+
+    isFile(file_path: str, storage_options: Dict[str, Any] = {}) -> bool:
+        Checks if a file exists at the specified path using the fsspec file system.
+
+    get_working_dir(stage: Stage, config: Dataset) -> str:
+        Retrieves the working directory for a stage based on stage options and configuration.
+
+    get_ed_list(config: Dataset, stage: Stage, transect_data: Union[Output, List[Dict[str, Any]], Dict[str, Any]]) -> List[open_converted]:
+        Retrieves a list of open_converted objects for echopype data.
+
+    get_zarr_list(config: Dataset, stage: Stage, transect_data: Union[Output, Dict[str, Any]]) -> List[xarray.Dataset]:
+        Retrieves a list of xarray.Dataset objects for zarr data.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import json
+import os
+from pathlib import Path
+from typing import Any, Dict, List, Tuple, Union
+from urllib.parse import urlparse
+
+import fsspec
+import xarray as xr
+from dateutil import parser
+from echopype import open_converted
+from fastapi.encoders import jsonable_encoder
+from fsspec.implementations.local import LocalFileSystem
+from prefect import task
+
+from echoflow.models.datastore import Dataset
+from echoflow.models.output_model import Output
+from echoflow.models.pipeline import Stage
+
+
+def download_temp_file(raw, working_dir: str, stage: Stage, config: Dataset):
+    """
+    Downloads a temporary raw file from a URL path.
+
+    Args:
+        raw (Dict[str, Any]): Raw file dictionary containing file information.
+        working_dir (str): Working directory where the file will be downloaded.
+        stage (Stage): Current processing stage object.
+        config (Dataset): Dataset configuration.
+
+    Returns:
+        Dict[str, Any]: Updated raw file dictionary with local file path.
+
+    Example:
+        raw_data = {'file_path': 'https://example.com/data.raw', ...}
+        working_directory = '/path/to/working_dir'
+        stage_object = ...
+        dataset_config = ...
+        updated_raw = download_temp_file(raw_data, working_directory, stage_object, dataset_config)
+    """
+
+    urlpath = raw.get("file_path")
+    fname = os.path.basename(urlpath)
+    out_path = working_dir+"/"+ raw.get("transect_num") +"_raw_files/"+fname
+    working_dir_fs = extract_fs(
+        out_path, storage_options=config.output.storage_options_dict)
+
+    if stage.options.get("use_raw_offline") == False or isFile(out_path, config.output.storage_options_dict) == False:
+        print("Downloading ...", out_path)
+        file_system = extract_fs(
+            urlpath, storage_options=config.args.storage_options_dict)
+        with file_system.open(urlpath, 'rb') as source_file:
+            with working_dir_fs.open(out_path, mode="wb") as f:
+                f.write(source_file.read())
+    raw.update({"local_path": out_path})
+    return raw
+
+
+def extract_fs(
+    url: str,
+    storage_options: Dict[Any, Any] = {},
+    include_scheme: bool = False,
+) -> Union[Tuple[Any, str], Any]:
+    """
+    Extracts the fsspec file system from a URL path.
+
+    Args:
+        url (str): URL path string.
+        storage_options (Dict[Any, Any]): Additional keywords to pass to the filesystem class.
+        include_scheme (bool): Flag to include scheme in the output of the function.
+
+    Returns:
+        Union[Tuple[Any, str], Any]: Filesystem for the given protocol and arguments.
+
+    Example:
+        fs = extract_fs('s3://mybucket/data', storage_options={'anon': True})
+    """
+    parsed_path = urlparse(url)
+    file_system = fsspec.filesystem(parsed_path.scheme, **storage_options)
+    if include_scheme:
+        return file_system, parsed_path.scheme
+    return file_system
+
+
+def make_temp_folder(folder_name: str, storage_options: Dict[str, Any]) -> str:
+    """
+    Creates a temporary folder locally or remotely using fsspec.
+
+    Args:
+        folder_name (str): Name of the folder.
+        storage_options (Dict[str, Any]): Storage options for fsspec.
+
+    Returns:
+        str: Path to the created temporary folder.
+
+    Example:
+        temp_folder = make_temp_folder('temp_folder', storage_options={'anon': True})
+    """
+
+    fsmap = fsspec.get_mapper(folder_name, **storage_options)
+    if fsmap.fs.isdir(fsmap.root) == False:
+        fsmap.fs.mkdir(fsmap.root, exists_ok=True, create_parents=True)
+    if isinstance(fsmap.fs, LocalFileSystem):
+        return str(Path(folder_name).resolve())
+    return folder_name
+
+
+@task
+def get_output_file_path(raw_dicts, config: Dataset):
+    """
+    Get the output file path based on the raw file dictionary datetime key.
+    It will grab the first file and use that as the file name in the form of 'DYYYYmmdd-THHMMSS.zarr'.
+
+    This method will also grab a value from the config.output.urlpath for where the file should go.
+
+    Parameters:
+        raw_dicts (List[Dict[str, Any]]): The list of raw URL dictionaries. Must have 'datetime' key!
+        config (Dataset): The pydantic Dataset object that contains all the necessary configurations.
+
+    Returns:
+        str: The full path string to the zarr file.
+
+    Example:
+        raw_files = [{'datetime': '2023-08-15T12:34:56', ...}, ...]
+        dataset_config = ...
+        zarr_file_path = get_output_file_path(raw_files, dataset_config)
+    """
+    if len(raw_dicts) < 1:
+        raise ValueError("There must be at least one raw file dictionary!")
+
+    first_file = raw_dicts[0]
+    datetime_obj = parser.parse(first_file.get("datetime"))
+    if config.args.transect is None:
+        # Only use datetime
+        out_fname = datetime_obj.strftime("D%Y%m%d-T%H%M%S.zarr")
+    else:
+        # Use transect number
+        transect_num = first_file.get("transect_num", None)
+        date_name = datetime_obj.strftime("D%Y%m%d-T%H%M%S")
+        out_fname = f"x{transect_num:04}-{date_name}.zarr"
+    return "/".join([config.output.urlpath, out_fname])
+
+
+def isFile(file_path: str, storage_options: Dict[str, Any] = {}):
+    """
+    Check if a file exists at the specified path using the fsspec file system.
+
+    Parameters:
+        file_path (str): The path to the file.
+        storage_options (Dict[str, Any]): Storage options for fsspec.
+
+    Returns:
+        bool: True if the file exists, False otherwise.
+
+    Example:
+        exists = isFile('s3://mybucket/data.zarr', storage_options={'anon': True})
+    """
+    fs = extract_fs(file_path, storage_options=storage_options)
+    if file_path.endswith(".zarr"):
+        return fs.isdir(file_path)
+    return fs.isfile(file_path)
+
+
+def get_working_dir(stage: Stage, config: Dataset):
+    """
+    Get the working directory for a stage based on stage options and configuration.
+
+    Parameters:
+        stage (Stage): The current processing stage object.
+        config (Dataset): The Dataset configuration.
+
+    Returns:
+        str: The path to the working directory.
+
+    Example:
+        stage_object = ...
+        dataset_config = ...
+        working_directory = get_working_dir(stage_object, dataset_config)
+    """
+    if stage.options is not None and stage.options.get("out_path") is not None:
+        working_dir = make_temp_folder(stage.options.get(
+            "out_path"), config.output.storage_options_dict)
+    elif config.output.urlpath is not None:
+        working_dir = make_temp_folder(
+            config.output.urlpath+"/"+stage.name, config.output.storage_options_dict)
+    else:
+        working_dir = make_temp_folder(
+            "Echoflow_working_dir"+"/"+stage.name, config.output.storage_options_dict)
+
+    return working_dir
+
+
+@task
+def get_ed_list(config: Dataset, stage: Stage, transect_data: Union[Output, List[Dict], Dict]):
+    """
+    Get a list of open_converted objects for echopype data.
+
+    Parameters:
+        config (Dataset): The Dataset configuration.
+        stage (Stage): The current processing stage object.
+        transect_data (Union[Output, List[Dict[str, Any]], Dict[str, Any]]): The transect data or output.
+
+    Returns:
+        List[open_converted]: A list of open_converted objects.
+
+    Example:
+        dataset_config = ...
+        stage_object = ...
+        transect_output = ...
+        echopype_list = get_ed_list(dataset_config, stage_object, transect_output)
+    """
+    ed_list = []
+    if type(transect_data) == list:
+        for zarr_path_data in transect_data:
+            ed = open_converted(
+                converted_raw_path=str(zarr_path_data.get("out_path")),
+                storage_options=dict(config.output.storage_options_dict),
+            )
+            ed_list.append(ed)
+    elif type(transect_data) == dict:
+        ed = open_converted(
+            converted_raw_path=str(transect_data.get("out_path")),
+            storage_options=dict(config.output.storage_options_dict),
+        )
+        ed_list.append(ed)
+    else:
+        zarr_path_data = transect_data.data
+        ed = open_converted(
+            converted_raw_path=str(zarr_path_data.get("out_path")),
+            storage_options=dict(config.output.storage_options_dict),
+        )
+        ed_list.append(ed)
+    return ed_list
+
+
+@task
+def get_zarr_list(transect_data: Union[Output, Dict], storage_options: Dict[str, Any] = {}):
+    """
+    Get a list of xarray.Dataset objects for zarr data.
+
+    Parameters:
+        config (Dataset): Output storage options
+        transect_data (Union[Output, Dict[str, Any]]): The transect data or output.
+
+    Returns:
+        List[xarray.Dataset]: A list of xarray.Dataset objects.
+
+    Example:
+        storage_options = ...
+        transect_output = ...
+        zarr_list = get_zarr_list(transect_output, storage_options)
+    """
+    zarr_list = []
+    if type(transect_data) == dict:
+        zarr = xr.open_zarr(transect_data.get("out_path"),
+                            storage_options=storage_options)
+        zarr_list.append(zarr)
+    else:
+        zarr_path_data = transect_data.data
+        zarr = xr.open_zarr(zarr_path_data.get("out_path"),
+                            storage_options=storage_options)
+        zarr_list.append(zarr)
+
+    return zarr_list
+
+def process_output_transects(name: str, config: Dataset, ed_list: List[Dict[str, Any]]) -> List[Output]:
+    """
+    Process and aggregate output transects.
+
+    This function processes a list of echodata (ed) dictionaries and aggregates them based on transect numbers.
+    It raises a ValueError if any of the echodata dictionaries have an error flag set to True.
+
+    Parameters:
+        name (str): The name of the process.
+        ed_list (List[Dict[str, Any]]): A list of echodata dictionaries.
+        config (Dataset): Datastore configuration
+
+    Returns:
+        List[Output]: A list of Output instances, each containing aggregated data for a transect.
+
+    Raises:
+        ValueError: If any echodata dictionary has an error flag set to True.
+
+    Example:
+        ed_list = [
+            {"transect": 1, "data": {...}, "error": False},
+            {"transect": 1, "data": {...}, "error": False},
+            {"transect": 2, "data": {...}, "error": False},
+            {"transect": 2, "data": {...}, "error": False},
+            {"transect": 2, "data": {...}, "error": True}
+        ]
+
+        output_list = process_output_transects("Data Processing", ed_list)
+        # Returns a list of Output instances with aggregated data per transect.
+
+    """
+    error_flag = False
+    transect_dict = {}
+    outputs: List[Output] = []
+    for ed in ed_list:
+        if ed["error"] == True:
+            error_flag = True
+            print(ed['error_desc'])
+        transect = ed['transect']
+        if transect in transect_dict:
+            transect_dict[transect].append(ed)
+        else:
+            transect_dict[transect] = [ed]
+
+    for transect in transect_dict.keys():
+        output = Output()
+        output.data = transect_dict[transect]
+        outputs.append(output)
+    if error_flag:
+        store_json_output(data=outputs, config=config, name=name)
+        raise ValueError("Could not complete "+name+" successfully since 1 or more raw files" 
+                                + "failed to convert. Try fixing or skipping the files from the process."
+                                + "If `use_offline` flag is set to true, processed files will be skipped the next run.")
+    return outputs
+
+def store_json_output(data, config: Dataset, name: str):
+    """
+    Store the given data as JSON in the Echoflow working directory.
+
+    This function serializes the provided `data` and stores it as JSON in the `.echoflow`
+    directory within the user's home directory. The stored JSON data can later be retrieved
+    using the `get_output` function.
+
+    Note:
+        The stored data can be retrieved using the `get_output` function with the same type
+        of data.
+
+    Example:
+        >>> output_data = ["data_point_1", "data_point_2", "data_point_3"]
+        >>> store_json_output(output_data, config, "output_name")
+
+    Args:
+        data (Any): The data to be stored.
+        config (Dataset): The configuration dataset.
+        name (str): The name of the JSON output file.
+
+    """
+    home_directory = os.path.expanduser("~")
+    config_directory = os.path.join(home_directory, ".echoflow")
+    json_data_path = os.path.join(config_directory, "echoflow_working_data.json")
+    
+    serialized_data_list = jsonable_encoder(data)
+
+    # Serialize the list to JSON
+    json_data = json.dumps(serialized_data_list)
+
+    with open(json_data_path, "w") as outfile:
+        outfile.write(json_data)
+
+    if config.args.json_export:
+        out_path = make_temp_folder(
+            config.output.urlpath+"/json_metadata", config.output.storage_options_dict)
+        out_path = out_path+"/"+name+".json"
+        fs = extract_fs(out_path, config.output.storage_options_dict)
+        with fs.open(out_path, mode="w") as f:
+            f.write(json_data)
+
+
+    
+def get_output(type : str = "Output"):
+    """
+    Retrieve stored output data from the Echoflow working directory.
+
+    This function retrieves previously stored output data from the JSON file located in
+    the `.echoflow` directory within the user's home directory.
+
+    Args:
+        type (str, optional): The type of data to retrieve. Defaults to "Output". If a different
+        type is specified, the stored raw JSON data is returned without further processing.
+
+    Returns:
+        If type is "Output":
+            A list of Output instances containing the retrieved data.
+        If type is not "Output":
+            The raw JSON data as stored, if available.
+
+    Example:
+        >>> retrieved_data = get_output(type="custom_type")
+        >>> print(retrieved_data)
+        {"key": "value", ...}
+
+    """
+    data = None
+    home_directory = os.path.expanduser("~")
+    config_directory = os.path.join(home_directory, ".echoflow")
+    json_data_path = os.path.join(config_directory, "echoflow_working_data.json")
+    with open(json_data_path, "r") as outfile:
+        data = json.load(outfile)
+
+    if type != "Output":
+        return data  
+    output_list = []
+    for item in data:
+        output_item = Output(**item)
+        output_list.append(output_item)
+    return output_list
+    
+def cleanup(config: Dataset, stage: Stage, data: List[Output]):
+    """
+    Clean up working directory associated with a specific stage of processing.
+
+    This function removes the working directory corresponding to the given stage and dataset configuration.
+
+    Parameters:
+        config (Dataset): The dataset configuration.
+        stage (Stage): The processing stage for which to perform cleanup.
+        data (List[Output]): List of Output objects containing information about processed data.
+
+    Example:
+        >>> dataset_config = Dataset()
+        >>> processing_stage = Stage()
+        >>> output_data = [Output(data=[...]), Output(data=[...])]
+        >>> cleanup(dataset_config, processing_stage, output_data)
+    """
+    if stage is not None:
+        working_dir = get_working_dir(stage=stage, config=config)
+        fs = extract_fs(working_dir, storage_options=config.output.storage_options_dict)
+        print("Cleaning : ",working_dir)
+        try:
+            fs.rm(working_dir, recursive=True)
+            print("Cleanup complete")
+        except Exception as e:
+            print(e)
+            print("Failed to cleanup "+working_dir)
+
+
+def get_last_run_output(data: List[Output] = None, storage_options: Dict[str, Any]={}):
+    """
+    Retrieve Zarr arrays from the last run's output data.
+
+    This function extracts Zarr arrays from the output data of the last run and returns them as a list of lists.
+
+    Parameters:
+        data (List[Output], optional): List of Output objects containing information about processed data.
+        storage_options (Dict[str, Any], optional): Storage options for loading Zarr arrays.
+
+    Returns:
+        List[List]: A list of lists containing Zarr arrays from the last run's output data.
+
+    Example:
+        >>> output_data = [Output(data=[...]), Output(data=[...])]
+        >>> zarr_arrays = get_last_run_output(output_data, storage_options={"key": "value"})
+    """
+    outputs : List[List]= []
+    if data is None:
+        data = get_output()
+    if isinstance(data, list) and isinstance(data[0], Output):
+        try:
+            if isinstance(data[0].data, list):
+                for transect in data:
+                    ed_list = []
+                    for d in transect.data:
+                        ed = get_zarr_list(d, storage_options)
+                        ed_list.append(ed[0])
+                    outputs.append(ed_list)
+            else:
+                for d in data:
+                    ed = get_zarr_list(d, storage_options)
+                    ed_list.append(ed[0])
+                    outputs.append(ed_list)
+            return outputs
+        except Exception as e:
+            print("Could not load the results.")
+            return data
+    else:
+        return data
diff --git a/echoflow/utils/function_utils.py b/echoflow/utils/function_utils.py
new file mode 100644
index 0000000..84023cb
--- /dev/null
+++ b/echoflow/utils/function_utils.py
@@ -0,0 +1,64 @@
+"""
+This module provides utility functions for dynamic function invocation and retrieving function arguments.
+
+Module Functions:
+    dynamic_function_call(module_name: str, function_name: str) -> Optional[Callable]:
+        Dynamically calls a function from a specified module.
+
+    get_function_arguments(function: Callable) -> List[str]:
+        Retrieves the argument names of a given function.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import importlib
+import inspect
+
+
+def dynamic_function_call(module_name, function_name):
+    """
+    Dynamically calls a function from a specified module.
+
+    Args:
+        module_name (str): Name of the module.
+        function_name (str): Name of the function to call.
+
+    Returns:
+        Callable or None: The function object if found, otherwise None.
+
+    Example:
+        function = dynamic_function_call('my_module', 'my_function')
+        if function:
+            result = function()
+    """
+    try:
+        module = importlib.import_module(module_name)
+        function = getattr(module, function_name)
+        return function
+    except ImportError:
+        print(f"Module '{module_name}' not found.")
+    except AttributeError:
+        print(f"Function '{function_name}' not found in module '{module_name}'.")
+
+
+def get_function_arguments(function):
+    """
+    Retrieves the argument names of a given function.
+
+    Args:
+        function (Callable): The function object.
+
+    Returns:
+        List[str]: List of argument names.
+
+    Example:
+        def my_function(arg1, arg2):
+            pass
+        arguments = get_function_arguments(my_function)
+        # Result: ['arg1', 'arg2']
+    """
+    signature = inspect.signature(function)
+    parameters = signature.parameters
+    argument_names = [param.name for param in parameters.values()]
+    return argument_names
\ No newline at end of file
diff --git a/echoflow/utils/rest_utils.py b/echoflow/utils/rest_utils.py
new file mode 100644
index 0000000..2f3f66c
--- /dev/null
+++ b/echoflow/utils/rest_utils.py
@@ -0,0 +1,129 @@
+"""
+Module containing various utility functions for interacting with external systems.
+
+This module provides functions for sending HTTP requests, retrieving flow run history,
+and handling file systems and paths.
+
+Functions:
+    send_request(method, url, headers={}, payload=None):
+        Send an HTTP request to a specified URL with optional headers and payload.
+
+    get_last_flow_run(name, type="FLOW"):
+        Get the ID of the last flow run with the specified name.
+
+    get_last_run_history(name, type="FLOW"):
+        Get the history of the last flow run with the specified name.
+
+Author: Soham Butala
+Email: sbutala@uw.edu
+Date: August 22, 2023
+"""
+import http.client
+import json
+from typing import Any, Dict
+
+
+def send_request(method : str, url : str, headers : Dict[str, Any] = {}, payload : Any = None):
+    """
+    Send an HTTP request to a specified URL with optional headers and payload.
+
+    Parameters:
+        method (str): The HTTP method, e.g., "GET" or "POST".
+        url (str): The URL to send the request to.
+        headers (Dict[str, Any]): Optional headers to include in the request.
+        payload (Any): Optional payload data to send with the request.
+
+    Returns:
+        str: The response data as a string.
+
+    Example:
+        response = send_request(method="GET", url="https://example.com/api/data")
+    """
+    data = None
+    try:
+        conn = http.client.HTTPConnection("127.0.0.1", 4200)
+        if method == "POST":
+            headers.update({
+            'Content-Type': 'application/json'
+            })
+            conn.request(str(method), url, payload, headers)
+        elif method == "GET":
+            conn.request(method=str(method), url=url, headers=headers)
+
+        res = conn.getresponse()
+        data = res.read().decode("utf-8")
+    except Exception as e:
+        print(e)
+    finally:
+        conn.close()
+    return data
+
+
+def get_last_flow_run(name : str, type : str = "FLOW"):
+    """
+    Get the ID of the last flow run with the specified name.
+
+    Parameters:
+        name (str): The name of the flow.
+        type (str): The type of flow (default: "FLOW").
+
+    Returns:
+        str: The ID of the last flow run, or None if no flow runs were found.
+
+    Example:
+        flow_name = "MyFlow"
+        last_run_id = get_last_flow_run(name=flow_name)
+    """
+    data : str = None
+    id_value : str = None
+
+    payload = json.dumps({
+    "sort": "END_TIME_DESC",
+    "limit": 1,
+    "offset": 0,
+    "flows": {
+        "operator": "and_",
+        "name": {
+        "any_": [
+            name
+        ]
+        }
+    }
+    })
+    
+    data = send_request(method="POST", payload=payload, url="/api/ui/flow_runs/history")
+
+    if data is not None:
+        parsed_data = json.loads(data)
+
+        # Access the 'id' value from the first dictionary in the list
+        if parsed_data and isinstance(parsed_data, list):
+            first_dict = parsed_data[0]
+            id_value = first_dict.get('id')
+        
+    return id_value
+    
+
+def get_last_run_history(name: str, type : str = "FLOW"):
+    """
+    Get the history of the last flow run with the specified name.
+
+    Parameters:
+        name (str): The name of the flow.
+        type (str): The type of flow (default: "FLOW").
+
+    Returns:
+        Any: The run history data, or None if no history was found.
+
+    Example:
+        flow_name = "MyFlow"
+        last_run_history = get_last_run_history(name=flow_name)
+    """
+    id = get_last_flow_run(name=name)
+    run_history : str = None
+    if id is not None:
+        run_history = send_request(method="GET", url="/api/flow_runs/"+id+"/graph")
+        run_json = json.loads(run_history)
+        if run_json and not isinstance(run_json, str):
+            return run_json
+    return run_history
diff --git a/environment.yml b/environment.yml
new file mode 100644
index 0000000..9f256dc
--- /dev/null
+++ b/environment.yml
@@ -0,0 +1,219 @@
+name: echoflow
+channels:
+  - defaults
+dependencies:
+  - aiobotocore=2.4.2=py310hecd8cb5_0
+  - aiohttp=3.8.3=py310h6c40b1e_0
+  - aioitertools=0.7.1=pyhd3eb1b0_0
+  - aiosignal=1.2.0=pyhd3eb1b0_0
+  - appnope=0.1.2=py310hecd8cb5_1001
+  - asttokens=2.0.5=pyhd3eb1b0_0
+  - async-timeout=4.0.2=py310hecd8cb5_0
+  - backcall=0.2.0=pyhd3eb1b0_0
+  - blas=1.0=mkl
+  - bokeh=2.4.3=py310hecd8cb5_0
+  - brotlipy=0.7.0=py310hca72f7f_1002
+  - bzip2=1.0.8=h1de35cc_0
+  - ca-certificates=2023.05.30=hecd8cb5_0
+  - cchardet=2.1.7=py310hcec6c5f_0
+  - certifi=2023.7.22=py310hecd8cb5_0
+  - cffi=1.15.1=py310h6c40b1e_3
+  - charset-normalizer=2.0.4=pyhd3eb1b0_0
+  - comm=0.1.2=py310hecd8cb5_0
+  - cryptography=41.0.2=py310h3b477ad_0
+  - debugpy=1.6.7=py310hcec6c5f_0
+  - decorator=5.1.1=pyhd3eb1b0_0
+  - executing=0.8.3=pyhd3eb1b0_0
+  - freetype=2.12.1=hd8bbffd_0
+  - frozenlist=1.3.3=py310h6c40b1e_0
+  - fsspec=2023.3.0=py310hecd8cb5_0
+  - giflib=5.2.1=h6c40b1e_3
+  - idna=3.4=py310hecd8cb5_0
+  - intel-openmp=2023.1.0=ha357a0b_43547
+  - ipykernel=6.19.2=py310h20db666_0
+  - ipython=8.12.0=py310hecd8cb5_0
+  - jedi=0.18.1=py310hecd8cb5_1
+  - jinja2=3.1.2=py310hecd8cb5_0
+  - jmespath=0.10.0=pyhd3eb1b0_0
+  - jpeg=9e=h6c40b1e_1
+  - jupyter_client=8.1.0=py310hecd8cb5_0
+  - jupyter_core=5.3.0=py310hecd8cb5_0
+  - lcms2=2.12=hf1fd2bf_0
+  - lerc=3.0=he9d5cce_0
+  - libcxx=14.0.6=h9765a3e_0
+  - libdeflate=1.17=hb664fd8_0
+  - libffi=3.4.4=hecd8cb5_0
+  - libpng=1.6.39=h6c40b1e_0
+  - libsodium=1.0.18=h1de35cc_0
+  - libtiff=4.5.0=hcec6c5f_2
+  - libwebp=1.2.4=hf6ce154_1
+  - libwebp-base=1.2.4=h6c40b1e_1
+  - lz4-c=1.9.4=hcec6c5f_0
+  - markupsafe=2.1.1=py310hca72f7f_0
+  - matplotlib-inline=0.1.6=py310hecd8cb5_0
+  - mkl=2023.1.0=h59209a4_43558
+  - mkl-service=2.4.0=py310h6c40b1e_1
+  - mkl_fft=1.3.6=py310h3ea8b11_1
+  - mkl_random=1.2.2=py310h3ea8b11_1
+  - multidict=6.0.2=py310hca72f7f_0
+  - ncurses=6.4=hcec6c5f_0
+  - nest-asyncio=1.5.6=py310hecd8cb5_0
+  - numpy=1.25.0=py310h827a554_0
+  - numpy-base=1.25.0=py310ha186be2_0
+  - openssl=3.0.9=hca72f7f_0
+  - packaging=23.0=py310hecd8cb5_0
+  - parso=0.8.3=pyhd3eb1b0_0
+  - pexpect=4.8.0=pyhd3eb1b0_3
+  - pickleshare=0.7.5=pyhd3eb1b0_1003
+  - pillow=9.4.0=py310hcec6c5f_0
+  - pip=23.2.1=py310hecd8cb5_0
+  - platformdirs=2.5.2=py310hecd8cb5_0
+  - prompt-toolkit=3.0.36=py310hecd8cb5_0
+  - ptyprocess=0.7.0=pyhd3eb1b0_2
+  - pure_eval=0.2.2=pyhd3eb1b0_0
+  - pycparser=2.21=pyhd3eb1b0_0
+  - pydantic=1.10.8=py310h6c40b1e_0
+  - pygments=2.15.1=py310hecd8cb5_1
+  - pyopenssl=23.2.0=py310hecd8cb5_0
+  - pysocks=1.7.1=py310hecd8cb5_0
+  - python=3.10.12=h5ee71fb_0
+  - python-dateutil=2.8.2=pyhd3eb1b0_0
+  - pyyaml=6.0=py310h6c40b1e_1
+  - pyzmq=25.1.0=py310hcec6c5f_0
+  - readline=8.2=hca72f7f_0
+  - s3fs=2023.3.0=py310hecd8cb5_0
+  - s3transfer=0.6.0=py310hecd8cb5_0
+  - setuptools=68.0.0=py310hecd8cb5_0
+  - six=1.16.0=pyhd3eb1b0_1
+  - sqlite=3.41.2=h6c40b1e_0
+  - stack_data=0.2.0=pyhd3eb1b0_0
+  - tbb=2021.8.0=ha357a0b_0
+  - tk=8.6.12=h5d9f67b_0
+  - tornado=6.3.2=py310h6c40b1e_0
+  - traitlets=5.7.1=py310hecd8cb5_0
+  - typing-extensions=4.7.1=py310hecd8cb5_0
+  - typing_extensions=4.7.1=py310hecd8cb5_0
+  - urllib3=1.26.16=py310hecd8cb5_0
+  - wcwidth=0.2.5=pyhd3eb1b0_0
+  - wheel=0.38.4=py310hecd8cb5_0
+  - wrapt=1.14.1=py310hca72f7f_0
+  - xz=5.4.2=h6c40b1e_0
+  - yaml=0.2.5=haf1e3a3_0
+  - yarl=1.8.1=py310hca72f7f_0
+  - zeromq=4.3.4=h23ab428_0
+  - zlib=1.2.13=h4dc903c_0
+  - zstd=1.5.5=hc035e20_0
+  - pip:
+      - aiosqlite==0.19.0
+      - alembic==1.11.1
+      - anyio==3.7.1
+      - apprise==1.4.5
+      - asciitree==0.3.3
+      - asgi-lifespan==2.1.0
+      - asyncpg==0.28.0
+      - attrs==23.1.0
+      - azure-common==1.1.28
+      - azure-core==1.29.2
+      - azure-cosmos==4.5.0
+      - azure-identity==1.14.0
+      - azure-mgmt-containerinstance==10.1.0
+      - azure-mgmt-core==1.4.0
+      - azure-mgmt-resource==23.0.1
+      - boto3==1.24.59
+      - botocore==1.27.59
+      - cachetools==5.3.1
+      - cftime==1.6.2
+      - click==8.1.6
+      - cloudpickle==2.2.1
+      - colorama==0.4.6
+      - coolname==2.2.0
+      - croniter==1.4.1
+      - dask==2023.3.1
+      - dateparser==1.1.8
+      - distributed==2023.3.1
+      - docker==6.1.3
+      - echopype==0.7.2.dev40+g6671fec
+      - entrypoints==0.4
+      - exceptiongroup==1.1.2
+      - fastapi==0.100.1
+      - fasteners==0.18
+      - geographiclib==2.0
+      - geopy==2.3.0
+      - google-auth==2.22.0
+      - greenlet==2.0.2
+      - griffe==0.32.3
+      - h11==0.14.0
+      - h2==4.1.0
+      - hpack==4.0.0
+      - httpcore==0.17.3
+      - httpx==0.24.1
+      - hyperframe==6.0.1
+      - isodate==0.6.1
+      - jsonpatch==1.33
+      - jsonpointer==2.4
+      - jsonschema==4.18.4
+      - jsonschema-specifications==2023.7.1
+      - kubernetes==27.2.0
+      - locket==1.0.0
+      - mako==1.2.4
+      - markdown==3.4.4
+      - markdown-it-py==3.0.0
+      - mdurl==0.1.2
+      - more-itertools==8.13.0
+      - msal==1.23.0
+      - msal-extensions==1.0.0
+      - msgpack==1.0.5
+      - mypy-boto3-s3==1.28.19
+      - mypy-boto3-secretsmanager==1.28.24
+      - netcdf4==1.6.4
+      - numcodecs==0.11.0
+      - oauthlib==3.2.2
+      - orjson==3.9.2
+      - pandas==1.5.3
+      - partd==1.4.0
+      - pathspec==0.11.2
+      - pendulum==2.1.2
+      - portalocker==2.7.0
+      - prefect==2.11.2
+      - prefect-aws==0.3.6
+      - prefect-azure==0.2.11
+      - prefect-dask==0.2.4
+      - psutil==5.9.5
+      - pyasn1==0.5.0
+      - pyasn1-modules==0.3.0
+      - pyjwt==2.8.0
+      - pynmea2==1.19.0
+      - python-slugify==8.0.1
+      - pytz==2023.3
+      - pytzdata==2020.1
+      - readchar==4.0.5
+      - referencing==0.30.0
+      - regex==2023.6.3
+      - requests==2.31.0
+      - requests-oauthlib==1.3.1
+      - rich==13.5.2
+      - rpds-py==0.9.2
+      - rsa==4.9
+      - ruamel-yaml==0.17.32
+      - ruamel-yaml-clib==0.2.7
+      - scipy==1.11.1
+      - sniffio==1.3.0
+      - sortedcontainers==2.4.0
+      - sqlalchemy==2.0.19
+      - starlette==0.27.0
+      - tblib==2.0.0
+      - text-unidecode==1.3
+      - toml==0.10.2
+      - toolz==0.12.0
+      - typer==0.9.0
+      - tzdata==2023.3
+      - tzlocal==5.0.1
+      - uvicorn==0.23.2
+      - watchtower==3.0.1
+      - websocket-client==1.6.1
+      - websockets==11.0.3
+      - xarray==2023.7.0
+      - xarray-datatree==0.0.6
+      - zarr==2.16.0
+      - zict==3.0.0
+prefix: /Users/soham/anaconda3/envs/echoflow
diff --git a/notebooks/Archive.zip b/notebooks/Archive.zip
new file mode 100644
index 0000000..57008fc
Binary files /dev/null and b/notebooks/Archive.zip differ
diff --git a/notebooks/ConfigurationDemo.ipynb b/notebooks/ConfigurationDemo.ipynb
new file mode 100644
index 0000000..88e32b0
--- /dev/null
+++ b/notebooks/ConfigurationDemo.ipynb
@@ -0,0 +1,102 @@
+{
+ "cells": [
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "### Echoflow Setup Notebook\n",
+    "##### To create a new profile in prefect execute the `prefect cloud login` command or refer the prefect docs available at https://docs.prefect.io/2.11.4/cloud/cloud-quickstart/ for latest setup instructions. If you wish to use local prefect create a local profile. `echoflow_create_prefect_profile()` can be used to create a new profile. After creating a profile make sure you are using the appropriate profile before running this pipeline. `load_profile()` can be used to switch the profile. \n",
+    "\n",
+    "#### Note :\n",
+    "- Use pip install -e . to install all dependencies\n",
+    "- pip install prefect is not required since the above command will do it\n",
+    "- Below code uses KEY and SECRET for AWS stored as environment variables for example"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import os\n",
+    "key = os.environ[\"KEY\"]\n",
+    "secret = os.environ[\"SECRET\"]\n",
+    "from echoflow import echoflow_config_AWS, echoflow_create_prefect_profile, load_profile, echoflow_config_AZ_cosmos"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "profile = echoflow_create_prefect_profile(name=\"echoflow-local\", set_active=False)\n",
+    "profile = load_profile(name=\"echoflow-local\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "load_profile(name=\"default\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# AWS Example\n",
+    "from prefect_aws import AwsCredentials\n",
+    "\n",
+    "options={}\n",
+    "options[\"storage_type\"] = \"destination\"\n",
+    "\n",
+    "func = echoflow_config_AWS(aws_key=key, aws_secret=secret, options=options, active=False)\n",
+    "aws = await AwsCredentials.load(\"echoflow-aws-credentials\")\n",
+    "s3c = aws.get_s3_client()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Azure Cosmos Example\n",
+    "from prefect_azure import AzureCosmosDbCredentials\n",
+    "\n",
+    "\n",
+    "cs = \"\"\n",
+    "func = echoflow_config_AZ_cosmos(connection_string=cs, options=options, active=False)\n",
+    "az = await AzureCosmosDbCredentials.load(\"echoflow-az-credentials\")\n",
+    "azc = az.get_client()"
+   ]
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/FlowDemo.ipynb b/notebooks/FlowDemo.ipynb
new file mode 100644
index 0000000..b848d42
--- /dev/null
+++ b/notebooks/FlowDemo.ipynb
@@ -0,0 +1,95 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "import os\n",
+    "sys.path.append('/home/ubuntu/echoflow/echoflow')\n",
+    "from echoflow import echoflow_start, StorageType, load_block\n",
+    "\n",
+    "dataset_config = os.path.abspath(\"../echoflow/config/datastore.yaml\")\n",
+    "pipeline_config = os.path.abspath(\"../echoflow/config/pipeline.yaml\")\n",
+    "logfile_config = os.path.abspath(\"../echoflow/config/logging_config.yaml\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "aws = load_block(name=\"echoflow-aws-credentials\", type=StorageType.AWS)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params={'sonar_model': 'EK60', 'xml_path': 's3//'} options={'save_output': True, 'use_raw_offline': True, 'use_offline': True} prefect_config={'retries': 0}\n",
+      "[Errno 111] Connection refused\n",
+      "[Output(data=[{'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T171516.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T171516.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T174838.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T174838.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T182247.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T182247.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T185656.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T185656.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T193105.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T193105.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T200518.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T200518.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T203931.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T203931.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T211247.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T211247.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T213837.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T213837.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T220425.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T220425.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T223013.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T223013.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T225602.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T225602.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T232152.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T232152.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170627-T234752.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170627-T234752.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T001354.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T001354.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T003944.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T003944.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T010526.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T010526.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T013108.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T013108.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T015659.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T015659.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T022249.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T022249.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T024842.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T024842.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/7/Summer2017-D20170628-T031428.zarr', 'transect': 7, 'file_name': 'Summer2017-D20170628-T031428.raw', 'error': False}], passing_params={}), Output(data=[{'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T184837.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T184837.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T192246.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T192246.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T195655.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T195655.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T203107.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T203107.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T210519.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T210519.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T213928.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T213928.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T221341.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T221341.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T224409.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T224409.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T230959.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T230959.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170628-T233549.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170628-T233549.raw', 'error': False}, {'out_path': 's3://echoflow-workground/combined_files/echoflow_open_raw/10/Summer2017-D20170629-T000153.zarr', 'transect': 10, 'file_name': 'Summer2017-D20170629-T000153.raw', 'error': False}], passing_params={})]\n",
+      "\n",
+      "Completed stage name='echoflow_open_raw' module='echoflow.stages.subflows.open_raw' external_params={'sonar_model': 'EK60', 'xml_path': 's3//'} options={'save_output': True, 'use_raw_offline': True, 'use_offline': True} prefect_config={'retries': 0}\n",
+      "--------------------------------------------------\n",
+      "--------------------------------------------------\n",
+      "\n",
+      "Executing stage :  name='echoflow_compute_TS' module='echoflow.stages.subflows.compute_TS' external_params=None options={'use_offline': True} prefect_config=None\n",
+      "[Errno 111] Connection refused\n"
+     ]
+    }
+   ],
+   "source": [
+    "options = {\"storage_options_override\": False}\n",
+    "data  = echoflow_start(dataset_config=dataset_config, pipeline_config=pipeline_config, logging_config=logfile_config, storage_options=aws, options=options)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "data"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "Python 3 (ipykernel)",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  }
+ },
+ "nbformat": 4,
+ "nbformat_minor": 4
+}
diff --git a/notebooks/Hake.zip b/notebooks/Hake.zip
new file mode 100644
index 0000000..5b4b853
Binary files /dev/null and b/notebooks/Hake.zip differ
diff --git a/notebooks/HakeFlowDemo.ipynb b/notebooks/HakeFlowDemo.ipynb
deleted file mode 100644
index a39c9cd..0000000
--- a/notebooks/HakeFlowDemo.ipynb
+++ /dev/null
@@ -1,85 +0,0 @@
-{
- "cells": [
-  {
-   "cell_type": "markdown",
-   "id": "ff019352-e761-4184-8034-27b906ac2a7e",
-   "metadata": {},
-   "source": [
-    "# Hake Prefect Flow Demo\n",
-    "\n",
-    "This notebook shows how to use the level 0 `fetch_and_convert` data pipeline that extracts file urls from a url path,\n",
-    "then these file urls are read and converted to `zarr` format, following a Sonar Data Standard via `echopype` python library."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "a03f7fc3-06f8-400f-8037-003db51ca43e",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "from echoflow.stages.level0 import fetch_and_convert\n",
-    "from echoflow.stages.level1 import open_and_calibrate"
-   ]
-  },
-  {
-   "cell_type": "markdown",
-   "id": "1c19e30b-80c5-43da-8151-8417985e68bf",
-   "metadata": {},
-   "source": [
-    "With one line, the prefect flow will fetch, read, and convert the specified sonar data.\n",
-    "The exported `zarr` files will be split into weekly or transect files."
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "9ae1730e-c428-4516-bf63-1c79d0853bd5",
-   "metadata": {
-    "scrolled": true,
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "# Level 0\n",
-    "file_paths = fetch_and_convert(config=\"./hake.yaml\")"
-   ]
-  },
-  {
-   "cell_type": "code",
-   "execution_count": null,
-   "id": "5e3e90c2-979d-427d-938e-0aebc7073119",
-   "metadata": {
-    "tags": []
-   },
-   "outputs": [],
-   "source": [
-    "# Level 1\n",
-    "datasets_Sv = open_and_calibrate(config=\"./hake.yaml\", file_paths=file_paths)"
-   ]
-  }
- ],
- "metadata": {
-  "kernelspec": {
-   "display_name": "Python [conda env:ef-dev]",
-   "language": "python",
-   "name": "conda-env-ef-dev-py"
-  },
-  "language_info": {
-   "codemirror_mode": {
-    "name": "ipython",
-    "version": 3
-   },
-   "file_extension": ".py",
-   "mimetype": "text/x-python",
-   "name": "python",
-   "nbconvert_exporter": "python",
-   "pygments_lexer": "ipython3",
-   "version": "3.8.15"
-  }
- },
- "nbformat": 4,
- "nbformat_minor": 5
-}
diff --git a/notebooks/aop_test.ipynb b/notebooks/aop_test.ipynb
new file mode 100644
index 0000000..7527c1e
--- /dev/null
+++ b/notebooks/aop_test.ipynb
@@ -0,0 +1,82 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "ename": "TypeError",
+     "evalue": "metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases",
+     "output_type": "error",
+     "traceback": [
+      "\u001b[0;31m---------------------------------------------------------------------------\u001b[0m",
+      "\u001b[0;31mTypeError\u001b[0m                                 Traceback (most recent call last)",
+      "Cell \u001b[0;32mIn[3], line 11\u001b[0m\n\u001b[1;32m      8\u001b[0m             \u001b[39mcls\u001b[39m\u001b[39m.\u001b[39m_instances[\u001b[39mcls\u001b[39m] \u001b[39m=\u001b[39m \u001b[39msuper\u001b[39m(SingletonMeta, \u001b[39mcls\u001b[39m)\u001b[39m.\u001b[39m\u001b[39m__call__\u001b[39m(\u001b[39m*\u001b[39margs, \u001b[39m*\u001b[39m\u001b[39m*\u001b[39mkwargs)\n\u001b[1;32m      9\u001b[0m         \u001b[39mreturn\u001b[39;00m \u001b[39mcls\u001b[39m\u001b[39m.\u001b[39m_instances[\u001b[39mcls\u001b[39m]\n\u001b[0;32m---> 11\u001b[0m \u001b[39mclass\u001b[39;00m \u001b[39mSingletonClassA\u001b[39;00m(pydantic\u001b[39m.\u001b[39mBaseModel, metaclass\u001b[39m=\u001b[39mSingletonMeta):\n\u001b[1;32m     12\u001b[0m     attr : \u001b[39mstr\u001b[39m\n\u001b[1;32m     13\u001b[0m     \u001b[39mdef\u001b[39;00m \u001b[39msome_method\u001b[39m(\u001b[39mself\u001b[39m):\n",
+      "\u001b[0;31mTypeError\u001b[0m: metaclass conflict: the metaclass of a derived class must be a (non-strict) subclass of the metaclasses of all its bases"
+     ]
+    }
+   ],
+   "source": [
+    "import pydantic\n",
+    "\n",
+    "class SingletonMeta(type):\n",
+    "    _instances = {}\n",
+    "\n",
+    "    def __call__(cls, *args, **kwargs):\n",
+    "        if cls not in cls._instances:\n",
+    "            cls._instances[cls] = super(SingletonMeta, cls).__call__(*args, **kwargs)\n",
+    "        return cls._instances[cls]\n",
+    "\n",
+    "class SingletonClassA(pydantic.BaseModel, metaclass=SingletonMeta):\n",
+    "    attr : str\n",
+    "    def some_method(self):\n",
+    "        print(\"Doing something in Class A...\")\n",
+    "\n",
+    "class SingletonClassB(pydantic.BaseModel,metaclass=SingletonMeta):\n",
+    "    attrb: str\n",
+    "    def some_method(self):\n",
+    "        print(\"Doing something in Class B...\")\n",
+    "\n",
+    "# Example usage\n",
+    "instance1 = SingletonClassA(attr=\"a\")\n",
+    "instance2 = SingletonClassA(attr=\"a\")\n",
+    "\n",
+    "instance3 = SingletonClassB(attrb=\"b\")\n",
+    "instance4 = SingletonClassB(attrb=\"b\")\n",
+    "\n",
+    "print(instance1 is instance2)  # Output: True\n",
+    "print(instance3 is instance4)  # Output: True\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "echoflow",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/dask_issue.py b/notebooks/dask_issue.py
new file mode 100644
index 0000000..75d3f34
--- /dev/null
+++ b/notebooks/dask_issue.py
@@ -0,0 +1,7 @@
+from prefect_dask import DaskTaskRunner
+from prefect import task, flow
+
+@task
+def process_raw():
+    temp_file = Path()
+    raw = {'instrument': 'EK60', 'file_path': 's3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/Summer2017-D20170627-T203931.raw', 'transect_num': 7, 'month': 6, 'year': 2017, 'jday': 178, 'datetime': '2017-06-27T20:39:31'}
\ No newline at end of file
diff --git a/notebooks/database_test copy.ipynb b/notebooks/database_test copy.ipynb
new file mode 100644
index 0000000..08f273f
--- /dev/null
+++ b/notebooks/database_test copy.ipynb	
@@ -0,0 +1,182 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import sys\n",
+    "import os\n",
+    "sys.path.append('/Users/Soham/Desktop/EchoWorkSpace/echoflow/')\n",
+    "import json\n",
+    "import echoflow.stages_v2.utils.databse_utils as db\n",
+    "from echoflow.config.models.db_log_model import DB_Log, Log_Data, Process"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<sqlite3.Connection object at 0x1128bd7b0>\n"
+     ]
+    }
+   ],
+   "source": [
+    "conn = db.get_connection('./echoflow_db/')\n",
+    "print(conn)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 10,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "<sqlite3.Connection object at 0x10a6fa7a0>\n"
+     ]
+    }
+   ],
+   "source": [
+    "db.create_log_table(conn)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "[(1, '2023-07-20 17:48:14.839424', '2023-07-20 17:48:18.704082', '{\"Configuration\": {\"name\": \"Configuration\", \"process_stack\": [{\"name\": \"init_flow\", \"start_time\": \"2023-07-20 17:48:10.420600\", \"end_time\": \"2023-07-20 17:48:18.703947\", \"status\": false, \"error\": null}]}}', '1', None), (2, '2023-07-22 16:28:28.474798', '2023-07-22 16:28:31.631818', '{\"Configuration\": {\"name\": \"Configuration\", \"process_stack\": [{\"name\": \"glob_all_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:31.631585\", \"status\": true, \"error\": null}]}}', '0', None), (3, '2023-07-22 16:28:28.474798', '2023-07-22 16:28:32.010278', '{\"Configuration\": {\"name\": \"Configuration\", \"process_stack\": [{\"name\": \"glob_all_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:31.631585\", \"status\": true, \"error\": null}, {\"name\": \"parse_raw_paths\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.010169\", \"status\": true, \"error\": null}]}}', '0', None), (4, '2023-07-22 16:28:28.474798', '2023-07-22 16:28:32.186253', '{\"Configuration\": {\"name\": \"Configuration\", \"process_stack\": [{\"name\": \"glob_all_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:31.631585\", \"status\": true, \"error\": null}, {\"name\": \"parse_raw_paths\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.010169\", \"status\": true, \"error\": null}, {\"name\": \"club_raw_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.186187\", \"status\": true, \"error\": null}]}}', '0', None), (5, '2023-07-22 16:28:28.474798', '2023-07-22 16:28:32.221189', '{\"Configuration\": {\"name\": \"Configuration\", \"process_stack\": [{\"name\": \"glob_all_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:31.631585\", \"status\": true, \"error\": null}, {\"name\": \"parse_raw_paths\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.010169\", \"status\": true, \"error\": null}, {\"name\": \"club_raw_files\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.186187\", \"status\": true, \"error\": null}, {\"name\": \"init_flow\", \"start_time\": \"2023-07-22 16:28:25.235681\", \"end_time\": \"2023-07-22 16:28:32.220689\", \"status\": true, \"error\": null}]}}', '0', None), (6, '2023-07-12 09:00:00', '2023-07-22 16:52:18.249180', '{\"exampl.txt\": {\"name\": \"example.txt\", \"process_stack\": [{\"name\": \"Process 1\", \"start_time\": \"09:05:00\", \"end_time\": \"09:10:00\", \"status\": true, \"error\": \"\"}, {\"name\": \"Process 2\", \"start_time\": \"09:15:00\", \"end_time\": \"09:30:00\", \"status\": false, \"error\": \"ValueError Occurred\"}]}}', 'Completed', None), (7, '2023-07-12 09:00:00', '2023-07-22 16:53:34.993540', '{\"exampl.txt\": {\"name\": \"example.txt\", \"process_stack\": [{\"name\": \"Process 1\", \"start_time\": \"09:05:00\", \"end_time\": \"09:10:00\", \"status\": true, \"error\": \"\"}, {\"name\": \"Process 2\", \"start_time\": \"09:15:00\", \"end_time\": \"09:30:00\", \"status\": false, \"error\": \"ValueError Occurred\"}]}}', 'Completed', None), (8, '2023-07-12 09:00:00', '2023-07-22 16:55:38.794985', '{\"exampl.txt\": {\"name\": \"example.txt\", \"process_stack\": [{\"name\": \"Process 1\", \"start_time\": \"09:05:00\", \"end_time\": \"09:10:00\", \"status\": true, \"error\": \"\"}, {\"name\": \"Process 2\", \"start_time\": \"09:15:00\", \"end_time\": \"09:30:00\", \"status\": false, \"error\": \"ValueError Occurred\"}]}}', 'Completed', None)]\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[DB_Log(run_id=1, start_time='2023-07-20 17:48:14.839424', end_time='2023-07-20 17:48:18.704082', data={'Configuration': Log_Data(name='Configuration', process_stack=[Process(name='init_flow', start_time='2023-07-20 17:48:10.420600', end_time='2023-07-20 17:48:18.703947', status=False, error=None)])}, status='1', error=None),\n",
+       " DB_Log(run_id=2, start_time='2023-07-22 16:28:28.474798', end_time='2023-07-22 16:28:31.631818', data={'Configuration': Log_Data(name='Configuration', process_stack=[Process(name='glob_all_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:31.631585', status=True, error=None)])}, status='0', error=None),\n",
+       " DB_Log(run_id=3, start_time='2023-07-22 16:28:28.474798', end_time='2023-07-22 16:28:32.010278', data={'Configuration': Log_Data(name='Configuration', process_stack=[Process(name='glob_all_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:31.631585', status=True, error=None), Process(name='parse_raw_paths', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.010169', status=True, error=None)])}, status='0', error=None),\n",
+       " DB_Log(run_id=4, start_time='2023-07-22 16:28:28.474798', end_time='2023-07-22 16:28:32.186253', data={'Configuration': Log_Data(name='Configuration', process_stack=[Process(name='glob_all_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:31.631585', status=True, error=None), Process(name='parse_raw_paths', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.010169', status=True, error=None), Process(name='club_raw_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.186187', status=True, error=None)])}, status='0', error=None),\n",
+       " DB_Log(run_id=5, start_time='2023-07-22 16:28:28.474798', end_time='2023-07-22 16:28:32.221189', data={'Configuration': Log_Data(name='Configuration', process_stack=[Process(name='glob_all_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:31.631585', status=True, error=None), Process(name='parse_raw_paths', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.010169', status=True, error=None), Process(name='club_raw_files', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.186187', status=True, error=None), Process(name='init_flow', start_time='2023-07-22 16:28:25.235681', end_time='2023-07-22 16:28:32.220689', status=True, error=None)])}, status='0', error=None),\n",
+       " DB_Log(run_id=6, start_time='2023-07-12 09:00:00', end_time='2023-07-22 16:52:18.249180', data={'exampl.txt': Log_Data(name='example.txt', process_stack=[Process(name='Process 1', start_time='09:05:00', end_time='09:10:00', status=True, error=''), Process(name='Process 2', start_time='09:15:00', end_time='09:30:00', status=False, error='ValueError Occurred')])}, status='Completed', error=None),\n",
+       " DB_Log(run_id=7, start_time='2023-07-12 09:00:00', end_time='2023-07-22 16:53:34.993540', data={'exampl.txt': Log_Data(name='example.txt', process_stack=[Process(name='Process 1', start_time='09:05:00', end_time='09:10:00', status=True, error=''), Process(name='Process 2', start_time='09:15:00', end_time='09:30:00', status=False, error='ValueError Occurred')])}, status='Completed', error=None),\n",
+       " DB_Log(run_id=8, start_time='2023-07-12 09:00:00', end_time='2023-07-22 16:55:38.794985', data={'exampl.txt': Log_Data(name='example.txt', process_stack=[Process(name='Process 1', start_time='09:05:00', end_time='09:10:00', status=True, error=''), Process(name='Process 2', start_time='09:15:00', end_time='09:30:00', status=False, error='ValueError Occurred')])}, status='Completed', error=None)]"
+      ]
+     },
+     "execution_count": 3,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "logs = db.parse_all_log_data(conn)\n",
+    "logs"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "log_obj = DB_Log(\n",
+    "    start_time='2023-07-12 09:00:00',\n",
+    "    end_time='2023-07-12 10:00:00',\n",
+    "    data={\n",
+    "        \"exampl.txt\" : Log_Data(\n",
+    "        name='example.txt',\n",
+    "        process_stack=[\n",
+    "            Process(name='Process 1', start_time='09:05:00', end_time='09:10:00', status=True, error=\"\"),\n",
+    "            Process(name='Process 2', start_time='09:15:00', end_time='09:30:00', status=False, error=\"ValueError Occurred\")\n",
+    "        ]\n",
+    "    )},\n",
+    "    status='Completed',\n",
+    "    error=\"\"\n",
+    ")\n",
+    "\n",
+    "out = db.insert_log_data(conn,log_obj)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "[]"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "abc = out.fetchall()\n",
+    "abc"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "9"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "out.lastrowid"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "echoflow",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.11.4"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/echoflow.ipynb b/notebooks/echoflow.ipynb
new file mode 100644
index 0000000..0fcb227
--- /dev/null
+++ b/notebooks/echoflow.ipynb
@@ -0,0 +1,1798 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import echopype as ep\n",
+    "\n",
+    "from prefect import task, flow\n",
+    "from prefect_dask import DaskTaskRunner, get_dask_client\n",
+    "from distributed.utils import logging\n",
+    "import dask\n",
+    "from distributed import LocalCluster, Client"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "2023-08-08 12:40:02,675 - distributed.diskutils - INFO - Found stale lock file and directory '/var/folders/1c/c2gm4rw56831d9cfkdf10y7h0000gn/T/dask-worker-space/worker-pledh8l2', purging\n",
+      "2023-08-08 12:40:02,676 - distributed.diskutils - INFO - Found stale lock file and directory '/var/folders/1c/c2gm4rw56831d9cfkdf10y7h0000gn/T/dask-worker-space/worker-kd_qhh7y', purging\n",
+      "2023-08-08 12:40:02,676 - distributed.diskutils - INFO - Found stale lock file and directory '/var/folders/1c/c2gm4rw56831d9cfkdf10y7h0000gn/T/dask-worker-space/worker-6vr9lqmb', purging\n",
+      "2023-08-08 12:40:02,677 - distributed.diskutils - INFO - Found stale lock file and directory '/var/folders/1c/c2gm4rw56831d9cfkdf10y7h0000gn/T/dask-worker-space/worker-be34sx1z', purging\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "'http://127.0.0.1:8787/status'"
+      ]
+     },
+     "execution_count": 2,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "cluster = LocalCluster()\n",
+    "cluster.dashboard_link"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "client = Client(address=cluster.scheduler_address)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/plain": [
+       "<pooled rpc to 'tcp://127.0.0.1:57028'>"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "client.scheduler"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "@task\n",
+    "def open_raw(fpath):\n",
+    "    ed = ep.open_raw(fpath, sonar_model='ek60')\n",
+    "    print(\"opened raw\")\n",
+    "    ed.to_zarr(\"./combined_files\", overwrite=True, compute=False)\n",
+    "    \n",
+    "@flow(task_runner=DaskTaskRunner(address=cluster.scheduler_address))\n",
+    "def convert_files(fpaths):\n",
+    "    futures = [open_raw.submit(p) for p in fpaths]\n",
+    "    return [f.result() for f in futures]"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 6,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import glob\n",
+    "fpaths = glob.glob(\"../notebooks/offline_files/*.raw\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 7,
+   "metadata": {},
+   "outputs": [
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "opened raw\n",
+      "opened raw\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked channel\n",
+      "Rechunked NMEA_datagramRechunked time1\n",
+      "\n",
+      "opened raw\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked time1\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked channel\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked rollRechunked latitude\n",
+      "\n",
+      "RechunkedRechunked latitude\n",
+      " vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked longitude\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked sentence_type\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked pitch\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked roll\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked water_level\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked time1\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked time2\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked channel\n",
+      "Rechunked MRU_rotation_z\n",
+      "RechunkedRechunked position_offset_x\n",
+      " transducer_offset_xRechunked position_offset_y\n",
+      "Rechunked longitude\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked\n",
+      "Rechunked time1\n",
+      "RechunkedRechunked transducer_offset_y\n",
+      " time2\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked channel\n",
+      "Rechunked sentence_type\n",
+      " Rechunked transducer_offset_x\n",
+      "Rechunked channel\n",
+      "Rechunked pitch\n",
+      "Rechunked time1\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "frequency_nominal\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked absorption_indicative\n",
+      "RechunkedRechunked frequency_nominal\n",
+      " water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "opened raw\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked Rechunked channel\n",
+      "Rechunked time1position_offset_z\n",
+      "\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked Rechunkedtime1\n",
+      " beam_group\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked channel_mode\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beam_group\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked time1\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "RechunkedRechunked transmit_type\n",
+      "Rechunked beam_stabilisation angle_athwartship\n",
+      "\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked Rechunked beam_direction_x\n",
+      "Rechunkedbeamwidth_twoway_athwartship\n",
+      " beam_direction_y\n",
+      "Rechunked beam_direction_x\n",
+      "RechunkedRechunked beam_direction_z\n",
+      " Rechunked source_filenames\n",
+      "Rechunked angle_offset_alongshipbeam_direction_y\n",
+      "\n",
+      "Rechunked filenames\n",
+      "RechunkedRechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      " angle_offset_athwartship\n",
+      "Rechunked Rechunked backscatter_r\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunkedangle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      " equivalent_beam_angle\n",
+      "Rechunked Rechunkedequivalent_beam_angle\n",
+      " gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processingRechunked gain_correction\n",
+      "Rechunked \n",
+      "Rechunked channel\n",
+      "gpt_software_version\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "RechunkedRechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      " transmit_duration_nominal\n",
+      "RechunkedRechunked transmit_power\n",
+      " channel\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked ping_time\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked data_type\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked channel_mode\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked beam_direction_x\n",
+      "RechunkedRechunked backscatter_r\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      " beamwidth_twoway_athwartshipRechunked\n",
+      "Rechunked beam_direction_x\n",
+      " angle_offset_athwartship\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "RechunkedRechunked beam_direction_y\n",
+      " gain_correction\n",
+      "opened raw\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked beam_direction_z\n",
+      "RechunkedRechunked transmit_frequency_stop angle_offset_alongship\n",
+      "\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisationRechunked frequency_nominal\n",
+      "Rechunked angle_offset_athwartship\n",
+      "\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked sa_correction\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked gain_correction\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length\n",
+      "Rechunked Rechunked channel\n",
+      "RechunkedRechunked angle_athwartship\n",
+      " pulse_length_bin\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "ping_timeRechunked\n",
+      " equivalent_beam_angle\n",
+      "RechunkedRechunked  gain_correction\n",
+      "range_sampleRechunked angle_athwartship\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked\n",
+      " transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked Rechunked sample_interval\n",
+      "channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_power\n",
+      "RechunkedRechunked data_type\n",
+      " transmit_duration_nominal\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked transmit_power\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked data_typeRechunked channel_mode\n",
+      "\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked backscatter_r\n",
+      "opened raw\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked latitude\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked Rechunked NMEA_datagram\n",
+      "channel\n",
+      "RechunkedRechunked pulse_length_bin\n",
+      " time1\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "RechunkedRechunked  frequency_nominal\n",
+      "backscatter_r\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked Rechunked angle_athwartship\n",
+      "transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked backscatter_r\n",
+      "opened raw\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "RechunkedRechunked NMEA_datagram sound_speed_indicative\n",
+      "\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked time1\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_xRechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "opened raw\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked channel_mode\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked source_filenames\n",
+      "RechunkedRechunked filenames\n",
+      " position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked backscatter_r\n",
+      "opened raw\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_zRechunked angle_athwartship\n",
+      "Rechunked channel\n",
+      "opened raw\n",
+      "\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "RechunkedRechunked angle_offset_alongship\n",
+      " sound_speed_indicative\n",
+      "Rechunked Rechunked angle_offset_athwartshipfrequency_nominal\n",
+      "\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked gain_correction\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked Rechunked sample_interval\n",
+      "latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked sentence_type\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked pitch\n",
+      "Rechunked transmit_power\n",
+      "Rechunked roll\n",
+      "RechunkedRechunked vertical_offset\n",
+      " data_type\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked latitude\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked longitude\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked sentence_type\n",
+      "Rechunked channel_mode\n",
+      "RechunkedRechunked pitch\n",
+      " position_offset_y\n",
+      "Rechunked roll\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked time1\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked time2\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked transducer_offset_y\n",
+      "opened raw\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked Rechunked time1\n",
+      "frequency_nominal\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "opened raw\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked channel\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked time1\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked sa_correction\n",
+      "Rechunked channel\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked gain_correction\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked pulse_length\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked Rechunked NMEA_datagram\n",
+      "position_offset_x\n",
+      "RechunkedRechunked frequency_nominal\n",
+      "Rechunked time1\n",
+      " position_offset_y\n",
+      "Rechunked beam_type\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked time1\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "RechunkedRechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      " time2\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked channel\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked Rechunked beam_stabilisation\n",
+      "frequency_nominal\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked Rechunked beam_type\n",
+      "equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked NMEA_datagramRechunked beam_direction_y\n",
+      "Rechunked ping_time\n",
+      "Rechunked Rechunked range_sample\n",
+      "\n",
+      "beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked sample_interval\n",
+      "Rechunked time1\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked transmit_bandwidth\n",
+      "RechunkedRechunked angle_sensitivity_alongship angle_alongship\n",
+      "\n",
+      "Rechunked transmit_duration_nominal\n",
+      "RechunkedRechunked transmit_power\n",
+      " angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked gain_correction\n",
+      "Rechunked channel_mode\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked channel_mode\n",
+      "Rechunked frequency_nominal\n",
+      "RechunkedRechunked angle_athwartship\n",
+      " beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "RechunkedRechunked  beam_group_descr\n",
+      "beam_stabilisation\n",
+      "Rechunked beam_group\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked sa_correction\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked gain_correction\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked pulse_length\n",
+      "Rechunked gain_correction\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked angle_athwartshipRechunked pulse_length_bin\n",
+      "\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_alongship\n",
+      "opened raw\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_group\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "opened raw\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "RechunkedRechunked backscatter_r\n",
+      " angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked channel\n",
+      "opened raw\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked channel\n",
+      "Rechunked time1\n",
+      "Rechunked absorption_indicative\n",
+      "Rechunked sound_speed_indicative\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked latitude\n",
+      "Rechunked longitude\n",
+      "Rechunked sentence_type\n",
+      "Rechunked pitch\n",
+      "Rechunked roll\n",
+      "Rechunked vertical_offset\n",
+      "Rechunked water_level\n",
+      "Rechunked MRU_offset_x\n",
+      "Rechunked MRU_offset_y\n",
+      "Rechunked MRU_offset_z\n",
+      "Rechunked MRU_rotation_x\n",
+      "Rechunked MRU_rotation_y\n",
+      "Rechunked MRU_rotation_z\n",
+      "Rechunked position_offset_x\n",
+      "Rechunked position_offset_y\n",
+      "Rechunked position_offset_z\n",
+      "Rechunked time1\n",
+      "Rechunked time2\n",
+      "Rechunked channel\n",
+      "Rechunked transducer_offset_x\n",
+      "Rechunked transducer_offset_y\n",
+      "Rechunked transducer_offset_z\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked NMEA_datagram\n",
+      "Rechunked time1\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked source_filenames\n",
+      "Rechunked filenames\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked beam_group_descr\n",
+      "Rechunked beam_group\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "Rechunked beam_direction_x\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked gain_correction\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked frequency_nominal\n",
+      "RechunkedRechunked sample_interval\n",
+      " beam_type\n",
+      "Rechunked beamwidth_twoway_alongship\n",
+      "Rechunked beamwidth_twoway_athwartship\n",
+      "RechunkedRechunked beam_direction_x\n",
+      " transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked beam_direction_y\n",
+      "Rechunked beam_direction_z\n",
+      "Rechunked angle_offset_alongship\n",
+      "Rechunked angle_offset_athwartship\n",
+      "Rechunked transmit_powerRechunked\n",
+      " angle_sensitivity_alongship\n",
+      "Rechunked angle_sensitivity_athwartship\n",
+      "Rechunked equivalent_beam_angle\n",
+      "Rechunked Rechunked gain_correction\n",
+      "data_type\n",
+      "Rechunked gpt_software_version\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked transmit_frequency_start\n",
+      "Rechunked channel_mode\n",
+      "Rechunked transmit_frequency_stop\n",
+      "Rechunked transmit_type\n",
+      "Rechunked beam_stabilisation\n",
+      "Rechunked non_quantitative_processing\n",
+      "Rechunked channel\n",
+      "Rechunked ping_time\n",
+      "Rechunked range_sample\n",
+      "Rechunked sample_interval\n",
+      "Rechunked transmit_bandwidth\n",
+      "Rechunked transmit_duration_nominal\n",
+      "Rechunked transmit_power\n",
+      "Rechunked data_type\n",
+      "Rechunked range_sample_offset\n",
+      "Rechunked channel_mode\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked backscatter_r\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked angle_athwartship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked angle_alongship\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n",
+      "Rechunked frequency_nominal\n",
+      "Rechunked sa_correction\n",
+      "Rechunked gain_correction\n",
+      "Rechunked pulse_length\n",
+      "Rechunked channel\n",
+      "Rechunked pulse_length_bin\n"
+     ]
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None,\n",
+       " None]"
+      ]
+     },
+     "execution_count": 7,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "convert_files(fpaths)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "cluster.close()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "echopype",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/hake.yaml b/notebooks/hake.yaml
deleted file mode 100644
index 941e122..0000000
--- a/notebooks/hake.yaml
+++ /dev/null
@@ -1,19 +0,0 @@
-name: Bell_M._Shimada-SH1707-EK60
-sonar_model: EK60
-raw_regex: (.*)-?D(?P<date>\w{1,8})-T(?P<time>\w{1,6})
-args:
-  urlpath: s3://ncei-wcsd-archive/data/raw/{{ ship_name }}/{{ survey_name }}/{{ sonar_model }}/*.raw
-  # Set default parameter values as found in urlpath
-  parameters:
-    ship_name: Bell_M._Shimada
-    survey_name: SH1707
-    sonar_model: EK60
-  storage_options:
-    anon: true
-  transect:
-    # Transect file spec
-    # can be either single or multiple files
-    file: ./hake_transects_2017.zip
-output:
-  urlpath: ./combined_files
-  overwrite: true
\ No newline at end of file
diff --git a/notebooks/hake_transects_2017_original.zip b/notebooks/hake_transects_2017_original.zip
new file mode 100644
index 0000000..e3a7232
Binary files /dev/null and b/notebooks/hake_transects_2017_original.zip differ
diff --git a/notebooks/hake_transects_2017_original/x0007_fileset.txt b/notebooks/hake_transects_2017_original/x0007_fileset.txt
new file mode 100644
index 0000000..23071e6
--- /dev/null
+++ b/notebooks/hake_transects_2017_original/x0007_fileset.txt
@@ -0,0 +1,22 @@
+Summer2017-D20170627-T171516.raw
+Summer2017-D20170627-T174838.raw
+Summer2017-D20170627-T182247.raw
+Summer2017-D20170627-T185656.raw
+Summer2017-D20170627-T193105.raw
+Summer2017-D20170627-T200518.raw
+Summer2017-D20170627-T203931.raw
+Summer2017-D20170627-T211247.raw
+Summer2017-D20170627-T213837.raw
+Summer2017-D20170627-T220425.raw
+Summer2017-D20170627-T223013.raw
+Summer2017-D20170627-T225602.raw
+Summer2017-D20170627-T232152.raw
+Summer2017-D20170627-T234752.raw
+Summer2017-D20170628-T001354.raw
+Summer2017-D20170628-T003944.raw
+Summer2017-D20170628-T010526.raw
+Summer2017-D20170628-T013108.raw
+Summer2017-D20170628-T015659.raw
+Summer2017-D20170628-T022249.raw
+Summer2017-D20170628-T024842.raw
+Summer2017-D20170628-T031428.raw
\ No newline at end of file
diff --git a/notebooks/hake_transects_2017_original/x0010_fileset.txt b/notebooks/hake_transects_2017_original/x0010_fileset.txt
new file mode 100644
index 0000000..b8847a1
--- /dev/null
+++ b/notebooks/hake_transects_2017_original/x0010_fileset.txt
@@ -0,0 +1,11 @@
+Summer2017-D20170628-T184837.raw
+Summer2017-D20170628-T192246.raw
+Summer2017-D20170628-T195655.raw
+Summer2017-D20170628-T203107.raw
+Summer2017-D20170628-T210519.raw
+Summer2017-D20170628-T213928.raw
+Summer2017-D20170628-T221341.raw
+Summer2017-D20170628-T224409.raw
+Summer2017-D20170628-T230959.raw
+Summer2017-D20170628-T233549.raw
+Summer2017-D20170629-T000153.raw
\ No newline at end of file
diff --git a/notebooks/logging_demo.ipynb b/notebooks/logging_demo.ipynb
new file mode 100644
index 0000000..a5cda5a
--- /dev/null
+++ b/notebooks/logging_demo.ipynb
@@ -0,0 +1,207 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": 1,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import yaml\n",
+    "import logging\n",
+    "from prefect import flow, task\n",
+    "from typing import Union, Dict"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 2,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "class Singleton_Echoflow:\n",
+    "    _instance: \"Singleton_Echoflow\" = None\n",
+    "    logger: logging.Logger = None\n",
+    "\n",
+    "    def __new__(\n",
+    "        cls,\n",
+    "        log_file: Union[Dict[str, str], str] = None,\n",
+    "    ) -> \"Singleton_Echoflow\":\n",
+    "        \n",
+    "        if cls._instance is None:\n",
+    "            cls._instance = super().__new__(cls)\n",
+    "            if log_file is not None:\n",
+    "                cls._instance.logger = cls._instance.logger_init(log_file)\n",
+    "        return cls._instance\n",
+    "\n",
+    "    @classmethod\n",
+    "    def get_instance(self) -> \"Singleton_Echoflow\":\n",
+    "        return self._instance\n",
+    "    \n",
+    "    @classmethod\n",
+    "    def get_logger(self) -> \"logging.Logger\":\n",
+    "        return self.logger\n",
+    "\n",
+    "    def logger_init(self, log_file: Union[Dict[str, str], str]) -> None:\n",
+    "        if type(log_file) == str:\n",
+    "            with open(log_file, \"r\") as file:\n",
+    "                logging_config_dict = yaml.safe_load(file.read())\n",
+    "                logging.config.dictConfig(logging_config_dict)\n",
+    "        else:\n",
+    "            logging.config.dictConfig(log_file)\n",
+    "        print(\"logging configured\")\n",
+    "        return logging.getLogger(\"echoflow\")\n",
+    "\n",
+    "    def log(self, msg, level, extra):\n",
+    "        self.logger.log(level=logging.DEBUG, msg=msg, extra=extra)\n",
+    "        self.logger.log(level=logging.ERROR, msg=msg, extra=extra)\n",
+    "        self.logger.log(level=logging.INFO, msg=msg, extra=extra)\n",
+    "        self.logger.log(level=logging.WARNING, msg=msg, extra=extra)\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 3,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "@flow(name=\"flow1\")\n",
+    "def flow1():\n",
+    "    print(Singleton_Echoflow(log_file='../echoflow/config/logging_config.yaml'))\n",
+    "    print(\"logging done\")\n",
+    "    flow2()\n",
+    "\n",
+    "@flow(name=\"flow2\")\n",
+    "def flow2():\n",
+    "    gea = Singleton_Echoflow.get_instance()\n",
+    "    gea.log('Demo Message', level=\"DEBUG\", extra={\"mod_name\": \"loggingDemo\", \"func_name\": __name__})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 4,
+   "metadata": {},
+   "outputs": [
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">14:20:02.661 | <span style=\"color: #008080; text-decoration-color: #008080\">INFO</span>    | prefect.engine - Created flow run<span style=\"color: #800080; text-decoration-color: #800080\"> 'aquamarine-muskox'</span> for flow<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\"> 'flow1'</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "14:20:02.661 | \u001b[36mINFO\u001b[0m    | prefect.engine - Created flow run\u001b[35m 'aquamarine-muskox'\u001b[0m for flow\u001b[1;35m 'flow1'\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stdout",
+     "output_type": "stream",
+     "text": [
+      "logging configured\n",
+      "<__main__.Singleton_Echoflow object at 0x7f850621fe50>\n",
+      "logging done\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">14:20:03.007 | <span style=\"color: #008080; text-decoration-color: #008080\">INFO</span>    | Flow run<span style=\"color: #800080; text-decoration-color: #800080\"> 'aquamarine-muskox'</span> - Created subflow run<span style=\"color: #800080; text-decoration-color: #800080\"> 'amphibian-jellyfish'</span> for flow<span style=\"color: #800080; text-decoration-color: #800080; font-weight: bold\"> 'flow2'</span>\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "14:20:03.007 | \u001b[36mINFO\u001b[0m    | Flow run\u001b[35m 'aquamarine-muskox'\u001b[0m - Created subflow run\u001b[35m 'amphibian-jellyfish'\u001b[0m for flow\u001b[1;35m 'flow2'\u001b[0m\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "name": "stderr",
+     "output_type": "stream",
+     "text": [
+      "/Users/soham/anaconda3/envs/echoflow/lib/python3.10/site-packages/watchtower/__init__.py:429: WatchtowerWarning: Received message after logging system shutdown\n",
+      "  warnings.warn(\"Received message after logging system shutdown\", WatchtowerWarning)\n"
+     ]
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">14:20:03.094 | <span style=\"color: #008080; text-decoration-color: #008080\">INFO</span>    | Flow run<span style=\"color: #800080; text-decoration-color: #800080\"> 'amphibian-jellyfish'</span> - Finished in state <span style=\"color: #008000; text-decoration-color: #008000\">Completed</span>()\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "14:20:03.094 | \u001b[36mINFO\u001b[0m    | Flow run\u001b[35m 'amphibian-jellyfish'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m()\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/html": [
+       "<pre style=\"white-space:pre;overflow-x:auto;line-height:normal;font-family:Menlo,'DejaVu Sans Mono',consolas,'Courier New',monospace\">14:20:03.121 | <span style=\"color: #008080; text-decoration-color: #008080\">INFO</span>    | Flow run<span style=\"color: #800080; text-decoration-color: #800080\"> 'aquamarine-muskox'</span> - Finished in state <span style=\"color: #008000; text-decoration-color: #008000\">Completed</span>('All states completed.')\n",
+       "</pre>\n"
+      ],
+      "text/plain": [
+       "14:20:03.121 | \u001b[36mINFO\u001b[0m    | Flow run\u001b[35m 'aquamarine-muskox'\u001b[0m - Finished in state \u001b[32mCompleted\u001b[0m('All states completed.')\n"
+      ]
+     },
+     "metadata": {},
+     "output_type": "display_data"
+    },
+    {
+     "data": {
+      "text/plain": [
+       "[Completed(message=None, type=COMPLETED, result=UnpersistedResult(type='unpersisted', artifact_type='result', artifact_description='Unpersisted result of type `NoneType`'))]"
+      ]
+     },
+     "execution_count": 4,
+     "metadata": {},
+     "output_type": "execute_result"
+    }
+   ],
+   "source": [
+    "flow1()"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": 5,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "logging.getLogger(\"echoflow\").error(\"Demo Debug Message\", extra={\"mod_name\": \"loggingDemo\", \"func_name\": __name__})"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "echoflow",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/s3_demo.ipynb b/notebooks/s3_demo.ipynb
new file mode 100644
index 0000000..38a0f2a
--- /dev/null
+++ b/notebooks/s3_demo.ipynb
@@ -0,0 +1,177 @@
+{
+ "cells": [
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import boto3\n",
+    "import fsspec\n",
+    "import s3fs\n",
+    "from typing import Dict, Any, Tuple, Union\n",
+    "from urllib.parse import urlparse\n",
+    "\n",
+    "import sys\n",
+    "import os\n",
+    "sys.path.append('/Users/Soham/Desktop/EchoWorkSpace/echoflow')\n",
+    "pipeline_config = os.path.abspath(\"../echoflow/config/pipeline.yaml\")\n",
+    "from echoflow.stages_v2.utils.file_utils import extract_fs\n",
+    "from echoflow.stages_v2.utils.config_utils import glob_url"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "options = {\"key\": \"\", \"secret\":\"\"}\n",
+    "url = \"s3://echoflow-workground/x0007_fileset.txt\"\n",
+    "fs = extract_fs(url, dict(options))"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ls_res = fs.ls(path=\"s3://echoflow-workground\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "ls_res"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "storage_options = {}\n",
+    "path = \"s3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/*.raw\"\n",
+    "parsed_path = urlparse(path)\n",
+    "file_system = fsspec.filesystem(parsed_path.scheme, **storage_options)\n",
+    "file_system.anon"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "file_system.ls(path=\"s3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "def isFile(file_path: str):\n",
+    "    fs = extract_fs(file_path)\n",
+    "    if file_path.endswith(\".zarr\"):\n",
+    "        return fs.isdir(file_path)\n",
+    "    return fs.isfile(file_path)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "folder_name = \"s3://echoflow-workground/combined_files\"\n",
+    "file = \"echoflow-workground/combined_files/echoflow_open_raw/Summer2017-D20170627-T174838.raw\"\n",
+    "fsmap = fsspec.get_mapper(folder_name)"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "urlpath = \"s3://ncei-wcsd-archive/data/raw/Bell_M._Shimada/SH1707/EK60/Summer2017-D20170627-T174838.raw\"\n",
+    "file_system = extract_fs(urlpath)\n",
+    "with file_system.open(urlpath, 'rb') as source_file:\n",
+    "    with fsmap.fs.open(file, mode=\"wb\") as f:\n",
+    "        f.write(source_file.read())\n",
+    "        outp = f.full_name"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "newmapper = fsspec.get_mapper(fsmap.root+\"echoflow_open_raw/Summer2017-D20170627-T174838.raw\")"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "import boto3\n",
+    "\n",
+    "# Create a Boto3 S3 client\n",
+    "s3_client = boto3.client(\"s3\")\n",
+    "\n",
+    "# List all S3 buckets\n",
+    "response = s3_client.list_buckets()\n",
+    "\n",
+    "# Print bucket names\n",
+    "for bucket in response[\"Buckets\"]:\n",
+    "    print(bucket[\"Name\"])\n"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": []
+  }
+ ],
+ "metadata": {
+  "kernelspec": {
+   "display_name": "echoflow",
+   "language": "python",
+   "name": "python3"
+  },
+  "language_info": {
+   "codemirror_mode": {
+    "name": "ipython",
+    "version": 3
+   },
+   "file_extension": ".py",
+   "mimetype": "text/x-python",
+   "name": "python",
+   "nbconvert_exporter": "python",
+   "pygments_lexer": "ipython3",
+   "version": "3.10.12"
+  },
+  "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}
diff --git a/notebooks/x0007_fileset.txt b/notebooks/x0007_fileset.txt
new file mode 100644
index 0000000..3d21bf6
--- /dev/null
+++ b/notebooks/x0007_fileset.txt
@@ -0,0 +1,2 @@
+Summer2017-D20170627-T171516.raw
+Summer2017-D20170627-T174838.raw
\ No newline at end of file
diff --git a/setup.cfg b/setup.cfg
index 4645d5d..05536c2 100644
--- a/setup.cfg
+++ b/setup.cfg
@@ -33,10 +33,15 @@ install_requires =
     jinja2
     prefect-dask
     pydantic
-python_requires = >=3.8
+    prefect-aws
+    prefect-azure
+    prefect-dask
+    jupyter
+python_requires = >=3.8,<3.11
 setup_requires =
     setuptools_scm
 
+
 [options.extras_require]
 test =
     pytest
@@ -64,3 +69,7 @@ all =
     %(test)s
     %(lint)s
     %(docs)s
+
+[options.entry_points]
+console_scripts =
+    echoflow = echoflow.echoflow_cli:main
\ No newline at end of file
diff --git a/setup.py b/setup.py
index 1abbd06..7f1a176 100644
--- a/setup.py
+++ b/setup.py
@@ -1,4 +1,4 @@
-import setuptools
+from setuptools import setup
 
 if __name__ == "__main__":
-    setuptools.setup()
+    setup()