Improvement suggestion to import Argo drifters data

[monocilindro]

Argo drifters provide the required data to compute a vertical sound speed profile (T,S,P) , usually as txt or NetCDF files. (http://www.jcommops.org/board?t=argo ) That information could be used to extend or complement an onboard probe cast when surveying nearby a working Argo drifter.

[giumas]

I support this proposed improvement. @monocilindro, feel free to submit a PR with the code changes to support the Argo-generated files.

[msmith-ccom]

Hey @monocilindro , I am going to be taking a look into this enhancement.

Below is current research into issue. @giumas let me know if there is a preferred place to put this.

Implementation

There are currently two ways to integrate argo float functionality:

1. argopy python package designed to easily search for data.
2. Use ftp functionality

argopy

There is a python package written for interfacing with argo data (argopy). Looking at the package design, it appears built on the erddapy (the wrapper to the ERDDAP RESTful web services). A recent update to the erddapy package breaks argopy. The functionality can be restored by fixing the version to argopy=0.7.0 Additionally, the following package changes would be made to the environment:

Environment Changes

added / updated specs: - aiohttp[version='>=3.6.2'] - check-manifest - dask[version='>=2.9'] - erddapy[version='>=0.6'] - fsspec[version='>=0.7.4'] - gsw[version='>=3.3.1'] - ipython[version='>=5.0.0'] - netcdf4[version='>=1.3.1'] - numpydoc[version='>=0.6.0'] - pytest - scikit-learn[version='>=0.21'] - scipy[version='>=1.1.0'] - setuptools_scm - toolz[version='>=0.8.2'] - tqdm[version='>=4.46.0'] - twine - wheel - xarray==0.15.1

Based on the significant changes to the env and the currently broken functionality, I would suggest not using this package.

FTP Protocal

There are two ftp data serving sites:

• US-Godae
• Coriolis GDAC

The two data servers host data in 3 particular groupings:

1. dac: sorts the data by Data Assembly Centre (DAC)
2. geo: sorts the data by ocean basin
3. latest_data: includes the most recent data

It looks like geo is the simplest to search through/build urls. An example link for all data measured in November is shown below:
ftp://ftp.ifremer.fr/ifremer/argo/geo/atlantic_ocean/2020/11/

A generic URL can look like
ftp://ftp.ifremer.fr/ifremer/argo/geo/ basin / year / month /

Note:

I currently am not sure how best to identify if a float was close to an area. In the top level of the ftp server there are a number of text files. I need to go through and determine if one maintains a master data/position list of the floats for easier serching.

Profiles

Argo User Manual
Profiles are stored in NetCDF format (CF-1.6).

Primary dimensions:
N_PROFILE = The number of distinct profiles contained within a data file
N_LEVELS = The maximum number of pressure levels contained in a profile.
N_CALIB = The maximum number of calibrations performed on a profile.

All data is available to generate a soundspeed profile using the gsw package.

• Pressure = decibar
• Temperature = ITS-90 scale degree Celsius
• Practical Salinity = PSU

Additional Information

@monocilindro , you mentioned in an email that you have done some testing regarding improving the vertical profiles taken on board. If you have the Database file for that project and can share it that would be helpful in determining applicability. If not, can you expand on that if possible regarding the following:

• Maximum positional separation between float and location of sound speed profile it compliments.
• Maximum temporal separation between float profile and sound speed profile.
• It would be nice to see how different the ray traces look when using an argo profile versus the WOA data.

@giumas - let me know what you think Giuseppe

[monocilindro]

Hi again, @zxcvbnmike15 and @giumas. Currently, I have no access to the database used for this research some months ago but I can share some other data regarding an ongoing survey near the Canary Islands. I am attaching a zip file containing the following data:

• Vertical sound speed profile cast in POS 28.75 N - 016.3 W at 01:00 GMT (txt file)
• ARGO float data from ID 6901261, cycle 047 (netCDF and csv formats)
• ARGO float data from ID 6901143, cycle 291 (netCDF and csv formats)

Those are the closest real measures of T,S, and P both in time ( up to 4 days) and distance (up to 120 km) to the SVP probe cast onboard.

I hope this helps

SSM_data.zip

[giumas]

After a first look and based on @zxcvbnmike15's info, I would suggest to search in the latest_data folder on the FTP site.
Based on this page, the prepended 'D' means 'delayed', thus I would first implement using the real time ('R').

Giving a look at the content of a .nc file (e.g., 'R20201108_prof_0.nc'), the distance search should be quite easy using the LATITUDE and LONGITUDE variables.

Once identified the index of a close-enough profile (the file in the example has 'only' 358 profiles), that index should be used to access:

• PRES_ADJUSTED
• PSAL_ADJUSTED
• TEMP_ADJUSTED

[msmith-ccom]

@giumas agreed that latest data should be used. I mistakenly thought each file (regardless of source directory) contained only a single float's data.
latest_data files are sorted by data and include data from a variety of floats.
I would like to check and see what improvement is offered using the float data versus WOA before implementing in SSM source code.

[giumas]

@zxcvbnmike15 what is the current status of this issue?

[giumas]

@msmith-ccom, can I unassign this old ticket from you?