Updating file format specifications

[ebranlard]

Is your feature request related to a problem? Please describe.
Input files from openfast lack some strict "format specifications", the main issue being tables, "outlist" and comments.

Describe the solution you'd like
Transitioning to an existing file format such as yaml might require some important efforts, so I'm suggesting that we come up with some small adjustements of the current inputs files, to reach a level where we follow some fixed format specifications.

The main benefits would be:

• Uniformization of our "reader" code, in particular for tables
• Ease of implementation of scripts to read/write inputs files, without having to know about "table" locations, and openfast version.

Here's the format specifications I'm suggesting:

• First line, starts with something like #fileheader type:openfast format_version:3 (everything case incensitive, format version 2 corresponds to the current specification, version 3 to the ones I'm suggesting)

• Comments are indicated after the following markers: ['!', '#', '%', '---'] (the NWTC library uses ['!#%'] for some functions. I'm not sure we should keep '%' or '#', but '---' at beginning of line is convenient for existing files)

• Lines with Value/Key pairs follow the format: Value Key Comment (space separated, Comment does not require a comment character)

• Lines with ValueList /Key follow the format: Value1 , Value2, Value3 Key Comment (values should now be comma separated, the key follows the last item, Comment does not require a comment character)

• Tables are indicated with a line of the following form: #table key:value key2:value2 .. where key is in ['name','col','row','header','format'] (arbitrary order, case incensitive, space separated, and the only required parameter is row, the programmer may hardcode/override the other keys). The key format is a string indicating the types of each column using one character, where I=int , F=Float, B=boolean, S=String.
  The following lines are the header lines (length given by the header key), then the table lines. Arbitrary text may be placed at the end of the columns. After reading, the programmer will receive an array for each data type as optional arguments.

• Outlist is indicated with a line starting with #outlist (case incensitive). Arbitrary text may follow on that line. The remaining lines until the keyword END are the output channels.

Here's an tentative example of input file:

#fileheader type:openfast format_version:3
!A comment1
%A comment2
#A comment3
---- comment 4
-------------------
#table name:MyTable row:2 col:3     header:2     format:ISF   ignored comment
 Col1       Col2     Col3    ignored comment
(Unit1)   (Unit2)    (unit3)  ignored comment
   0         "a"      3.4     ignored comment
   1         "b"      2       ignored comment
-------------------
#table name:MyOtherTable row:3    ! A table where the programmer hardcoded the "keys"
 "value1"  
 "value2"  
 "value3"  
-------------------
0.0         SomeKey       ignored text
0.0, 1.0    SomeKeyList   ignored text
-------------------
#outlist   Ignored comment
"RotSpeed, RotPower"
"GenSpeed"
END

I've written these specifications such that limited changes would be required to the input files (and to the code) to switch to this format. I believe it would support the functionalities of most input files.
I've purposely kept it a bit "loose", with comment characters not required in some cases, but we can easily require this. Fortran also tends to treat space or comma separated fields similarly, but we can progressively make our reading functions more strict.

Let me know your thoughts, I'm happy to change any of these specifications based on your comments.

[andrew-platt]

Do you think we can allow arbitrary numbers of comment lines in separation sections?

And perhaps randomly placed comment lines within a value/key list?

I kind of like keeping the # as a comment character, but it makes more sense to appropriate it as a special character for delineating tables or sections.

[jjonkman]

I've talked with Emmanuel about this and like the proposal. With just a small changes to the existing input file formats, it will be much easier to develop scripts to read and write all input files, including for converting a model from one version of OpenFAST to another, and for generating all of the input files needed for a loads analysis or other batch analysis based on a master set of input files.

[andrew-platt]

I was looking into the ProcessComFile routine that is used in the AirFoilInfo.f90 code. This strips all the comment lines from the input file, and puts everything into memory in the FileInfoType structure. This may be a very convenient starting point with all non-comment character starting lines held in memory. This could make it much simpler to do the file parsing once read into memory. I'm in the process of implementing this in the f/LidarSim branch.

One really nice feature of it is that it provides the line number for each line of each file held in memory (this supports nested files with the @filename method). That makes it possible to tell the end user which line was an issue.

[ebranlard]

Sounds good to me to expand on ProcessComFile.

As for comment characters, I don't have a strong opinion regarding ! and #, nor for the choice of the special character #.

[ebranlard]

I've added a toy example/test here:

https://github.com/ebranlard/openfast/tree/f/inputformat/test-input