4.0.10 - Nov 2024 Refresh

[sylvia-leaf]

SFe Version 4.0.10 (Draft Specification) - November 2024 Refresh

SFe is now static (for now)

This is the tenth draft milestone of the SFe specification version 4.

We're focusing on changes that help compatibility between different SFe players, by adding features such as extended version sub-chunks, feature flags and more. We're also streamlining the format by reducing the number of variants; this has recently been a point of contention.

Making SFe variants easier to understand

We've listened to feedback, and now we're going to reduce the number of variants of SFe.

Previously, there was SFe32, SFe64L and SFe64. Now, we've postponed the divergence of features between 32-bit and 64-bit formats to version 5.0. Now, there is one SFe specification for both 32-bit and 64-bit versions of the format, and the only current difference is the chunk header format. We also plan on adding a new "dynamic" chunk header designed by SFe Team member spessasus that can scale from 32-bit to 64-bit and beyond in a future version (likely 4.5).

It is called "dynamic" because the length of the chunk size in the header is dynamic and depends on another value. This allows the format to be much more scalable, and wastes less space when compared to standard RIFF64. We will also replace the sfbk fourcc with sfen so legacy SF programs don't accidentally attempt to run an SFe bank.

We've also added a section detailing LTS versions of SFe, which will continue getting feature updates even after structurally incompatible changes (with a higher major version) have been made. For example, when the structurally incompatible SFe 5 releases, it is expected that we maintain feature additions to SFe 4 for some time after. This should strike the perfect balance between supporting legacy players and getting features included quickly!

It may be possible to also abandon the SFty sub-chunk, making the proposition of selecting a variant of SFe even simpler. Even if the SFty sub-chunk is not abandoned, it will still be much simpler for everyone.

Introducing the SFvx sub-chunk

The SFvx sub-chunk includes extended version information about the SFe bank, including:

• specification version
  • differs from ifil version
• specification type
  • drafts, milestones and final versions are supported
• draft milestone (when applicable)
• full version tag (when applicable)

By including such a sub-chunk in the ISFe-info subchunk, we can solve the problem with providing more version information without breaking compatibility with legacy SF players. The ifil value is strictly defined as two WORD values in legacy SF.

Feature flags and the flag sub-chunk

After the SFvx sub-chunk is the flag sub-chunk. This implements feature flags, and is split into branches, leaves and flags. Generally, branches correspond to types of features, leaves correspond to specific features and flags represent the features themselves.

The idea is that a bank has feature flags that are determined by the features in use by samples/instruments/presets. The SFe editor will add the feature flags automatically. The list of feature flags is included in the compatibility specification for 4.0.10, but should be moved to the same document in the final specification.

When an SFe player reads the bank, it will recognise the feature flags that it supports, but if it detects an unrecognised flag, then the user is warned. The user is also warned when flags that are not available in the specification corresponding to the SFvx sub-chunk are found, as the presence of such flags may indicate proprietary extensions.

We also include a "private-use" area (branch 240 to 255), but as the SFe license is copyleft and strongly discourages unpublished proprietary extensions, features may be moved to the main area of flags.

Other versioning changes

Previous drafts have described the first version of SFe as version 4.00. Two zeros are used due to legacy SF using this in its naming conventions (for example, the versions written in SFSPEC21.PDF and SFSPEC24.PDF are 2.01 and 2.04 respectively).

However, some programs have used the terms 2.1 and 2.4 to describe legacy SF. Therefore, we've made the decision to remove the leading zeros. This means that version 4.00 becomes 4.0 and 4.01 becomes 4.1. This is easier to understand for those who are not familiar with the legacy versioning system.

This is also a good time to mention that there will not be illogical version jumping in SFe, unlike in legacy SF. For example, E-mu updated version 1.0 to 1.5 and then to 2.00. As expected, 2.00 was replaced by 2.01, but 2.02 and 2.03 were skipped and 2.04 was the final version ever released by E-mu. Meanwhile in SFe, version 4.0 will be replaced by 4.1 or 5.0, not 4.2, 4.3 or 4.4.

In the future, we'll refer to major versions as "episodes". This naming is used to make it a bit easier for us to describe the development process of SFe versions.

Removing the info-list subchunk length limits

These limits inherited from legacy SF don't make sense for modern computers. Therefore, we're removing them, allowing bank developers to have an unlimited length for information.

Fixing a few other issues

We've also fixed some issues, including an incorrect reference to a chunk that was intended for version 4.4 being listed in section 4 as being intended for version 4.5 and some references in sections 7.4 and 7.5 to features that are to be implemented in version 4.1. We are also rewriting the definitions of more fields that convey multiple values using different bits in a similar vein to the byBankMSB and byBankLSB changes.

License changes

To make the SFe license compliant with the Open Source Definition, we've removed the requirements to share all modifications implemented in programs under the SFe license (or a compatible license) or not removing the link to the latest version of the specification (for final versions). Instead, these are simply considered "good practice".

This new version of the license supersedes the previous version. All SFe data that has been released in the past is now available without the need to share all modifications or to retain the link to the latest version.

It is likely that the SFe program specification containing all "good practice" tips will instead include these requirements in the future.

We will write the full version of the license in time, but there are other things that we need to focus on.

More features for SFe in the future!

According to feedback from team member davy7125 we're adding a few features to the plan to versions beyond 4.0!

We've got a list of baseline features that SFe should include in first implementations:

• default modulators
  • already planned for 4.1
  • using the DMOD chunk
• initial value of MIDI control changes
  • listed as "not sure it is useful though"
  • may be included in 4.1
• sample storage
  • listed as solved by Werner SF3
• round-robin
  • will be added in 4.2
  • 4.2 will consist mostly of generator enum definitions
• independent envelopes/LFOs for pitch/attenuation/filter
  • will likely be added in 4.3 but may be moved forward to 4.2
• sample modes
  • likely to be added in 4.5
• MSB/LSB banks
  • will be added in 4.0
• 64-bit mode
  • static 64-bit SFe is included in the 4.0 specification
  • dynamic SFe is 64-bit by default and will be released in 5.0
• conversion between sfz/sf2
  • this is achieved using the abstraction level system
  • AL6, AL5 and AL4 are SFZ, AL2 is SFe and AL1 is SiliconSFe
  • basic specification for AL6, AL5 and AL4 will be ready for 4.6
  • full AL3 specification will be 4.7 or 5.0
  • during episode 4, we add DLS features, and during episode 5, SFZ features

The comment also includes other features that are less-critical and will be delayed for future versions. Here is a similar list:

• multiple loops at sample level - sometime in episode 6
• different kinds of filters - very likely to be 4.3
• modulator inputs - might be as early as 4.1 but it's unknown what this means
• conditional starts/key switches - 4.7, but if significant structural changes are required, during episode 5
• exclusive class normal release - likely 4.2
• field length limitations - 4.0 for SFe64 only
• negative attenuation - 4.0, its feature flag is already there
• real attenuation - 4.0, its feature flag is already there
• sm32 chunk - 4.0

About other future plans

So, at this point, we are almost ready to release the final version of SFe version 4.0. However, we need to communicate with program developers such as FluidSynth, so we can get an agreement of features to be implemented, as well as a final specification of Werner SF3 that can be included in SFe.

Ideally, this should be done before the release of Polyphone 3 (the first planned version of Polyphone with SFe used by default). We should also communicate with other SF editing and playback programs (for example the Swami editor and the Bassmidi player) to ensure that SFe does not simply become a "Polyphone and FluidSynth Custom Features" that suffers from the same problems as SFCF. Remember that one of the fundamental goals of SFe is to tackle the inevitable problem of "format wars".

Once we've created the near-finalised text of specification version 4.0 (this will appear as a draft milestone), then we can work on creating the logos and niceties so we can release a finalised specification. The finalised specification is likely to have a somewhat different structure to the draft specifications, which closely followed the structure of legacy SF2.04. Parts that are identical to legacy SF2.04 and are not planned to be changed will also not be included in the final specification. The final specification will also include one document with all the information, instead of split file, program and compatibility specifications.

We've merged versions 4.2 and 4.3 because version 4.2 was planned to only include the MIDI lyrics specification. Because such a feature is only part of the program specification, we've decided to merge it into format specification 4.3. Now, 4.2 will include the format specification changes originally planned for 4.3 along with the MIDI lyrics specification.

This also means that 4.4 and 4.5 are now renamed 4.3 and 4.4. Versions 4.5 and later are likely to still happen, with the end-goal of implementing all DLS features.

What's Changed

• 4.00.9b by @sylvia-leaf in 4.00.9b #27
• Merge pull request 4.00.9b #27 from SFe-Team-was-taken/main by @sylvia-leaf in Merge pull request #27 from SFe-Team-was-taken/main #31
• Concept for version 4.0.10 (merged SFe32 and SFe64!) by @sylvia-leaf in Concept for version 4.0.10 (merged SFe32 and SFe64!) #37

Full Changelog: 4.00.9...4.0.10

---

This discussion was created from the release 4.0.10 - Nov 2024 Refresh.