Musings: Parameter DataType Engine Design

[unityconstruct]

Parameter DataType Engine Design

MIDI_COMM_BITS	MIDI_PACKING_STRATEGY_TYPE	MIDI_PACKING_STATEGY_PARAM_COUNT
7	A	1
7	A	2
7	A	3
7	A	4
7	A	5

PARAM_ID	PARAM_DATA_TYPE	PARAM_VALUE_MIN	PARAM_VALUE_MAX
VOICE1_FILTER_CUTOFF	PARAM_128VALUE_-64_63	-64	63
VOICE1_FILTER_RESONANCE	PARAM_128VALUE_0_127	0	127
VOICE1_FILTER_TYPE	PARAM_128VALUE_1_128	1	128

--- Enum for the Parameter's  Real-World MIN/MAX values
MidiParameterDataType = {
    "PARAM_128VALUE_-64_63",
    "PARAM_128VALUE_0_127",
    "PARAM_128VALUE_1_128"
}

--- Enum for Hardwards Packing Logic type
PackingStrategy = {
    "BIT7_MSB_LSB",
    "BIT7_LSB_MSB",
    "BIT7_LSB_MSB_1_PARAMS",
    "BIT7_LSB_MSB_2_PARAMS",
    "BIT7_LSB_MSB_3_PARAMS",
    "BIT7_LSB_MSB_4_PARAMS"
}

--- Enum for Comm Direction
CtrlrMidiCommDirection = {
    "ToHardware",
    "ToCtrlr"
}

--- MidiParameter Object
CtrlrMidiParameter = {
    PARAM_NAME = "DEFAULT",
    PARAM_ID = "2580",
    PARAM_DATA_TYPE = MidiParameterDataType.PARAM_128VALUE_0_127,
    PARAM_VALUE = 15,
}

--- MidiMessage Package containing MidiParameterObjects
CtrlrMidiParameterPackage = {
    Parameters = CtrlrMidiParameter{},
    PackingStrategyToHardware = PackingStrategy.BIT7_MSB_LSB,
    PackingStrategyToCtrlr = PackingStrategy.BIT7_LSB_MSB_3_PARAMS
}


--[[
    DO THE CONVERSION..!! WORK THAT MAGIC!!! I DON'T CARE WHAT IT DOES!! I GAVE IT CORRECT VALUES!!!
--]]

---Convert Parameters to a MidiMessage using a CtrlrMidiParameterPackage containing one or more parameters and a comm direction
---function will call needed conversion function based on the Package's PackagingStrategy, direction, number & datatypes of the parameter
---@param package CtrlrMidiParameterPackage
---@param direction CtrlrMidiCommDirection
function CtrlrMidiPackagerConvert( package, direction )
    -- call MidiParameterPackerPrep(package) -- to get PackingStrategy & direction
    -- foreach CtrlrMidiParameter in package -- to translate Parameter VALUE to its MidiParameterDataType used for communication
      -- paramHexValues[i] = call MidiParameterPacker(package.Parameters.Count)
    -- end
    -- midiPayload = call MidiPackage(paramHexValues)
    -- call MidiSend(midiPayload)
end

-- CtrlrMidiParameterDataTypeConversionType = {
--     PARAM_DATA_TYPE = "", 
--     MIDI_PACKING_STRATEGY_TYPE = CtrlrMidiParameterPackage.PackingStrategyToCtrlr
-- }

-- MidiParamterMinMaxType = {
--     "MIN__64_MAX63",
--     "MIN0_MAX127,
--     "MIN1_MAX128"

-- }

[Godlike-Productions]

Thanks for the efforts here. It's to be applauded, though its a big job.

I'll add a few others that I've come across.

MIDI Packing. Another common one is nibblized data (2 bytes with 4 bits each) to give 256 values. There are different ways to nibblize data but usually it's the last 4 bits of each byte that are important, but the order often changes.

Another is using the 127 values for large data ranges, normally program numbers. There are a number of ways that this is done. Some use the first 100 values, and then a bank control (CC0/32 for bank control and then program change is the most common one), but also sometimes it is split into banks of 128.

Also, particularly with older synths, bitmaps are used where each bit of byte are used as switches. These can be challenging to set as bit flipping isn't supported well in Lua 5.1 (it is better with 5.2)

In terms of control schemes there are really 4 that I'm aware of.

The first is raw MIDI CC mapped to a synth parameter. These are the easiest. There is a little bit of standardization as to what CC controls what, but it is ignored as much as it is implemented. Generally MIDI CC's 0 to 11, 32, 64 and 65 are standard across synths.

Next is NRPM, where a NRPM MIDI CC pair opens up a second pair of MIDI CC's to give a much larger number of parameters that can be accessed, and with 2 byte precision. There are RPM's such as Pitchbend as well, that are kind of a subset of these. These give 14 bit precision and 14 bit's of parameter selection. Again, relatively easy to implement because they are all based on MIDI CC's. I think the first 2 MIDI CC's are standardized, but I think there are 2 sets of MIDI CC options for the second pair, as well as the option to use Data (MIDI CC6) as a value byte.

One other thing to note about MIDI CC's is that for limited parameter selections (for example LFO shape), some manufacturers use the first numbers starting at 0, some use ranges, some use numbers somewhere in the middle, and I've even seen examples where negative numbers are used by lowering to a lower controller number and using the highest numbers (though usually these are where there are more than 127 possible values)

KPN, which stand for Kurzweil Parameter Numbers. These are kind of like a super NRPM that uses 7 consecutive MIDI CC's to access thousands and thousands of parameters, with up to 3 data bytes. These have to be sent in strict order, but they do support a kind of running message. This is particular to Kurzweil PC3's and later. I do have the XML files that document these, at least for the PC3, but the set of controls is massive. It is more efficient than SYSEX for the massive number of parameters available on these synths, but not everything is controlled with KPN's. SYSEX is also required for some elements, notably algorithm wiring.

Sysex, this is the most complex version and there really isn't a standard, as it is determined by the manufacturer for the model. SYSEX supports everything from patch dumps, to screen dumps, to parameter modification, to entire control schemes. Generally the control scheme is documented for a synth (though often in cryptic difficult to digest volumes at the end of manuals, or as a second manual that you needed to request from the manufacturer when the synth was still supported). Patch dumps are sometimes documented, generally for simple fixed architecture synths, but often they are not.

Sometimes there is hidden sysex that isn't documented, but still useful. It should be noted that there are 2 forms of sysex headers depending on the manufacturer, because the number of manufacturers grew larger than what was originally allowed for in the spec, so they just added more bytes to the header.

Finally there is a MIDI sample dump standard, which may be used to load sample data over MIDI. Slow, (unless it also supports SCSI, which is a thing), but sometimes the only way to get sample data to some synths.

Regarding envelopes, ADSR is by far the most common. Probably used by 80% of synths, but DADSR, DAHDSR, and multi segment envelopes are also used. The Kurzweil for example has a Impact parameter, 3 x Attack Stages, 3 x Delay stages, a Sustain stage and 3 x release stages, as well as a looping parameter, and to make it more difficult it also has ASR envelopes with optional hold as a second set of envelopes.

Occasionally a machine will support MIDI messages that don't adhere to the MIDI standard. Of note is the TR909, which uses a non standard message (f3 header if I remember correctly) for one of the switches. I can't remember which, but I did use it in a GTRC module I developed for it (to store a 909 snapshot). So we need to be able to maintain support for this, though honestly this is such a rare case, that LUA is probably the place for it.