Refactoring style

[ohmtech-rdi]

Background

We have a current scheme where a user must add a style to a control in order to determine what is used.

For example:

• style rogan.1ps means use the Alpha 9mm pot with D shaft, and finally the rogan size 1, with pointer and skirt
• style thonk.pj398sm.knurled means use the thonkiconn with knurled nut
• style led.3mm.red means use a 3mm red LED
• style dailywell.2m1 means use 2-position on-on switch, but style dailywell.2m3 is 3-position on-off-on

We have all sort of behaviors:

• Some styles imply a component that doesn't appear in the name (like rogan.1ps)
• Some styles are most likely implicit, since they are the only options, or it doesn't really matter if another component is used as long as it provides the proper function (like thonk.pj398sm.knurled)
• Some styles repeat the type of the control, which is superfluous (like led.3mm.red)
• Some names are just confusing, as they don't make the function clear (like dailywell.2m1 and dailywell.2m3)

Improving problem definition

From a semantic point of view, and to help with the upcoming manufacturer feature, we change the style semantic so that it matches the user intent, for example:

• The user wants a red LED and doesn't really care about the size,
• The user want the small size 1 rogan with pointer and skirt, doesn't really care about what is on the PCB,
• The user want the CV nut to be knurled, and doesn't really care about what is on the PCB.

The user intent is mainly what the end-user will see and haptics, that they can experience in the simulator.

Finally this becomes crucial when bringing the manufacturer feature, since the user for example doesn't care if an Alpha or Bourns pot is used, and more importantly, if the manufacturer is going to use a THT or SMT version for a component.

Improving current solution

The way styles are done from a language point of view is somehow an ordered list of features, but there are no reason really to order those features: led.3mm.red could be also led.red.3mm.
For this, instead of having a style describe a constant, we could say that we have a list of constants instead, for which the style doesn't matter:

style led, 3mm, red
// same as
style led, red, 3mm

We can:

• Remove redundant information (like led)
• Make functions more apparent (like on_on for a 2-position switch)
• Have default fallbacks (like 3mm for a Led, or using thonkiconn for a CvIn), so that having no user-defined style at all is still ok.

control pot Pot {
   style rogan, size1, skirted
   style rogan, size1 // default pointer, no skirt, white
   style sifam, big, base_white, cap_white 
   // in all cases implies Alpha 9mm, for now
}

control enc Encoder {
   style rogan, size1 // default no pointer, no skirt, black
   // in all cases implies Bourns PEC11R, for now
}

control trim Trim {
   style tall // implies tall Songhuei 9mm
   // no style implies tall
}

control led Led {
   style 3mm, red
   style red, 3mm
   style red // implies 3mm)
   // no style, implies 3mm, red
}

control led_bi LedBi {
   style red_green // default if omitted
}

control button Button {
   style tl1105, black // tl1105 with black cap, default
   style ckd6, round, black // CKD6R black
}

control sw Switch {
   style on_off_on // implies dailywell.2ms3
}

control cv CvIn {
   style hex // default
   style knurled
}

The main problem that arises is probably getting to know the available style options for each control.
This can be done in the documentation, and also through hinting or helping with messages from the erbb command.

control freq Pot {
   style rohan, size3
}

$ erbb build hardware
error: unknown option 'rohan'
note: did you mean 'rogan'?
help: visit Pot style options:
   https://eurorack-blocks.readthedocs.io/en/latest/controls/Pot.html#style-property

Implementation

Because of the explosion of style options, we need to find a way to move them out of the grammar.
For this we can just create "constants", that are almost like identifiers, except they can start with a number.

We need also the Xcode syntax highlighter and Pygments lexer to recognise those constants.

Finally on one side we have a set of contants, and a list of set of options on the other side, ordered by more common first.
Matching 2 sets gives:

• An intersection set of matching options
• A set of options that the user want but can't be found (error or warning?)
• A set of options that are available but not specified by the user (implicit selected options)

For each potential final style we have a score:

• Number of matching options
• Number of unmatching options
• Order in the list

We do a "lexical" compare:
If 2 sets have the same number of matching options, then select the ones with the less unmatching options, and finally the one earliest in the order.

[ohmtech-rdi]

This new abstraction looks like it's going to work like a charm. Testing it on samples modules, it really feels like expressing an intent in a simpler way. I added also some help from the parser to detect mistakes and make suggestions, for example:

control freq Pot {
   style rohan, large
}

$ erbb build hardware
Drop.erbui:30:13: error: Unknown style 'rohan'
         style rohan, large
               ^~~~~
<builtin>: note: did you mean 'rogan'?
         style rogan, large
               ^~~~~

Or when an option is not compatible (for example thonk has only unskirted for size 2 and 1, not 3 ie. "large"):

control freq Pot {
   style rogan, large, unskirted
}

Drop.erbui:30:27: error: Incompatible style 'unskirted'
         style rogan, large, unskirted
                             ^~~~~~~~~
<builtin>: note: for selected styles 'size3, 3ps, rogan, skirt, d_shaft, large'

From an implementation point of view, we have a new analyser that computes a number of parts from the given style.
Then generators relie on parts rather than style, which makes thing much more precise and avoids all sorts of guesswork.

For one kind of control (for example a Pot) we already have the manufacturer abstraction built-in. This means that we could for example check the target manufacturer catalog, and if a style is not available, fallback to everything available for DIY, and emit a warning in that case (as this could lead to additional charges from the manufacturer).

[ohmtech-rdi]

• Refactor erbui control style #399 would be ready to merge

[ohmtech-rdi]

There is still a conceptual issue left.
When refactoring the code, some new concept appeared, from the list of parts in a Control:

• The order is important, and the first part should be the "main part" as use for some of the UI code generation for example
• We don't get to know what is sitting on the PCB and what are extras (like knobs)

Since it is still unclear if we could have the case that for example a list of parts contains multiple components that would go on the PCB or other usage dictated by the generators, we will assume the generator knows what it is doing with this list of parts.

The problem with this approach, is that generators will ignore parts in which they are not interested into, and this could create a problem if a new part is introduced and we forget to add the support code for this part in a specific generator.

Until we have more insight about this, we will leave the system as it is.

[ohmtech-rdi]

After quite some contorsions, semantic style is finally here 🎉

And by contorsions, we mean that quite a lot of successive small improvements have been needed to get there in the end:

• Refactor erbui front pcb #397
• Erbui front pcb load unique #410
• Erbui front pcb control annotate #411
• Erbui front pcb merge #403
• Use control kind instead of style for default label positioning #414
• Param range norm bipolar discrete #415
• Add erb tl1105 ckd6r and led 3mm generic rotate support #416
• Simulator generic rotate #417
• Erbui front pcb net based process #418
• Erbui front pcb decorations #419
• Erbui front pcb net bom #420
• Add missing 1rblk cap for tl1105 #421
• Erbui analyser style #422
• Erbui front panel dxf #423
• Erbui pcb based label positioning #424
• Erbui net based vcvrack code #425
• Erbui net based UI code #426
• Erbui remove legacy ast style #427
• Make bom generation generic by using a declarative bom format #428
• Regroup hardware build in a single erbui function #429
• Erbui builtin manufacturer diy wire manual #430
• Erbui semantic style #431
• Erbui use semantic style #432

One main problem is that generators needed too much semantic information about the components that are used. This is now centralized in a new style analyser.

The information itself (for the PCB, BOM, DXF, etc. generators) was coming from too many sources, so we decided to store as much data as possible directly in the Kicad generated files:

• The new generated net files allows us to have a key/value container, which is used for the BOM obviously, but also to store which C++ class should be used by the simulator,
• The kicad_pcb files have been enriched with everything related to the design, such as holes for drilling, or control's decoration visual bounds for automatic label placement.

The whole system as been made in particular to allow manufacturer for which the definition can't be opened for the public (for IP reasons).

Backward compatibility is present, so one might still use for example style led.3mm.red instead of just style red. Though the old style name will be deprecated with the introduction of new manufacturers.

Finally, the new system will also allow to have 3D previews to check for the general look of the module (as seeing it only from an orthographic front is misleading — eg. Rogan 5PS feels smaller than Rogan 3PS), and maybe even realistic module packshots for marketing early tests.