It turns out that without the fix, the transmit properly sends 7-bit data, but receive still sometimes sets the 8th bit high. I need to explore more...

On my journey to understanding this better — It's been a while since I've explored this in depth (for TeensyDMX) — I think I'm re-discovering that the "data" includes everything but the start and stop bits. This means that my proposed solution, setting M7, will really cause a 6-bit data byte if there's parity. If I'm right, then my solution isn't correct. I'm closer...

See: https://forum.pjrc.com/threads/71505-7-bit-support-seems-to-be-missing-on-Teensy-4

That top bit, because it's the last in the stream, turns out to be the value of the parity bit. On to a fix...

Okay, I think I got it. There has to be an appropriate mask depending on the mode.

I tested this with a 7E1 device, both directions.

I updated the PR to include a commit for fixing teensy3 7-bit support.

Tested with the same 7E1 device.

Here's my logic for fixing this: All formats other than 7-bit don't set the parity bit when a user reads bytes. It is unexpected for the 7-bit format to set the parity bit. This PR fixes this case. If this is not fixed, then, in my opinion, it should be documented.

Quick update: As I mentioned on the forum thread. The other 8 bit and 9 bit also return the parity. But by default the software FIFO queues are 8 bits per element, so the data is lost.

There is a #define commented out in the HardwareSerial.h that enables the FIFO queue to be uint16_t instead and it does return the actual data including the parity bit.

Posted test sketch in the forum thread, but with that bit on, it outputs:

Test format: SERIAL_7E1 Write time us: 170
41(A) 42(B) C3(C) 44(D) C5(E) C6(F) 47(G) 48(H) C9(I) CA(J) 4B(K) CC(L) 4D(M) 4E(N) CF(O) 50(P)  : 7B FC 7D 7E FF 

Test format: SERIAL_7O1 Write time us: 171
C1(A) C2(B) 43(C) C4(D) 45(E) 46(F) C7(G) C8(H) 49(I) 4A(J) CB(K) 4C(L) CD(M) CE(N) 4F(O) D0(P)  : FB 7C FD FE 7F 

Test format: SERIAL_8N1 Write time us: 171
41(A) 42(B) 43(C) 44(D) 45(E) 46(F) 47(G) 48(H) 49(I) 4A(J) 4B(K) 4C(L) 4D(M) 4E(N) 4F(O) 50(P)  : FB FC FD FE FF 

Test format: SERIAL_8E1 Write time us: 188
41(A) 42(B) 143(C) 44(D) 145(E) 146(F) 47(G) 48(H) 149(I) 14A(J) 4B(K) 14C(L) 4D(M) 4E(N) 14F(O) 50(P)  : 1FB FC 1FD 1FE FF 

Test format: SERIAL_8O1 Write time us: 188
141(A) 142(B) 43(C) 144(D) 45(E) 46(F) 147(G) 148(H) 49(I) 4A(J) 14B(K) 4C(L) 14D(M) 14E(N) 4F(O) 150(P)  : FB 1FC FD FE 1FF 

Test format: SERIAL_9N1 Write time us: 188
41(A) 42(B) 43(C) 44(D) 45(E) 46(F) 47(G) 48(H) 49(I) 4A(J) 4B(K) 4C(L) 4D(M) 4E(N) 4F(O) 50(P)  : 1FB 1FC 1FD 1FE 1FF 

Test format: SERIAL_9E1 Write time us: 205
41(A) 42(B) 243(C) 44(D) 245(E) 246(F) 47(G) 48(H) 249(I) 24A(J) 4B(K) 24C(L) 4D(M) 4E(N) 24F(O) 50(P)  : 1FB 3FC 1FD 1FE 3FF 

Test format: SERIAL_9O1 Write time us: 205
241(A) 242(B) 43(C) 244(D) 45(E) 46(F) 247(G) 248(H) 49(I) 4A(J) 24B(K) 4C(L) 24D(M) 24E(N) 4F(O) 250(P)  : 3FB 1FC 3FD 3FE 1FF 

Note in each of the output lines for the different Formats I output A-P and I also now after the : in each line try to output the top values for these formats. As to make sure whole range is returned.
Note: With 9 bit output, you need to use a special write method, as shown in the sketch