Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

hook-mdev: better /dev/disk/by-id via custom mdev script for wwid parsing and fallbacks #227

Merged
merged 3 commits into from
Jun 1, 2024

Conversation

rpardini
Copy link
Contributor

@rpardini rpardini commented Jun 1, 2024

hook-mdev: add persistent-storage script from upstream Alpine (ipsis-literis)

hook-mdev: rename to persistent-storage.sh and shellfmt it (no real changes)

  • simple rename & shellfmt; separate commit for easy future rebasing

hook-mdev: add customized persistent-storage script which tries harder to produce /dev/disk/by-id entries by parsing wwid

  • this is still not-even-close to systemd's udev, but should at least add something to by-id when Alpine's mdev wouldn't
    • if device reports model and serial, all was/is good;
    • introduce:
      • if model or serial missing, try parsing them from wwid;
        • falls back to using the sanitized wwid as serial if parsing fails
      • last resort: falls back to using 'noserial' as serial for devices that only have a model.
  • also adds util-linux related apks, which brings a more capable blkid
  • for context: https://gitlab.alpinelinux.org/alpine/mdev-conf/-/commits/master/?ref_type=HEADS
  • add a log of logging, which you can see with cat /var/log/mdev.log on Hook's console

@rpardini
Copy link
Contributor Author

rpardini commented Jun 1, 2024

Before this PR (Hook hook-default-amd64 via LK's qemu)

/dev/disk
├── by-id
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

On Ubuntu, with systemd's udev:

/dev/disk
├── by-id
│   ├── ata-QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── scsi-0ATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-0QEMU_QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-1ATA_QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── scsi-1ATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── scsi-SATA_QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   └── scsi-SATA_QEMU_HARDDISK_QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
├── by-path
│   ├── pci-0000:00:1f.2-ata-1 -> ../../sda
│   ├── pci-0000:00:1f.2-ata-1-part1 -> ../../sda1
│   ├── pci-0000:00:1f.2-ata-1-part14 -> ../../sda14
│   ├── pci-0000:00:1f.2-ata-1-part15 -> ../../sda15
│   ├── pci-0000:00:1f.2-ata-1.0 -> ../../sda
│   ├── pci-0000:00:1f.2-ata-1.0-part1 -> ../../sda1
│   ├── pci-0000:00:1f.2-ata-1.0-part14 -> ../../sda14
│   ├── pci-0000:00:1f.2-ata-1.0-part15 -> ../../sda15
│   ├── pci-0000:00:1f.2-ata-3 -> ../../sr0
│   └── pci-0000:00:1f.2-ata-3.0 -> ../../sr0
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

With this PR (Hook hook-default-amd64 via LK's qemu)

/dev/disk
├── by-id
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── ata-ro-QEMU_HARDDISK_QM00001 -> ../../sr0
│   ├── scsi-ro-t10.ATA_____QEMU_DVD-ROM____________________________QM00005 -> ../../sr0
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001 -> ../../sda
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part1 -> ../../sda1
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part14 -> ../../sda14
│   └── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

@rpardini rpardini marked this pull request as draft June 1, 2024 10:25
@rpardini
Copy link
Contributor Author

rpardini commented Jun 1, 2024

Still not good enough for my USB-connected disks. Those have wwid with some naa., have a model sysfs file (which is being ignored) but don't have a serial sysfs. Drafted PR until minimally addressed

@rpardini rpardini force-pushed the somewhat-better-mdev-disk-by-id branch from 5ef1d10 to c10fd8a Compare June 1, 2024 12:31
@rpardini
Copy link
Contributor Author

rpardini commented Jun 1, 2024

Force-pushed fixed version. I had made a gross mistake (hardcoded sda) on the original commits; added a few fallbacks for disks without model / serial / wwid in that order, eventually using 'noserial' hardcoded.

This makes most disk devices at least appear in disk/by-id, even if they most probably won't match systemd's udev nomenclature, at least we have 'some' id. Using /dev/sdb et al directly is a recipe for disaster.

@rpardini rpardini marked this pull request as ready for review June 1, 2024 12:35
@rpardini
Copy link
Contributor Author

rpardini commented Jun 1, 2024

This PR (after fixes), qemu

/dev/disk
├── by-id
│   ├── ata-QEMU_HARDDISK_QM00001 -> ../../sda
│   ├── ata-QEMU_HARDDISK_QM00001-part1 -> ../../sda1
│   ├── ata-QEMU_HARDDISK_QM00001-part14 -> ../../sda14
│   ├── ata-QEMU_HARDDISK_QM00001-part15 -> ../../sda15
│   ├── ata-ro-QEMU_DVD-ROM_QM00005 -> ../../sr0
│   ├── scsi-ro-t10.ATA_____QEMU_DVD-ROM____________________________QM00005 -> ../../sr0
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001 -> ../../sda
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part1 -> ../../sda1
│   ├── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part14 -> ../../sda14
│   └── scsi-t10.ATA_____QEMU_HARDDISK___________________________QM00001-part15 -> ../../sda15
├── by-label
│   ├── UEFI -> ../../sda15
│   └── cloudimg-rootfs -> ../../sda1
├── by-partuuid
│   ├── 153d8d43-4a97-4dc6-af23-fde0cd384151 -> ../../sda15
│   ├── 182cef04-542b-4904-be23-94040d967d93 -> ../../sda1
│   └── 35d4ae99-c70b-470b-8de3-24db7e8c3474 -> ../../sda14
└── by-uuid
    ├── 75B6-4176 -> ../../sda15
    └── 840082fd-4646-4649-bf63-d92ca9fe2b97 -> ../../sda1

A different VM, using virtio disks with explicitly-set serial numbers:

/dev/disk
├── by-id
│   ├── virtio-cidata_serial -> ../../vda
│   └── virtio-root_disk_serial -> ../../vdb
├── by-label
│   └── cidata -> ../../vda
└── by-uuid
    └── 2024-06-01-11-43-11-85 -> ../../vda

@rpardini rpardini changed the title hook-mdev: better /dev/disk/by-id via custom mdev script & better by-label/uuid via util-linux hook-mdev: better /dev/disk/by-id via custom mdev script for wwid parsing and fallbacks Jun 1, 2024
rpardini added 3 commits June 1, 2024 10:51
…s-literis)

- original persistent-storage from upstream Alpine:
  - source https://gitlab.alpinelinux.org/alpine/mdev-conf/-/blob/master/persistent-storage?ref_type=heads
  - revision a21d1053dd5ca37538b0435d66c58a30cb273658
- this is being added as reference for future rebases; it will be shellfmt'ed and modified in later commits

Signed-off-by: Ricardo Pardini <[email protected]>
… changes)

- simple rename & shellfmt; separate commit for easy future rebasing

Signed-off-by: Ricardo Pardini <[email protected]>
…der to produce /dev/disk/by-id entries by parsing `wwid`

- this is still not-even-close to systemd's udev, but should at least add _something_ to by-id when Alpine's mdev wouldn't
  - if device reports model and serial, all was/is good;
  - introduce:
    - if model or serial missing, try parsing them from wwid;
      - falls back to using the sanitized wwid as serial if parsing fails
    - last resort: falls back to using 'noserial' as serial for devices that only have a model.
- also adds `util-linux` related apks, which brings a more capable `blkid`
- for context: https://gitlab.alpinelinux.org/alpine/mdev-conf/-/commits/master/?ref_type=HEADS
- add a log of logging, which you can see with `cat /var/log/mdev.log` on Hook's console

Signed-off-by: Ricardo Pardini <[email protected]>
@jacobweinstock jacobweinstock force-pushed the somewhat-better-mdev-disk-by-id branch from c10fd8a to c361f85 Compare June 1, 2024 16:52
@jacobweinstock jacobweinstock added the ready-to-merge Signal to Mergify to merge the PR. label Jun 1, 2024
@mergify mergify bot merged commit 19d73bd into tinkerbell:main Jun 1, 2024
30 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
ready-to-merge Signal to Mergify to merge the PR.
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants