Skip to content

Commit

Permalink
Merge branch 'master' into gh-pages
Browse files Browse the repository at this point in the history
  • Loading branch information
@jeffpar
jeffpar committed Sep 8, 2023
2 parents c52f794 + f3d2dcc commit b5edf95
Show file tree
Hide file tree
Showing 5 changed files with 120 additions and 91 deletions.
9 changes: 6 additions & 3 deletions .vscode/launch.json
View file
Original file line number Diff line number Diff line change
Expand Up @@ -478,10 +478,13 @@
"request": "launch",
"program": "${workspaceFolder}/tools/pc/pc.js",
"args": [
"--start=ibm5160",
"--floppy"
"load info",
"--sys=pcdos:2",
"--drivetype=100:3:23",
"--test",
"--halt"
],
"cwd": "${workspaceFolder}/tools/pc/disks/empty",
"cwd": "${workspaceFolder}/tools/pc/empty",
"stopOnEntry": false,
"console": "integratedTerminal",
"outFiles": [
Expand Down
4 changes: 2 additions & 2 deletions blog/_posts/2023/2023-09-05-wrapping-up-support-for-fat.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -242,9 +242,9 @@ To confirm this, I tested a drive configuration (162 cylinders, 4 heads, and 17

Here's how I built it using `pc.js`:

% pc.js --sys=pcdos --ver=2.00 --drivetype=162:4:17 --test --save=test.img
% pc.js --sys=pcdos --ver=2.00 --drivetype=162:4:17 --trim --save=test.img

The undocumented `--test` flag tells `pc.js` to bypass its normal DOS-compatibility rules and build/format the disk with an "optimized" 12-bit FAT, and the `--save` option saves the disk image without starting a machine. And although DOS 2.00 didn't like it, `test.img` was an otherwise perfectly valid and usable disk image, and `fsck_msdos` on macOS reported no problems.
The undocumented `--trim` flag tells `pc.js` to bypass its normal DOS-compatibility rules and build/format the disk with an "optimized" 12-bit FAT, and the `--save` option saves the disk image without starting a machine. And although DOS 2.00 didn't like it, `test.img` was an otherwise perfectly valid and usable disk image, and `fsck_msdos` on macOS reported no problems.

It's understandable that DOS 2.00 would be skeptical of its own BPBs, in part because BPBs were a new feature that probably evolved during the development of DOS 2.00, so they would have been dealing with disks with no BPBs, out-dated BPBs, or even invalid BPBs. However, perhaps the biggest problem was FDISK, because whenever FDISK created a DOS partition, it would simply update the partition table in the Master Boot Record and then reboot, leaving the partition's boot sector and any BPB it previously contained in place. And that old BPB might be completely inappropriate.

Expand Down
159 changes: 91 additions & 68 deletions machines/pcx86/modules/v3/diskinfo.js
View file
Original file line number Diff line number Diff line change
Expand Up @@ -1495,7 +1495,7 @@ export default class DiskInfo {

if (driveInfo.partitioned) {
bMediaID = 0xF8;
cHiddenSectors = 1 + (driveInfo.hiddenSectors || 0);
cHiddenSectors = driveInfo.hiddenSectors || 1;
cDiagnosticSectors = cHeads * cSectorsPerTrack;
}

Expand Down Expand Up @@ -1550,7 +1550,6 @@ export default class DiskInfo {
* DOS that performs these same tests using the total sectors value from the partition table, then it may be
* using a slightly different value and therefore arriving at different defaults.
*/
let cRecalcs = 2;
if (!driveInfo.clusterSize) {
if (cTotalSectors <= 512) { // 0x0200 (256Kb)
cSectorsPerCluster = 1;
Expand Down Expand Up @@ -1593,6 +1592,88 @@ export default class DiskInfo {
}
cRootSectors = Math.ceil((rootEntries * 32) / cbSector);

/*
* This is our code to ensure that the the last sector of the BIOS (IO.SYS or IBMBIO.COM) falls on the
* last sector of a track.
*
* This weird requirement is due to how PC DOS and MS-DOS 2.x/3.x boot sectors read the first system file
* into memory: they read the file one track at a time; the first track read may be partial, because it
* starts with whatever the file's first sector is, but every subsequent read is a whole track, even if the
* file doesn't occupy the entire track.
*
* This would be OK if there was ample memory, but the boot sector doesn't relocate itself from 0:7C00,
* and with its stack sitting just below that address, there's room for only about 28K of file data. For
* reference, IO.SYS in MS-DOS 3.30 is about 22K, so there's enough room, but if the final sector is near
* the start of a track, then the final full track read (8.5K for a track with 17 sectors) runs the risk
* of overwriting the stack and/or the boot sector itself.
*
* See https://www.os2museum.com/wp/hang-with-early-dos-boot-sector/ for more details; it's accurate except
* for the implication that a contemporaneous disk using only 17 sectors per track was safe (it was not).
*
* To make matters *slightly* worse, the affected boot sectors didn't accurately calculate the sector size
* of the system file correctly; in keeping with the overall "sloppy" approach, they simply divided the file
* size by the sector size and then *always* added 1 (they should have added 1 only if there was a remainder).
*
* This affects any version of IO.SYS or IBMBIO.COM that is an exact multiple of 512 (such as IBMBIO.COM
* from PC DOS 2.00, which is 4608 bytes or 9 sectors; the boot sector will read 10 sectors instead). Although
* interestingly, DOS 2.x "precalculates" that number and stores it in the BPB at offset 0x20, whereas DOS 3.x
* actually reads the file size from the directory entry and performs the calculation at runtime. In both
* cases though, the calculation is "sloppy".
*
* Having perfect hindsight, we can help the boot sector avoid running into trouble by performing the same
* sloppy sector size calculation ourselves, dividing it by sectors per track, and ensuring that the remainder
* matches the number of free sectors in the first data track (and adjusting volume sector usage until it does).
* As a result, the system file will end at the end of a track, and the boot sector never risks reading too
* much data.
*
* Finally, a note about disks with a cluster size of 2 or more sectors: on such disks, the final *cluster*
* of IO.SYS/IBMBIO.COM may not end on a track boundary, but that's OK, because the boot sector is only
* reading sectors, not clusters. Any "overhang" is merely wasted cluster space and does not affect us here.
*/
let cFileSectors = aFileData[0]? Math.trunc(aFileData[0].size / cbSector) + 1 : 0;
let adjustTotalSectors = function() {
let fAdjusted = false;
if (cFileSectors) {
let maxAdjustments = driveInfo.hiddenSectors? 0 : cSectorsPerTrack;
while (maxAdjustments--) {
let cInitSectors = cHiddenSectors + cReservedSectors + cFATs * cFATSectors + cRootSectors;
/*
* I used to calculate the number of free sectors in the first track with free sectors:
*
* let cFreeSectors = cSectorsPerTrack - (cInitSectors % cSectorsPerTrack);
*
* and break when cFreeSectors >= cFileSectors, because that meant the file was contained
* entirely within that track, but that's not sufficient, because if the disk is using a large
* number of sectors/track (eg, 63) AND the file happens to be at the start of the track, then
* a full track (31.5K) will be read, which will trash the boot sector. We REALLY need to push
* the file to the END of the track, even if it's already fully contained within the track.
*/
if ((cInitSectors + cFileSectors) % cSectorsPerTrack == 0) {
break;
}
/*
* I used to increase root directory sectors, since we were at least getting some benefit from the
* adjustment:
*
* cRootSectors++;
* rootEntries += (cbSector >> 5);
*
* However, that created compatibility issues (see the verDOS code above for specific thresholds we
* need to honor). Next, I tried tweaking reserved sectors, but guess what? Few if any versions of DOS
* actually honor reserved sectors (they assume it's 1 and crash if it isn't):
*
* cReservedSectors++;
*
* So we're left with adjusting hidden sectors, which requires a corresponding adjustment to total sectors:
*/
cHiddenSectors++;
cTotalSectors--;
fAdjusted = true;
}
}
return fAdjusted;
};

/*
* Now we get to a thornier matter: when calculating how many clusters will fit on a disk, the calculation
* SHOULD begin with total DATA sectors, not total DISK sectors. But that presents a chicken-and-egg problem,
Expand All @@ -1607,6 +1688,7 @@ export default class DiskInfo {
* I learned about the latter by watching IO.SYS from MS-DOS 3.30 read the entire FAT into memory (at 0000:7DC6):
* if it reads more than 32K of FAT data, it will start trashing memory.
*/
let cRecalcs = 4;
let grossClusters, minClusters, maxClusters, initSectors = cSectorsPerCluster;
do {
minClusters = (typeFAT == 12)? 0 : DiskInfo.FAT12.MAX_CLUSTERS + 1;
Expand Down Expand Up @@ -1674,7 +1756,13 @@ export default class DiskInfo {
continue;
}
if (grossClusters <= maxClusters) {
if (cFATSectors * cbSector <= 32 * 1024) break;
if (cFATSectors * cbSector <= 32 * 1024) {
if (adjustTotalSectors()) {
if (!cRecalcs--) break;
continue;
}
break;
}
}
if (cSectorsPerCluster == maxSectorsPerCluster) {
if (typeFAT == 12) {
Expand Down Expand Up @@ -1728,71 +1816,6 @@ export default class DiskInfo {
setBoot(DiskInfo.BPB.TRACKSECS, 2, cSectorsPerTrack);
setBoot(DiskInfo.BPB.DRIVEHEADS, 2, cHeads);

/*
* We're now at the point where we ensure that the the last sector of the BIOS (IO.SYS or IBMBIO.COM)
* falls on the last sector of a track.
*
* This weird requirement is due to how PC DOS and MS-DOS 2.x/3.x boot sectors read the first system file
* into memory: they read the file one track at a time; the first track read may be partial, because it
* starts with whatever the file's first sector is, but every subsequent read is a whole track, even if the
* file doesn't occupy the entire track.
*
* This would be OK if there was ample memory, but the boot sector doesn't relocate itself from 0:7C00,
* and with its stack sitting just below that address, there's room for only about 28K of file data. For
* reference, IO.SYS in MS-DOS 3.30 is about 22K, so there's enough room, but if the final sector is near
* the start of a track, then the final full track read (8.5K for a track with 17 sectors) runs the risk
* of overwriting the stack and/or the boot sector itself.
*
* See https://www.os2museum.com/wp/hang-with-early-dos-boot-sector/ for more details; it's accurate except
* for the implication that a contemporaneous disk using only 17 sectors per track was safe (it was not).
*
* To make matters *slightly* worse, the affected boot sectors didn't accurately calculate the sector size
* of the system file correctly; in keeping with the overall "sloppy" approach, it simply divides the file
* size by the sector size and then *always* adds 1 (it should have added 1 only if there was a remainder).
* However, unless IO.SYS or IBMBIO.COM was an *exact* multiple of 512, this calculation would generally
* end up with the correct answer.
*
* Having perfect hindsight, we can help the boot sector avoid running into trouble by performing the same
* sloppy sector size calculation ourselves, dividing it by sectors per track, and ensuring that the remainder
* matches the number of free sectors in the first data track (and adjusting volume sector usage until it does).
* As a result, the system file will end at the end of a track, and the boot sector never risks reading too
* much data.
*/
let cFileSectors = 0;
if (aFileData[0]) {
let maxAdjustments = driveInfo.hiddenSectors? 0 : cSectorsPerTrack;
cFileSectors = Math.ceil(aFileData[0].size / cbSector);
while (maxAdjustments--) {
let cInitSectors = cHiddenSectors + cReservedSectors + cFATs * cFATSectors + cRootSectors;
let cFreeSectors = cSectorsPerTrack - (cInitSectors % cSectorsPerTrack);
/*
* I used to ALSO break whenever cFileSectors - cFreeSectors < 0, because that meant the file was
* contained entirely within a single track, but that's not sufficient, because if the disk is using
* a large number of sectors/track (eg, 63) AND the file happens to be at the start of the track,
* then a full track (31.5K) will be read, which will trash the boot sector. We REALLY need to push
* the file to the END of the track, even if it's fully contained within the track.
*/
if ((cFileSectors - cFreeSectors) % cSectorsPerTrack == 0) break;
/*
* I used to increase root directory sectors, since we were at least getting some benefit from the
* adjustment:
*
* cRootSectors++;
* rootEntries += (cbSector >> 5);
*
* However, that created compatibility issues (see the verDOS code above for specific thresholds we
* need to honor). Next, I tried tweaking reserved sectors, but guess what? Few if any versions of DOS
* actually honor reserved sectors (they assume it's 1 and crash if it isn't):
*
* cReservedSectors++;
*
* So we're left with adjusting hidden sectors, which requires a corresponding adjustment to total sectors:
*/
cHiddenSectors++;
cTotalSectors--;
}
}

if (cTotalSectors <= 0xffff) {
setBoot(DiskInfo.BPB.DISKSECS, 2, cTotalSectors);
} else {
Expand Down
4 changes: 4 additions & 0 deletions tools/diskimage/README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,10 @@ To extract a specific file from a disk image:

diskimage.js /diskettes/pcx86/sys/dos/ibm/2.00/PCDOS200-DISK1.json --extract=COMMAND.COM

To display the contents of a specific file in a disk image:

diskimage.js /diskettes/pcx86/sys/dos/ibm/3.00/PCDOS300-DISK2.json --type=VDISK.LST

To extract files from a disk image into a specific directory (eg, tmp):

diskimage.js /diskettes/pcx86/sys/dos/ibm/2.00/PCDOS200-DISK1.json --extract --extdir=tmp
Expand Down
Loading

0 comments on commit b5edf95

Please sign in to comment.