Memory manager hangs on physical hardware

[the-grue]

I have been having issues with my laptops and desktop/server hardware where it runs fine as long as I stay in real mode, but as soon as I activate a memory manager ($386.sys), it hangs and locks up the system to the point only a cold boot will get you out.

I am currently working on a new memory manager that seems to be allowing me to resolve or at least bypass this issue. I believe it may have something to do with non-contiguous memory regions, so that is what I am currently looking at.

Is anyone else running on real hardware, with a pentium or newer hardware? Do you have the same issues?

[roelandjansen]

did you try the /P option on the $386.sys driver?

[the-grue]

did you try the /P option on the $386.sys driver?

Yeah, I have tried all of the possible combinations, even some of the "undocumented" ones.

We can close this for now since nobody else seems to be seeing it. I'll continue to poke at it as time permits.

[roelandjansen]

a long shot here -- there are BIOS options for virtualization etc. You could see if that
"fixes" the issue? On the other hand -- the weird part is that under vmware -- that passes on all flags afaik, it works pretty well.

[the-grue]

Yeah, the /P flag and turning off virtualization settings in VirtualBox works fine too. That's where I do most of my testing and full builds prior to running anything on bare metal. One machine I am having issues with is a pre-virtualization P4. It hangs and the disk activity light stays on. My guess, on seeing this happen pretty constantly, is that maybe something in the int 13 code in $386.sys may have a bug. I added some flags to get it to work on a different P4 based machine, basically it tells it to stop counting memory when it finds a gap or you can tell it how much memory to count up to. Both of these machines have 512M in them. Could be chipset, could be the int13, could be I am way off base. (Wouldn't be the first time!) I no longer think it is due to memory gaps as I originally thought though.

…

________________________________ From: Roeland Jansen <[email protected]> Sent: Monday, August 12, 2019 3:07 AM To: roelandjansen/pcmos386v501 <[email protected]> Cc: James T. Sprinkle <[email protected]>; Author <[email protected]> Subject: Re: [roelandjansen/pcmos386v501] Memory manager hangs on physical hardware (#76) a long shot here -- there are BIOS options for virtualization etc. You could see if that "fixes" the issue? On the other hand -- the weird part is that under vmware -- that passes on all flags afaik, it works pretty well. — You are receiving this because you authored the thread. Reply to this email directly, view it on GitHub<#76?email_source=notifications&email_token=AAXVDGASVSXTSJA7YB3OZKDQEEY43A5CNFSM4GZ3JKJKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4CCVLQ#issuecomment-520366766>, or mute the thread<https://github.com/notifications/unsubscribe-auth/AAXVDGCQC2SJSO2CSMBAGT3QEEY43ANCNFSM4GZ3JKJA>.

[darkstar252]

Does $386.sys support INT 15 AX=E820h?

http://www.uruk.org/orig-grub/mem64mb.html

[the-grue]

Hi @darkstar252 ! It doesn't use that to calculate memory. Instead, it uses a "brute force" scan once it is in protected mode. Since most, if not all, newer BIOS supports that call, an update to the memory manager could use that, but it could break backward compatibility.

That is another thing we need to discuss for the project. Do we maintain backward compatibility as we go forward or do we drop legacy features (support for FAT12 on old 5.25 floppies, for example) and legacy hardware (pre 386 for example)?

[darkstar252]

This has the same when Udo Kunht was starting to modify Open DOS 7.01, and he added FAT32 support to that kernel and some enhancements.
I don’t think to add needed features for modern hardware was bad, without breaking compatibility with the older ones.

[stsp]

You can in principle use e820 only
if it is available, and use the old techniqie
otherwise. I dont know how it "scans"
the memory in prot mode, but if it
pokes into the regions that are reserved
in e820, then it does an extremely
malicious thing that may crash the
real machine and is only good for
emulators. So if not removing that
technique, then at least make it a
last resort.
As for removing things like fat12 -
for what? Removing for the sake of
removal is usually a bad idea.
Just my 2 penses.

[stsp]

fat12 is needed to boot from floppy
image on emulator, for example.

[the-grue]

support for FAT12 on old 5.25 floppies

@stsp I didn't say to remove FAT12 entirely, just the support matrices and code required to support really old floppy formats. Anybody actually using 8" disks? 5.25 w/ less than 360K? etc.

@darkstar252 the bios function requires use of 32-bit registers. Because of that, it would require a 386+ to use it and make MOS no longer work with anything less. This was the point of my comment about backward compatibility. I believe MOS uses 8086 instructions only until you attempt to go into protected mode with the memory managers. Then it will use 386 instructions. Granted, if you ran the command in the memory manager before the switch to 32bit protected mode, that may work.

@stsp It does poke/peek (more like clobber and verify) at memory locations to determine if memory is there. It does this after it scans for the first meg of memory.

;======================================================================
;,fs
; test_extmem
;
; test for a contiguous range of extended memory
;
; in: es -> gdtpage (setup for full access)
; ebx -> phyadr of range to be tested
;
; out: ebx = # of pages found
;
;,fe
;======================================================================
assume ds:mmgrp
test_extmem:
push eax
push edx
push edi
mov edi,ebx
mov dx,7755h ; test pattern
xor eax,eax ; to blank out found blocks
cld
txm1:
xif edi == 1000000h && [m17wrap] == 'Y' then txm2 ; quit before a wrap
mov es:[edi],dl
jmp $+2 ; tiptoe around the wyse 386
mov es:[edi+1],dh
jmp $+2
cmp es:[edi],dl
jmp $+2
xif flags == ne then txm2
cmp es:[edi+1],dh
jmp $+2
xif flags == ne then txm2
push edi
mov ecx,4096/4 ; if get this far, zero out the page
rep stosd
jmp $+2
pop edi
cmp es:[edi],eax ; did the zero'ing take?
xif flags == ne then txm2
add edi,4096 ; advance to next page
jmp txm1
txm2:
sub edi,ebx ; determine total range
shr edi,12 ; divide by 4096 to calc # of pages
mov ebx,edi
pop edi
pop edx
pop eax
ret

[stsp]

If e820 requires the use of 32bit
registers and 386+ then on 286-
you won't run $386.sys driver.
Is there a big reason to run $386.sys
on 286- anyway?
I think you can't get away w/o e820
these days, but I dont think it sbould
go in as a compatibility barrier. You can
still stick to realmode on 286, in which case
e820 is not needed.

[darkstar252]

One simply question. Is the $386.sys for 386+? And $286.sys for 286+? And one question more. I want to translate the text messages for PC-MOS 386 in all of the utilities kernel etc. Can I? and This needs another discussion on howto make the code friendly for international translations?

[the-grue]

$386.sys is for 386+. $286.sys is for 286 only.

I guess I am not being clear in my concern relating to backwards compatibility. I am speaking about the kernel and the tools. If we introduce 32 bit instructions into either of them, we break the ability of folks using 286, 186, 8086, 8088, etc from being able to use the OS. The second you include instructions from a higher instruction set, you make it incompatible with processors that don't include those instructions.

Now, if you wanted to include all the 32 bit code or instructions/interrupts that use the 32 bit registers in $386.sys, that should be fine, since it expects to be run on a 32 bit processor and can use 32 bit instructions. You would just need to run the e820 interrupt while still in real mode.

As far as translations, we could probably do that by pulling out all of the message strings from the various components and putting them in language files. Then, perhaps, you could load the language file of your choice as a device or TSR or something. Just a thought. In order to do something like this, wouldn't we also need to have custom font files to support character sets not included in the standard ASCII set?

[roelandjansen]

regarding translations -- it's well known that translations happen to have a few side-effects.

• First of all, an issue will hardly be googleable as it's a language that hardly anyone uses.
  (this happens with windows and linux too -- the dutch translations hardly hits cpompared to
  english)
• Secondly, most translations are done in a very bad way. Many translations don't end up well.
  (floppy disk is being transtaled to zachte schijf and harddisk to harde schijf -- the people who
  translated this should be shot or something ;-))

I always say that if you change CPU to centrale processor eenheid, it just does one thing. It makes an unknown item translated into something even less known. If someone needs to know what a CPU is I tell them once instead of trranslating, ending up with two issues -- tech people capable of telling what it is don't understand the translation well. And if that part is done, translation back to teh real name, you end up explaining it anyways.

Have I said already that internationalization/localization beyond time/date is something I don't like? ;-)

In any case, to be a it more serious, translation is a possibility but I believe there re more important things to do.

[darkstar252]

The translations are in spanish sorry for my poor explanation of i want to make.

[stsp]

Yes, e820 should go to $386.sys and
is hardly needed outside of it. I was
quite sure that on modern bioses you
can even run it from prot mode, but
osdev wiki says only e881 is guaranteed
to work from prot mode. In any case,
$386.sys knows how to go to v86, or
indeed one can just do e820 before the
switch to pm.

[HStewartGitHome]

This is Stewart Hyde (SAH), I was one of original developers of PC-MOS back in late 80's and very early 90's.I actually work on $386.sys and actually found but in IBM 386SLC.

One thing to look for and exactly very stupid, is memory being display to screen when system boots up. I was able to get it work in virtual machine since but had to limited the memory. I would search the code where it display memory. with more memory it could be overwriting the amount display on screen. I remember this happen with larger machine I remember being tested 486 with 12Meg of memory and Gig of storage.

Also I would add to configuration, to tell memory manger not to load anything in high memory to isolate any issues.

It been such a long time and trying to remember but I do have download source.

[HStewartGitHome]

286 was used only in special cases and some had special cards on cpu to allow paging
386 is for 386 and higher and but not sure tested on Pentium.

[the-grue]

Welcome @HStewartGitHome ! I was doing some updates on $386.sys awhile back and I still have the branch on my repo. Not sure where I left off, but I believe one thing I did was allow you to pass a maximum memory parameter and I also changed the display to megabytes instead of kilobytes. I started from a copy of _386.ASM and my working copy is _NEWMM.ASM in the branch. With these changes, I was able to get $NEWMM.SYS to work on a P4 laptop with 512M RAM just fine and it would display the full 512M of RAM.

I started thinking maybe the issue I was having was something to do with the IRQ mapping on some of my test devices and then I got sidetracked on other projects.

[HStewartGitHome]

One thing my boss who created the memory manger created is a device that help debug the memory manger was a parallel cable with has led hook to it and in code you can output a byte and if it locks you know where it locks up. I used this device to track down the bug with the IBM 386SLC

[the-grue]

Very interesting! You wouldn't happen to have a schematic of that device, would you? Or was this part of the periscope tooling I see reference to throughout the code.

[stsp]

Such devices are very common.
You can buy one, or google for the
schematic with "lpt port leds" query
or alike.

[stsp]

https://randomskk.net/lptmon/#hardware
http://ponty.github.io/MyElectronicProjects/projects/parallel_port_monitor/parallel_port_monitor.html

[the-grue]

Thanks @stsp