Re: About firmware facilities

[Vladimir 'phcoder' Serbinenko]

Brendan Trotter wrote:
>> No. Usuable means only that firmware isn't destroyed. Any device may
>> be in a different state
>>     
>
> Any device (that the firmware assumes is in a certain state) may be
> left in a different state (that the firmware no longer knows about)?
>
> For a very simple example, imagine if the BIOS leaves the floppy motor
> on, and GRUB's own floppy driver uses the floppy and then turns the
> motor off. Then the OS uses the firmware to read from floppy, but the
> firmware thinks the floppy motor is still on and attempts to read from
> the floppy without turning the floppy motor on.
>
> If GRUB has it's own device drivers, and GRUB doesn't restore devices
> to the state that the firmware expects the devices to be in, then the
> firmware is unusable.
>
>   
Most OSes should use their own drivers to access devices.
> If an OS can't use the firmware, then the OS must rely on GRUB for
> everything instead, including strange "OS specific" things that nobody
> has seen any other OS do before.
>
>   
If nobody uses a particular feature in firmware then you shouldn't use
it either. Unused firmware features are often buggy. Moreover firmware
on x86 is useful only for bootstrap and once bootstrap is completed you
should forget it exists except some firmware-specific tasks as setting
boot device.
>>>>> Due to limitations in the original multi-boot specification my code
>>>>> switches back to real mode and uses the BIOS to do memory detection,
>>>>> do video mode detection, switch video modes and gather other
>>>>> information.
>>>>>           
>>>> Have you actually read the multiboot specification? Booter passes info
>>>> about memory and video mode in mbi (video for multiboot isn't
>>>> implemented yet). If you need firmware for basic bootup you're clearly
>>>> doing something wrong and are firmware-dependent. Of course it's your
>>>> freedom to make suboptimal software.
>>>>         
>>> I've read the multi-boot specification. I've also read the code in
>>> GRUB-legacy that does memory detection, and I'm unwilling to allow my
>>> code to rely on it for "quality control" reasons. Without going into
>>> details, GRUB-legacy tends to do a "minimal" job and then expects the
>>> user to fix the problem if/when it goes wrong (but even then it only
>>> offers a "uppermem" command without providing a way for the user to
>>> specify a complete system memory map).
>>>
>>>       
>> What is "minimal job" and "quality control"? We use standard
>> E820+(optionally)badram command. I've seen no OS do any more than
>> this.
>>     
>
> My code tries "int 0x15, eax=0xE820" expecting 24 bytes per area (ACPI
> 3.0); then it tries "int 0x15, eax=0xE820" expecting 20 bytes per
> area. If "int 0x15, eax=0xE820" isn't supported by the BIOS then you
> can assume it's an old computer (and old computers are painful).
>
> It tries "int 0x15, ax=0xE801", then "int 0x15, ah=0xC7", then "int
> 0x15, ah=0x8A", then "int 0x15, ah=0xDA88", then "int 0x15, ah=0x88",
> then CMOS locations 0x70 and 0x71.
Read code. GRUB fallback to old methods if newer aren't available.
>  If all of this fails (which does
> happen on some computers) then it does manual probing.
>   

> Some of the old BIOS functions have limited range - for example
> they'll return "number of KiB blocks at 0x00100000" as a 16-bit
> integer, and can't return more than 64 MiB. In this case my code won't
> know if the value returned by the BIOS has been limited to 0xFFFF, so
> it'll do manual probing to detect any more RAM above the reported
> area. Some computers have an "ISA hole" from 0x00F00000 to 0x00FFFFFF.
> Because of this all older BIOS functions that report "amount of RAM at
> 0x00100000" may return 14 MiB (from 0x00100000 to 0x00EFFFFF) when
> there's actually another area of RAM at 0x01000000. In this case my
> code won't know if the value returned by the BIOS is right or not, and
> it'll do manual probing to detect any more RAM at 0x00100000. If my
> code has to do manual probing, then it assumes there's an "ISA hole"
> from 0x00F00000 to 0x00FFFFFF (regardless of whether there is or not)
> as this hole was used for memory mapped video cards (which would seem
> like RAM).
>   
Have you thought that manual probe may detect MMIO as additional "RAM"?
Video RAM isn't the only case of MMIO.
I prefer to function correctly on bug-free firmware at cost of quirks on
buggy ones rather than other way round
> For all BIOS functions used my code avoids all known BIOS bugs (and
> there's plenty of them). This includes "sanitizing" the data returned
> from "int 0x15, eax=0xE820" - sorting the list and handling any
> overlapping areas.
>   
have you ever looked at mmap folder?
> I've never needed to provide a way for the end-user to override my
> memory detection.
>
>   
Neither did we. But test your manual probing at 4GiB system - it's
likely to detect all MMIO addresses as RAM.


>>> For memory detection, ACPI 3.0 allows the BIOS (" INT 15H, E820H") to
>>> return extended attributes - mostly only a volatile/non-volatile flag.
>>> This isn't in GRUB's information. ACPI 3.0 also allows the BIOS to
>>> return areas of the type "AddressRangeUnusable" (e.g. faulty RAM).
>>>       
>> This is mostly unnecessary. Basically you need only to know if you can
>> use a memory range or not. The only useful additional code would be
>> ReclaimMemory
>>     
>
> To handle standby states correctly the OS may need to know which areas
> are volatile and which areas aren't (which can include knowing the
> difference between volatile system areas and non-volatile system
> areas). Some OSs also want to know if there's any faulty RAM present
> in the system or not (and additional information about any area
> reserved for "hot-plug" RAM, and NUMA ranges, but that information
> comes from ACPI tables not BIOS functions so the OS can get this
> information without GRUB).
>
>   
I'm ok with defining additional types in multiboot1. But OS considering
multiboot type to be BIOS type is buggy
>>
>> GRUB can't do this right now because it doesn't recieve badram info
>> soon enough. And even if it does most kernels expect first MiB to be
>> usable.
>>     
>
> You're right - all kernels that are designed to use "multi-boot
> specification version 1" expect to be loaded at 0x00100000 and that
> RAM below the EBDA is usable. I'm not sure what kernels designed for
> "multi-boot specification version 2" expect...
>
>   
Read what I said
>> Such list is a blatant encapsulation breach. If you want such test,
>> add it to bootloader, not OS.
>>     
>
> When the OS is running and detects a RAM fault, you want the OS to run
> a copy of GRUB (maybe inside an emulator or something) so the OS can
> tell GRUB about the RAM fault, and GRUB can tell the OS if the RAM
> fault might cause problems if the computer is rebooted (and so the OS
> can send an email to the network administrators or something *before*
> the computer is rebooted)?
>
> You can't assume that the OS that is running is the same OS that
> installed GRUB; or that the OS that is running has access to wherever
> GRUB is installed; or that GRUB will be able to detect any faulty RAM
> during boot.
>
>   
You're in circular logic. You assume that booter is using faulty RAM but
supplying RAM it used correctly.
>>
>> This information is available with a simple loop over mbi. I would
>> rathjer avoid overcomplicating the standard because it increases a
>> chance of having "half-compliant" OSes and "half-compliant" booters.
>>     
>
> I'd rather have "fully compliant" OSes that are easier to write than
> "fully compliant" OSes that are a pain in the neck to write because
> you have to parse everything in the multi-boot information structure
> before you can write to any RAM (except for your own ".bss").
>
> If I can't rely on the firmware (like I currently do) then I have to
> rely on GRUB, and have to copy everything from the multi-boot
> information structure into my ".bss". So, how much extra space do I
> need to allow in my ".bss"? What is the maximum number of drive
> structures? What's the maximum number of memory map entries? What's
> the maximum length of the command line? The multi-boot specification
> doesn't say.
>
>   
First do a small parse and count how many memory the structures you need
to take.

>>> The "Boot device" field in the multi-boot information structure should
>>> be improved to handle a variety of conditions; including if the disk
>>> was an emulated disk (e.g. "El Torito" emulating a hard drive). The
>>> BIOS drive number isn't much use (especially if the firmware is
>>> coreboot, UEFI, OpenFirmware, etc), and should be replaced with
>>> something that identifies the corresponding drive structure (this
>>> includes USB).
>>>       
>> Boot device shouldn't be used at all. It was a mistake. Booter has no
>> good way to know how OS will see the device. You should pass this
>> parameter via commandline either as device name or UUID. You have
>> scripting to automate this
>>     
>
> A user who's using GRUB to boot Ubuntu decides to install my code in
> another partition, then modify GRUB's configuration (in Ubuntu) so
> that GRUB will also boot my code. Now my code needs to rely on the
> user to not stuff up GRUB's configuration for my code?
>
>   
You can simply tell him to add "source" line
>>> The "boot loader name" field is nice, but it needs a "boot loader
>>> version" field to go with it.
>>>       
>> it's a part of name. This field is more for displaying anyway and OS
>> shouldn't do any checks based on it
>>     
>>> A "firmware type" field is also needed.
>>>       
>> Can you respond in appropriate thread?
>>     
>
> Ok - where is the appropriate thread?
>
>   
ML search is your friend
>>> The OS image also needs a different magic number to indicate that the
>>> OS image is designed for future versions of the multi-boot
>>> specification (rather than the old/current version). If the OS image
>>> uses the new magic number, then the OS image must also include an
>>> "version of the multi-boot specification that this image complies
>>> with" field. If the OS image indicates that it's intended for a newer
>>> version of the multi-boot specification than the boot loader complies
>>> with, then the boot loader refuses to boot and displays a "this boot
>>> loader needs to be upgraded" error. If the OS image has the old magic
>>> number, and if the firmware is "PC BIOS" then the boot loader should
>>> boot the old OS image. If the OS image has the old magic number, and
>>> if the firmware is not "PC BIOS" then the boot loader refuses to boot
>>> and displays a "this OS requires a PC BIOS" error message.
>>>       
>> Already implemented through feature fields
>>     
>
> Is there a private version of the multi-boot specification that I'm
> not aware of yet; or does GRUB fail to comply with the current
> multi-boot specification?
>
>   
No. Read specification again