IDE/PATA disk hda changed to sda in 2.6.20 kernel?

[saikee]

I have been alerted that the support for IDE/Pata disks hda, hdb, hdc and hdd is being "withdrawn" so that in future they will form parts of the SCSI/Sata/USB disk family using the notation of sda, sdb, sdc, sdd, sde...etc.

Can anyone shed some light on it?

I was told by our Linux experts here that the major and minor numbers for IDE/Pata are different to those used by SCSI/Sata/USB. That the 256 raw devices of the former are made up by 4 disks each having a maximum of 64 names (e.g. hda, hda1, hda2, hda3.....hda63). The latter 256 raw devices come from 16 disks each with 16 names like sda, sda1, sda2, sda3....sda15. Thus the two different types hard disks are hardware-detectable permitting such an implementation.

The change is going to be an upheavel to Linux users enjoying more than 15 partitions in a IDE/Pata disk. I am one of those suckers.

The strange thing is that I have installed a couple of distros with 2.6.20 kernels (Suse 9.3 and Slax 6.0.0) into a Pata disk by the conventional names and completely unaware of any change. The new kernels did absolutely nothing to my existing setup which has hda1-ha63, hdc1-hdc63 and hdd1-hdd63, sda1-sda15 & sdb1-sdb6. The two above distros allowed me to install them in hdd55 and hdd52 partitions respectively and run normally just like any other distro.

I suppose the current 2.6.20 kernel has the ability to continue to work with existing hard disks. If that is the case the change isn't mandatory. I am looking into the future to see if my PC could blown up when a new 2.6.2x kernel suddenly stops supporting the existing naming convention of the IDE/Pata disks.

[retsaw]

What is happening is that the PATA drivers are being migrated to the libATA system in the kernel which is what the SATA drivers use. When this affects you depends on what chipset your PATA IDE interface uses as the drivers are not being done all at the same time. I believe that you will have the option of which driver to use for a while (this will surely also depend on your distro also providing that option), but I don't know in what time-frame the old drivers will be phased out.

Oddly, when trying out Kubuntu 7.04 (using the 2.6.20 kernel) on my spare system which has a SiS chipset, the install CD appeared to use the old drivers as the drives had the old hda device name, but when I booted up it was using sda.

I would suggest using LVM with all the distros that support it so you can use more than the 16 partitions per drive that you'll probably be limited to otherwise. Of course, you'll still have the option of using an older kernel if you need to once your old drivers have been removed from the current kernel.

[saikee]

retsaw,

What is my chance of creating the hda2, hda3 in /dev to get the disk read as before?

Are we talking the same (block, major 3, minor 2) for hda2, as an example, in /dev can no longer be understood by future Linux. I did post this thread (http://www.justlinux.com/forum/showthread.php?t=147954) on get help to create more partitions in Pata (within the 64 device names limit) and that works like a treat.

It has to be pretty radical to throw out the Pata system thereby making all existing Pata partitions prior to the 2.6.20 kernel unreadable at some future date.

[retsaw]

retsaw,

What is my chance of creating the hda2, hda3 in /dev to get the disk read as before?

Are we talking the same (block, major 3, minor 2) for hda2, as an example, in /dev can no longer be understood by future Linux.You could create a device called hda which is a symlink or copy of the device file sda as long as it pointed to the same major and minor as sda i.e. (block, major 8, minor 0). If you're using udev you could create a rule to do this. But creating the old hda devices as before would not work. This change might be the main reason Ubuntu switched to refering to partitions by their UUID (which won't change unless they are reformatted) instead of their device name.
It has to be pretty radical to throw out the Pata system thereby making all existing Pata partitions prior to the 2.6.20 kernel unreadable at some future date.I haven't really looked into the reasons why they are making this change, I'm sure you could find a discussion/explanation of it if you look though.

[bwkaz]

The reason is that the PATA code in the kernel is extremely crufty. The libata code is much, much cleaner -- and not only internally: libata handles errors a lot better than the current IDE subsystem does, for instance.

And it's not like they're going to choose a kernel version and add the libata drivers, but remove all the old IDE drivers. libata is available now, and both drivers will be available for a good long time -- the IDE drivers may start being unmaintained, but they won't be removed before their deprecation period passes. (See the Documentation/feature-removal-schedule.txt file in the kernel sources for the current schedule of when various features will be removed: ide will follow a similar period of deprecation. And "unmaintained" doesn't mean that when an external interface changes, the IDE drivers will break, it just means that it's possible that everyone will stop developing those drivers.)

The same thing was slated for the 2.5 kernel series, but libata itself didn't exist yet, so the code was all new. There was a huge shakeup around the IDE drivers then, because they were old and crufty, but the new code that got written destroyed some data, and so people got to be wary of it. Finally the old 2.4 IDE subsystem was put back, but the kernel people have wanted to get rid of that for quite a while. Now they finally are, since libata is finally a viable replacement. (And it's being used for SATA.)

What is my chance of creating the hda2, hda3 in /dev to get the disk read as before? Zero -- but only zero if you disable the ide subsystem. It's turned on by default in the kernel config, and probably will be for a while (certainly until almost everyone's disk controller can be driven by libata, which isn't true yet).

(And for anyone else wondering about the change from hdXY to sdXY, the naming won't affect you if your distro uses an initramfs that works properly, and all the partition references in fstab are to the by-id symlinks in /dev/disk. Of course this also requires udev. Anyway, that's part of the reason those features were added: to survive changes like this.)

[saikee]

bwkaz,

That is a very nice explanation and I am certainly benefited from it.

Can you go one more step, if you have the information relating to the hardware side, by giving us some idea on how the connection of the Bios information on the disk is made with the Linux's (block, major No., minor No.) reference?

I think a Pata hda has (block, 3, 0) reference and the one for sda would be (block, 8, 0). Therefore if the hda is dropped then there will be no block device with a major number 3 presumably and if it is the only disk then the sda reference will be used.

Would I be correct in thinking the kernel keeps a record of the hardware link (of the actual disk in the form of interupt or address of some kind) corresponding to the reference (block, 8, 0)? I would be interest to see that record even if I cant make out any sense out of it. That information will improve my understanding of the magic of mounting a hardware device on the Linux/Unix filing system tree.

I always enjoy reading your replies.

[bwkaz]

Can you go one more step, if you have the information relating to the hardware side, by giving us some idea on how the connection of the Bios information on the disk is made with the Linux's (block, major No., minor No.) reference? It's not -- at least not directly. :)

Linux doesn't use the BIOS, except for a very short time at boot (and for some things related to ACPI that require BIOS support). When the kernel boots, when it loads the driver for either your IDE chipset (the "legacy" IDE driver) or your SATA chipset (the libata driver), that driver scans whatever buses you have that it can drive (e.g. PCI, PCIe, whatever), and discovers any controllers it can drive that are connected.

Once the driver has discovered the controllers, it asks them which devices are connected to which ports. Once it has that information for each device, it registers the device with the IDE or SCSI subsystem (which handles the various I/O requests on the device files). The IDE or SCSI subsystem then sends the uevents for the new devices to udev, and (say) the hda device file gets created with the proper major and minor number.

(IDE has a very specific order defined: primary master is hda, etc., etc. SATA and SAS (serial SCSI) don't do this, and normal SCSI doesn't have to either: when you create a SCSI-subsystem disk, it takes whichever major number and letter are next in line. This is even more chaotic with the introduction of parallel SCSI scanning in one of the recent 2.6 kernels, but it doesn't matter if you use the persistent symlinks provided by udev.)

When userspace makes an open() or read() or whatever call into the kernel on any block device, that call gets forwarded to the code that registered itself as the handler for that block device (and the lookup is done by the block major number). (Character devices are handled the same way, but character-major-3 is distinct from block-major-3.) This code is inside the IDE subsystem, I think; it forwards the request to whichever driver registered hda or hdb, depending on the minor number. (Because both hda and hdb use major number 3.)

Therefore if the hda is dropped then there will be no block device with a major number 3 presumably Correct: nothing on the system will be registered to handle block major 3. If you try to create an hda file with major 3 and minor zero, you'll get an ENXIO error trying to do anything with it (this corresponds to the "no such device" error message).

and if it is the only disk then the sda reference will be used. Well, the kernel will try to use whatever device file you try to open. But if the file isn't there, or has no driver registered, then you get errors. :)

Would I be correct in thinking the kernel keeps a record of the hardware link (of the actual disk in the form of interupt or address of some kind) corresponding to the reference (block, 8, 0)? Sort of, indirectly.

The kernel keeps track of the file-operations structure that gets registered to handle block major 3. This structure has pointers to an open function, read and write functions, seek functions, an ioctl handler function, a close function, and possibly a few other bookkeeping functions. When user code tries to do anything with that file, the kernel's device-file handler code looks up the proper file-operations structure, and calls the appropriate function.

That function in turn looks up which lower-level driver (IDE, libata, real SCSI, SAS, or even SCSI-over-a-network) handles the type of request it has to do (e.g. read X blocks starting at offset Y from the disk), and makes that request to the lower-level driver. The lower-level driver in turn tells the controller that it drives to do whatever needs to be done, and the controller signals the lower-level driver (via an interrupt) when the procedure is finished. (DMA, etc. is all involved here.)

I would be interest to see that record even if I cant make out any sense out of it. I'm not sure it's available, actually. The closest thing might be some of the structure in /sys. For instance, /sys/block/hda/ has a symlink named "device" in it, that points at the sysfs directory for the physical device that corresponds to hda. (On my system, that's ../../devices/pci0000:00/0000:00:12.0/ide0/0.0, or /sys/devices/<blah>.)

Then, that /sys/devices/... directory has a "subsystem" symlink that points at the subsystem that this driver is driven by (in my case, /sys/bus/ide). Also, if you go up two directories (to /sys/devices/pci0000:00/0000:00:12.0 on my system), you get a bunch of entries corresponding to the disk controller: a symlink to its driver's directory, for instance (my driver symlink points at the "ALI15x3_IDE" driver).

But the structure of the function calls isn't really given anywhere -- probably because it's too deep.

[saikee]

bwkaz,

I shall have to read your post many times to understand the subject better but it has more or less confirmed my thought on the subject. Your explanation helps to clear my doubt of hanging onto the hda convention as I now know why I wouldn't get anywhere once the switch becomes permanent,

I have not expect to understand the registers, interrupts, addressess and the like deep down at the hardware level and need you to tell me where to stop.

I feel sad to see Pata convention of hda to hdd goes as so far it is hardware and location specific and doesn't change in any Linux. However I can move with the progress in the kernel. The pace of Linux development has been breathtaking.

Thank again for your enlightment.

[saikee]

I have recently assembled a new PC with an Intel Core 2 Duo CPU on a 975x chipset mobo and so loaded a couple of Linux to try the hardware out as it was reported earlier Linux wasn't supporting the 965 chipset, let alone the 975x. I was pleasantly surprised all the Linux performed superbly without any driver other than those inside the distros. The only drivers I fed were for a XP which needs assistance to recognise a Sata, nic, sound, mobo and of course the video.

To play safe I use the current distros and that was why I run in to the Pata disk name change. The new mobo has only 1 IDE instead of the normal 2 IDE channels. What is more curious is the IDE channel is the secondary and the Sata I installed comes up as the primary master in the Bios. I was using only a Sata and so didn't run into the Pata disk name change problem initially.

Due to this thread I investigate by sticking a 300Gb Pata with 63 partitions inside. Here is the results

Ubuntu (2.6.20) sees the Pata as sdb but cannot display it due to "unable to seek"
Puppy (2.6.18) sees the Pata as hdd and able to access the full 63 partitions
Salx (2.6.20) sees the Pata as hdd and able to access the full 63 partitions
Fedora (2.6.20) sees the Pata as sdb and able to access only the first 15 partitions
Debian (2.6.18) sees the Pata as sdb but cannot display it due to "unable to seek"

Slax is an odd one out as its 2.6.20 kernel still recognise Pata partitions. We could, however, discount it for being a small distro many Linux features were fully implemented in this distro.

FC6 see only 15 partitions in the Pata disk may deserve a credit here. It manages this because it is the only distro (including everyone of its family) that reports no more than 15 partitions in every Pata disk even before the kernel 2.6.20. However it does mount, access and work with higher Pata partitions with older kernels. I have proof of this because I have operational FC3 installed in hda59, FC4 in hda22 and FC5 in hdc56.

Ubuntu is publicly known to throw away the Pata disk convention because of its popularity. I give it credit in refusing to read the 63-partition Pata because a sdb, for being a SCSI/Sata/USB disk the sdb is not supposed to have more than 15 partitions and so it regards my disk as foreign. As Ubuntu does not know how to handle the extra partitions it plays safe and refuse sto have anything to do with it. That is what a robust OS supposed to do.

Debian 4 is a strange setup. Its kernel is supposed to be able to read the hdd but it reported "unable to seek" in sdb and there is no hdd reported. Thus Debian might have implemented the new libATA drivers to follow the modern trend. That is no different to distros with kernel older than 2.6 can support Sata disks, like the current Slackware 11.

No doubt in due time it would be all clear but the Pata disk name change can have a big impact on many users updating kernels or introducing new distros into existing systems.

[bwkaz]

Salx (2.6.20) sees the Pata as hdd and able to access the full 63 partitions I thought I said this earlier, but the new IDE-over-libata drivers are optional at this point. Just because you're using 2.6.20 (or 2.6.20.x, or 2.6.21, or 2.6.21.x) doesn't mean you're using libata. If this distro sees the device as hdd, then it'll work fine as hdd; that means this distro's kernel doesn't have IDE-over-libata enabled by default.

:)

FC6 see only 15 partitions in the Pata disk may deserve a credit here. It manages this because it is the only distro (including everyone of its family) that reports no more than 15 partitions in every Pata disk even before the kernel 2.6.20. However it does mount, access and work with higher Pata partitions with older kernels. I have proof of this because I have operational FC3 installed in hda59, FC4 in hda22 and FC5 in hdc56. This may be because it just doesn't create the device files for higher partitions. Its kernel is probably able to use them (and since the devices show up as hdX instead of sdX, it's not using the SCSI layer); if you create them manually it'll probably work just fine.

Not sure why they would have decided to do that, but it may have been because they figured nobody would use that many partitions. Or it may have been because they knew about the move to libata and wanted to make sure none of their users ran into problems.

Debian 4 is a strange setup. Its kernel is supposed to be able to read the hdd but it reported "unable to seek" in sdb It reported this after you did... what exactly? After you tried to run "fdisk -l" or equivalent? Or on bootup? Or when you did something else? That makes a difference. :)

Thus Debian might have implemented the new libATA drivers to follow the modern trend. Only if the device showed up as /dev/sdXY... If it didn't show up at all, then I can't say what they did.

No doubt in due time it would be all clear but the Pata disk name change can have a big impact on many users updating kernels or introducing new distros into existing systems. Not really. In order to be bitten by this, you would have to, first, avoid using the udev persistent-disk symlinks. This change is what those symlinks exist to handle. Then, you'd have to either (a) avoid reading any information at all about this until several years down the road, when you upgrade to a kernel that doesn't have the legacy IDE drivers, or (b) purposefully disable the old-IDE driver when you built your new kernel (and enable the new IDE-libata driver).

If the old-IDE driver is enabled at all in 2.6.20.x, then it gets used. The libata driver is not the default yet, if both are present.

[saikee]

bwkaz,

I reported the various Linux behaviors as a testimony of your in-depth knowledge in the subject.

The Fedora behaviour is exactly as you have predicted. You taught me how to create the device names in this thread (http://www.justlinux.com/forum/showthread.php?t=147954&highlight=device) when the distro did not do it. I was therefore able to follow your guidance and installed Fedora in partition it never dreams possible.

For Debian when I issued "fdisk -l" it listed out sda but reported "unable to seek" in sdb. It then reported no such device when I issued command "fdisk /dev/hdd", which Puppy and Slax were able to display up to 63th partition. To me if Debian name the same Pata as a sdb and has no knowledge of hdd then it must be using the libATA driver in its 2.6.18 kernel now.

On the switching of the Pata and Sata disk name the issue can be complicated in practice. A good example is the current Ubuntu Christian 3.1 which is so new that it is still displaying in the front page of Distrowatch.com.

When I try to install this distro into a box with several Pata disks each having 60+ partitions the boot-up Linux see all of them as Pata partitions when I listed them in "fdisk -l". If I try to install then the installer will get stuck in the section of

"Starting the partitioner"

and remains there forever. Obviously the LIve CD recognises the Pata partitions but the installer doesn't. I guess it has to stop because the installer cannot cope with apparently-renamed Sata disks with more than 15 partitions inside. In fact I always have a bit of bother from the Ubuntu family seeing my Pata disks as their installer frequently get stuck at the same place. I could install the same distro in a simpler disk with smaller number of partitions and then migrate it back to the a disk with a large number of partitions. In future I won't be able to do it and have conform to the 15 maximum number of partitions in each disk.

[bwkaz]

For Debian when I issued "fdisk -l" it listed out sda but reported "unable to seek" in sdb. Hmm.

Any errors from the kernel when this happens? That doesn't sound right...

(Actually, is this a SATA controller running under IDE compatibility mode in the BIOS? Or is it a real 80-pin PATA connection?)

It then reported no such device when I issued command "fdisk /dev/hdd", So it either doesn't have a driver for your IDE chipset, or it has some other kind of bug. Or it's using libata, but see below.

To me if Debian name the same Pata as a sdb and has no knowledge of hdd then it must be using the libATA driver in its 2.6.18 kernel now. AFAIK 2.6.18 can't run PATA-over-libata. But I'm not positive on that; it may have been possible for the Debian people to backport it. However, it's still quite new, and it seems like it'd be a really bad idea for a huge distro like that to use it quite yet. If I were deciding on things like that, I'd wait until at least 2.6.22, but probably longer. (That will be three releases for it to have stabilized; the drivers were introduced in 2.6.19.)

[saikee]

Straight from Debian terminal
debian:/home/saikee# fdisk -l

Disk /dev/sda: 500.1 GB, 500107862016 bytes
255 heads, 63 sectors/track, 60801 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/sda1 * 1 12158 97659103+ 7 HPFS/NTFS
/dev/sda2 12159 36473 195310237+ 7 HPFS/NTFS
/dev/sda3 36474 48755 98655165 5 Extended
/dev/sda4 48756 60801 96759495 7 HPFS/NTFS
/dev/sda5 36474 36595 979933+ 82 Linux swap / Solaris
/dev/sda6 36596 37811 9767488+ 83 Linux
/dev/sda7 37812 39027 9767488+ 83 Linux
/dev/sda8 39028 40243 9767488+ 83 Linux
/dev/sda9 40244 41459 9767488+ 83 Linux
/dev/sda10 41460 42675 9767488+ 83 Linux
/dev/sda11 42676 43891 9767488+ 83 Linux
/dev/sda12 43892 45107 9767488+ 83 Linux
/dev/sda13 45108 46323 9767488+ 83 Linux
/dev/sda14 46324 47539 9767488+ 83 Linux
/dev/sda15 47540 48755 9767488+ 83 Linux

Unable to seek on /dev/sdb
debian:/home/saikee# fdisk /dev/hdd

Unable to open /dev/hdd
debian:/home/saikee# fdisk /dev/sdb

The number of cylinders for this disk is set to 16709.
There is nothing wrong with that, but this is larger than 1024,
and could in certain setups cause problems with:
1) software that runs at boot time (e.g., old versions of LILO)
2) booting and partitioning software from other OSs
(e.g., DOS FDISK, OS/2 FDISK)

Unable to seek on /dev/sdb
debian:/home/saikee# uname -a
Linux debian 2.6.18-4-686 #1 SMP Wed May 9 23:03:12 UTC 2007 i686 GNU/Linux
debian:/home/saikee#


The mobo is a Abit AWD9 MAX with one IDE channel with a ATA 100/66/33 IDE connector. I have a 80-conductor cable connecting the CD/DVD/writer and a Pata disk (in a mobile rack). There are two Sata controller too; a Intel ICH7R and Silicon Image 3132 for up to 7 Sata disks.

I have Puppy, with a kerenl 2.6.18, installed in the same hard disk in sda7 and it sees the full complement from hdd1 to hdd63.

The information in the terminal suggests Debian sees a hard disk under the name of sdb but was unable to seek. That disk has 63 partitions and would be illegal or impossible as a sdb. To me the partitioning program fdisk cannot display the partitions beyond the 15th and so it plays safe by not having anything to do with it.

[bwkaz]

The information in the terminal suggests Debian sees a hard disk under the name of sdb but was unable to seek. Which was why I mentioned the kernel logs -- I really don't think that fdisk has any clue that it won't be able to use large partition numbers on SCSI block devices. Instead, I think it's trying to seek to the next partition in the partition table (to find out where the *next* partition is: all the logical drives are in a huge linked list on the disk), and the seek is failing for some reason. (In other words, fdisk is using only the hdd/sda/sdb device, not the partition-number devices. It's interpreting the partition table on its own.) The issue here is that fdisk isn't giving you the error that it got from the kernel seek call, it's just saying "failed".

Perhaps this kernel doesn't have support for large block devices, or large files? Or perhaps this version of fdisk wasn't compiled with large-file support? If it's the kernel, then there should be some message in the kernel logs; if it's fdisk, then you'd probably have to fix it to report the error to you. (Actually you may have to do that anyway...)

What happens if you fire up hexedit on the raw disk device on that system, and try to seek to an offset larger than 4GB? (0xffffffff) Does it give you any better error?