Linux/m68k ext2fs byteswapping mini-HOWTO

Michael Schmitz <schmitz@lcbvax.cchem.berkeley.edu>


0) Preamble

If you have installed Debian or Red Hat on your machine from scratch, you should not need to follow these instructions; all recent versions of the ext2fs utilities have been creating little-endian format partitions.

To check to see whether you have the little-endian format, use the following command as root: dumpe2fs device | head, replacing device with the name of the device to be checked (e.g. /dev/sda1). If the device needs to be byte-swapped, you will see Note: This is a byte-swapped file system.

If you have no devices that need to be byte-swapped, you can safely ignore this HOWTO.

1) Scope and structure of this document

This document describes the necessary steps you need to take in order to convert old-style (that is, native m68k byte order) ext2 filesystems for use with kernel release 2.1.21 and later.

It is intended for the general Linux-m68k user running a system (and probably administrating it in some way or other), and assumes you are familiar with basic system maintainance tasks and general Unix concepts.

If you don't know what ext2fs, byteorder and so on is all about, and if you don't run a 2.1 kernel, please read on anyway - it might apply to you at a later stage. (If the term 'm68k' doesn't tell you anything, however, this document is definitely only half as entertaining :-)

Especially if you saw the lines

EXT2-fs warning: old big-endian filesystem detected!
EXT2-fs warning: support for this filesystem will disappear in future kernels.
EXT2-fs warning: please run 'e2fsck -s' from e2fsprogs-1.06-fix or later
EXT2-fs warning: to convert it to the new standard.
you should take the steps described in this document SOON, as the time for the big-endian ext2fs runs out real soon now.

The following sections describe briefly what all this byteswapping stuff is about (well, not too briefly; please come up with a more concise description :-), what information about your particular setup is required to successfully convert your old ext2 partitions, and what files you need to download to do this. The last two sections describe the actual conversion procedure for two possible scenarios.

2) Background information

Linux users should be familiar with the term 'filesystem' and with the Linux 'default filesystem', the second extendef filesystem or ext2fs in particular. So I won't waste time explaining the details here, just a few comments on what this recent change in the m68k ext2fs code is about, and how it will affect you.

Both Linux-i386 and Linux-m68k use the same ext2 filesystem (and all the other Linux ports as well), but did you ever try to mount a ext2 fs created on m68k on some intel box?? It simply isn't recognized, because the data stored on disk that comprises administrative information about filesystem layout, free and occupied space and so on (the so-called 'metadata' as opposed to 'user data', what you finally get when reading a file or directory) is in a different byte order. The user data itself differs in byte order as well, but you would have expected that anyway, right?

For about a year now, the m68k ext2fs code was equipped to recognize both big endian (native) and little endian (intel) ext2fs format (by simply looking at the superblock magic number and deciding of it made sense immediately or only after byteswapping). The Intel kernel (which regards itself as the mainstream version, that's kind of a 'Right Thing (tm)' issue) never acquired that feature, and we're going to be the Good Guys and abandon it again when finally merging with the mainstream source distribution in the near future. Kernel release 2.1.21 is in fact the first release without big endian ext2fs support.

So what's the consequence of switching to little endian ext2fs only? Mainly that there's only one ext2fs format in future, and every Linux system regardless of its architecture and platform can read and write disks from every other system (might be interesting for removable disks in particular).

A little drawback is that all systems for which the filesystem isn't in native byte order anymore must do a little extra work inside the kernel when reading or writing the metadata, as all multi-byte-fields of the ext2fs data structures have to be converted from or to little endian format. For practical purpose, that overhead never showed up to be even detectable as far as I recall.

The major drawback for m68k users that started back in the good old days, when the world was mainly big endian, is the need to convert their old ext2 filesystems (which are still big-endian and were supported until recently). This conversion is done by a specially equipped e2fsck and comprises rewriting all filesystem metadata on disk to little endian format. This is a potentially hazardous step (just imagine the e2fsck crashing while half-way done with the conversion) leaving you with a inconsistent filesystem if it fails, so you might want to back up your i partitions before. But on the other hand, the conversion has been done by a good deal of testers, and errors have been sorted out during this procedure (I've converted my last native endian partition a week ago, and didn't encounter any problems; I even did it while running 2.1.21 having neglected the issue a bit too long ...). So it's probably safe to convert your filesystem without backing it up (but YMMV as usual!).

Now I hear some mumbling 'I remember having done such a byteswapping before - can't recall details, but I'm quite sure ...?' ... You're right there - but that was another story, or rather several other old stories (last seen when the big-endian m68k ext2fs format was changed with respect to the bitmap byte order, switching to little- endian bitmap format in 1.2.10 and again in 1.2.13pl4). And it's a long time ago (Dec. 1995), so you probably want to try this current byteswap craze as well. Simply boot kernel release 2.0.28 and look out for ext2fs warnings ...

What's the difference, this time ?? Simply the fact that _all_ metadata are converted now, and the 1.2.13pl4 changes affected only the bitmaps.

I'm well aware of all the trouble that happened during the 1.2.13pl4 conversion period - that has been the main reason for supporting two different byte orders for some time while encouraging people to make the conversion at their convenience some time during that period. I've been opposing to doing anything that requires yet another byteswapping ever again for a long time - but finally, when left with the choices of integrating the m68k sources into the main kernel tree with switching (and even giving up support for native byte order format) or having to patch Linus' changes into a different source every time, you can't reasonably chose the latter :-) I hope this document helps a little avoiding problems that might occur when trying 'just run e2fsck -s' ...

3) Required information about your setup you need during the procedure

Basically, two kinds of information are required to do the filesystem conversion: How to determine which ext2 filesystems need to be converted?

Simple: boot the 2.0.28 m68k kernel, and watch out for any 'ext2fs warnings' during the filesystem mount stage (and make sure you mount all your ext2 partitions).

Write down the partition names of all partitions generating a warning message (partition names are of the form [sd, hd, ad, ...][a-z][1-16] ) on a piece of paper, and refer to these names later in the procedure.

How to determine the name of the swap partition?

Look at /etc/fstab; there should be at least one entry like

/dev/sda4	swap	swap	none	0	0
or
/var/swapfile	swap	swap	none	0	0
The first refers to a dedicated swap partition, just write down the partition name in the first column.

The second refers to a swap file on a regular data partition, and here you need the name of the file as well as the name of the partition on which it resides. The filename appears in the first column again, try to figure out the partition name from the path name and your mount scheme defined in /etc/fstab.

4) Files required for conversion

To perform the ext2fs byteswap, the e2fsck binary was extended to do a new pass before the regular consistency checking if run with either the '-s' or '-S' switch. If -s is specifed, during this additional pass all metadata is converted to little endian format, and if -S is specified, this pass converts all metadata to big endian format (might be helpful if you want to use the filesystem with a kernel release prior to 2.0.15).

Make sure you have a recent version of e2fsck that not only supports these options, but has a bug fixed that caused the conversion to fail in rare cases. The correct binary can be found on the major m68k ftp servers as part of the e2fsprogs-1.06-swap-fixed archive. For the Debian people: version 1.06-4.1 (or later) is safe!

If you chose to do the conversion from a ramdisk filesystem image, make sure you have a ramdisk that contains the right version of e2fsck (the most recent ramdisk image you should use is currently Geert Uytterhoeven's filesys-ELF-2.0.x-1400K-2.gz, located in the subdirectory v2.0/filesys under the Linux-m68k area on ftp.uni-erlangen.de and its mirrors. The ramdisk takes up 1.4 MB when booted, and that might be a bit much for low-RAM systems. Please report to the linux-m68k mailing list or to the newsgroup if you run into trouble with a 4MB RAM system!).

Now for the real beef: How to do the conversion to little endian byte order on old ext2 filesystems?

A little caveat ahead: you should probably do the conversion steps running a 2.0 kernel or anything prior to 2.1.21 (but as far as I recall 1.2.13pl10 or earlier will probably not work, at least not with the mentioned ramdisk filesystem - this in fact requires 2.0.18 or later kernels).

I've successfully run e2fsck -p and e2fsck -s on a native filesystem using 2.1.21 after the mount of this filesystem failed, but that's definitely not for the faint at heart ...

5) Doing the conversion from a booted hard disk partition

(I'm silently assuming you've installed the most recent version of e2fstools, are running on plenty of free RAM or have activated swap before attempting to do the following steps ...)

There are two ways to do the filesystem conversion: Every filesystem except for the root filesystem may be converted while running from a booted hard disk filesystem. Just unmount the filesystem, and run the command

# e2fsck -s /dev/<your filesystem name here>
where <your filesystem name here> is replaced with the Linux device name of the filesystem to be converted, i.e. hda4 or sda5. If you're unsure what names to use there, look at /etc/fstab ot at the 'mount' output (before unmounting it, of course).

i.e. 'e2fsck -s /dev/sda5' was used to convert the old-style filesystem on partition 5 of my first SCSI disk.

If e2fsck complains about the filesystem being not properly unmounted (as will happen if you attempt to mount an old filesystem with 2.1.21) or tells you it needs a 'freshly checked filesystem', just run e2fsck -f before e2fsck -s (but you had better respected that caveat before anyway).

That's all - repeat for each partition you wrote down in step 3, and mount them again!

If you need to convert the root partition, however, refer to the next section (or resort to another root partition you might happen to keep around for emergencies. But of course byteswap that one first :-)).

6) Doing the conversion using a ramdisk filesystem

The second way to do the conversion is to boot your system from a ramdisk filesystem image. This way, you don't have to care for proper unmounting of the partitions in the right order (and the root partition cannot be unmounted anyway). But this method has its drawbacks as well:

Especially if using the ramdisk method on a system with only 4MB RAM the main memory may be exhausted fast, so take care to use a kernel with only the minimum number of drivers configured and/or boot in the lowest possible screen resolution.

e2fsck requires huge amounts of memory sometimes, so make sure you activate swapping to your usual swap partition, using the command

# swapon /dev/<my swap partition> 
as soon as you have booted the ramdisk and logged in.

<my swap partition> should be changed to the name of your swap partition as you figured out before; if you are using a swap file on one of your partitions instead of a dedicated swap partition, you're a bit out of luck here as you need to mount the partition holding the swap file in order to use it. Hence, you cannot convert the partition holding the swap file itself; you'll need at least to set up a temporary swap file while converting the partition holding your original swap file.

(You can use 'dd' to make a suitably sized file, mkswap <filename> to create a swap area on this file, and finally swapon <filename> to activate it for this conversion run.)

Boot the ramdisk using the command line '-r filesys root=/dev/ram' (plus additional switches you might need for your setup) with amiboot or ataboot (assuming the compressed filesystem image was copied to 'filesys' on your disk).

Log in to the ramdisk system, activate swap (see above) and convert your root partition (or all of your partitions) using the command

# e2fsck -s /dev/<partition name>
as described above.

<partition name> is the Linux name of the partition you want to convert (you wrote these on a piece of paper earlier on, didn't you?)

If e2fsck complains about the filesystem being not properly unmounted or tells you it needs a 'freshly checked filesystem', just run e2fsck -f before e2fsck -s, as mentioned before.

Consult the README files coming with the ramdisk filesystem for details on how to log on (mostly 'root' as username without a password), and consult the e2fsck and swapon, mkswap and dd manpages for details on these programs if you need further information.

7) Bug reports, suggestions, acknowledgements etc...

I'd like to acknowledge Andreas Schwab for providing the diffs for e2fstools whenever a problem showed up, and Geert Uytterhoeven for providing the minimum install-and-convert ramdisk used to convert the root filesystem.

Please direct bug reports (problems you encounter when following the procedure outlined in this document) to the Linux-m68k mailing list <linux-m68k@lists.linux-m68k.org>, there will be always someone out there to read your complaint and hopefully help you.

Any suggestions to improve this description is greatly appreciated, please mail to the list or directly to me on these matters.


Michael Schmitz <schmitz@lcbvax.cchem.berkeley.edu> 6-Feb-1997