I demand that Ant may or may not have written...

>>> I did something stupid earlier. I accidently ran fsck command on a
>>> mounted HDD in Debian. I quickly ctrl-c'ed to get out of it. Now, I am
>>> having HDD problems and I can't seem to recover/undo what I did. I
>>> rebooted and my box didn't come back (can't check the box since I am at
>>> work and doing it via SSH).

>>> dmesg showed this after I before I rebooted it remotely with shutdown -r
>>> now at: http://pastebin.ca/672751
>> [sample from that paste follows]
>>> [...] ext3_free_blocks_sb: bit already cleared for block 228823
>>> Remounting filesystem read-only
[more "bit cleared" errors, and some "bad entry in directory" errors]

>> Ouch.

>> It looks like some objects were updated while fsck was running, and fsck
>> has consequently overwritten newer metadata with older. Your ^C may well
>> have prevented further damage...

> Oh good.

.... and here's the bad news: (AIUI) fsck tries to write everything which it's
changed in one go when it's finished scanning the partition.

>>> Then, I decided to reboot and nothing came back (can't SSH back in). I
>>> think it got stuck somewhere?
[snip]
>>> If it indeed stuck, how do I fix this after I get home to see the
>>> console?
>> Logging in & re-running fsck should be fine, but you should take that
>> advice about using a CD and making a copy (consider "belt & braces").
>> (The Debian install CD will be fine for this.)

> I have KNOPPIX v4.1 CD so I hope that's enough. I always have this CD
> and the latest versions for emergencies for like this one!

If it's new enough or your hardware is old enough, it should be fine...

>> If you do log in and run fsck directly, you should find that the damaged
>> partition is either mounted read-only or not yet mounted.

> Do I just run fsck /dev/hda and it let run with defaults?

Yes (well, /dev/hda1), but expect some questions and expect to have to do
some cleanup work (as mentioned elsewhere in this thread) afterwards.

Again, consider making a copy of the unrepaired partition in case something
goes wrong or you need a reference point.

--
| Darren Salt | linux or ds at | nr. Ashington, | Toon
| RISC OS, Linux | youmustbejoking,demon,co,uk | Northumberland | Army
| + Output less CO2 => avoid massive flooding. TIME IS RUNNING OUT *FAST*.

I'd like to, but I'm observing National Apathy Week.

> >>> I did something stupid earlier. I accidently ran fsck command on a
> >>> mounted HDD in Debian. I quickly ctrl-c'ed to get out of it. Now, I am
> >>> having HDD problems and I can't seem to recover/undo what I did. I
> >>> rebooted and my box didn't come back (can't check the box since I am at
> >>> work and doing it via SSH).

> >>> dmesg showed this after I before I rebooted it remotely with shutdown -r
> >>> now at: http://pastebin.ca/672751
> >> [sample from that paste follows]
> >>> [...] ext3_free_blocks_sb: bit already cleared for block 228823
> >>> Remounting filesystem read-only
> [more "bit cleared" errors, and some "bad entry in directory" errors]

> >> Ouch.

> >> It looks like some objects were updated while fsck was running, and fsck
> >> has consequently overwritten newer metadata with older. Your ^C may well
> >> have prevented further damage...

> > Oh good.

> ... and here's the bad news: (AIUI) fsck tries to write everything which it's
> changed in one go when it's finished scanning the partition.
>
> >>> Then, I decided to reboot and nothing came back (can't SSH back in). I
> >>> think it got stuck somewhere?
> [snip]
> >>> If it indeed stuck, how do I fix this after I get home to see the
> >>> console?
> >> Logging in & re-running fsck should be fine, but you should take that
> >> advice about using a CD and making a copy (consider "belt & braces").
> >> (The Debian install CD will be fine for this.)

> > I have KNOPPIX v4.1 CD so I hope that's enough. I always have this CD
> > and the latest versions for emergencies for like this one!

> If it's new enough or your hardware is old enough, it should be fine...

> >> If you do log in and run fsck directly, you should find that the damaged
> >> partition is either mounted read-only or not yet mounted.

> > Do I just run fsck /dev/hda and it let run with defaults?

> Yes (well, /dev/hda1), but expect some questions and expect to have to do
> some cleanup work (as mentioned elsewhere in this thread) afterwards.

> Again, consider making a copy of the unrepaired partition in case something
> goes wrong or you need a reference point.

Well, I don't really have the storage for it. I copied my data/work
datas though and i have a backup of /etc.

Anyways, bad news:

I came home. I saw a bunch of password errors for sulogin or something.
I couldn't type anything. I was able to press numlock to see light turn
on and off. I couldn't reboot with the three keys so I had to power
shutdown. I repowered back on and Grub said error code 2. It didn't say
much in Grub's Web site.

I booted up KNOPPIX v5.0 (not 5.1.1) and I couldn't mount the hda1 in
KDE: "mount: wrong fs type, bad option, bad superblock on /dev/hda1." I
ran fsck /dev/hda and got:

root@1[knoppix]# fsck /dev/hda
fsck 1.39-WIP (09-Apr-2006)
e2fsck 1.39-WIP (09-Apr-2006)
Couldn't find ext2 superblock, trying backup blocks...
fsck.ext2: Bad magic number in super-block while trying to open /dev/hda

The superblock could not be read or does not describe a correct ext2
filesystem. If the device is valid and it really contains an ext2
filesystem (and not swap or ufs or something else), then the superblock
is corrupt, and you might try running e2fsck with an alternate
superblock:
e2fsck -b 8193 <device>


I ran fdisk /dev/hda to see if I can read it at all:
Command (m for help): p

Disk /dev/hda: 80.0 GB, 80026361856 bytes
255 heads, 63 sectors/track, 9729 cylinders
Units = cylinders of 16065 * 512 = 8225280 bytes

Device Boot Start End Blocks Id System
/dev/hda1 1 36 289138+ 83 Linux
/dev/hda2 37 9729 77859022+ 5 Extended
/dev/hda5 37 1860 14651248+ 83 Linux
/dev/hda6 1861 2468 4883728+ 83 Linux
/dev/hda7 2469 2590 979933+ 83 Linux
/dev/hda8 2591 2712 979933+ 83 Linux
/dev/hda9 2713 3320 4883728+ 83 Linux
/dev/hda10 3443 3736 2361523+ 83 Linux
/dev/hda11 3737 9729 48138741 83 Linux
/dev/hda12 3321 3442 979933+ 83 Linux

Partition table entries are not in disk order


I am going to try e2fsck -b 8193 /dev/hda (heard ext3 is based on ext2
with journaling).

It looks like I am going to need to reinstall from scratch if it is not
recoverable.

Note: My HDDs are just IDE. Nothing fancy.
--
"He who cannot pick up an ant, and wants to pick up an elephant will
some day see his folly." --African
/\___/\
/ /\ /\ \ Ant @ http://antfarm.ma.cx (Personal Web Site)
| |o o| | Ant's Quality Foraged Links (AQFL): http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

In alt.os.linux.debian Ant <ANTant@zimage.com> wrote:
> > >>> I did something stupid earlier. I accidently ran fsck command on a
> > >>> mounted HDD in Debian. I quickly ctrl-c'ed to get out of it. Now, I am
> > >>> having HDD problems and I can't seem to recover/undo what I did. I
> > >>> rebooted and my box didn't come back (can't check the box since I am at
> > >>> work and doing it via SSH).

> > >>> dmesg showed this after I before I rebooted it remotely with shutdown -r
> > >>> now at: http://pastebin.ca/672751
> > >> [sample from that paste follows]
> > >>> [...] ext3_free_blocks_sb: bit already cleared for block 228823
> > >>> Remounting filesystem read-only
> > [more "bit cleared" errors, and some "bad entry in directory" errors]

> > >> Ouch.

> > >> It looks like some objects were updated while fsck was running, and fsck
> > >> has consequently overwritten newer metadata with older. Your ^C may well
> > >> have prevented further damage...

> > > Oh good.

> > ... and here's the bad news: (AIUI) fsck tries to write everything which it's
> > changed in one go when it's finished scanning the partition.
> >
> > >>> Then, I decided to reboot and nothing came back (can't SSH back in). I
> > >>> think it got stuck somewhere?
> > [snip]
> > >>> If it indeed stuck, how do I fix this after I get home to see the
> > >>> console?
> > >> Logging in & re-running fsck should be fine, but you should take that
> > >> advice about using a CD and making a copy (consider "belt & braces").
> > >> (The Debian install CD will be fine for this.)

> > > I have KNOPPIX v4.1 CD so I hope that's enough. I always have this CD
> > > and the latest versions for emergencies for like this one!

> > If it's new enough or your hardware is old enough, it should be fine...

> > >> If you do log in and run fsck directly, you should find that the damaged
> > >> partition is either mounted read-only or not yet mounted.

> > > Do I just run fsck /dev/hda and it let run with defaults?

> > Yes (well, /dev/hda1), but expect some questions and expect to have to do
> > some cleanup work (as mentioned elsewhere in this thread) afterwards.

> > Again, consider making a copy of the unrepaired partition in case something
> > goes wrong or you need a reference point.

> Well, I don't really have the storage for it. I copied my data/work
> datas though and i have a backup of /etc.

> Anyways, bad news:

> I came home. I saw a bunch of password errors for sulogin or something.
> I couldn't type anything. I was able to press numlock to see light turn
> on and off. I couldn't reboot with the three keys so I had to power
> shutdown. I repowered back on and Grub said error code 2. It didn't say
> much in Grub's Web site.

> I booted up KNOPPIX v5.0 (not 5.1.1) and I couldn't mount the hda1 in
> KDE: "mount: wrong fs type, bad option, bad superblock on /dev/hda1." I
> ran fsck /dev/hda and got:

> root@1[knoppix]# fsck /dev/hda
> fsck 1.39-WIP (09-Apr-2006)
> e2fsck 1.39-WIP (09-Apr-2006)
> Couldn't find ext2 superblock, trying backup blocks...
> fsck.ext2: Bad magic number in super-block while trying to open /dev/hda

> The superblock could not be read or does not describe a correct ext2
> filesystem. If the device is valid and it really contains an ext2
> filesystem (and not swap or ufs or something else), then the superblock
> is corrupt, and you might try running e2fsck with an alternate
> superblock:
> e2fsck -b 8193 <device>


> I ran fdisk /dev/hda to see if I can read it at all:
> Command (m for help): p

> Disk /dev/hda: 80.0 GB, 80026361856 bytes
> 255 heads, 63 sectors/track, 9729 cylinders
> Units = cylinders of 16065 * 512 = 8225280 bytes

> Device Boot Start End Blocks Id System
> /dev/hda1 1 36 289138+ 83 Linux
> /dev/hda2 37 9729 77859022+ 5 Extended
> /dev/hda5 37 1860 14651248+ 83 Linux
> /dev/hda6 1861 2468 4883728+ 83 Linux
> /dev/hda7 2469 2590 979933+ 83 Linux
> /dev/hda8 2591 2712 979933+ 83 Linux
> /dev/hda9 2713 3320 4883728+ 83 Linux
> /dev/hda10 3443 3736 2361523+ 83 Linux
> /dev/hda11 3737 9729 48138741 83 Linux
> /dev/hda12 3321 3442 979933+ 83 Linux

> Partition table entries are not in disk order


> I am going to try e2fsck -b 8193 /dev/hda (heard ext3 is based on ext2
> with journaling).

> It looks like I am going to need to reinstall from scratch if it is not
> recoverable.

> Note: My HDDs are just IDE. Nothing fancy.

See http://pastebin.com/m5ca8c3f0 first.


I basically held down Y for like a few minutes during fsck /dev/hda1.

I tried to mount this partition with KDE, but it didn't see anything (uh
oh). All other partitions except the swap one seemed fine.

I am going to go reboot.
--
"He who cannot pick up an ant, and wants to pick up an elephant will
some day see his folly." --African
/\___/\
/ /\ /\ \ Ant @ http://antfarm.ma.cx (Personal Web Site)
| |o o| | Ant's Quality Foraged Links (AQFL): http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

I demand that Ant may or may not have written...

> In alt.os.linux.debian Ant <ANTant@zimage.com> wrote:
[huge snip]
>> See http://pastebin.com/m5ca8c3f0 first.
>> I basically held down Y for like a few minutes during fsck /dev/hda1.

>> I tried to mount this partition with KDE, but it didn't see anything (uh
>> oh). All other partitions except the swap one seemed fine.

>> I am going to go reboot.

> Grub 1.5 now says error 15 meaning file not found according to Google
> search results. Crap, hda1 is still empty. AUGH!

Well, at least you have /var - you can make use of dpkg's --get-selections
and --set-selections options.

> Back in KNOPPIX, I found out where all the datas went in hda1. They all
> ended up in lost+found/ directory: http://pastebin.ca/673132 ...

Anything which you've customised and can identify from that lot (fstab and
xorg.conf, for example), you should restore after reinstalling.

You may be able to do a minimal install in your root partition, after which
you can mount your existing /usr and /var then just force reinstallation of
(more or less) all packages. Afterwards, you can unmount /usr and /var then
remove the content which was hidden by the mount points.

(Untested. This advice is given without warranty or any guarantee of fitness
for purpose. May cause nasal demons. No sacrifice necessary.)

> I thought I could move these back, but I realized I am missing other
> files like Kernel images. I think I am stuck and need to do a reinstall
> soon.

Looks like it...

--
| Darren Salt | linux or ds at | nr. Ashington, | Toon
| RISC OS, Linux | youmustbejoking,demon,co,uk | Northumberland | Army
| + Burn less waste. Use less packaging. Waste less. USE FEWER RESOURCES.

I'm a figment of my imagination.

In comp.os.linux.setup Darren Salt <news@youmustbejoking.demon.cu.invalid> wrote:
> I demand that Ant may or may not have written...

> > In alt.os.linux.debian Ant <ANTant@zimage.com> wrote:
> [huge snip]
> >> See http://pastebin.com/m5ca8c3f0 first.
> >> I basically held down Y for like a few minutes during fsck /dev/hda1.

> >> I tried to mount this partition with KDE, but it didn't see anything (uh
> >> oh). All other partitions except the swap one seemed fine.

> >> I am going to go reboot.

> > Grub 1.5 now says error 15 meaning file not found according to Google
> > search results. Crap, hda1 is still empty. AUGH!

> Well, at least you have /var - you can make use of dpkg's --get-selections
> and --set-selections options.

Well, I do run apt-get clean once in a while so thoe packages get nuked.
It's already too late since I already cleaned out /var and /tmp
directories/partitions.


> > Back in KNOPPIX, I found out where all the datas went in hda1. They all
> > ended up in lost+found/ directory: http://pastebin.ca/673132 ...

> Anything which you've customised and can identify from that lot (fstab and
> xorg.conf, for example), you should restore after reinstalling.

Yeah, I used the one from my backup though. Those were like a month old
so it wasn't too bad.


> You may be able to do a minimal install in your root partition, after which
> you can mount your existing /usr and /var then just force reinstallation of
> (more or less) all packages. Afterwards, you can unmount /usr and /var then
> remove the content which was hidden by the mount points.

That is what I did (mounting).


> (Untested. This advice is given without warranty or any guarantee of fitness
> for purpose. May cause nasal demons. No sacrifice necessary.)

> > I thought I could move these back, but I realized I am missing other
> > files like Kernel images. I think I am stuck and need to do a reinstall
> > soon.

> Looks like it...

Yep. Done. Just need to reinstall and reconfigure things.
--
"He who cannot pick up an ant, and wants to pick up an elephant will
some day see his folly." --African
/\___/\
/ /\ /\ \ Ant @ http://antfarm.ma.cx (Personal Web Site)
| |o o| | Ant's Quality Foraged Links (AQFL): http://aqfl.net
\ _ / Please remove ANT if replying by e-mail.
( )

If you have a backup, do a restore. Problem solved.

If you don't have a backup, you are a moron and deserve
to lose all of your data. Consider it the price you pay
for being an idiot, and start doing regualr backups.

I hope this helps...