Unix Technical Forum

Analyzing "panic"

This is a discussion on Analyzing "panic" within the comp.unix.solaris forums, part of the Solaris Operating System category; --> Hello. This morning, I found the following in the log of one of my systems: Dec 20 08:12:51 winds02 ...


Go Back   Unix Technical Forum > Unix Operating Systems > Solaris Operating System > comp.unix.solaris

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 01-12-2008, 05:03 AM
Alexander Skwar
 
Posts: n/a
Default Analyzing "panic"

Hello.

This morning, I found the following in the log of one
of my systems:

Dec 20 08:12:51 winds02 unix: [ID 799565 kern.notice] BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
Dec 20 08:12:51 winds02 unix: [ID 839527 kern.notice] umount:
Dec 20 08:12:51 winds02 unix: [ID 123557 kern.notice] alignment error:
Dec 20 08:12:51 winds02 unix: [ID 381800 kern.notice] addr=0x657700646204ed
Dec 20 08:12:51 winds02 unix: [ID 101969 kern.notice] pid=14803, pc=0x11defb4, sp=0x2a100b4f0a1, tstate=0x9900001604, context=0xfbc
Dec 20 08:12:51 winds02 unix: [ID 743441 kern.notice] g1-g7: 78, f, f, 300a79a5a00, 10, 0, 300011e2020
Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f520 unix:die+9c (34, 2a100b4f800, 657700646204ed, 0, 2a100b4f5e0, c1e00000)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000c0800000 0000000000000034 0000000000000000 000003000006c430
Dec 20 08:12:51 winds02 %l4-7: 000003000006c480 000003000006f858 0000000000000000 000000000107b000
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f600 unix:trap+690 (2a100b4f800, 10000, 0, 800009, 0, 300011e2020)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 00000600064ab0c8 0000000000000034 0000060001060e80
Dec 20 08:12:51 winds02 %l4-7: 0000000000010011 0000000000010009 0000000000010000 0000000000010200
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f750 unix:ktl0+48 (300407b9700, 0, 657700646204ed, 0, 1000, 1205800)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000005 0000000000001400 0000009900001604 000000000101afa0
Dec 20 08:12:51 winds02 %l4-7: 0000000000000013 000006000482ed40 0000000000000000 000002a100b4f800
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f8a0 genunix:___const_seg_900000212+1e0bc (300407b9700, 60000c03d98, 1205800, 657700646204ed, 180c000, 60014875980)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0065770064620065 0000000070742f53 0000000000002000 0000000000000002
Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6b01 0000000000000000
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f950 genunix:dounmount+bc (60009ca95c0, 0, 0, 2, 300407b9700, 1)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000018b3040 000000000000001f 000000000000000e 00000000026d6b00
Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6aff 0000000000000000
Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4fa00 genunix:umount2+140 (26df8, 0, ffbfff38, 11aac, 0, 0)
Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 0000000000000000 00000000cb0a0000 000000000000cb0a
Dec 20 08:12:51 winds02 %l4-7: 0000000000000001 0000000000000000 0000000000000000 0000060009ca95c0
Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
Dec 20 08:12:51 winds02 genunix: [ID 672855 kern.notice] syncing file systems...
Dec 20 08:12:51 winds02 genunix: [ID 904073 kern.notice] done
Dec 20 08:12:51 winds02 genunix: [ID 111219 kern.notice] dumping to /dev/dsk/c1t1d0s1, offset 429588480, content: kernel
Dec 20 08:12:51 winds02 genunix: [ID 409368 kern.notice] 100% done: 316443 pages dumped, compression ratio 3.71,
Dec 20 08:12:51 winds02 genunix: [ID 851671 kern.notice] dump succeeded
Dec 20 08:12:51 winds02 genunix: [ID 540533 kern.notice] SunOS Release 5.10 Version Generic_127111-03 64-bit
Dec 20 08:12:51 winds02 genunix: [ID 943907 kern.notice] Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.
Dec 20 08:12:51 winds02 Use is subject to license terms.
Dec 20 08:12:51 winds02 eri: [ID 786680 kern.notice] SUNW,eri0 : 100 Mbps full duplex link up
Dec 20 08:12:51 winds02 savecore: [ID 570001 auth.error] reboot after panic: BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
Dec 20 08:23:42 winds02 savecore: [ID 748169 auth.error] saving system crash dump in /pool/savecore/*.0

What do I make out of this?

Alexander
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 01-12-2008, 05:03 AM
Richard B. Gilbert
 
Posts: n/a
Default Re: Analyzing "panic"

Alexander Skwar wrote:
> Hello.
>
> This morning, I found the following in the log of one
> of my systems:
>
> Dec 20 08:12:51 winds02 unix: [ID 799565 kern.notice] BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
> Dec 20 08:12:51 winds02 unix: [ID 839527 kern.notice] umount:
> Dec 20 08:12:51 winds02 unix: [ID 123557 kern.notice] alignment error:
> Dec 20 08:12:51 winds02 unix: [ID 381800 kern.notice] addr=0x657700646204ed
> Dec 20 08:12:51 winds02 unix: [ID 101969 kern.notice] pid=14803, pc=0x11defb4, sp=0x2a100b4f0a1, tstate=0x9900001604, context=0xfbc
> Dec 20 08:12:51 winds02 unix: [ID 743441 kern.notice] g1-g7: 78, f, f, 300a79a5a00, 10, 0, 300011e2020
> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f520 unix:die+9c (34, 2a100b4f800, 657700646204ed, 0, 2a100b4f5e0, c1e00000)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000c0800000 0000000000000034 0000000000000000 000003000006c430
> Dec 20 08:12:51 winds02 %l4-7: 000003000006c480 000003000006f858 0000000000000000 000000000107b000
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f600 unix:trap+690 (2a100b4f800, 10000, 0, 800009, 0, 300011e2020)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 00000600064ab0c8 0000000000000034 0000060001060e80
> Dec 20 08:12:51 winds02 %l4-7: 0000000000010011 0000000000010009 0000000000010000 0000000000010200
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f750 unix:ktl0+48 (300407b9700, 0, 657700646204ed, 0, 1000, 1205800)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000005 0000000000001400 0000009900001604 000000000101afa0
> Dec 20 08:12:51 winds02 %l4-7: 0000000000000013 000006000482ed40 0000000000000000 000002a100b4f800
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f8a0 genunix:___const_seg_900000212+1e0bc (300407b9700, 60000c03d98, 1205800, 657700646204ed, 180c000, 60014875980)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0065770064620065 0000000070742f53 0000000000002000 0000000000000002
> Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6b01 0000000000000000
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f950 genunix:dounmount+bc (60009ca95c0, 0, 0, 2, 300407b9700, 1)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000018b3040 000000000000001f 000000000000000e 00000000026d6b00
> Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6aff 0000000000000000
> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4fa00 genunix:umount2+140 (26df8, 0, ffbfff38, 11aac, 0, 0)
> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 0000000000000000 00000000cb0a0000 000000000000cb0a
> Dec 20 08:12:51 winds02 %l4-7: 0000000000000001 0000000000000000 0000000000000000 0000060009ca95c0
> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
> Dec 20 08:12:51 winds02 genunix: [ID 672855 kern.notice] syncing file systems...
> Dec 20 08:12:51 winds02 genunix: [ID 904073 kern.notice] done
> Dec 20 08:12:51 winds02 genunix: [ID 111219 kern.notice] dumping to /dev/dsk/c1t1d0s1, offset 429588480, content: kernel
> Dec 20 08:12:51 winds02 genunix: [ID 409368 kern.notice] 100% done: 316443 pages dumped, compression ratio 3.71,
> Dec 20 08:12:51 winds02 genunix: [ID 851671 kern.notice] dump succeeded
> Dec 20 08:12:51 winds02 genunix: [ID 540533 kern.notice] SunOS Release 5.10 Version Generic_127111-03 64-bit
> Dec 20 08:12:51 winds02 genunix: [ID 943907 kern.notice] Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.
> Dec 20 08:12:51 winds02 Use is subject to license terms.
> Dec 20 08:12:51 winds02 eri: [ID 786680 kern.notice] SUNW,eri0 : 100 Mbps full duplex link up
> Dec 20 08:12:51 winds02 savecore: [ID 570001 auth.error] reboot after panic: BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
> Dec 20 08:23:42 winds02 savecore: [ID 748169 auth.error] saving system crash dump in /pool/savecore/*.0
>
> What do I make out of this?
>
> Alexander


A very quick look suggests a programming error in "mount" that caused
the "alignment error". It seems a little odd to say the least. A
broken mount should not have made it out the door!

If you have a support contract, call support. If not, I don't think
there is anything you can do about it except hope that it doesn't happen
again.

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 01-12-2008, 05:03 AM
Alexander Skwar
 
Posts: n/a
Default Re: Analyzing "panic"

Richard B. Gilbert <rgilbert88@comcast.net> wrote:

> Alexander Skwar wrote:
>> Hello.
>>
>> This morning, I found the following in the log of one
>> of my systems:
>>

[...]
> A very quick look suggests a programming error in "mount" that caused
> the "alignment error". It seems a little odd to say the least. A
> broken mount should not have made it out the door!


Thanks.

> If you have a support contract, call support. If not, I don't think
> there is anything you can do about it except hope that it doesn't happen
> again.


The "issue" appeared, when I rebooted the system (during shutdown).
When I saw that, I did some more "init 6"'s, and the panic hasn't
been shown again.

As I cannot reproduce it, I guess I can do nothing but hope.

But thanks for having a quick look!

Alexander
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 01-12-2008, 05:03 AM
Andrew Gabriel
 
Posts: n/a
Default Re: Analyzing "panic"

In article <476A7B8E.5060104@comcast.net>,
"Richard B. Gilbert" <rgilbert88@comcast.net> writes:
> Alexander Skwar wrote:
>> Hello.
>>
>> This morning, I found the following in the log of one
>> of my systems:
>>
>> Dec 20 08:12:51 winds02 unix: [ID 799565 kern.notice] BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
>> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
>> Dec 20 08:12:51 winds02 unix: [ID 839527 kern.notice] umount:
>> Dec 20 08:12:51 winds02 unix: [ID 123557 kern.notice] alignment error:
>> Dec 20 08:12:51 winds02 unix: [ID 381800 kern.notice] addr=0x657700646204ed
>> Dec 20 08:12:51 winds02 unix: [ID 101969 kern.notice] pid=14803, pc=0x11defb4, sp=0x2a100b4f0a1, tstate=0x9900001604, context=0xfbc
>> Dec 20 08:12:51 winds02 unix: [ID 743441 kern.notice] g1-g7: 78, f, f, 300a79a5a00, 10, 0, 300011e2020
>> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f520 unix:die+9c (34, 2a100b4f800, 657700646204ed, 0, 2a100b4f5e0, c1e00000)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000c0800000 0000000000000034 0000000000000000 000003000006c430
>> Dec 20 08:12:51 winds02 %l4-7: 000003000006c480 000003000006f858 0000000000000000 000000000107b000
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f600 unix:trap+690 (2a100b4f800, 10000, 0, 800009, 0, 300011e2020)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 00000600064ab0c8 0000000000000034 0000060001060e80
>> Dec 20 08:12:51 winds02 %l4-7: 0000000000010011 0000000000010009 0000000000010000 0000000000010200
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f750 unix:ktl0+48 (300407b9700, 0, 657700646204ed, 0, 1000, 1205800)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000005 0000000000001400 0000009900001604 000000000101afa0
>> Dec 20 08:12:51 winds02 %l4-7: 0000000000000013 000006000482ed40 0000000000000000 000002a100b4f800
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f8a0 genunix:___const_seg_900000212+1e0bc (300407b9700, 60000c03d98, 1205800, 657700646204ed, 180c000, 60014875980)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0065770064620065 0000000070742f53 0000000000002000 0000000000000002
>> Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6b01 0000000000000000
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4f950 genunix:dounmount+bc (60009ca95c0, 0, 0, 2, 300407b9700, 1)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 00000000018b3040 000000000000001f 000000000000000e 00000000026d6b00
>> Dec 20 08:12:51 winds02 %l4-7: 0000030000072980 0000000000000000 00000000026d6aff 0000000000000000
>> Dec 20 08:12:51 winds02 genunix: [ID 723222 kern.notice] 000002a100b4fa00 genunix:umount2+140 (26df8, 0, ffbfff38, 11aac, 0, 0)
>> Dec 20 08:12:51 winds02 genunix: [ID 179002 kern.notice] %l0-3: 0000000000000000 0000000000000000 00000000cb0a0000 000000000000cb0a
>> Dec 20 08:12:51 winds02 %l4-7: 0000000000000001 0000000000000000 0000000000000000 0000060009ca95c0
>> Dec 20 08:12:51 winds02 unix: [ID 100000 kern.notice]
>> Dec 20 08:12:51 winds02 genunix: [ID 672855 kern.notice] syncing file systems...
>> Dec 20 08:12:51 winds02 genunix: [ID 904073 kern.notice] done
>> Dec 20 08:12:51 winds02 genunix: [ID 111219 kern.notice] dumping to /dev/dsk/c1t1d0s1, offset 429588480, content: kernel
>> Dec 20 08:12:51 winds02 genunix: [ID 409368 kern.notice] 100% done: 316443 pages dumped, compression ratio 3.71,
>> Dec 20 08:12:51 winds02 genunix: [ID 851671 kern.notice] dump succeeded
>> Dec 20 08:12:51 winds02 genunix: [ID 540533 kern.notice] SunOS Release 5.10 Version Generic_127111-03 64-bit
>> Dec 20 08:12:51 winds02 genunix: [ID 943907 kern.notice] Copyright 1983-2007 Sun Microsystems, Inc. All rights reserved.
>> Dec 20 08:12:51 winds02 Use is subject to license terms.
>> Dec 20 08:12:51 winds02 eri: [ID 786680 kern.notice] SUNW,eri0 : 100 Mbps full duplex link up
>> Dec 20 08:12:51 winds02 savecore: [ID 570001 auth.error] reboot after panic: BAD TRAP: type=34 rp=2a100b4f800 addr=657700646204ed mmu_fsr=0
>> Dec 20 08:23:42 winds02 savecore: [ID 748169 auth.error] saving system crash dump in /pool/savecore/*.0
>>
>> What do I make out of this?
>>
>> Alexander

>
> A very quick look suggests a programming error in "mount" that caused
> the "alignment error". It seems a little odd to say the least. A
> broken mount should not have made it out the door!


umount(1M) is a user level program but this crashed in the kernel.
I haven't got a sparc system handy I can disassemble dounmount,
but it looks to me like it might have branched from dounmount()
to a garbage location instead of one of the relevant filesystem's
entrypoints (such a VFS_UNMOUNT).

It might be interesting to see if mdb on the crashdump could
give a better backtrace...

cd /pool/savecore/; echo '$C' | mdb -k 0

> If you have a support contract, call support. If not, I don't think
> there is anything you can do about it except hope that it doesn't happen
> again.
>


--
Andrew Gabriel
[email address is not usable -- followup in the newsgroup]
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 01-12-2008, 05:03 AM
Alexander Skwar
 
Posts: n/a
Default Re: Analyzing "panic"

Andrew Gabriel <andrew@cucumber.demon.co.uk> wrote:
> In article <476A7B8E.5060104@comcast.net>,
> "Richard B. Gilbert" <rgilbert88@comcast.net> writes:


> umount(1M) is a user level program but this crashed in the kernel.
> I haven't got a sparc system handy I can disassemble dounmount,
> but it looks to me like it might have branched from dounmount()
> to a garbage location instead of one of the relevant filesystem's
> entrypoints (such a VFS_UNMOUNT).


Hm. Before I rebooted, I noticed, that I was unable to access NFS
shares that this system exports (ie. this system also acts as a
NFS server). Might be related, don't you think?

> It might be interesting to see if mdb on the crashdump could
> give a better backtrace...
>
> cd /pool/savecore/; echo '$C' | mdb -k 0


000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
657700646204ed, 180c000, 60014875980)
000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308, 2)

Is that better?

Best regards,

Alexander
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 01-12-2008, 05:03 AM
Andrew Gabriel
 
Posts: n/a
Default Re: Analyzing "panic"

In article <2505468.VPmkaakTx6@kn.gn.rtr.message-center.info>,
Alexander Skwar <alexander@skwar.name> writes:
> Andrew Gabriel <andrew@cucumber.demon.co.uk> wrote:
>> In article <476A7B8E.5060104@comcast.net>,
>> "Richard B. Gilbert" <rgilbert88@comcast.net> writes:

>
>> umount(1M) is a user level program but this crashed in the kernel.
>> I haven't got a sparc system handy I can disassemble dounmount,
>> but it looks to me like it might have branched from dounmount()
>> to a garbage location instead of one of the relevant filesystem's
>> entrypoints (such a VFS_UNMOUNT).

>
> Hm. Before I rebooted, I noticed, that I was unable to access NFS
> shares that this system exports (ie. this system also acts as a
> NFS server). Might be related, don't you think?
>
>> It might be interesting to see if mdb on the crashdump could
>> give a better backtrace...
>>
>> cd /pool/savecore/; echo '$C' | mdb -k 0

>
> 000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
> 657700646204ed, 180c000, 60014875980)
> 000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
> 000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
> 000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308, 2)
>
> Is that better?


A bit.

Try:
echo '300407b9700:rint -t vnode_t' | mdb -k 0
to see what vnode fop_inactive() has here.

Also try:
echo '60009ca95c0:rint -t vfs_t' | mdb -k 0

--
Andrew Gabriel
[email address is not usable -- followup in the newsgroup]
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 01-12-2008, 05:03 AM
Alexander Skwar
 
Posts: n/a
Default Re: Analyzing "panic"

Andrew Gabriel <andrew@cucumber.demon.co.uk> wrote:

> In article <2505468.VPmkaakTx6@kn.gn.rtr.message-center.info>,
> Alexander Skwar <alexander@skwar.name> writes:
>> Andrew Gabriel <andrew@cucumber.demon.co.uk> wrote:
>>> In article <476A7B8E.5060104@comcast.net>,
>>> "Richard B. Gilbert" <rgilbert88@comcast.net> writes:


>>> It might be interesting to see if mdb on the crashdump could
>>> give a better backtrace...
>>>
>>> cd /pool/savecore/; echo '$C' | mdb -k 0

>>
>> 000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
>> 657700646204ed, 180c000, 60014875980)
>> 000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
>> 000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
>> 000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308,
>> 2)
>>
>> Is that better?

>
> A bit.
>
> Try:
> echo '300407b9700:rint -t vnode_t' | mdb -k 0
> to see what vnode fop_inactive() has here.


adm@winds02 /pool/savecore $ echo '300407b9700:rint -t vnode_t' | sudo mdb -k 0
{
kmutex_t v_lock = {
void *[1] _opaque = [ 0 ]
}
uint_t v_flag = 0x100
uint_t v_count = 0x1
void *v_data = 0x300407c0a10
struct vfs *v_vfsp = 0x60014875980
struct stdata *v_stream = 0
enum vtype v_type = 2 (VDIR)
dev_t v_rdev = 0xffffffffffffffff
struct vfs *v_vfsmountedhere = 0
struct vnodeops *v_op = 0x60001ad5080
struct page *v_pages = 0
pgcnt_t v_npages = 0
pgcnt_t v_msnpages = 0
struct page *v_scanfront = 0
struct page *v_scanback = 0
struct filock *v_filocks = 0
struct shrlocklist *v_shrlocks = 0
krwlock_t v_nbllock = {
void *[1] _opaque = [ 0 ]
}
kcondvar_t v_cv = {
ushort_t _opaque = 0
}
void *v_locality = 0
struct fem_head *v_femhead = 0
char *v_path = 0x3001cab4008 "
/pool/home/askwar/Gentoo-Home/.zfs/snapshot/auf-usb-stick-20071212/Notebook/usr/
portage"
uint_t v_rdcnt = 0
uint_t v_wrcnt = 0
u_longlong_t v_mmap_read = 0
u_longlong_t v_mmap_write = 0
void *v_mpssdata = 0
hrtime_t v_scantime = 0
ushort_t v_mset = 0
uint_t v_msflags = 0
struct vnode *v_msnext = 0
struct vnode *v_msprev = 0
krwlock_t v_mslock = {
void *[1] _opaque = [ 0 ]
}
}

/pool/home/askwar.... is NFS exported. And, as I said, I noticed that
I could not access the NFS server anymore. A "showmount -e winds02"
returned nothing (winds02 is the name of the server).

> Also try:
> echo '60009ca95c0:rint -t vfs_t' | mdb -k 0


adm@winds02 /pool/savecore $ echo '60009ca95c0:rint -t vfs_t' | sudo mdb -k 0
{
struct vfs *vfs_next = 0
struct vfs *vfs_prev = 0
vfsops_t *vfs_op = vfssw+0x538
struct vnode *vfs_vnodecovered = 0
uint_t vfs_flag = 0x2400
uint_t vfs_bsize = 0x2000
int vfs_fstype = 0xa
fsid_t vfs_fsid = {
int [2] val = [ 0x1540000, 0x2 ]
}
void *vfs_data = 0x30025969468
dev_t vfs_dev = 0x5500000000
ulong_t vfs_bcount = 0
struct vfs *vfs_list = 0x1b0f00001b0f
struct vfs *vfs_hash = 0
ksema_t vfs_reflock = {
void *[2] _opaque = [ 0, 0 ]
}
uint_t vfs_count = 0
mntopts_t vfs_mntopts = {
uint_t mo_count = 0
mntopt_t *mo_list = 0
}
refstr_t *vfs_resource = 0
refstr_t *vfs_mntpt = 0
time_t vfs_mtime = 2007 Dec 13 08:08:40
vfs_impl_t *vfs_implp = 0
struct zone *vfs_zone = zone0
struct vfs *vfs_zone_next = 0
struct vfs *vfs_zone_prev = 0
}

Alexander
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 01-12-2008, 05:03 AM
James Carlson
 
Posts: n/a
Default Re: Analyzing "panic"

Alexander Skwar <alexander@skwar.name> writes:
> >> 000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
> >> 657700646204ed, 180c000, 60014875980)
> >> 000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
> >> 000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
> >> 000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308,

[...]
> struct vfs *v_vfsp = 0x60014875980


Try doing a "0x60014875980::whatis" in mdb. If it says that this
buffer was freed, then I think you've hit CR 6564619.

--
James Carlson, Solaris Networking <james.d.carlson@sun.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 01-12-2008, 05:03 AM
Alexander Skwar
 
Posts: n/a
Default Re: Analyzing "panic"

· James Carlson <james.d.carlson@sun.com>:

> Alexander Skwar <alexander@skwar.name> writes:
>> >> 000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
>> >> 657700646204ed, 180c000, 60014875980)
>> >> 000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
>> >> 000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
>> >> 000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308,

> [...]
>> struct vfs *v_vfsp = 0x60014875980

>
> Try doing a "0x60014875980::whatis" in mdb. If it says that this
> buffer was freed, then I think you've hit CR 6564619.


adm@winds02 /pool/savecore $ echo '0x60014875980::whatis' | sudo mdb -k 0
60014875980 is 60014875980+0, freed from kmem_alloc_192

I suppose during an init 6, a forced unmount is being done, right?

Alexander Skwar
--
panic("esp: what could it be... I wonder...");
linux-2.2.16/drivers/scsi/esp.c

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 01-12-2008, 05:04 AM
James Carlson
 
Posts: n/a
Default Re: Analyzing "panic"

Alexander Skwar <usenet@alexander.skwar.name> writes:

> · James Carlson <james.d.carlson@sun.com>:
>
> > Alexander Skwar <alexander@skwar.name> writes:
> >> >> 000002a100b4f0a1 fop_inactive+0x54(300407b9700, 60000c03d98, 1205800,
> >> >> 657700646204ed, 180c000, 60014875980)
> >> >> 000002a100b4f151 dounmount+0xbc(60009ca95c0, 0, 0, 2, 300407b9700, 1)
> >> >> 000002a100b4f201 umount2+0x140(26df8, 0, ffbfff38, 11aac, 0, 0)
> >> >> 000002a100b4f2e1 syscall_trap32+0xcc(26df8, 0, ffbfff38, 11aac, ff36e308,

> > [...]
> >> struct vfs *v_vfsp = 0x60014875980

> >
> > Try doing a "0x60014875980::whatis" in mdb. If it says that this
> > buffer was freed, then I think you've hit CR 6564619.

>
> adm@winds02 /pool/savecore $ echo '0x60014875980::whatis' | sudo mdb -k 0
> 60014875980 is 60014875980+0, freed from kmem_alloc_192
>
> I suppose during an init 6, a forced unmount is being done, right?


It can be, yes.

--
James Carlson, Solaris Networking <james.d.carlson@sun.com>
Sun Microsystems / 35 Network Drive 71.232W Vox +1 781 442 2084
MS UBUR02-212 / Burlington MA 01803-2757 42.496N Fax +1 781 442 1677
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 06:39 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com