View Single Post

   
  #3 (permalink)  
Old 04-07-2008, 08:29 AM
Hans Martin Schwengeler
 
Posts: n/a
Default Re: How to analyze core dump after system crash?

In article <fta22m$g1c$1@news.yaako.com>,
"newbie" <new@new.com> writes:
> Hi all,
>
> Our HP C3600 Unix workstaion always crashes recently, 4 or 5 times everyday.
> which debug tool can be used to analyze core.26.1.gz and vmunix.gz?
>
> Thanks a lot.
> Case
>
> Some information are shown as below:
>
> # uname -a
> HP-UX HP001 B.10.20 A 9000/785 2000429216 two-user license
> # pwd
> /var/adm/crash
> # ls -tl
> total 34
> drwxr-xr-x 2 root 0 1024 Apr 5 01:14 core.26
> -rwxr-xr-x 1 root 0 2 Apr 5 01:14 bounds
> drwxr-xr-x 2 root 0 1024 Apr 5 01:05 core.25
> drwxr-xr-x 2 root 0 1024 Apr 5 00:51 core.24
> drwxr-xr-x 2 root 0 1024 Apr 4 23:57 core.23
> drwxr-xr-x 2 root 0 1024 Apr 3 06:48 core.22
> drwxr-xr-x 2 root 0 1024 Apr 3 03:49 core.21
> drwxr-xr-x 2 root 0 1024 Apr 3 03:24 core.20
> drwxr-xr-x 2 root 0 1024 Apr 3 02:12 core.19
> drwxr-xr-x 2 root 0 1024 Apr 3 00:37 core.18
> drwxr-xr-x 2 root 0 1024 Apr 2 23:56 core.17
> drwxr-xr-x 2 root 0 1024 Apr 2 20:21 core.16
> drwxr-xr-x 2 root 0 1024 Apr 2 19:54 core.15
> drwxr-xr-x 2 root 0 1024 Apr 2 05:57 core.14
> drwxr-xr-x 2 root 0 1024 Apr 2 04:26 core.13
> drwxr-xr-x 2 root 0 1024 Apr 2 03:52 core.12
> drwxr-xr-x 2 root 0 1024 Apr 2 03:11 core.11
>
> # cd core.26
> # ls -a
> . .. INDEX core.26.1.gz vmunix.gz
> # ls -al
> total 32246
> drwxr-xr-x 2 root 0 1024 Apr 5 01:14 .
> drwxr-xr-x 24 root 0 1024 Apr 5 01:14 ..
> -rw-r--r-- 1 root 0 586 Apr 5 01:17 INDEX
> -rw-r--r-- 1 root 0 12941263 Apr 5 01:17 core.26.1.gz
> -rw-r--r-- 1 root 0 3539310 Apr 5 01:14 vmunix.gz
>
> # more syslog.log
> Apr 5 01:17:51 HP001 syslogd: restart
> Apr 5 01:17:52 HP001 vmunix: vuseg=a31d000
> Apr 5 01:17:52 HP001 vmunix: inet_cltsk inet_cotsk nfs_init3 added
> vfs type nfs3 at slot 5
> Apr 5 01:17:52 HP001 vmunix: NOTICE: cachefs_link(): File system was
> registered at index 6.
> Apr 5 01:17:52 HP001 vmunix: NOTICE: autofs_link(): File system was
> registered at index 7.
> Apr 5 01:17:52 HP001 vmunix: PciCID10eb: FO Card Driver Rev. 01.00.00 -
> Copyright (c) 1998 Hewlett
> Packard BSTD R&D
> Apr 5 01:17:52 HP001 vmunix: PciCID10eb: driver successfully installed.
> Apr 5 01:17:52 HP001 vmunix:
> Apr 5 01:17:52 HP001 vmunix: t61622: GPIO TAMS61622 card driver. Copyright
> 1999-2001 TAMS Inc. 970-
> 669-6553
> Apr 5 01:17:52 HP001 vmunix: t61622: INSTALLING the driver... Looking for
> TAMS61622 hardware...
> Apr 5 01:17:52 HP001 vmunix: t60488 driver installed
> Apr 5 01:17:52 HP001 vmunix: 10 ccio2
> Apr 5 01:17:52 HP001 vmunix: 10/0 lba
> Apr 5 01:17:52 HP001 vmunix: 10/0/12/0 btlan3
> Apr 5 01:17:52 HP001 vmunix: 10/0/13/0 audio
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/0 side
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/0.7 tgt
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/0.7.0 sctl
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/1 superio
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/1/1 asio0
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/1/2 asio0
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/1/3 SCentIf
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/2 hcd
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/2.1 hub
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/2.1.2 hid
> Apr 5 01:17:52 HP001 vmunix: 10/0/14/2.1.3 hid
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/0 c720
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/0.7 tgt
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/0.7.0 sctl
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1 c720
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.5 tgt
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.5.0 sdisk
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.6 tgt
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.6.0 sdisk
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.7 tgt
> Apr 5 01:17:52 HP001 vmunix: 10/0/15/1.7.0 sctl
> Apr 5 01:17:52 HP001 vmunix: 10/1 lba
> Apr 5 01:17:52 HP001 vmunix: t61622[4]: FOUND TAMS61622 hardware.
> Attaching...
> Apr 5 01:17:52 HP001 vmunix: t61622[4]: ATTACHED successfully. Enabling...
> Apr 5 01:17:52 HP001 vmunix: 10/1/4/0 t61622
> Apr 5 01:17:52 HP001 vmunix: PciCID10eb: PCI CIC detected.
> Apr 5 01:17:52 HP001 vmunix: PciCID10eb: PCI CIC type is F330.
> Apr 5 01:17:52 HP001 vmunix: PciCID10eb: claiming this device.
> Apr 5 01:17:52 HP001 vmunix: 10/1/5/0 PciCID10eb
> Apr 5 01:17:52 HP001 vmunix: 10/1/6/0 t60488
> Apr 5 01:17:52 HP001 vmunix: 10/4 lba
> Apr 5 01:17:52 HP001 vmunix: 10/4/3/0 t60488
> Apr 5 01:17:52 HP001 vmunix: 10/6 lba
> Apr 5 01:17:52 HP001 vmunix: Variable size pages used to map 1000 graf
> pages at fb000000
> Apr 5 01:17:52 HP001 vmunix: 10/6/2/0 graph3
> Apr 5 01:17:52 HP001 vmunix: 32 processor
> Apr 5 01:17:52 HP001 vmunix: 49 memory
> Apr 5 01:17:52 HP001 vmunix: btlan3: Initializing 10/100BASE-TX card at
> 10/0/12/0....
> Apr 5 01:17:52 HP001 vmunix: T60488: card identified and enabled
> Apr 5 01:17:52 HP001 vmunix: t61622[4]: DMA support: Yes
> Apr 5 01:17:52 HP001 vmunix: t61622[4]: ENABLED successfully. TAMS61622
> ready.
> Apr 5 01:17:52 HP001 vmunix: T60488: card identified and enabled
> Apr 5 01:17:52 HP001 vmunix:
> Apr 5 01:17:52 HP001 vmunix: System Console is on the ITE
> Apr 5 01:17:52 HP001 vmunix: Networking memory for fragment reassembly is
> restricted to 186916864 b
> ytes
> Apr 5 01:17:52 HP001 vmunix: Logical volume 64, 0x3 configured as ROOT
> Apr 5 01:17:52 HP001 vmunix: Logical volume 64, 0x2 configured as SWAP
> Apr 5 01:17:52 HP001 vmunix: Logical volume 64, 0x2 configured as DUMP
> Apr 5 01:17:52 HP001 vmunix: Swap device table: (start & size given in
> 512-byte blocks)
> Apr 5 01:17:52 HP001 vmunix: entry 0 - major is 64, minor is 0x2;
> start = 0, size = 3891200
> Apr 5 01:17:52 HP001 vmunix: WARNING: Insufficient space on dump device to
> save full crashdump.
> Apr 5 01:17:52 HP001 vmunix: Only 1992294400 of 2147484672 bytes will
> be saved.
> Apr 5 01:17:52 HP001 vmunix: Dump device table: (start & size given in
> 1-Kbyte blocks)
> Apr 5 01:17:52 HP001 vmunix: entry 0 - major is 31, minor is
> 0x36000; start = 52064, size =
> 1945600
> Apr 5 01:17:52 HP001 vmunix: Starting the STREAMS daemons.
> Apr 5 01:17:52 HP001 vmunix: B2352B HP-UX (B.10.20) #1: Sun Jun 9
> 08:03:38 PDT 1996
> Apr 5 01:17:52 HP001 vmunix:
> Apr 5 01:17:52 HP001 vmunix: Memory Information:
> Apr 5 01:17:52 HP001 vmunix: physical page size = 4096 bytes, logical
> page size = 4096 bytes
> Apr 5 01:17:52 HP001 vmunix: Physical: 2097152 Kbytes, lockable:
> 1573580 Kbytes, available: 180
> 7012 Kbytes
> Apr 5 01:17:52 HP001 vmunix:
> Apr 5 01:17:52 HP001 vmunix: btlan3: NOTE: MII Link Status Not OK - Switch
> Connection to AUI at 10/
> 0/12/0....
> Apr 5 01:17:52 HP001 vmunix: btlan3: timeout: DMA timeout occurred at
> 10/0/12/0
> Apr 5 01:17:52 HP001 vmunix: btlan3: reset state is 550 at 10/0/12/0....
> Apr 5 01:17:52 HP001 vmunix: btlan3: WARNING: AUI Loopback Failed at
> 10/0/12/0....
> Apr 5 01:17:51 HP001 nettl[818]: nettl starting up.
> Apr 5 01:18:23 HP001 inetd[1036]: Reading configuration
> Apr 5 01:18:23 HP001 inetd[1036]: ftp/tcp: Added service, server
> /usr/lbin/ftpd
> Apr 5 01:18:23 HP001 inetd[1036]: telnet/tcp: Added service, server
> /usr/lbin/telnetd
> Apr 5 01:18:23 HP001 inetd[1036]: tftp/udp: Added service, server
> /usr/lbin/tftpd
> Apr 5 01:18:23 HP001 inetd[1036]: login/tcp: Added service, server
> /usr/lbin/rlogind
> Apr 5 01:18:23 HP001 inetd[1036]: shell/tcp: Added service, server
> /usr/lbin/remshd
> Apr 5 01:18:23 HP001 inetd[1036]: exec/tcp: Added service, server
> /usr/lbin/rexecd
> Apr 5 01:18:23 HP001 inetd[1036]: ntalk/udp: Added service, server
> /usr/lbin/ntalkd
> Apr 5 01:18:23 HP001 inetd[1036]: auth/tcp: Added service, server
> /usr/lbin/identd
> Apr 5 01:18:23 HP001 inetd[1036]: printer/tcp: Added service, server
> /usr/sbin/rlpdaemon
> Apr 5 01:18:23 HP001 inetd[1036]: daytime/tcp: Added service, server
> internal
> Apr 5 01:18:23 HP001 inetd[1036]: daytime/udp: Added service, server
> internal
> Apr 5 01:18:23 HP001 inetd[1036]: time/tcp: Added service, server internal
> Apr 5 01:18:23 HP001 inetd[1036]: time/udp: Added service, server internal
> Apr 5 01:18:23 HP001 inetd[1036]: echo/tcp: Added service, server internal
> Apr 5 01:18:23 HP001 inetd[1036]: echo/udp: Added service, server internal
> Apr 5 01:18:23 HP001 inetd[1036]: discard/tcp: Added service, server
> internal
> Apr 5 01:18:23 HP001 inetd[1036]: discard/udp: Added service, server
> internal
> Apr 5 01:18:23 HP001 inetd[1036]: chargen/tcp: Added service, server
> internal
> Apr 5 01:18:23 HP001 inetd[1036]: chargen/udp: Added service, server
> internal
> Apr 5 01:18:24 HP001 inetd[1036]: kshell/tcp: Added service, server
> /usr/lbin/remshd
> Apr 5 01:18:24 HP001 inetd[1036]: klogin/tcp: Added service, server
> /usr/lbin/rlogind
> Apr 5 01:18:24 HP001 inetd[1036]: dtspc/tcp: Added service, server
> /usr/dt/bin/dtspcd
> Apr 5 01:18:24 HP001 inetd[1036]: recserv/tcp: Added service, server
> /usr/lbin/recserv
> Apr 5 01:18:24 HP001 inetd[1036]: spc/tcp: Added service, server
> /usr/bms/bin/softspcd
> Apr 5 01:18:24 HP001 inetd[1036]: instl_boots/udp: Added service, server
> /opt/ignite/lbin/instl_boo
> td
> Apr 5 01:18:24 HP001 inetd[1036]: rpc.cmsd/udp: Added service, server
> /usr/dt/bin/rpc.cmsd
> Apr 5 01:18:24 HP001 inetd[1036]: rpc.ttdbserver/tcp: Added service, server
> /usr/dt/bin/rpc.ttdbser
> ver
> Apr 5 01:18:24 HP001 inetd[1036]: Configuration complete
> Apr 5 01:18:36 HP001 FontServer[1257]: Warning: Rasterizer
> "/usr/lib/X11/fs/fwrast.sl" could not be
> loaded: No such file or directory
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/hp_japan
> ese/100dpi/"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Cannot initialize font path
> element: "/usr/lib/X11/
> fonts/hp_japanese/75dpi/"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/hp_korea
> n/75dpi/"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/hp_chine
> se_s/75dpi/"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/hp_chine
> se_t/75dpi/"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/fontwave
> /fwd"
> Apr 5 01:18:37 HP001 FontServer[1257]: Warning: Bad font path element:
> "/usr/lib/X11/fonts/ifojpn.s
> t"
> Apr 5 01:18:41 HP001 xntpd[1279]: xntpd version=3.5f; Tue Jul 1 16:35:35
> PDT 1997 (1)
> Apr 5 01:18:41 HP001 xntpd[1279]: tickadj = 625, tick = 10000, tvu_maxslew
> = 61875
> Apr 5 01:18:41 HP001 xntpd[1279]: precision = 7 usec
> Apr 5 01:19:19 HP001 /usr/dt/bin/rpc.ttdbserver[1838]:
> _Tt_db_server_db("/"): 4 (/TT_DB/access_tabl
> e)
> Apr 5 01:19:19 HP001 vmunix: /: bad dir ino 19228 at offset 0: mangled
> entry
> Apr 5 01:19:19 HP001 /usr/dt/bin/rpc.ttdbserver[1838]:
> _Tt_db_server_db("/"): 4 (/TT_DB/access_tabl
> e)
> Apr 5 01:19:20 HP001 last message repeated 10 times
> Apr 5 01:21:12 HP001 syslog: su : + 0 prod536-root
>
>
>
> begin 666 hobbit@hswn.dk.vcf
> M0D5'24XZ5D-!4D0-"E9%4E-)3TXZ,BXQ#0I..CMH;V)B:71 :'-W;BYD:PT*
> M1DXZ:&]B8FET0&AS=VXN9&L-"D5-04E,.U!2148[24Y415).150Z:&]B8FET
> K0&AS=VXN9&L-"E)%5CHR,# X,#0P-E0P.#,V-#5:#0I%3D0Z5D-!4D0-"@``
> `
> end
>


Hello,

I don't like the line in syslog:
vmunix: /: bad dir ino 19228 at offset 0: mangled entry

It seems that the / filesystem need checking, probably it's corrupt.
Maybe the disk is defective. You might first try a full filesystem
check at the single user prompt (boot with hpux -is).

I'd try to make a Ignite-UX backup onto tape and the reinstall the
system from tape. Ot just do a fresh install onto a new disk.

Yours, Hans Martin
Reply With Quote