I have an oracle 10g RAC with sun v880 server running solaris 10.
After having hardware problem on the server and fixing it, now when I
reboot the secondery server, the master server crashes and I have to do
hard reboot. Any tought???

I think this is the error for it,

2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
down.
2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
kgxgnpstat: error: CLSS service is shutting down
kjzmpoll: kgxgnpstat err[16 -2147438272]
Cluster Synchronization Service is shutting down
DIAG detachs from KGXGN service
error 29746 detected in background process
OPIRIP: Uncaught error 447. Error stack:
ORA-00447: fatal error in background process
ORA-29746: Cluster Synchronization Service is being shut down.

souri

Souri Bozorgmehri wrote:
> I have an oracle 10g RAC with sun v880 server running solaris 10.
> After having hardware problem on the server and fixing it, now when I
> reboot the secondery server, the master server crashes and I have to do
> hard reboot. Any tought???
>
> I think this is the error for it,
>
> 2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
> down.
> 2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
> kgxgnpstat: error: CLSS service is shutting down
> kjzmpoll: kgxgnpstat err[16 -2147438272]
> Cluster Synchronization Service is shutting down
> DIAG detachs from KGXGN service
> error 29746 detected in background process
> OPIRIP: Uncaught error 447. Error stack:
> ORA-00447: fatal error in background process
> ORA-29746: Cluster Synchronization Service is being shut down.
>
> souri

What release level exactly on 10g? I would think you are perhaps
missing some oracle maintenance that might prevent the crash and burn
of the other node.

Souri Bozorgmehri wrote:
> I have an oracle 10g RAC with sun v880 server running solaris 10.
> After having hardware problem on the server and fixing it, now when I
> reboot the secondery server, the master server crashes and I have to do
> hard reboot. Any tought???
>
> I think this is the error for it,
>
> 2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
> down.
> 2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
> kgxgnpstat: error: CLSS service is shutting down
> kjzmpoll: kgxgnpstat err[16 -2147438272]
> Cluster Synchronization Service is shutting down
> DIAG detachs from KGXGN service
> error 29746 detected in background process
> OPIRIP: Uncaught error 447. Error stack:
> ORA-00447: fatal error in background process
> ORA-29746: Cluster Synchronization Service is being shut down.
>
> souri

When you say reboot the secondary server are you doing a literal
operating system reboot? I'd suggest rerunning the cluster verify
tool for starters.

And, as hpuxrac asked, 10g RAC is meaningless. Which version precisely
out to 4 decimal points?
--
Daniel A. Morgan
University of Washington
damorgan@x.washington.edu
(replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org

DA Morgan wrote:
> Souri Bozorgmehri wrote:
> > I have an oracle 10g RAC with sun v880 server running solaris 10.
> > After having hardware problem on the server and fixing it, now when I
> > reboot the secondery server, the master server crashes and I have to do
> > hard reboot. Any tought???
> >
> > I think this is the error for it,
> >
> > 2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
> > down.
> > 2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
> > kgxgnpstat: error: CLSS service is shutting down
> > kjzmpoll: kgxgnpstat err[16 -2147438272]
> > Cluster Synchronization Service is shutting down
> > DIAG detachs from KGXGN service
> > error 29746 detected in background process
> > OPIRIP: Uncaught error 447. Error stack:
> > ORA-00447: fatal error in background process
> > ORA-29746: Cluster Synchronization Service is being shut down.
> >
> > souri
>
> When you say reboot the secondary server are you doing a literal
> operating system reboot? I'd suggest rerunning the cluster verify
> tool for starters.
>
> And, as hpuxrac asked, 10g RAC is meaningless. Which version precisely
> out to 4 decimal points?
> --
> Daniel A. Morgan
> University of Washington
> damorgan@x.washington.edu
> (replace x with u to respond)
> Puget Sound Oracle Users Group
> www.psoug.org


The version of oracle is 10.2.0.1.
When I say reboot, I mean either stop RAC services or actually
rebooting the solaris box.

Thanks,

Souri Bozorgmehri wrote:
> DA Morgan wrote:
>> Souri Bozorgmehri wrote:
>>> I have an oracle 10g RAC with sun v880 server running solaris 10.
>>> After having hardware problem on the server and fixing it, now when I
>>> reboot the secondery server, the master server crashes and I have to do
>>> hard reboot. Any tought???
>>>
>>> I think this is the error for it,
>>>
>>> 2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
>>> down.
>>> 2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
>>> kgxgnpstat: error: CLSS service is shutting down
>>> kjzmpoll: kgxgnpstat err[16 -2147438272]
>>> Cluster Synchronization Service is shutting down
>>> DIAG detachs from KGXGN service
>>> error 29746 detected in background process
>>> OPIRIP: Uncaught error 447. Error stack:
>>> ORA-00447: fatal error in background process
>>> ORA-29746: Cluster Synchronization Service is being shut down.
>>>
>>> souri
>> When you say reboot the secondary server are you doing a literal
>> operating system reboot? I'd suggest rerunning the cluster verify
>> tool for starters.
>>
>> And, as hpuxrac asked, 10g RAC is meaningless. Which version precisely
>> out to 4 decimal points?
>> --
>> Daniel A. Morgan
>> University of Washington
>> damorgan@x.washington.edu
>> (replace x with u to respond)
>> Puget Sound Oracle Users Group
>> www.psoug.org
>
>
> The version of oracle is 10.2.0.1.
> When I say reboot, I mean either stop RAC services or actually
> rebooting the solaris box.
>
> Thanks,

I'd run cluster verify. I suspect you've got a problem at that level.

If not then look very carefully at the network hardware and
configuration. What you are describing is not normal behaviour.
--
Daniel A. Morgan
University of Washington
damorgan@x.washington.edu
(replace x with u to respond)
Puget Sound Oracle Users Group
www.psoug.org

Hello,

The error message you received is from database level. Your
Clusterlayer is having the problems. To find out what the reall problem
is you need to analyse the logfiles of the CRS stack, if you are using
Oracle Clusterware.

If you are using vendor clusterware you still can check the part which
are running.
To me it sounds like "you added new hardware" ? what is changed, new ip
?

Running cluvfy can help to identify where the problem area is.

Greetings,
BCB
www.rachelp.nl


DA Morgan wrote:
> Souri Bozorgmehri wrote:
> > DA Morgan wrote:
> >> Souri Bozorgmehri wrote:
> >>> I have an oracle 10g RAC with sun v880 server running solaris 10.
> >>> After having hardware problem on the server and fixing it, now when I
> >>> reboot the secondery server, the master server crashes and I have to do
> >>> hard reboot. Any tought???
> >>>
> >>> I think this is the error for it,
> >>>
> >>> 2006-08-29 16:15:47.362: [ CSSCLNT]clssgsGroupGetStatus: CSS shutting
> >>> down.
> >>> 2006-08-29 16:15:47.368: [ CSSCLNT]clssgsGroupGetStatus: returning 22
> >>> kgxgnpstat: error: CLSS service is shutting down
> >>> kjzmpoll: kgxgnpstat err[16 -2147438272]
> >>> Cluster Synchronization Service is shutting down
> >>> DIAG detachs from KGXGN service
> >>> error 29746 detected in background process
> >>> OPIRIP: Uncaught error 447. Error stack:
> >>> ORA-00447: fatal error in background process
> >>> ORA-29746: Cluster Synchronization Service is being shut down.
> >>>
> >>> souri
> >> When you say reboot the secondary server are you doing a literal
> >> operating system reboot? I'd suggest rerunning the cluster verify
> >> tool for starters.
> >>
> >> And, as hpuxrac asked, 10g RAC is meaningless. Which version precisely
> >> out to 4 decimal points?
> >> --
> >> Daniel A. Morgan
> >> University of Washington
> >> damorgan@x.washington.edu
> >> (replace x with u to respond)
> >> Puget Sound Oracle Users Group
> >> www.psoug.org
> >
> >
> > The version of oracle is 10.2.0.1.
> > When I say reboot, I mean either stop RAC services or actually
> > rebooting the solaris box.
> >
> > Thanks,
>
> I'd run cluster verify. I suspect you've got a problem at that level.
>
> If not then look very carefully at the network hardware and
> configuration. What you are describing is not normal behaviour.
> --
> Daniel A. Morgan
> University of Washington
> damorgan@x.washington.edu
> (replace x with u to respond)
> Puget Sound Oracle Users Group
> www.psoug.org