NOTE: in all logs we just replaced real host names with XXXX

We are getting these errors on our web servers intermittently. When
these error happen, all web servers "choke", they do not respond to ANY
request, not just DB requests. The DB server itself shows almost no
activity at all during that time. Usually the problem clears itself
after few minutes then everything goes back to normal. We see the same
errors in logs on probably 9 out of 13 webservers at least for the last
time it happened.

Some clients are running client 8.1.7 on WIN2K
Some are running client 9.2.0.1 on 2003,
Both show the error.

DB server is 9.2.0.5 on solaris 5.9

the error always appears to be as follows, the client program is either
intefo.exe or cfserver.exe (our client programs),
---------------------------------------------------------------
Fatal NI connect error 12560, connecting to:

(DESCRIPTION=(ADDRESS=(PROTOCOL=BEQ)(PROGRAM=oracl e)(ARGV0=oracleORCL)(ARGS='(DESCRIPTION=(LOCAL=YES )(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=OR CL)(CID=(PROGRAM=D:\CFusion\Bin\cfserver.exe)(HOST =xxxxxx)(USER=SYSTEM))))

VERSION INFORMATION:
TNS for 32-bit Windows: Version 9.2.0.1.0 - Production
Oracle Bequeath NT Protocol Adapter for 32-bit Windows: Version
9.2.0.1.0 - Production
Windows NT TCP/IP NT Protocol Adapter for 32-bit Windows: Version
9.2.0.1.0 - Production
Time: 01-MAR-2006 14:20:03
Tracing not turned on.
Tns error struct:
nr err code: 0
ns main err code: 12560
TNS-12560: TNSrotocol adapter error
ns secondary err code: 0
nt main err code: 530
TNS-00530: Protocol adapter error
nt secondary err code: 126
nt OS err code: 0

---------------------------------------------------------------

While the error shows BEQ, the tnsnames.ora on the clients has no BEQ
protocol defined. The connection to the DB server is TCP, as follows:

---------------------------------------------------------------
PRODUCTION.XXXXXX =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = XXXXXX)(PORT = 1521))
)
(CONNECT_DATA =
(SID = FPROD)
)
)
---------------------------------------------------------------

the listener.ora on the DB server has no BEQ protocol either. Here it
is:

---------------------------------------------------------------
LISTENER =
(DESCRIPTION_LIST =
(DESCRIPTION =
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = IPC)(KEY = EXTPROC))
)
(ADDRESS_LIST =
(ADDRESS = (PROTOCOL = TCP)(HOST = XXXXXX)(PORT = 1521)
(QUEUESIZE=100
))
)
)
)

LOGGING_LISTENER=off

SID_LIST_LISTENER =
(SID_LIST =
(SID_DESC =
(SID_NAME = PLSExtProc)
(ORACLE_HOME = /u5/app/oracle/product/9.2.0)
(PROGRAM = extproc)
)
(SID_DESC =
(GLOBAL_DBNAME = FPROD.XXXXXX)
(ORACLE_HOME = /u5/app/oracle/product/9.2.0)
(SID_NAME = FPROD)
)
(SID_DESC =
(GLOBAL_DBNAME = FPROD1.XXXXXX)
(ORACLE_HOME = /u5/app/oracle/product/9.2.0)
(SID_NAME = FPROD1)
)
)

---------------------------------------------------------------

Any suggestions?

Thank you,

Roy

Roy M wrote:
> NOTE: in all logs we just replaced real host names with XXXX
>
Why?

>
> (DESCRIPTION=(ADDRESS=(PROTOCOL=BEQ)(PROGRAM=oracl e)(ARGV0=oracleORCL)(ARGS='(DESCRIPTION=(LOCAL=YES )(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=OR CL)(CID=(PROGRAM=D:\CFusion\Bin\cfserver.exe)(HOST =xxxxxx)(USER=SYSTEM))))

(ARGV0=oracleORCL) indicates you have ORCL set as SID.

> PRODUCTION.XXXXXX =
> (DESCRIPTION =
> (ADDRESS_LIST =
> (ADDRESS = (PROTOCOL = TCP)(HOST = XXXXXX)(PORT = 1521))
> )
> (CONNECT_DATA =
> (SID = FPROD)
> )
> )

This is a different SID!
It suggests your ColdFusion environment settings are
defaulted (ORCL would be a default SID), so change them.
Also, system would be an extremely bad idea.

Another possibility is, of course, you are being hacked.
> ---------------------------------------------------------------
>
> the listener.ora on the DB server has no BEQ protocol either. Here it
> is:

No of course not!
BEQUEATH is a direct call (any c or Unix-programmer recognizes
ARGV0 and ARGS) - you don't even *need* a listener for BEQueath
connections, just the correct environment variables (ORACLE_HOME
and ORACLE_SID)

--
Regards,
Frank van Bortel

Top-posting is one way to shut me up...

Thank you Frank, that's very useful

> NOTE: in all logs we just replaced real host names with XXXX
this is just paranoid me. I hope it doesn't offend you!

I am not getting it only from my ColdFusion environment, As you can see
below I am also getting the error from inetinfo.exe
------------------------------------------------------------------
Fatal NI connect error 12560, connecting to:

(DESCRIPTION=(ADDRESS=(PROTOCOL=BEQ)(PROGRAM=oracl e)(ARGV0=oracleORCL)(ARGS='(DESCRIPTION=(LOCAL=YES )(ADDRESS=(PROTOCOL=beq)))'))(CONNECT_DATA=(SID=OR CL)(CID=(PROGRAM=C:\WINNT\System32\inetsrv\inetinf o.exe)(HOST=XXXXXX)(USER=SYSTEM))))

VERSION INFORMATION:
TNS for 32-bit Windows: Version 8.1.7.0.0 - Production
Oracle Bequeath NT Protocol Adapter for 32-bit Windows: Version
8.1.7.0.0 - Production
Windows NT TCP/IP NT Protocol Adapter for 32-bit Windows: Version
8.1.7.0.0 - Production
Time: 08-MAR-2006 13:34:09
.....
.....
-------------------------------------------------------------------------------------
if I understand correctly BEQ is meant to use locally, so any calls to
BEQ cannot be originated from our programs. there is no DB installed at
our web servers! so does it mean its a hack, spyware, etc. trying to
access the DB using def values? would that explain why the web server
"chokes" when it happens? also, why does it happen to most web servers
simultaneously?

do you have any knowledge on specific malware that works like that?

our IIS seettings declare the user to run our pages, so the SYSTEM does
not seem to be from our application either

is there any other direction we should be looking at?


thanks again,

Roy

First hit while searching on inetinfo.exe:

Note: inetinfo.exe is also a process which is registered as the
Trojan.W32.RONTOKBRO. This virus is distributed via the Internet
through e-mail and comes in the form of an e-mail message, in the hopes
that you open it's hostile attachment. The worm has it's own SMTP
engine which means it gathers E-mails from your local computer and
re-distributes itself. In worst cases this worm can allow attackers to
access your computer, stealing passwords and personal data. It is a
registered security risk and should be removed immediately.

Take it from there