Hi,

I'm currently involved in a evaluation of an application that uses of Oracle
RAC to scale horizontally.

At full production volumes the supplier is recommending a 12-node database
cluster - using 2CPU commodity servers. Although the transaction rates are
likely to be low, probably only:
- 1 update transaction (consisting of perhaps 10 updates/inserts) per
second.
- 10 queries/sec
on average during the day with more queries and less updates overnight .
Peak volumes are unlikely to be significantly higher.

In the application architecture of the product some application components
are hosted on the database server. This provides a standard building block
to scale the application - so the vendor says.

Normally I would expect to separate the application tier from the database
tier, and believe that a single database server (4CPU's perhaps) would
easily be able to handle this kind of transaction load. But with this
shared server the vendor recommends scaling is by adding these combined
application/database server's into the cluster.

My understanding is that Oracle RAC would typically be deployed with < 5
nodes. So I'm looking to identify any real world examples of RAC clusters
and the transaction rates/numbers of nodes.

I would also be interested to know if there are:
- any recommendations as to the maximum number of nodes in a RAC cluster
- any RAC design principles that we should ensure the vendor has followed
- areas we could/should monitor when performing an assessment of the
application.
- ways of predicting RAC performance as additional nodes are added (we are
unlikely to be able to test the application with 12 nodes so will have to
factor this up from fewer nodes).

Any other comments on the general stability of the Oracle RAC platform in
real world production environments would also be appreciated.

regards

Ian

For the transaction load mentioned I do not see why you would need 12
nodes even using two-cpu (Windows?) machines. If the vendor is the
software product vendor and they recommend 12 nodes then I would
suspect that thier application is the primary reason why and not the
database.

I believe that a very important factor is Oracle database performance
is the disk farm. Exactly what kind of disk is going to be used here?

Depending on the project time frame and hardward procurement lead time
you might want to consider building a smaller cluster, testing, and
scaling up only if necessary.

We run a far higher transaction load using only a two node cluster with
multiple RAC databases running on the two six-cpu machines. Each
machine is under 50% so that in the event a machine dies and will take
more than a couple of hours to recover we can run everything on the
remaining machine. But then we run under AIX on the Power PC so that
leaves WinTel in the dirt.

HTH -- Mark D Powell --

Ian Turner wrote:

> Hi,
>
> I'm currently involved in a evaluation of an application that uses of Oracle
> RAC to scale horizontally.
>
> At full production volumes the supplier is recommending a 12-node database
> cluster - using 2CPU commodity servers. Although the transaction rates are
> likely to be low, probably only:
> - 1 update transaction (consisting of perhaps 10 updates/inserts) per
> second.
> - 10 queries/sec
> on average during the day with more queries and less updates overnight .
> Peak volumes are unlikely to be significantly higher.
>
> In the application architecture of the product some application components
> are hosted on the database server. This provides a standard building block
> to scale the application - so the vendor says.
>
> Normally I would expect to separate the application tier from the database
> tier, and believe that a single database server (4CPU's perhaps) would
> easily be able to handle this kind of transaction load. But with this
> shared server the vendor recommends scaling is by adding these combined
> application/database server's into the cluster.
>
> My understanding is that Oracle RAC would typically be deployed with < 5
> nodes. So I'm looking to identify any real world examples of RAC clusters
> and the transaction rates/numbers of nodes.
>
> I would also be interested to know if there are:
> - any recommendations as to the maximum number of nodes in a RAC cluster
> - any RAC design principles that we should ensure the vendor has followed
> - areas we could/should monitor when performing an assessment of the
> application.
> - ways of predicting RAC performance as additional nodes are added (we are
> unlikely to be able to test the application with 12 nodes so will have to
> factor this up from fewer nodes).
>
> Any other comments on the general stability of the Oracle RAC platform in
> real world production environments would also be appreciated.
>
> regards
>
> Ian

Based on what you've written I'd hide my checkbook and find another
vendor.

You are incorrect that RAC is typically deployed with less than 5 nodes:
I've seen lots of 8+ node clusters. But I don't see based on what you've
written that such is required.

I would suggest that you start with 2 x 2CPU nodes and then buy more
if required by testing.

More troubling is that this may well be an indication that they are
providing rotten software with block contention issues. If you contact
me off-line I can put you in touch with a lab that will allow you to
pre-test the application on RAC clusters before you buy.
--
Daniel A. Morgan
University of Washington
damorgan@x.washington.edu
(replace 'x' with 'u' to respond)

> You are incorrect that RAC is typically deployed with less than 5
nodes:

Synonym for typical ... regular.

What statistics can you show give evidence that RAC is typically
deployed with more than 4 nodes? Possibly oracle has that information
.... let's see you dig something up that we can authenticate. Proving a
statement ... Tom Kyte does that regularly. So does Mr. Lewis. Other
people just make up stuff.

Maybe it's just an english language thing. Seeing 8+ node clusters
doesn't mean they are in the majority or even close ... which would
meet the definition of typical.

The rest of what Mr. Morgan said appears correct except for the
possibly wild speculation about block contention.

Personally I would recommend vendor references that might include
visits to current customers and on-site demonstrations with current
customers of the application vendor application running in production.
Have them explain their recommended architecture and pro's/con's ...
and keep drilling for detail.

Maybe it's just a lack of information or not enough information but
something doesn't sound right so far.

As an ex-Oracle employee and one of the RAC specialists, I saw (and still
see) a heck of a lot of RAC installations. There is no "typical", but there
are certainly more 2 to 4 node installations out there than 12 nodes.

The most unsettling about this "vendor" that I see is that running the
middle-tier and the database (especially RAC) on the same servers is a big
problem.

I suspect this vendors product has not been design or optimized for the
Oracle technical stack. Do yourself a favor and bring in some outside help
with this one before letting any PO's out.

Unless there are some giant connection and analytics requirements, a 12-node
cluster might be wrong in this case. I have seen simple 2 CPU, 2 node
clusters handle giant jobs - the need for 12 nodes may very well be due to
poor block contention schema design. Until you ask some questions and
perhaps get some help, I'd put the brakes on this on.

If for no other reason than running the middle-tier and the database on the
same hardware is just plain wrong; certainly will invalidate any perceived
HA gains from RAC.


On 4/19/05 5:50 PM, in article
1113958253.931994.62960@g14g2000cwa.googlegroups.c om, "hpuxrac"
<johnbhurley@sbcglobal.net> wrote:

>> You are incorrect that RAC is typically deployed with less than 5
> nodes:
>
> Synonym for typical ... regular.
>
> What statistics can you show give evidence that RAC is typically
> deployed with more than 4 nodes? Possibly oracle has that information
> ... let's see you dig something up that we can authenticate. Proving a
>
> statement ... Tom Kyte does that regularly. So does Mr. Lewis. Other
> people just make up stuff.
>
> Maybe it's just an english language thing. Seeing 8+ node clusters
> doesn't mean they are in the majority or even close ... which would
> meet the definition of typical.
>
> The rest of what Mr. Morgan said appears correct except for the
> possibly wild speculation about block contention.
>
> Personally I would recommend vendor references that might include
> visits to current customers and on-site demonstrations with current
> customers of the application vendor application running in production.
> Have them explain their recommended architecture and pro's/con's ...
> and keep drilling for detail.
>
> Maybe it's just a lack of information or not enough information but
> something doesn't sound right so far.
>

Here a short general rule of thumb for RAC:

(1) If you need PERFORMANCE - scale vertically (put more cpu's,
memory, etc.) in your database machine.

(2) If you need HIGHAVAILABILITY scale horizontal means - use a cluster.

It is true that if you add cluster nodes instead of scaling vertically -
you will gain some more performance BUT - not as much as scaling
vertically - and at a higher cost (hardware + more administration)

Walter Dorninger

On Tue, 19 Apr 2005 13:39:45 +0100, Ian Turner wrote:

> Hi,
>
> I'm currently involved in a evaluation of an application that uses of Oracle
> RAC to scale horizontally.
>
> At full production volumes the supplier is recommending a 12-node database
> cluster - using 2CPU commodity servers. Although the transaction rates are
> likely to be low, probably only:
> - 1 update transaction (consisting of perhaps 10 updates/inserts) per
> second.
> - 10 queries/sec
> on average during the day with more queries and less updates overnight .
> Peak volumes are unlikely to be significantly higher.
>
> In the application architecture of the product some application components
> are hosted on the database server. This provides a standard building block
> to scale the application - so the vendor says.
>
> Normally I would expect to separate the application tier from the database
> tier, and believe that a single database server (4CPU's perhaps) would
> easily be able to handle this kind of transaction load. But with this
> shared server the vendor recommends scaling is by adding these combined
> application/database server's into the cluster.
>
> My understanding is that Oracle RAC would typically be deployed with < 5
> nodes. So I'm looking to identify any real world examples of RAC clusters
> and the transaction rates/numbers of nodes.
>
> I would also be interested to know if there are:
> - any recommendations as to the maximum number of nodes in a RAC cluster
> - any RAC design principles that we should ensure the vendor has followed
> - areas we could/should monitor when performing an assessment of the
> application.
> - ways of predicting RAC performance as additional nodes are added (we are
> unlikely to be able to test the application with 12 nodes so will have to
> factor this up from fewer nodes).
>
> Any other comments on the general stability of the Oracle RAC platform in
> real world production environments would also be appreciated.
>
> regards
>
> Ian

Ian Turner wrote:
>
> Hi,
>
> I'm currently involved in a evaluation of an application that uses of Oracle
> RAC to scale horizontally.
>
> At full production volumes the supplier is recommending a 12-node database
> cluster - using 2CPU commodity servers. Although the transaction rates are
> likely to be low, probably only:
> - 1 update transaction (consisting of perhaps 10 updates/inserts) per
> second.
> - 10 queries/sec
> on average during the day with more queries and less updates overnight .
> Peak volumes are unlikely to be significantly higher.
>
> In the application architecture of the product some application components
> are hosted on the database server. This provides a standard building block
> to scale the application - so the vendor says.
>
> Normally I would expect to separate the application tier from the database
> tier, and believe that a single database server (4CPU's perhaps) would
> easily be able to handle this kind of transaction load. But with this
> shared server the vendor recommends scaling is by adding these combined
> application/database server's into the cluster.
>
> My understanding is that Oracle RAC would typically be deployed with < 5
> nodes. So I'm looking to identify any real world examples of RAC clusters
> and the transaction rates/numbers of nodes.
>
> I would also be interested to know if there are:
> - any recommendations as to the maximum number of nodes in a RAC cluster
> - any RAC design principles that we should ensure the vendor has followed
> - areas we could/should monitor when performing an assessment of the
> application.
> - ways of predicting RAC performance as additional nodes are added (we are
> unlikely to be able to test the application with 12 nodes so will have to
> factor this up from fewer nodes).
>
> Any other comments on the general stability of the Oracle RAC platform in
> real world production environments would also be appreciated.
>
> regards
>
> Ian

One very simple issue will be the license cost for a 12-node
RAC....ouch!
--
Connor McDonald
Co-author: "Mastering Oracle PL/SQL - Practical Solutions"
Co-author: "Oracle Insight - Tales of the OakTable"

web: http://www.oracledba.co.uk
web: http://www.oaktable.net
email: connor_mcdonald@yahoo.com


"GIVE a man a fish and he will eat for a day. But TEACH him how to fish,
and...he will sit in a boat and drink beer all day"

------------------------------------------------------------

hpuxrac wrote:

> Maybe it's just an english language thing. Seeing 8+ node clusters
> doesn't mean they are in the majority or even close ... which would
> meet the definition of typical.

Not one of the RAC projects I have worked or bid on this year has been
less than 4 nodes. So while there may be many of them ... "typical"
does indeed have a meaning and "most" <> "typical" in English.
--
Daniel A. Morgan
University of Washington
damorgan@x.washington.edu
(replace 'x' with 'u' to respond)

Connor McDonald wrote:

> One very simple issue will be the license cost for a 12-node
> RAC....ouch!

Unless one really needs those 24 CPUs. What is the cost delta
between 24 RAC licenses (one per CPU) and the cost of the
hardware? One should keep in mind too that with this quantity
the person pricing the licenses might find room for flexibility.
--
Daniel A. Morgan
University of Washington
damorgan@x.washington.edu
(replace 'x' with 'u' to respond)

Hi,

For us licensing isn't an issue in terms of cost.... we have an all you can
eat Oracle license.

thanks

Ian
"DA Morgan" <damorgan@x.washington.edu> wrote in message
news:1114012378.142107@yasure...
> Connor McDonald wrote:
>
> > One very simple issue will be the license cost for a 12-node
> > RAC....ouch!
>
> Unless one really needs those 24 CPUs. What is the cost delta
> between 24 RAC licenses (one per CPU) and the cost of the
> hardware? One should keep in mind too that with this quantity
> the person pricing the licenses might find room for flexibility.
> --
> Daniel A. Morgan
> University of Washington
> damorgan@x.washington.edu
> (replace 'x' with 'u' to respond)