Hi people,

I need some help to design a Disaster Recovery solution.

The idea is to make a replication server and each one hour, for
example, update content from one to other.

I've thought on Progress, Proquiet and Snapshots on Solaris (with
fssnap). Can it work ? What is your experience on them ?

I've suggest Fathom HA but for money reasons, it cannot be possible,
at this moment.

The other idea is to use rsync command to send files in secure mode,
at this case, Progress files, database, etc. Do you have implement it
?

I wait your suggestions.

Thanks,

Toni

PS: I'm a non-experiened on Progress database

We currently use this method with a 9.1D system but it has problems.
1. Due to a bug in progress, you can never truncate the bi files on the DR
machine (there is a knowledge base article on this but I forget the number),
if you do it may cause the AI files to stop applying (but it may not!).
2. Progress are still investigating another possible bug where the DR system
is not re using BI space (this is how we found out about #1).
3. When generating DB backups to initially set up or rebuild the databases
on the DR machine, remeber to use the -norecover option, otherwise the DR
system may not apply the AI files (but again, it may!).

When we implemented this (about18 months ago) Progress reccomended this
method, but due to the amount of problems reported they no longer do.

Another tip, make sure your method of transfering the AI files is solid. We
have a problem whereby the Live server will re-send an AI file if it hasn't
been processed from the last transfer (incase it never got there), if the AI
file is in the process of being applied on the DR machine, Progress really
gets confused and starts reporting Disk Full errors. A rebuild of the DR DB
is then required.

I wouldn't reccomend it for mission critical systems if you are
inexperienced. On the other hand it is an excelent way to learn (the hard
way!) how AI works.

Oh and another thing, make sure you start AI on your live server BEFORE you
back up the DB's. Starting AI updates the DB internal date and time stamps
so they are then out of sync with the backups. (Another one we learnt the
hard way)!

Cheers,
Rob.

"Kevin" <Joe@Joe.com> wrote in message
news:Xns93ACAC7FB734FJoeJoecom@195.8.68.207...
> tonid_news@hotmail.com (ToniD) wrote in
> news:c0de09b.0307020456.7b0c981b@posting.google.co m:
>
> > Hi people,
> >
> > I need some help to design a Disaster Recovery solution.
> >
> > The idea is to make a replication server and each one hour, for
> > example, update content from one to other.
> >
> > I've thought on Progress, Proquiet and Snapshots on Solaris (with
> > fssnap). Can it work ? What is your experience on them ?
> >
> > I've suggest Fathom HA but for money reasons, it cannot be possible,
> > at this moment.
> >
> > The other idea is to use rsync command to send files in secure mode,
> > at this case, Progress files, database, etc. Do you have implement it
> > ?
> >
> > I wait your suggestions.
> >
> > Thanks,
> >
> > Toni
> >
> > PS: I'm a non-experiened on Progress database
> >
>
> The easiest method is log based replication.. Backup your DB, enable AI.
> Restore the backup on the replication server. Write a script on the
> production machine that checks every x minutes for the oldest full file..
> If no full file exists, force AI to switch and grab the full file and
> copy it to the replication server. Make sure the file arrived okay, and
> mark it as empty on the production machine. Have another job on the
> replication server that checks for the existance of the file, and applies
> it to the restored db..
>
> Keep in mind that a 50% license fee is required from Progress for the
> replication machine.
>
> I guess Proquiet and Snapshots on Soalris would work fine, but there will
> always be a level of interruption, which you may not want.
>
> Kevin

"Rob Lyons" <rob.lyons@blueyonder.co.uk> wrote in
news:MEGMa.72$p16.4@news-binary.blueyonder.co.uk:

> We currently use this method with a 9.1D system but it has problems.
> 1. Due to a bug in progress, you can never truncate the bi files on
> the DR machine (there is a knowledge base article on this but I forget
> the number), if you do it may cause the AI files to stop applying (but
> it may not!). 2. Progress are still investigating another possible bug
> where the DR system is not re using BI space (this is how we found out
> about #1). 3. When generating DB backups to initially set up or
> rebuild the databases on the DR machine, remeber to use the -norecover
> option, otherwise the DR system may not apply the AI files (but again,
> it may!).
>
> When we implemented this (about18 months ago) Progress reccomended
> this method, but due to the amount of problems reported they no longer
> do.
>
> Another tip, make sure your method of transfering the AI files is
> solid. We have a problem whereby the Live server will re-send an AI
> file if it hasn't been processed from the last transfer (incase it
> never got there), if the AI file is in the process of being applied on
> the DR machine, Progress really gets confused and starts reporting
> Disk Full errors. A rebuild of the DR DB is then required.
>
> I wouldn't reccomend it for mission critical systems if you are
> inexperienced. On the other hand it is an excelent way to learn (the
> hard way!) how AI works.
>
> Oh and another thing, make sure you start AI on your live server
> BEFORE you back up the DB's. Starting AI updates the DB internal date
> and time stamps so they are then out of sync with the backups.
> (Another one we learnt the hard way)!
>
> Cheers,
> Rob.
>

There is a bug #20030327-035 which will be fixed in 9.1D07 to do with
inconsistent in-flight information in AI files and truncating the bi
file, but it would appear that it is very sporadic..

Bug #20030122-011 was logged for 9.1D05 and was fixed in 9.1D06.
Apparantley this only manifests itself if the target BI cluster size is
larger than the source cluster size.

Hope this is of some use.

Kevin

"Rob Lyons" <rob.lyons@blueyonder.co.uk> wrote in
news:MEGMa.72$p16.4@news-binary.blueyonder.co.uk:

> We currently use this method with a 9.1D system but it has problems.
> 1. Due to a bug in progress, you can never truncate the bi files on
> the DR machine (there is a knowledge base article on this but I forget
> the number), if you do it may cause the AI files to stop applying (but
> it may not!).

This is not a bug, it's part of the way Progress databases work.
If crash recovery has not been run, it will when you truncate
the .bi. That changes the state of the database and that's
why it prevents applying .ai files - because the database changed
and no longer is the same as the original database.

The reason for it not doing that all the time is when you
already truncated the .bi and it had no notes to apply for
crash recovery. Then it might as well be the same thing.

> 2. Progress are still investigating another possible bug
> where the DR system is not re using BI space (this is how we found out
> about #1).


Also not a bug, but a very unfortunate part of the circular
ring method used for the .bi. That comes from long standing
transactions inside the database. That prevents one cluster
from being closed and as soon as that cluster becomes the
first in the ring, all other empty clusters cannot be reused.
And so the .bi grows with new clusters.

> 3. When generating DB backups to initially set up or
> rebuild the databases on the DR machine, remeber to use the -norecover
> option, otherwise the DR system may not apply the AI files (but again,
> it may!).
>

Exactly - that's the solution for not forcing the crash recovery
that would modify the database and invalidate it against new .ai files.
Stay clear of that, and you're fine.

> When we implemented this (about18 months ago) Progress reccomended
> this method, but due to the amount of problems reported they no longer
> do.
>

Actually they do recommend it.

Thanks for all your points, it seems yet again that the Progress community
know more than Progress themselves!

I referred to them as bugs as that it what Progress called them!

As for the 06 service pack, we installed this recently and are monitoring.
However we do not hold much hope as Progress also informed us that the BI
file growth "bug" was fixed in 03, 04 and 05, despite no sign of it in the
release notes (none in 06 either).
The reason we truncated the BI files on our DR system is Progress told us
to! 2 of the 4 databases failed to apply AI files. We were then passed to
another support person (Bodo Manke, cool name) who said that the bug was
that you could truncate the BI files on a standby database and rfutil
shouldn't let you!

I wouldn't mind some more info on the circular ring method mentioned. We
have read all that Progress can offer on how the AI roll forward works with
standby BI files, but nothing seems to explain what happens.
E.G. We have a standby DB with 2 BI extents. One stays at 40MB but the other
one seems to randomly grow for no reason, it is currently at 680MB and
seems to jump around 80 - 120 MB at a time when it applies AI files of only
20 - 50 MB. Every few months it blows the 2GB limit and we have to rebuild
the DB. This is why Progress recommended truncating the BI. Which didn't
work.

It was the afore mentioned Bodo that told us that Progress were no longer
recommending this method because of the amount of problems reported (however
he was trying to sell us Fathom at the same time).

Regards,
Rob.


<pro> wrote in message news:Xns93AD6133C57EApro@216.168.3.44...
> "Rob Lyons" <rob.lyons@blueyonder.co.uk> wrote in
> news:MEGMa.72$p16.4@news-binary.blueyonder.co.uk:
>
> > We currently use this method with a 9.1D system but it has problems.
> > 1. Due to a bug in progress, you can never truncate the bi files on
> > the DR machine (there is a knowledge base article on this but I forget
> > the number), if you do it may cause the AI files to stop applying (but
> > it may not!).
>
> This is not a bug, it's part of the way Progress databases work.
> If crash recovery has not been run, it will when you truncate
> the .bi. That changes the state of the database and that's
> why it prevents applying .ai files - because the database changed
> and no longer is the same as the original database.
>
> The reason for it not doing that all the time is when you
> already truncated the .bi and it had no notes to apply for
> crash recovery. Then it might as well be the same thing.
>
> > 2. Progress are still investigating another possible bug
> > where the DR system is not re using BI space (this is how we found out
> > about #1).
>
>
> Also not a bug, but a very unfortunate part of the circular
> ring method used for the .bi. That comes from long standing
> transactions inside the database. That prevents one cluster
> from being closed and as soon as that cluster becomes the
> first in the ring, all other empty clusters cannot be reused.
> And so the .bi grows with new clusters.
>
> > 3. When generating DB backups to initially set up or
> > rebuild the databases on the DR machine, remeber to use the -norecover
> > option, otherwise the DR system may not apply the AI files (but again,
> > it may!).
> >
>
> Exactly - that's the solution for not forcing the crash recovery
> that would modify the database and invalidate it against new .ai files.
> Stay clear of that, and you're fine.
>
> > When we implemented this (about18 months ago) Progress reccomended
> > this method, but due to the amount of problems reported they no longer
> > do.
> >
>
> Actually they do recommend it.
>
>

"Rob Lyons" <rob.lyons@blueyonder.co.uk> wrote in
news:7V_Ma.116$BD5.38@news-binary.blueyonder.co.uk:

> Thanks for all your points, it seems yet again that the Progress
> community know more than Progress themselves!
>
> I referred to them as bugs as that it what Progress called them!

As I said in my other post, there are bugs logged for these issues,
therefore, they are bugs.. :-)

>
> As for the 06 service pack, we installed this recently and are
> monitoring. However we do not hold much hope as Progress also informed
> us that the BI file growth "bug" was fixed in 03, 04 and 05, despite
> no sign of it in the release notes (none in 06 either).

This is often the case if there is no work request associated with the
bug entry. In the bug DB, the WR# is 'Internal' (for one of the bugs I
mentioned earlier), which generally means someone was testing something
and found the problem, and it will not find it's way into the Service
Pack release notes.

> The reason we truncated the BI files on our DR system is Progress told
> us to! 2 of the 4 databases failed to apply AI files. We were then
> passed to another support person (Bodo Manke, cool name) who said that
> the bug was that you could truncate the BI files on a standby database
> and rfutil shouldn't let you!

Nice idea, but that is not enforcable. rfutil (or proutil rather) has no
idea that this db is going to be participating in future roll forward
operations, therefore we cannot stop anyone from truncating the BI as it
might be part of the "recovery operation". Perhaps a message that the DB
has recently participated in a roll forward operation, asking if you want
to go ahead with the truncate..

>
> I wouldn't mind some more info on the circular ring method mentioned.
> We have read all that Progress can offer on how the AI roll forward
> works with standby BI files, but nothing seems to explain what
> happens. E.G. We have a standby DB with 2 BI extents. One stays at
> 40MB but the other one seems to randomly grow for no reason, it is
> currently at 680MB and seems to jump around 80 - 120 MB at a time when
> it applies AI files of only 20 - 50 MB. Every few months it blows the
> 2GB limit and we have to rebuild the DB. This is why Progress
> recommended truncating the BI. Which didn't work.

What OS are you running?? Unless you are on Linux or SCO, then 2GB is no
longer a limit. BI files in V9.1B and up can be up to 32TB. SCO & Linux
are still limited to 2GB per file, but you can have multiple of them. I
know this does no solve the issue, but it can prevent you from hitting
the limit.. Potentially, backup the DB, truncate and attempt a roll
forward. If it works, great. If not, restore and try roll forward again,
letting the BI grow a little more..

The weird grwoth you describe has been seen before, but only under very
specific circumstances. Do you have the WR number from Tech support??

What is the current BI cluster size and -G on both the production and
warm spare DBs?

>
> It was the afore mentioned Bodo that told us that Progress were no
> longer recommending this method because of the amount of problems
> reported (however he was trying to sell us Fathom at the same time).

Understandable.. :-) I haven't seen this before, and don't have the time
this week to run any tests, but early next week I should be able to do
some testing.

>
> Regards,
> Rob.
>
>

Cheers,

Kevin

Cheers Kev, escuse the different name, I am logging on from work!.

Work No's are (we have had several as Progress keep closing it and
re-opening it). My manager is now dealing with it so not sure of the
current status.

W302200028

W305130050

RobL <RobL.q7b4m@progresstalk.com> wrote in
news:RobL.q7b4m@progresstalk.com:

>
> Cheers Kev, escuse the different name, I am logging on from work!.
>
> Work No's are (we have had several as Progress keep closing it and
> re-opening it). My manager is now dealing with it so not sure of the
> current status.
>
> W302200028
>
> W305130050
>
>
>

No problem.. I'll have a look as soon as I am back in the office (tomorrow
morning)..

Kevin

RobL <RobL.q7b4m@progresstalk.com> wrote in
news:RobL.q7b4m@progresstalk.com:

>
> Cheers Kev, escuse the different name, I am logging on from work!.
>
> Work No's are (we have had several as Progress keep closing it and
> re-opening it). My manager is now dealing with it so not sure of the
> current status.
>
> W302200028
>
> W305130050
>
>
>

Hi Rob,

Had a look at the two work requests.. Both are currently closed.. As
mentioned in W305130050 bug #20030122-011 was an internal thing,
apparantley to do with internal monitoring of the bi size, not bi growth.

Since the 9.1D06 serivce pack (I see the las comm was 17/06/03) has there
been any change in the behaviour?? Does the bi still grow??

Do you still have 9.1C on the box?? If you do, do me a favour and check the
startup.pf file in $DLC, specifically for any -G entries..

Cheers,

Kevin

Thanks Kevin,
I am not at work today so can not check, however know -G isn't used as I
checked this before for Bodo.
Things have seemed better for a while (no recent BI growth) so it may be OK
now with 06 on.
We upgraded to 9.1D a while ago so this may have helped.

Thanks for looking into it.
Rob.

"Kevin" <Joe@Joe.com> wrote in message
news:Xns93B2CA322B53EJoeJoecom@217.158.240.11...
> RobL <RobL.q7b4m@progresstalk.com> wrote in
> news:RobL.q7b4m@progresstalk.com:
>
> >
> > Cheers Kev, escuse the different name, I am logging on from work!.
> >
> > Work No's are (we have had several as Progress keep closing it and
> > re-opening it). My manager is now dealing with it so not sure of the
> > current status.
> >
> > W302200028
> >
> > W305130050
> >
> >
> >
>
> Hi Rob,
>
> Had a look at the two work requests.. Both are currently closed.. As
> mentioned in W305130050 bug #20030122-011 was an internal thing,
> apparantley to do with internal monitoring of the bi size, not bi growth.
>
> Since the 9.1D06 serivce pack (I see the las comm was 17/06/03) has there
> been any change in the behaviour?? Does the bi still grow??
>
> Do you still have 9.1C on the box?? If you do, do me a favour and check
the
> startup.pf file in $DLC, specifically for any -G entries..
>
> Cheers,
>
> Kevin