Unix Technical Forum

Massive Delete

This is a discussion on Massive Delete within the Informix forums, part of the Database Server Software category; --> Hello Again Friends, Thanks for the help you've provided me so far. So... Here are some stats on a ...


Go Back   Unix Technical Forum > Database Server Software > Informix

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-20-2008, 01:08 PM
Tam OShanter
 
Posts: n/a
Default Massive Delete

Hello Again Friends,
Thanks for the help you've provided me so far.

So...
Here are some stats on a table I have:

total record count 20,454,537.00
total record count to be moved to DW and deleted 11,000,921.00

row size of 413 bytes.

How do I go about deleting this data without rendering my server unusable?

I saw a suggestion of deleting records in 500 record batches, but that is a
helluva lot of batches to get done what I want.

Any other ideas?

Thanks in advance for the advice.

Tam.


Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-20-2008, 01:08 PM
Quman
 
Posts: n/a
Default Re: Massive Delete

Normally, we have a script(perl,...) to do this,

Inside the loop, we can control the number of rows we want to delete
in a short transaction. You can also have a couple of seconds sleep
for next round if you like.

You can just stopt(CTRL-C) it any time and restart it later.

thanks,
Quman

On 9/26/06, Tam OShanter <tam@oshanter.com> wrote:
> Hello Again Friends,
> Thanks for the help you've provided me so far.
>
> So...
> Here are some stats on a table I have:
>
> total record count 20,454,537.00
> total record count to be moved to DW and deleted 11,000,921.00
>
> row size of 413 bytes.
>
> How do I go about deleting this data without rendering my server unusable?
>
> I saw a suggestion of deleting records in 500 record batches, but that is a
> helluva lot of batches to get done what I want.
>
> Any other ideas?
>
> Thanks in advance for the advice.
>
> Tam.
>
>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-20-2008, 01:08 PM
Tam OShanter
 
Posts: n/a
Default Re: Massive Delete

Okay,
So I cannot do this in one transaction without killing my server so...
Given my row size, what considerations do I have when trying to decide the
optimal number of rows to delete in a given batch?

Thoughts?

Tam.

"Quman" <yquman@gmail.com> wrote in message
news:mailman.630.1159295524.20706.informix-list@iiug.org...
> Normally, we have a script(perl,...) to do this,
>
> Inside the loop, we can control the number of rows we want to delete
> in a short transaction. You can also have a couple of seconds sleep
> for next round if you like.
>
> You can just stopt(CTRL-C) it any time and restart it later.
>
> thanks,
> Quman
>
> On 9/26/06, Tam OShanter <tam@oshanter.com> wrote:
>> Hello Again Friends,
>> Thanks for the help you've provided me so far.
>>
>> So...
>> Here are some stats on a table I have:
>>
>> total record count 20,454,537.00
>> total record count to be moved to DW and deleted 11,000,921.00
>>
>> row size of 413 bytes.
>>
>> How do I go about deleting this data without rendering my server
>> unusable?
>>
>> I saw a suggestion of deleting records in 500 record batches, but that is
>> a
>> helluva lot of batches to get done what I want.
>>
>> Any other ideas?
>>
>> Thanks in advance for the advice.
>>
>> Tam.
>>
>>
>> _______________________________________________
>> Informix-list mailing list
>> Informix-list@iiug.org
>> http://www.iiug.org/mailman/listinfo/informix-list
>>



Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-20-2008, 01:08 PM
bozon
 
Posts: n/a
Default Re: Massive Delete

Quman,
How does the script limit the number of records deleted? There isn't a
command to do "delete first 50 from <table> " is there ?
I did something similar to fix in place alters ( I of course updated a
column that wasn't in an index or trigger, instead of deleting the
record) but to get the records to work on I unloaded the primary key of
the records (because you can't even "select first 50 <pk> from <table>
into temp nibble_<table> with no log

If you are going to do this during down time you can do the following:

begin work;

rename <table> to old_<table> ;

create table <table> (...);

alter table <table> type(raw);
insert into <table> select * from old_<table> where <records_to_keep>;
alter table <table> type(standard);
drop table old_<table>;

<create indexes>
<create constraints>
<recreate fk constraints from other tables>

commit work;

Since the inserts are being done to a raw table it won't be logged (of
course it won't be replicated either).
Quman wrote:
> Normally, we have a script(perl,...) to do this,
>
> Inside the loop, we can control the number of rows we want to delete
> in a short transaction. You can also have a couple of seconds sleep
> for next round if you like.
>
> You can just stopt(CTRL-C) it any time and restart it later.
>
> thanks,
> Quman
>
> On 9/26/06, Tam OShanter <tam@oshanter.com> wrote:
> > Hello Again Friends,
> > Thanks for the help you've provided me so far.
> >
> > So...
> > Here are some stats on a table I have:
> >
> > total record count 20,454,537.00
> > total record count to be moved to DW and deleted 11,000,921.00
> >
> > row size of 413 bytes.
> >
> > How do I go about deleting this data without rendering my server unusable?
> >
> > I saw a suggestion of deleting records in 500 record batches, but that is a
> > helluva lot of batches to get done what I want.
> >
> > Any other ideas?
> >
> > Thanks in advance for the advice.
> >
> > Tam.
> >
> >
> > _______________________________________________
> > Informix-list mailing list
> > Informix-list@iiug.org
> > http://www.iiug.org/mailman/listinfo/informix-list
> >


Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-20-2008, 01:08 PM
Quman
 
Posts: n/a
Default Re: Massive Delete

I attached an esql.ec file. That can do our job.
If your table does not have unique ID column, you need modfy it.

I prefer using PERL to do it now. this is our early version.

Thanks,
Quman


/*
* File: CleanupRepTab.ec
*
* CleanupRepTab.ec is an ESQL/C program to cleanup database tables that
* are replicated by Informix Enterprise Replication server between multiple
* sites. For each table to be cleaned, there is a cleanup SQL WHERE clause
* stored in the db_cleanup_rep table. CleanupRepTab.ec will lookup the
* db_cleanup_rep table and delete the rows which satisfy the condition.
* db_cleanup_rep has the following schema,
* create table db_cleanup_rep (
* id serial not null ,
* tabname char(32) not null ,
* colname char(32) not null ,
* delcond varchar(255),
* status char(8),
* primary key (id) constraint db_cleanup_rep_pk );
*
* id: a serial number generated for each row.
* tabname: the name of the table to be cleaned, e.g., arch_recall_reqs,
* file_change_notice, etc.
* colname: a column name of the table, its type must be Integer and Unique.
* delcond: the condition used to delete rows of the table.
* status: "active" means the clean activity is on, "inactive" means off.

* The final results of cleanup are stored in two files: db_cleanup_sum.msg and
* db_cleanup_all.msg. db_cleanup_sum.msg contains the summary about
the deletion
* for each table. db_cleanup_all.msg contains all the ids of cleaned rows for
* each table.
*/


#include <stdio.h>
#include <time.h>
#include <sys/types.h>
#include <stdlib.h>
#include <sqlhdr.h>
#include <sqlca.h>

#define TABLIMIT 100 /*number of tables can be handled */
#define ROWLIMIT 320000 /*max number of rows in a table handled at one
time run*/

void deleteRows(char* tabname, char* colname,
char* delcond, FILE *fp_sum, FILE *fp_all){

int ids[ROWLIMIT];
int thecount=0;

EXEC SQL BEGIN DECLARE SECTION;
int theid;
char buff[255]; char buff2[255];
EXEC SQL END DECLARE SECTION;

/* select the row ids into a array*/
sprintf(buff, " SELECT %s FROM %s WHERE %s INTO TEMP toDelRow
WITH NO LOG \n",
colname, tabname, delcond);
fputs(buff, fp_sum);

/*printf("stmt1 : %s\n \n", buff);*/
EXEC SQL EXECUTE IMMEDIATE :buff;
EXEC SQL PREPARE sele FROM 'select * from toDelRow';
EXEC SQL DECLARE C2 CURSOR FOR sele;
EXEC SQL OPEN C2;

for (; ; ) {

EXEC SQL FETCH C2 into :theid;
if ( sqlca.sqlcode != 0 || thecount > ROWLIMIT) { break; }
ids[thecount]=theid;
/*printf(" i= %d id= %d \n", thecount,theid);*/
thecount++;
}

EXEC SQL CLOSE C2;
EXEC SQL DROP TABLE toDelRow;

/* delete from target table row by row */
for ( int i=0;i<thecount; i++) {

sprintf(buff2,"DELETE FROM %s WHERE %s = %d \n",
tabname, colname, ids[i]);
/*printf(" stmt2 = %s \n", buff2);*/
EXEC SQL begin work without replication;
EXEC SQL EXECUTE IMMEDIATE :buff2;
EXEC SQL commit work;

/* keep a log in the file*/
fputs(buff2, fp_all);
}

/* record summary deletion info*/
sprintf(buff,"===> %s rows deleted: %d \n \n", tabname, thecount);
fputs(buff, fp_sum);
}


int main(int argc, char *argv[]) {

static int mycount=0;
char* tname[TABLIMIT];
char* cname[TABLIMIT];
char* cond[TABLIMIT];

FILE *fp_sum, *fp_all;
fp_sum= fopen("db_cleanup_rep.sum","w+");
fp_all= fopen("db_cleanup_rep.all","w+");

time_t current_time; time(&current_time);
fputs(ctime(&current_time), fp_sum);
fputs(ctime(&current_time), fp_all);

EXEC SQL BEGIN DECLARE SECTION;
int id;
char tabname[32];
char colname[32];
char delcond[255];
EXEC SQL END DECLARE SECTION;

/* allocate memory for arraies */
for (int i=0; i<TABLIMIT ; i++){
tname[i]=malloc(32);
cname[i]=malloc(32);
cond[i]=malloc(255);
}

/*select table clean info into arraies*/
EXEC SQL DATABASE 'noaa';
EXEC SQL PREPARE sel FROM
"SELECT * FROM db_cleanup_rep where status='active'";
EXEC SQL DECLARE C1 CURSOR FOR sel;
EXEC SQL OPEN C1;

for ( ;; ) {

EXEC SQL FETCH C1 into :id, :tabname, :colname, :delcond;
if ( sqlca.sqlcode != 0 ){ break; }
sprintf(tname[mycount],"%s",tabname);
sprintf(cname[mycount],"%s",colname);
sprintf(cond[mycount],"%s",delcond);

mycount++;
}
EXEC SQL CLOSE C1;

/*printf(" count= %d \n", mycount);*/
/*handle it table by table*/
for ( int i=0;i<mycount; i++) {

/*printf("mycount= %d, i=%d, %s, %s\n \n", mycount, i,
cname[i], cond[i]);*/
deleteRows(tname[i],cname[i],cond[i], fp_sum, fp_all);
}

fclose(fp_sum); fclose(fp_all);


}






On 26 Sep 2006 12:04:25 -0700, bozon <curtis@crowson1.com> wrote:
> Quman,
> How does the script limit the number of records deleted? There isn't a
> command to do "delete first 50 from <table> " is there ?
> I did something similar to fix in place alters ( I of course updated a
> column that wasn't in an index or trigger, instead of deleting the
> record) but to get the records to work on I unloaded the primary key of
> the records (because you can't even "select first 50 <pk> from <table>
> into temp nibble_<table> with no log
>
> If you are going to do this during down time you can do the following:
>
> begin work;
>
> rename <table> to old_<table> ;
>
> create table <table> (...);
>
> alter table <table> type(raw);
> insert into <table> select * from old_<table> where <records_to_keep>;
> alter table <table> type(standard);
> drop table old_<table>;
>
> <create indexes>
> <create constraints>
> <recreate fk constraints from other tables>
>
> commit work;
>
> Since the inserts are being done to a raw table it won't be logged (of
> course it won't be replicated either).
> Quman wrote:
> > Normally, we have a script(perl,...) to do this,
> >
> > Inside the loop, we can control the number of rows we want to delete
> > in a short transaction. You can also have a couple of seconds sleep
> > for next round if you like.
> >
> > You can just stopt(CTRL-C) it any time and restart it later.
> >
> > thanks,
> > Quman
> >
> > On 9/26/06, Tam OShanter <tam@oshanter.com> wrote:
> > > Hello Again Friends,
> > > Thanks for the help you've provided me so far.
> > >
> > > So...
> > > Here are some stats on a table I have:
> > >
> > > total record count 20,454,537.00
> > > total record count to be moved to DW and deleted 11,000,921.00
> > >
> > > row size of 413 bytes.
> > >
> > > How do I go about deleting this data without rendering my server unusable?
> > >
> > > I saw a suggestion of deleting records in 500 record batches, but that is a
> > > helluva lot of batches to get done what I want.
> > >
> > > Any other ideas?
> > >
> > > Thanks in advance for the advice.
> > >
> > > Tam.
> > >
> > >
> > > _______________________________________________
> > > Informix-list mailing list
> > > Informix-list@iiug.org
> > > http://www.iiug.org/mailman/listinfo/informix-list
> > >

>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-20-2008, 01:08 PM
Superboer
 
Posts: n/a
Default Re: Massive Delete

may be you can design a good fragmentation strategy on this table
so you can detach the fragment with rows you do not need, this will not
cause
long trx rollbacks etc. and is also really fast!!!!!!
check it out.

Superboer.



Tam OShanter schreef:

> Hello Again Friends,
> Thanks for the help you've provided me so far.
>
> So...
> Here are some stats on a table I have:
>
> total record count 20,454,537.00
> total record count to be moved to DW and deleted 11,000,921.00
>
> row size of 413 bytes.
>
> How do I go about deleting this data without rendering my server unusable?
>
> I saw a suggestion of deleting records in 500 record batches, but that is a
> helluva lot of batches to get done what I want.
>
> Any other ideas?
>
> Thanks in advance for the advice.
>
> Tam.


Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-20-2008, 01:08 PM
Jean Sagi
 
Posts: n/a
Default Re: Massive Delete

....detach the fragment...

That sound interesting.


J.

Superboer escribió:
> may be you can design a good fragmentation strategy on this table
> so you can detach the fragment with rows you do not need, this will not
> cause
> long trx rollbacks etc. and is also really fast!!!!!!
> check it out.
>
> Superboer.
>
>
>
> Tam OShanter schreef:
>
>
>>Hello Again Friends,
>>Thanks for the help you've provided me so far.
>>
>>So...
>>Here are some stats on a table I have:
>>
>> total record count 20,454,537.00
>> total record count to be moved to DW and deleted 11,000,921.00
>>
>>row size of 413 bytes.
>>
>>How do I go about deleting this data without rendering my server unusable?
>>
>>I saw a suggestion of deleting records in 500 record batches, but that is a
>>helluva lot of batches to get done what I want.
>>
>>Any other ideas?
>>
>>Thanks in advance for the advice.
>>
>>Tam.

>
>
> _______________________________________________
> Informix-list mailing list
> Informix-list@iiug.org
> http://www.iiug.org/mailman/listinfo/informix-list
>

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-20-2008, 01:09 PM
Tam OShanter
 
Posts: n/a
Default Re: Massive Delete

Yes. It does sound quite interesting.
Can you provide more information, or at least a hint on where to start
looking for practical examples?

Thanks Friends,

Tam.
"Jean Sagi" <jeansagi@myrealbox.com> wrote in message
news:mailman.646.1159358485.20706.informix-list@iiug.org...
> ...detach the fragment...
>
> That sound interesting.
>
>
> J.
>
> Superboer escribió:
>> may be you can design a good fragmentation strategy on this table
>> so you can detach the fragment with rows you do not need, this will not
>> cause
>> long trx rollbacks etc. and is also really fast!!!!!!
>> check it out.
>>
>> Superboer.
>>
>>
>>
>> Tam OShanter schreef:
>>
>>
>>>Hello Again Friends,
>>>Thanks for the help you've provided me so far.
>>>
>>>So...
>>>Here are some stats on a table I have:
>>>
>>> total record count 20,454,537.00
>>> total record count to be moved to DW and deleted 11,000,921.00
>>>
>>>row size of 413 bytes.
>>>
>>>How do I go about deleting this data without rendering my server
>>>unusable?
>>>
>>>I saw a suggestion of deleting records in 500 record batches, but that is
>>>a
>>>helluva lot of batches to get done what I want.
>>>
>>>Any other ideas?
>>>
>>>Thanks in advance for the advice.
>>>
>>>Tam.

>>
>>
>> _______________________________________________
>> Informix-list mailing list
>> Informix-list@iiug.org
>> http://www.iiug.org/mailman/listinfo/informix-list
>>



Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-20-2008, 01:09 PM
bozon
 
Posts: n/a
Default Re: Massive Delete

That is similar to my job that fixes in place alters. I would think
that the select all records into the temp table might be a performance
hit. Don't get me wrong I don't like my method of unloading the
records, which is something like "unload to <file> select first <x>
<pk_id_col> from <table> where <pk_id_col> > <last_id_selected> order
by <pk_id_col>" because you can't do the more natural "select first <x>
<pk_id_col> from <table> where <pk_id_col> > <last_id_selected> order
by <pk_id_col>" into temp <temp table name> with no log " because it
gives you a syntax error, method either.

adding first and skip etc. to the temp table select statement would be
a nice feature. I of course don't think it would be that hard to
implement. I certainly wouldn't have to do anything. ;-)

Quman wrote:
> I attached an esql.ec file. That can do our job.
> If your table does not have unique ID column, you need modfy it.
>
> I prefer using PERL to do it now. this is our early version.
>
> Thanks,
> Quman
>
>
> /*
> * File: CleanupRepTab.ec
> *
> * CleanupRepTab.ec is an ESQL/C program to cleanup database tables that
> * are replicated by Informix Enterprise Replication server between multiple
> * sites. For each table to be cleaned, there is a cleanup SQL WHERE clause
> * stored in the db_cleanup_rep table. CleanupRepTab.ec will lookup the
> * db_cleanup_rep table and delete the rows which satisfy the condition.
> * db_cleanup_rep has the following schema,
> * create table db_cleanup_rep (
> * id serial not null ,
> * tabname char(32) not null ,
> * colname char(32) not null ,
> * delcond varchar(255),
> * status char(8),
> * primary key (id) constraint db_cleanup_rep_pk );
> *
> * id: a serial number generated for each row.
> * tabname: the name of the table to be cleaned, e.g., arch_recall_reqs,
> * file_change_notice, etc.
> * colname: a column name of the table, its type must be Integer and Unique.
> * delcond: the condition used to delete rows of the table.
> * status: "active" means the clean activity is on, "inactive" means off.
>
> * The final results of cleanup are stored in two files: db_cleanup_sum.msg and
> * db_cleanup_all.msg. db_cleanup_sum.msg contains the summary about
> the deletion
> * for each table. db_cleanup_all.msg contains all the ids of cleaned rows for
> * each table.
> */
>
>
> #include <stdio.h>
> #include <time.h>
> #include <sys/types.h>
> #include <stdlib.h>
> #include <sqlhdr.h>
> #include <sqlca.h>
>
> #define TABLIMIT 100 /*number of tables can be handled */
> #define ROWLIMIT 320000 /*max number of rows in a table handled at one
> time run*/
>
> void deleteRows(char* tabname, char* colname,
> char* delcond, FILE *fp_sum, FILE *fp_all){
>
> int ids[ROWLIMIT];
> int thecount=0;
>
> EXEC SQL BEGIN DECLARE SECTION;
> int theid;
> char buff[255]; char buff2[255];
> EXEC SQL END DECLARE SECTION;
>
> /* select the row ids into a array*/
> sprintf(buff, " SELECT %s FROM %s WHERE %s INTO TEMP toDelRow
> WITH NO LOG \n",
> colname, tabname, delcond);
> fputs(buff, fp_sum);
>
> /*printf("stmt1 : %s\n \n", buff);*/
> EXEC SQL EXECUTE IMMEDIATE :buff;
> EXEC SQL PREPARE sele FROM 'select * from toDelRow';
> EXEC SQL DECLARE C2 CURSOR FOR sele;
> EXEC SQL OPEN C2;
>
> for (; ; ) {
>
> EXEC SQL FETCH C2 into :theid;
> if ( sqlca.sqlcode != 0 || thecount > ROWLIMIT) { break; }
> ids[thecount]=theid;
> /*printf(" i= %d id= %d \n", thecount,theid);*/
> thecount++;
> }
>
> EXEC SQL CLOSE C2;
> EXEC SQL DROP TABLE toDelRow;
>
> /* delete from target table row by row */
> for ( int i=0;i<thecount; i++) {
>
> sprintf(buff2,"DELETE FROM %s WHERE %s = %d \n",
> tabname, colname, ids[i]);
> /*printf(" stmt2 = %s \n", buff2);*/
> EXEC SQL begin work without replication;
> EXEC SQL EXECUTE IMMEDIATE :buff2;
> EXEC SQL commit work;
>
> /* keep a log in the file*/
> fputs(buff2, fp_all);
> }
>
> /* record summary deletion info*/
> sprintf(buff,"===> %s rows deleted: %d \n \n", tabname, thecount);
> fputs(buff, fp_sum);
> }
>
>
> int main(int argc, char *argv[]) {
>
> static int mycount=0;
> char* tname[TABLIMIT];
> char* cname[TABLIMIT];
> char* cond[TABLIMIT];
>
> FILE *fp_sum, *fp_all;
> fp_sum= fopen("db_cleanup_rep.sum","w+");
> fp_all= fopen("db_cleanup_rep.all","w+");
>
> time_t current_time; time(&current_time);
> fputs(ctime(&current_time), fp_sum);
> fputs(ctime(&current_time), fp_all);
>
> EXEC SQL BEGIN DECLARE SECTION;
> int id;
> char tabname[32];
> char colname[32];
> char delcond[255];
> EXEC SQL END DECLARE SECTION;
>
> /* allocate memory for arraies */
> for (int i=0; i<TABLIMIT ; i++){
> tname[i]=malloc(32);
> cname[i]=malloc(32);
> cond[i]=malloc(255);
> }
>
> /*select table clean info into arraies*/
> EXEC SQL DATABASE 'noaa';
> EXEC SQL PREPARE sel FROM
> "SELECT * FROM db_cleanup_rep where status='active'";
> EXEC SQL DECLARE C1 CURSOR FOR sel;
> EXEC SQL OPEN C1;
>
> for ( ;; ) {
>
> EXEC SQL FETCH C1 into :id, :tabname, :colname, :delcond;
> if ( sqlca.sqlcode != 0 ){ break; }
> sprintf(tname[mycount],"%s",tabname);
> sprintf(cname[mycount],"%s",colname);
> sprintf(cond[mycount],"%s",delcond);
>
> mycount++;
> }
> EXEC SQL CLOSE C1;
>
> /*printf(" count= %d \n", mycount);*/
> /*handle it table by table*/
> for ( int i=0;i<mycount; i++) {
>
> /*printf("mycount= %d, i=%d, %s, %s\n \n", mycount, i,
> cname[i], cond[i]);*/
> deleteRows(tname[i],cname[i],cond[i], fp_sum, fp_all);
> }
>
> fclose(fp_sum); fclose(fp_all);
>
>
> }
>
>
>
>
>
>
> On 26 Sep 2006 12:04:25 -0700, bozon <curtis@crowson1.com> wrote:
> > Quman,
> > How does the script limit the number of records deleted? There isn't a
> > command to do "delete first 50 from <table> " is there ?
> > I did something similar to fix in place alters ( I of course updated a
> > column that wasn't in an index or trigger, instead of deleting the
> > record) but to get the records to work on I unloaded the primary key of
> > the records (because you can't even "select first 50 <pk> from <table>
> > into temp nibble_<table> with no log
> >
> > If you are going to do this during down time you can do the following:
> >
> > begin work;
> >
> > rename <table> to old_<table> ;
> >
> > create table <table> (...);
> >
> > alter table <table> type(raw);
> > insert into <table> select * from old_<table> where <records_to_keep>;
> > alter table <table> type(standard);
> > drop table old_<table>;
> >
> > <create indexes>
> > <create constraints>
> > <recreate fk constraints from other tables>
> >
> > commit work;
> >
> > Since the inserts are being done to a raw table it won't be logged (of
> > course it won't be replicated either).
> > Quman wrote:
> > > Normally, we have a script(perl,...) to do this,
> > >
> > > Inside the loop, we can control the number of rows we want to delete
> > > in a short transaction. You can also have a couple of seconds sleep
> > > for next round if you like.
> > >
> > > You can just stopt(CTRL-C) it any time and restart it later.
> > >
> > > thanks,
> > > Quman
> > >
> > > On 9/26/06, Tam OShanter <tam@oshanter.com> wrote:
> > > > Hello Again Friends,
> > > > Thanks for the help you've provided me so far.
> > > >
> > > > So...
> > > > Here are some stats on a table I have:
> > > >
> > > > total record count 20,454,537.00
> > > > total record count to be moved to DW and deleted 11,000,921.00
> > > >
> > > > row size of 413 bytes.
> > > >
> > > > How do I go about deleting this data without rendering my server unusable?
> > > >
> > > > I saw a suggestion of deleting records in 500 record batches, but that is a
> > > > helluva lot of batches to get done what I want.
> > > >
> > > > Any other ideas?
> > > >
> > > > Thanks in advance for the advice.
> > > >
> > > > Tam.
> > > >
> > > >
> > > > _______________________________________________
> > > > Informix-list mailing list
> > > > Informix-list@iiug.org
> > > > http://www.iiug.org/mailman/listinfo/informix-list
> > > >

> >
> > _______________________________________________
> > Informix-list mailing list
> > Informix-list@iiug.org
> > http://www.iiug.org/mailman/listinfo/informix-list
> >


Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-20-2008, 01:09 PM
normajeansebastian via DBMonster.com
 
Posts: n/a
Default Re: Massive Delete

http://www-306.ibm.com/software/data...ry/ids_94.html
scroll down and select the "Performance Guide, Version 9.4"
in this manual, go to chapter 9, "Fragmentation Guidelines".

fragmenting by expression takes some prep work but once set up it really
simplifies massive deletes.
we use it to speed our partial database refreshes.

good luck,
Norma Jean



Tam OShanter wrote:
>Yes. It does sound quite interesting.
>Can you provide more information, or at least a hint on where to start
>looking for practical examples?
>
>Thanks Friends,
>
>Tam.
>> ...detach the fragment...
>>

>[quoted text clipped - 41 lines]
>>> Informix-list@iiug.org
>>> http://www.iiug.org/mailman/listinfo/informix-list


--
Message posted via DBMonster.com
http://www.dbmonster.com/Uwe/Forums....ormix/200609/1

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 09:31 AM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com