SEO

#1 (**permalink**) 04-19-2008, 10:46 AM

Hi all,

I need to do a bulk data loading around 704GB (log file size) at present  in 8 hrs (1 am - 9am). The data file size may increase 3 to 5 times in future.

Using COPY it takes 96 hrs to finish the task.
What is the best way to do it ?

HARDWARE: SUN THUMPER/ RAID10
OS : SOLARIS 10.
DB: Greenplum/Postgres

Regards,
Srikanth k Potluri
+63 9177444783(philippines)

#2 (**permalink**) 04-19-2008, 10:46 AM

Potluri Srikanth wrote:
> Hi all,
>
> I need to do a bulk data loading around 704GB (log file size) at
> present in 8 hrs (1 am - 9am). The data file size may increase 3 to
> 5 times in future.
>
> Using COPY it takes 96 hrs to finish the task.
> What is the best way to do it ?
>
> HARDWARE: SUN THUMPER/ RAID10
> OS : SOLARIS 10.
> DB: Greenplum/Postgres

If you're using Greenplum, you should probably be talking to the
Greenplum folks. IIRC, they have made some fairly large changes to the
load process, so they'll be the ones with the proper answers for you.

//Magnus

--
Sent via pgsql-performance mailing list (pgsql-performance@postgresql.org)
To make changes to your subscription:
http://www.postgresql.org/mailpref/pgsql-performance

Thread Tools
Show Printable Version Email this Page
Display Modes
Linear Mode Switch to Hybrid Mode Switch to Threaded Mode