Unix Technical Forum

Dump database more than 1 flat file

This is a discussion on Dump database more than 1 flat file within the pgsql Admins forums, part of the PostgreSQL category; --> Hi people, How can I export (dump) database more than 1 file? I have great table and can't split. ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Admins

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-10-2008, 08:17 AM
A.Burbello
 
Posts: n/a
Default Dump database more than 1 flat file

Hi people,

How can I export (dump) database more than 1 file?
I have great table and can't split.
Like Oracle, just set the parameter filesize and the
files name, and dump in several files.

Could I do it the same way in PG?

Thank you in advance.


Abra sua conta no Yahoo! Mail, o único sem limite de espaço para armazenamento!
http://br.mail.yahoo.com/

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-10-2008, 08:17 AM
Scott Marlowe
 
Posts: n/a
Default Re: Dump database more than 1 flat file

On Dec 15, 2007 11:42 AM, A.Burbello <burbello3000@yahoo.com.br> wrote:
> Hi people,
>
> How can I export (dump) database more than 1 file?
> I have great table and can't split.
> Like Oracle, just set the parameter filesize and the
> files name, and dump in several files.


You could pipe the output of pg_dump to split, or split it after it's
been created.

But is there a reason for doing this or is it just some practice you're into?

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-10-2008, 08:18 AM
A.Burbello
 
Posts: n/a
Default Re: Dump database more than 1 flat file

I consider this way a good practice to transport the
files. This is because I have table that has more than
30GB.
But if the OS was windows, couldn't split the files
because postgres doesn't has this feature!
Could be a good option if postgres had native.

Weel, I will try this:
eg: $ pg_dump postgres -U postgres -f split.txt |
split --bytes=10m

Thank you.






--- Scott Marlowe <scott.marlowe@gmail.com> escreveu:

> On Dec 15, 2007 11:42 AM, A.Burbello
> <burbello3000@yahoo.com.br> wrote:
> > Hi people,
> >
> > How can I export (dump) database more than 1 file?
> > I have great table and can't split.
> > Like Oracle, just set the parameter filesize and

> the
> > files name, and dump in several files.

>
> You could pipe the output of pg_dump to split, or
> split it after it's
> been created.
>
> But is there a reason for doing this or is it just
> some practice you're into?
>




Abra sua conta no Yahoo! Mail, o único sem limite de espaço para armazenamento!
http://br.mail.yahoo.com/

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-10-2008, 08:18 AM
Scott Marlowe
 
Posts: n/a
Default Re: Dump database more than 1 flat file

On Dec 17, 2007 2:06 PM, A.Burbello <burbello3000@yahoo.com.br> wrote:
> I consider this way a good practice to transport the
> files. This is because I have table that has more than
> 30GB.
> But if the OS was windows, couldn't split the files
> because postgres doesn't has this feature!
> Could be a good option if postgres had native.
>
> Weel, I will try this:
> eg: $ pg_dump postgres -U postgres -f split.txt |
> split --bytes=10m


http://gnuwin32.sourceforge.net/packages/coreutils.htm

for now. I used these back in the day (NT4.0 SP4 or so) and they
worked a charm back then. Heck, even ln worked ( in a manner of
speaking ) back then.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-10-2008, 08:18 AM
Geoffrey
 
Posts: n/a
Default Re: Dump database more than 1 flat file

Scott Marlowe wrote:
> On Dec 17, 2007 2:06 PM, A.Burbello <burbello3000@yahoo.com.br> wrote:
>> I consider this way a good practice to transport the
>> files. This is because I have table that has more than
>> 30GB.
>> But if the OS was windows, couldn't split the files
>> because postgres doesn't has this feature!
>> Could be a good option if postgres had native.
>>
>> Weel, I will try this:
>> eg: $ pg_dump postgres -U postgres -f split.txt |
>> split --bytes=10m

>
> http://gnuwin32.sourceforge.net/packages/coreutils.htm
>
> for now. I used these back in the day (NT4.0 SP4 or so) and they
> worked a charm back then. Heck, even ln worked ( in a manner of
> speaking ) back then.


There's also Uwin from AT&T:

http://www.research.att.com/sw/tools/uwin

list of available commands and man pages:

http://www.research.att.com/~gsf/man/man1/


--
Until later, Geoffrey

Those who would give up essential Liberty, to purchase a little
temporary Safety, deserve neither Liberty nor Safety.
- Benjamin Franklin

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-10-2008, 08:18 AM
A.Burbello
 
Posts: n/a
Default Re: Dump database more than 1 flat file

Thank you for your response!

I tested the split command and I got another problem
because for each file doesn't have the "COPY" header.

As well I said early, one table has more than 30G and
I need to import to another server (linux), but by dvd
media because doens't have network connection.

Whats the best way to do this.
Any suggestion?

If doesn't have another way, how can I put the
"header" in the begin of file without open?
With "cat >>" command I put in the end.

Could you help me?

Once again, thank you!








--- Scott Marlowe <scott.marlowe@gmail.com> escreveu:

> On Dec 17, 2007 2:06 PM, A.Burbello
> <burbello3000@yahoo.com.br> wrote:
> > I consider this way a good practice to transport

> the
> > files. This is because I have table that has more

> than
> > 30GB.
> > But if the OS was windows, couldn't split the

> files
> > because postgres doesn't has this feature!
> > Could be a good option if postgres had native.
> >
> > Weel, I will try this:
> > eg: $ pg_dump postgres -U postgres -f split.txt |
> > split --bytes=10m

>
>

http://gnuwin32.sourceforge.net/packages/coreutils.htm
>
> for now. I used these back in the day (NT4.0 SP4 or
> so) and they
> worked a charm back then. Heck, even ln worked ( in
> a manner of
> speaking ) back then.
>
> ---------------------------(end of
> broadcast)---------------------------
> TIP 9: In versions below 8.0, the planner will
> ignore your desire to
> choose an index scan if your joining column's
> datatypes do not
> match
>




Abra sua conta no Yahoo! Mail, o único sem limite de espaço para armazenamento!
http://br.mail.yahoo.com/

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-10-2008, 08:18 AM
Phillip Smith
 
Posts: n/a
Default Re: Dump database more than 1 flat file

> If doesn't have another way, how can I put the
> "header" in the begin of file without open?
> With "cat >>" command I put in the end.


After the split files are loaded on the new server file system:
cat splitfile*.txt > bigfile.txt
psql < bigfile.txt


THINK BEFORE YOU PRINT - Save paper if you don't really need to print this

*******************Confidentiality and Privilege Notice*******************

The material contained in this message is privileged and confidential to
the addressee. If you are not the addressee indicated in this message or
responsible for delivery of the message to such person, you may not copy
or deliver this message to anyone, and you should destroy it and kindly
notify the sender by reply email.

Information in this message that does not relate to the official business
of Weatherbeeta must be treated as neither given nor endorsed by Weatherbeeta.
Weatherbeeta, its employees, contractors or associates shall not be liable
for direct, indirect or consequential loss arising from transmission of this
message or any attachments
e-mail.

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-10-2008, 08:18 AM
Alvaro Herrera
 
Posts: n/a
Default Re: Dump database more than 1 flat file

A.Burbello wrote:

> If doesn't have another way, how can I put the
> "header" in the begin of file without open?
> With "cat >>" command I put in the end.


Don't -- you can put the header in a separate file and do something like

(cat header-file ; cat split-1 ; cat tail-file ) | psql

lather, rinse, repeat. (Each split-N file would be an output file from
split). I'm not sure if you also need the tail-file containing the \.
The parens tell the shell to execute all those commands and send the
output of all of them into the single psql process.

Of course you can automate it further, once you've copied all the DVDs
into a directory in the destination server:

for file in split-*; do
( cat header-file ; cat $file ; cat tail-file ) | psql
done

--
Alvaro Herrera http://www.CommandPrompt.com/
The PostgreSQL Company - Command Prompt, Inc.

---------------------------(end of broadcast)---------------------------
TIP 7: You can help support the PostgreSQL project by donating at

http://www.postgresql.org/about/donate

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-10-2008, 08:18 AM
Thomas Pundt
 
Posts: n/a
Default Re: Dump database more than 1 flat file

Hi,

On Dienstag, 18. Dezember 2007, Alvaro Herrera wrote:
| A.Burbello wrote:
| > If doesn't have another way, how can I put the
| > "header" in the begin of file without open?
| > With "cat >>" command I put in the end.
|
| Don't -- you can put the header in a separate file and do something like
|
| (cat header-file ; cat split-1 ; cat tail-file ) | psql

even simpler: "cat" usually takes any number of file name arguments,
so usually you simply can do

cat header-file split-1 tail-file | psql

No need for parentheses and starting a new process for each file.

Ciao,
Thomas

--
Thomas Pundt <thomas.pundt@rp-online.de> ---- http://rp-online.de/ ----

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-10-2008, 08:18 AM
Chris Browne
 
Posts: n/a
Default Re: Dump database more than 1 flat file

phillip.smith@weatherbeeta.com.au ("Phillip Smith") writes:
>> If doesn't have another way, how can I put the
>> "header" in the begin of file without open?
>> With "cat >>" command I put in the end.

>
> After the split files are loaded on the new server file system:
> cat splitfile*.txt > bigfile.txt
> psql < bigfile.txt
>
>
> THINK BEFORE YOU PRINT - Save paper if you don't really need to print this
>
> *******************Confidentiality and Privilege Notice*******************
>
> The material contained in this message is privileged and confidential to
> the addressee. If you are not the addressee indicated in this message or
> responsible for delivery of the message to such person, you may not copy
> or deliver this message to anyone, and you should destroy it and kindly
> notify the sender by reply email.
>
> Information in this message that does not relate to the official business
> of Weatherbeeta must be treated as neither given nor endorsed by Weatherbeeta.
> Weatherbeeta, its employees, contractors or associates shall not be liable
> for direct, indirect or consequential loss arising from transmission of this
> message or any attachments
> e-mail.


You should think before you attach these sorts of "confidentiality" notices.

If you put them on messages like this that are *OBVIOUSLY* being sent
to publicly-available mailing lists, this may undermine future claims
of material being supposedly-confidential.

In other words, by WRONGLY attaching confidentiality notices, courts
might decide to ignore them even in cases where you imagined they
ought to be legitimate...
--
let name="cbbrowne" and tld="linuxfinances.info" in String.concat "@" [name;tld];;
http://linuxfinances.info/info/x.html
The cigars in Los Angeles that were Duchamp-signed and then smoked.
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 02:36 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com