Hi,
I've constructed below after researching from the cdi archive.
Initially it worked for a small db and now it fails for larger.

dbexport

mknod /tmp/test p
gzip -9f </tmp/test >/tmp/test.exp &
dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss



dbimport

mknod /tmp/test p
cd /tmp
gunzip -c > /tmp/test < /tmp/test.exp &
dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql

After creating the db and loading couple tables it fails with below message.

*** load table ***
Read tape failed

*** Import data is corrupted!

0 - Unknown error message 0.

From the same gzip file If I were to

gunzip -c > /tmp/test < /tmp/test.exp

and

dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql

import completes successfully.

My goal is to create one export file and put it to tape while tackling
informix 2Gb file size limit on IDS 7.31 UC5 family.

Any input is appreciated.

Thanks,

Alkin T.

Informix DBA

alkin.tezuysal@gmail.com

Thanks for both the answers.

8 kb seems to be working for now. I'll be trying on a large db (about 35gb)
very soon

and see what happens.

Thanks,

Alkin T.



Might want to try using a block size of 8 (kbytes) for the (pipe) stream....
I traced ontape, and this is the size it required. Perhaps the others share
the same concept...








Martin Fuerderer

<MARTINFU@de.ibm. To: "Alkin Tezuysal" <alkin.tezuysal@gmail.com>

com> cc: informix-list@iiug.org, owner-informix-list@iiug.org

Sent by: Subject: Re: dbexport / dbimport using named pipe

owner-informix-li

st@iiug.org



09/13/2005 03:58

AM









Hi,

the problem is, that there is a difference between a tape device and a pipe.
dbimport is made for working with tape devices. This also works well for
files in a file system without too much difference and dbimport supports
files as well (albeit in your version only up to 2 GB, that's correct).

A pipe (named or not) has different characteristics than other devices.
Particularly, a pipe can have a state where there is no data in the pipe
because the writer to the pipe is slower than the reader.

With a pipe this is a transient but not a fatal problem. The reader only has
to wait until new data arrives from the writer. Once that has happened, the
reader can continue reading (and actually getting) data from the pipe.
Therefore the reader from a pipe has to repeatedly try to read from the pipe
when there is no data ... until new data arrives or a fatal error happens
(e.g. pipe has been destroyed).

Contrary, with a tape/file device a read that does not get any data means
that there is no more data. Any retry is pointless. If it happens before all
expected data has been read, then this is a fatal condition.

And this is how dbimport handles it.

dbimport does not have special provisions for handling a pipe, therefore you
are seeing this problem. Of course you can try to minimize the probability
of the problem by giving the gzip (the writer to the pipe) a head start,
e.g. put a sleep of several seconds inbetween the gzip and dbimport
commands. However, there will never be a guarantee that it will always work.
It may also depend on how big the pipe can become - which may be a
configurable parameter in the OS ... ?

On some platforms, the "tar" system utility provides parameter "-B"

exactly to solve this sort of problem. It will make tar to repeatedly read
from a pipe if it is empty, or to repeatedly try to write to the pipe if it
is currently full. To my knowledge no such thing has been implemented for
dbexport and dbimport.

Regards,

Martin

--

Martin Fuerderer

IBM Informix Development Munich, Germany Information Management

owner-informix-list@iiug.org wrote on 13.09.2005 01:29:50:

> Hi,

> I've constructed below after researching from the cdi archive.

> Initially it worked for a small db and now it fails for larger.

>

> dbexport

>

> mknod /tmp/test p

> gzip -9f </tmp/test >/tmp/test.exp &

> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss

>

>

> dbimport

>

> mknod /tmp/test p

> cd /tmp

> gunzip -c > /tmp/test < /tmp/test.exp & dbimport test -c -t /tmp/test

> -b 16 -s 12000000 -f /tmp/test.sql

>

> After creating the db and loading couple tables it fails with below

message.

>

> *** load table ***

> Read tape failed

>

> *** Import data is corrupted!

>

> 0 - Unknown error message 0.

>

> From the same gzip file If I were to

>

> gunzip -c > /tmp/test < /tmp/test.exp

>

> and

>

> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql

>

> import completes successfully.

>

> My goal is to create one export file and put it to tape while

> tackling informix 2Gb file size limit on IDS 7.31 UC5 family.

>

> Any input is appreciated.

>

> Thanks,

>

> Alkin T.

>

> Informix DBA

>

> alkin.tezuysal@gmail.com

sending to informix-list







This e-mail is intended only for the addressee named above.

As this e-mail may contain confidential or privileged information, if you
are not the named addressee, you are not authorized to retain, read, copy or
disseminate this message or any part of it.

"Alkin Tezuysal" <alkin.tezuysal@gmail.com> wrote in message
news:mdudnW3v_4jqkrveRVn-vw@comcast.com...
> Hi,
> I've constructed below after researching from the cdi archive.
> Initially it worked for a small db and now it fails for larger.
>
> dbexport
>
> mknod /tmp/test p
> gzip -9f </tmp/test >/tmp/test.exp &
> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss
>
>
>
> dbimport
>
> mknod /tmp/test p
> cd /tmp
> gunzip -c > /tmp/test < /tmp/test.exp &
> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql
>
> After creating the db and loading couple tables it fails with below
> message.
>
> *** load table ***
> Read tape failed
>
> *** Import data is corrupted!
>
> 0 - Unknown error message 0.
>
> From the same gzip file If I were to
>
> gunzip -c > /tmp/test < /tmp/test.exp
>
> and
>
> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql
>
> import completes successfully.
>
> My goal is to create one export file and put it to tape while tackling
> informix 2Gb file size limit on IDS 7.31 UC5 family.
>
> Any input is appreciated.
>
> Thanks,
>
> Alkin T.
>
> Informix DBA
>
> alkin.tezuysal@gmail.com
>
>

Hi All,

dbimport 2gb issue again;

As expected dbimport is failing once it reaches 2GB limit. Instead of asking
the next tape
it's failing with
*** load table ***
Read tape failed

*** Import data is corrupted!

0 - Unknown error message 0.

It seems my client doesn't have a support agreement with Informix. Looking
at the release notes I've found below defects
still remaining open even latest version of IDS 7.31 family.

Can somebody (Informix Tech. Support guys hello) verify whether this is a
bug for IDS 7.31.UC5 or not ?

PRODUCT: DBATOOLS
54311 DBIMPORT WITH BLOCKSIZE OF 32 AND TAPESIZE OF 2097151

CAUSES READ TAPE FAILED MESSAGE AT END OF FIRST TAPE

Date:2002/07/03

Version: 7.31.UD5

CUSTOMER-REPORTED BUGS STILL OPEN WITH THE RELEASE OF 7.31.UD5

146032
DBATOOLS - DBIMPORT
DBIMPORT FAILS WITH READ TAPE FAILED, IMPORT DATA CORRUPTED,UNKNOWN ERROR
MESSAGE 0, AND TAPE SIZE SET TO 2097151 AND BLKSIZE OF 16

132316
DBATOOLS - DBIMPORT
DBIMPORT FAILS WITH READ TAPE FAILED WHEN ATTEMPTING TO READ A TAPE WITH A
FIXED RECORD SIZE LESS

THAN 1024 BYTES



Any idea, input is appreciated.



Thanks in Advance.

"Alkin Tezuysal" <alkin.tezuysal@gmail.com> wrote in message
news:IY-dneYy2NPhfLXeRVn-3w@comcast.com...
> Thanks for both the answers.
>
> 8 kb seems to be working for now. I'll be trying on a large db (about
> 35gb) very soon
>
> and see what happens.
>
> Thanks,
>
> Alkin T.
>
>
>
> Might want to try using a block size of 8 (kbytes) for the (pipe)
> stream.... I traced ontape, and this is the size it required. Perhaps the
> others share the same concept...
>
>
>
>
>
>
>
>
> Martin Fuerderer
>
> <MARTINFU@de.ibm. To: "Alkin Tezuysal" <alkin.tezuysal@gmail.com>
>
> com> cc: informix-list@iiug.org, owner-informix-list@iiug.org
>
> Sent by: Subject: Re: dbexport / dbimport using named pipe
>
> owner-informix-li
>
> st@iiug.org
>
>
>
> 09/13/2005 03:58
>
> AM
>
>
>
>
>
>
>
>
>
> Hi,
>
> the problem is, that there is a difference between a tape device and a
> pipe. dbimport is made for working with tape devices. This also works well
> for files in a file system without too much difference and dbimport
> supports files as well (albeit in your version only up to 2 GB, that's
> correct).
>
> A pipe (named or not) has different characteristics than other devices.
> Particularly, a pipe can have a state where there is no data in the pipe
> because the writer to the pipe is slower than the reader.
>
> With a pipe this is a transient but not a fatal problem. The reader only
> has to wait until new data arrives from the writer. Once that has
> happened, the reader can continue reading (and actually getting) data from
> the pipe. Therefore the reader from a pipe has to repeatedly try to read
> from the pipe when there is no data ... until new data arrives or a fatal
> error happens (e.g. pipe has been destroyed).
>
> Contrary, with a tape/file device a read that does not get any data means
> that there is no more data. Any retry is pointless. If it happens before
> all expected data has been read, then this is a fatal condition.
>
> And this is how dbimport handles it.
>
> dbimport does not have special provisions for handling a pipe, therefore
> you are seeing this problem. Of course you can try to minimize the
> probability of the problem by giving the gzip (the writer to the pipe) a
> head start, e.g. put a sleep of several seconds inbetween the gzip and
> dbimport commands. However, there will never be a guarantee that it will
> always work. It may also depend on how big the pipe can become - which may
> be a configurable parameter in the OS ... ?
>
> On some platforms, the "tar" system utility provides parameter "-B"
>
> exactly to solve this sort of problem. It will make tar to repeatedly read
> from a pipe if it is empty, or to repeatedly try to write to the pipe if
> it is currently full. To my knowledge no such thing has been implemented
> for dbexport and dbimport.
>
> Regards,
>
> Martin
>
> --
>
> Martin Fuerderer
>
> IBM Informix Development Munich, Germany Information Management
>
> owner-informix-list@iiug.org wrote on 13.09.2005 01:29:50:
>
>> Hi,
>
>> I've constructed below after researching from the cdi archive.
>
>> Initially it worked for a small db and now it fails for larger.
>
>>
>
>> dbexport
>
>>
>
>> mknod /tmp/test p
>
>> gzip -9f </tmp/test >/tmp/test.exp &
>
>> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss
>
>>
>
>>
>
>> dbimport
>
>>
>
>> mknod /tmp/test p
>
>> cd /tmp
>
>> gunzip -c > /tmp/test < /tmp/test.exp & dbimport test -c -t /tmp/test
>
>> -b 16 -s 12000000 -f /tmp/test.sql
>
>>
>
>> After creating the db and loading couple tables it fails with below
>
> message.
>
>>
>
>> *** load table ***
>
>> Read tape failed
>
>>
>
>> *** Import data is corrupted!
>
>>
>
>> 0 - Unknown error message 0.
>
>>
>
>> From the same gzip file If I were to
>
>>
>
>> gunzip -c > /tmp/test < /tmp/test.exp
>
>>
>
>> and
>
>>
>
>> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql
>
>>
>
>> import completes successfully.
>
>>
>
>> My goal is to create one export file and put it to tape while
>
>> tackling informix 2Gb file size limit on IDS 7.31 UC5 family.
>
>>
>
>> Any input is appreciated.
>
>>
>
>> Thanks,
>
>>
>
>> Alkin T.
>
>>
>
>> Informix DBA
>
>>
>
>> alkin.tezuysal@gmail.com
>
> sending to informix-list
>
>
>
>
>
>
>
> This e-mail is intended only for the addressee named above.
>
> As this e-mail may contain confidential or privileged information, if you
> are not the named addressee, you are not authorized to retain, read, copy
> or disseminate this message or any part of it.
>
> "Alkin Tezuysal" <alkin.tezuysal@gmail.com> wrote in message
> news:mdudnW3v_4jqkrveRVn-vw@comcast.com...
>> Hi,
>> I've constructed below after researching from the cdi archive.
>> Initially it worked for a small db and now it fails for larger.
>>
>> dbexport
>>
>> mknod /tmp/test p
>> gzip -9f </tmp/test >/tmp/test.exp &
>> dbexport test -t /tmp/test -f /tmp/test.sql -b 16 -s 12000000 -ss
>>
>>
>>
>> dbimport
>>
>> mknod /tmp/test p
>> cd /tmp
>> gunzip -c > /tmp/test < /tmp/test.exp &
>> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql
>>
>> After creating the db and loading couple tables it fails with below
>> message.
>>
>> *** load table ***
>> Read tape failed
>>
>> *** Import data is corrupted!
>>
>> 0 - Unknown error message 0.
>>
>> From the same gzip file If I were to
>>
>> gunzip -c > /tmp/test < /tmp/test.exp
>>
>> and
>>
>> dbimport test -c -t /tmp/test -b 16 -s 12000000 -f /tmp/test.sql
>>
>> import completes successfully.
>>
>> My goal is to create one export file and put it to tape while tackling
>> informix 2Gb file size limit on IDS 7.31 UC5 family.
>>
>> Any input is appreciated.
>>
>> Thanks,
>>
>> Alkin T.
>>
>> Informix DBA
>>
>> alkin.tezuysal@gmail.com
>>
>>
>
>