SEO

Hello All,


We are getting very strange problem ..


Scenario..
************************************************** ******
We Have one data field in some table..type of this field
Is *TEXT*.
We are storing around 1,000 (One Thousand) lines of text
In that field.

If it's run though separate thread,
When we say select <columnname> from <tablename>
We are getting
*could not receive data from server: No such file or directory *
messages without Result

If it was in main thread,
It Works fine.
************************************************** ******


Our Analysis::
************************************************** ******
It looks like stack space issue.when we allocate stack space
Of 1.3 GB to new thread then we will get the Desired
Outpt I.e All 1,000 lines without any problem.

1)That String is barely 80 k ..there is no direct
Relationship with that size.

2)We can't allocate 1.3 GB,and we also don't have any
Recusrsive functions,

We would like to know is there any Recursive functions
In postgres -PQExec which might be eating all stack space..
Or what cud be the reason form libpq's point of view.

************************************************** ******



Please give u'r thougts,

Thanks in advance for U'r Suggetions..
Prasanna.

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

On Fri, Apr 14, 2006 at 09:13:26AM +0530, Mavinakuli, Prasanna (STSD) wrote:
> Hello All,
>
> We are getting very strange problem ..
>
> ************************************************** ******
> We Have one data field in some table..type of this field
> Is *TEXT*.
> We are storing around 1,000 (One Thousand) lines of text
> In that field.
>
> If it's run though separate thread,
> When we say select <columnname> from <tablename>
> We are getting
> *could not receive data from server: No such file or directory *
> messages without Result
>
> If it was in main thread,
> It Works fine.
> ************************************************** ******

So you have a multithreaded program. Have you setup something so that
you don't have two threads trying to do things with libpq at the same
time? Because that won't work.

Secondly, it can't have anything to do with stack-space because when
you run out of stack space you get a segmentation fault, not a nice
error.

Either post a complete example of the failure (with source code) or
perhaps the strace output would be enough.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFEQPpwIB7bNG8LQkwRAi6VAKCIyCn78cg9W6r6HhZtMh HAifztBgCfZx8I
B64NcXae4uCgtvT08H3b/xI=
=jriJ
-----END PGP SIGNATURE-----

On Tue, Apr 18, 2006 at 12:31:57PM +0530, Mavinakuli, Prasanna (STSD) wrote:
> Hello Martijin,
> Thx for u'r Suggetions..
>
> Here is Complete source code..

Ok, I can't get it to fail. I cleaned up the code because it wouldn't
compile as is (g++ 3.3.5 on Debian). I've put the cleaned up version
here:

http://svana.org/kleptog/temp/test.cpp

You might need to change the QUERY_STRING and the CONNECT_STRING back.
Could you try running this to see if you can get it to fail? If it's
does fail, could you post the output of:

strace -f <program>

somewhere (*don't* post it to the list). Somewhere like:
http://phpfi.com/
should work well.

> Desc about following code::::
>
> Code has a function named as queryDB, which makes a connection to our DB
> and then tries to query
> Particular table which has huge string in it's field.
>
> If this method called from main thread then it *DOES* work properly.
>
> But if we make that as routine function for a thread it doesn't work
> instead we will get that following status :::
> ( PGRES_FATAL_ERROR )
> ***could not receive data from server: Invalid argument***

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is a
> tool for doing 5% of the work and then sitting around waiting for someone
> else to do the other 95% so you can sue them.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFERK42IB7bNG8LQkwRAlQ7AJ0TlOK1pPdv2/lrRNMzOzvHzGF6DgCcD3Jy
VjtXW/jWt+Ezwh84a4CXxg0=
=XoHw
-----END PGP SIGNATURE-----

Hello Martijin,
Thx for u'r Suggetions..

Here is Complete source code..

Desc about following code::::

Code has a function named as queryDB, which makes a connection to our DB
and then tries to query
Particular table which has huge string in it's field.

If this method called from main thread then it *DOES* work properly.

But if we make that as routine function for a thread it doesn't work
instead we will get that following status :::
( PGRES_FATAL_ERROR )
***could not receive data from server: Invalid argument***


******************************CODE**************** *************

#include <iostream>
#include "libpq-fe.h"
#include "libpq/libpq-fs.h"
#include <string>
#include <stdlib.h>
#include <netinet/in.h>
#include <arpa/inet.h>
#include <sys/timeb.h>
#include <time.h>
#include <unistd.h>
#include <pwd.h>
#include <sys/timeb.h>
#include <errno.h>
#include <sys/socket.h>
#include <netdb.h>
#include <netinet/in.h>
#include "common/gs/utils/UtilsConsts.h"
#include <utmp.h>
#include <utmpx.h>
#include <sys/stat.h>
#include <cstdlib>
#include <cstdio>

void* queryDB(void* p)
{

cout<<"I Am in Routine Function *** "<<endl;
PGconn *_conn;
PGresult *_res;
char *dbn,*usn;
char *error;
cout<<"i am connecting:"<<endl;
_conn=PQconnectdb("dbname = LOGDB user = sfmdb port=10864 ");
if(PQstatus(_conn)==CONNECTION_OK)
cout<<"connection is ready"<<endl;
dbn=PQdb(_conn);
usn=PQuser(_conn);
if(PQstatus(_conn)==CONNECTION_BAD)
cout<<"the connection is not hapened"<<endl;
string string1,string2,string3;
_res=PQexec(_conn,"select * from throttlingconfig ;");
if (!( PQresultStatus(_res) == PGRES_COMMAND_OK
||PQresultStatus(_res) == PGRES_TUPLES_OK ) )
{
/*If this Function is *NOT* called from main thread then
we will end up by entering here. */
string msg = PQresultErrorMessage(_res);
PQclear(_res);
cout<<"Query Execution is Not OK **********"<<msg<<endl;
}
else
{
/*If this Function is called from main thread then we will
end up by entering here. */
int no_of_tuples,no_of_columns;
char *str1;
string stg;
no_of_columns=PQnfields(_res);
no_of_tuples=PQntuples(_res);

cout<<"no of tuples "<<no_of_tuples<<endl;
cout<<"Number Of Columns "<<no_of_columns<<endl;

for(int u=0;u<no_of_tuples;u++)
{
for(int y=0;y<no_of_columns;y++)
{
str1=PQgetvalue(_res,u,y);
cout<<"*************Printing *** "<<u<<"th Row
"<<y<<"th Column****************"<<endl;
cout<<str1<<endl;
}
cout<<endl;
}
PQclear(_res);
}
PQfinish(_conn);
return NULL;
}
int main()
{
//size_t _stackSize(512 * 1024 * 2);

//calling queryDB from other thread ..DOESN'T Work here
size_t _stackSize(512 * 1024 * 2 * 20) ;
pthread_attr_t attr;
pthread_attr_init ( &attr ) ;
pthread_t thread_t;
int ret = pthread_attr_setstacksize( &attr, _stackSize) ;
pthread_attr_setdetachstate(&attr, PTHREAD_CREATE_JOINABLE);
int retVal = pthread_create(&thread_t, &attr,queryDB,NULL );
int rc,status;
rc = pthread_join(thread_t, (void **)&status);

//calling queryDB from main thread WORKS here..

queryDB(NULL);
return 1;
}


Thx a lot,
And looking for u'r reply,

Prasanna.




-----Original Message-----
From: Martijn van Oosterhout [mailto:kleptog@svana.org]
Sent: Saturday, April 15, 2006 7:22 PM
To: Mavinakuli, Prasanna (STSD)
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] HUGE Stack space is gettiing consumed

On Fri, Apr 14, 2006 at 09:13:26AM +0530, Mavinakuli, Prasanna (STSD)
wrote:
> Hello All,
>
> We are getting very strange problem ..
>
> ************************************************** ******
> We Have one data field in some table..type of this field Is *TEXT*.
> We are storing around 1,000 (One Thousand) lines of text In that
> field.
>
> If it's run though separate thread,
> When we say select <columnname> from <tablename> We are getting *could

> not receive data from server: No such file or directory * messages
> without Result
>
> If it was in main thread,
> It Works fine.
> ************************************************** ******

So you have a multithreaded program. Have you setup something so that
you don't have two threads trying to do things with libpq at the same
time? Because that won't work.

Secondly, it can't have anything to do with stack-space because when you
run out of stack space you get a segmentation fault, not a nice error.

Either post a complete example of the failure (with source code) or
perhaps the strace output would be enough.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is
> a tool for doing 5% of the work and then sitting around waiting for
> someone else to do the other 95% so you can sue them.

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

On Thu, Apr 20, 2006 at 08:20:30PM +0530, Mavinakuli, Prasanna (STSD) wrote:
>
> Hello Martijin,
>
> I am not able to make it ..Code what u sent is *not* giving desired
> result..
> That is -when we try to fetch huge data in a thread routine it says
> "could not receive data from server: Error 0"
> If it was in main thread then it goes thorugh..I.e It's able to get the
> huge data.

Well, I don't understand. There's no interesting difference between the
code I sent and the code you sent...

> Will send the tusc output (stack trace)

Not stack trace, strace which is a system call trace. Don't forget -f
to trace the threads also.
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> From each according to his ability. To each according to his ability to litigate.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.1 (GNU/Linux)

iD8DBQFER6jsIB7bNG8LQkwRAnKLAJ4lvJ3agW8tPU3wOB1U2F 8e86YNKgCdEE4r
J6jYBQ9TXl06EINWKJwNlS0=
=l+eR
-----END PGP SIGNATURE-----

Hello Martijin,

I am not able to make it ..Code what u sent is *not* giving desired
result..
That is -when we try to fetch huge data in a thread routine it says
"could not receive data from server: Error 0"
If it was in main thread then it goes thorugh..I.e It's able to get the
huge data.


Environment..

Postgres Version:7.4.3
Compiler:aCC 5.0
OS:HP-UX PARISC 11.23

Code has put in the following link..
http://phpfi.com/113850

Will send the tusc output (stack trace)


Thx,
Prasanna.

-----Original Message-----
From: Martijn van Oosterhout [mailto:kleptog@svana.org]
Sent: Tuesday, April 18, 2006 2:46 PM
To: Mavinakuli, Prasanna (STSD)
Cc: pgsql-general@postgresql.org
Subject: Re: [GENERAL] HUGE Stack space is gettiing consumed

On Tue, Apr 18, 2006 at 12:31:57PM +0530, Mavinakuli, Prasanna (STSD)
wrote:
> Hello Martijin,
> Thx for u'r Suggetions..
>
> Here is Complete source code..

Ok, I can't get it to fail. I cleaned up the code because it wouldn't
compile as is (g++ 3.3.5 on Debian). I've put the cleaned up version
here:

http://svana.org/kleptog/temp/test.cpp

You might need to change the QUERY_STRING and the CONNECT_STRING back.
Could you try running this to see if you can get it to fail? If it's
does fail, could you post the output of:

strace -f <program>

somewhere (*don't* post it to the list). Somewhere like:
http://phpfi.com/
should work well.

> Desc about following code::::
>
> Code has a function named as queryDB, which makes a connection to our
> DB and then tries to query Particular table which has huge string in
> it's field.
>
> If this method called from main thread then it *DOES* work properly.
>
> But if we make that as routine function for a thread it doesn't work
> instead we will get that following status :::
> ( PGRES_FATAL_ERROR )
> ***could not receive data from server: Invalid argument***

Have a nice day,
--
Martijn van Oosterhout <kleptog@svana.org> http://svana.org/kleptog/
> Patent. n. Genius is 5% inspiration and 95% perspiration. A patent is
> a tool for doing 5% of the work and then sitting around waiting for
> someone else to do the other 95% so you can sue them.

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq