Unix Technical Forum

4 pgcrypto regressions failures - 1 unsolved

This is a discussion on 4 pgcrypto regressions failures - 1 unsolved within the pgsql Hackers forums, part of the PostgreSQL category; --> http://www.pgbuildfarm.org/cgi-bin/s...-10%2022:30:03 New sha2 code on Solaris 2.8 / SPARC. Seems like it has problems memcpy'ing to a non-8-byte-aligned uint64 ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Hackers

Register FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-11-2008, 04:49 AM
Marko Kreen
 
Posts: n/a
Default 4 pgcrypto regressions failures - 1 unsolved


http://www.pgbuildfarm.org/cgi-bin/s...-10%2022:30:03

New sha2 code on Solaris 2.8 / SPARC. Seems like it has
problems memcpy'ing to a non-8-byte-aligned uint64 *.

Attached patch fixes it by simplifying the _Final code and
getting rid of the pointer.

(I redefined bzero and bcopy but now I think they should be
replaced directly - patch later.)



http://www.pgbuildfarm.org/cgi-bin/s...-11%2006:00:04
http://www.pgbuildfarm.org/cgi-bin/s...-10%2018:25:11

The new sha2.c checks not only whether BYTE_ENDIAN is
LITTLE_ENDIAN or BIG_ENDIAN but also whether it is set.
And the test fails on both Cygwin and MINGW.

As gcc evaluates "#if UNDEF1 == UNDEF2" as true and there
were no compile failures reported with current code, that
means currently internal AES, SHA1 and MD5 used randomly
big-endian, little-endian or both variants of code.

If there was no regression failures on those platforms,
it must be only by dumb luck.

Attached patch includes sys/param.h, where I found them on
MINGW, and puts stricter checks into all files.

After I see successful run in pgbuildfarm, I send it for
stable branches too.



http://www.pgbuildfarm.org/cgi-bin/s...-11%2002:30:00

NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
AES, but most of PGP tests use it as it is the preferred cipher.
And the AES tests fails anyway. I guess it can stay as expected
failure.



http://www.pgbuildfarm.org/cgi-bin/s...-11%2003:30:04

Linking problem with zlib on Solaris 9/x86. I am clueless about
this. I can anyone look into it?

Error message:
ld: fatal: relocations remain against allocatable but non-writable sections


--
marko



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-11-2008, 04:49 AM
Andrew Dunstan
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen said:
>


>
>

http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00>
> NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
> AES, but most of PGP tests use it as it is the preferred cipher.
> And the AES tests fails anyway. I guess it can stay as expected
> failure.
>


Please try to avoid expected failures if possible. If you must have them,
move them into a test file of their own. Consider the possibility of using
alternative .out files.

This doesn't matter to Buildfarm quite so much for contrib as it does for
main or PL regression sets, as contrib is the last thing checked, so a
failure there doesn't block any other steps in the checking process. Still,
a contrib failure will show up as yellow rather than green.

Buildfarm does not currently have a way of knowing what failure is expected
and what is not - it currently treats any regression failure as unexpected.
Changing that is on my TODO list, but it's not going to happen any time soon.

cheers

andrew



---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 04-11-2008, 04:49 AM
Marko Kreen
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 05:50:32AM -0500, Andrew Dunstan wrote:
> Marko Kreen said:
> http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00>
> > NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
> > AES, but most of PGP tests use it as it is the preferred cipher.
> > And the AES tests fails anyway. I guess it can stay as expected
> > failure.

>
> Please try to avoid expected failures if possible. If you must have them,
> move them into a test file of their own. Consider the possibility of using
> alternative .out files.


I need either to use included rijndael.c for AES with older
OpenSSL or rerun all tests to be Blowfish-only.

I want to standardise on AES so the former is preferred.

Now there's a choice:

1. Check OpenSSL version in main configure
2. #include "rijndael.c" in openssl.c

I guess 1. is nicer. I try to hack something together.

--
marko


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #4 (permalink)  
Old 04-11-2008, 04:49 AM
Marko Kreen
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 02:59:54PM +0300, Marko Kreen wrote:
> On Mon, Jul 11, 2005 at 05:50:32AM -0500, Andrew Dunstan wrote:
> > Marko Kreen said:
> > http://www.pgbuildfarm.org/cgi-bin/show_log.pl?nm=canary&dt=2005-07-11%2002:30:00>
> > > NetBSD 1.6 with older OpenSSL. OpenSSL < 0.9.7 does not have
> > > AES, but most of PGP tests use it as it is the preferred cipher.
> > > And the AES tests fails anyway. I guess it can stay as expected
> > > failure.

> >
> > Please try to avoid expected failures if possible. If you must have them,
> > move them into a test file of their own. Consider the possibility of using
> > alternative .out files.

>
> I need either to use included rijndael.c for AES with older
> OpenSSL or rerun all tests to be Blowfish-only.
>
> I want to standardise on AES so the former is preferred.
>
> Now there's a choice:
>
> 1. Check OpenSSL version in main configure
> 2. #include "rijndael.c" in openssl.c
>
> I guess 1. is nicer. I try to hack something together.


I tried 1. but that was messing with main build system for no
good reason. As the openssl.c would still be mess, so I went
with 2.

Result is - it's not so bad. As I used rijndael.c to provide
OpenSSL's own interface, I even got rid of all the ifdefs inside
the code.

--
marko



---------------------------(end of broadcast)---------------------------
TIP 5: don't forget to increase your free space map settings

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #5 (permalink)  
Old 04-11-2008, 04:49 AM
Tom Lane
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:
> Result is - it's not so bad. As I used rijndael.c to provide
> OpenSSL's own interface, I even got rid of all the ifdefs inside
> the code.


Looks good, but I'm still getting these compile warnings:

openssl.c: In function `ossl_des3_ecb_encrypt':
openssl.c:484: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type
openssl.c:484: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type
openssl.c: In function `ossl_des3_ecb_decrypt':
openssl.c:498: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type
openssl.c:498: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type

The following addition to the patch shuts up gcc with openssl 0.9.7a,
but I'm not sure if it will break anything with older openssl ---
comments?

regards, tom lane


*** /home/postgres/pgsql/contrib/pgcrypto/openssl.c Sun Jul 10 12:35:38 2005
--- new/openssl.c Mon Jul 11 10:06:30 2005
***************
*** 446,452 ****
ossldata *od = c->ptr;

for (i = 0; i < dlen / bs; i++)
! DES_ecb3_encrypt(data + i * bs, res + i * bs,
&od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 1);
return 0;
}
--- 480,487 ----
ossldata *od = c->ptr;

for (i = 0; i < dlen / bs; i++)
! DES_ecb3_encrypt((const_DES_cblock *) (data + i * bs),
! (DES_cblock *) (res + i * bs),
&od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 1);
return 0;
}
***************
*** 460,466 ****
ossldata *od = c->ptr;

for (i = 0; i < dlen / bs; i++)
! DES_ecb3_encrypt(data + i * bs, res + i * bs,
&od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 0);
return 0;
}
--- 495,502 ----
ossldata *od = c->ptr;

for (i = 0; i < dlen / bs; i++)
! DES_ecb3_encrypt((const_DES_cblock *) (data + i * bs),
! (DES_cblock *) (res + i * bs),
&od->u.des3.k1, &od->u.des3.k2, &od->u.des3.k3, 0);
return 0;
}

---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #6 (permalink)  
Old 04-11-2008, 04:49 AM
Tom Lane
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:
> (I redefined bzero and bcopy but now I think they should be
> replaced directly - patch later.)


Please. We do not use those old functions in the Postgres code;
memcpy, memmove, memset, etc are the project standard.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #7 (permalink)  
Old 04-11-2008, 04:49 AM
Marko Kreen
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 10:10:12AM -0400, Tom Lane wrote:
> Marko Kreen <marko@l-t.ee> writes:
> > Result is - it's not so bad. As I used rijndael.c to provide
> > OpenSSL's own interface, I even got rid of all the ifdefs inside
> > the code.

>
> Looks good, but I'm still getting these compile warnings:
>
> openssl.c: In function `ossl_des3_ecb_encrypt':
> openssl.c:484: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type
> openssl.c:484: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type
> openssl.c: In function `ossl_des3_ecb_decrypt':
> openssl.c:498: warning: passing arg 1 of `DES_ecb3_encrypt' from incompatible pointer type
> openssl.c:498: warning: passing arg 2 of `DES_ecb3_encrypt' from incompatible pointer type
>
> The following addition to the patch shuts up gcc with openssl 0.9.7a,
> but I'm not sure if it will break anything with older openssl ---
> comments?


They won't matter on older OpenSSL, as the macros will recast
again. But on 0.9.7e the signature is:

void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output,
DES_key_schedule *ks1,DES_key_schedule *ks2,
DES_key_schedule *ks3, int enc);

so it seems to me that with your patch the warnings will appear
on newer OpenSSL. (Confirmed)

--
marko


---------------------------(end of broadcast)---------------------------
TIP 9: In versions below 8.0, the planner will ignore your desire to
choose an index scan if your joining column's datatypes do not
match

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #8 (permalink)  
Old 04-11-2008, 04:49 AM
Marko Kreen
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

On Mon, Jul 11, 2005 at 10:13:22AM -0400, Tom Lane wrote:
> Marko Kreen <marko@l-t.ee> writes:
> > (I redefined bzero and bcopy but now I think they should be
> > replaced directly - patch later.)

>
> Please. We do not use those old functions in the Postgres code;
> memcpy, memmove, memset, etc are the project standard.


Indeed. But I'll wait until the previous sha2 patch is applied
as they would conflict.

--
marko


---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #9 (permalink)  
Old 04-11-2008, 04:49 AM
Tom Lane
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:
> On Mon, Jul 11, 2005 at 10:10:12AM -0400, Tom Lane wrote:
>> The following addition to the patch shuts up gcc with openssl 0.9.7a,
>> but I'm not sure if it will break anything with older openssl ---
>> comments?


> They won't matter on older OpenSSL, as the macros will recast
> again. But on 0.9.7e the signature is:


> void DES_ecb3_encrypt(const unsigned char *input, unsigned char *output,
> DES_key_schedule *ks1,DES_key_schedule *ks2,
> DES_key_schedule *ks3, int enc);


> so it seems to me that with your patch the warnings will appear
> on newer OpenSSL. (Confirmed)


Grumble --- you're right. It's probably not worth ifdef'ing the code to
suppress the warnings on 0.9.7a ...

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #10 (permalink)  
Old 04-11-2008, 04:49 AM
Tom Lane
 
Posts: n/a
Default Re: 4 pgcrypto regressions failures - 1 unsolved

Marko Kreen <marko@l-t.ee> writes:
> New sha2 code on Solaris 2.8 / SPARC. Seems like it has
> problems memcpy'ing to a non-8-byte-aligned uint64 *.
> ...
> Attached patch includes sys/param.h, where I found them on
> MINGW, and puts stricter checks into all files.


Applied.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 11:24 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com