Those 3 patches hopefully end my pgcrypto patch-bombing.
(Well, at least I am offline next couple weeks...)

1. Small fixes
2. Fortuna fixes
3. New README

There are 3 important points about new readme:

* It is formatted for use by asciidoc. To be able to generate
html from it. As I consider the text version to be main
target, I tried to as few formatting cruft as possible.

Here are both versions online:

http://grue.l-t.ee/~marko/src/pgcrypto/README.html
http://grue.l-t.ee/~marko/src/pgcrypto/README.txt

* I've seen people suggest to use MD5 for passwords.
Now there is special section which compares crypt algorithms
to md5 and sha1.

* I announce the removal of (digest/cipher/hmac)_exists
functions in 8.2. Reason for it is that they are useless:

- The informational question can be answered by calling the
main function and looking for error.
- They are useless for user programs, as you cannot replace
one algorithm with another on the run - missing algorithm
will be always hard error.
- The question 'does it exists' even should not be asked,
user should rather use algorithms that are _always_ there.

The situation would be different for functions that would
return a _list_ of supported algorithms. But I doubt even
the usefulness of such functions.

--
marko


---------------------------(end of broadcast)---------------------------
TIP 6: explain analyze is your friend

- Fix couple comments.

- internal.c didnt clean hash contexts when freeing

- zero the system randomness buffer too

- Google tells that we can use /dev/urandom on Cygwin,
HPUX and AIX. Add them to random.c

- remove a debug reference from pgp.h


Index: pgsql/contrib/pgcrypto/fortuna.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/fortuna.c
--- pgsql/contrib/pgcrypto/fortuna.c
*************** static void init_state(FState *st)
*** 174,181 ****
}

/*
! * Must not reseed more ofter than RESEED_PER_SEC
! * times per second.
*/
static int too_often(FState *st)
{
--- 174,181 ----
}

/*
! * The time between reseed must be at least RESEED_INTERVAL
! * microseconds.
*/
static int too_often(FState *st)
{
*************** static void reseed(FState *st)
*** 241,247 ****

memset(&key_md, 0, sizeof(key_md));
memset(buf, 0, BLOCK);
- n = k = 0;
}

/*
--- 241,246 ----
Index: pgsql/contrib/pgcrypto/internal.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/internal.c
--- pgsql/contrib/pgcrypto/internal.c
*************** int_md5_free(PX_MD * h)
*** 127,132 ****
--- 127,133 ----
{
MD5_CTX *ctx = (MD5_CTX *) h->p.ptr;

+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*************** int_sha1_free(PX_MD * h)
*** 174,179 ****
--- 175,181 ----
{
SHA1_CTX *ctx = (SHA1_CTX *) h->p.ptr;

+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*************** int_sha256_free(PX_MD * h)
*** 221,226 ****
--- 223,229 ----
{
SHA256_CTX *ctx = (SHA256_CTX *) h->p.ptr;

+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*************** int_sha384_free(PX_MD * h)
*** 267,272 ****
--- 270,276 ----
{
SHA384_CTX *ctx = (SHA384_CTX *) h->p.ptr;

+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*************** int_sha512_free(PX_MD * h)
*** 314,319 ****
--- 318,324 ----
{
SHA512_CTX *ctx = (SHA512_CTX *) h->p.ptr;

+ memset(ctx, 0, sizeof(*ctx));
px_free(ctx);
px_free(h);
}
*************** init_md5(PX_MD * md)
*** 326,331 ****
--- 331,337 ----
MD5_CTX *ctx;

ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));

md->p.ptr = ctx;

*************** init_sha1(PX_MD * md)
*** 345,350 ****
--- 351,357 ----
SHA1_CTX *ctx;

ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));

md->p.ptr = ctx;

*************** init_sha256(PX_MD * md)
*** 364,369 ****
--- 371,377 ----
SHA256_CTX *ctx;

ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));

md->p.ptr = ctx;

*************** init_sha384(PX_MD * md)
*** 383,388 ****
--- 391,397 ----
SHA384_CTX *ctx;

ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));

md->p.ptr = ctx;

*************** init_sha512(PX_MD * md)
*** 402,407 ****
--- 411,417 ----
SHA512_CTX *ctx;

ctx = px_alloc(sizeof(*ctx));
+ memset(ctx, 0, sizeof(*ctx));

md->p.ptr = ctx;

*************** static void system_reseed(void)
*** 829,834 ****
--- 839,845 ----
fortuna_add_entropy(SYSTEM_ENTROPY, buf, n);

seed_time = t;
+ memset(buf, 0, sizeof(buf));
}

int
Index: pgsql/contrib/pgcrypto/pgp-compress.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/pgp-compress.c
--- pgsql/contrib/pgcrypto/pgp-compress.c
*************** restart:
*** 270,276 ****
dec->stream.avail_out = dec->buf_len;
dec->pos = dec->buf;

! // Z_NO_FLUSH, Z_SYNC_FLUSH,
flush = dec->stream.avail_in ? Z_SYNC_FLUSH : Z_FINISH;
res = inflate(&dec->stream, flush);
if (res != Z_OK && res != Z_STREAM_END)
--- 270,280 ----
dec->stream.avail_out = dec->buf_len;
dec->pos = dec->buf;

! /*
! * Z_SYNC_FLUSH is tell zlib to output as much as possible.
! * It should do in anyway (Z_NO_FLUSH), but seems to reserve
! * the right not to. So lets follow the API.
! */
flush = dec->stream.avail_in ? Z_SYNC_FLUSH : Z_FINISH;
res = inflate(&dec->stream, flush);
if (res != Z_OK && res != Z_STREAM_END)
Index: pgsql/contrib/pgcrypto/pgp-decrypt.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/pgp-decrypt.c
--- pgsql/contrib/pgcrypto/pgp-decrypt.c
*************** static void mdc_free(void *priv)
*** 339,345 ****
ctx->mdc_ctx = NULL;
}

- // fixme: clarify
static int mdc_finish(PGP_Context *ctx, PullFilter *src,
int len, uint8 **data_p)
{
--- 339,344 ----
*************** static int mdc_finish(PGP_Context *ctx,
*** 364,369 ****
--- 363,369 ----
return 0;
}

+ /* safety check */
if (ctx->in_mdc_pkt > 1)
{
px_debug("mdc_finish: several times here?");
*************** static int mdc_finish(PGP_Context *ctx,
*** 371,376 ****
--- 371,377 ----
}
ctx->in_mdc_pkt++;

+ /* is the packet sane? */
if (res != 20)
{
px_debug("mdc_finish: read failed, res=%d", res);
Index: pgsql/contrib/pgcrypto/random.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/random.c
--- pgsql/contrib/pgcrypto/random.c
***************
*** 44,50 ****
*/
#if defined(__linux__) || defined(__FreeBSD__) || defined(__OpenBSD__) \
|| defined(__NetBSD__) || defined(__DragonFly__) \
! || defined(__darwin__) || defined(__SOLARIS__)

#define TRY_DEV_RANDOM

--- 44,52 ----
*/
#if defined(__linux__) || defined(__FreeBSD__) || defined(__OpenBSD__) \
|| defined(__NetBSD__) || defined(__DragonFly__) \
! || defined(__darwin__) || defined(__SOLARIS__) \
! || defined(__hpux) || defined(__HPUX__) \
! || defined(__CYGWIN__) || defined(_AIX)

#define TRY_DEV_RANDOM

Index: pgsql/contrib/pgcrypto/pgp.h
================================================== =================
*** pgsql.orig/contrib/pgcrypto/pgp.h
--- pgsql/contrib/pgcrypto/pgp.h
*************** unsigned pgp_armor_dec_len(unsigned len)
*** 238,245 ****
int pgp_compress_filter(PushFilter **res, PGP_Context *ctx, PushFilter *dst);
int pgp_decompress_filter(PullFilter **res, PGP_Context *ctx, PullFilter *src);

- extern void (*pgp_packet_debug) (int tag, uint8 *buf, int len);
-
int pgp_key_alloc(PGP_PubKey **pk_p);
void pgp_key_free(PGP_PubKey *pk);
int _pgp_read_public_key(PullFilter *pkt, PGP_PubKey *pk);
--- 238,243 ----

--

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

After studying Fortuna more, I found out that I missed
some of the details.

- reseeding should happen only if pool #0 has aquired additional
entropy.

- a 'rekeying' operation should happend after each request and
also after 1M of extracted data. That means taking next two
blocks and using it as new key.

- Fortuna _really_ wants entropy sources to be somewhat unpredictible.

So roll dice when adding it and also add them to pools randomly,
not sequentially.

This hopefully makes harder for someone to doctor with the
internal state (as in our case user can directly control
what goes into it).

That also drops the idea of several sources - which really
fits more to hardware backed event sources.

- add a really obvious obfuscation: take the absolutely first
block be initial counter value. If Fortuna (AES to be exact)
is secure with known counter value, then it should be also
secure with unknown counter value. This does not go against
the important property of counter - that the bit-pattern repeat
period should be as long as possible.

- S2K functions should use px_get_pseudo_random_bytes not
px_get_random_bytes.

Index: pgsql/contrib/pgcrypto/fortuna.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/fortuna.c
--- pgsql/contrib/pgcrypto/fortuna.c
***************
*** 94,107 ****
/* for one big request, reseed after this many bytes */
#define RESEED_BYTES (1024*1024)


/*
* Algorithm constants
*/

- /* max sources */
- #define MAX_SOURCES 8
-
/* Both cipher key size and hash result size */
#define BLOCK 32

--- 94,109 ----
/* for one big request, reseed after this many bytes */
#define RESEED_BYTES (1024*1024)

+ /*
+ * Skip reseed if pool 0 has less than this many
+ * bytes added since last reseed.
+ */
+ #define POOL0_FILL (256/8)

/*
* Algorithm constants
*/

/* Both cipher key size and hash result size */
#define BLOCK 32

*************** struct fortuna_state {
*** 118,126 ****
uint8 key[BLOCK];
MD_CTX pool[NUM_POOLS];
CIPH_CTX ciph;
- unsigned source_pos[MAX_SOURCES];
unsigned reseed_count;
struct timeval last_reseed_time;
};
typedef struct fortuna_state FState;

--- 120,130 ----
uint8 key[BLOCK];
MD_CTX pool[NUM_POOLS];
CIPH_CTX ciph;
unsigned reseed_count;
struct timeval last_reseed_time;
+ unsigned pool0_bytes;
+ unsigned rnd_pos;
+ int counter_init;
};
typedef struct fortuna_state FState;

*************** static void md_result(MD_CTX *ctx, uint8
*** 161,167 ****
memset(&tmp, 0, sizeof(tmp));
}

-
/*
* initialize state
*/
--- 165,170 ----
*************** static void init_state(FState *st)
*** 174,179 ****
--- 177,208 ----
}

/*
+ * Endianess does not matter.
+ * It just needs to change without repeating.
+ */
+ static void inc_counter(FState *st)
+ {
+ uint32 *val = (uint32*)st->counter;
+ if (++val[0])
+ return;
+ if (++val[1])
+ return;
+ if (++val[2])
+ return;
+ ++val[3];
+ }
+
+ /*
+ * This is called 'cipher in counter mode'.
+ */
+ static void encrypt_counter(FState *st, uint8 *dst)
+ {
+ ciph_encrypt(&st->ciph, st->counter, dst);
+ inc_counter(st);
+ }
+
+
+ /*
* The time between reseed must be at least RESEED_INTERVAL
* microseconds.
*/
*************** static void reseed(FState *st)
*** 207,215 ****
MD_CTX key_md;
uint8 buf[BLOCK];

! /* check frequency */
! if (too_often(st))
! return;

/*
* Both #0 and #1 reseed would use only pool 0.
--- 236,243 ----
MD_CTX key_md;
uint8 buf[BLOCK];

! /* set pool as empty */
! st->pool0_bytes = 0;

/*
* Both #0 and #1 reseed would use only pool 0.
*************** static void reseed(FState *st)
*** 244,292 ****
}

/*
* update pools
*/
! static void add_entropy(FState *st, unsigned src_id, const uint8 *data, unsigned len)
{
unsigned pos;
uint8 hash[BLOCK];
MD_CTX md;

- /* just in case there's a bug somewhere */
- if (src_id >= MAX_SOURCES)
- src_id = USER_ENTROPY;
-
/* hash given data */
md_init(&md);
md_update(&md, data, len);
md_result(&md, hash);

! /* update pools round-robin manner */
! pos = st->source_pos[src_id];
md_update( &st->pool[pos], hash, BLOCK);

! if (++pos >= NUM_POOLS)
! pos = 0;
! st->source_pos[src_id] = pos;

memset(hash, 0, BLOCK);
memset(&md, 0, sizeof(md));
}

/*
! * Endianess does not matter.
! * It just needs to change without repeating.
*/
! static void inc_counter(FState *st)
{
! uint32 *val = (uint32*)st->counter;
! if (++val[0])
! return;
! if (++val[1])
! return;
! if (++val[2])
! return;
! ++val[3];
}

static void extract_data(FState *st, unsigned count, uint8 *dst)
--- 272,351 ----
}

/*
+ * Pick a random pool. This uses key bytes as random source.
+ */
+ static unsigned get_rand_pool(FState *st)
+ {
+ unsigned rnd;
+
+ /*
+ * This slightly prefers lower pools - thats OK.
+ */
+ rnd = st->key[st->rnd_pos] % NUM_POOLS;
+
+ st->rnd_pos++;
+ if (st->rnd_pos >= BLOCK)
+ st->rnd_pos = 0;
+
+ return rnd;
+ }
+
+ /*
* update pools
*/
! static void add_entropy(FState *st, const uint8 *data, unsigned len)
{
unsigned pos;
uint8 hash[BLOCK];
MD_CTX md;

/* hash given data */
md_init(&md);
md_update(&md, data, len);
md_result(&md, hash);

! /*
! * Make sure the pool 0 is initialized,
! * then update randomly.
! */
! if (st->reseed_count == 0 && st->pool0_bytes < POOL0_FILL)
! pos = 0;
! else
! pos = get_rand_pool(st);
md_update( &st->pool[pos], hash, BLOCK);

! if (pos == 0)
! st->pool0_bytes += len;

memset(hash, 0, BLOCK);
memset(&md, 0, sizeof(md));
}

/*
! * Just take 2 next blocks as new key
*/
! static void rekey(FState *st)
{
! encrypt_counter(st, st->key);
! encrypt_counter(st, st->key + CIPH_BLOCK);
! ciph_init(&st->ciph, st->key, BLOCK);
! }
!
! /*
! * Fortuna relies on AES standing known-plaintext attack.
! * In case it does not, slow down the attacker by initialising
! * the couter to random value.
! */
! static void init_counter(FState *st)
! {
! /* Use next block as counter. */
! encrypt_counter(st, st->counter);
!
! /* Hide the key. */
! rekey(st);
!
! /* The counter can be shuffled only once. */
! st->counter_init = 1;
}

static void extract_data(FState *st, unsigned count, uint8 *dst)
*************** static void extract_data(FState *st, uns
*** 294,324 ****
unsigned n;
unsigned block_nr = 0;

! /*
! * Every request should be with different key,
! * if possible.
! */
! reseed(st);
!
! /*
! * If the reseed didn't happen, don't use the old data
! * rather encrypt again.
! */

while (count > 0) {
- /* must not give out too many bytes with one key */
- if (block_nr > (RESEED_BYTES / CIPH_BLOCK))
- {
- reseed(st);
- block_nr = 0;
- }
-
/* produce bytes */
! ciph_encrypt(&st->ciph, st->counter, st->result);
! block_nr++;
!
! /* prepare for next time */
! inc_counter(st);

/* copy result */
if (count > CIPH_BLOCK)
--- 353,369 ----
unsigned n;
unsigned block_nr = 0;

! /* Can we reseed? */
! if (st->pool0_bytes >= POOL0_FILL && !too_often(st))
! reseed(st);
!
! /* Is counter initialized? */
! if (!st->counter_init)
! init_counter(st);

while (count > 0) {
/* produce bytes */
! encrypt_counter(st, st->result);

/* copy result */
if (count > CIPH_BLOCK)
*************** static void extract_data(FState *st, uns
*** 328,334 ****
--- 373,389 ----
memcpy(dst, st->result, n);
dst += n;
count -= n;
+
+ /* must not give out too many bytes with one key */
+ block_nr++;
+ if (block_nr > (RESEED_BYTES / CIPH_BLOCK))
+ {
+ rekey(st);
+ block_nr = 0;
+ }
}
+ /* Set new key for next request. */
+ rekey(st);
}

/*
*************** static void extract_data(FState *st, uns
*** 338,344 ****
static FState main_state;
static int init_done = 0;

! void fortuna_add_entropy(unsigned src_id, const uint8 *data, unsigned len)
{
if (!init_done)
{
--- 393,399 ----
static FState main_state;
static int init_done = 0;

! void fortuna_add_entropy(const uint8 *data, unsigned len)
{
if (!init_done)
{
*************** void fortuna_add_entropy(unsigned src_id
*** 347,353 ****
}
if (!data || !len)
return;
! add_entropy(&main_state, src_id, data, len);
}

void fortuna_get_bytes(unsigned len, uint8 *dst)
--- 402,408 ----
}
if (!data || !len)
return;
! add_entropy(&main_state, data, len);
}

void fortuna_get_bytes(unsigned len, uint8 *dst)
Index: pgsql/contrib/pgcrypto/fortuna.h
================================================== =================
*** pgsql.orig/contrib/pgcrypto/fortuna.h
--- pgsql/contrib/pgcrypto/fortuna.h
***************
*** 32,45 ****
#ifndef __FORTUNA_H
#define __FORTUNA_H

- /*
- * Event source ID's
- */
- #define SYSTEM_ENTROPY 0
- #define USER_ENTROPY 1
-
void fortuna_get_bytes(unsigned len, uint8 *dst);
! void fortuna_add_entropy(unsigned src_id, const uint8 *data, unsigned len);

#endif

--- 32,39 ----
#ifndef __FORTUNA_H
#define __FORTUNA_H

void fortuna_get_bytes(unsigned len, uint8 *dst);
! void fortuna_add_entropy(const uint8 *data, unsigned len);

#endif

Index: pgsql/contrib/pgcrypto/internal.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/internal.c
--- pgsql/contrib/pgcrypto/internal.c
***************
*** 42,50 ****
#include "fortuna.h"

/*
! * How often to try to acquire system entropy. (In seconds)
*/
! #define SYSTEM_RESEED_FREQ (3*60*60)


#ifndef MD5_DIGEST_LENGTH
--- 42,63 ----
#include "fortuna.h"

/*
! * System reseeds should be separated at least this much.
*/
! #define SYSTEM_RESEED_MIN (20*60) /* 20 min */
! /*
! * How often to roll dice.
! */
! #define SYSTEM_RESEED_CHECK_TIME (10*60) /* 10 min */
! /*
! * The chance is x/256 that the reseed happens.
! */
! #define SYSTEM_RESEED_CHANCE (4) /* 256/4 * 10min ~ 10h */
!
! /*
! * If this much time has passed, force reseed.
! */
! #define SYSTEM_RESEED_MAX (12*60*60) /* 12h */


#ifndef MD5_DIGEST_LENGTH
*************** px_get_pseudo_random_bytes(uint8 *dst, u
*** 823,842 ****
}

static time_t seed_time = 0;

static void system_reseed(void)
{
uint8 buf[1024];
int n;
time_t t;

t = time(NULL);
! if (seed_time && (t - seed_time) < SYSTEM_RESEED_FREQ)
return;

n = px_acquire_system_randomness(buf);
if (n > 0)
! fortuna_add_entropy(SYSTEM_ENTROPY, buf, n);

seed_time = t;
memset(buf, 0, sizeof(buf));
--- 836,875 ----
}

static time_t seed_time = 0;
+ static time_t check_time = 0;

static void system_reseed(void)
{
uint8 buf[1024];
int n;
time_t t;
+ int skip = 1;

t = time(NULL);
!
! if (seed_time == 0)
! skip = 0;
! else if ((t - seed_time) < SYSTEM_RESEED_MIN)
! skip = 1;
! else if ((t - seed_time) > SYSTEM_RESEED_MAX)
! skip = 0;
! else if (!check_time || (t - check_time) > SYSTEM_RESEED_CHECK_TIME)
! {
! check_time = t;
!
! /* roll dice */
! px_get_random_bytes(buf, 1);
! skip = buf[0] >= SYSTEM_RESEED_CHANCE;
! }
! /* clear 1 byte */
! memset(buf, 0, sizeof(buf));
!
! if (skip)
return;

n = px_acquire_system_randomness(buf);
if (n > 0)
! fortuna_add_entropy(buf, n);

seed_time = t;
memset(buf, 0, sizeof(buf));
*************** int
*** 854,860 ****
px_add_entropy(const uint8 *data, unsigned count)
{
system_reseed();
! fortuna_add_entropy(USER_ENTROPY, data, count);
return 0;
}

--- 887,893 ----
px_add_entropy(const uint8 *data, unsigned count)
{
system_reseed();
! fortuna_add_entropy(data, count);
return 0;
}

Index: pgsql/contrib/pgcrypto/pgp-s2k.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/pgp-s2k.c
--- pgsql/contrib/pgcrypto/pgp-s2k.c
*************** pgp_s2k_fill(PGP_S2K *s2k, int mode,int
*** 225,237 ****
case 0:
break;
case 1:
! res = px_get_random_bytes(s2k->salt, PGP_S2K_SALT);
break;
case 3:
! res = px_get_random_bytes(s2k->salt, PGP_S2K_SALT);
if (res < 0)
break;
! res = px_get_random_bytes(&tmp, 1);
if (res < 0)
break;
s2k->iter = decide_count(tmp);
--- 225,237 ----
case 0:
break;
case 1:
! res = px_get_pseudo_random_bytes(s2k->salt, PGP_S2K_SALT);
break;
case 3:
! res = px_get_pseudo_random_bytes(s2k->salt, PGP_S2K_SALT);
if (res < 0)
break;
! res = px_get_pseudo_random_bytes(&tmp, 1);
if (res < 0)
break;
s2k->iter = decide_count(tmp);
Index: pgsql/contrib/pgcrypto/pgp-pgsql.c
================================================== =================
*** pgsql.orig/contrib/pgcrypto/pgp-pgsql.c
--- pgsql/contrib/pgcrypto/pgp-pgsql.c
*************** PG_FUNCTION_INFO_V1(pg_dearmor);
*** 87,123 ****
} while (0)

/*
* Mix user data into RNG. It is for user own interests to have
* RNG state shuffled.
*/
static void add_entropy(text *data1, text *data2, text *data3)
{
PX_MD *md;
! uint8 sha1[20];
! int res;

if (!data1 && !data2 && !data3)
return;

! res = px_find_digest("sha1", &md);
! if (res < 0)
return;

! if (data1)
! px_md_update(md, VARDATA(data1), VARSIZE(data1) - VARHDRSZ);
! if (data2)
! px_md_update(md, VARDATA(data2), VARSIZE(data2) - VARHDRSZ);
! if (data3)
! px_md_update(md, VARDATA(data3), VARSIZE(data3) - VARHDRSZ);

! px_md_finish(md, sha1);
! px_md_free(md);

! res = px_add_entropy(sha1, 20);
! memset(sha1, 0, 20);

! if (res < 0)
! ereport(NOTICE, (errmsg("add_entropy: %s", px_strerror(res))));
}

/*
--- 87,146 ----
} while (0)

/*
+ * Mix a block of data into RNG.
+ */
+ static void add_block_entropy(PX_MD *md, text *data)
+ {
+ uint8 sha1[20];
+
+ px_md_reset(md);
+ px_md_update(md, VARDATA(data), VARSIZE(data) - VARHDRSZ);
+ px_md_finish(md, sha1);
+
+ px_add_entropy(sha1, 20);
+
+ memset(sha1, 0, 20);
+ }
+
+ /*
* Mix user data into RNG. It is for user own interests to have
* RNG state shuffled.
*/
static void add_entropy(text *data1, text *data2, text *data3)
{
PX_MD *md;
! uint8 rnd[3];

if (!data1 && !data2 && !data3)
return;

! if (px_get_random_bytes(rnd, 3) < 0)
return;

! if (px_find_digest("sha1", &md) < 0)
! return;

! /*
! * Try to make the feeding unpredictable.
! *
! * Prefer data over keys, as it's rather likely
! * that key is same in several calls.
! */

! /* chance: 7/8 */
! if (data1 && rnd[0] >= 32)
! add_block_entropy(md, data1);
!
! /* chance: 5/8 */
! if (data2 && rnd[1] >= 160)
! add_block_entropy(md, data2);
!
! /* chance: 5/8 */
! if (data3 && rnd[2] >= 160)
! add_block_entropy(md, data3);

! px_md_free(md);
! memset(rnd, 0, sizeof(rnd));
}

/*

--

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

- reformat in asciidoc syntax
- clarify various parts
- add high-level description of PGP structure
- add a comparison of crypt vs. regular hashes


Index: pgsql/contrib/pgcrypto/README.pgcrypto
================================================== =================
*** pgsql.orig/contrib/pgcrypto/README.pgcrypto
--- pgsql/contrib/pgcrypto/README.pgcrypto
***************
*** 1,362 ****

! pgcrypto 0.4 - cryptographic functions for PostgreSQL.
! ================================================== ====
! by Marko Kreen <marko@l-t.ee>


! INSTALLATION
! ============

! Edit makefile, if you want to use any external library.

! NB! Default randomness source is libc random() function. This
! is so only to get pgcrypto build everywhere. Randomness is
! needed for gen_salt() and pgp_encrypt() functions. So if you plan
! using those, you should definitely change that by editing Makefile.
! You can should use urandom device if your OS supports it, otherwise
! link pgcrypto against OpenSSL library and use its PRNG.

! After editing Makefile:

- make
- make install

! To run regression tests, install both PostgreSQL and pgcrypto
! and then run

! make installcheck

! SQL FUNCTIONS
! =============

! If any of arguments are NULL they return NULL.

! digest(data::bytea, type::text)::bytea

! Type is here the algorithm to use. E.g. 'md5', 'sha1', ...
! Returns binary hash.

! digest_exists(type::text)::bool

! Returns BOOL whether given hash exists.

! hmac(data::bytea, key::bytea, type::text)::bytea

- Calculates Hashed MAC over data. type is the same as
- in digest(). Returns binary hash. Similar to digest()
- but noone can alter data and re-calculate hash without
- knowing key. If the key is larger than hash blocksize
- it will first hashed and the hash will be used as key.
-
- [ HMAC is described in RFC2104. ]

! hmac_exists(type::text)::bool
! Returns BOOL. It is separate function because all hashes
! cannot be used in HMAC.

! crypt(password::text, salt::text)::text

- Calculates UN*X crypt(3) style hash. Useful for storing
- passwords. For generating salt you should use the
- gen_salt() function. Usage:

! New password:
!
! UPDATE .. SET pswhash = crypt(new_psw, gen_salt('md5'));
!
! Authentication:

! SELECT pswhash = crypt(given_psw, pswhash) WHERE .. ;
!
! returns BOOL whether the given_psw is correct. DES crypt
! has max key of 8 bytes, MD5 has max key at least 2^32-1
! bytes but may be larger on some platforms...

- Builtin crypt() supports DES, Extended DES, MD5 and Blowfish
- (variant 2a) algorithms.

! gen_salt(type::text)::text

! Generates a new random salt for usage in crypt(). Type
!
! 'des' - Old UNIX, not recommended
! 'md5' - md5-based crypt()
! 'xdes' - 'Extended DES'
! 'bf' - Blowfish-based, variant 2a

! When you use --enable-system-crypt then note that system
! libcrypt may not support them all.

! gen_salt(type::text, rounds::int4)::text

- same as above, but lets user specify iteration count
- for algorithm. Number is algorithm specific:

! type default min max
! ---------------------------------
! xdes 725 1 16777215
! bf 6 4 31

! In case of xdes there is a additional limitation that the
! count must be a odd number.

! The higher the count, the more time it takes to calculate
! crypt and therefore the more time to break it. But beware!
! With too high count it takes a _very_long_ time to
! calculate it.

! For maximum security, you should choose the 'bf' crypt
! and use maximum number of rounds you can still tolerate.

! armor(bytea)::text
! dearmor(text)::bytea

! Those wrap/unwrap data into PGP Ascii Armor which
! is basically Base64 with CRC and additional formatting.

! pgp_sym_encrypt(data::text, key::text)::bytea
! pgp_sym_encrypt(data::text, key::text, arg::text)::bytea
! pgp_sym_encrypt_bytea(data::bytea, key::text)::bytea
! pgp_sym_encrypt_bytea(data::bytea, key::text, arg::text)::bytea

- pgp_sym_decrypt(data::bytea, key::text)::text
- pgp_sym_decrypt(data::bytea, key::text, arg::text)::text
- pgp_sym_decrypt_bytea(data::text, key::text)::bytea
- pgp_sym_decrypt_bytea(data::text, key::text, arg::text)::bytea

! Encrypt data into OpenPGP Symmetrically Encrypted Data
! message. And decrypt it from it.

! Note that the pgp_sym_encrypt_bytea functions tag the data
! as binary, as the pgp_sym_encrypt will tag the data as text.
! You can not decrypt the binary data as text. But you can
! decrypt text data as binary. This rule avoids having
! broken textual data in PostgreSQL.

! Both encrypt and decrypt accept also third argument, which
! is parameters to the function in following format:
!
! parm=val[,parm=val]...
!
! Example:

! select pgp_sym_encrypt('data', 'psw',
! 'compress-algo=2, unicode-mode=1');

! Accepted parameters are:

- cipher-algo: bf, aes, aes128, aes192, aes256
- Cipher algorithm to use. OpenSSL gives additional algorithms:
- 3des, cast5
- Default: aes128
-
- compress-algo: 0, 1, 2
- Which compression algorithm to use.
- 0 - no compression
- 1 - ZIP compression
- 2 - ZLIB compression [=ZIP plus meta-data and block-CRC's]
- Default: 0

- compress-level: 0, 1-9
- How much to compress. Bigger level compresses smaller
- but also slower. 0 disables compression.
- Default: 6

! convert-crlf: 0, 1
! Whether to convert \n into \r\n when encrypting and
! \r\n to \n when decrypting. RFC2440 specifies that
! text packets should use "\r\n" line-feeds.
! Use this to get fully RFC-compliant behaviour.
! Default: 0

! disable-mdc: 0, 1
! Do not protect data with SHA-1. Note that SHA-1 protected
! packet is from upcoming update to RFC2440. (Currently at
! version RFC2440bis-13.) You need to disable it if you need
! compatibility with ancient PGP products. Recent gnupg.org
! and pgp.com software supports it fine.
! Default: 0
!
! enable-session-key: 0, 1
! Use separate session key.
! Default: 0
!
! s2k-mode: 0, 1, 3
! Which S2K algorithm to use. 0 is dangerous - without salt.
! Default: 3
!
! s2k-digest-algo: md5, sha1
! Which digest algorithm to use in S2K calculation.
! Default: SHA-1

! s2k-cipher-algo: bf, aes, aes128, aes192, aes256
! Which cipher to use for encrypting separate session key.
! Default: same as cipher-algo.

! unicode-mode: 0, 1
! Whether to convert textual data from database internal
! encoding to UTF-8 and back.
! Default: 0

! Only 'convert-crlf' applies to both encrypt and decrypt,
! all others apply only to encrypt - decrypt gets the
! settings from PGP data.
!
!
! pgp_pub_encrypt(data::text, key::bytea)::bytea
! pgp_pub_encrypt(data::text, key::bytea, arg::text)::bytea
! pgp_pub_encrypt_bytea(data::bytea, bytea::text)::bytea
! pgp_pub_encrypt_bytea(data::bytea, bytea::text, arg::text)::bytea

- pgp_pub_decrypt(data::bytea, key::bytea)::text
- pgp_pub_decrypt(data::bytea, key::bytea, psw::text)::text
- pgp_pub_decrypt(data::bytea, key::bytea, psw::text, arg::text)::text
- pgp_pub_decrypt_bytea(data::text, key::bytea)::bytea
- pgp_pub_decrypt_bytea(data::text, key::bytea, psw::text)::bytea
- pgp_pub_decrypt_bytea(data::text, key::bytea, psw::text, arg::bytea)::bytea

! Encrypt data into OpenPGP Public-Key Encrypted Data
! message. And decrypt it from it. The arg parameter is
! described in pgp_sym_* section.

! The key must be a public-key packet for pgp_pub_encrypt
! functions and a secret key packet for pgp_pub_decrypt
! functions. Trying to encrypt with secret key gives a error.
! While being technically possible, it is probably a sign of
! user error and leaking secret keys.

! Here is a example how to generate them:
!
! Generate a new key:
!
! gpg --gen-key
!
! You need to pick "DSA and Elgamal" key type, others
! are sign-only.

! List keys:
!
! gpg --list-secret-keys

! Export ascii-armored public key:

! gpg -a --export KEYID > public.key
!
! Export ascii-armored secret key:

! gpg -a --export-secret-keys KEYID > secret.key

! You need to use dearmor() on them before giving giving
! them to pgp_pub_* functions. Ofcourse, if you can handle
! binary data, you can drop "-a" from gpg.


! pgp_key_id(key / data)

! It shows you either key ID if given PGP public or secret
! key. Or it gives the key ID what was used for encrypting
! the data, if given encrypted data.

! It can return 2 special key ID's:

! SYMKEY - it got symmetrically encrypted data.
! ANYKEY - the data packet key ID is clear. That means
! you should try all you secret keys on it.

- encrypt(data::bytea, key::bytea, type::text)::bytea
- decrypt(data::bytea, key::bytea, type::text)::bytea
- encrypt_iv(data::bytea, key::bytea, iv::bytea, type::text)::bytea
- decrypt_iv(data::bytea, key::bytea, iv::bytea, type::text)::bytea

! Encrypt/decrypt data with cipher, padding data if needed.

! Pseudo-noteup:

! algo ['-' mode] ['/pad:' padding]

! Supported algorithms:
!
! bf - Blowfish
! aes, rijndael - Rijndael-128

! Others depend on library and are not tested enough, so
! play on your own risk.

! Modes: 'cbc' (default), 'ecb'. Again, library may support
! more.

! Padding is 'pkcs' (default), 'none'. 'none' is mostly for
! testing ciphers, you should not need it.

! So, example:

- encrypt(data, 'fooz', 'bf')
-
- is equal to

! encrypt(data, 'fooz', 'bf-cbc/padkcs')

! IV is initial value for mode, defaults to all zeroes.
! It is ignored for ECB. It is clipped or padded with zeroes
! if not exactly block size.


! ALGORITHMS
! ==========

! The standard functionality at the moment consists of

! Hashes: md5, sha1
! Ciphers: bf, aes
! Modes: cbc, ecb

- TODO: write standard names for optional ciphers too.

! LIBRARIES
! =========

! * crypt()

- internal: des, xdes, md5, bf

! -lcrypt: ??? (whatever you have)

! * other:

! [ This only lists stuff that the libraries claim to support. So
! pgcrypto may work with all of them. But ATM tested are only the
! standard ciphers. On others pgcrypto and library may mess something
! up. You have been warned. ]

! internal (default):
! Hashes: MD5, SHA1
! Ciphers: Blowfish, Rijndael-128


! OpenSSL (0.9.7):
! Hashes: MD5, SHA1, RIPEMD160, MD2
! Ciphers: Blowfish, AES, CAST5, DES, 3DES
! License: BSD-like with strong advertisement
! Url: http://www.openssl.org/


! CREDITS
! =======

I have used code from following sources:

! DES crypt() by David Burren and others FreeBSD libcrypt
! MD5 crypt() by Poul-Henning Kamp FreeBSD libcrypt
! Blowfish crypt() by Solar Designer www.openwall.com
! Blowfish cipher by Niels Provos OpenBSD sys/crypto
! Rijndael cipher by Brian Gladman OpenBSD sys/crypto
! MD5 and SHA1 by WIDE Project KAME kame/sys/crypto

- LEGALESE
- ========

! * I owe a beer to Poul-Henning.

* This product includes software developed by Niels Provos.


--- 1,675 ----

! pgcrypto - cryptographic functions for PostgreSQL
! =================================================
! Marko Kreen <marko@l-t.ee>


! 1. Installation
! -----------------

! Run following commands:

! make
! make install
! make installcheck

! The `make installcheck` command is important. It runs regression tests
! for the module. They make sure the functions here produce correct
! results.


! 2. Notes
! ----------

! 2.1. Configuration
! ~~~~~~~~~~~~~~~~~~~~

! pgcrypto configures itself according to the findings of main PostgreSQL
! `configure` script. The options that affect it are `--with-zlib` and
! `--with-openssl`.

! Without zlib, the PGP functions will not support compressed data inside
! PGP encrypted packets.

! Without OpenSSL public-key encryption does not work, as pgcrypto does
! not contain yet math functions for large integers.

! There are some other differences between with and without OpenSSL build:

! `----------------------------`---------`------------
! Functionality built-in OpenSSL
! ----------------------------------------------------
! MD5 yes yes
! SHA1 yes yes
! SHA256/384/512 yes since 0.9.8
! Any other digest algo no yes (1)
! Blowfish yes yes
! AES yes yes (2)
! DES/3DES/CAST5 no yes
! Raw encryption yes yes
! PGP Symmetric encryption yes yes
! PGP Public-Key encryption no yes
! ----------------------------------------------------

! 1. Any digest algorithm OpenSSL supports is automatically picked up.
! This is not possible with ciphers, which need to be supported
! explicitly.

! 2. AES is included in OpenSSL since version 0.9.7. If pgcrypto is
! compiled against older version, it will use built-in AES code,
! so it has AES always available.


! 2.2. NULL handling
! ~~~~~~~~~~~~~~~~~~~~

! As standard in SQL, all functions return NULL, if any of it's arguments
! are NULL. This may create security risks on careless usage.


! 2.3. Deprecated functions
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~

! The `digest_exists()`, `hmac_exists()` and `cipher_exists()` functions
! are deprecated. The plan is to remove those in PostgreSQL 8.2.


! 2.4. Security
! ~~~~~~~~~~~~~~~

! All the functions here run inside database server. That means that all
! the data and passwords move between pgcrypto and client application in
! clear-text. Thus you must:

! 1. Connect locally or use SSL connections.
! 2. Trust both system and database administrator.

! If you cannot, then better do crypto inside client application.


! 3. General hashing
! --------------------

! 3.1. digest(data, type)
! ~~~~~~~~~~~~~~~~~~~~~~~~~

! digest(data text, type text) RETURNS bytea
! digest(data bytea, type text) RETURNS bytea

! Type is here the algorithm to use. Standard algorithms are `md5` and
! `sha1`, although there may be more supported, depending on build
! options.

! Returns binary hash.

! If you want hexadecimal string, use `encode()` on result. Example:

! CREATE OR REPLACE FUNCTION sha1(bytea) RETURNS text AS $$
! SELECT encode(digest($1, 'sha1'), 'hex')
! $$ LANGUAGE SQL STRICT IMMUTABLE;


! 3.2. hmac(data, key, type)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

! hmac(data text, key text, type text) RETURNS bytea
! hmac(data bytea, key text, type text) RETURNS bytea

! Calculates Hashed MAC over data. `type` is the same as in `digest()`.
! If the key is larger than hash block size it will first hashed and the
! hash will be used as key.

! It is similar to digest() but the hash can be recalculated only knowing
! the key. This avoids the scenario of someone altering data and also
! changing the hash.

! Returns binary hash.



! 4. Password hashing
! ---------------------

! The functions `crypt()` and `gen_salt()` are specifically designed
! for hashing passwords. `crypt()` does the hashing and `gen_salt()`
! prepares algorithm parameters for it.

! The algorithms in `crypt()` differ from usual hashing algorithms like
! MD5 or SHA1 in following respects:

! 1. They are slow. As the amount of data is so small, this is only
! way to make brute-forcing passwords hard.
! 2. Include random 'salt' with result, so that users having same
! password would have different crypted passwords. This also
! additional defense against reversing the algorithm.
! 3. Include algorithm type in the result, so passwords hashed with
! different algorithms can co-exist.
! 4. Some of them are adaptive - that means after computers get
! faster, you can tune the algorithm to be slower, without
! introducing incompatibility with existing passwords.

! Supported algorithms:
! `------`-------------`---------`----------`---------------------------
! Type Max password Adaptive Salt bits Description
! ----------------------------------------------------------------------
! `bf` 72 yes 128 Blowfish-based, variant 2a
! `md5` unlimited no 48 md5-based crypt()
! `xdes` 8 yes 24 Extended DES
! `des` 8 no 12 Original UNIX crypt
! ----------------------------------------------------------------------


! 4.1. crypt(password, salt)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~

! crypt(password text, salt text) RETURNS text

! Calculates UN*X crypt(3) style hash of password. When storing new
! password, you need to use function `gen_salt()` to generate new salt.
! When checking password you should use existing hash as salt.

! Example - setting new password:

! UPDATE .. SET pswhash = crypt('new password', gen_salt('md5'));

! Example - authentication:

! SELECT pswhash = crypt('entered password', pswhash) WHERE .. ;

! returns true or false whether the entered password is correct.
! It also can return NULL if `pswhash` field is NULL.


! 4.2. gen_salt(type)
! ~~~~~~~~~~~~~~~~~~~~~

! gen_salt(type text) RETURNS text

! Generates a new random salt for usage in `crypt()`. For adaptible
! algorithms, it uses the default iteration count.

! Accepted types are: `des`, `xdes`, `md5` and `bf`.


! 4.3. gen_salt(type, rounds)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

! gen_salt(type text, rounds integer) RETURNS text

! Same as above, but lets user specify iteration count for some
! algorithms. The higher the count, the more time it takes to hash
! ti password and therefore the more time to break it. Although with
! too high count the time to calculate a hash may be several years
! - which is somewhat impractical.

! Number is algorithm specific:

! `-----'---------'-----'----------
! type default min max
! ---------------------------------
! `xdes` 725 1 16777215
! `bf` 6 4 31
! ---------------------------------

! In case of xdes there is a additional limitation that the count must be
! a odd number.

! Notes:

! - Original DES crypt was designed to have the speed of 4 hashes per
! second on the hardware that time.
! - Slower that 4 hashes per second would probably damper usability.
! - Faster that 100 hashes per second is probably too fast.
! - See next section about possible values for `crypt-bf`.


! 4.4. Comparison of crypt and regular hashes
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

! Here is a table that should give overview of relative slowness
! of different hashing algorithms.

+ * The goal is to crack a 8-character password, which consists:
+ 1. Only from lowercase letters
+ 2. Numbers, lower- and uppercase letters.
+ * The table below shows how much time it would take to try all
+ combinations of characters.
+ * The `crypt-bf` is featured in several settings - the number
+ after slash is the `rounds` parameter of `gen_salt()`.

! `------------'----------'--------------'--------------------
! Algorithm Hashes/sec Chars: [a-z] Chars: [A-Za-z0-9]
! ------------------------------------------------------------
! crypt-bf/8 28 246 years 251322 years
! crypt-bf/7 57 121 years 123457 years
! crypt-bf/6 112 62 years 62831 years
! crypt-bf/5 211 33 years 33351 years
! crypt-md5 2681 2.6 years 2625 years
! crypt-des 362837 7 days 19 years
! sha1 590223 4 days 12 years
! md5 2345086 1 day 3 years
! password 143781000 25 mins 18 days
! ------------------------------------------------------------

! * The machine used is 1.5GHz Pentium 4.
! * crypt-des and crypt-md5 algorithm numbers are taken from
! John the Ripper v1.6.38 `-test` output.
! * MD5 numbers are from mdcrack 1.2.
! * SHA1 numbers are from lcrack-20031130-beta.
! * MySQL password() numbers are from my own tests.
! (http://grue.l-t.ee/~marko/src/mypass/)
! * `crypt-bf` numbers are taken using simple program that loops
! over 1000 8-character passwords. That way I can show the speed with
! different number of rounds. For reference: `john -test` shows 213
! loops/sec for crypt-bf/5. (The small difference in results is in
! accordance to the fact that the `crypt-bf` implementation in pgcrypto
! is same one that is used in John the Ripper.)

! Note that the "try all combinations" is not a realistic exercise.
! Usually password cracking is done with the help of dictionaries, which
! contain both regular words and various mutations of them. So, even
! somewhat word-like passwords will be cracked much faster than the above
! numbers suggest, and a 6-character non-word like password may escape
! cracking. Or may not.


! 5. PGP encryption
! -------------------

! The functions here implement the encryption part of OpenPGP (RFC2440)
! standard.


! 5.1. Overview
! ~~~~~~~~~~~~~~~

! Encrypted PGP message consists of 2 packets:

! - Packet for session key - either symmetric- or public-key encrypted.
! - Packet for session-key encrypted data.

! When encrypting with password:

+ 1. Given password is hashed using String2Key (S2K) algorithm. This
+ is rather similar to `crypt()` algorithm - purposefully slow
+ and with random salt - but is produces a full-length binary key.
+ 2. If separate session key is requested, new random key will be
+ generated. Otherwise S2K key will be used directly as session key.
+ 3. If S2K key is to be used directly, then only S2K settings will be put
+ into session key packet. Otherwise session key will be encrypted with
+ S2K key and put into session key packet.

! When encrypting with public key:

+ 1. New random session key is generated.
+ 2. It is encrypted using public key and put into session key packet.

! Now common part, the session-key encrypted data packet:
!
! 1. Optional data-manipulation: compression, conversion to UTF-8,
! conversion of line-endings.
! 2. Data is prefixed with block of random bytes. This is equal
! to using random IV.
! 3. A SHA1 hash of random prefix and data is appended.
! 4. All this is encrypted with session key.
!
!
! 5.2. pgp_sym_encrypt(data, psw)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! pgp_sym_encrypt(data text, psw text [, options text] ) RETURNS bytea
! pgp_sym_encrypt_bytea(data bytea, psw text [, options text] ) RETURNS bytea
!
! Return a symmetric-key encrypted PGP message.
!
! Options are described in section 5.7.
!
!
! 5.3. pgp_sym_decrypt(msg, psw)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! pgp_sym_decrypt(msg bytea, psw text [, options text] ) RETURNS text
! pgp_sym_decrypt_bytea(msg bytea, psw text [, options text] ) RETURNS bytea
!
! Decrypt a symmetric-key encrypted PGP message.
!
! Options are described in section 5.7.
!
!
! 5.4. pgp_pub_encrypt(data, pub_key)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! pgp_pub_encrypt(data text, key bytea [, options text] ) RETURNS bytea
! pgp_pub_encrypt_bytea(data bytea, key bytea [, options text] ) RETURNS bytea
!
! Encrypt data with a public key. Giving this function a secret key will
! produce a error.
!
! Options are described in section 5.7.
!
!
! 5.5. pgp_pub_decrypt(msg, sec_key [, psw])
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! pgp_pub_decrypt(msg bytea, key bytea [, psw text [, options text]] ) \
! RETURNS text
! pgp_pub_decrypt_bytea(msg bytea, key bytea [,psw text [, options text]] ) \
! RETURNS bytea
!
! Decrypt a public-key encrypted message with secret key. If the secret
! key is password-protected, you must give the password in `psw`. If
! there is no password, but you want to specify option for function, you
! need to give empty password.
!
! Options are described in section 5.7.
!
!
! 5.6. pgp_key_id(key / msg)
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! pgp_key_id(key or msg bytea) RETURNS text
!
! It shows you either key ID if given PGP public or secret key. Or it
! gives the key ID what was used for encrypting the data, if given
! encrypted message.
!
! It can return 2 special key ID's:
!
! SYMKEY::
! The data is encrypted with symmetric key.
!
! ANYKEY::
! The data is public-key encrypted, but the key ID is cleared.
! That means you need to try all your secret keys on it to see
! which one decrypts it. pgcrypto itself does not produce such
! messages.
!
! Note that different keys may have same ID. This is rare but normal
! event. Client application should then try to decrypt with each one,
! to see which fits - like handling ANYKEY.
!
!
! 5.7. armor / dearmor
! ~~~~~~~~~~~~~~~~~~~~~~
!
! armor(data bytea) RETURNS text
! dearmor(data text) RETURNS bytea
!
! Those wrap/unwrap data into PGP Ascii Armor which is basically Base64
! with CRC and additional formatting.
!
!
! 5.8. Options for PGP functions
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! Option are named to be similar to GnuPG. Values should be given after
! equal sign, different options from each other with commas. Example:
!
! pgp_sym_encrypt(data, psw, 'compress-also=1, cipher-algo=aes256')
!
! All of the options except `convert-crlf` apply only to encrypt
! functions. Decrypt functions get the parameters from PGP data.
!
! Most interesting options are probably `compression-algo` and
! `unicode-mode`. The rest should have reasonable defaults.
!
!
! cipher-algo::
! What cipher algorithm to use.
!
! Values: bf, aes128, aes192, aes256 (OpenSSL-only: `3des`, `cast5`)
! Default: aes128
! Applies: pgp_sym_encrypt, pgp_pub_encrypt
!
!
! compress-algo::
! Which compression algorithm to use. Needs building with zlib.
!
! Values:
! 0 - no compression
! 1 - ZIP compression
! 2 - ZLIB compression [=ZIP plus meta-data and block-CRC's]
! Default: 0
! Applies: pgp_sym_encrypt, pgp_pub_encrypt
!
! compress-level::
! How much to compress. Bigger level compresses smaller but is slower.
! 0 disables compression.
!
! Values: 0, 1-9
! Default: 6
! Applies: pgp_sym_encrypt, pgp_pub_encrypt
!
! convert-crlf::
! Whether to convert `\n` into `\r\n` when encrypting and `\r\n` to `\n`
! when decrypting. RFC2440 specifies that text data should be stored
! using `\r\n` line-feeds. Use this to get fully RFC-compliant
! behavior.
!
! Values: 0, 1
! Default: 0
! Applies: pgp_sym_encrypt, pgp_pub_encrypt, pgp_sym_decrypt, pgp_pub_decrypt
!
! disable-mdc::
! Do not protect data with SHA-1. Only good reason to use is this
! option is to achieve compatibility with ancient PGP products, as the
! SHA-1 protected packet is from upcoming update to RFC2440. (Currently
! at version RFC2440bis-14.) Recent gnupg.org and pgp.com software
! supports it fine.
!
! Values: 0, 1
! Default: 0
! Applies: pgp_sym_encrypt, pgp_pub_encrypt
!
! enable-session-key::
! Use separate session key. Public-key encryption always uses separate
! session key, this is for symmetric-key encryption, which by default
! uses S2K directly.
!
! Values: 0, 1
! Default: 0
! Applies: pgp_sym_encrypt
!
! s2k-mode::
! Which S2K algorithm to use.
!
! Values:
! 0 - Dangerous! Without salt.
! 1 - With salt but with fixed iteration count.
! 3 - Variable iteration count.
! Default: 3
! Applies: pgp_sym_encrypt
!
! s2k-digest-algo::
! Which digest algorithm to use in S2K calculation.
!
! Values: md5, sha1
! Default: sha1
! Applies: pgp_sym_encrypt
!
! s2k-cipher-algo::
! Which cipher to use for encrypting separate session key.
!
! Values: bf, aes, aes128, aes192, aes256
! Default: same as cipher-algo.
! Applies: pgp_sym_encrypt
!
! unicode-mode::
! Whether to convert textual data from database internal encoding to
! UTF-8 and back. If your database already is UTF-8, no conversion will
! be done, only the data will be tagged as UTF-8. Without this option
! it will not be.
!
! Values: 0, 1
! Default: 0
! Applies: pgp_sym_encrypt, pgp_pub_encrypt
!
!
! 5.9. Generating keys with GnuPG
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! Generate a new key:
!
! gpg --gen-key
!
! You need to pick "DSA and Elgamal" key type, others are sign-only.
!
! List keys:
!
! gpg --list-secret-keys
!
! Export ascii-armored public key:
!
! gpg -a --export KEYID > public.key
!
! Export ascii-armored secret key:
!
! gpg -a --export-secret-keys KEYID > secret.key
!
! You need to use `dearmor()` on them before giving giving them to
! pgp_pub_* functions. Or if you can handle binary data, you can drop
! "-a" from gpg.
!
!
! 5.10. Limitations of PGP code
! ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
!
! - No support for signing. That also means that it is not checked
! whether the encryption subkey belongs to master key.
!
! - No support for RSA keys. Only Elgamal encryption keys are supported
!
! - No support for several encryption subkeys.
!
!
! 6. Raw encryption
! -------------------
!
! Those functions only run a cipher over data, they don't have any advanced
! features of PGP encryption. In addition, they have some major problems:
!
! 1. They use user key directly as cipher key.
! 2. They don't provide any integrity checking, to see
! if the encrypted data was modified.
! 3. They expect that users manage all encryption parameters
! themselves, even IV.
! 4. They don't handle text.
!
! So, with the introduction of PGP encryption, usage of raw
! encryption functions is discouraged.
!
!
! encrypt(data bytea, key bytea, type text) RETURNS bytea
! decrypt(data bytea, key bytea, type text) RETURNS bytea
!
! encrypt_iv(data bytea, key bytea, iv bytea, type text) RETURNS bytea
! decrypt_iv(data bytea, key bytea, iv bytea, type text) RETURNS bytea
!
! Encrypt/decrypt data with cipher, padding data if needed.
!
! `type` parameter description in pseudo-noteup:
!
! algo ['-' mode] ['/pad:' padding]
!
! Supported algorithms:
!
! * `bf` - Blowfish
! * `aes` - AES (Rijndael-128)
!
! Modes:
!
! * `cbc` - next block depends on previous. (default)
! * `ecb` - each block in encrypted separately.
! (for testing only)
!
! Padding:
!
! * `pkcs` - data may be any length (default)
! * `none` - data must be multiple of cipher block size.
!
! IV is initial value for mode, defaults to all zeroes. It is ignored for
! ECB. It is clipped or padded with zeroes if not exactly block size.
!
! So, example:
!
! encrypt(data, 'fooz', 'bf')
!
! is equal to
!
! encrypt(data, 'fooz', 'bf-cbc/padkcs')
!
!
! 7. Credits
! ------------

I have used code from following sources:

! `--------------------`-------------------------`----------------------
! Algorithm Author Source origin
! ----------------------------------------------------------------------
! DES crypt() David Burren and others FreeBSD libcrypt
! MD5 crypt() Poul-Henning Kamp FreeBSD libcrypt
! Blowfish crypt() Solar Designer www.openwall.com
! Blowfish cipher Niels Provos OpenBSD sys/crypto
! Rijndael cipher Brian Gladman OpenBSD sys/crypto
! MD5 and SHA1 WIDE Project KAME kame/sys/crypto
! SHA256/384/512 Aaron D. Gifford OpenBSD sys/crypto
! ----------------------------------------------------------------------


! 8. Legalese
! -------------

+ * I owe a beer to Poul-Henning.
* This product includes software developed by Niels Provos.


+ 9. References/Links
+ ---------------------
+
+ 9.1. Useful reading
+ ~~~~~~~~~~~~~~~~~~~~~
+
+ http://www.openwall.com/crypt/[]::
+ Describes the crypt-blowfish algorithm.
+
+ http://www.stack.nl/~galactus/remail...hrase-faq.html[]::
+ How to choose good password.
+
+ http://world.std.com/~reinhold/diceware.html[]::
+ Interesting idea for picking passwords.
+
+ http://www.interhack.net/people/cmcu...e-oil-faq.html[]::
+ Describes good and bad cryptography.
+
+
+ 9.2. Technical references
+ ~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+ http://www.ietf.org/rfc/rfc2440.txt[]::
+ OpenPGP message format
+
+ http://www.imc.org/draft-ietf-openpgp-rfc2440bis[]::
+ New version of RFC2440.
+
+ http://www.ietf.org/rfc/rfc1321.txt[]::
+ The MD5 Message-Digest Algorithm
+
+ http://www.ietf.org/rfc/rfc2104.txt[]::
+ HMAC: Keyed-Hashing for Message Authentication
+
+ http://www.usenix.org/events/usenix99/provos.html[]::
+ Comparison of crypt-des, crypt-md5 and bcrypt algorithms.
+
+ http://csrc.nist.gov/cryptval/des.htm[]::
+ Standards for DES, 3DES and AES.
+
+ http://en.wikipedia.org/wiki/Fortuna_(PRNG)[]::
+ Description of Fortuna CSPRNG.
+
+ http://jlcooke.ca/random/[]::
+ Jean-Luc Cooke Fortuna-based /dev/random driver for Linux.
+
+ http://www.cs.ut.ee/~helger/crypto/[]::
+ Collection of cryptology pointers.
+

--

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly

Marko Kreen <marko@l-t.ee> writes:
> - Fix couple comments.
> - internal.c didnt clean hash contexts when freeing
> - zero the system randomness buffer too
> - Google tells that we can use /dev/urandom on Cygwin,
> HPUX and AIX. Add them to random.c
> - remove a debug reference from pgp.h

Applied.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Marko Kreen <marko@l-t.ee> writes:
> After studying Fortuna more, I found out that I missed
> some of the details.

> - reseeding should happen only if pool #0 has aquired additional
> entropy.

> - a 'rekeying' operation should happend after each request and
> also after 1M of extracted data. That means taking next two
> blocks and using it as new key.

> - Fortuna _really_ wants entropy sources to be somewhat unpredictible.

> So roll dice when adding it and also add them to pools randomly,
> not sequentially.

> This hopefully makes harder for someone to doctor with the
> internal state (as in our case user can directly control
> what goes into it).

> That also drops the idea of several sources - which really
> fits more to hardware backed event sources.

> - add a really obvious obfuscation: take the absolutely first
> block be initial counter value. If Fortuna (AES to be exact)
> is secure with known counter value, then it should be also
> secure with unknown counter value. This does not go against
> the important property of counter - that the bit-pattern repeat
> period should be as long as possible.

> - S2K functions should use px_get_pseudo_random_bytes not
> px_get_random_bytes.

Applied.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 3: Have you checked our extensive FAQ?

http://www.postgresql.org/docs/faq

Marko Kreen <marko@l-t.ee> writes:
> - reformat in asciidoc syntax
> - clarify various parts
> - add high-level description of PGP structure
> - add a comparison of crypt vs. regular hashes

Applied.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 1: if posting/reading through Usenet, please send an appropriate
subscribe-nomail command to majordomo@postgresql.org so that your
message can get through to the mailing list cleanly