Unix Technical Forum

Floating point problem?

This is a discussion on Floating point problem? within the pgsql Bugs forums, part of the PostgreSQL category; --> This has become more of a prominent problem as we've added more data integrity checks in our code... We ...


Go Back   Unix Technical Forum > Database Server Software > PostgreSQL > pgsql Bugs

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 04-10-2008, 10:05 AM
Chito Lapena
 
Posts: n/a
Default Floating point problem?

This has become more of a prominent problem as we've added more data
integrity checks in our code... We are getting errors like the following
from our debug output:

EMP ID: 3 Ledger balance mismatch - record: 116.1 Ledger: 116.1 diff:
1.52588e-05

record: float4
ledger: float4

diff = ledger - record

This is causing a large number of our commit transactions to abort
because we are basically checking the balance record of a ledger (in one
table) VS the last balance in that ledger (in another table) and it is
being flagged as not equal with the infinitesimal difference. We have
tried typecasting and that is not working. We are going to kludge this
to work by replacing the 'is equal to' operator with a subtract
operation compared with a small number. Needless to say, we would rather
not do it this way.

These are 2 servers that are having the same problem

Linux ims.rx.uga.edu 2.6.12-1.1381_FC3smp #1 SMP Fri Oct 21 04:03:26 EDT
2005 i686 i686 i386 GNU/Linux

Linux leave.forestry.uga.edu 2.6.9-1.667 #1 Tue Nov 2 14:41:25 EST 2004
i686 i686 i386 GNU/Linux

thanks for your attention to this

-Chito Lapena
webims.com
912-695-8076

---------------------------(end of broadcast)---------------------------
TIP 4: Have you searched our list archives?

http://archives.postgresql.org

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 04-10-2008, 10:05 AM
Tom Lane
 
Posts: n/a
Default Re: Floating point problem?

Chito Lapena <chito@webims.com> writes:
> This has become more of a prominent problem as we've added more data
> integrity checks in our code... We are getting errors like the following
> from our debug output:


> EMP ID: 3 Ledger balance mismatch - record: 116.1 Ledger: 116.1 diff:
> 1.52588e-05


> record: float4
> ledger: float4


If you are expecting exact results, you should be using an exact
datatype, ie numeric. Floating-point calculations are inherently
inexact ... *especially* with float4, which only has about six
significant digits at best.

regards, tom lane

---------------------------(end of broadcast)---------------------------
TIP 2: Don't 'kill -9' the postmaster

Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 09:37 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2008, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com