This is a discussion on Floating point problem? within the pgsql Bugs forums, part of the PostgreSQL category; --> This has become more of a prominent problem as we've added more data integrity checks in our code... We ...
| |||||||
| FAQ | Members List | Calendar | Search | Today's Posts | Mark Forums Read |
| ||||
| This has become more of a prominent problem as we've added more data integrity checks in our code... We are getting errors like the following from our debug output: EMP ID: 3 Ledger balance mismatch - record: 116.1 Ledger: 116.1 diff: 1.52588e-05 record: float4 ledger: float4 diff = ledger - record This is causing a large number of our commit transactions to abort because we are basically checking the balance record of a ledger (in one table) VS the last balance in that ledger (in another table) and it is being flagged as not equal with the infinitesimal difference. We have tried typecasting and that is not working. We are going to kludge this to work by replacing the 'is equal to' operator with a subtract operation compared with a small number. Needless to say, we would rather not do it this way. These are 2 servers that are having the same problem Linux ims.rx.uga.edu 2.6.12-1.1381_FC3smp #1 SMP Fri Oct 21 04:03:26 EDT 2005 i686 i686 i386 GNU/Linux Linux leave.forestry.uga.edu 2.6.9-1.667 #1 Tue Nov 2 14:41:25 EST 2004 i686 i686 i386 GNU/Linux thanks for your attention to this -Chito Lapena webims.com 912-695-8076 ---------------------------(end of broadcast)--------------------------- TIP 4: Have you searched our list archives? http://archives.postgresql.org |
| ||||
| Chito Lapena <chito@webims.com> writes: > This has become more of a prominent problem as we've added more data > integrity checks in our code... We are getting errors like the following > from our debug output: > EMP ID: 3 Ledger balance mismatch - record: 116.1 Ledger: 116.1 diff: > 1.52588e-05 > record: float4 > ledger: float4 If you are expecting exact results, you should be using an exact datatype, ie numeric. Floating-point calculations are inherently inexact ... *especially* with float4, which only has about six significant digits at best. regards, tom lane ---------------------------(end of broadcast)--------------------------- TIP 2: Don't 'kill -9' the postmaster |