View Single Post

   
  #1 (permalink)  
Old 04-08-2008, 06:17 PM
gcarter@acm.org
 
Posts: n/a
Default Sybase/TransactSQL calculations

Below are the results of AxBxCxD on 2 records 9609 and 9610 using
Transact-SQL. As you can see for record 9609 the Excel result matches
the Sybase result. However, the results differ for record 9610. Is this
a precision/rounding issue?

Thanks,
GC



RecordNo 9609 9610
A -13.179101 -7.655611
B 1.437439 14.195282
C 0.9998 0.988299
D 200000000 50000000
Excel Results -3,788,072,986.3173 -5,370,098,386.8263
Sybase Results -3,788,072,986.3173 -5,370,099,582.2354
Diff 0.0000 1,195.4091

Reply With Quote