Unix Technical Forum

Pricision of float

This is a discussion on Pricision of float within the SQL Server forums, part of the Microsoft SQL Server category; --> Hi, Just wonder why PRINT CAST(0.0573542567654 AS float) will give the rounded reult 0.0573543 rather than the original number? ...


Go Back   Unix Technical Forum > Database Server Software > Microsoft SQL Server > SQL Server

FAQ Members List Calendar Search Today's Posts Mark Forums Read
  #1 (permalink)  
Old 02-28-2008, 08:24 PM
Benny
 
Posts: n/a
Default Pricision of float

Hi,

Just wonder why

PRINT CAST(0.0573542567654 AS float)

will give the rounded reult

0.0573543

rather than the original number?

"float" should be 'big' enough to hold numbers that have even more
decimal places. How come it round up at the 7 decimal place?

since I need to do some calculations with accumulated values. The
rounded figure will cause significant error after a number of opertions.

Are there any way to work around it?

thanks






*** Sent via Developersdex http://www.developersdex.com ***
Don't just participate in USENET...get rewarded for it!
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #2 (permalink)  
Old 02-28-2008, 08:25 PM
oj
 
Posts: n/a
Default Re: Pricision of float

Float is never exact. You will need decimal or numeric instead.

--
-oj
http://www.rac4sql.net


"Benny" <anonymous@devdex.com> wrote in message
news:3ff9ec52$0$202$75868355@news.frii.net...
> Hi,
>
> Just wonder why
>
> PRINT CAST(0.0573542567654 AS float)
>
> will give the rounded reult
>
> 0.0573543
>
> rather than the original number?
>
> "float" should be 'big' enough to hold numbers that have even more
> decimal places. How come it round up at the 7 decimal place?
>
> since I need to do some calculations with accumulated values. The
> rounded figure will cause significant error after a number of opertions.
>
> Are there any way to work around it?
>
> thanks
>
>
>
>
>
>
> *** Sent via Developersdex http://www.developersdex.com ***
> Don't just participate in USENET...get rewarded for it!



Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
  #3 (permalink)  
Old 02-28-2008, 08:25 PM
Erland Sommarskog
 
Posts: n/a
Default Re: Pricision of float

Benny (anonymous@devdex.com) writes:
> Just wonder why
>
> PRINT CAST(0.0573542567654 AS float)
>
> will give the rounded reult
>
> 0.0573543
>
> rather than the original number?
>
> "float" should be 'big' enough to hold numbers that have even more
> decimal places. How come it round up at the 7 decimal place?
>
> since I need to do some calculations with accumulated values. The
> rounded figure will cause significant error after a number of opertions.
>
> Are there any way to work around it?


PRINT does not work on floats, it works with string data, so there is
an implicit conversion to varchar, and that conversion gives you
seven decimals by default.

Here is one option:

PRINT convert(varchar(30), CAST(0.0573542567654 AS float), 2)

You can also use the str() function to format floats as string. Look
up this function in Books Online.


--
Erland Sommarskog, SQL Server MVP, sommar@algonet.se

Books Online for SQL Server SP3 at
http://www.microsoft.com/sql/techinf...2000/books.asp
Digg this Post!Add Post to del.icio.usBookmark Post in TechnoratiFurl this Post!
Reply With Quote
Reply


Thread Tools
Display Modes

Posting Rules
You may not post new threads
You may not post replies
You may not post attachments
You may not edit your posts

vB code is On
Smilies are On
[IMG] code is On
HTML code is Off
Trackbacks are On
Pingbacks are On
Refbacks are On
Forum Jump


All times are GMT. The time now is 11:28 PM.


Powered by vBulletin® Version 3.6.5
Copyright ©2000 - 2009, Jelsoft Enterprises Ltd.
SEO by vBSEO 3.2.0
www.UnixAdminTalk.com