precision question...

In my sp (sql server) Im selecting a value like this:

SELECT ROUND(WHATEVER,3) AS WHATEVER FROM MYTABLE

I want the value rounded with 3 decimals.

I execute the SP and in the returned dataset I get the value, and there it is something like "3,000,000" Even though I stated that I want 3 decimals only. I guess it has something to do with the dbtype of the column in the dataset, but where should I then perform the round?

I have another problem!!!!! In dev studio, I often get this message:

Quote:

---

An unhandled exception of type 'System.OutOfMemoryException' occurred in system.drawing.dll

Additional information: Out of memory.

---

kind regards
henrik

When you Round a number you are still dealing with the original properties of the original data type. That is to say if your column was a Decimal(8,4) 1234.5678 after you perform the round function you will still have the same datatype--decimal(8,4)

so

Code:

---

round(1234.5678 , 3)

---

will yield 1234.5680

you have to cast into a new datatype if you do not want the trailing '0' at the end like this

Code:

---

cast(round(1234.5678,3) as decimal(7,3))

---

this yields 1234.568

Hope that helps

Tal McMahon
Competitive Egde Software Inc.
Makers Of Report Exec© Enterprise