Results 1 to 2 of 2

Thread: Help with Calculations

  1. Mar 22nd, 2012, 02:41 AM #1
    vuyiswamb
    vuyiswamb is offline

    Thread Starter
    Fanatic Member vuyiswamb's Avatar
    Join Date
    Jan 2007
    Location
    South Africa
    Posts
    830

    Help with Calculations

    I need your help . i have a spreadsheet In the Spread Sheet in the First line there is a field “Unit Type Total PQ”.

    That field is calculated from =F2*D2 and the value of F2 = 0.05 and D2 = 12 so Excel reports that the Field “Unit Type Total PQ” results into 0.56.

    If I take a CALCULATOR and multiply 0.05 * 12 I get 0.6

    And if I go to SQL I get with this

    Code:
    SELECT  CONVERT(DECIMAL(10,2),0.05) * 12
    I get 0.60

    Why is Excell giving me Incorrect values or how can i fix my SQL
    Reply With Quote Reply With Quote
  2. Mar 22nd, 2012, 04:25 AM #2
    abhijit
    abhijit is offline
    PowerPoster abhijit's Avatar
    Join Date
    Jun 1999
    Location
    Chit Chat Forum.
    Posts
    3,228

    Re: Help with Calculations

    I suspect that is being caused in Excel due to formatting and rounding of decimal places. The 0.05 you see could be a result of rounding. You need to see that cell upto 4 or 6 decimal places to determine if that value is 0.05 or is it something else.
    Everything that has a computer in will fail. Everything in your life, from a watch to a car to, you know, a radio, to an iPhone, it will fail if it has a computer in it. They should kill the people who made those things.- 'Woz'
    save a blobFileStreamDataTable To Text Filemy blog
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules


Click Here to Expand Forum to Full Width