Loss Of Precision

[dsheller]

Precision of Vectors:

I have a number that i'm trying to pass to my MakeVector function, which accepts a single as the datatype. The number is 22201.97376. So I pass that number along to makevector but the problem is it chops it down to two decimal places... and I need those extra points of precision to make things work. Is there anything I can do? I can't change the data type because eventually it ends up as a single, when it is stored in a D3DVector.

[Yhoko]

Show that function pls?

[dsheller]

Public Function MakeVector(x As Single, y As Single, z As Single) As D3DVECTOR
    With MakeVector
        .x = x
        .y = y
        .z = z
    End With
    
End Function

Then I use it like so:
Mesh.Move MakeVector( 22201.97376, 0, -5 )

[Jacob Roman]

Where did you get the idea that it chops it down to two decimal places though? Single is sufficient enough. Cause if you use Doubles, DX is gonna convert it to Single anyways, and might slow your program down. However in other cases, Double is ok. Such as for collision when Single isn't accurate enough.

[dsheller]

I got the idea from when I stepped through the code... when this line:
Mesh.Move MakeVector( 22201.97376, 0, -5 )

is called it goes in to the MakeVector function which reports the x variables as 22201.97, so it obviously loses 3 digits of precision.

[Jacob Roman]

Hmmmm, that's funny. I did an experiment, and no matter what I did, it always turned into 22201.97. I guess VB converts it automatically.

[dsheller]

Yea... I have no idea what to do about it =/

[wossname]

Stab in the dark but have you tried this?

Mesh.Move MakeVector( 22201.97376F, 0, -5 )

[dsheller]

Appending F doesn't do anything, just a compile error.

[Jacob Roman]

Try # instead. It might work.

[NoteMe]

What D3DVECTOR vector are you using. The one in DX? If so, it is float by default. Your GPU is using floats. It is made to be fast not for precission. So it is nothing you can do about it except making your own struct/class....

[dsheller]

Well all the functions expect a D3DVector so how would I go about implementing that note?

[NoteMe]

I am not sure what you are doing here, but usualy it is not nesesary in a game. If it is for you, then the other functions have to be rewritten too. Remember that the drawing does never have to be as precise as the calculation. So you can have two sets of coordinates and vectors. One for drawing, and one for collision detection or what ever...