[RESOLVED] How to fast split Number integet part and decimal part ?

[dilettante]

If we knew what the goal was we might be able to suggest alternatives.

I don't see what trying to divide a Single, Double, etc. into two Long values might be useful for... especially if you just format those as String values afterward. Perhaps you can enlighten us?

[DEXWERX]

If we knew what the goal was we might be able to suggest alternatives.

I don't see what trying to divide a Single, Double, etc. into two Long values might be useful for... especially if you just format those as String values afterward. Perhaps you can enlighten us?

Nope Sorry... we have to go through all possible ways of doing things the wrong way first.

Now I decided to accelerate it with multi cpu core.

Now that would be impressive! Make sure you check out Krool's simple multithreading component, to help you on your journey.
And post the code back here.

You can find the DLL in the codebank here --> http://www.vbforums.com/showthread.p...ultithreading)

[Schmidt]

...you say your Data is very huge, it would be helpful to know where the Data comes from and in what Format
why do you want to split the number anyway ?

My guess is, that he has the (Double-based) Data in memory (in an UDT-Array or something) -
and then simply wants to export into a TextFile "as fast as possible" (e.g. writing his real-numbers in CSV-format).

@quickbbbb
You should be more forthcoming with background-information about your problem(s) in the future -
since most "seemingly complicated" tasks are quite "wellknown ones" (for long-term regulars in a community-forum -
approaches to solve these problems efficiently, already existing for a long time - no need for "IP-protectiveness").


If my guess is right, then the Problem could be expressed (in code) this way:

Private Type t2Cols 'a "two Double-Columns" Test-UDT-def
  Col1 As Double
  Col2 As Double
End Type

Const TestCount& = 10 ^ 6 '1Mio "Test-Row-Entries"

Private Arr2Cols(1 To TestCount) As t2Cols 'allocate an appropriate 2-Col-Array
 
Private Sub Form_Load()
  Rnd -1  '... ensure, that the array is filled with always the same random content
  Dim i As Long
  For i = 1 To UBound(Arr2Cols)
    Arr2Cols(i).Col1 = Rnd * 1000
    Arr2Cols(i).Col2 = Rnd * 1000
  Next
End Sub

Private Sub Form_Click()
  Dim TT!, i&, FNr&
  TT = Timer
    FNr = FreeFile
    Open "C:\Temp\Export.txt" For Output As FNr
      For i = 1 To TestCount
        Print #FNr, Str$(Arr2Cols(i).Col1) & "," & Str$(Arr2Cols(i).Col2)
      Next
    Close FNr
  Caption = Timer - TT
End Sub

The above Test (when you click the Form) will need about 2.5 seconds,
to produce an about 35MB large CSV-export-file (with two columns and 1Mio Lines).

And no, the problem is indeed not "IO-bound" ...
Meaning, that writing 35MB of FileContent should not take longer than:
- about 0.1sec with an SSD
- about 0.3-0.5sec when writing to a rotating DiskDrive

In my tests (against an SSD) I get a timing of 2.2sec total.
So, 2.2sec total minus 0.1sec SSD-FileIO-time => ... 2.1 seconds, which are "burned in String-Conversion-efforts".

So the OP is right in thinking, that splitting a Double into "Long-Integer-Parts" could speed things up significantly.
In my tests, using a Double-to-LongParts-conversion-routine, I was able to reduce those 2.1 seconds to 0.2seconds,
ending up with a total-time to produce the CSV-file of: 0.3seconds.

Olaf

[LaVolpe]

@Olaf. Change Str$() to CStr$() and see a reduction in time from ~2.5 to ~1.6 seconds.
Also, writing the UDT members in binary mode (Put vs Print), was actually slower. Just FYI

As far as converting double to 2 longs being significantly faster to print... If you are suggesting that writing 8 bytes (2 Longs) vs 16 bytes (2 doubles) to file is faster, no one is going to disagree with you

[Schmidt]

@Olaf. Change Str$() to CStr$() and see a reduction in time from ~2.5 to ~1.6 seconds.
Also, writing the UDT members in binary mode (Put vs Print), was actually slower. Just FYI

When the task is, to write a *standardized* CSV-Format (which defines Commas as Col-separators and "dots" as decimal-points)
then CStr is not really suitable as a Double-to-Text-conversion-routine.

Please look at my "Problem-Description-CodeSnippet" only as a "first naive attempt"
(producing correct CSV, independent of a Users locale - but with "room for optimization").

And as said, using math-ops to "break-up" a Double-value into Integer-parts beforehand,
can help to bring down the over-all-timing to about 0.3seconds for producing said CSV-file...

Olaf

[LaVolpe]

I think the OP has "resolved" this thread but hasn't marked it resolved (see last post of previous page).

But bottom line, the OP appears to be using doubles when currency is likely a better variable type, mentioned by Elroy. And, I also think that separating the double to 2 longs is honestly a scenario of forcing a square block into a round hole because it appears the OP's intent is to use his "fast CStr() replacement" function which may only work with Longs. Assumption based on previous posts. Maybe OP should look at a fast replacement for Currency?

With using doubles, extracting the fractional value using simple subtraction can result in a different value than expected. As shown with: 12345.6789# - 12345# = 0.678900000000795, not 0.6789.

Converting such a value to two Longs isn't a problem. But some hint needs to be understood to determine how to interpret the fractional part, i.e., multiplication factor. Otherwise, how would a fractional part (0.025) be stored as a long and keeping the leading zero. That hint would then be able to convert a Long value like 25 to its fractional true value of 0.025. Even LSET could be used to convert Double or Currency to 2 Longs and back to original vartype. But the Longs may not look like the value they represent until converted back.

[Schmidt]

... bottom line, the OP appears to be using doubles when currency is likely a better variable type, ...

Not really, because sometimes precision of more than 4 decimal-places is needed -
and the problem of fast TextFile-Output (fast conversion of "fractional numbers" to text) is still unsolved
(when we stick to the problem I've opened up in my post #43).

I also think that separating the double to 2 longs is honestly a scenario of forcing a square block into a round hole...

With using doubles, extracting the fractional value using simple subtraction can result in a different value than expected.
As shown with: 12345.6789# - 12345# = 0.678900000000795, not 0.6789.

Converting such a value to two Longs isn't a problem.
But some hint needs to be understood to determine how to interpret the fractional part,...

Of course - a hint is needed ... so what about solving both problems with the following "Split-approach" (offering max. 9 digits of rounded precision):

Private Sub SplitDblParts(ByVal Dbl#, IntPart&, DecPart&, DecLen&)
  If Dbl < 0 Then Dbl = -Dbl 'treat neg. Values the same as pos. Values here
  IntPart = Fix(Dbl): DecPart = (Dbl - IntPart) * 1000000000# 'make DecPart an Int with 9 Digits
  If DecPart = 0 Then
     DecLen = 1
  ElseIf DecPart > 999999999 Then
     DecLen = 1: DecPart = 0: IntPart = IntPart + 1
  Else
     DecLen = 9
     Dim i As Long
     For i = 1 To 8
       If DecPart Mod 10 Then Exit For Else DecPart = DecPart \ 10: DecLen = DecLen - 1
     Next
  End If
End Sub

The above routine is acting as the workhorse in my approach, to write the CSV-file in 0.27seconds (instead of 2.2secs, when we'd use the naive, first attempt in #43).

Olaf

[LaVolpe]

If you are going to save customized long values (including the DecLen part), why not use a simple UDT and LSET? I know I must be missing something in the problem you are addressing.

For example:

Private Type LongLong
    hiPart As Long
    loPart As Long
End Type
Private Type DblDbl
    Value As Double
End Type

Private Sub SplitDblParts2(ByVal Dbl#, IntPart&, DecPart&)

    Dim udtDbl As DblDbl
    Dim udtLng As LongLong
    
    udtDbl.Value = Dbl
    LSet udtLng = udtDbl
    
    IntPart = udtLng.loPart
    DecPart = udtLng.hiPart
End Sub

Above can be called as shown below and easily restored by reading the loPart & hiPart and using LSet to get the Double. No restriction for the number of decimal places, handles negative values, and not restricted to 2GB Long value. Also, the above is more than 2x faster than your posted SplitDblParts. On my older PC, 1 million iterations, .109 seconds vs .25 in the IDE.

    Dim udtDbl As DblDbl
    Dim udtLng As LongLong
    
    SplitDblParts2 12345.6789, udtLng.loPart, udtLng.hiPart

    LSet udtDbl = udtLng

[Schmidt]

...I know I must be missing something in the problem you are addressing...

I've described the problem (which I think is similar to the one the OP has) in #43.

E.g. when your Customer demands that you export "a bunch of large Files" (let's say 10), each containing about "1Mio Rows of two fractional Numbers" in a two-column CSV-format.
So, you exporting Double-Values in a TextFormat is mandatory (part of the Customers "interface for an established workflow"...

Now, if you stick to the naive approach (as outlined in #43), then producing (writing) these 10 large CSV-Files -
will take up about 25 seconds.

What I also aready mentioned is, that it is possible to write these 10 Files in 3 seconds total instead.

That's (IMO) the speedup the OP is looking for.

Olaf

[LaVolpe]

Well, I'm not even sure a CSV is even required. The OP may have mentioned that and I missed it. What I did recall was that he was looking for 2 Longs from 1 Double. I'm not even sure he was addressing leading zeros in the fractional portion if a Long would be used. If wanting 2 Longs is an absolute requirement, then using LSet option may be a suitable solution. Won't look like the actual Double value when written to file, but will any pair of Longs? Instead of his original post of using CDec(), storing 1 Long & 1 Double, storing 2 customized Longs might be better, regardless what those Longs look like. At least it meets the 2 Long values requirement.

Edited: A problem of using LSet as shown in previous reply is that more characters will be written to file. For example, instead of writing 12345.6789 (10 characters), this will be written: 1086856534,-432932703 (21 characters including comma). Therefore, the file will be larger and should take longer to write. Whereas your solution should write less than 21 characters even if the DecLen value is written and the difference in increased time of creating your customized longs could be offset with the shorter time it takes to write a smaller file? Honestly, I just find it odd to want to write 2+ Long values, as text, vs. writing just 1 Double/Currency value as text.

[Schmidt]

Well, I'm not even sure a CSV is even required.

It was just an example, to underline that (in the end) a fast text-conversion of a fractional number is needed (IMO).

The OP may have mentioned that and I missed it.

Yep, I think that is the case because in #20 he wrote:

my data amount very huge , i need a function faster than cstr()

vba.Cstr convert long type to String = 4 second
My.Cstr convert long type to String = 2 second...

Above describing, what he already has accomplished, when converting Integer-Values into Strings.

the above was then followed by:

...now i need let My.Cstr can convert float type to String faster than vba.Cstr

...describing his intent, to write a faster replacment for VBA.CStr also for "fractional numbers" (float-types).

And that requires (beside the Split around the decimal-point) as he and you have already acknowledged, an additional "hint"-Value (because of potentially leading Zeros):

...so i need know decimal part length

So in my opinion, it is definitely about "Float-to-Text"-conversion (faster than VBA.CStr - probably in a locale-invariant version).

If you want to avoid thinking about the FileSystem (forgetting about CSV) -
just imagine a WebServer, which has to send e.g. a larger array of Double-Values in Text-Format back to the Client -
(e.g. serialized into UTF8-JSON-text-format beforehand) ... here a factor 10 better response-time would be a desired outcome as well.

Olaf

[quickbbbb]

Private Sub SplitDblParts(ByVal Dbl#, IntPart&, DecPart&, DecLen&)
  If Dbl < 0 Then Dbl = -Dbl 'treat neg. Values the same as pos. Values here

Fabulous! thank you very much

========================

sorry every one, my english is poor , can not has full expression and understand what you say

Most of them are used almost Google translation put together for read and write