[RESOLVED] Fast and proper way to get the file name and folder part from a full path
In other thread it was raised a discussion about whether using the PathFindFileName API or not to get the file name part from a path.
For not being off topic there, and because the issue seems important enough to me, I open this new thread to discuss different approaches to get the file name part and the folder part given a full path in a String.
I compared two ways, one using the PathFindFileName API and other using InStrRev and found the API to be quite faster.
Module 1 code:
Code:
Option Explicit
Private Declare Function PathFindFileName Lib "shlwapi" Alias "PathFindFileNameW" (ByVal pszPath As Long) As Long '//LPWSTR
Public Function GetFileName1(nFullPath As String) As String
Dim iPtr As Long
iPtr = StrPtr(nFullPath)
GetFileName1 = Mid$(nFullPath, (UnsignedSub(PathFindFileName(iPtr), iPtr)) \ 2 + 1)
End Function
Public Function GetFolder1(nFullPath As String) As String
Dim iPtr As Long
iPtr = StrPtr(nFullPath)
GetFolder1 = Left$(nFullPath, (UnsignedSub(PathFindFileName(iPtr), iPtr)) \ 2)
End Function
Private Function UnsignedSub(ByVal Start As Long, ByVal Decr As Long) As Long
UnsignedSub = ((Start And &H7FFFFFFF) - (Decr And &H7FFFFFFF)) Xor ((Start Xor Decr) And &H80000000)
End Function
Public Function GetFileName2(nFullPath As String) As String
Dim iPos As Long
iPos = InStrRev(nFullPath, "\")
If iPos > 0 Then
GetFileName2 = Mid$(nFullPath, iPos + 1)
Else
GetFileName2 = nFullPath
End If
End Function
Public Function GetFolder2(nFullPath As String) As String
Dim iPos As Long
iPos = InStrRev(nFullPath, "\")
If iPos > 0 Then
GetFolder2 = Left$(nFullPath, iPos)
End If
End Function
Form1 code:
Code:
Option Explicit
Private Sub Command1_Click()
Dim iT1
Dim c As Long
Dim f1 As String
Dim f2 As String
Const cIterations As Long = 10000000
f1 = "c:\tmp\mysub\whatever\myfile.txt"
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName1(f1)
Next
Text1.Text = "GetFileName API: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName2(f1)
Next
Text1.Text = Text1.Text & "GetFileName InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder1(f1)
Next
Text1.Text = Text1.Text & "GetFolder API: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder2(f1)
Next
Text1.Text = Text1.Text & "GetFolder InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
End Sub
Re: Fast and proper way to get the file name and folder part from a full path
Code:
dim fn() as string
fn() = split(fullpath,"/"):filename = fn(ubound(fn()))
Very hacky way to do it, but it'll output the filename to "filename" and you then have the filename's length and can do a LEFT$() for the rest of the full path to get the path...string manipulation tends to be fairly slow so I don't expect it to beat the API, but it's worth adding to your test to see where it scores :-)
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by SmUX2k
Code:
dim fn() as string
fn() = split(fullpath,"/"):filename = fn(ubound(fn()))
Very hacky way to do it, but it'll output the filename to "filename" and you then have the filename's length and can do a LEFT$() for the rest of the full path to get the path...string manipulation tends to be fairly slow so I don't expect it to beat the API, but it's worth adding to your test to see where it scores :-)
Did you time it?
Does it have any advantage regarding speed or correctness?
Re: Fast and proper way to get the file name and folder part from a full path
You might consider:
Code:
Option Explicit
Private Declare Function PathFindFileName Lib "shlwapi" Alias "PathFindFileNameW" ( _
ByVal pPath As Long) As Long
Private Declare Function PathRemoveFileSpec Lib "shlwapi" Alias "PathRemoveFileSpecW" ( _
ByVal pPath As Long) As Long
Private Declare Sub PutMem4 Lib "msvbvm60" (ByVal pVar As Long, ByVal NewVal As Long)
Private Declare Function SysAllocString Lib "oleaut32" (ByVal pCopyFrom As Long) As Long
Private Function PathGetFileName(ByRef Path As String) As String
If Len(Path) > 0 Then
PutMem4 VarPtr(PathGetFileName), SysAllocString(PathFindFileName(StrPtr(Path)))
End If
End Function
Private Function PathGetFolderName(ByVal Path As String) As String
If Len(Path) > 0 Then
PathRemoveFileSpec StrPtr(Path)
PutMem4 VarPtr(PathGetFolderName), SysAllocString(StrPtr(Path))
End If
End Function
Private Sub Main()
Dim P As String
P = "a:\b\c\d e f.txt"
MsgBox PathGetFolderName(P) _
& vbNewLine _
& PathGetFileName(P)
End Sub
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by Eduardo-
Did you time it?
Does it have any advantage regarding speed or correctness?
Timing it would require setting it up in a project to compare it to your other options, and though it would show all results the speed would be different for me with a high end AMD...but I was curious, I expected it to outperform instrrev, and it was pitifully slow (the one method that gives both filename and path took 4x longer than instrrev took to do both separately):-)
And no, doesn't really have any advantages...it'll always output the filename and you'll always know the rest of the string before that is the path, so it's 100% correct, but I suspect they're all as reliable...it's just another way to do it...the folder is easily pulled out by getting the length of the full path and deducting the length of the filename then using LEFT$ to grab the path...it could probably be coded in a way that you have both returned as path/filename from a function, meaning whatever time it takes is divided by two if you're counting both results.
For the record, the code is:
Code:
Public Function GetFolder3(nFullPath As String, outPath As String, outFN As String)
Dim fn() As String
fn() = Split(nFullPath, "/"): outFN = fn(UBound(fn()))
outPath = Left$(nFullPath, Len(nFullPath) - Len(outFN))
End Function
Called with
Code:
GetFolder3 f1, vlin, vlout
I'm sure you don't need the code given that it is far slower, but still...it's there :-P
Re: Fast and proper way to get the file name and folder part from a full path
OK, test program updated:
Module1:
Code:
Option Explicit
Private Declare Function PathFindFileName Lib "shlwapi" Alias "PathFindFileNameW" (ByVal pszPath As Long) As Long
Private Declare Function PathRemoveFileSpec Lib "shlwapi" Alias "PathRemoveFileSpecW" (ByVal pPath As Long) As Long
Private Declare Sub PutMem4 Lib "msvbvm60" (ByVal pVar As Long, ByVal NewVal As Long)
Private Declare Function SysAllocString Lib "oleaut32" (ByVal pCopyFrom As Long) As Long
Public Function GetFileName_API1(nFullPath As String) As String
Dim iPtr As Long
iPtr = StrPtr(nFullPath)
GetFileName_API1 = Mid$(nFullPath, (UnsignedSub(PathFindFileName(iPtr), iPtr)) \ 2 + 1)
End Function
Public Function GetFolder_API1(nFullPath As String) As String
Dim iPtr As Long
iPtr = StrPtr(nFullPath)
GetFolder_API1 = Left$(nFullPath, (UnsignedSub(PathFindFileName(iPtr), iPtr)) \ 2)
End Function
Private Function UnsignedSub(ByVal Start As Long, ByVal Decr As Long) As Long
UnsignedSub = ((Start And &H7FFFFFFF) - (Decr And &H7FFFFFFF)) Xor ((Start Xor Decr) And &H80000000)
End Function
Public Function GetFileName_InStrRev(nFullPath As String) As String
Dim iPos As Long
iPos = InStrRev(nFullPath, "\")
If iPos > 0 Then
GetFileName_InStrRev = Mid$(nFullPath, iPos + 1)
Else
GetFileName_InStrRev = nFullPath
End If
End Function
Public Function GetFolder_InStrRev(nFullPath As String) As String
Dim iPos As Long
iPos = InStrRev(nFullPath, "\")
If iPos > 0 Then
GetFolder_InStrRev = Left$(nFullPath, iPos)
End If
End Function
Public Function GetFileName_SmUX2k(nFullPath As String) As String
Dim fn() As String
fn() = Split(nFullPath, "/")
GetFileName_SmUX2k = fn(UBound(fn()))
End Function
Public Function GetFolder_SmUX2k(nFullPath As String) As String
Dim fn() As String
Dim outFN As String
fn() = Split(nFullPath, "/")
outFN = fn(UBound(fn()))
GetFolder_SmUX2k = Left$(nFullPath, Len(nFullPath) - Len(outFN))
End Function
Public Function GetFileName_dilettante(ByRef Path As String) As String
If Len(Path) > 0 Then
PutMem4 VarPtr(GetFileName_dilettante), SysAllocString(PathFindFileName(StrPtr(Path)))
End If
End Function
Public Function GetFolder_dilettante(ByVal Path As String) As String
If Len(Path) > 0 Then
PathRemoveFileSpec StrPtr(Path)
PutMem4 VarPtr(GetFolder_dilettante), SysAllocString(StrPtr(Path))
End If
End Function
Form1:
Code:
Option Explicit
Private Sub Command1_Click()
Dim iT1
Dim c As Long
Dim f1 As String
Dim f2 As String
Const cIterations As Long = 10000000
f1 = "c:\tmp\mysub\whatever\myfile.txt"
Text1.Text = ""
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_InStrRev(f1)
Next
Text1.Text = Text1.Text & "GetFileName_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_InStrRev(f1)
Next
Text1.Text = Text1.Text & "GetFolder_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_API1(f1)
Next
Text1.Text = Text1.Text & "GetFileName_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_API1(f1)
Next
Text1.Text = Text1.Text & "GetFolder_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFileName_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFolder_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_dilettante(f1)
Next
Text1.Text = Text1.Text & "GetFileName_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFolder_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
End Sub
Result:
I would like to know if there is any advantage in dilettante approach (regarding correctness?) since the functions don't seem to be faster.
Re: Fast and proper way to get the file name and folder part from a full path
A lot of overhead can be removed by using a type library. That can remove quite a few of the calls involved as well as eliminating the cost of calling through Declares.
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by dilettante
A lot of overhead can be removed by using a type library. That can remove quite a few of the calls involved as well as eliminating the cost of calling through Declares.
Hmm, ok, that may be an option, but the idea is to have these functions handy, so that you can easily copy and paste them into any project easily.
Re: Fast and proper way to get the file name and folder part from a full path
Interesting, when I run that code in the IDE on my system the API and INSTRREV methods are almost identical to each other with the API having a very slight advantage. The slower ones on the other hand are painfully slow. both instr and api are in the 2s where the slower methods are in the 20s
Re: Fast and proper way to get the file name and folder part from a full path
Oh, and something I should mention...GetTickCount() API will return the number of milliseconds (like timer, but timer returns the number of seconds since the PC booted while GTK returns the number of milliseconds). Using this, you don't necessarily need to use such a high number of iterations...it's used in exactly the same way as timer, you just need to reference the API and replace "timer" with "GetTickCount()"
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by SmUX2k
Oh, and something I should mention...GetTickCount() API will return the number of milliseconds (like timer, but timer returns the number of seconds since the PC booted while GTK returns the number of milliseconds). Using this, you don't necessarily need to use such a high number of iterations...it's used in exactly the same way as timer, you just need to reference the API and replace "timer" with "GetTickCount()"
I use that because it works good enough for me, and with it I don't need to copy any API declaration.
In fact, I never used GetTickCount (that I recall), maybe it is better because it is what almost anybody else I see uses.
Yes, more accuracy --> less iterations... and more copy paste.
Re: Fast and proper way to get the file name and folder part from a full path
I didn't say you had to use InStrRev I said that's one way to accomplish it. I further noted how the API did it (character walk forward, return last pos). Also, vbspeed.net has alternate implementations of InStrRev that are substantially faster; see also, GetFile implementations.
What's the speed difference of a pure VB6 reimplementation?
Will the API still come out on top vs all the native VB6 alternatives? I'd be curious to know. Of course, even if faster, presumably we'd still be naughty for using them, as dilettante has declared there's one correct way, using the API
Last edited by fafalone; Apr 25th, 2022 at 11:48 PM.
Re: Fast and proper way to get the file name and folder part from a full path
I never declared any such thing. My point is that these shell functions are provided for a reason. Using them makes sense in order to avoid potential errors reinventing the wheel.
Re: Fast and proper way to get the file name and folder part from a full path
I added vbSpeed InStrRev08 as InStrRev2 in the test, performing better than the original InStrRev but not better than the API.
But vbSpeed GetFile05 and GetPath05 perform a bit better than the API.
I also added another function to the comparison pool, GetExtension to get the file extension.
What I don't like of the vbSpeed functions is that they require a class to fire its terminate event, something that cannot be guaranteed while working on the IDE.
Re: Fast and proper way to get the file name and folder part from a full path
My own functions. It looks for both Path-Delimiters: "\" and "/"
No idea about performance.
Note: Done in Office-VBA 64-Bit (Note "LongPtr" and "PtrSafe")
And yes, i'm aware of the StrReverse
Code:
'#########################################################################################################################
'StrCSpn scans Source for the first occurrence of any of the characters that are part of CharSet,
'Returns the length of the initial part of Source not (!!) containing any of the characters that are part of CharSet.
'This is the length of Source if none of the characters in CharSet are found in source.
Public Declare PtrSafe Function StrCSpn Lib "SHLWAPI" Alias "StrCSpnW" (ByVal lpSource As LongPtr, ByVal lpCharSet As LongPtr) As Long
Public Declare PtrSafe Function StrCSpnI Lib "SHLWAPI" Alias "StrCSpnIW" (ByVal lpSource As LongPtr, ByVal lpCharSet As LongPtr) As Long
'#########################################################################################################################
'###############################################################################################
'RPos: find last occurence of Delimiter(s) in Source
'###############################################################################################
Public Function RPos(ByVal Source As String, _
ByVal Delimiters As String, _
Optional ByVal CompareMethod As VbCompareMethod = vbBinaryCompare) As Long
If CompareMethod <> vbDatabaseCompare And Delimiters <> "" Then 'If vbDatabaseCompare or no Delimiters --> Return 0
If CompareMethod = vbBinaryCompare Then 'Case-Sensitive
RPos = Len(Source) - StrCSpn(StrPtr(StrReverse(Source)), StrPtr(Delimiters))
Else
RPos = Len(Source) - StrCSpnI(StrPtr(StrReverse(Source)), StrPtr(Delimiters))
End If
Else
RPos = 0
End If
End Function
Private Function FileNameHelper(ByVal Source As String) As String
FileNameHelper = Mid$(Source, RPos(Source, "/\") + 1)
End Function
'###############################################################################################
'ExtractFilePath: Extract FilePath from Fullpath
'Flag IncludeDel: False --> Delete Delimiter from Result, True --> Don't delete Delimiter from Result
'###############################################################################################
Public Function ExtractFilePath(ByVal Source As String, Optional ByVal IncludeDel As Boolean = True) As String
Dim i As Long
If IncludeDel Then i = 0 Else i = -1
ExtractFilePath = Left$(Source, RPos(Source, "/\") + i)
End Function
'###############################################################################################
'ExtractFileName: Extract Filename from Fullpath with/without Extension
'###############################################################################################
Public Function ExtractFileName(ByVal Source As String, Optional ByVal IncludeExt As Boolean = True) As String
Dim sFileName As String
sFileName = FileNameHelper(Source)
If IncludeExt Then ExtractFileName = sFileName Else ExtractFileName = Left$(sFileName, RPos(sFileName, ".") - 1)
End Function
'###############################################################################################
'ExtractFileExt: Extract Extension from Fullpath
'###############################################################################################
Public Function ExtractFileExt(ByVal Source As String) As String
Dim sFileName As String
sFileName = FileNameHelper(Source)
ExtractFileExt = Mid$(sFileName, RPos(sFileName, ".") + 1)
End Function
Last edited by Zvoni; Tomorrow at 31:69 PM.
----------------------------------------------------------------------------------------
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------------------
People call me crazy because i'm jumping out of perfectly fine airplanes.
---------------------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad
Re: Fast and proper way to get the file name and folder part from a full path
Eh? Not even that bad....
Last edited by Zvoni; Tomorrow at 31:69 PM.
----------------------------------------------------------------------------------------
One System to rule them all, One Code to find them,
One IDE to bring them all, and to the Framework bind them,
in the Land of Redmond, where the Windows lie
---------------------------------------------------------------------------------
People call me crazy because i'm jumping out of perfectly fine airplanes.
---------------------------------------------------------------------------------
Code is like a joke: If you have to explain it, it's bad
Re: Fast and proper way to get the file name and folder part from a full path
also, we need to remember that some comparison when we use 1000+ loops are using memory buffer,
so just the first one is "slow" the other can go fast.
I mean, who needs that many requests for the same string?
so remove
f1 as string
f1 = ****
and add
Code:
Private Function f1() As String
Dim i&
f1 = "C:\"
For i = 1 To 10
f1 = f1 & Chr(65 + Int((Rnd * 25)))
Next i
f1 = f1 & "\"
For i = 1 To 10
f1 = f1 & Chr(65 + Int((Rnd * 25)))
Next i
f1 = f1 & ".TXT"
End Function
change
Const cIterations As Long = 500000
and remember randomize timer
Re: Fast and proper way to get the file name and folder part from a full path
Good point baka about the CPU caching the results.
But now I see that what you are measuring, is mainly the time spent in the f1 function, and that's why you get so close results for all the functions, because the impact of what we actually are trying to measure is minimized in the overall time elapsed there (that is mainly making the paths).
So I propose to make all the paths in advance and to store them in an array.
This is Form1 code:
Code:
Option Explicit
Private mf1(100000) As String
Private Sub Command1_Click()
Dim iT1
Dim c As Long
'Dim f1 As String
Dim f2 As String
Const cIterations As Long = 10000000
'f1 = "c:\tmp\mysub\whatever\myfile.txt"
Init_f1
Text1.Text = ""
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_API1(f1)
Next
Text1.Text = Text1.Text & "GetFileName_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_API1(f1)
Next
Text1.Text = Text1.Text & "GetFolder_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetExtension_API1(f1)
Next
Text1.Text = Text1.Text & "GetExtension_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_InStrRev(f1)
Next
Text1.Text = Text1.Text & "GetFileName_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_InStrRev(f1)
Next
Text1.Text = Text1.Text & "GetFolder_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFileName_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFolder_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_dilettante(f1)
Next
Text1.Text = Text1.Text & "GetFileName_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(f1)
Next
Text1.Text = Text1.Text & "GetFolder_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_InStrRev2(f1)
Next
Text1.Text = Text1.Text & "GetFileName_InStrRev2: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_InStrRev2(f1)
Next
Text1.Text = Text1.Text & "GetFolder_InStrRev2: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_vbSpeed(f1)
Next
Text1.Text = Text1.Text & "GetFileName_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_vbSpeed(f1)
Next
Text1.Text = Text1.Text & "GetFolder_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetExtension_vbSpeed(f1)
Next
Text1.Text = Text1.Text & "GetExtension_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFileName(f1)
Next
Text1.Text = Text1.Text & "ExtractFileName Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFilePath(f1)
Next
Text1.Text = Text1.Text & "ExtractFilePath Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFileExt(f1)
Next
Text1.Text = Text1.Text & "ExtractFileExt Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
End Sub
Private Function f1File() As String
Dim i&
f1File = "C:\"
For i = 1 To 10
f1File = f1File & Chr(65 + Int((Rnd * 25)))
Next i
f1File = f1File & "\"
For i = 1 To 10
f1File = f1File & Chr(65 + Int((Rnd * 25)))
Next i
f1File = f1File & ".TXT"
End Function
Private Sub Init_f1()
Dim c As Long
For c = 0 To UBound(mf1)
mf1(c) = f1File
Next
End Sub
Private Function f1() As String
Static c As Long
c = c + 1
If c > 100000 Then c = 0
f1 = mf1(c)
End Function
My results:
It seems that the original API functions are now on pair with vbSpeed's ones.
Only GetExtension has a substantial difference.
Thank you!
Last edited by Eduardo-; Apr 26th, 2022 at 06:20 AM.
Re: Fast and proper way to get the file name and folder part from a full path
not sure.
I have a "builder", that creates tons of data that is later used by my game. (It creates multiple files, databases, graphic files, sound etc, index stuff, sorting etc, we are talking 100MB+ of data)
the first time I start the builder is slow, but after that its ultrafast. even if I close the program and start it again one hour later.
it seems to cache lots of data that is used again. even a "randomized" array, can be used again.
so I did another try but this time:
1000000 times (so 1/10 of what you use) so even if F1 eat CPU, it should be almost the same time for all the F1 created. as they are all the same function, same string length.
but the files are:
Code:
Const bT$ = "C:\"
Const rT$ = ".TXT"
Private Function f1() As String
Dim i&
f1 = bT
For i = 1 To 2
f1 = f1 & Chr(65 + Int((Rnd * 25)))
Next i
f1 = f1 & "\"
For i = 1 To 2
f1 = f1 & Chr(65 + Int((Rnd * 25)))
Next i
f1 = f1 & rT
End Function
so, if you multiply by 10 you are not getting the same value as you have it.
and I mean for the "bad" ones, like
GetFolder_dilettante: 1,96
that compared to GetExtension_vbSpeed 1,11 is not as much difference as 1,37 <> 9,38 from your result.
Re: Fast and proper way to get the file name and folder part from a full path
Concatenating strings with '&' takes lot of time.
It is the same time for all, yes, but if you are measuring something very fast adding something very slow for all items, the differences (of what you actually want to compare) will of course be minimized.
On the other hand, do you think that the CPU will cache the results of 100,000 randomized items?
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by dilettante
I never declared any such thing. My point is that these shell functions are provided for a reason. Using them makes sense in order to avoid potential errors reinventing the wheel.
Yes you did.
Originally Posted by dilettante
Do what you want, but (A) this is the correct way to do it, and (B) that wasn't the point.
There actually is no risk of failure since LARGEADDRESSAWARE isn't safe for VB6 programs anyway and it should not be used.
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by baka
if u want the strings to be made before calculation, better to create a unique string for all
I don't think that's necessary. More than 10 should do it (I guess). And 100,000 should be far more than enough.
How much would the CPU cache? I think only a couple of registers.
Perhaps someone with knowledge about this issue could clarify it (wqweto, The trick, Niya, others).
On the other hand, the real actual data won't be completely different anyways, they'll repeat at some point too.
I mean, if we are dealing with file paths, some files paths will repeat from time to time.
100,000 different files seems already too randomized compared to what real data will be most of the times IMO.
Re: Fast and proper way to get the file name and folder part from a full path
not sure, but my builder-tool that is doing tons of processing seems to go at least x10 speed the second time I run it.
so if u create a string in a specific region in the memory, it can be accessed again
if u randomize 10.000 strings, the second time they are used they are now accessed faster.
maybe not as fast and maybe not by all functions/API. but if u compare my result with yours theres a difference.
the question is. why if not the cpu is accessing that memory faster.
but looking at the result is not that is totally different from yours.
the fastest seems to be the same here, but the differences in some of the result is not as big.
here we have: x4,5 times slower. so the 28 in length is not the answer for the difference.
why do we have this difference?
I think this is very important.
since we have done many test in the past, comparing different methods.
but if the caching is there, calling the memory faster "if" they are in the memory region and if called a second time. it could ruin the comparison.
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by baka
not sure, but my builder-tool that is doing tons of processing seems to go at least x10 speed the second time I run it.
There must be something else.
Originally Posted by baka
so if u create a string in a specific region in the memory, it can be accessed again
if u randomize 10.000 strings, the second time they are used they are now accessed faster.
If they are accessed faster, better, because we don't want to measure the time spent in accessing the strings in memory, but the time spent in the functions that we are comparing.
Originally Posted by baka
maybe not as fast and maybe not by all functions/API. but if u compare my result with yours theres a difference.
I didn't understand that.
Originally Posted by baka
the question is. why if not the cpu is accessing that memory faster.
I repeat: it doesn't matter. If the memory was accessed with no time at all, it would be ideal, because we are not trying to measure that. They are just "parasitic" times that are involved.
The problem of having a single, constant, never changing string was that the compiler made some short-circuiting for optimization (or something like that). So, with just a few different elements it should be enough IMO.
as u can see. 4,5-5 times slower is GetFolder_dilettante in my run, while 7,62 times slower in your run, and if we compare the "old fixed string" we have 12,86 times slower.
the 3 methods are using Init_f1
mine is creating 1 string for each call, so I have 17.000.000 strings created before I run the loops
while yours is only creating 100.000 different strings that are used sequentially
with 10.000.000 x 17 loops, each unique string will be used 1.700 times
I do not call any "function" in the loop to get the "string" instead I have the string in the array,
that could also increase speed a bit. that could be the reason yours is 50% slower in comparison?
and if we compare the old fixed string, its 150% slower in comparison.
as u can see. 4,5-5 times slower is GetFolder_dilettante in my run, while 7,62 times slower in your run, and if we compare the "old fixed string" we have 12,86 times slower.
the 3 methods are using Init_f1
mine is creating 1 string for each call, so I have 17.000.000 strings created before I run the loops
while yours is only creating 100.000 different strings that are used sequentially
with 10.000.000 x 17 loops, each unique string will be used 1.700 times
I do not call any "function" in the loop to get the "string" instead I have the string in the array,
that could also increase speed a bit. that could be the reason yours is 50% slower in comparison?
and if we compare the old fixed string, its 150% slower in comparison.
I didn't understand one single thing.
Please post the project to be able to run it, see what it is doing and come to a conclusion what it means.
Re: Fast and proper way to get the file name and folder part from a full path
its a comparison between GetFolder_dilettante and GetExtension_vbSpeed
for all 4 methods.
so if it takes 1 minute to run GetExtension_vbSpeed it takes 5 minutes to run using GetFolder_dilettante using 12 len string method.
thats what Im comparing.
and as u see. the 4 methods show different comparison. it should not be that huge of a difference.
just add:
this is the 28 string len (since I don't have enough memory I can only do 500.000 loops with 28 len, in 12 len I can increase to 1.000.000 loops)
Code:
Option Explicit
Const cIterations As Long = 500000
Private mf1(1 To 17, 1 To cIterations) As String * 28
Private Sub Command1_Click()
Dim iT1
Dim c As Long
Dim f2 As String
Init_f1
Text1.Text = ""
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_API1(mf1(1, c))
Next
Text1.Text = Text1.Text & "GetFileName_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_API1(mf1(2, c))
Next
Text1.Text = Text1.Text & "GetFolder_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetExtension_API1(mf1(3, c))
Next
Text1.Text = Text1.Text & "GetExtension_API1: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_InStrRev(mf1(4, c))
Next
Text1.Text = Text1.Text & "GetFileName_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_InStrRev(mf1(5, c))
Next
Text1.Text = Text1.Text & "GetFolder_InStrRev: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_SmUX2k(mf1(6, c))
Next
Text1.Text = Text1.Text & "GetFileName_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(mf1(7, c))
Next
Text1.Text = Text1.Text & "GetFolder_SmUX2k: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_dilettante(mf1(8, c))
Next
Text1.Text = Text1.Text & "GetFileName_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_SmUX2k(mf1(9, c))
Next
Text1.Text = Text1.Text & "GetFolder_dilettante: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_InStrRev2(mf1(10, c))
Next
Text1.Text = Text1.Text & "GetFileName_InStrRev2: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_InStrRev2(mf1(11, c))
Next
Text1.Text = Text1.Text & "GetFolder_InStrRev2: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFileName_vbSpeed(mf1(12, c))
Next
Text1.Text = Text1.Text & "GetFileName_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetFolder_vbSpeed(mf1(13, c))
Next
Text1.Text = Text1.Text & "GetFolder_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = GetExtension_vbSpeed(mf1(14, c))
Next
Text1.Text = Text1.Text & "GetExtension_vbSpeed: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFileName(mf1(15, c))
Next
Text1.Text = Text1.Text & "ExtractFileName Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFilePath(mf1(16, c))
Next
Text1.Text = Text1.Text & "ExtractFilePath Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
iT1 = Timer
For c = 1 To cIterations
f2 = ExtractFileExt(mf1(17, c))
Next
Text1.Text = Text1.Text & "ExtractFileExt Zvoni: " & Round(Timer - iT1, 2) & vbCrLf
End Sub
Private Function f1File() As String
Dim i&
f1File = "C:\"
For i = 1 To 10
f1File = f1File & Chr(65 + Int((Rnd * 25)))
Next i
f1File = f1File & "\"
For i = 1 To 10
f1File = f1File & Chr(65 + Int((Rnd * 25)))
Next i
f1File = f1File & ".TXT"
End Function
Private Sub Init_f1()
Dim c As Long
Dim d As Long
For c = 1 To cIterations
For d = 1 To 17
mf1(d, c) = f1File
Next d
Next
End Sub
its a factor of 5,77
the other test we have factors: 5 and 4,5 and 7,62 and 12,86
do you understand?
how reliable are the test?
in your first try, with one string (that all the function will use) we get a factor of 12,86
but if all strings are unique we get a factor of 4.5 - 5,7.
that means, using one string will make GetExtension_vbSpeed seems to be 12 times better.
but using unique strings its around 5 times better.
this means that theres a cache going on, the cpu is remembering the string and in some cases making the function faster.
but not for all types of functions/apis.
since in a real situation we are not looping millions of times the same string, we need a more reliable comparison.
thats why I wrote about using unique strings to see how all the methods works.
Re: Fast and proper way to get the file name and folder part from a full path
Most of us already have a type library registered on dev machines to do this, so:
Code:
Option Explicit
'Requires a reference to:
'
' Edanmo's OLE interfaces & functions v1.81 (olelib.tlb) OR EQUIVALENT.
Private Function PathGetFileName(ByRef Path As String) As String
PathGetFileName = SysAllocString(PathFindFileName(Path))
End Function
Private Function PathGetFolderName(ByVal Path As String) As String
PathRemoveFileSpec Path
PathGetFolderName = SysAllocString(StrPtr(Path))
End Function
Private Sub Main()
Dim P As String
P = "a:\b\c\d.txt"
MsgBox PathGetFolderName(P) _
& vbNewLine _
& PathGetFileName(P)
End Sub
And of course there is nothing to deploy so the cost is nothing and both operations should run a little faster. I'm not sure why there is an obsession over speed for this though anyway. A need for speed here should be pretty rare.
Yes, that one (8,88 from GetFolder_dilettante) is not very fast.
But what was the point with bringing those numbers, since we were already clear that using a single fixed string was a mistake, and we already had agreed not to do that anymore?
The discussion was whether to have 100,000 different items were enough or not, or if 10 different items were already enough.
At least that was what I had in mind.
And I'll forget about options that are already clear that are slower, I would not include them anymore in the tests if they are source of confusion.
The only ones that seems to be in race are the API1 and vbSpeed ones.
They are the ones highlighted in the image of post #20.
Re: Fast and proper way to get the file name and folder part from a full path
Originally Posted by dilettante
And of course there is nothing to deploy so the cost is nothing and both operations should run a little faster.
I don't have it (registered).
Originally Posted by dilettante
I'm not sure why there is an obsession over speed for this though anyway.
Did you see any "obsession"?
Originally Posted by dilettante
A need for speed here should be pretty rare.
Yes, it is rare. I never cared/needed for the speed on these functions until now.
But since we are already discussing it, should we stop?
I opened this thread for two things: (1) "Fast" and (2) "proper".
I hoped that perhaps you had some answer for "proper".
But it seems that all proposed functions return proper results, and they will do it in the future since it is very unlikely that Windows will change the character for directory separation ("\") or extension separation (".").
I also tested mines with paths over MAX_PATH and they worked. What else could be not correct?
In the Windows API (with some exceptions discussed in the following paragraphs), the maximum length for a path is MAX_PATH, which is defined as 260 characters.
The Windows API has many functions that also have Unicode versions to permit an extended-length path for a maximum total path length of 32,767 characters.