[RESOLVED] Using DIR function with multiple specific file types

[SomeYguy]

Hi everyone,

I've been all over the web and also experimenting on my own and cannot find an answer to this.

One of my programs uses a hidden VB FileListBox for filename loading and extraction. Works fine except for the upper limit of 32767 items in the FileListBox. I've been experimenting with using the DIR function in order to eliminate the hidden FileListBox and to streamline code. So far so good.

But here's the problem - I can find no way to use multiple specific file types - i.e. multiple wildcard patterns such as "*.bmp;*.jpg;*.txt" etc.

If using just one file type then all is well but not multiple types. Here is my code (Hope I post it correctly, its my first time):

Private Sub ListFiles(strPath As String, Optional Extension As String)
    
    '- This is a good alternative to using a hidden FileListBox to list files.
    '- BUT There is (as always) a caveat. In this case it is that multiple
    '- file patterns are not accepted.
    
    Dim File As String
    Dim fCount As Long
    Dim p As Long
    
    Erase strFiles
    
    
    If Right$(strPath, 1) <> "" Then strPath = strPath & ""
    
    If Trim$(Extension) = "" Then
        Extension = "*.*"
    ElseIf Left$(Extension, 2) <> "*." Then
        Extension = "*." & Extension
    End If
    
    File = Dir$(strPath & Extension)
    
    Do While Len(File)
        fCount = fCount + 1
        ReDim Preserve strFiles(fCount)
        strFiles(fCount) = File
        File = Dir$
    Loop
   '  .....Etc.
    
    
    
End Sub

Any help is appreciated, thank you

[jdc2000]

How many wild card patterns do you think you may need to match - a reasonable number (few) or hundreds +? If just a few, create a string array with a list of them and then add another loop to your code to execute it for each successive wild card value.

[SomeYguy]

Only 8-10 file types for now but maybe more later.

I thought about doing as you suggest, but it might raise a performance hit. But, it may also be the only way......

[SomeYguy]

Oops, hit post button twice

[wqweto]

I thought about doing as you suggest, but it might raise a performance hit.

The bottleneck when enumerating files/folders is *always* disk access. When enumerating *.bmp wilcard for instance, the OS has to read the contents of the whole directory so this is *not* faster than enumerating everything w/ *.*

In your case I would consider full enumeration from OS (i.e. *.* + vbDirectory) and then filter "client-side" w/ VB's Like operator in a loop. It's not worth it performance-wise to prepare a single regexp from an array of mask wilcards -- something like /pattern1|pattern2|pattern2|.../ -- as this is error prone and you have to test very well file name escapes not to leak control symbols.

cheers,
</wqw

[dilettante]

There is also the option of a query against the Windows Search database.

[wqweto]

Yes, and if 32k entries limit is expected to be reached then OP might try a direct MFT parsing approach on an NTFS volume too (although its greatest strength shows for recursive enumerations).

cheers,
</wqw>

[fafalone]

The PathMatchSpec API supports multiple filters, passed as semicolon delimited e.g. *.jpg;*.gif;*.bmp etc., I think that's a much better option than the Like statement when you're going beyond a single pattern.

Private Declare Function PathMatchSpecW Lib "shlwapi" (ByVal pszFileParam As Long, ByVal pszSpec As Long) As Long

Private Function ShowFile(sName As String, sFilter As String) As Boolean
If PathMatchSpecW(StrPtr(sName), StrPtr(sFilter)) Then
    ShowFile = True
Else
    ShowFile = False
End If
End Function

Note you pass just the filename, not the full path. The code above supports Unicode.

[wqweto]

PathMatchSpecW seems like a perfect fit from the wonderful world of shlwapi functions :-))

The performance of Like operator is not that awful for the API to be much better but it's the # symbol and [a-z] class matches that differ to rudimentary filesystem wildcards and this might be a problem in some edge-cases.

Thumbs up! (I think I could use this API in some of my (public) classes too, provided that it's impl since win2k)

cheers,
</wqw>

[SamOscarBrown]

A 'simpler' option MAY be (depending upon your program), multiple FileListBoxes. Perhaps one for each of the different file types.

Just a thought, seeing as how your ONE FLB works fine, but is limited to the 32767.

[fafalone]

@wqweto yes MSDN lists it as available in Win2k pro and Win2k server.

[DataMiser]

Over 32k files in a folder seems more than a bit excessive to me. Why so many files in a single folder?
Perhaps creating some sub directories and copying the files of the given types to their own child folder may be a solution.

[Elroy]

It looks like you're getting good advice, but here's some code where I do something similar. Take a look at the LoadMovieFiles procedure. It loads a listbox with certain movie files it finds.

Public Sub LoadMovieFiles(lst As ListBox, sPathToExamine As String)
    Dim sFle As String
    '
    sFle = Dir$(AddSlash(sPathToExamine) & "*.*")
    Do
        If Len(sFle) = 0 Then Exit Do
        If FileIsVideo(sFle) Then
            lst.AddItem sFle
        End If
        sFle = Dir$
    Loop
End Sub

Public Function AddSlash(sFldr As String) As String
    ' Only adds it if not there.
    ' This can be used as EITHER a sub or function, as it modifies the input.
    If Right$(sFldr, 1) <> "\" Then sFldr = sFldr & "\"
    AddSlash = sFldr
End Function

Public Function FileIsVideo(sFile As String) As Boolean
    FileIsVideo = (LCase$(Left$(Right$(sFile, 3), 2)) = "mp") Or _
                  (LCase$(Left$(Right$(sFile, 4), 2)) = "mp") Or _
                  (LCase$(Right$(sFile, 3)) = "avi") Or _
                  (LCase$(Right$(sFile, 3)) = "wmv")
End Function

Maybe those ideas will help you to get going.

Good Luck,
Elroy

[SomeYguy]

Over 32k files in a folder seems more than a bit excessive to me. Why so many files in a single folder?
Perhaps creating some sub directories and copying the files of the given types to their own child folder may be a solution.

I agree. But this program will deal primarily with image files, and some folks just keep dumping their digital images into the same folder for years n years . I'd like to extend the upper limit to ~ 100k or so. Also, I'm streamlining and so using a hidden FileListBox just bugs me .

[SomeYguy]

Unrelated: Not sure why but when I reply I get a wait 30 seconds warning even though I haven't yet submitted and then the post shows up twice?

[SamOscarBrown]

Just sent note to MOD about your 'wait' issue

[Bobbles]

I have noticed the double posting for a couple of weeks now.
If you go into advanced edit(creation), it does not happen.
If you do what you have been doing (not advanced) it actually does send it on your first attempt.
If you close the Tab, and go back into the thread via the Forum list of the various threads, you will find that your post is in there.

[techgnome]

Unrelated: Not sure why but when I reply I get a wait 30 seconds warning even though I haven't yet submitted and then the post shows up twice?

It's a bug with the forum software... showed up after an upgrade a while back. No one's sure what's causing it or how to fix it. Has something to do with one of the extesions.... BUT if you look, after getting the wait message, and scroll down... you'll see that your post DID in fact post... and that's why you're getting the wait message.... the wait is a part of the anti-spam feature that forces posters to wait a certain amount of time between posts. In this case, there's something that causes the post to go through the first time, and then again a second time, which then gets caught, resulting in the wait message.... and of course, people aren't paying attention, wait, and post it again, and results in the double post.

-tg

[DataMiser]

Unrelated: Not sure why but when I reply I get a wait 30 seconds warning even though I haven't yet submitted and then the post shows up twice?

That happens to me all the time but it is after I hit post reply the first time I get the message and the screen looks like it has not been posted. If I wait and hit the post button then it is a double post. If I simply go back to the thread I can see that the message was posted and never duplicated.

[qvb6]

The double post issue is known and discussed in the Forum Feedback.

[Shaggy Hiker]

Also, some folks have suggested that this ONLY happens in the quick reply, so if you hit Go Advanced, and use that to submit messages, then perhaps it won't happen. I haven't tested it out, myself, very well. In fact, this will be my first test.

[SomeYguy]

Very good on the double post issue and thanks. I'm using Advanced now rather than quick so here goes .

[SomeYguy]

YAY!! Multiple file pattern w/Dir issue is resolved! Per cool little API code from Fafalone, I used the PathMatchSpecW API function. Works well.

I shoehorned it all into one sub, and here is the code in case anybody else can use.

Private Declare Function PathMatchSpecW Lib "shlwapi" (ByVal pszFileParam As Long, ByVal pszSpec As Long) As Long
 
 
Private Sub ListFiles(strPath As String, Optional strFilePattern As String)
    
    '- This is a good alternative to using a hidden FileListBox to list files.
    '- Note the use of Dir$ rather than Dir for that function. I read somewhere
    '- on an MVP site that Dir$ is marginally faster.
    
    Dim File              As String
    Dim fCount          As Long
    Dim nX               As Long
    
    Dim strFiles()      As String
    '- Define file pattern
    If strFilePattern = "" then strFilePattern = "*.*"
    '- Fix path if needed.
    If Right$(strPath, 1) [noparse]<> "\" Then strPath = strPath & "\"[/noparse]
    '- We want to see all files before we filter
    File = Dir$(strPath & "*.*")
    '- Do it
    Do While Len(File) > 0
        fCount = fCount + 1
        ReDim Preserve strFiles(fCount)
        strFiles(fCount) = File
        '- Pattern filter
        If PathMatchSpecW(StrPtr(File), StrPtr(strFilePattern)) Then
        
        '- ** This is where you do something with the returned file names.
        '- Can add to a listbox, string array, or ? **
        
        '- Example: List1.Additem File
        End If
        File = Dir$
    Loop
    
End Sub

EDIT: Code updated to include suggestions.

Thanks again!

[SomeYguy]

.....

[fafalone]

Right(File, Len(File) - InStrRev(File, ""))

That will return a blank string, it should be Right(File, Len(File) - InStrRev(File, "\"))

**I presume that's what you actually posted but it seems the forum code converts "\" into "" for some reason (I used the NOPARSE tag to get it to show)

[LaVolpe]

Right(File, Len(File) - InStrRev(File, ""))

That will return a blank string, it should be Right(File, Len(File) - InStrRev(File, "\"))

**I presume that's what you actually posted but it seems the forum code converts "\" into "" for some reason (I used the NOPARSE tag to get it to show)

Actually, is that check needed at all. Doesn't Dir$() return just the file name? Also, looks like not specifically handling Dir$() when it returns "." or ".."; though may not be important in that scenario? Looks like the 1st instance is handled by calling Dir$() before the loop.

Edited: ignore that last part as it applies when enumerating directories, not the case here

Though "File = Dir$" line should be moved to just before the "Loop" statement after "End If". As it is right now, the 1st file in the directory is never being checked.

[fafalone]

Oops yeah Dir just returns the filename, been a looooooong time since I enumerated files without some fancy way lol

[LaVolpe]

Oops yeah Dir just returns the filename, been a looooooong time since I enumerated files without some fancy way lol

Me too. Can't remember offhand what scenarios . or .. is returned in the enumeration

[fafalone]

I ran that code on a folder 3 levels deep printing all names, didn't get . or .. in the list.

[LaVolpe]

I ran that code on a folder 3 levels deep printing all names, didn't get . or .. in the list.

Ugh, now you made me go and find the scenario... it's when you include vbDirectory in the Dir$() command

[SomeYguy]

Right(File, Len(File) - InStrRev(File, ""))

That will return a blank string, it should be Right(File, Len(File) - InStrRev(File, "\"))

**I presume that's what you actually posted but it seems the forum code converts "\" into "" for some reason (I used the NOPARSE tag to get it to show)

Ugh. You're right, forum code took out my backslashes. I edited using the noparse tags and backslashes are showing now but so are the noparse tags .

[SomeYguy]

Actually, is that check needed at all. Doesn't Dir$() return just the file name? Also, looks like not specifically handling Dir$() when it returns "." or ".."; though may not be important in that scenario? Looks like the 1st instance is handled by calling Dir$() before the loop.

Edited: ignore that last part as it applies when enumerating directories, not the case here

Though "File = Dir$" line should be moved to just before the "Loop" statement after "End If". As it is right now, the 1st file in the directory is never being checked.

Great catches and thanks . In this scenario, I only need the file names within the current directory.

EDIT: Thanks especially for catching that "File = Dir$" line placement error. That cured a little issue which was driving me nuts!

[Bobbles]

Ugh. You're right, forum code took out my backslashes. I edited using the noparse tags and backslashes are showing now but so are the noparse tags .

If there is any chance that the posted code is confused(mangled), perhaps attach a small project