Adding files to a list with a filter

Printable View

Nov 25th, 2012, 01:43 AM
Simon Canning

Adding files to a list with a filter

I have a function that searches through folders recursively. I pass in an arraylist of strings to use as a file filter.

For some reason, no matter how many extensions I pass in, only files that are found with the first extension are added.

May I please have some help to fix this code.

Here is my code:

Code:

Public Function SearchAndAddToList(ByVal path As String, ByVal Recursive As Boolean, stringArrayListOfFileFilters As ArrayList) If Not Directory.Exists(path) Then Exit Function Dim initDirInfo As New DirectoryInfo(path) Dim tmpListOfAllMedia As MediaItem Dim stringFilenameFilter As String formAddFolder.toolstripStatus.Text = "Retrieving file information from: " & path For Each oFileInfo In initDirInfo.GetFiles Application.DoEvents() For x = 0 To stringArrayListOfFileFilters.Count - 1 stringFilenameFilter = stringArrayListOfFileFilters(x) If oFileInfo.Name Like stringFilenameFilter Then tmpListOfAllMedia.FileInfoObject = oFileInfo tmpListOfAllMedia.MediaObject = MediaPlayerObject.newMedia(oFileInfo.FullName) listOfAllMedia.Add(tmpListOfAllMedia) End If Next Next If Recursive Then For Each oDirInfo In initDirInfo.GetDirectories SearchAndAddToList(oDirInfo.FullName, True, stringArrayListOfFileFilters) Next End If End Function

An example of the filters that I pass to this function is:

*.mp3
*.docx
*.txt

In this situation, only *.mp3 files are added to the list.

Example 2:

*.docx
*.mp3
*.txt

In this example, only *.docx files are added.

Why would this be? And more importantly, how can I fix this code?

Thanks

Re: Adding files to a list with a filter

I can see several problems with your code:
1) You have declared a function, but as it never returns anything, you might as well use a sub.
2) Using Application.DoEvents is really not the way to do this (it never is!!!) - you should use a backgroundworker or simply a wait-pointer, since the search is likely to be completed fairly quickly.

I have made a rather simple version of your code (ie. return value is just a list of string) that uses a wait-pointer, but rewriting it to fit a bgw shouldn't be much work.

vb.net Code:

Public Function SearchForMedia(path As String, recursive As Boolean, _
                                   extensions As SortedSet(Of String)) As List(Of String)
        If Not Directory.Exists(path) Then Return Nothing
 
        ' Declare a new list to hold the returned values.
        Dim return_list As New List(Of String)
 
        Me.Cursor = Cursors.WaitCursor
 
        ' Find the files as specified in the extension set.
        FindMediaFilesInDirectory(path, recursive, extensions, return_list)
 
        Me.Cursor = Cursors.Default
 
        Return return_list
    End Function
    Private Sub FindMediaFilesInDirectory(current_dir As String, recursive As Boolean, _
                                          extensions As SortedSet(Of String), foundfiles As List(Of String))
        For Each f As String In Directory.GetFiles(current_dir)
            If extensions.Contains(Path.GetExtension(f)) Then foundfiles.Add(f)
        Next
        If recursive Then
            For Each d As String In Directory.GetDirectories(current_dir)
                FindMediaFilesInDirectory(d, True, extensions, foundfiles)
            Next
        End If
    End Sub

To use it:

vb.net Code:

Dim l As List(Of String) = SearchForMedia(My.Computer.FileSystem.SpecialDirectories.MyPictures, _
                                                  True, _
                                                  New SortedSet(Of String) From {".jpg", ".png", ".bmp"})

Tom

Nov 25th, 2012, 09:00 AM
dunfiddlin

Re: Adding files to a list with a filter

Erm ....

Dim episodes = My.Computer.FileSystem.GetFiles(FolderList.SelectedItem, FileIO.SearchOption.SearchAllSubDirectories, "*.avi", "*.mp4", "*.mkv")

... (a line from my own media seracher) does more or less all of this in one line!
Nov 25th, 2012, 09:22 AM
ThomasJohnsen

Re: Adding files to a list with a filter

You're right as usual. But now you know how it's implemented :)
And besides the OP seemed to want to encapsulate the results into some classes while searching, which means that he has to do the whole recursive search anyways.
Nov 25th, 2012, 05:16 PM
Simon Canning

Re: Adding files to a list with a filter

Quote:

uses a wait-pointer

I cannot actually see where this is in your code. Can you please explain how you have done this?
Nov 25th, 2012, 05:53 PM
Simon Canning

Re: Adding files to a list with a filter

Here is my code that I currently use to find files:

Code:

Public Function SearchAndAddToListWithFilter(ByVal path As String, ByVal Recursive As Boolean, arrayListOfFilters As ArrayList, ByRef listOfFiles As List(Of FileInfo)) If Not Directory.Exists(path) Then Exit Function Dim initDirInfo As New DirectoryInfo(path) For Each oFileInfo In initDirInfo.GetFiles Application.DoEvents() For x = 0 To arrayListOfFilters.Count - 1 If (oFileInfo.Name Like arrayListOfFilters(x)) Then listOfFiles.Add(oFileInfo) End If Next Next If Recursive Then For Each oDirInfo In initDirInfo.GetDirectories SearchAndAddToListWithFilter(oDirInfo.FullName, True, arrayListOfFilters, listOfFiles) Next End If End Function

And here is how to use it:

Code:

Dim stringFilterList As String = "*.mp3, *.docx, *.mp3, *.txt, *music*, *test*" Dim arrayListOfFilenameFilters As New ArrayList(stringFilterList.Split(",")) Dim stringFolderPath As String = "C:\temp\folder\" Dim booleanSearchSubFolders As Boolean = True Dim listOfFilesFoundViaSearch As New List(Of FileInfo) SearchAndAddToListWithFilter(stringFolderPath, booleanSearchSubFolders, arrayListOfFilenameFilters, listOfFilesFoundViaSearch) For x = 0 To listOfFilesFoundViaSearch.Count - 1 MsgBox(listOfFilesFoundViaSearch(x).FullName) Next

In this above code I use "Application.DoEvents()". I gather that I should not use this. Can I please have some help in what I should use instead of "Application.DoEvents()" in my above code?

Also, I am wanting to filter the files that are added to the list. This filter is not just for extensions (as the above code was modified to). My above code only filters files that correspond to the first filter in the list of filters.

Can I please have some help to fix this?

Re: Adding files to a list with a filter

Quote:

Originally Posted by Simon Canning

Can I please have some help to fix this?

First off - this is a commented/edited version of your code - should be mainly self-explanatory:

vb.net Code:

'This is a modified version of your code - I have put in remarks, where changes have been made:
    '1: Parameters are declared as ByVal as a default, so there's no need to type it every time.
    '2: There is no need to use ByRef on a List, since it is a reference type and thus can be added
    '   to and removed from both with ByVal or ByRef declaration.
    '3: List(Of T) performs better than ArrayList in most cases and is type-safe (source MSDN).
    '4: Changed path to Path and listOfFiles to ListOfFiles to have naming consistancy.
    Private Sub YourSearchAndAddToListWithFilter(Path As String, Recursive As Boolean, ListOfFilters As List(Of String), ListOfFiles As List(Of FileInfo))
        If Not Directory.Exists(Path) Then Exit Sub
 
        Dim InitDirInfo As New DirectoryInfo(Path) '5: Changed from initDirInfo
 
        For Each oFileInfo As FileInfo In InitDirInfo.GetFiles() '6: Added type and Method paranthesis.
            'Application.DoEvents() '7: Commented out
            For Each Filter As String In ListOfFilters '8: Changed to foreach which is more natural
                If oFileInfo.Name Like Filter Then '9: Unnecessary paranthesis.
                    ListOfFiles.Add(oFileInfo)
                End If
            Next
        Next
 
        'The 2 nested loops above can be written in a more natural way as:
        'For Each Filter As String In ListOfFilters
        '    For Each oFileInfo As FileInfo In InitDirInfo.GetFiles(Filter)
        '        ListOfFiles.Add(oFileInfo)
        '    Next
        'Next
 
        If Recursive Then
            For Each oDirInfo As DirectoryInfo In InitDirInfo.GetDirectories() '10: Added type and paranthesis.
                YourSearchAndAddToListWithFilter(oDirInfo.FullName, True, ListOfFilters, ListOfFiles)
            Next
        End If
    End Sub
    Public Sub YourSearchForMedia()
        'Here I mainly changed names to follow your own naming conventions and changed the loop from
        'an integer loop to the more natural foreach. Also MsgBox() should be MessageBox.Show().
        'If you want to keep spaces in your StringFilterList, you should use something like:
        ' StringFilterList.Split(",").Select(Function(s) Trim(s)).ToList()
        'instead of just splitting.
        Dim StringFilterList As String = "*.png,*.bmp,*oa*"
        Dim ListOfFilenameFilters As List(Of String) = StringFilterList.Split(",").ToList()
        Dim StringFolderPath As String = "<mydir>"
        Dim SearchSubFolders As Boolean = True
        Dim ListOfFilesFoundViaSearch As New List(Of FileInfo)
 
        YourSearchAndAddToListWithFilter(StringFolderPath, SearchSubFolders, ListOfFilenameFilters, ListOfFilesFoundViaSearch)
 
        For Each oFileInfo As FileInfo In ListOfFilesFoundViaSearch
            'MessageBox.Show(oFileInfo.FullName)
            Console.WriteLine(oFileInfo.FullName) 'Using this instead of having 1000 files displayed.
        Next
    End Sub

For your purposes however, I'd advice you to simply use:

vb.net Code:

'Instead of using a sub and modify a paramter, I chose the more natural function to return a result.
    'Also filters are passed as a string, so you don't need to split the string on each call.
    Private Function MySearchWithFilter(Path As String, Recursive As Boolean, Filters As String) As ObjectModel.ReadOnlyCollection(Of String)
        If Not My.Computer.FileSystem.DirectoryExists(Path) Then Return Nothing
 
        Dim Search As SearchOption = If(Recursive, SearchOption.AllDirectories, SearchOption.TopDirectoryOnly)
 
        Return My.Computer.FileSystem.GetFiles(Path, Search, Filters.Split(","c))
    End Function
    Public Sub MySearchForMedia()
        Dim FilterList As String = "*.png,*.bmp,*oa*"
        Dim FolderPath As String = "<mydir>"
        Dim SearchSubFolders As Boolean = True
 
        For Each FileName As String In MySearchWithFilter(FolderPath, SearchSubFolders, FilterList)
            Console.WriteLine(FileName)
        Next
    End Sub

If you are going to be making more complicated searches (using for instance Regex), your code is easy to modify, whereas my shorter example cannot be extended.

When making operations that take a substantial amount of time, there basically are two options:
1) Accepting that the user-interface freezes for a short amount of time - one can show a waiting pointer (Me.Cursor = Cursors.WaitCursor) or similar to indicate that something is going on.
2) Using a background thread to run the process on. This frees the user-interface (except possibly controls associated with the process). This is most likely not neccessary in your example, unless you search through many nested directories with several hundreds of files. It is not a terribly complicated thing to do, but there are some cross-thread considerations to be made - there are several good guides on how it's done on these fora.

Tom

#EDIT: Forgot about the Application.DoEvents() - do a search on it on these fora. There are many discussions about it :)

#EDIT2: If you choose to still rely on your own version, you might wan't to change the usage of DirInfo and FileInfo. The string versions are faster, so unless you need the additional info in those classes, you should avoid them. A bit of additional speed can be found by using EnumerateFiles/EnumerateDirectories instead of GetFiles/GetDirectories on very large directories.