I need to scan a list of files on our FTP site, finding ones with dates in between the given range... problem is, there are over 40K files in the location

this works to get a list of the files,
Code:
Dim sftp As FtpWebRequest = FtpWebRequest.Create("ftp://XXX.XX.X.XXX/BDT/")
        sftp.Method = WebRequestMethods.Ftp.ListDirectory
        Application.DoEvents()
        tssl_Status.Text = "Logging In..."
        Application.DoEvents()
        sftp.Credentials = New NetworkCredential("[USERNAME]", "[PASSWORD]")
        sftp.Timeout = -1
        Dim response As FtpWebResponse = sftp.GetResponse
        tssl_Status.Text = "Retreiving File List..."
        Application.DoEvents()
        Dim reader As New StreamReader(response.GetResponseStream)
        Dim str() As String = Split(reader.ReadToEnd, vbNewLine)
then i have to take each one and do a get file date
Code:
Dim FI As FtpWebRequest = FtpWebRequest.Create("ftp://XXX.XX.X.XXX/BDT/" & File)
                    FI.Method = WebRequestMethods.Ftp.GetDateTimestamp
                    FI.Credentials = New NetworkCredential("[USERNAME]", "[PASSWORD]")
                    FI.Timeout = -1
                    response = FI.GetResponse
                    Dim DateMod As DateTime = response.LastModified
                    
                    If DateMod >= startdate And DateMod <= enddate Then
now, as you can imagine, this is quite slow when checking 44,562 files....

so, i tried to use

sftp.Method = WebRequestMethods.Ftp.ListDirectoryDetails

but it errors whenpulling the response intot the streamreader... if i let it sit a minute, then hit F5 it will finish, but wont work stright through


SO what do i do? is there a better way?? Any help or tips would be great!!