Fast File Search Utility App Demo With Source

[jpbro]

JPBFileFinder is a tool for quick and easy file searches on your computer. You can search any folder (optionally recursively) to find files based on properties such as name, extension, size, attributes, hash, and content (ANSI, Unicode, or Hex bytes). Multi-criteria searches are "AND" based for simplicity - all criteria must be true for a file to be considered a match.



JPBFileFinder is a fairly comprehensive and (hopefully) fully functional RC6-based app. I've tried to touch as many RC6/RC6Widgets objects as possible to demonstrate their use in a real-world application scenario. I've done this to learn a bit more about RC6 Forms/Widgets mostly, but also to server as a nice demonstration project for newcomers to RC6. Hopefully it will be of use to some of you.

It's not perfect, but I am quite happy with it so far. For one thing, it is very fast! A file contents search for a short test string took about 5.5 minutes across 50,000+ files/~8.5GB of data, making about 3,000 matches. A similar performance test using my old favourite search app (HDSearchAndStats.exe) took closer to 7 minutes.

A second comparison where I constrained the content search to *.txt files only, took less than 3 seconds with JPBFileFinder, and about 30 seconds with HDSearchAndStats. This was after I removed some symbolic links that tripped up HDSearchAndStats and forced it into an infinite loop - there's no such problem with JPBFileFinder

In a third comparison, I grabbed every file/folder recursively (no filter/search criteria) in C:\Windows\. It took JPBFileFinder just over 30 seconds to match almost 300,000 files. HDSearchAndStats took about 7 minutes to reach 99%, when it then completely froze for an additional HOUR AND 12 MINUTES while it filled the results list (trying it's damnedest to melt one logical processor core in the meantime, and having memory usage climb up to >350MB commit vs. 265MB for JPBFileFinder). This clearly demonstrates the advantages of RC6Widgets Virtual List/Grid approach.



So not a bad effort by JPBFileFinder & RC6 I would say Even if you have no interest in programming with RC6, you might find JPBFileFinder to be a useful tool.

Getting Started:

• Extract the contents of the following attachment to your computer: JPBFileFinderAppAndSource10.zip
• Go into the JPBFileFinderAppAndSource folder.
• Download and extract the contents of RC6BaseDlls.zip (available at https://vbrichclient.com/downloads/RC6BaseDlls.zip) to the \JPBFileFinderAppAndSource\JPBFileFinder\System\ folder.
• Open \JPBFileFinderAppAndSource\Source\Helpers\JPBFileHelpers.vbp and compile it to \JPBFileFinderAppAndSource\JPBFileFinder\System\
• Close VB6.
• Open \JPBFileFinderAppAndSource\Source\JPBFileFinder.vbp
• Click the Project menu, then click References.
• Select JPBFileHelpers.
• Click the File menu, then click Make JPBFileFinder.exe
• Navigate to \JPBFileFinderAppAndSource\JPBFileFinder\ and then click OK.
• Close VB6.
• Run \JPBFileFinderAppAndSource\JPBFileFinder\JPBFileFinder.exe
• Have fun searching

NOTE: You can copy the \JPBFileFinder\ folder with the compiled binaries to any computer and run \JPBFileFinder\JPBFileFinder.exe *without* needing to register anything. Try it!

[jpbro]

For those of you interested in exploring the source code, and that have an interest in RC6/RC6Widgets, below is a list of the classes that I have used, and a brief explanation of what they are used for. Explore the source code to deepen your understanding of them, and feel free to ask me questions here.

RC6 Classes Used:

cCrypt
- For hashing files and byte arrays (SHA-2/384/512/MD5)

cFSO
- For working with the filesystem (enumerating folders & files, checking folder/file properties, reading & writing files).

cBigInt
- For working with HUGE numbers (>64-bit max)

cArrayList
- For keeping track of folders to scan, with easy queue/dequeue

cMemDb/cRecordset
- For storing drop-down list values and match results.

cCommand
- For parameter based building of down-down list data

cDirList
- For quickly enumerating files and folders

cTimer
- For pumping the Search loop while keeping the UI thread responsive

cStringBuilder
- For fast string concatenation

cConstructor
- For registry free instantiation of RC6 objects

cActiveScript
- For JS regex engine support

cStream
- For searching file contents. I'm doing this with a "chunked" approach so that files >2GB can be scanned.

cThreadHandler
- For threaded content searching & hashing of large files, keeping the UI thread responsive.

cWidgetForm
- For building RC6-based forms for search options, progress, and results.


RC6Widgets Classes Used:

cwButton
- For demonstrating the use of RC6 Button widgets.

cwDropDownList
- For demonstrating the use of RC6 Drop Down List (i.e. ComboBox) widgets.

cwTextBox
- For demonstrating the use of RC6 TextBox widgets.

cwLabel
- For demonstrating the use of RC6 Label widgets.

cwProgressBar
- For demonstrating the use of RC6 Progress Bar widgets.

cwGrid, cwVList
- For demonstrating the use of RC6 Grid widgets in virtual list mode.




Enjoy! Comments, questions, and criticisms are always welcome.

[Carlos Rocha]

Thank you

[jpbro]

Cheers Carlos

[pekko]

Looks not so good on 100% scaling

[jpbro]

Thanks pekko, I'll take a look and get it fixed.

[jpbro]

I've updated the source in the first post to fix display problems due to bad DPI conversions. When I changed the code to use the WidgetForm Zoom feature, I forget to get rid of my manual scaling calculations in the displayScalePoint method. I've cleaned all of that up and things should look good now at all DPIs (tested at 100%, 125%, 150%, 175% and 200%).

Thanks for reporting the problem pekko!

[jpbro]

I forgot to mention (in case Olaf ever happens by here) that after fixing (or at least hopefully fixing) the DPI stuff, the cwDropDown list item heights are too short:



I'm not sure why, but I'll try to figure it out. In the meantime, if anyone else knows what's going on, I'd love to hear about it.

[jpbro]

I forgot to mention (in case Olaf ever happens by here) that after fixing (or at least hopefully fixing) the DPI stuff, the cwDropDown list item heights are too short:
...
I'm not sure why, but I'll try to figure it out. In the meantime, if anyone else knows what's going on, I'd love to hear about it.

OK I figured this one out and updated the source in the first post.

It looks like setting the DropDownList.VList.RowHeight doesn't take the WidgetForm.Zoom level into account, so you have to perform the scale calculations manually. It seems to me that the Zoom level should be taken into account here though, as we are on the zoomed side of the Zoomed/Unzoomed threshold within the widget, no?

[pekko]

Thanks for the quick fix

Here's another problem
Do recursive search from folder C:\Windows
File Content ANSI Text abcdef
Program crashes

[pekko]

This was in compiled exe. Haven't test in IDE

[jpbro]

Thanks for all the testing pekko

I've updated the source in the first post to fix the problem. I've been very light on the error handling in this project, so it was blowing up when it tried to read file content of any file it didn't have permission to read. Now these files will be skipped silently. There are probably some other areas where this might happen - one that comes to mind is if a file is deleted after its path is cached in the list of files to scan, so I'll run some tests and try to handle this case better.

[jpbro]

Latest version (source in first post) should no longer crash when a file is deleted during a scan. Deleted files will just be skipped silently.

[pekko]

Works ok

Development proposal:
When you do a search that takes a long time and you press cancel, it would be nice if the program would show the results it has found so far.

[jpbro]

Works ok

Great, thanks for letting me know.

Development proposal:
When you do a search that takes a long time and you press cancel, it would be nice if the program would show the results it has found so far.

Good idea, I had the same thought The latest version in the first post will now show partial results (if any) when the search is cancelled.

[xiaoyao]

Very powerful, very proficient in using various functions of rc6, and the interface is also very good-looking. Searching files is also fast

It would be even better if various functions can be encapsulated into a class, and everyone can call it without UI. Integrate your functionality directly into my software.
For example: search file content ("d:", "*.bas", "createthread")

[taishan]

Latest version (source in first post) should no longer crash when a file is deleted during a scan. Deleted files will just be skipped silently.

Does the project attachment in post #1 work for you? I'm getting an invalid link.

[jpbro]

It works for me - try refreshing the page. I did a last second update after my post in the other thread, so you may have clicked the link in the other thread before I updated, then clicked the link in this post after I updated.

[taishan]

WOW! Now that is super impressive, it works like Greased Lightning! It finished searching everything in User\Documents (7,706 files, 512 MB) milliseconds after I clicked "Search". I think I'm going to love this virtual Listview...
Maybe add a line to the Readme to skip steps 2 & 3 if the programmer already has RC6 dlls registered elsewhere. I was ready for the DirectCom registry-free RichClient reference, as I've already got that working with my current app.
There's a ton of good things going here! Cheers & Thanks!

[taishan]

Question: will this work on a Unicode file path?

Sub Main()
   ChDir App.Path

Edit: I renamed the folder \香港JPBFileFinder\ and it throws a run-time 76, path not found. I think you'll have to change that to the other method
Cheers

[jpbro]

Glad you like what you see

Good catch with the ChDir call too - I tried to make everything unicode aware in the MRc6Base module, but messed up in the main app. Does changing the ChDir to SetCurrentDirectoryW work?

Private Declare Function SetCurrentDirectory Lib "kernel32" Alias "SetCurrentDirectoryW" (ByVal lpPathName As Long) As Long

Sub Main()
   SetCurrentDirectory StrPtr(PathImage)

[taishan]

I thought it would work but we get an error -2147221504 Extender Not Available - which I guess is triggered by Cairo.SetDPIAwareness

45 minutes later:
Ok, I tried every permutation I could think of setting the initial directory, and finally settled on :

SetCurrentDirectory StrPtr(PathAppSystem) '// PathAppSystem is your existing routine

I've been meaning to ask Olaf for the most efficient way to reg-free the richclient familly of dlls, while working inIDE and deploying, and using Properties seems to be better than Functions...? At least on my existing app and now this one... And I don't understand why.

On that note, the RichClient instantiation/configuration using Properties instead of Functions is the only one that works for me with \UnicodeChars\ExecutableLocation\System\*.dll --- which is this on my machine:

"O:\Downloads\JPBFileFinderAppAndSource8\Deploy\香港JPBFileFinder\JPBFileFinder.exe"
"O:\Downloads\JPBFileFinderAppAndSource8\Deploy\香港JPBFileFinder\System\RC6.dll"

So, I replaced MStartup.bas with the following --- and commented out your RC6 instantiating New_c and Cairo Functions in MRc6Base.bas:

Option Explicit

Private Declare Function SetCurrentDirectory Lib "kernel32" _
                         Alias "SetCurrentDirectoryW" ( _
                         ByVal lpPathName As Long) As Long

Private Declare Function LoadLibraryW Lib "kernel32" ( _
                         ByVal StrPtr_FileName As Long) As Long
                         
Private Declare Function GetInstanceEx Lib "DirectCom" ( _
                         ByRef StrPtr_FName As Long, _
                         ByRef StrPtr_ClassName As Long, _
                         Optional ByVal UseAlteredSearchPath As Boolean = True) As Object

Private mf_Main As FMain

Public Property Get New_c() As cConstructor

    Static statNew_c As cConstructor
    If statNew_c Is Nothing Then    'try to instantiate it for the first time
        'we run compiled, so we try a regfree instancing of New_c first ...
        If App.LogMode Then
            'pre-loading DirectCOM.dll first from our \Bin\-Folder
            LoadLibraryW StrPtr(App.Path & "\System\DirectCOM.dll")
            'now using GetInstanceEx from DirectCOM.dll
            Set statNew_c = GetInstanceEx(StrPtr(App.Path & "\System\RC6.dll"), StrPtr("cConstructor"))
        End If
        ' in IDE-Mode (or in case of failure) we try to instantiate from a registered version normally per 'New'
        If statNew_c Is Nothing Then Set statNew_c = New cConstructor
    End If

    Set New_c = statNew_c    'just hand over the (now initialized) Static-Variable

End Property

Public Property Get Cairo() As cCairo

    Static statCairo As cCairo
    If statCairo Is Nothing Then    'try to ...
        Set statCairo = New_c.Cairo '...instantiate it for the first time
    End If
    Set Cairo = statCairo           'just  hand over the (now initialized) Static-Variable

End Property

Sub Main()

                    ' We want to make sure our "Current" directory is the app dir
                    ' so any relative path references resolve to the expected folder.
                    ' This smells a bit cargo cult-y, but it can't hurt.
                    
    SetCurrentDirectory StrPtr(PathAppSystem)
   
   ' Initialize RC6 Objects.
   '
   ' We have 3 ways to initialize the RC6 ActiveX objects:
   '
   ' 1) By using the Windows Registry through the normal "As New" VB6 means
   ' 2) By skipping the Windows Registry using a Manifest
   ' 3) By skipping the Windows Registry by using Olaf Schmidt's DirectCOM.dll
   '
   ' We're going to use option #3 since it is the easiest way to get registry-free instantiation of COM objects
   
   Cairo.SetDPIAwareness
   
   ' Initialize main form and show it
   Set mf_Main = New FMain
   
   mf_Main.Form.Show
      
   If Not envRunningInIde Then 'use the RC6-MessagePump, when we run compiled
      Cairo.WidgetForms.EnterMessageLoop  'this will (due to its optional Param) automatically cleanup the RC6-lib on exit
   Else
     Load FDummy 'in IDE-mode, we ensure the IDEs message-pump for better debugging-support (by loading an invisible Dummy-Form)
   End If

End Sub

[taishan]

BTW: One aspect of your project that strikes me as superior is your detailed comments and thorough explanations. Cheers for that !!!

[jpbro]

So I've traced the problem to RC6 Cairo.WidgetForms.Create, and the error occurs in the CWidgetRoot class RenderContent method. Not sure what the Unicode current directory name would cause a problem there yet though. The method seems to be failing to detect that our render host is a cWidgetForm class and is then switching to UserControl host mode, but since we're not rendering to a UserControl we get an error when trying to access the UserControl.Extender object. I'll have to do some more digging...

[jpbro]

So it looks like the return value of TypeName(RenderHost) in a non-unicode folder is "cWidgetForm", but when the folder has Unicode characters then TypeName(RenderHost) returns "Object" - for some reason TypeName can't determine the specific object type. When this happens, RC6 assumes we're rendering to a UserControl (even though we aren't). I wonder if testing TypeOf RenderHost Is cWidgetForm will work?

[jpbro]

I wonder if testing TypeOf RenderHost Is cWidgetForm will work?

CORRECTION: That does appear to work! Olaf will likely know the reason for the different typename results though, and may have a better solution, so we should wait to see what he has to say (if he happens to be reading).

[taishan]

I was able to compile my version and run as .exe, but not in the IDE because I encountered your problem. Did you get it working with Function instantiation instead of Property? Cheers

[jpbro]

I was able to compile my version and run as .exe, but not in the IDE because I encountered your problem.

Hmm..I'm able to run compiled or not. Does the path to your VB6.exe have problematic Unicode characters?

Did you get it working with Function instantiation instead of Property? Cheers

TBH I didn't try your Function->Property modification because I can't imagine how that would make a difference. But (big BUT) if it does make a difference, then I'm not the right guy to ask. it might be a better question for the guys that have deep-dived into into the internal workings of the compiler.

[taishan]

No Unicode path (Chars) to VB6. Windows is American English. I just test everything with Unicode 'gotchas' because my program runs in 30 languages changeable on-the-fly, and is completely portable with reg-free RC6, SxS for VBCCR and Eduardo's Tab - I verify every library is running from exe path with SysInternals ProcessExplorer. The in-place registration RC6 .vbs should not be shipped to end user.
formwidget inIDE choked exact same place you mentioned, but compiled and ran the .exe fine from unicode path mentioned above.
Your instantiation functions would not work in same circumstances, had to change to Property.

[taishan]

Sorry I dragged this thread off topic, back to your excellent code, I'm learning all kinds of great stuff:

If Not envRunningInIde Then
      Set widgetNewDropDownList = New_c.RegFree.GetInstanceEx(WidgetDllPath, WidgetClassName(widget_DropDownList))
End If

Widgets are a bit daunting, but after I see your solutions, it's given me optimism that eventually this is a great way to go for UI.
Cheers & thanks!

[taishan]

This search is so good and fast searching 30 TB on my desktop, that I wish I was smart enough to hook/highjack the search bar in explorer and make it open this program. I will have to train myself to use this for searches...

[taishan]

Can we add a right-click shell extension "Search with JPBFileFinder" - it's the best search tool I've found. I just forget to fire it up when I'm searching...
It would be great to have an installation program for portable apps. I don't feel comfortable telling users to "unzip it anywhere"... A more standard looking installation interface would cut down on user support.

[jpbro]

Can we add a right-click shell extension "Search with JPBFileFinder" - it's the best search tool I've found. I just forget to fire it up when I'm searching...
It would be great to have an installation program for portable apps. I don't feel comfortable telling users to "unzip it anywhere"... A more standard looking installation interface would cut down on user support.

Glad you like the tool, but it is important to understand that this was just a demo/first foray for me into vbWidgets...it's definitely not a complete product. Primarily I wanted to give vbWidgets a serious try, and the only way I know how to do that is to make a small app. Secondarily I wanted to demonstrate some of the power of RC3/4/5/6 in the more complete setting of a fully functioning app, not just a bunch of classes/definitions in the Object Browser.

That said, I've been working on some improvements (and a Shell Extension might come out of this, we'll see). I've also been playing around with twinBasic as a front-end instead of vbWidgets. I like to see what the UI looks like and iterate against what I see, so vbWidgets has a lot of friction for me compared to a visual designer. Anyway, here's where I'm at so far (tB on the left, vbWidgets on the right):



Anyway, tB and vbWidgets both have their advantages and disadvantages, but I'm enjoying doing the UI stuff more in tB. All the non-UI stuff will continue to be RC6 for the foreseeable future.

[taishan]

Understood on the vbWidgets. A vbWidgets Visual Designer would be extremely helpful.
Or maybe design with VB, and then translate the controls coordinates with the Forms 2 vbWidgets example Olaf put on the Zoom display thread.
Is there a noticable performance gain with TB compiled at 64 bit?
Cheers

[tmighty2]

Thank you very much!!

[tmighty2]

jpbro, may I ask you if you are currently using widgetforms or VB6 forms in your projects generally?

[jpbro]

jpbro, may I ask you if you are currently using widgetforms or VB6 forms in your projects generally?

No I don't generally. This project was my first real attempt to learn and use RC5/6 widgetforms/widgets, and I found it didn't really work for the way I prefer to design forms. I'm not a "plan in advance" or on-paper type for UI stuff, I prefer to work visually and experiment with different layouts that I can see before I settle on something. So the VB6 form designer or HTML/XAML just fit my working style better.

That said, I've recently started using a mix of the RC6 WebView2 wrapper and regular VB6/ActiveX controls around it to build tools for my main app. It is a combination that has been both fun and productive, so for now I will be going that route (at least until I go web-only, or to tB64 under which RC6 won't currently work. That will be a bit of a bummer).

[wuxiu]

In Rc6, GetFontHeight Use this method: cCairoContext.MeasureContext.GetFontHeight?And GetfontWidth Use this mehod: cCairoContext.MeasureContext.GetTextExtents()

So, the Funcion widgetNewDropDownLis is modified as follows:

Public Function widgetNewDropDownList(po_ParentForm As RC6.cWidgetForm, ByVal p_Name As String, p_LeftPx As Long, ByVal p_TopPx As Long, ByVal p_WidthPx As Long, Optional ByVal p_HeightPx As Long = -1) As RC6Widgets.cwDropDownList
   If Not envRunningInIde Then
      Set widgetNewDropDownList = New_C.RegFree.GetInstanceEx(WidgetDllPath, WidgetClassName(widget_DropDownList))
   End If
   If widgetNewDropDownList Is Nothing Then Set widgetNewDropDownList = New RC6Widgets.cwDropDownList
         
   With widgetNewDropDownList
      With .VList
         .Widget.FontName = optFontNameDefault
         .Widget.FontSize = optFontSizeBase

'         .RowHeight = displayDefaultFontHeightPx * displayDpiScale
         Dim MCC As RC6.cCairoContext
         Set MCC = widgetLastAdded(po_ParentForm).MeasureContext
         .RowHeight = MCC.GetFontHeight + 5

      End With
      .Widget.BackColor = optBackcolorDefault
      .Widget.ForeColor = optTextcolorDefault
      
      .Widget.FontName = optFontNameDefault
      .Widget.FontSize = optFontSizeBase
   End With
   
   po_ParentForm.Widgets.Add widgetNewDropDownList, _
                             p_Name, _
                             p_LeftPx, p_TopPx, _
                             p_WidthPx, PassedHeightOrDefaultHeightPx(p_HeightPx)
End Function

And in RC6, cwDropDownList can Auto-height property , so we wouldn't need to do this ourselves.
so the Funcion GetOptimalVlistHeight and drpXXXDropDown _PopupWidgetRequestFuncion can be can be commented out.

Attachment 194207

[jpbro]

Hi wuxiu,

Thank you for the feedback and recommended changes, wasn't aware of the MeasureContext property until now.

I made a small change to get rid of the MCC temp object variable as I think it looks a bit cleaner, and I multiply your magic padding number 5 by the displayDpiScale for consistency at any DPI, but otherwise I think the code functions the same:

Public Function widgetNewDropDownList(po_ParentForm As RC6.cWidgetForm, ByVal p_Name As String, p_LeftPx As Long, ByVal p_TopPx As Long, ByVal p_WidthPx As Long, Optional ByVal p_HeightPx As Long = -1) As RC6Widgets.cwDropDownList
   If Not IsRunningInIde Then
      Set widgetNewDropDownList = New_C.RegFree.GetInstanceEx(WidgetDllPath, WidgetClassName(widget_DropDownList))
   End If
   If widgetNewDropDownList Is Nothing Then Set widgetNewDropDownList = New RC6Widgets.cwDropDownList
         
   With widgetNewDropDownList
      With .VList
         .Widget.FontName = optFontNameDefault
         .Widget.FontSize = optFontSizeBase
      
         .RowHeight = .Widget.MeasureContext.GetFontHeight + 5 * displayDpiScale   ' Thanks to wuxio @ https://www.vbforums.com/showthread.php?898206-Fast-File-Search-Utility-App-Demo-With-Source&p=5670777&viewfull=1#post5670777
      End With
      
      .Widget.BackColor = optBackcolorDefault
      .Widget.ForeColor = optTextcolorDefault
      
      .Widget.FontName = optFontNameDefault
      .Widget.FontSize = optFontSizeBase
   End With
   
   po_ParentForm.Widgets.Add widgetNewDropDownList, _
                             p_Name, _
                             p_LeftPx, p_TopPx, _
                             p_WidthPx, PassedHeightOrDefaultHeightPx(p_HeightPx)
End Function

I'll update the source in the first post shortly. Thanks again!

[jpbro]

Updated first post to use wuxio's recommendations. Also updated to the latest version of MRegFree.bas (formerly MRC6Base.bas) and removed some code/modules that were no longer necessary.

Thanks again wuxiu