dcsimg
Results 1 to 27 of 27

Thread: VB6 Automated Source Code Processing Helper

  1. #1

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    VB6 Automated Source Code Processing Helper

    WORK IN PROGRESS

    This is a work in progress - it does a pretty good job of doing what it is trying to do, but there are definitely holes in the implementation. If you can provide any source code that fails, I will be happy to fix my code to accommodate it!

    WHAT IS THIS?

    This project parses VBG & VBP files, building a list of all source code files (with limitations, see the KNOWN/EXPECTED PROBLEMS section below). You can then loop through each source file, and each line of code therein to perform just about any line-level custom processing you can imagine. The project includes fast PCRE2 regex support for finding matching lines of code (in forward and reverse directions).

    When you find a line you are looking for, you can optionally replace the text with something else, and when you are done you can save the file back to disk.

    Attachment 164403

    A VERY VERY VERY IMPORTANT WARNING
    YOU can destroy your source code files using this project as it allows you to modify source code and write it back to disk! Bugs (mine or yours) can be catastrophic, so please make sure you backup your source code before processing it!

    NOTE: The included example parser does not modify source files and in my opinion can be considered safe for all uses. That said, I still always recommend having backups made before using this project on your source code.

    I TAKE NO RESPONSIBILITY FOR ANY TERRORS THIS PROJECT MAY UNLEASH UPON YOUR COMPUTER/FILES - YOU HAVE BEEN WARNED!

    EXAMPLE USES

    I use this project for a number of processes on my main project, including:

    • Find methods that call "Erl" but contain lines that are not numbered
    • Finding missing GUIDs in methods where I reference a method GUID for logging/debugging purposes.
    • Finding duplicate GUIDs (copy & paste errors) in lists of field codes, and replacing duplicates with new GUIDs.
    • Finding methods that do not have an error handler active.
    • Finding methods where I accidentally used the NEW keyword instead of RC5 regfree instantiation (which would cause problems after deployment since the libraries aren't registered on the end-user's computer).


    KNOWN/EXPECTED PROBLEMS
    I've only tested this project on my own source code, so I expect problems when you try it on your own projects with your own coding styles. I'm happy to fix these issues, especially if you provide code examples.

    Handling source code saved in other locales has not been tested at all, and based on the conversation in this thread, I expect problems (possibly serious ones!).

    The project currently only supports FRM, BAS, CLS, and CTL files. More exotic things like data reports, web classes, property pages, etc... are not yet supported.

    PRE-REQUISITES


    IMPORTANT: For VBPCRE2 you will need to ensure that pcre2-16.dll is saved alongside VbPcre2.dll and then register VbPcre2.dll.

    For Krool's OCXs you will need to register both OCX files.

    For Olaf's vbRichClient5, you will need to ensure you have registered it properly using the .BAT file included with the binary distribution.

    USAGE
    There's an included example processor class (CExampleProcessor) included in the project that demonstrates some of the features of the project. The example processor finds methods that do NOT include an "ON ERROR GOTO <label>" line, so it is useful for finding places you forgot to include an error handler.

    To try it out, start the project then click Select Files and select any VBP or VBG file.

    Once the list of files is parsed and populated, click the Run button and wait while the source files are processed by the CExampleProcessor class.

    After processing is complete, click Copy to Clipboard to copy the report/log to the clipboard and paste it into your favourite text editor for review. In particular, look for WARNING lines.

    BUILDING YOUR OWN SOURCE CODE PROCESSORS

    NOTE: I'll expand on these instructions as the project matures.

    To build your own source processor, add a class to the main VB6SourceProcessor project and implement the ISourceProcessor interface in the new class. In the ISourceProcessor_ProcessFile method, perform whatever physical line level pattern matching & processing you desire.

    Remember to instantiate your class and call .ProcessFile on it in the frmMain.cmdRun_Click event!

    FUTURE ENHANCEMENTS
    1. This project only has a concept of physical lines right now, but I'd like to add the ability to handle "logical" lines (for example, multiple lines ending in an underscore would become a single logical line, and a single line with multiple statements separated by a colon would be considered multiple logical lines). ADDED IN LATEST VERSION
    2. I would like to offer the ability to backup original source files before writing modified versions back to disk.
    3. I would like to add the ability to filter log messages by level (INFO, IMPORTANT, WARNING, ERROR), and maybe by keyword.
    4. I want to add the ability to cancel processing.


    I imagine more enhancement ideas will come as people start using the project, so please let me know if you have any ideas on how to make the project better.

    PROJECT SOURCE CODE

    LATEST SOURCE CODE HERE: VB6SourceProcessor.zip

    Hope some of you find this project useful!
    Last edited by jpbro; Jan 11th, 2019 at 08:52 PM. Reason: Updated Project Source

  2. #2
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Hi jpbo, nice to see you, I haven't tested your code yet, but it looks great. I'm currently trying to develop WebPage Builder with RC5.Cairo. After a few days, I'll test your code carefully. Thank you, jpbro.

    Merry Christmas and Happy New Year.

  3. #3

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Hi dreammanor - thanks for stopping by I haven't been posting too much at the forums lately as I've been busy with work, but I thought I'd take a day to clean up this old project of mine and post here as a small gift to the community. I hope you find it useful!

    Just please heed my warning - this project is potentially dangerous as it allows you to mess around with source code outside of the IDE, so please make sure you backup your code before you try anything out!

    I'd also love to see some small sample projects with Chinese characters to see how my project handles it - if you can post any small examples, I'd really appreciate it.

    Your web page builder project sounds very ambitious, and I wish you all the best.

    Merry Christmas and Happy New Year to you too, and best of luck to you and your family this coming year and beyond!

  4. #4
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Thank you jpbro, and wish you and your family good health and happiness forever.

  5. #5

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Updated to fix a problem handling Unicode paths.

  6. #6
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Hi jpbro, I tested your code today. The project can successfully open a VBP file, but when I press the "Run" button, the following error message appears:

    Could not initialize PCRE2 library! Last DLL Error: 0

    Error code:
    Code:
    Public Function RegexMatchLine(ByVal p_StartAtOneBasedLineIndex As Long, ByVal p_Regex As String, Optional ByVal p_CaseSensitive As Boolean) As Long
       ' Return Line # of first found match
    
       Dim ii As Long
    
       If mo_Regex Is Nothing Then
          Set mo_Regex = New VBPCRE2.cPcre2
       End If
    
       If p_CaseSensitive Then
          mo_Regex.IgnoreCase = False
       Else
          mo_Regex.IgnoreCase = True
       End If
    
       For ii = p_StartAtOneBasedLineIndex To Me.PhysicalLineCount
          Set mo_LastMatch = mo_Regex.Match(Me.PhysicalLineContent(ii), p_Regex)
          If mo_LastMatch.Count Then
             RegexMatchLine = ii
             Exit Function
          End If
       Next ii
    End Function
    After compiling into the executable file, the same error still occurs. (Note: DLL files are already registered correctly)
    Last edited by dreammanor; Dec 28th, 2018 at 07:04 AM.

  7. #7

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Hi dreammanor, thanks for trying out the project. Do you have the pcre2-16.dll file in the same folder as your registered VbPcre2.dll? pcre2-16.dll is available here: https://github.com/jpbro/VbPcre2/tree/master/bin

  8. #8

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    I've just update the project in the first post.

    It includes a number of improvements:

    • You will now see a warning when you click the Run button, just to make sure you understand the risks associated with using this tool:

      Name:  warning.png
Views: 320
Size:  21.1 KB
    • If you don't want to see that warning, you can disable it by returning FALSE in modOptions.optRequireRunConfirmation()
    • After parsing & processing files, a message is shown with a bit of information about the processing/parsing:

      Name:  finished.png
Views: 316
Size:  8.5 KB
    • There's a new tab based interface that better separates the File & Log grids.
    • I've added a bunch of comments to the source (more to come).
    • I've made an initial stab at parsing Physical Lines to Logical Lines - this is a work in progress, and there will likely be bugs. If you have any projects that generate ERRORs in the log after clicking the Run button I would love to see them so I can fix the parsing.


    Couple of thoughts:

    • I don't know if "parsing" is the right word for what I'm doing with the source files when determining the logical lines. I'm open to a correction here, and I will adjust the language in the project accordingly.
    • For logical lines, I've come up with the following list. Have I missed anything? Are there better/more correct names to use for any of these?


    List of Logical Line "Types"
    Code:
       logicallinetype_Empty = &H1& ' Blank line (empty or whitespace only)
       logicallinetype_Header = &H2& ' Stuff in the header section of an FRM file, or "Attributes"
       logicallinetype_Attribute = &H4& ' Hidden VB attribute (e.g. ProcedureID)
       logicallinetype_Option = &H8&  ' e.g. Option Explicit
       logicallinetype_ConditionalCompilation = &H10& ' e.g. #IF #ELSE #END IF
       logicallinetype_Comment = &H20&  ' Comment line
       logicallinetype_ApiDeclaration = &H40&  ' e.g. Declare Sleep lib "user32.dll"
       
       logicallinetype_EnumStart = &H80&  ' Start of an Enum e.g. Public Enum MyEnum
       logicallinetype_EnumMember = &H100&  ' elements/members of an Enum. E.g. enum_MyEnum1, enum_MyEnum2
       logicallinetype_EnumEnd = &H200&  ' End of en enum, e.g. "End Enum"
       
       logicallinetype_TypeStart = &H400&  ' Start of a UDT e.g. Public Type MyType
       logicallinetype_TypeMember = &H800&  ' Members of a UDT
       logicallinetype_TypeEnd = &H1000&  ' End of a UDT, e.g "End Type"
       
       logicallinetype_ConstantDeclaration = &H2000&   ' e.g. Private Const mc_MyConst As Long = 1
       logicallinetype_VariableDeclaration = &H4000&   ' e.g. Dim X As Long, Private Y As Integer
       logicallinetype_MethodStart = &H8000& ' Sub/Function/Property start line
       logicallinetype_MethodEnd = &H10000  ' Sub/Function/Property end line
       logicallinetype_Label = &H20000   ' Line label
       logicallinetype_Statement = &H40000    ' Regular code line/statement
    Last edited by jpbro; Dec 28th, 2018 at 05:17 PM.

  9. #9
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by jpbro View Post
    Hi dreammanor, thanks for trying out the project. Do you have the pcre2-16.dll file in the same folder as your registered VbPcre2.dll? pcre2-16.dll is available here: https://github.com/jpbro/VbPcre2/tree/master/bin
    Hi jpbro, Sorry, I only downloaded VBPCRE2.dll. I mistakenly thought that pcre2-16.dll is not needed on Win10. After downloading pcre2-16.dll, the project can run normally.

    I tested Krool's ComCtlsDemo.vbp with your tool, which took 172 seconds to complete the project analysis. If it's used to analyze my project, it may take a long time, maybe more than 30 minutes(I'll test my project after a few days). If the "Pause" and "Cancel" buttons can be added to the frmProgress window, that would be great.

    This tool is very useful, maybe I can apply it to my code editor in the future to check the syntax errors of golang and javascript. I still need to spend some time studying it, especially the use of regular expressions, I have learned several times, but always forget.

    Thank you, jpbro. Happy new Year!
    Last edited by dreammanor; Dec 28th, 2018 at 06:35 PM.

  10. #10
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    33,309

    Re: VB6 Automated Source Code Processing Helper

    I made a slight edit to the original post. For a long time now, this site has not allowed the posting of compiled binaries to any place other than the UtilityBank. Obviously that's a significant issue if you are linking to a site like GitHub, because you have no control over what is posted there. For that reason, we are currently allowing links to sites with binaries, as long as that site is GitHub. I removed the one link that wasn't GitHub. After all, anybody from this forum that can't find vbRichClient5 isn't really trying.
    My usual boring signature: Nothing

  11. #11

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by dreammanor View Post
    Hi jpbro, Sorry, I only downloaded VBPCRE2.dll. I mistakenly thought that pcre2-16.dll is not needed on Win10. After downloading pcre2-16.dll, the project can run normally.
    No need to apologize! I wasn't very clear in my original instructions. I've since updated them to let everyone know what files are needed. In any case, I'm glad that's what the problem was, and that it is now resolved.

    Quote Originally Posted by dreammanor View Post
    I tested Krool's ComCtlsDemo.vbp with your tool, which took 172 seconds to complete the project analysis. If it's used to analyze my project, it may take a long time, maybe more than 30 minutes(I'll test my project after a few days). If the "Pause" and "Cancel" buttons can be added to the frmProgress window, that would be great.
    Thanks for the performance notes - if you are using the latest version, it is definitely slower than before because of the new experimental "logical line" parsing. I think I will add an option to disable this, or at least perform that parsing lazily (for example, the first time you attempt to access Logical Line information in a CSourceInfo object, it could do the Logical Line parsing, but not before then). For many use cases, this will help performance significantly. Pause/Cancel are also good ideas, so I will add those too.

    Quote Originally Posted by dreammanor View Post
    This tool is very useful, maybe I can apply it to my code editor in the future to check the syntax errors of golang and javascript. I still need to spend some time studying it, especially the use of regular expressions, I have learned several times, but always forget.
    I hope you will find it useful going forward. Regarding the regexes, one of my planned enhancements is too add a bunch of useful regexes as constants (or perhaps wrapped behind friendly named Sub/Method calls). But if you have anything you are trying to accomplish, let me know and I will try to help. Happy New Year to you too!
    Last edited by jpbro; Jan 1st, 2019 at 06:43 PM.

  12. #12

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Just updated the latest source in the first post with some dramatic performance improvements.

    First, there's a new option that is enabled by default (optLazyParseLogicalLines). When this is enabled, parsing source files for logical lines will be deferred until the first access of a logical line related property instead of happening as soon as a source file is loaded and split into physical lines. This will dramatically improve the initial loading/populating of the source files list, and since there are currently no Logical Line related properties (since the feature is still experimental) then overall processing will be highly performant.

    NOTE: If you want to help me out, please disable the lazy parsing of logical lines option by returning FALSE from optLazyParseLogicalLines in modOptions. This will increase load times but it will help me find errors in my parsing, as I'm sure there are many!

    Second, most of the processing time in the example processor was due to inefficient regex scanning of lines in the backward direction. In the example processor, we find a Method start line, then find a matching Method end line, and the look backwards from the end to find an "ON ERROR GOTO <Label>" line. In the original implementation of the RegexMatchLineBackward method, searching would always work backward to the beginning of the file even if we only needed to search back to the Method start line. In projects with lots of lines and no (or few) ON ERROR GOTO <Label> lines, this would result in horrible performance.

    I've added an optional StopAfterOneBasedLineNumber parameter to the RegexMatchLineBackward method that we can set to a line number that we know is a logical place to stop looking for matches (e.g. a method starting line number - anything before that line is not relevant). This change results in a significant performance improvement in most cases. On my machine, processing Krool's common controls project took over 100 seconds before the change, and now takes under 3 seconds.

    Thanks to @dreammanor for reporting about his experience with the project that resulted in these improvements!
    Last edited by jpbro; Dec 29th, 2018 at 08:11 AM.

  13. #13

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Just updated the source in post #1 with a minor bug fix for the misreporting of the # of files on the "Files" tab (cosmetic issue only), and a new "Stats" tab that shows some rudimentary statistics for your project (I plan to expand on the stats reported here in the future).

  14. #14
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Hi jpbro, sorry for the late reply. I tested your project today. The test results are as follows:

    Old VB6SourceProcessor
    (1) Process Krool's ComCtlsDemo.vbp: (Project Files:76; Run-Log:11374)
    Time1(Select Files): 3.36 seconds;
    Time2(Run): 224.57 seconds

    (2) Process DreamManor's Large.vbp: (Project Files:567; Run-Log:47647)
    Time1(Select Files): 17.88 seconds;
    Time2(Run): 361.82 seconds

    New VB6SourceProcessor
    (1) Process Krool's ComCtlsDemo.vbp: (Project Files:76; Run-Log:11374)
    Time1(Select Files): 0.67 seconds;
    Time2(Run): 5.86 seconds
    (2) Process DreamManor's Large.vbp: (Project Files:567; Run-Log:47647)
    Time1(Select Files): 8.26 seconds;
    Time2(Run): 39.26 seconds

    The performance of the new VB6SourceProcessor is greatly improved. When processing Krool's ComCtlsDemo.vbp, it's almost 40 times faster; when processing my project, it's almost 10 times faster (my project is stored in U-Disk). Very nice, extremely grateful.

    In addition, Olaf has demonstrated a more concise and convenient timing test method:
    Code:
      New_c.Timing True   '--- Start timing
      
      '...
      '...
      '...
      
      MsgBox New_c.Timing   '---End timing and display the time spent --
    Last edited by dreammanor; Jan 4th, 2019 at 11:00 PM.

  15. #15

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Hi Dreammanor, thanks for re-running the tests and reporting the results. I'm glad you are seeing the same significant performance improvements

    Also re: New_C.Timing, thanks for that. I was aware of it, but I happen to prefer seeing the timing in seconds as opposed to milliseconds, which is why I usually do it the other way. I should probably just roll my own function for the formatting though.

    Happy New Year!

  16. #16

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    BIG update to the project (latest source available in the first post). I've added logical line processing to the project, so we can now handle things like line continuations and multiple statements separated by ":" characters on a single line. Performing matches against logical lines is more robust compared to matching against physical lines.

    Special thanks to LaVolpe for pointing out that I missed some logical line types (Static methods and Events in particular) and providing some general guidance.

    The current list of logical line types that this project detects and parses is as follows:

    Code:
    Public Enum e_LogicalLineType
       [_logicallinetype_AutoDetect] = 0
       
       logicallinetype_Empty = &H1& ' Blank line (empty or whitespace only)
       logicallinetype_Header = &H2& ' Stuff in the header section of an FRM file, or "Attributes"
       logicallinetype_Attribute = &H4& ' Hidden VB attribute (e.g. ProcedureID)
       logicallinetype_Option = &H8&  ' e.g. Option Explicit
       logicallinetype_ConditionalCompilation = &H10& ' e.g. #IF #ELSE #END IF
       logicallinetype_Comment = &H20&  ' Comment line
       logicallinetype_ApiDeclaration = &H40&  ' e.g. Declare Sleep lib "user32.dll"
       
       logicallinetype_EnumStart = &H80&  ' Start of Enum e.g. Public Enum MyEnum
       logicallinetype_EnumMember = &H100&  ' Members/Elements of Enum - enum_MyEnum1, enum_MyEnum2
       logicallinetype_EnumEnd = &H200&  ' End of Enum, e.g. "End Enum"
       
       logicallinetype_TypeStart = &H400&  ' Start of UDT e.g. Public Type MyType
       logicallinetype_TypeMember = &H800&  ' Elements/Members of a UDT
       logicallinetype_TypeEnd = &H1000&  ' End of a UDT, e.g "End Type"
       
       logicallinetype_EventDeclaration = &H2000& ' e.g. Public Event MyEvent()
       logicallinetype_ConstantDeclaration = &H4000&   ' e.g. Private Const mc_MyConst As Long = 1
       logicallinetype_VariableDeclaration = &H8000&   ' e.g. Dim X As Long, Private Y As Integer
       logicallinetype_MethodStart = &H10000  ' Sub/Function/Property start line
       logicallinetype_MethodEnd = &H20000  ' Sub/Function/Property end line
       logicallinetype_Label = &H40000   ' Line label
       
       logicallinetype_OnErrorGotoLabel
       logicallinetype_OnErrorGotoZero
       logicallinetype_OnErrorGotoNegativeOne
       logicallinetype_OnErrorGotoResumeNext
       
       logicallinetype_Resume  ' E.g. Resume Next
       
       logicallinetype_Statement = &H80000    ' Regular code line/statement
       
       ' Special "shortcut" enums that combine multiple logical line types
       logicallinetype_MultiHidden = logicallinetype_Header Or logicallinetype_Attribute
       
       logicallinetype_MultiVisible = logicallinetype_Option Or logicallinetype_ConditionalCompilation Or logicallinetype_Comment Or logicallinetype_EventDeclaration Or logicallinetype_EnumStart Or logicallinetype_EnumMember Or logicallinetype_EnumEnd Or logicallinetype_TypeStart Or logicallinetype_TypeMember Or logicallinetype_TypeEnd Or logicallinetype_ConstantDeclaration Or logicallinetype_EventDeclaration Or logicallinetype_VariableDeclaration Or logicallinetype_MethodStart Or logicallinetype_MethodEnd Or logicallinetype_Statement
       
       logicallinetype_MultiAll = &HFFFFFFFF
    End Enum
    Currently logical lines are read-only - so you can run matches against them, but you can't modify them. I will be adding the ability to modify logical lines as soon as I can.

    There are a bunch of new methods available on the CSourceInfo class now:

    FindNextLogicalLineType - Detect a logical line by type (as in the the enum list above). You can combine logical line enum types to detect the next matching type from a list of types.

    FindNextLogicalLineType - Same as above but in the backward looking direction.

    RegexMatchLogicalLine - Perform a regex match against a range of logical lines. You can also limit the logical line types to match against by passing an e_LogicalLineType enum (or combination of enums for multiple line type matches).

    LogicalLineCount - Get a count of the number of logical lines in a module.

    LogicalLineContent - Get the content of a specific logical line.

    LogicalLineTypeInRange - Return TRUE if a passed logical line type enum is present within a range of lines.

    MethodNameFromLogicalLine - Returns the name of the method to which the passed logical line number belongs.

    That's it for now - there's almost guaranteed to be problems with the logical line parsing and I would love to get examples from anyone who runs into problems so I can fix them.

    Enjoy
    Last edited by jpbro; Jan 11th, 2019 at 04:50 PM.

  17. #17

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    I should note that the Logical Line processing will slow things down a bit again, although it will produce more correct results over a wider variety of coding styles.

    Processing Krool's project against logical lines now takes about 80 seconds on my machine. Still decent considering all the extra benefits we will get from making logical line processing available.

    That said, you can still use the older/faster physical line processing methods if you don't need to consider things like line continuations & multiple statements per line that have been separated by a ":" for you processing needs. By default Logical Lines won't be parsed unless you call a logical line method, so there's no performance hit against the older physical line methods if the logical line methods aren't used.

  18. #18
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    18,002

    Re: VB6 Automated Source Code Processing Helper

    jpbro, couple of observations

    You may be familiar with this reference, if not here it is. Describes in wonky terms how VB interprets stuff like physical and logical lines and much more. https://msdn.microsoft.com/en-us/library/ee199815.aspx

    This isn't meant to nit picky and I don't think I fully understand your RegEx expressions to say what I'm about to jot down isn't already handled.

    1. Attempting to find 2+ words in a single InStr() call can fail. This is because VB allows continuations in just about every way you can imagine with only a few exceptions: no continuations for enum members and obviously cannot split a "word/token". So if looking for something like "end type", "property get" can fail because those words may be split on different lines

    2. Think you are missing "Implements" statement detection, if you intended to trap these too.

    3. Variables in a declarations section can also be preceded with Public and Global. I see that you are testing for Dim, Private & Static

    4. DefType statements? https://docs.microsoft.com/en-us/pre...421(v%3dvs.60)

    5. Don't know any modern coder that uses this, but another type of comment block uses the keyword "Rem". When used, it must start a statement. Can't be appended to a normal statement.
    i.e., this is invalid: X = Y Rem Some Comments
    but this is valid: X = Y: Rem Some Comments

    6. Don't know if you want to address these: On [expression] GoTo labelA, labelB, etc and/or On [expression] GoSub labelC, labelD, etc and/or Gosub. If you do handle GoSub calls, might want to treat them like a block also, since the "sub" needs a "Return" statement.
    https://docs.microsoft.com/en-us/pre...175(v%3dvs.60)
    https://docs.microsoft.com/en-us/pre...378(v%3dvs.60)

    7. Just a reminder: "With" statements. Briefly talked about it before

    A coder's fun project is never finished, is it?
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  19. #19

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Hi LaVolpe,

    First of all, thank you very much for taking the time to look at the project and provide criticism. It is very appreciated, and don't worry about being nit-picky with me, I'm happy to have my nits picked

    [QUOTE=LaVolpe;5349033]You may be familiar with this reference, if not here it is. Describes in wonky terms how VB interprets stuff like physical and logical lines and much more. https://msdn.microsoft.com/en-us/library/ee199815.aspx

    I was not aware of that reference, thanks for providing it. I somewhat shamefully admit that my attempts at parsing/interpreting VB6 source is based almost entirely on the way I code. I also somewhat shamefully admit that my style of coding is not based on any formal education, just many years of hacking away, so my codebase is definitely an inadequate sample to base a parser/interpreter on alone. I think I should have been clearer that this project is a work in progress, so I will make a note of that in the first post.

    As an aside - my lack of formal education and the fact that I program in different languages from time to time has probably led to some "bad" VB6 nomenclature on my part. Things like my use of the word "method" vs. "procedure" and some other language I've used is probably incorrect or confusing. This project was originally developed in a more basic form some years ago just to help ensure some QA with my own code before compilation/distrubution, so there will be some growing pains as I make it more generic and useful for all.

    Quote Originally Posted by LaVolpe View Post
    I don't think I fully understand your RegEx expressions to say what I'm about to jot down isn't already handled.
    Based on your notes below, I think you've understood them enough. There's the old quote about Regexes that I like:

    Some people, when confronted with a problem, think "I know, I'll use regular expressions." Now they have two problems.
    But for whatever reason I have a fondness for regexes...probably related to some mental pathology that keeps me writing new software in VB6 at this late date

    Quote Originally Posted by LaVolpe View Post
    1. Attempting to find 2+ words in a single InStr() call can fail. This is because VB allows continuations in just about every way you can imagine with only a few exceptions: no continuations for enum members and obviously cannot split a "word/token". So if looking for something like "end type", "property get" can fail because those words may be split on different lines
    I'll review my code, but I thought I was only passing Logical Lines as parsed and normalized units that couldn't fail like this. I may be wrong about that though, so I will double-check all my tests.

    Quote Originally Posted by LaVolpe View Post
    2. Think you are missing "Implements" statement detection, if you intended to trap these too.
    You are 100% correct - those would be interpreted as just "Statements" but I agree I should have a logical line type for "Implements" lines, thank you

    Quote Originally Posted by LaVolpe View Post
    3. Variables in a declarations section can also be preceded with Public and Global. I see that you are testing for Dim, Private & Static
    Well that's a "duh" on my part (especially missing "Public"....I don't use "Global" myself, but missing "Public" was just a flub). Thanks again!

    Quote Originally Posted by LaVolpe View Post
    Definitely an omission on my part, I've seen them but forgot they existed. Thanks again! Maybe I should save the thanks to the end

    Quote Originally Posted by LaVolpe View Post
    5. Don't know any modern coder that uses this, but another type of comment block uses the keyword "Rem". When used, it must start a statement. Can't be appended to a normal statement.
    i.e., this is invalid: X = Y Rem Some Comments
    but this is valid: X = Y: Rem Some Comments
    Totally forgot about "REM" statements! I haven't used them since my Commodore 64 days

    Quote Originally Posted by LaVolpe View Post
    6. Don't know if you want to address these: On [expression] GoTo labelA, labelB, etc and/or On [expression] GoSub labelC, labelD, etc and/or Gosub. If you do handle GoSub calls, might want to treat them like a block also, since the "sub" needs a "Return" statement.
    https://docs.microsoft.com/en-us/pre...175(v%3dvs.60)
    https://docs.microsoft.com/en-us/pre...378(v%3dvs.60)
    GoSubs are another one I forgot about because I don't use them, but thanks for the reminder.

    Quote Originally Posted by LaVolpe View Post
    7. Just a reminder: "With" statements. Briefly talked about it before
    That's yet another "duh" on my part...I use "With" blocks all the time and just forgot to implement tests for them.

    Looks like it is time to get back to work!

    Quote Originally Posted by LaVolpe View Post
    A coder's fun project is never finished, is it?
    Now that's the truth! But I wouldn't have it any other way...I caught the programming "bug" when I was 5 and was typing out "Compute!" magazine programs on my VIC 20 (with the help of my Mom who did most of the typing while I recited the lines) and I've been working on "unfinished" projects ever since. Maybe that's one of the reasons that Jeff Atwood's concept of The Infinite Version resonates with me.

    Anyway, I would like to extend a final proper "Thank you very much!" for taking the time to take a look at my project and provide such valuable feedback! It's truly a privilege to get advice from VB6 Masters such as yourself.

  20. #20
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    18,002

    Re: VB6 Automated Source Code Processing Helper

    You're welcome, glad I could pick your nits -- jeez that sounds so wrong; maybe my attempt at humor stops there.

    Compute! Mag? Mine was PC Computing. Loved playing with those DOS functions found towards end of magazine. Those required you to type hex programs using EdLin to create .com files directly from the DOS prompt. Hmmm, or was that Compute! magazine? I had subscriptions for both.
    Last edited by LaVolpe; Jan 11th, 2019 at 10:59 PM.
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  21. #21

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by LaVolpe View Post
    You're welcome, glad I could pick your nits -- jeez that sounds so wrong; maybe my attempt at humor stops there.
    LOL, agreed that that line of humour might be done

    Quote Originally Posted by LaVolpe View Post
    Compute! Mag? Mine was PC Computing. Loved playing with those DOS functions found towards end of magazine. Those required you to type hex programs using EdLin to create .com files directly from the DOS prompt. Hmmm, or was that Compute! magazine? I had subscriptions for both.
    Now that I think of it, I actually had "Compute!'s Gazette", which was the Commodore version of Compute!. Probably the same format, just with different focus. the VIC20 and Commode 64 both booted up to BASIC, so all of the code in those magazines was BASIC code - normally with a tonne of "DATA" statements followed by "random" numbers (to a young brain they seemed random at least - that part always seemed like magic). Getting access to .RES files sure beat DATA statements. PS: I just checked, and VB6 doesn't support DATA statements I came late to the PC actually, I think I got my first PC around 1997 or so. I stuck with Commodore long past any rational point (VIC20 > C64/128 > Amiga).

    Cheers to those old systems that got us here!

    PS: I'm almost done some changes to my code to address your comments, thanks again for taking the time to provide them!

  22. #22

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Well that's a bummer - taking into account all the additional Logical Line types, I'm coming to 33 types. I've been using bitwise comparisons in order to allow for combining logical line types for matches, limiting me to 32 enums. I'm going to have to rethink this a bit.

  23. #23
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Hi jpbro, your project is great. After I finish my work, I'll study and test it carefully.

  24. #24
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    18,002

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by jpbro View Post
    Well that's a bummer - taking into account all the additional Logical Line types, I'm coming to 33 types. I've been using bitwise comparisons in order to allow for combining logical line types for matches, limiting me to 32 enums. I'm going to have to rethink this a bit.
    Well, you have a few you've designated for "ends", i.e., type end, enum end, etc

    You can use just one bit for a generic "end block" , example:
    Code:
       logicallinetype_EndBlock = &H80000000
       logicallinetype_EnumStart = &H80&
       logicallinetype_EnumMember = &H100&  
       logicallinetype_EnumEnd = logicallinetype_EnumStart Or logicallinetype_EndBlock
       
       logicallinetype_TypeStart = &H400&
       logicallinetype_TypeMember = &H800&  ' Elements/Members of a UDT
       logicallinetype_TypeEnd = logicallinetype_TypeStart Or logicallinetype_EndBlock 
       
       logicallinetype_MethodStart = &H10000  ' Sub/Function/Property start line
       logicallinetype_MethodEnd = logicallinetype_MethodStart Or logicallinetype_EndBlock
    likewise, you can free up another bit by using just one bit for both the enum member & type member, similar logic as above, i.e., logicallinetype_TypeMember = logicallinetype_TypeStart Or logicallinetype_Member

    Edited: Above said, do you need to OR any of the line types to begin with? In other words, would you be using values like logicallinetype_TypeStart Or logicallinetype_EnumStart or other combinations? If not, then you have 2 billion possibilities left: just apply values 1-n vs. bits.

    In my projects, sometimes I'll do both: I may have a block of enum values that will be OR'd together and a block that will never be, but want to keep them in the same Enumeration. So, I'll separate the blocks so that neither will overlap each other, i.e.,
    Code:
    Enum Whatever
       member_notOr1 = 1    ' start not-OR values: this block of numbers will never be OR'd with each other
       member_notOr2
       member_notOr3
       member_notOr4
       ... etc
       member_canOr1 = &H100  ' start OR values: 0 to 255 are  reserved for "not Or" values
       member_canOr2 = &H200  ' this block can be OR'd with each other and/or the previous block if needed
       member_canOr3 = &H400
       ... etc
    End Enum
    Last edited by LaVolpe; Jan 15th, 2019 at 08:03 AM.
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  25. #25

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    [QUOTE=LaVolpe;5349921]Well, you have a few you've designated for "ends", i.e., type end, enum end, etc

    You can use just one bit for a generic "end block" , example:
    ...
    likewise, you can free up another bit by using just one bit for both the enum member & type member, similar logic as above, i.e., logicallinetype_TypeMember = logicallinetype_TypeStart Or logicallinetype_Member

    That was smart, thanks for that tip

    Quote Originally Posted by LaVolpe View Post
    Edited: Above said, do you need to OR any of the line types to begin with? In other words, would you be using values like logicallinetype_TypeStart Or logicallinetype_EnumStart or other combinations? If not, then you have 2 billion possibilities left: just apply values 1-n vs. bits.
    I did want to keep it as an option to OR line types together for matching purposes. For example, you could do a regex match on just logicallinetye_TypeStart Or logicallinetype_EnumStart only to make sure that some kind of naming convention is adhered to. It saves time on the matching because the recordset to match against will only include TypeStart and EnumStart lines. Now it might be rare for this to happen, but I think there are cases where it can be useful.

    Quote Originally Posted by LaVolpe View Post
    In my projects, sometimes I'll do both: I may have a block of enum values that will be OR'd together and a block that will never be, but want to keep them in the same Enumeration. So, I'll separate the blocks so that neither will overlap each other, i.e.,
    Code:
    Enum Whatever
       member_notOr1 = 1    ' start not-OR values: this block of numbers will never be OR'd with each other
       member_notOr2
       member_notOr3
       member_notOr4
       ... etc
       member_canOr1 = &H100  ' start OR values: 0 to 255 are  reserved for "not Or" values
       member_canOr2 = &H200  ' this block can be OR'd with each other and/or the previous block if needed
       member_canOr3 = &H400
       ... etc
    End Enum
    That's a smart approach too, I'll have to give some more thought as to whether I can split up my enums into ORable/non-ORable blocks. Thanks again for all the great advice!

  26. #26

    Thread Starter
    Frenzied Member
    Join Date
    Aug 2010
    Location
    Canada
    Posts
    1,221

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by dreammanor View Post
    Hi jpbro, your project is great. After I finish my work, I'll study and test it carefully.
    Thanks for the kind words dreammanor - I hope you eventually find the project useful, and I'm looking forward to getting feedback from you as you try it out.

  27. #27
    Frenzied Member
    Join Date
    Sep 2012
    Posts
    1,360

    Re: VB6 Automated Source Code Processing Helper

    Quote Originally Posted by jpbro View Post
    Thanks for the kind words dreammanor - I hope you eventually find the project useful, and I'm looking forward to getting feedback from you as you try it out.
    I have a very accurate feeling about the technology I need. When I first saw VBFastCGI, I knew that it was what I had been searching for for a long time, but I only started using it a year later. Your VB6SourceProcessor is exactly what I need, but I need to wait until my current project is done before I start using it. Thank you, jpbro. Your product always has a unique idea.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width