Out of memory error due to project size (using RC6)

[tmighty2]

Hello!

I use RC6 in all my projects.
There is one project that is really big, and it seems to me that the "Out of memory" error which I could not really track down over the last 5 years and which keeps reoccuring even though I totally restructured the project might stem from the fact that there is a real compiler error in my code, but due to the size of my project + the size of RC6, VB6 is unable to jump to the error as normally to indicate where exactely it is.

Perhaps this is a question to Olaf as he might be the biggest user of RC6:

Do you think my assumption is plausible? If yes, do you have any suggestion for me?

Thank you!

[wqweto]

> Do you think my assumption is plausible? If yes, do you have any suggestion for me?

Very much the case with big projects. RC6 might not be the sole culprit (if at all). Try compiling with line numbers removed. Try removing big (1000s of LOC) comments. Try using As Object instead of As MyClass on public methods to reduce cyclic references on multi-project solutions.

I recently had to replace stock link.exe (version 6.0) with version 7.1 (from some later DDK) on the build server just to be able to link one of our bigger COM components which was failing with a suprious error at link time.

cheers,
</wqw>

[tmighty2]

Thank you.
Do you also have any suggestion regarding #If compiler statements?
Are they like comments and rather contributing to the problem than helping?

[VanGoghGaming]

Does that new linker provide superior executables (like with newer ASM instructions that were not available for older CPUs)?

[vbwins]

Does that new linker provide superior executables (like with newer ASM instructions that were not available for older CPUs)?

My question also but I doubt it.

[wqweto]

Thank you.
Do you also have any suggestion regarding #If compiler statements?
Are they like comments and rather contributing to the problem than helping?

Not sure about conditional compilation because we don't use it for large chunks of code, just for couple of consts per project e.g. APP_EDITION = Express vs Professional vs Enterprise, etc.

> Does that new linker provide superior executables (like with newer ASM instructions that were not available for older CPUs)?

Don't think the newer version adds anything of value. I just needed *something* to compile the final binary, something which works on Windows Server 2003.

cheers,
</wqw>

[vbwins]

It is possible that out of memory could actually be from string allocation. Are you running the exe on 32 or 64 bit? Are you linked LAA?

How easy is this to reproduce?

As a long shot try adding this to your main form and call it once at startup

private declare function SetOaNoCache lib "oleaut32.dll" () as long

Call it like

SetOaNoCache

This turns off the small string cache in OLE

By chance I was playing around with this today. I included it in one of my large multithreaded projects when using a memory leak checker years ago and decided to see what it was costing me in terms of performance which was a bit.

To my suprise when I tried this in the lab it increased the commited memory of my project by 25%. In prod with a large workload this pushed the thing over the edge and bang. Out of memory and out of string space errors. So back in it went.

[tmighty2]

It's not a runtime error, but compile / IDE error.

[jpbro]

I recently had to replace stock link.exe (version 6.0) with version 7.1 (from some later DDK) on the build server just to be able to link one of our bigger COM components which was failing with a suprious error at link time.

Just out of curiosity, was this the error you were getting in your large project?

: warning C4761: integral size mismatch in argument; conversion supplied
: fatal error C1001: INTERNAL COMPILER ERROR
      (compiler file 'E:\8783\vc98\p2\src\P2\main.c', line 494)

I bump into this one with one of my large UserControls and I have to keep refactoring bits to get it under the size threshold to compile. If replacing link.exe with 7.1 would fix this problem it would save me a bit of trouble. Thanks

[wqweto]

Just out of curiosity, was this the error you were getting in your large project?

: warning C4761: integral size mismatch in argument; conversion supplied
: fatal error C1001: INTERNAL COMPILER ERROR
      (compiler file 'E:\8783\vc98\p2\src\P2\main.c', line 494)

I bump into this one with one of my large UserControls and I have to keep refactoring bits to get it under the size threshold to compile. If replacing link.exe with 7.1 would fix this problem it would save me a bit of trouble. Thanks

This looks like C2 compiler internal error. The linker error sounded a bit different as it had .obj files included.

cheers,
</wqw>

[Arnoutdv]

There must be different versions of C2.exe lingering around.
I have 2 versions:
1. C:\Program Files (x86)\Microsoft Visual Studio\VB98\c2.exe
2000-07-15, Version: 6.0.8783.0
2. Installation folder
1998-06-17: Versio 6.0.8084.0

[jpbro]

This looks like C2 compiler internal error. The linker error sounded a bit different as it had .obj files included.

cheers,
</wqw>

Gotcha, thanks...back to refactoring!

[jpbro]

There must be different versions of C2.exe lingering around.
I have 2 versions:
1. C:\Program Files (x86)\Microsoft Visual Studio\VB98\c2.exe
2000-07-15, Version: 6.0.8783.0
2. Installation folder
1998-06-17: Versio 6.0.8084.0

FWIW I have the 6.0.8783 version installed.

[ThiagoPSanches]

I've faced this issue several times.
The easiest fix for me was moving some functions/subs from the Form/UserControl to a separate module.

[jpbro]

I've faced this issue several times.
The easiest fix for me was moving some functions/subs from the Form/UserControl to a separate module.

Thanks Thiago - that's also what I am doing now whenever I go over the threshold, was hoping I might not have to go through the trouble though!

Even better would be if users stopped asking for new features for this UC, but that's not likely

[Schmidt]

For the record...

The loading of the RC6-lib(s) will add roughly 5-7MB static (one-time) Mem-consumption to the hosting process.
(which is not all that much these days).

Other than that, the most probable reason for huuuge (unexpected) mem-consumption is perhaps:
- the usage of the Cairo.CreateSurface(..., Width, Height) -call
- ... when not using Pixel, but VB-Twips instead (via passing of ScaleWidth, ScaleHeight values)

This might happen in case one wants to create a fully Form- or UserControl-covering Surface-Object -
but forgetting to switch the Form. or Usercontrol.Scalemode to vbPixels beforehand.

...which in case of e.g. 15 TwipsPerPixel will then cause a Surface-Memory-allocation, which is:
15*15 = 225 times larger than necessary.

Might be helpful, to search VB.Form- and -Usercontrol-code for this kind of thing...

Olaf

[tmighty2]

Thanks for the contributions so far.
Apart from using RC6 like crazy, I also use Chilkat. A few days ago, I removed the Chilkat reference and used As Object everywhere. Since then, the out of memory errors have become less frequent — but they still occur.
I also applied the large address aware patch to VB6, but that didn’t change anything either.

Here is a little breakdown of what happened so far:

About 1.5 years ago, I started encountering silent crashes in compiled VB6 applications — no error message, no exception, just immediate termination at runtime. The IDE worked fine, and compilation succeeded without issues. The crashes happened only in the compiled EXE.

After a lot of debugging, I suspected the c32bppDIB OCX to be involved. Since I was using it across all of my projects, I spent over six months replacing it with RC6 Cairo as a modern alternative and hopefully safer alternative. But even after the complete migration, the silent crash still occurred.

At that point, I began restructuring my entire codebase, suspecting deeper issues. I had hundreds of Enums and Type definitions scattered across Forms and Modules. To consolidate and gain control, I moved them into a single interface definition and started compiling them into a type library.

This process exposed a lot of hidden problems — circular dependencies, broken declarations, incompatible field types. Cleaning those up improved the structure significantly, but the silent runtime crash remained.

So I built a tool that would automatically generate Enums and UDTs into the .idl, compile the .tlb, and then compile and run a VB6 project that uses it. The idea was to detect exactly when the resulting EXE would crash at runtime.

That tool eventually revealed that a certain combination of string and long in an UDT will compile fine, but the resulting EXE crashes immediately when run. Every time. No message. Nothing. Just exit.

This kind of failure is completely invisible to the VB6 IDE or compiler. Only the compiled binary shows the problem — and only when executed.

Once I identified and removed those structures, the silent crash was finally gone.

But then, in my largest project, I started hitting a new problem: “Out of memory” error in the IDE. And what makes it so strange — this project used to be even larger in the past, and compiled just fine.

Still investigating.

[VanGoghGaming]

UDTs containing strings are fine only if you do not pass them to API functions. Most likely that's where you encounter your crashes.

[tmighty2]

Please define "fine".

[VanGoghGaming]

"As Expected"! You haven't posted a code snippet that crashes for you so it's only an educated guess that your crashes involve UDTs with strings being passed as parameters for API functions.

[scprk]

I had the same issue (even without rc6). Out of memory came in my huge project 'cause I have crossed the upper limit for var and object in VB6 compiler. With CodeSmart I searched for function/sub unused or dead code, then I removed. Next I searched for vars with name used rarely and changed with a/b/c/... and so on. After this my huge project compile with success. I hope this 'll be useful.

[wqweto]

With bigger projects one has to impose some artificial limitations to keep codebase healthy. For instance we are running all our projects in IDE with “Break on All Errors” option set. This reveals errors “covered” with OERN which is a hideous practice to begin with. Our code is 100% not raising COM errors unless unexpected condition occurs, condition which needs being logged and handled (so it will stop filling logs).

Second artificial limitation we imposed is being able to hit End button in IDE and continue coding as if nothing happened. This took years to deal with. First we switched to MST for IDE-safe subclassing/timers (no hooks in IDE) and then we had to ASM thunk IUnknown::Release impl. for all our lightweight COM objects we used which was hard to recognize as crashing the IDE but obvious in hindsight.

[VanGoghGaming]

What thunk did you put in IUnknown::Release? Is Release even called when you click the End button?

[yereverluvinuncleber]

Has the OP tried compiling with TwinBasic yet?

Perhaps this may be a route out of your woes.

[wqweto]

What thunk did you put in IUnknown::Release? Is Release even called when you click the End button?

Release is called on every outstanding COM instance incl. VB's objects. It is the interpreter which is paused so you don't get Class_Terminate raised. When you put AddressOf of a VB procedure for Release impl. on End it cannot check if interpreter is paused before executing any p-code as the check would be some more p-code (it can but this has to be ASM thunk) and this usually brings the IDE down.

Here is snippet of our IPAO hook impl. redux, based on vbAccelerators one:

' ===========================================================================
' Light-weight object definition
' ===========================================================================

Public Type IPAOHookStruct
    lpVTable        As Long     'VTable pointer
    IPAORealPtr     As Long     'Weak-ref for forwarding calls
    CtlPtr          As Long     'Weak-ref for making Friend calls
    SinkPtr         As Long
    ThisPtr         As Long
    CtlName         As String
End Type

Private Function pvGetVTable() As Long
    Dim STR_RELEASE_THUNK       As String: STR_RELEASE_THUNK = "i1QkBItCBIsIUP9RCMIEAA==" ' 13.5.2020 20:15:19
    Const RELEASE_THUNK_SIZE    As Long = 16

    ' Set up the vTable for the interface and return a pointer to it
    With m_uVTable
        If .VTable(0) = 0 Then
            .VTable(0) = VBA.CLng(AddressOf QueryInterface)
            .VTable(1) = VBA.CLng(AddressOf AddRef)
            .VTable(2) = pvThunkAllocate(STR_RELEASE_THUNK, RELEASE_THUNK_SIZE)
            .VTable(3) = VBA.CLng(AddressOf GetWindow)
            .VTable(4) = VBA.CLng(AddressOf ContextSensitiveHelp)
            .VTable(5) = VBA.CLng(AddressOf TranslateAccelerator)
            .VTable(6) = VBA.CLng(AddressOf OnFrameWindowActivate)
            .VTable(7) = VBA.CLng(AddressOf OnDocWindowActivate)
            .VTable(8) = VBA.CLng(AddressOf ResizeBorder)
            .VTable(9) = VBA.CLng(AddressOf EnableModeless)
            '--- init guid
            With IID_IOleInPlaceActiveObject
               .Data1 = &H117
               .Data4(0) = &HC0
               .Data4(7) = &H46
            End With
        End If
        pvGetVTable = VarPtr(.VTable(0))
    End With
End Function

Private Function pvToIOleIPAO(ByVal lPtr As Long) As IOleInPlaceActiveObject
    Call vbaObjSetAddref(pvToIOleIPAO, lPtr)
End Function

'Private Function Release(This As IPAOHookStruct) As Long
'    Release = pvToIOleIPAO(This.IPAORealPtr).Release
'End Function

The Release ASM thunk is reimplementing VB6 Release procedure i.e. it's aiming this particular member variables layout. Here is the source code for the base64 string above.

cheers,
</wqw>

[tmighty2]

I have been able to resolve a reproduceable crash:


I had used

#If 

#Else

#End If

very often to rule out certain components as being responsible for the crashes.

On one form I did it in such a way that VB6 would crash each time it would try to run the project.
It looked like this:

Option Explicit
    
Public Event Finished()


Private m_sTranslationWorkingOn$

#If SUPPORTS_MAIL_DOWNLOADING Or SUPPORTS_MAIL_SENDING Then
Private m_bUnloadWhenDone As Boolean

Private m_bBusy As Boolean

Private m_UIDsOfAlreadyDownloadedMails As cHashD
Private m_UIDsOfMailsToBeDownloaded As cHashD
Private m_iCountEmailsToDownload&
Private m_iTimeWhenAllEmailsWereDownloadedToDetermineNextRound&
Public Event MailsToBeDownloadedAnalyzed(ByVal uCount As Long)

Private m_iLastTickWhenCheckingCanDo&
Private m_bCanDo As Boolean
Private m_iErrorCount&

Private m_sLastTopic$
Private m_sDownloadingXofY As String
Private m_iCountDownloaded&

Private m_CurTaskToKnowWhatWeAreJustDealingWith As eEmailTask
Private m_iCountMailsOnServer&
Private m_iRowIDSend&
Private m_Task As Object
Private m_cMailManSMTPForSendingOnly As Object
Private m_Imap As Object

Private WithEvents m_Parent As Form

Private m_MailAccount As udtMailAccount

Private Sub pGetCurrentMailAccount()

    #If USESCOMPONENTSA Then
        m_MailAccount = User.Settings.MailAccount
    #ElseIf ISPRIVATEAPP Then
        m_MailAccount = Settings.MailAccount
    #Else
        Stop
    #End If
End Sub

It becam obvious to me that there was a problem when I noticed that VB6 would more and more timer events when I click the component:



When I removed some of the #If Then's the crash problem was gone.

[tmighty2]

I also had situations where VB6 would crash upon pressing the start button in the IDE.
I tracked the error down to the following being responsible:

I often use Debug.Print, and it happens that I forget to remove it, so sometime the debug window is flooded by messages like

False
False
False
0
0
False
False

... and I don't quickly find out where it originates from.

That is when I decided to replace all Debug.Print with DebugPrintEx.
I created the following sub:

Public Sub DebugPrintEx(u As Variant)
   Debug.Assert False
End Sub

It works fine and lets me find the origin.

However, it turned out that just this sub in a certain class together with a certain module caused the IDE to reliably crash upon pressing the Start button.
It took me several months to stumble over the cause for this crash.

I don't know what I should make of it: Is there any learning experience from it? I have not found one except the fact that I want to use open source software only that I can debug properly.
In this case, I used any tool I could including WinDbg and fafalone's file watcher, but it would not help.

[tmighty2]

Does anybody know a way to quickly reveal such "crash corners"?

[wqweto]

> However, it turned out that just this sub in a certain class together with a certain module caused the IDE to reliably crash upon pressing the Start button.

That is weird and highly unlikely. Usually the “damage” is done in other places but it is just revealed by innoculous looking code. Corrupt memory, subclassing, dangling pointers are dangerous stuff but rarely a Debug.Assert can bring the IDE down.

[SearchingDataOnly]

A few years ago, when I had a project with more than 700 classes, the compiler would get a lot of inexplicable errors, including variables that weren't defined, but didn't show which line of code was wrong in which file.

In the end, I had to break this project down into three smaller projects (3 active-dlls), each containing no more than 600 classes.

[tmighty2]

A few years ago, when I had a project with more than 700 classes, the compiler would get a lot of inexplicable errors, including variables that weren't defined, but didn't show which line of code was wrong in which file.

In the end, I had to break this project down into three smaller projects (3 active-dlls), each containing no more than 600 classes.

Why so many classes if I may ask?

[VanGoghGaming]

I've run into a few situations where pressing F5 would crash the IDE because it's relying on old, possibly cached, data and addresses. Since then I've made it a habit to always press Ctrl-F5 for start with full compile every time.

[SearchingDataOnly]

Why so many classes if I may ask?

That's because at the time I was trying to translate the .Net core-libs (which has tens of thousands of classes) into the VB6 libs. When I translated 1,600 classes, I gave up. This is because VB6 is not suitable for very large projects.

[wqweto]

I've run into a few situations where pressing F5 would crash the IDE because it's relying on old, possibly cached, data and addresses. Since then I've made it a habit to always press Ctrl-F5 for start with full compile every time.

This can be alleviated by unchecking Background compile option (or something similar) which effectively turns F5 into an Ctrl-F5

[tmighty2]

I know but this does not help. It will run fine.

However, I had the idea of trying to convert my VB6 project to TB, and TB choked at this in an ActiveX exe:

Public Sub Add(uObj As Object, _
                  Optional uKeepFixedDistanceTo_Left As Boolean = False, _
                  Optional uKeepFixedDistanceTo_Top As Boolean = False, _
                  Optional uKeepFixedDistanceTo_Right As Boolean = False, _
                  Optional uKeepFixedDistanceTo_Bottom As Boolean = False, _
                  Optional ByVal uEnsureInitialPosition As Boolean = False, _
                  Optional ByVal uGrowOnly As Boolean = False, _
                 Optional ByVal uPreventOffscreen As Boolean = False)

I did forget to add ByRef / ByVal, and perhaps Optional in a class in an ActiveX Exe was a problem.

I removed the Optional keywords, and since then, the problem seems to be gone.

[VanGoghGaming]

All function parameters are initialized with their default values so you could spare a lot of typing by not including "= False" every time.

[tmighty2]

I have set the following error options in VB6 now, and since then I got pointed to 2 problems:
Public Function ....
Was not valid, and VB6 demanded Friend instead.

- Ask for saving changes
- Halt on all errors, and then I switched to only in classes.