[twinBASIC] Memory List Manager

[fafalone]

Memory List Manager v2.1

Memory List Manager is a small, lightweight utility to clear standby memory and flush caches, without needing to load a full heavyweight system resources app like SystemInformer, the code for which this project is based on. Windows keeps memory contents it thinks you might use again soon loaded and marked as standby, theoretically this memory is 'available' but memory management isn't always perfect, and high persistent standby and especially modified memory can cause reduced system performance, certain issues with some memory intensive apps or even lead to spurious out of memory errors and system destabilization.

Updates
Version 2.1.4 (29 Jun 2024)
-Added toggle for memory bar at top
-GUI will now update when jump list commands are used
-Always call UpdateMemoryInfo immediately after op
-Bug fix: Autooptmize taskbar command not working
-Bug fix: Taskbar command accelerators not supported.
-Bug fix: Mem free display mismatch
-Temp. workaround for exit while minimized failure

Version 2.0.2 (21 Jun 2024)
-Bug fix, debug MsgBox on exit still enabled. Built with new version info fix.
Version 2.0 (21 Jun 2024)
-Added graphical Memory Bar to visualize modified/standby usage
Can change colors, double click key (small circle) to set
-Added Auto-optimize option for common procedure
-Added option to monitor status and auto-exec AutoOptimize
-Added minimize to tray option with commands available from a popup menu when right-clicking the tray icon. (Vista+)
-New settings are saved to/loaded from registry
-Moved strings to constant list at top for easier translation.

Version 1.2: Bug fix: System memory free visual glitch and commit charge % calculated incorrectly.
Version 1.1: Bug in wait cursor code.

Requirements

-Windows XP or newer; some features require Windows 7 or Windows 8.1.
-Program must be run as administrator as it needs the SeProfileSingleProcessPrivilege for basic functions and SeIncreaseQuotaPrivilege for some.\

Build:
-twinBASIC Beta 553 or newer (Note: The manifest specifies requireAdministrator so running the built exe from the IDE will fail if tB is not also running as admin.)
-(Included in project file) Windows Development Library for twinBASIC v8.3.428 or newer. Note: To run from the IDE, the project must be built, and built again after any compiler restart. This is due to resource icons only being available from the compiled exe; the app loads them from the exe when running from the IDE.

Available commands:

Clear standby RAM: Clears standby memory of all priority levels.

Clear Low-Priority Standby RAM Only: Clear only the low priority standby memory. On Windows 10, this seems to mean Priority 0 only.

Flush modified RAM: Commits memory waiting to be written to disk.

Empty Working Sets: Empties system and user working set set memory to modified or standby. If you're going to use this, you should use it prior to clearing standby memory.

Extra:

Combine pages: De-duplicates certain memory contents.

Flush registry cache: Commits pending registry operations to disk.

Flush system file cache: Clears files being held in memory for fast access.

Command line and Jump list

In addition to the GUI, the above commands can be silently run without opening the app GUI by using the following command line switches:

/clearstdby - Clear standby memory
/clearlpstdby - Clear low-priority standby memory
/flushmod - Flush modified memory
/emptyws - Empty working sets.
/combine - Combine pages.
/flushreg - Flush registry cache
/flushfiles - Flush file cache

Note: Currently, only one command at a time is supported.

Further, on Windows 7 and above, the taskbar icon has a jump list to quickly access commands:



and on Windows Vista and above, there's a minimize to tray option; left-click to restore the UI, right-click for a context menu with all commands and exit:




How it works

These actions are done using the NtSetSystemInformation API; it's just straightforward calls:

    Public Function ClearStandby(Optional bLowPriority As Boolean = False) As NTSTATUS
        Dim nCmd As SYSTEM_MEMORY_LIST_COMMAND
        If bLowPriority Then
            nCmd = MemoryPurgeLowPriorityStandbyList
        Else
            nCmd = MemoryPurgeStandbyList
        End If
        Dim status As NTSTATUS
        SetCursor LoadCursor(0, IDC_WAIT)
        status = NtSetSystemInformation(SystemMemoryListInformation, nCmd, LenB(nCmd))
        SetCursor LoadCursor(0, IDC_ARROW)
        Return status
    End Function

    Public Function CombinePages(Optional pNumCombined As LongLong) As NTSTATUS
        Dim mci As MEMORY_COMBINE_INFORMATION_EX
        Dim status As NTSTATUS
        etCursor LoadCursor(0, IDC_WAIT)
        status = NtSetSystemInformation(SystemCombinePhysicalMemoryInformation, mci, LenB(Of MEMORY_COMBINE_INFORMATION_EX))
        pNumCombined = mci.PagesCombined
        SetCursor LoadCursor(0, IDC_ARROW)
        Return status
    End Function

    Public Function FlushRegistryCache() As NTSTATUS
        etCursor LoadCursor(0, IDC_WAIT)
        FlushRegistryCache = NtSetSystemInformation(SystemRegistryReconciliationInformation, ByVal 0&, 0&)
        SetCursor LoadCursor(0, IDC_ARROW)
    End Function

    Public Function FlushFileCache(Optional pSize As LongLong) As NTSTATUS
        Dim sfi As SYSTEM_FILECACHE_INFORMATION
        Dim sfiSet As SYSTEM_FILECACHE_INFORMATION
        Dim status As NTSTATUS
        Dim bRet As Byte
        Dim cb As Long
        etCursor LoadCursor(0, IDC_WAIT)
        status = RtlAdjustPrivilege(SE_INCREASE_QUOTA_PRIVILEGE, 1, 0, bRet)
        status = NtQuerySystemInformation(SystemFileCacheInformationEx, sfi, LenB(Of SYSTEM_FILECACHE_INFORMATION), cb)
        If NT_SUCCESS(status) Then
            sfiSet.MinimumWorkingSet = MAXSIZE_T
            sfiSet.MaximumWorkingSet = MAXSIZE_T
            status = NtSetSystemInformation(SystemFileCacheInformationEx, sfiSet, LenB(Of SYSTEM_FILECACHE_INFORMATION))
            pSize = sfi.CurrentSize
        End If
        SetCursor LoadCursor(0, IDC_ARROW)
        Return status
    End Function

Download from GitHub
Includes .twinproj project file, browsable exported source code, and binary builds. Also full readme and license.

[Thierry69]

Nice !

[yereverluvinuncleber]

Very nice. I once wrote a tool in DCL to flush the working sets of VMS system users (100+ realtime users) and setting the WSExtents more appropriately to match user's requirements. Ah, those were the days, when memory management was more manual.

[yereverluvinuncleber]

Can you stick a compiled binary up there on Github too, so that casual users can just try it as well as more committed developers. There is no problem in doing so on github, it is allowed.

[fafalone]

Can you stick a compiled binary up there on Github too, so that casual users can just try it as well as more committed developers. There is no problem in doing so on github, it is allowed.

I've included those yes; you have to click on 'Releases' on the right side column:



This and a few other tools like my event tracer, radio manager, and run as trusted installer, I maintain up to date binary releases too for that exact purpose, because they're of use as standalone applications for non-programmers too.

[yereverluvinuncleber]

Ah, very good. I should do that too, when I have something worth releasing. Love your stuff Faf.

[yereverluvinuncleber]

Faf,
Who would be the typical user of your memlist utility and in what situations would they use it?

[Zphere]

Very nice and useful tool. I also like it's minimalistic design.
Will try to report if necessary - thanks.

[fafalone]

Faf,
Who would be the typical user of your memlist utility and in what situations would they use it?

Anybody who doesn't reboot very often and uses memory intensive apps that fill up standby memory and cause the problems I mentioned, and who want a quicker path to clearing it than more heavyweight utils, that might also do things like spin up all your idle hard drives while querying unrelated system information. There's value in simple tools for just the task at hand vs going through something like System Informer or RAMMap that have these options buried several dialogs in.

I'll be using this extensively. But I made a polished UI for others and so it looks nicer for me too.

[fafalone]

Project Updated - Version 2.0



Version 2.0.2 (21 Jun 2024)
-Bug fix, debug MsgBox on exit still enabled. Built with new version info fix.
Version 2.0 (21 Jun 2024)
-Added graphical Memory Bar to visualize modified/standby usage
Can change colors, double click key (small circle) to set
-Added Auto-optimize option for common procedure
-Added option to monitor status and auto-exec AutoOptimize
-Added minimize to tray option with commands available from a popup menu when right-clicking the tray icon. (Vista+)
-New settings are saved to/loaded from registry
-Moved strings to constant list at top for easier translation.




Download from GitHub (source and binary builds available)

[yereverluvinuncleber]

Can we have an option to run with/without the graphics at the top? I preferred the elegance of the original without. I don't mind the visualisation but a right click option to hide/show it? For me, that would be better.

[fafalone]

Sure I can add that. Bit busy right now but I'll get to it this coming week.

[yereverluvinuncleber]

Perfect, I am in no rush.

[Zphere]

fafalone,
(bug?-) report for v1.2 and 2.02:
Env: Win11 23H2 and 64GB physical RAM.
There seems to bee something wrong with the calculated sizes at start of the programm as well as after its actions.
Example at start:
"Current free 4,18GB (93%)"
"Total 63,79GB".
The percentage amount of free ram is correct (ususally 93-94 percent), but shown is the amount of Bytes/GB used (4.1GB in the example above).
Sizes (and therefore lengths) of the status bars as well as the corresponding icons on top are also flipped (v.2.02).

(bug?-) report for v2.02:
whenever you minimise the proggy in the system tray or task bar,
clicking the context menu entry "Auto-optimize" is quitted with an "Invalid command line" message box.
The other entries work as expected.

[Zphere]

...another one I just discovered (both versions) when minimising to taskbar:
- option "Memory List Manager" does not bring back the GUI
- option "Close task" works as expected (process vanishes from taskmgr)
- option "Close window" (which should also close the program) does not work. Memory List Manager still remains as open process in taskmgr.

[fafalone]

The "Memory List Manager" you see in the bottom group is a Windows shortcut to open a new instance of the program. I'll see if I can get the previous instance detector to restore the old one right when its minimized; but the command goes to Windows, not the program.

The 'Close window' not working when it's minimized is currently impacting all tB apps; I'm looking for a workaround but might need to wait for Wayne to fix it.

Can you elaborate on what you mean with the status bars? They're not meant to show the same thing as the "Current free" label; indeed the "Current free" as calculated in the label is unrelated to "Free" in the bar, as in the bar "Free" excludes standby etc and comes from SYSTEM_MEMORY_LIST_INFORMATION.FreePageCount, not .NumberOfPhysicalPages - .AvailablePages. It's meant to approximate the bar in the system Resource Monitor app; and with the exception that RM has some 3rd way of calculating free that puts it larger than .FreePageCount but way less than normal "Free", it seems to be in the right proportions--- it's meant to visualize the proportion of the 4 categories it shows in relation to eachother only; so if it's doing that it's right but if not let me know.

The other issues have been fixed.

I'm running down some odd behavior when dpi aware is enabled and hoping Wayne takes a look at the exit issue, so I'll probably wait another day or two to post the update. In the mean time, in Sub Main, change If Len(sCmd) < 8 Then 'Length less than shortest command to < 4 to fix the invalid command line, and the right number for the % should be lblFree.Caption = FormatSizeStd((sbi.NumberOfPhysicalPages - spi.AvailablePages) * PAGE_SIZE) & " (" & Round((CSng(sbi.NumberOfPhysicalPages - spi.AvailablePages) / CSng(sbi.NumberOfPhysicalPages) * 100), 0) & "%)" in frmMain.UpdateMemoryInfo.

[Zphere]

Okay, maybe pics tell more than words:
IDE: TwinBasic Beta v.568, switched to win64 compiling for the resulting exe.

Attachment 192094

I adapted rounding to two digits after the comma which helps to get more
precise results for crosschecking calculations and which is aligned with the
number format of other user information you provided.

As you see total phys. ram is 63,79GB.
For me (and my calculator :-)) 90,36% of 63,79GB are 57,64GB (93,79/100 * 90,36).
Therefore, "Current free" should show "57,64 GB (90,36%)" NOT 6,15 GB.
Programmatically we get "Current free" by
"FormatSizeStd(spi.AvailablePages * PAGE_SIZE)" (=57,94GB which is correct).

Without "disturbing" a user with detailed information, the total amount
of currently used RAM should be 63,79GB - 57,64GB = 6,15GB (which is correctly calculated).

Back to the bars and dots with number informations:
Problem is the "Free:" propotion of the status bar.
It shows 29,68 MB which is definitely wrong.
It must be the 57,64GB mentioned above.
Due to its small byte amount the other bars, particulary the "Used:" bar
grows to the right and visualises a wrong propotion regarding to the "Free:" bar
(that's what I meant with "flipped" in my previous post).

From my understanding the complete code line for this label should be:
lblFree.Caption = FormatSizeStd(spi.AvailablePages * PAGE_SIZE) & " (" & Round((CSng(spi.AvailablePages) / CSng(sbi.NumberOfPhysicalPages)) * 100, 2) & "%)"
- to get the current free RAM.

BTW, I've seen that the displayed amount of "Free:" RAM near the dot is the same as shown/calculated in the "Memory Lists" frame; Label "Free".
Maybe you meant freed? - which is certainly not the same as the free available RAM.

I hope my report is of help.
Again, I really like your proggy and would be very pleased to use it as trustworthy tool in my belt.

[Zphere]

I forgot:
I changed the code line to your proposed value ("If Len(sCmd) < 4 Then").
After minimising to the taskbar and clicking "Auto-optimize" the TwinBasic 64Bit nag-screen appears
but the GUI does not pop-up. I have to left-click the icon in the taskbar.
After appearance of the GUI I see that nothing has changed
(values are the same as before and it seems that the Auto-optimize command was never executed).

[fafalone]

I can't see the pic you posted; VBF has a longstanding bug where you have to post it from the 'Go advanced' page.

Ok so for the Current free label I see what you mean; what you posted it right.

For the bar; it's not the same definition of "free". "Free" in "Current free" means what the *system* considers free: Available - Used. The bar is meant to show what is *actually* free-- memory that is neither used, modified, or standby. SYSTEM_MEMORY_LIST_INFORMATION.FreePageCount. That's the whole point of this tool-- your system slows down when you appear to have free memory because it's not really free, it's used as modified or standby memory-- it's *available*, not free. Perhaps I should change the 'Current free' label to 'Current available'.

I forgot:
I changed the code line to your proposed value ("If Len(sCmd) < 4 Then").
After minimising to the taskbar and clicking "Auto-optimize" the TwinBasic 64Bit nag-screen appears
but the GUI does not pop-up. I have to left-click the icon in the taskbar.
After appearance of the GUI I see that nothing has changed
(values are the same as before and it seems that the Auto-optimize command was never executed).

The way jump lists work is they launch a new instance of the program using a command line associated with the item clicked. The GUI is not meant to pop up; the point of that is to use the program while it's minimized. Sub Main() checks the command line, and if present, executes the command(s) and quits without launching the GUI. If there's no command line it shows the GUI. You'd have to click update info to see the changes because the first instance doesn't currently know the 2nd was launched... I'll add some kind of IPC mechanism to do it automatically for the next version.

[Zphere]

I can't see the pic you posted; VBF has a longstanding bug where you have to post it from the 'Go advanced' page.

Apologies, I didn't know.

For the bar; it's not the same definition of "free". "Free" in "Current free" means what the *system* considers free:...

fafalone, I perfectly understand what you're trying to say because I'm really quite familar with system RAM settings.
What I was covertly trying to say in my posts is that the bars and dots (and also the associated numerical information) are linguistically
misleading implicitly convey something different to the user than what is actually meant.

The GUI is not meant to pop up...

Sorry, I got that mixed up. Please ignore this part of my previous post.
However, then I would like to post a request (now I understand the "Update Info" button).
When I click Auto-optimise within the GUI everything gets updated.
As a user (and after the GUI "experience" which comes first) I expect that this will be also done if I click the corresponding context menu option when minimised to taskbar.
I also expect an updated GUI then when I decide to bring back the GUI some later.

[fafalone]

Apologies, I didn't know.


fafalone, I perfectly understand what you're trying to say because I'm really quite familar with system RAM settings.
What I was covertly trying to say in my posts is that the bars and dots (and also the associated numerical information) are linguistically
misleading implicitly convey something different to the user than what is actually meant.

I'm not sure what other nomenclature I can use, other than changing "Currently free" to "Currently available"... For consistency I used the same labeling as Resource Monitor; it's also what SystemInformer/ProcessHacker uses. I could add tooltips with full explanations on the key dots though.

Sorry, I got that mixed up. Please ignore this part of my previous post.
However, then I would like to post a request (now I understand the "Update Info" button).
When I click Auto-optimise within the GUI everything gets updated.
As a user (and after the GUI "experience" which comes first) I expect that this will be also done if I click the corresponding context menu option when minimised to taskbar.
I also expect an updated GUI then when I decide to bring back the GUI some later.

Yes I've added a custom SendMessage/FindWindow setup so the new instance communicates to the previous to update its UI; just need to test it.

[yereverluvinuncleber]

I suggest balloon tooltips to explain mostly everything that needs an explanation. As I always assume at least one of my users is probably an idiot therefore an informative balloon tooltip is a must for generally everything.

[fafalone]

I've posted a beta release of version 2.1 that should address everything... I found a temporary workaround for it not exiting; when you use the taskbar close window, it gets a WM_CLOSE message it doesn't get through normal exit (but also when closed from system menu), so receiving that message sets a flag to call PostQuitMessage in Form_Unload, which seems to make it exit mostly orderly. My SendMessage IPC scheme worked in testing here; and I've removed the 'defer to autoupdate' condition so it always immediately refreshes info after an op now.

Let me know if I've missed anything or if there's anything else.

https://github.com/fafalone/MemListMgr/tree/main/Beta

[Zphere]

Thank you,
a small side note: the compiled 64bit exe on Github for v2.1 is still 2.0.

[fafalone]

Oops forgot I changed the build path, thanks, fixed.