Introduction

CSharedMemCache is a name-based key-value memory cache that can be shared across processes.

It is best suited to a small-to-medium, read-heavy/write-light, same-session cache shared by a handful of trusted VB6 processes, where values are expensive to generate, invalidation is explicit, and contention is modest.

It is not a substitute for a full-featured cache such as Redis—with expiration, eviction, statistics, enumeration, clustering, and other operational features—or a persistent, transactional datastore such as SQLite.

Background

Skip this bit if you aren't interested in story time.

I'm working on a multi-process web application server layer for my existing VB6 backend code and it needs to generate HTML for presentation. Some of that HTML is expensive to generate but rarely (if ever) changes, so I figured some kind of cache was called for.

Many years ago, Olaf Schmidt introduced me to file mappings as a way to share data across processes, using a serializable RC6.cCollection as a key-value store, so I started there. The problem was that, when generating a value took a long time, multiple processes could attempt to generate the same value at the same time. This worked, but it was obviously inefficient.

More recently, I created a MutexEx project and figured it could be used to block additional generation attempts while one was already in progress, then release the waiting processes when the work was complete. That would allow only one process to perform the expensive operation while the others waited for the result.

After a bit of work, the approach did the job, and I felt somewhat like how a real programmer must feel when they use foundational building blocks to support a higher-level program.

I could (maybe should!) have stopped there, but I like to share, so I thought I would clean things up a bit and post the project here for anyone else who might find it useful.

The first step was to run the code past ChatGPT for review. It did a very helpful job of identifying some potential pitfalls and helping me understand and correct them. Not that LLMs need any more advertising, but this step likely saved many hours of debugging some gnarly cross-process issues that would otherwise have been difficult to reproduce reliably, so I highly recommend it to other mere mortal programmers.

The next realization was that, with Olaf having (sadly!) largely disappeared from the VB6 community, most people are understandably avoiding taking on RC6 as a new dependency. This left the slightly daunting task of implementing a serializable Collection/Dictionary class of my own.

SIDE NOTE: existing RC6 users can swap RC6.cCollection back into this project and use it instead of the included implementation. RC6.cCollection is more feature-rich, more battle-tested, and still faster in a some areas.

Those earlier feelings of being a real programmer now seemed like a dream, so I turned back to ChatGPT. Armed with Olaf's cHashD code, I prompted ChatGPT to use that as inspiration while adding serialization/deserialization for the supported VB6 intrinsic types, arrays, and COM objects implementing the standard IPersist* interfaces. Ideally, it would support both binary and JSON serialization (as a bonus).

About 45 minutes and a swath of rain forest later, ChatGPT produced something that *almost* compiled!

After another 30 minutes or so of back-and-forth, the bloody thing worked. Crazy.

Over the course of the rest of the day, we worked through performance improvements (which started out quite poorly but quickly became much better), added features, cleaned up the structure and API, and tackled the genuinely difficult problems such as naming things.

Performance still isn't quite at Olaf levels in every category, but the gap has been reduced from an order of magnitude or more in the initial pass to a much more reasonable percentage. In some operations, the new implementation actually appears to be faster (at least in limited synthetic testing).

I was now a manager! This means I can take all the credit, right? After roughly a day of work, the supporting serializable collection class was complete, along with a reasonably extensive set of tests to give me the "warm and fuzzies".

CSerializableCollection Features

The serializable collection class is the base storage system for the CSharedMemCache. It acts as a replacement for the RC6.cCollection class (partial only - just the methods required for CSharedMemCache - an RC6.cCollection can be substituted in for it if you prefer). CSerializableCollection includes:

  • Binary serialization and deserialization
  • JSON parsing and generation
  • Compact Base64-wrapped JSON (ideal for database storage)
  • Human-readable and editable JSON, with minified, spaces, and tabs formatting options
  • Serialization of supported VB6 intrinsic values
  • Serialization of Multidimensional arrays, including non-zero lower bounds
  • Serialization Variant arrays containing nested supported values
  • COM persistence through IPersistStream and IPersistStreamInit
  • Readable COM persistence through IPersistPropertyBag and IPersistPropertyBag2
  • A temporary in-memory property-bag bridge
  • A native string dictionary used by the collection and persistence code
  • Correctness, round-trip, contention, and performance tests. NOTE: Test modules are included in the ZIP, but not the project. You can add them to the project and execute TestAll if you want to ensure everything passes yourself.


Objects supporting property-bag persistence can be represented as named values in readable JSON. Objects supporting only stream persistence are preserved as opaque Base64 data, along with the class information needed to recreate them.