Virtual Memory

[joey o.]

HI,
I'm writing a graphics program that records mouse moves into virtual memory. I have a few questions I'm hoping someone might know the answer to:

1.Is there a way to turn up the users virtual memory to maximum when the program initializes and reset it back when done?
2.Can I utilize some other form of memory that bypasses the system settings?
3.If I write the mouse move positions to a text file can I dynamically change properties of objects in this program as if I'm writing direct to the VB editor without loss of performance?
4.(Related to question 3)Is there a way to dynamically write script and place it in VB during a programs run time?

Any help on any question will be appreciated. What I'm really looking for is to write and store modules of VB programs inside or outside of VB so they can be writen(most important), called, and utilized during the program's run time.

[joey o.]

This sat all day without a response. I thought I'd give it another go.
Joey O.

[joey o.]

Giving this one more go round,
The folks in the States couldn't or wouldn't answer. Surely someone must know at least one of these???

[Paul282]

You sounded lonely so I thought I'd visit!

Part of your question is ambiguous.
Your VB app has access to it's own VM space, which by windows default is 4gb. Every program has 4G to work with even though the PC may not. When the PC runs out of physical memory it writes the pages to a pageing file on the hard disk. If you know you will require more memory than is available then it's worth you writing to the HDD yourself so you can be sure to keep the CPU entensive part in memory and simple storage on disk.

As for the other part... I've often thought about this too. To have a program create modules for itself or incorporate modules at runtime.

There are a few options but none of them perfect (many of them not even good)

1. investigate the CallByName function and allow your software to read text in and hold code to be executed that way (code needs to be reloaded if the app is stopped)

2. Compile DLLs and figure out a way of passing the names of them to your software at run time (not easy)

Remember the problem is that your code is compiled and in order to add to the compilation the soft needs to be killed in order to be saved over. the other option is to create a second loader program that is spawned from your code and kills your app, makes the modifications (leave that up to you) then restarts your software. I used this method as an update feature on software sent over the web, you could download an update and the loader would install and restart the app.

If I'm babbling in the wrong direction please let me know

[joey o.]

Your not babbling, But I'm confused as to why my arrays draw an out of memory mesage when the VM of the machine is set to minimum if the program is from what you say,accessing it's own VM space of 4G.Am I screwing up or did I miss something? Is there an API or shell call I can make to insure VM is sufficient?

Thanks to all - Please continue to help

[Paul282]

Well, it's possible that the overall VM setting is too low, that is, the pagefile size. Right click on MyComputer, click proberties and one of the tabs (depending on the OS) has VM settings, increase the figure (double it if you have lots of HDD drive space left). I think you'll need to reboot the computer to have the settings take effect.

How big is the array (size and data type)?
How much RAM is on the PC
what's the OS?
What's the VM set to?

Cheers

[joey o.]

Array is big. It records all x,y coords for specific tasks. Easily over 10000+. I'm trying to capture command clicks to complete it. Thats why I need to be able to read back the events. No problem with maxing VM for me, but I don't want to have to put unsightly VM instuctions in the program, I want it to me maxed at start up automatically then put back to original setting on unload. In other words I need to automate the settings for Virtual Memory.
Thanks - More help welcome!

[Sam Finch]

Right, First I'll try to explain Virtual Memeory

Virtual Memory is how windows lets programs organise their memory in a nice organised way while still meing able to manage the physical memory and keep everything ticking along smoothly.

Virtual memory is ecactly what it says, it's not real.

One way to imagine it is like telephone numbers.

If you imagine that you are Pentium Man, with super pentium powers of being able to do calculations at lightning. (for some reason I use superheroes to explain a lot of things)

the trouble is you have very little short term memory and no long term memory at all.

But you do have a telephone and a sidekick.

Your Sidekicks name is PageFile boy, He controls the telephone system, if you are working on a calculation for your boss(the VB Programmer, most powerful superhero of them all) and your boss asks you to assign some memory, you will tell Pagefile boy to assign a telephone number to a person, you don't care who that person is, as long as he is on the end of a particular telephone number, you can then ring him up and ask him to remember a number, and you can ring him up again and ask him what that number was whenever you want. He doesn't care what his telephone number is, he just gets the call and either remembers what the number was, or tells it to you.

Pagefile boy is very good at his job, he will never asign 2 telephone numbers to the same person at the same time.

You simply follow instructions, ringing up people, asking for the number, doing sms with these numbers and ringing up other people to remember them, VB Programmer always tells you which numbers to use and which ones to assign. (you also deassign telephone numbers to free up memory)

Just Incedenaly, sometimes you have to switch processes, which means you are working on a different program, This is where pagefileboy really earns his keep, he can instantly switch all the telephone numbers around, deassigning them all then assigning them to what they were when you last worked on this process)

Hopefully that was clear enough to understand how our superheroes work, Now once apon a time Pentium man had a visit from Bill Gates, who copied this system and used it to make an operating system.

The people on the end of the telephone line were replaced with chips called Physical memory, Pentium man was replaced with a CPU and pagefile boy was replaced with something called the pagefile, which does his job.

Virtual memory is like the actual telephone numbers that Pentium man used, if they are assigned to people (physical memory) they are backed by physical memory.

There are 3 basic types of memory error you can get.

1. Pentium man can be asked to phone a number with nobody on the other end (he has to access virtual memory that is not backed by physical memory) this causes a general protection fault and the process terminates.
   
2. Pentium man asks Pagefile boy to assign another telephone number, but there are not enough Telephone numbers, or pagefile boy hasn't been given permission to use them. This raises an out of stack space error.
   
3. You ask pagefile boy to assign another telephone number but they are all used up

Now looking at case 3 we realise that there are 2 types of people in the world, smatr people(RAM) and dumb people(Hard drive) pagefile boy tries not to assign telephone numbers to the dumb people because they can't remember the number they have to write it down, and it takes them ages to find a pen and paper and then it takes them ages to find where thay wrote the number down. Put if pagefile boy runs out of smart people he has to start assigning them to the dumb people (in fact he's quite clever and shifts memory around, assigning numbers that arn't used much to the dumb people)

But pagefile boy is only allowed to assign numbers to a limeted amount of dumb people, if he runs out of dumb people he raises an out of memory error.


Now you can change the amount of hard drive space that windows can use for virtual memory, it's in your system options in the control pannel (it changes exactly where the option is from Version to version, but it's probably in the advanced tab) and I believe that you can change it with an API, but I don't know how. You can also increase the amount of VM(telephone numbers) the pagefile is allowed, up to a maximum of 4G as Paul is saying.


One of the problems with just increing the amount of HDD available (I think it varies between NT and 9x whether it set a maximum pagefile size or a maximum amount of HD space allowed) is that when the system starts backing bits of memory with the hard drive it slows things down considerably, so you really should avoid doing this.

The Best Way is to write the data directly to the Hard drive there's a tutorial Here

that's your best solution.


On dynamicly writing Script you can use the Microsoft Script Control.

You can add code to this as a string and execute a line of code as a string (so add some subroutines to it then call it from a string)

Unfortunatley you can't access variables in your code from it, you have to declare variables in the code you add to it and then set values whenever you change them.

press control T and select Microsoft Script Control from the list. then add one of these.

I've never actually used one before so I can't really help with all the details. the best thing to do is keep experimenting with it, and ask questions up here if you don't get anywhere.

[joey o.]

Sam,
That was great and I would hope John finds this and puts it in a tutorial or something. I'm wicked happy about finally understanding this stuff as well as discovering the script control. I'd still be interested in the API for increasing the VM memory though, for quick storage and retrievals ie: collections in classes for folks whose VM is set to minimum.
Thanks again,
Joey O

[Sam Finch]

Hmm, "The SuperHero's Guide to Visual Basic."


I'm Afraid I don't know the API for increasing the pagefile size, in any case I think you'd have to set it and then reeboot, which isn't really what you'd want to do.

There is one Called VirtualAllocEx, this allocates your program an amount of virtual memory, the trouble is you can't then access it with VB variables, you have to use CopyMemory to get at it and it's a real hassle and not worth it, also you have to back the memory yourself using VirtualLock or VirtualProtect and unless you are really organised in how you use this memory you will end up using more memory keeping track of where everything is than you would just storing it normally.

I have never run out of Virtual Memory, you get an enormous amount. If you are pushing your memory to these sorts of limits then not only are you slowing your own app down but you're slowing the whole machine down to a snail's pace.

You really shouldn't be diong this, if you're strting to store this much then write it to the hard drive, it's actually faster than storing it in memory.

[joey o.]

Thanks again Sam,
Read the tutorial and liked it. My fear was loosing performance in the read. I was using sprites to draw for a while but the gets and lets made the program jerky while .wav files played. Using the VM was instant and smooth. I'm trying to store using binary, I hope it works!
Thanks

[Paul282]

I'm still a little unclear on why your array takes up so much space!

Public Type MouseCoords
   X as Integer
   Y as Integer
End Type

Public MouseLog() as MouseCoords

With 50,000 Co-ords that's 200K of RAM (plus a bit for the array etc)

how are you storing this data?

[joey o.]

OK,
Maybe not so big, but size is not really the problem.All the NT machines in my office are defaulted to minimum on VM setting. I have to turn them up to use collections in VB to avoid the "out of stack" error. I'm not sure if this is the default for NT and Win98 or unique to my workplace. I was going to be distributing a graphics program that utilizes the VM and was concerned new users would say "to hell with it" if it didn't work right out of the jewel case. Although I haven't tried to write it to a text file yet, I'm concerned that it won't get the info quick enough to draw what it needs to at each refrenced co-ordinate of the mouse move.

[Paul282]

Hang on !!!!!!

You're getting an Out of Stack space error ????

I'm not sure if Sam's had enough to drink to fly off on one more of his super hero descriptions but the stack is:

A fixed amount of memory used by Visual Basic to preserve local variables and arguments during procedure calls.

To quote Microsoft. Actually it's a little more complex than that.

Try some of these things...

Declare any large arrays at a module level (better still dynamically, If you've ever coded in C++ you'll know the advantages of allocating Heap memory rather than wasting stack space)

Use Normal rather than fixed length strings

be careful with nesting functions too far and using do-events statements in the nest.

There is more info in MSDN, I don't think you need to fiddle with the VM settings...


[Edited by Paul282 on 08-23-2000 at 02:39 AM]