Slow animation of a lot of data

Comrades,

I have an issue here which I am looking for ideas on. I am creating a program that models the movement of some atoms and then animates them using the basic Paint method in visual basic. The animation is to have basic remote control functions, such as play, pause, fast forward, rewind and skip to any particular frame.

The program is in two parts. The program models the movements of atoms in a background thread. This program was 90% written by other people and is very long and complicated, so in your answers please do not suggest I change the modelling aspect because it's just not possible. It is my job to do the animation.

The modelling program outputs the positions of each atom (just an x,y coordinate) at each time step. We could easily be looking at 200,000 time steps and 50/100 atoms. That means there are 200,000 * 100 * 2 = 40,000,000 numbers that need to be stored. The locations are all single precision numbers.

At the moment the modelling code stores these positions in arrays. When you hit the start button both the modelling and animation start. The modelling of course zooms ahead of the animation as it is running in a background thread as fast as it can, whilst the animation paints the atoms onto the form at intervals specified by a timer.

What happens though is, as the model rolls on and the number of time steps increases, the animation becomes God slow. I have run a couple of tests:

With the timer set at 1 ms (although it doesn't run this fast) it takes ~25 seconds per 1,000 steps of animation initially. It takes ~8 seconds to model each 1,000 steps.

After leaving the program to run for a bit, by the time 268,000 steps have been modelled it is taking ~15 seconds to model each 1,000 steps. That's not really a concern for me, given that I won't need to run the program for much longer than that and the speed is acceptable, even if it has roughly halved.

The program is that by this time 44,000 steps have been animated and it now takes ~90 seconds to animate each 1,000 steps! That's really slow and it shows. The thing is really laggy.

So, what I need is some way of having consistently smooth animation. Here are a few extra points.

1) The atoms are moving every time step so every bit of data is important
2) The program was initially created with the animation and modelling occuring concurrently. A time step was modelled, and then it was animated. This wasn't good though because there was a lag between animation steps, and this got signifcantly worse with poorer computers which struggled to do the modelling.
3) I have attempted having all of the data from the modelling code not put into an array but instead put into a text file. I then call in the data line by line from the text file at each time step for animation. This is laggy, I suppose due to the time taken to load the data from the text file. It's normally quicker if the data is already there in memory.

So yeah, sorry comrades for the long post, but any suggestions would be most appreciated. Unfortunately I cannot change the data output from the modelling program, and I'm not even sure it'd be possible given that the position of each atom at each timestep is important. It doesn't seem like there is a large size of data, just lots and lots of tiny bits of data. My knowledge of computer science isn't good enough to know how hard that makes it for a computer.

Thanks in advance,

Dave

Correct me if I'm wrong, but it seems like your model is running in a background thread which asynchronously outputs the positions of the atoms in some kind of array. So as far as I can see, the array contains one entry for every time step, where each entry is then the collection of atom positions (50-100 numbers), right?

Are you saying that the model, running in a background thread, fills the array and at the same time the animation code reads it, one entry at a time with a specified interval.

For this to work, it seems logical to me that the model should output the atom locations faster than the animation code reads them. Otherwise the animation is going to overtake the model code and will be trying to animate atom positions that have not yet been calculated.

If this is correct, then I don't understand your point 2. If the model and animation run at the same time, one step at a time, and there is a lag between animation steps then it seems the model is too slow to keep up. If that is the case, how can you ever expect the model to keep up with the animation if you run it as quick as possible (without waiting for the animation steps).


Is it an option to simply wait for the model to complete and then run the animation? That way the model does not have to wait for the animation (if it is faster), the animation does not have to wait for the model, and there should be no lag. I don't expect keeping 40.000.000 numbers in memory will pose much problems. It might be a little hard on ram memory but you can easily test that.

NickThissen, thank you for your swift reply.

Yes, the model running the background thread fills the array at the same time that the animation code reads it. The modelling is faster than the animation, unless when I sometimes set the animation speed to be very fast. By very fast, I mean I set the animation to only show every 2nd, 10th, 50th etc step. In this situation sometimes the animation can "catch up" to the modelling and there is a buffering effect, but this isn't really an issue.

So yes, we seem to be eye to eye on that issue.

As for point 2, I think the main issue was that on slower computers (and I unfortunately have to deal with some ancient machines) the modelling would be very slow, yet the animation would remain smooth no matter how good or bad the computer was (except when it starts to lag after being left to run for a while). Animating each step as soon as it was modelled was not as smooth as animating the position data from the model, so the decision was made to split the two processes.

For your final point, no, it is not an option to wait for the model to complete and then run the animation. I don't think the model has to wait for the animation (the models are produced independent of the animation) and I don't think that the animation has to wait for the models (results show that unless the animation is set at a super speed then the models will be produced quickly enough).

My thoughts still return to the memory being bombarded by millions of bits of data, even if they are only small. The models also produce some big text files which contain significantly more data per time step than just a location of the atoms. These text files are not used for animation. Yet, after over 200,000 time steps these files are less than a gigabyte. These are quite big but surely RAM could handle a gig and even still, the position data for the animation would be way less than this. I have also conducted tests when having this large data output on and off and it makes no difference to the modelling and animation speeds.

If the model is not quick enough on slow computers then there is simply no way to make the animation smooth. If the model is slower than the animation, the animation has to wait until the model produces the next output step, and if that takes a few seconds, there's really nothing you can do except improve the model.


Anyway, assuming the model is faster than the animation speed, it seems there is a problem somewhere in the animation. You say it keeps getting slower and slower as time progresses, yet this doesn't really make sense to me. All it is doing is reading the locations of 100 atoms from an array and drawing them on the screen, right? And it does this every x milliseconds, once for each step. There is no reason drawing the first step should be any faster than drawing the 44.000th step, unless you are doing something strange with the drawing. I am assuming you are only drawing the new locations of the atoms; you are not constantly drawing ALL positions of all previous steps (and the new step simply overwrites the old positions so you don't see them) I hope?

I don't have too much experience with animated graphics, but I would assume you would simply handle the Paint event of some kind of panel or PictureBox, retrieve the required locations and draw them. Then you call the Invalidate or Refresh method of that panel in a timer, once every x milliseconds.

Code:

---

Private allPositions As List(Of List(Of Point))
Private stepIndex As Integer = -1

Private Sub Timer1_Tick(..) Handles Timer1.Tick
  stepIndex += 1
  PictureBox1.Invalidate() 
End Sub

Private Sub PictureBox1_Paint(..) Handles PictureBox1.Paint
  Dim positions As List(Of Point) = allPositions(stepIndex)
  For Each p As Point in positions
      e.Graphics.DrawEllipse(p.X, p.Y, etc)
  Next
End Sub

---

Something like that at least... The 'allPositions' field is a collection of time steps, where each time step is a collection of points (the position of each atom). When the timer ticks, you increment a stepIndex field. In the PictureBox Paint event you retrieve the positions from that step index (assuming they exist already, you should check that of course) and draw them.

Thank you again for your reply.

My animation is as simple as it gets. In the Form_Paint subroutine I just use:

vb Code:

For i = 0 To NumberOfAtoms
e.Graphics.DrawImage(AtomImage(i), intXposition(i), intYposition(i), 24, 24)
Next

And then the timer uses a me.invalidate command.

Okay, I'm going to run a test.

At the moment I am thinking that the animation becomes slower because the array of position data gets very large. You have suggested that perhaps the animation gets slower because somehow the program is painting lots of stuff and clogging up the system.

So, I'll set the model going but not the animation. When the model has reached say 200,000 time steps then I'll start the animation. If it is slow from the start then my theory may be right, as it is struggling due to the large amount of data in the array. If the animation starts of fast, then what you said may be right!

I'll get back with the results within 30 minutes :)

Okay, interesting results. I ran the model for 200,000 time steps and then started the animation and it's smooth! This is odd considering there is so much position data in the memory.

It seems as though the Form_Paint method is getting tired after painting so much :P It obviously need to go to the gym more.

Quote:

---

Originally Posted by AussieDave

Okay, interesting results. I ran the model for 200,000 time steps and then started the animation and it's smooth! This is odd considering there is so much position data in the memory.

It seems as though the Form_Paint method is getting tired after painting so much :P It obviously need to go to the gym more.

---

Was the modeler still running when the animation started?

Yep, the modelling program was still running when the animation started.

Is there any kind of governor on the modeller? I ask because you said, "At the moment the modelling code stores these positions in arrays. When you hit the start button both the modelling and animation start. The modelling of course zooms ahead of the animation as it is running in a background thread as fast as it can, whilst the animation paints the atoms onto the form at intervals specified by a timer." and "At the moment the modelling code stores these positions in arrays."

The fact that there is a lot of data in the array should not matter. You will only be using a tiny bit of that data (the atom positions at one time step), the rest is data from other time steps.
Furthermore I believe getting a value from an array by index is a O(1) operation so its speed is independent of the number of items in the array.

If the animation is smooth when you start it (even when the model is already halfway done) and then starts slowing down this indicates that something is going wrong with the drawing.
You might accidently be drawing duplicates, or you might not be retrieving the positions in the right way, I dunno...

Can you show us some more code?

Quote:

---

Originally Posted by NickThissen

The fact that there is a lot of data in the array should not matter. You will only be using a tiny bit of that data (the atom positions at one time step), the rest is data from other time steps.
Furthermore I believe getting a value from an array by index is a O(1) operation so its speed is independent of the number of items in the array.

If the animation is smooth when you start it (even when the model is already halfway done) and then starts slowing down this indicates that something is going wrong with the drawing.
You might accidently be drawing duplicates, or you might not be retrieving the positions in the right way, I dunno...

Can you show us some more code?

---

I don't think that it is the amount of data, but if the modeller is CPU bound then that will be an issue, especially on older single core machines.

Quote:

---

Originally Posted by dbasnett

I don't think that it is the amount of data, but if the modeller is CPU bound then that will be an issue, especially on older single core machines.

---

You mean that the model is using up the CPU so that there is no 'juice' left for the animation? I guess that's possible... Then how do you explain that the animation runs smooth when you start it while the model has been going for a while? If the model has been going for a while then the CPU should already be too busy and the animation should be slow even the first few time steps. The OP mentions it isn't, it is smooth at the start, no matter if the start is the first time step or the 200.000th time step.

Quote:

---

Originally Posted by NickThissen

You mean that the model is using up the CPU so that there is no 'juice' left for the animation? I guess that's possible... Then how do you explain that the animation runs smooth when you start it while the model has been going for a while? If the model has been going for a while then the CPU should already be too busy and the animation should be slow even the first few time steps. The OP mentions it isn't, it is smooth at the start, no matter if the start is the first time step or the 200.000th time step.

---

I am just asking. Without some idea of how the modeller works this is all guesswork. I don't disagree that it could be a fundamental error in the drawing.

Quote:

---

Originally Posted by AussieDave

<snip>At the moment I am thinking that the animation becomes slower because the array of position data gets very large.</snip>

---

'Gets very large' suggests to me that somewhere or other the arrays are being reallocated to accommodate the new data. Is this correct? Do the arrays in question grow or are they static is size?

remote control :
http://channel9.msdn.com/coding4fun/...tendos-Wiimote

I haven't tested it

moti, I think your AI needs feeding, it's getting delusional.

Interesting to see that I have unearthed some debate here :p

I am unfortunately not allowed to post any exact source code here. This is something to do with patenting and such. I'm not really sure, as it's to do with the modelling code which is of course the main aspect of this project and I don't have a lot to do with it. The animation is really just an addon that I am working on. The modelling program is also several thousand lines long so it'd be very hard for me, who has had little to do with its writing, to pick out the relevant parts.

What I'm going to do now is run the modelling for 200,000 steps and then tell it to stop. At this point I'll start the animation and see if it slows down still :-)

...put a breakpoint and write out the "NumberOfAtoms" as the modelling progresses. I think that's slowing you down some how. I'm going on this assumption based off the For loop posted in #5


EDIT: After re-looking over the loop, that has to be it. It almost looks like you're literally re-drawing every single calculated Atom...but I could be mistaken.
EDIT2: A thought for the animation. why not just draw the latest calculated one and have an option when the animation is paused to "reveal" all the motion up to that point? I'm just brainstorming out loud here now...it's 4:30am and I should probably sleep :p

If he was drawing every position of every atom it would account for a big slow down but it would also be pretty obvious if that was happening as the animation would show extending lines and not dots.

I was not sure if it might be to do with the storage of the positions, if storage is being allocated dynamically in one big lump then as it gets close to 200'000 steps with say 50 atoms assuming 8 bytes per position then there is potentially up to 76Mb of data being reallocated per frame, that said the loop shown in #5 does not really indicate storage of that sort.

There is also the possibility that storage starts hitting the page file at some point.

I haven't been able to run the test I said I would yet, as other things have come up. I will run it tomorrow.

Milk, I am definitely only drawing each atom once per time step. There is no trail, you just see 50 or so atoms jiggling around.

Hi Dave,

I haven't looked at this thread in detail, but the code in post #5 suggests to me that the painting process could be more efficient.

Firstly, DrawImage with width and height arguments takes longer than the one without (I have measured about 2.5x in tests). In the loop of #5, you are forcing the scaling process for each atom (even if it is hidden by another atom or off-bounds). Scaling an image in GDI+ is processor-intensive because it doesn't use the graphics hardware. If you are generating the images yourself, you might be able to use an array of pre-scaled images instead.

Secondly, it's more efficient to invalidate only the rectangles where the atoms are drawn intsead of the whole form. It sounds like a lot of work for 100 atoms, but the integer arithmetic for defining 200 rectangles is a lot quicker than updating the pixels on the screen. I say 200 rectangles, because you need to invalidate both the "before" and "after" positions (if you want an example, see post #10 in ZoomPictureBox in my sig.) I think the invalidation process forms a union of all the invalid rectangles, so no pixel is drawn twice.

BB

Quote:

---

Originally Posted by boops boops

Secondly, it's more efficient to invalidate only the rectangles where the atoms are drawn intsead of the whole form. It sounds like a lot of work for 100 atoms, but the integer arithmetic for defining 200 rectangles is a lot quicker than updating the pixels on the screen. I say 200 rectangles, because you need to invalidate both the "before" and "after" positions (if you want an example, see post #10 in ZoomPictureBox in my sig.) I think the invalidation process forms a union of all the invalid rectangles, so no pixel is drawn twice.

BB

---

Being a model for atomic motion (presumably), I doubt passing rectangles to the Invalidate method will have any measurable effect. Unless the model models some very low pressure gas or something the atoms will be very close together, and the picturebox will probably be around 75&#37; full and 25% empty space. Take into account the old positions as well that number will most likely be very near 100% (any empty space is very likely filled in the next time step, as atoms push eachother away so they tend to fill the empty spaces). In that case, I think invalidating all the atom positions will probably be slower than just invalidating the entire form. You might be drawing 98% of the pixels instead of all pixels, I doubt that difference outweighs the calculating of 200 rectangles and their union.

But then again, I doubt this would make such a large difference anyway. The OP mentions clearly the animation is smooth for the first couple hundred (thousand?) time steps and only gets slower after a while. That indicates that the problem is not that the drawing itself is slow (only that consecutive drawings are slow).

Quote:

---

Originally Posted by NickThissen

Being a model for atomic motion (presumably), I doubt passing rectangles to the Invalidate method will have any measurable effect. Unless the model models some very low pressure gas or something the atoms will be very close together, and the picturebox will probably be around 75% full and 25% empty space. Take into account the old positions as well that number will most likely be very near 100% (any empty space is very likely filled in the next time step, as atoms push eachother away so they tend to fill the empty spaces). In that case, I think invalidating all the atom positions will probably be slower than just invalidating the entire form. You might be drawing 98% of the pixels instead of all pixels, I doubt that difference outweighs the calculating of 200 rectangles and their union.

But then again, I doubt this would make such a large difference anyway. The OP mentions clearly the animation is smooth for the first couple hundred (thousand?) time steps and only gets slower after a while. That indicates that the problem is not that the drawing itself is slow (only that consecutive drawings are slow).

---

I agree that relative slowness or irregularity of the frame generating model sounds a more likely explanation of the problems described. But since the OP is concerned about the rendering process, I thought it would be useful to point out a few ways it could be tuned up.

My main point was to avoid unnecessary scaling in the Paint event, but selective invalidation could make a difference too. How big you draw the atoms in a 2D representation relative to the distances does not depend on intramolecular forces but on graphic conventions/choices. The OP wanted to render each atom to a 24 * 24 pixels square; 200 of those would give 115,200 pixels (although in reality the "before" and "after" rectangles would largly overlap if the animation is running at a sane speed). A form measuring 500 * 500 (for example) would be signficantly bigger. The integer arithmetic for calculating the rectangles would take trivial time compared to the screen update.EDIT: reality check. I just tried rendering an image to 100 24*24 pixel squares at random locations on a 500*500 pixel picturebox. Invalidating the 100 rectangles offered no advantage over invalidating the whole picture box. The time was if anything slightly slower. So you are right there, Nick.

What is unclear at the moment is whether the images being drawn in the Paint event are provided by the model or by the OP's code. If the latter, I would be interested to know how that is done.

BB

Cheers for the further discussion fellas. I am currently at university and am excited to test out some things when I get home (the latest version of the code is on the home computer).

BB, despite my problem largely being a "decaying" animation speed, your suggestions to improve the animation speed in general are most helpful. Whilst your second idea may take a while to implement (or it may not, I haven't read up on it yet) your first idea is rather simple and it seems so obvious.

The atom graphics are not created by any code, they were just created by me in Photoshop. They are just circles but I made them look pretty in Photoshop instead of being boring like the ones you could create in VB code. When I get home I will change them to already be the right size and no longer have to scale them. From memory (I unfortunately don't have the images on hand) they are close to the right size. The scaling process may still be equally slow regardless of how much scaling is required though?

In terms of how much free space there is, there is a reasonable amount of overlap, especially given that the pictures depicting the size of the atoms are disproportionately large (otherwise they would be impossible to see). I am not sure that using an algorithm to make sure each pixel is only drawn once would help too much in this case though but that is a bit of a guess.

The movements of the atoms are only confined to part of the screen. In this section there is a background image that is just a bitmap loaded in. It is not generated by the modelling program (well, it was a long time ago but now is just a static bitmap) and I fear that when invoking the Paint routine the form is having to repaint the background image every time, which seems like a waste since it doesn't change at all. Also, around the section with all the atoms are various buttons, lists and other controls. I am assuming that at the moment the Paint routine is having to repaint all of those things at every time step as well? Again, that seems like a waste and I think that's what you are trying to get at when you talk about a more selective image invalidation.

Quote:

---

Originally Posted by AussieDave

The atom graphics are not created by any code, they were just created by me in Photoshop. They are just circles but I made them look pretty in Photoshop instead of being boring like the ones you could create in VB code. When I get home I will change them to already be the right size and no longer have to scale them. From memory (I unfortunately don't have the images on hand) they are close to the right size. The scaling process may still be equally slow regardless of how much scaling is required though?

---

I would think that the speed of scaling doesn't care how much scaling there is; it is simply a calculation for each pixel (this pixel goes there and becomes this large) and a subsequent drawing of the new pixels. Sure, drawing a HUGE image takes longer than drawing a small image but that isn't due to the scaling itself. Scaling the atom images in every time step is a bad idea though, BB is right there, you should either make sure the images are scaled correctly, or scale them before running the animation so it doesn't have to happen every time step (it only takes 'long' relative to a time step, but not long at all if you do it once before the animation).

Quote:

---

Originally Posted by AussieDave

In terms of how much free space there is, there is a reasonable amount of overlap, especially given that the pictures depicting the size of the atoms are disproportionately large (otherwise they would be impossible to see). I am not sure that using an algorithm to make sure each pixel is only drawn once would help too much in this case though but that is a bit of a guess.

---

You'd need a hell of a resolution to see pixels the size of atoms:lol: But yeah, that's what I guessed too, the change probably isn't too large because in any typical atomic movement model the atoms (well, the images of the atoms) are so close together that there's not a lot of 'free space' that doesn't need to be redrawn. If it's just a few atoms moving around an otherwise empty space then drawing just the changed pixels would improve the speed of course.
However, none of this explains why the animation is smooth at the start but becomes progressively slower.

Quote:

---

Originally Posted by AussieDave

The movements of the atoms are only confined to part of the screen. In this section there is a background image that is just a bitmap loaded in. It is not generated by the modelling program (well, it was a long time ago but now is just a static bitmap) and I fear that when invoking the Paint routine the form is having to repaint the background image every time, which seems like a waste since it doesn't change at all. Also, around the section with all the atoms are various buttons, lists and other controls. I am assuming that at the moment the Paint routine is having to repaint all of those things at every time step as well? Again, that seems like a waste and I think that's what you are trying to get at when you talk about a more selective image invalidation.

---

If you are invalidating the PictureBox then the rest of the form is not repainted (unless Windows for some reason thinks it must be, like when you resize the form or drag some other window over it, which I assume you're not doing).
Invalidate tells the control you call it on that some regions of the control must be repainted. You can tell it which regions (by passing a Rectangle or Region object, which is what BB suggested) but by default it repaints the entire control. Since the buttons and lists next to the PictureBox are not part of the PictureBox, those are not repainted.

The background image might pose a problem though, but I'm not sure. You cuold try putting that on another PictureBox, behind the actual PictureBox that shows the animation. You handle the Paint event (and call Invalidate on) the top PictureBox (which shows the atoms) but don't touch the background PictureBox. If the top PictureBox is transparent then this could work, but as I said, I haven't tried it. From memory though I think the background PictureBox should show through.

Dave, I hope you noticed my edit to my previous post: if the atoms are scattered over the drawing surface, invalidating the individual rectangles may not offer much if any benefit. As for prescaling, if the images are already roughly the right size the gain will be less spectacular than I suggested. In a quick test (just leaving out the Width and Height arguments) I got a time ratio of about 3:2 scaled/unscaled. Scaling down large images makes much more difference.

Looking back to your post #1, it seems odd to me that you were even trying a clock rate of 1 ms. There's not much point in trying to animate anything faster than about 15 ms. per frame, and an interval of 30 ms. is usually good enough. At 200,000 frames that would correspond to an animation lasting 100 minutes. You didn't say what kind of processes are being modelled, but I wonder if you really need that much data to visualize them? (For a simple a 3D rotation of the molecule around a given 3D axis, 360 frames would be more than enough:D)

BB

edit: I haven't read Nick's reply yet.

@ NT:

I don't actually use any picture boxes. All of the things are just drawn on the form.

@ BB:

Yes, I did read your edit. As for the cloke rate, it's actually not close to 1ms. That's just what the timer control is set at, as a minimum interval value. Given the time taken to process the animation, it actually takes a fair bit longer than 1 second. As for the amount of data, no one is going to sit for hours to watch this stuff but the model runs for a long time to see how it evolves with time. I want to be able to jump to any step from 1 to 200,000 or whatever and hit the play button :) There are also things in the model that might kick in at a certain step and such.

Anyway I'm home now and I have a bunch of tests to do. I'll let you know of the result.

Quick update. It would appear that my atom images are 160x160!!! Changing them to 24x24 and removing the rescaling should make a difference then!

First of all, try using a PictureBox then. It is designed to handle drawing (also custom drawing) much more efficiently then other controls, and invalidating the entire Form is completely unnecessary. In that case, you will be drawing all other controls as well.
Your code doesn't have to change much, just replace 'Me.Invalidate' with 'PictureBox1.Invalidate', and handle the PictureBox1_Paint event rather than the Form_Paint event.

Second, a Timer control doesn't typically go lower than something like 50ms interval. Setting it to 1ms is not going to change that, and might even hurt performance (I don't know). In any case, I'm a little lost on what you're doing now. You have a Timer set to 1ms that Invalidates the entire form every tick, yet your animation only changes once every second or so? That means you are repainting your entire form 1000 times for no reason at all (nothing changed), assuming the timer goes that fast (which it doesn't).

Preliminary results: Unfortunately having the images already prescaled doesn't make things much faster. Removing the background does make things a bit faster though but I do need the background. Maybe there is something I can investigate here because it appears that the background is being drawn every time step whereas it should just be drawn once and sit there.

These things aren't the main issue though, the decay is. I'm now going to set it running for a long time but not play the animation. After 200,000 steps or so I'll stop the model and start the animation and see how things go :)

Quote:

---

Originally Posted by NickThissen

First of all, try using a PictureBox then. It is designed to handle drawing (also custom drawing) much more efficiently then other controls, and invalidating the entire Form is completely unnecessary. In that case, you will be drawing all other controls as well.
Your code doesn't have to change much, just replace 'Me.Invalidate' with 'PictureBox1.Invalidate', and handle the PictureBox1_Paint event rather than the Form_Paint event.

Second, a Timer control doesn't typically go lower than something like 50ms interval. Setting it to 1ms is not going to change that, and might even hurt performance (I don't know). In any case, I'm a little lost on what you're doing now. You have a Timer set to 1ms that Invalidates the entire form every tick, yet your animation only changes once every second or so? That means you are repainting your entire form 1000 times for no reason at all (nothing changed), assuming the timer goes that fast (which it doesn't).

---

@ Point 1: I'll try using a picture box

@ Point 2: It was misleading of me to say I want to have it animate every 1ms. The animation (i.e. the me.invalidate command) runs at every tick of the timer. That happens to be set at 1ms because that is the smallest possible timer interval. From some basic tests, I am aware that the timer in fact does not tick every ms, it is in fact a fair bit slower than that. So yeah, I'm not asking for a 1ms refresh rate, I'm just asking for the fastest possible refresh rate. In terms of the rate at which I would want the animation, I'm not sure yet. It will probably just come down to what looks smooth.

Even a timer interval of 50 ms (the minimum) would be 200 frames per second. That is a very large number even when talking about high performance video games for example. I think 50 frames per second is already much more than you need. Anything below 30-40 frames per second becomes visibly 'choppy' for typical people (though that probably depends on what you are viewing, in a fast paced game 30 frames per seconds might not be enough, yet a movie is typically only 24 frames per second and still looks perfectly smooth).

Try setting your timer interval to 200 ms (50 frames per second) and see if you see any difference.

Quote:

---

Originally Posted by NickThissen

Second, a Timer control doesn't typically go lower than something like 50ms interval. Setting it to 1ms is not going to change that, and might even hurt performance (I don't know). In any case, I'm a little lost on what you're doing now. You have a Timer set to 1ms that Invalidates the entire form every tick, yet your animation only changes once every second or so? That means you are repainting your entire form 1000 times for no reason at all (nothing changed), assuming the timer goes that fast (which it doesn't).

---

Actually that is not true. Refresh or Update forces immediate repainting, and they could indeed slow things down enormously when used inappropriately. Invalidate (without Update) is asynchronous: it uses a queue, which is cleared only when the processor has some idle time. That's when the invalidated areas are summed, which is why it doesn't matter if you invalidate the same pixels several times.

With that in mind, I wonder if the model thread is grabbing all the processor cycles so it is delaying the repainting? If so, maybe just inserting a sleep instruction into the model code (e.g. 10 ms. every 5 frames) might leave enough processor cycles to allow repainting to take place. I am not all that well up on multithreading, so I stand to be corrected here.

BB

Edit: Nick, please check your arithmetic;).

@ NT

As soon as I increase the timer's interval length the animation becomes slower. It is God-slow by the time it gets to 200.

@ BB

That's very interesting regarding invalidate and update. Perhaps I should set the timer interval to something high like 100ms and then use update instead of just invalidate?

And I just finished simulating 100,000 steps but there was an error in my code which turned of the modelling process so I'll have to do it again :p

Quote:

---

Originally Posted by boops boops

Actually that is not true. Refresh or Update forces immediate repainting, and they could indeed slow things down enormously when used inappropriately. Invalidate (without Update) is asynchronous: it uses a queue, which is cleared only when the processor has some idle time. That's when the invalidated areas are summed, which is why it doesn't matter if you invalidate the same pixels twice or 1000 times.

With that in mind, I wonder if the model thread is grabbing all the processor cycles so it is delaying the repainting? If so, maybe just inserting a sleep instruction into the model code (e.g. 10 ms. every 5 frames) might leave enough processor cycles to allow repainting to take place. I am not all that well op on multithreading, so I stand to be corrected here.

BB

---

Ok, you are right that it probably won't redraw 1000 times for 1000 Invalidate calls. But, it will redraw 'as much as possible', if the cpu has enough time to redraw 394 times within that 1 second, then it will indeed redraw 394 times, while it should only redraw a couple of times (50 or something). 394 redraws per second is ridiculous. For an animation of an atomic model, even 50 is ridiculously high imo but I'll leave that decision up to the Dave.
(Yes, 394 is a random number that came up by smashing my hands on the numpad)

As for pausing the model thread; I assume the model runs in a background thread anyway so that wouldn't make much difference. It's worth a shot though if the OP can modify the model code at all.


@Dave,
Try some timing tests. You could do something simple as putting a Debug.WriteLine in the Timer_Tick event that prints out the current time (DateTime.Now). Do the same at the start and end of the drawing code. Something like this

Code:

---

Sub Timer1_Tick()
  Debug.WriteLine(String.Format("Timer has ticked at {0} and {1} ticks.", _
      DateTime.Now.ToString("hh:mm:ss"), _
      DateTime.Now.Ticks)

  '... Existing code
End Sub

Sub AnimationStep()
  Debug.WriteLine(String.Format("Animation step {0} has started at {1} and {2} ticks.", _
      currentStep, _
      DateTime.Now.ToString("hh:mm:ss"), _
      DateTime.Now.Ticks)

  '... Existing animation code

  Debug.WriteLine(String.Format("Animation step {0} has finished at {1} and {2} ticks.", _
      currentStep, _
      DateTime.Now.ToString("hh:mm:ss"), _
      DateTime.Now.Ticks)     
Emd Sub

---

Now you can see exactly at which times the timer ticks, and how long it takes for the animation (drawing) code to run. (A tick is 0.1 ms or 100 nanoseconds I think). From that you can probably tell where the code is slowing down. Perhaps it is the timer ticking too late (I doubt it), or it is the animation code running slow.

That's an interesting thought re pausing the model thread. Right now I'm running 100,000 steps. In a few minutes that will be done and I'll be able to animate it. If it still slows down then I doubt pausing the model thread would help very much given that it will be off here!

22,000 steps into the animation and it is definitely getting slow. I like your idea NT and will try it after this. Currently I am doing something similar, logging the time taken to both model and animate each 1000 steps. I'll show the resulsts once I get to about 30,000 steps animated, but from the naked eye I can tell it is slowing down significantly.

Quote:

---

Originally Posted by NickThissen

As for pausing the model thread; I assume the model runs in a background thread anyway so that wouldn't make much difference. It's worth a shot though if the OP can modify the model code at all.

---

Another idea, assuming you can modify the model code in minor respects. For viewing the animation while the frames are being generated, don't use a timer at all. Just put Invalidate in the model code, after each frame is generated. As Invalidate is asynchronous it won't cause cross-theading problems. Maybe you need to add a touch of Thread.Sleep now and then too, but that could be tried later if necessary. BB

Here are the results from my little test of running the model for 100,000 steps then stopping it and running the animation. Here is the time log for the modelling, out of interest.

1000 3.447
2000 9.673
3000 3.276
4000 2.823
5000 9.563
6000 7.566
7000 2.73
8000 5.741
9000 4.789
10000 3.666
11000 4.54
12000 4.04
13000 2.995
14000 4.166
15000 4.04
16000 4.103
17000 3.416
18000 7.364
19000 4.414
20000 3.526
21000 5.008
22000 4.227
23000 3.791
24000 3.463
25000 5.008
26000 4.976
27000 4.634
28000 4.305
29000 3.713
30000 4.056
31000 3.885
32000 4.446
33000 4.024
34000 4.524
35000 4.431
36000 4.337
37000 4.368
38000 4.586
39000 4.571
40000 4.618
41000 4.336
42000 4.634
43000 4.492
44000 4.571
45000 4.712
46000 4.96
47000 4.914
48000 4.15
49000 4.165
50000 4.009
51000 4.213
52000 4.492
53000 4.119
54000 4.914
55000 4.352
56000 4.836
57000 4.446
58000 4.4
59000 5.163
60000 4.899
61000 5.475
62000 5.351
63000 5.195
64000 5.179
65000 5.476
66000 4.976
67000 5.616
68000 5.429
69000 5.662
70000 5.773
71000 5.382
72000 5.538
73000 5.6
74000 5.335
75000 5.772
76000 5.975
77000 5.569
78000 5.476
79000 5.569
80000 5.772
81000 5.944
82000 5.881
83000 5.991
84000 5.865
85000 6.178
86000 5.834
87000 6.069
88000 6.084
89000 6.084
90000 5.397
91000 5.46
92000 5.585
93000 5.57
94000 5.397
95000 5.507
96000 5.663
97000 6.084
98000 6.24
99000 6.599

And for the animation:

1000 20.249
2000 21.216
3000 21.981
4000 23.244
5000 24.258
6000 25.569
7000 26.52
8000 27.596
9000 28.814
10000 29.843
11000 30.95
12000 32.121
13000 33.181
14000 34.336
15000 35.428
16000 36.567
17000 37.69
18000 38.859
19000 39.905
20000 40.982
21000 42.073
22000 43.259
23000 44.352
24000 45.552
25000 46.597
26000 47.596
27000 48.719
28000 49.796
29000 51.043
30000 52.105
31000 53.29
32000 54.397
33000 55.427
34000 56.644
35000 57.705
36000 58.766
37000 59.951
38000 61.012
39000 62.135
40000 63.196
41000 64.366
42000 65.427
43000 66.566
44000 67.673
45000 68.843
46000 69.857
47000 70.84
48000 72.041
49000 73.211
50000 74.195
51000 75.426
52000 76.456
53000 77.673
54000 78.734
55000 79.966

It's clear that the animation is slowing down, but this log doesn't tell you where. If you also log the time that the timer ticks (as well as the start and end times of the animation time step) then you can see where the problem lies.

As we can see, the modelling doesn't change in speed much but the animation gets significantly slower. This is with the bitmaps prescaled and the background image on. I haven't changed the code to use a picture box yet.

@ NT

Yep, I'll do that next.

Okay, I'm finally back working on this. After NT fixed another issue I had (thanks heaps!) I am now back to checking the animation speed.

So far I have made the change to doing the animation on a picture box, not on a form. Changing the code for it was very simple. To be honest, I reckon it's actually performing slightly worse now. I haven't tested how the animation "decays" yet but it seems as though it is a bit...jerky, right from the start. It just jerks here and there, not consistently. When animating on the form it would be very smooth at the start.

I'm testing the "decay" now. The stutter that is present when viewing the stuff in a picture box, not the form, isn't consistent or very prominent but it is noticable.

Okay, it decays just as much when using the picture box. Next on the agenda is to test different aspects of the code to ifind where the delay is coming from!

Okay, I have done a little test.

I counted the time taken to run all of the stuff in the picturebox.paint routine. It consistently takes 0.016 seconds with very little variation and no decay whatsoever. Something else must be the cause!

I imagine your program should be structured like this...

Model --> Buffer --> Animation

The Model should only be concerned with the moment, i.e. current positions and other attributes of the particles.

The Animation should only be concerned the positions of the particles at a particular timeslice.

The Buffer stores all the positions of every timeslice and therefore grows as the model runs.

The most likely culprit of the slow down is the Buffer.

Hey Milk, thanks for the reply.

My only reason to doubt your suggestion is one of the tests I ran. I ran the model for a large amount of steps and stored everything into memory, so a huge buffer if you will.

I then stopped the modelling and ran the animation. It was fast at the start and then "decayed".

Maybe the computer finds it easier accessing data at the front of a very long array than at the end? That seems a bit weird though.

The fact that the Paint routine speed doesn't decay doesn't mean the problem is not with the painting. The problem I am talking about, and have been talking about since the start of the thread, is that you are simply drawing too much to the screen. The code to do that doesn't take long, but the actual drawing does!
Still, without seeing all your code I can't say much.

Fair enough observation, the computer does not find it easier to access from the front of an array (elementaddress = baseaddress + elementsize * index). It could still be the buffer that is culprit.

Can you show us how the buffer is structured and the code where the Animation part extracts the appropriate timeslice data?

Edit: Nick, I'm seeing the animation part as a trivial amount of drawing. 50 icon sized bitmaps? Something is very wrong if that is taking a long time to render. I had a play at the weekend animating a gravity model of 1000 particles with trails (like little comets) and it ran happily at 30 fps in a window of 1024x1024, the animation included over 1000000 per pixel calculations per frame (for the fading trails). This is on a 7 year old pentium.

@NT: I debate that too much stuff is being drawn on the screen because there are only every 50-100 atoms. You say that the code to do it doesn't take too long but the drawing does. If I set a start time at the top of the OnPaint routine and then a finish time at the end of it, and then print it to a log, does that interval not contain all of the drawing? Does the drawing instead get passed to a seperate thread perhaps and gets done in its own time?

@ Milk. The buffer is very simple. The model runs and outputs all of the positions at all time into some arrays. I wouldn't even call it a buffer, it just contains everything. The animation can start at any time and just plots what is available. The modelling is faster than the animation except when the animation is set to a very high speed (I can set it to only animate the nth step if needed) so there normally isn't an issue of running out of data to animate.

Quote:

---

Originally Posted by AussieDave

@NT: I debate that too much stuff is being drawn on the screen because there are only every 50-100 atoms. You say that the code to do it doesn't take too long but the drawing does. If I set a start time at the top of the OnPaint routine and then a finish time at the end of it, and then print it to a log, does that interval not contain all of the drawing? Does the drawing instead get passed to a seperate thread perhaps and gets done in its own time?

---

As far as I understand it, and that might not be 100% correct, the code in the Paint routine (code such as Graphics.DrawEllipse, DrawLine, etc) merely instructs the control what to paint. The painting itself is done by windows I guess.
There are 'only 50-100 atoms', true, but there are 50-100 atoms per time step. If yuo're doing everything correctly, you should only be drawing 50-100 atoms at a time. If you're not, you might be drawing an EXTRA 50-100 atoms every time step, resulting in 50-100-150-200-250-300-350, etc, and this number rises very fast.

Again, without seeing yuor code there is no way to tell, but I think it could explain why it is slowing down.

I appreciate that having every atom at every time step eventually being added to the page would slow things down but I can guaruntee that there are only every 50 or so at a time.

Not going to lie, I'm very tempted to say "**** it" to the whole thing and go with some graphics-based rendering like Open GL. Our model is currently in 2D but a 3D version could be easily made. Unfortunately I know nothing about anything other than GDI+, and I probably should fix this issue first.

I'm sure if you posted some code we can help you further. Without code it's just impossible. I can understand you're not allowed to post the model code but just post what you can. You can leave out the details if you are confident they don't matter.

Agreed, without showing us the code you will only get speculative answers.

Here is my attempt at posting the relevant code. It's just so big that it's not an easy task! The RunModel() routine is run over and over in a background thread.


Private Sub wrkModel_DoWork(ByVal sender As System.Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles wrkModel.DoWork
Do While RunModel = True
RunModel()
If RunModel = False Then
wrkModel.CancelAsync()
End If
Loop
End Sub

Public Sub RunModel()
'run all of the model stuff here
'append the positions of each atom to the position vectors
timestep += 1
End Sub


That's basically how the modelling aspect works. For animation I have:


Private Sub timAnimate_Tick(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles timAnimate.Tick
ShowAnimation()
CurrentAnimationStep += AnimationSpeed 'AnimationSpeed = 1 by default
End Sub

Public Sub ShowAnimation()
PictureBox.invalidate()
End Sub


And in the OnPaint routine I have:


For i = 0 To NumberOfAtoms
intXposition = XOffset + ConvertScale(AnimationPositions(CurrentAnimationStep).X(i), "X") - 12
intYposition = YOffset + ConvertScale(AnimationPositions(CurrentAnimationStep).Y(i), "Y") - 12
e.Graphics.DrawImage(AtomGraphics, intXposition, intYposition)
Next

Hmm, the code seems to have come out looking rather red and ugly. Hopefully it is still readable though. It is such a long program but I hope I have pulled out the important bits.

The ConvertScale function is just used to convert the position of the atom from the model into a pixel coordinate to be drawn.

I wonder if the fact that you're drawing images could cause this, perhaps you're generating the image again and again (maybe not you but the program behind the scenes). Could you try replacing DrawImage with a simple DrawEllipse (or FillEllipse) to draw a circle?

I'll give that a shot.

The same decay occurs when using ellipses instead of bitmaps.