I have a routine in VB that reads a text file (random access) full of drawing coordinates and then draws everything onto a picturebox on my form.
Since the text files are usually larger than 100 MB in size, this process takes a long time, usually over 3 minutes!!!
I'm wondering, how much faster would the same thing take in c++? If there is a significant difference then I may be forced to switch over since 3 minutes is unacceptable for loading this image onto the picture box. More specifically, how does c++ file access differ from VB in speed, and how does c++ drawing methods differe from VB in speed?
I might not be a C++ expert but file access (Reading & Writng) speed differs significantly! The thing is, it's not just the language! 100 MB? You're practically loading a game level. That takes pretty long no matter what language you will be using! C++ might save you a quarter of the time but it ain't gonna save the bad method you're using. Try revising your graphics format. Other than that, there are code improvements that you can do in Visual Basic or any other language for that matter. For example, try accomplishing multiple steps in the least amount of code. That's just my opinion though! Good luck!
100mb is ALOT of data... i can only assume you meant 100kb...
if you're reading it in byte by byte, that'll take forever... in which case, the best thing to do, is make a buffer and GET the data in one line, not in a loop... it will gain you a 85%+ increase in speed on large files.
Originally posted by Mushroom Realm Visual Basic is a scripting language, c++ interacts with memory directly, not only would it run faster the whole program would be smaller too
VB is a compiled language. not scripting language. the code is not encrypted or anything like that when compiled. it is made into a true win32 executable.
Remember, if someone's post was not helpful, you can always rate their post negatively .
This is comprised of probably 500000 or more records, each of which contains 2 x and y coordinates, the colour of the line/arc/circle, and the line thickness.
This data is for a large and very detailed map of a large city. It's used by utility companies. We have a program here called CableCad that can view this map in less than a minute. In addition to this when zooming in/out after the image has been drawn the first time, the image loads incredibly fast.....faster than any other program out there. Unfortunately, CableCad does not offer all the features we think are critical for this application (I won't go into this in more detail).
What my program does, is it opens up a dxf file (which has been generated from within CableCad) that is usually over 300 MB in size (dxf files are HUGE). My program uses a 'process dxf routine' function which takes the dxf file, extracts only the data I need in order to draw everything onto the map, organizes the data into different drawing 'objects' (for example, all lines are sent to 1 file, all circles to another file, all arcs to another file, all text to another file).
The idea is that the process dxf routine is only executed maybe every week (since the original files are updated about every week). By processing the dxf file like this, we can speed up the loading time tremendously.
I've included a screen shot of the picture box with the map loaded.
So would I be better off to read in say 1000 records at a time? Currently I just read in a record at a time. I guess since each record is the same size (type object) I could do this...?
Well, you could stop at 1000, but I would just load the whole thing into memory... in one fell swoop...
However, I have a few questions:
1) You say you have a text file with these coordinates and related info, but then you mentioned you are using Random Access methods to get this info.
Well, Binary Access would greatly speed this up...
So, if its a binary file, its random access...
If not, its more likely a sequential text file...
I imagine you are really talking about a sequential text file, since this was generated by another program, supposedly for export to other applications.
One more thing, if you are seriously about creating images in detail like the one you posted, you better be able to do DirectX programming...
If you are using basic operations such as PSet and Line, well, and it is taking you 3 minutes to create an image, then you will have to reogranize your code, or move to DirectX.
I think the majority of the 3 minute load time is spent accessing the file and/or refreshing the image as I draw stuff on to the screen. To minimize the # of refresh calls I only refresh the image every 10000 lines.
BTW...I have a small idea to speed up the loading significantly. Make the image invisible as you load it and then display it all in the end! That will dramatically speed up loading time. Just an idea but if you don't intend to do it that way, it's your loss. Kuz the system will take plenty of time to display and refresh the image time and time again and that will get in the way of the loading procedure.
I need to be able to show the image loading as it loads, since the user can choose to cancel loading at any time, or zoom in before the loading process is complete.
WIth binary access, can you read a 'section' of data directly into a type variable, such as with random access?
This to me seems like a great benefit and one that would speed things up considerably. Otherwise you would have to read in data, then break it apart into separate variables etc.
Originally posted by ae_jester I think the majority of the 3 minute load time is spent accessing the file and/or refreshing the image as I draw stuff on to the screen. To minimize the # of refresh calls I only refresh the image every 10000 lines.
If there are over 500,000 lines, and you are refreshing every 10,000 lines, then that is also a major speed factor. If I were you, I would only draw it once at the end, and just update a progress bar or something to let the user know it's still loading.
Well, Windows uses page files, so 100mb isn't a problem....
As far as your file access, that is random access... and you can read it with binary access...
If memory serves me correct, Single Types take 2 bytes..., longs take 4... so the LenB of your mapLineType should be 2*5=10 + 4= 14 bytes long.
So in a byte array, you'll have 14 bytes for each record.
Now, before moving ahead, the code you posted, you note you are saving the lines back to file... which means you are editing the data? and then resaving those lines to file?
You will need to give more info before I recommend moving to binary access... because with Binary, you must perform conversions, and depending on if you have to save or edit the data, it can make a big difference on how to approach it code wise.
At one point in time I considered drawing the image to a device context in memory and then just copying this to the picture box every few seconds. I thought this might be faster, but I never got anywhere with it because of a number of scaling problems and lack of experience working with these.
If there are over 500,000 lines, and you are refreshing every 10,000 lines, then that is also a major speed factor. If I were you, I would only draw it once at the end, and just update a progress bar or something to let the user know it's still loading.
Actually I tried refreshing every 40000 lines and the time it took to load was very similar to refreshing every 10000 lines.....So I'm pretty sure the file access is the main cause of the speed problems.
if you run my code example on 500,000 lines, you can see it only takes VB 4-5 seconds (Depending on your processor) to create all those lines. So it is definitely the file access aspect.
Now, this populates 500,000 random lines also, and that takes 2 seconds itself, and this would not be a requirement of your program since you are obviously reading from an existing file, so the first part of my example would be unnecessary and irrelevant to the final outcome of speed.
Public Type mapLineType
LineType As Byte
XPosStart As Single
YPosStart As Single
XPosEnd As Single
YPosEnd As Single
Colour As Long
Thickness As Single
End Type
So with an array all elements are of the same type right?
So could your example be modified to use something like the above? Or would I have to treat everything as Single's since the coordinates themselves can be decimal numbers?
If I try setting the array to 2000 nothing happens. Now when I add error checking code in it says "Overflow".........but without the error code there i get no error message, just nothing at all happens!
Well, here you go.... the file access (load) part is faster than my original byte array version... which I can't explain, so don't bother asking...
You'll note the bold line in the code below... this line decreases the speed at which the program fills the screen to 11 seconds from 4.5 seconds on my machine... for some reason changing .DrawWidth property of the picturebox incurs a heavy performance hit.... this is where writing directly to video card memory(DirectX) would prove useful... especially when zooming the image... I would ask some people in the Graphics programming section of this forum...
I add a DoEvents after the bolded line to allow for user interaction, such as moving the form, or perhaps for a cancel button. DoEvents only added 1.5 seconds to the overall time on my machine.
Well, i've been playing around and here's a few things you should know....
Set your picturebox1.AutoRedraw=False
Set your picturebox1.ClipControls = False
This allows for a great improvement in speed...
I have included a form for download that you can run , which compares the intrinsic VB Line methods, versus the Win32 API (C functions) LineTo methods... as you will see, both run at the same speed regardless, other than you don't have to set picturebox1.autoredraw or clipcontrols=false to realize that speed for API version. Plus when compiled, the API version seems to work better as far as retaining the image made, since you haven't had the need to adjust the clipcontrols or the autoredraw properties.
Last edited by nemaroller; Sep 16th, 2002 at 05:23 PM.
There isn't 500,000 records there is close to 4 million!
On this computer it takes 37 seconds just to open the file and plug the data into the array!!!! This isnt much good especially when the user wants to zoom into a small region. I tried opening the file in 'sections', for example 100000 records at a time (anything larger than this causes annoying delays during the drawing process), which did work but really didn't improve the overall time from before by a big margin.
There is no way I can set the AutoRedraw property to false, because then if another window is moved over the image, or the user switches to another app, the image dissapears when they come back. Unless there is some method of avoiding this???
I'm going to try the API method and see if that does much
Is it faster than the commercial product you use to load images? And if not, is that running on a different computer with higher processing speed? (Some workstations have dual processors)...
as I said above, the API method will retain the image properly...
Last edited by nemaroller; Sep 18th, 2002 at 03:27 PM.
Reply With Quote
.
