Why So Slow??

[Buy2easy.com]

WHY IS THIS PROGRAM SO STINKIN SLOW?? I rote it today it reads throught the entire file..about 5 megs and it is super slow. I have heard c++ programmers for years say that c++ is the way to go if you want your programs to be fast. Well i wrote a more complex program in vb.net it runs in windows this below code runs in a console so it is less complex and my vb.net(more complex) will smoke this simple little program below.

Can anyone tell me why? the CaseConverter is just a seperate thing to convert the string to lower case so it does not make a big difference. Oh yea i am running the below code on a pentium 4 and the vb.net code on a very much slower pentium 2 and it still smokes the below code.

Thanks for any help.



// Included Stuff & Other Stuff//
#include <iostream>
#include <fstream>
#include <string>
#include <vector>
#include "caseconverter.cpp"

using namespace std;

typedef vector<string> StringArray;

//Main Function: //

int
main(void)
{

ifstream fin;
string Buffer;
string sString;
string addStr;
StringArray Results;
StringArray::iterator sIT;
CaseConverter Converter;
int Found = 0;

// Open file to be read

fin.open("bible(KJV).txt");

cout << "Please enter a search string: ";
getline(cin,sString,'\n');
Converter.cConverter(&sString);
cout << "Searching for: " << sString;


// Check to see if it opened

if(fin.fail())
{
cout << "\nError Opening File\n"; //if the file is not found then display error and exit
return 0;
}

// Search for the string

while(getline(fin,Buffer))
{
Converter.cConverter(&Buffer);

if(Buffer.find(sString) != -1)
{
Results.push_back(Buffer);

Found += 1;
}
}

// Close file

fin.close();

// Display info about search

cout << "\n There were " << Found << " verses that contained " << sString << "\n";


return 0;
}

[Buy2easy.com]

and on top of that the c++ compiled *.exe is 400 + kb while the vb.net *.exe is only 50 or less KB

why is the C++ *.exe so big?

[Arawn]

Can't help you much on why it's so slow, But if you're using VC++ for this, you're probably using the debug configuration, usually adds a few hundered k onto it. If you set the config to Release then it'll probably drop down to less then 100k...

[CornedBee]

Why is it so big?
Unlike your VB.Net program, which needs a few megabytes of .Net framework to run, while the C++ program will run all alone, using not a single DLL besides the standard windows DLLs.
If this is alredy release then the 400 kB come from the statically linked C++ standard library you're using. Your code alone would be about 1-2 kB large. Add to that standard exe bloating and you'll end up with a 15-20 kB app.
The VB.Net app on the other hand needs 50 kB only for your code and .Net exe bloating...


Why is it so slow?
Because the programming language doesn't make a f***ing difference when all you're doing is accessing a 5MB file on the hard disk. The other computer probably has a faster hard disk, or maybe the file access is somehow more efficient (larger buffer or something), but it hasn't got anything to do with the language.

[jim mcnamara]

Or disk controller -

IDE vs SCSI for example.

[Buy2easy.com]

but the laptop that i am running the vb.net program on it about 3 or 4 years old and the desktop that i am running the c++ program on is a new dell deminsion w/a pentium 4 processor.

The debug thing was why it was so big but i still can not understand why the c++ code is slower accessing.


Now even with them both running on the same cpu the vb.net is still faster accessing.

I also thought that .net was slower than c++ or at least that is what i was told.

But the c++ program does take up way less of the processor than the .net program so i guess that is a plus

[CornedBee]

Write some complicated mathematical algorithm in both languages and see which is faster.

About the disk: it is well possible that MS was more careful implementing the .Net io than the C++ fstream. Or maybe you specified in VB some performance improvement options that aren't included in the C++ standard.

[made_of_asp]

.NET is far slower than C.

Try using C libraries instead of C++'s STL. It will drop your EXE size below 30k and will run faster. Use Win32 API, and it will be even faster.

[CornedBee]

Yeah, you can specify optimizing flags to CreateFile, which will really speed that up.

[amac]

The main problem with his program isn't that he is using STL or C++ for that matter. The problem is he is not using it "properly".

The STL vector class is just a "managed" array is it not? When I say "managed", I mean that resizing is done automatically.

Either way, the biggest problem with using them in looks is that
they allocate a new array, copy the contents, delete the old
(memory allocation is slow). Not only do you have to do this for
every iteration, but each string object will be copied(allocate
memory, copy contents). Don't forget that the old list has to be
deleted(calls to delete).

For instance, take a look at this.

std::vector<std::string> array;

array.push_back("Element 1");  // Array allocated to one, copy "Element 1" into index 0.

array.push_back("Element 2"); // Array allocated to two, copy old array(allocate memory for index 0, and copy
// string contents), delete old array, copy "Element 2" into index 0

array.push_back("Element 3"); // Array allocated to three, copy old array(allocate memory for index 0 & 1, 
// copy string contents for element 0 & 1), delete old array, 
// copy "Element 3" into index 2.

array.push_back("Element 4"); // Do I need to right this one out.

Anyway, the amount of memory allocation, deallocation, copy
string contents that is going on here I imagine become
exponential to the number of elements.

That being said. Try replacing std::vector with, std::slist and if
you don't have that (MSVC++ 6.0 doesn't have it), then try
std::list.

Lets see the difference.

std::slist<std::string> array;

array.push_back("Element 1");  // Allocate list node, copy "Element 1" into node.

array.push_back("Element 2"); // Allocate list node, copy "Element 2" into node.

array.push_back("Element 3"); // Allocate list node, copy "Element 3" into node.

array.push_back("Element 4"); // Do I need to right this one out.

It should be pretty obvious.

The other option that may speed it up is if you store pointer's to
strings in your list. That way string contents will not need to be
copied, only the pointer value.

You could also write your own allocator, so that you can allocate
say 20 items at a time rather than 1 (this would help more than
you might think).

Hope this helps.

[CornedBee]

I'm very sorry to say that, amac, but that's just plain wrong. Simple as that.

A vector's memory strategy is different. It's implementation-specific, but no implementation grows just by one every time something is added.
MS's implementation grows by 50%, SGI's doubles its size.
And the .Net collections need to grow too.

The string reallocations don't count, the .Net framework reallocates too, and allocation on a managed heap takes longer than on the normal heap.

So it would make sense to specify an initial size, but it won't speed up the thing very much. Neither will a linked list do.

StringArray Results(20);

[made_of_asp]

Try this:

void read(char* szFile)
{
DWORD dwRead;

HANDLE hFile = CreateFile(szFile, GENERIC_READ, NULL, NULL, OPEN_EXISITNG, NULL, NULL);

DWORD dwFile = GetFileSize(hFile, NULL);

LPSTR pHeap = (LPSTR)VirtualAlloc(NULL, dwFile+1, MEM_COMMIT, PAGE_READWRITE);

ReadFile(hFile, pHeap, dwFile, &dwRead, NULL);

CloseHandle(hFile);

VirtualFree(pHeap, NULL, MEM_RELEASE);

}

[CornedBee]

Another optimization (and a typo correction):

HANDLE hFile = CreateFile(szFile, GENERIC_READ, NULL, NULL, OPEN_EXISTING, FILE_FLAG_SEQUENTIAL_SCAN, NULL);

[Darkwraith]

How bout just using the get function and get each character individually. That should take care of the array and string allocations.

However, this method gets a little messy if you are looking for pieces of words...

[made_of_asp]

Originally posted by Darkwraith
How bout just using the get function and get each character individually. That should take care of the array and string allocations.

However, this method gets a little messy if you are looking for pieces of words...

reading character by character is slow. The code i provided above could be read character by character anyway, with minimal overhead.

[Darkwraith]

why is it slow?

We reduced all of the overhead of allocating strings and arrays. Our we got rid of everything except the algorithm which can be easily converted to assembler (the god of all optimizations.)

Just get one character. If they are not equal start back at the beginning and wait for the new word else start referencing the next character. Once the whole word has been reached, increment the whole find function.

[made_of_asp]

The file is much better off read in a buffer, than you can manipulate it all you want without moving through the file.

You can manipulate a buffer easier than a file (usually).

[CornedBee]

Darkwraith: the problem is only the hard disk access. Code is fast, hard disks are slow.

made_of_asp's method with my correction is the fastest because it does only one large access. The buffer in memory can then be manipulated quickly.

Using get takes many many accesses - depending on buffer size.

[Darkwraith]

Ah. So it is the get function that kills the speed.

Thx.