Reading from a file

[SteveCRM]

I need to read from a text file. It is the html of a search engines results page, so its a lot of html

what would be the best way to load it and be able to take parts out etc (im stripping the links out)? I don't think loading it into a huge variable would be very effiecient.

[Smithy006]

Hi Steve,

You'll need to directly access the text file, read a chunk of it and store it in the temporary value then check if the chunk has a link in it if so then store it to where ever you need to store the link to.

You'll need to open the file using fstream.h, use a for statement to keep reading until EOF. You'll be able to identify what a link is because A "href=" will be in front of the link.

I hope this helps.

Best Regards,

Ben Smith.

[SteveCRM]

so how large of sections of the file do you think I should take at a time?

Thanks

[CornedBee]

About 1 MB maybe? It really depends. If you allocate the buffer on the stack you shouldn't use more than 500 k because the stack is only 1 mb large. I usually read in the whole file no matter how large it is. You can also learn about VirtualAlloc or file mappings. Both should help you.

[parksie]

Originally posted by Smithy006
You'll need to open the file using fstream.h, use a for statement to keep reading until EOF. You'll be able to identify what a link is because A "href=" will be in front of the link.

Actually, fstream on its own.

fstream.h is part of the old iostreams library and is officially deprecated.

[Zaei]

fstream is just nasty.

Z.

[SteveCRM]

So then what should I use to read my file?

[abdul]

You can also use the C, FILE, class and other functions to work with the files.
Here is an example:

FILE *myfile;
char c[5];
myfile = fopen("myfile", "r");
do {
      c = fgetc (myfile);
      count<<c;
    } while (c != EOF);

fclose(myfile); 

[jim mcnamara]

#include <stdio.h>
// I just chose 8193 because 8192 byes are sector size on my work box
void main(void){
   char big[8193];
   FILE *in;
   in = fopen("myfile","r");
   while (!feof(in)) {
          if ( fgets(big,sizeof(big),in) != NULL{
                      process(big);
          }
   }
}

void process(char *t){
        // do stuff to your string here
}

[parksie]

Although you should really use iostreams for I/O in C++...otherwise you're using C.

Just get Kedaman in here, he's been straightening me out on the whole "get your act together and write OOP C++ rather than just bunging things into classes" (my words, not his )

[Zaei]

I refuse to write slow C++ code, when I can just as easily write fast C code inside of a class (if that makes any sense =). fstream would require lots of fun looping stuff to write lots of data, while C style File IO can spit it out a single call (most cases).

Z.

[parksie]

Originally posted by Zaei
I refuse to write slow C++ code, when I can just as easily write fast C code inside of a class (if that makes any sense =). fstream would require lots of fun looping stuff to write lots of data, while C style File IO can spit it out a single call (most cases).

Z.

Umm...iostreams aren't limited to >> and <<. You can still use .read and .write I think, or something similar.

[Zaei]

Yeah, you can (now that I take the time to look), but I find it a WHOLE lot easier to use C Style functions (guess its just an opinion thing).

Z.

[parksie]

Yeah, nobody's stopping you - everyone can make their own choices

Although admittedly, if I'm going for size I have a small wrapper class around CreateFile

[SteveCRM]

jim I got your code to work in a console, but in my windows program it keeps crashing.

Im using URLDownloadToFile....maybe it can't make the file quick enough before it reads it???

[CornedBee]

CreateFile is the official WinAPI way. Then there are the file stream classes, Ansi C file operations and Win16 OpenFile. I think it's just a matter of taste. CreateFile will probably be the fastest because the windows CRT and C++RT are just wrappers around API calls (ever looked at the CRT source of printf?)