Results 1 to 22 of 22

Thread: Reading data from files [RESOLVED]

  1. #1

    Thread Starter
    Hyperactive Member Jareware's Avatar
    Join Date
    Nov 2000
    Location
    Silicon Valley, CA
    Posts
    275

    Question Reading data from files [RESOLVED]

    I have a textfile a bit over 2 MB of size, and it takes painfully long to open it (almost 60 seconds). I'm using the following code to input the data:
    VB Code:
    1. Open App.path & "\index" For Input As #FF
    2.         Do
    3.             Line Input #FF, inputString ' Read a line
    4.             wholeLog = wholeLog & inputString & vbNewLine ' Append newly read line
    5.         Loop Until EOF(FF)
    6.     Close #FF
    Is there some faster method of reading the data from the file?


    -JR-
    Last edited by Jareware; Jan 6th, 2003 at 09:26 AM.
    "Blessed are we who can laugh at ourselves for we shall never cease to be amused"
    - Unknown

  2. #2
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    It will be the append command thats slowing it down not the read. I have a program that reads all lines in text files and its like lightning. Comment out the append line and see how fast it is then
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  3. #3

    Thread Starter
    Hyperactive Member Jareware's Avatar
    Join Date
    Nov 2000
    Location
    Silicon Valley, CA
    Posts
    275
    What would be the fastest way to store the entire file in a string-variable then? I am myself writing a file searching application, and the file I'm opening is an index of some 600 word documents (that have been parsed and useful data from them has been extracted and stored in XML in the index file).

    Thank you for your interest in my problem - my project is not making much progress while it takes a minute for the program to load!


    -JR-
    "Blessed are we who can laugh at ourselves for we shall never cease to be amused"
    - Unknown

  4. #4
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    try using binary mode so you can read the whole file into one string without loops, use something like this:
    VB Code:
    1. Open "file.dat" for Binary As #1
    2. Dim FileData As String * LOF(1)
    3. Get #1, , FileData
    4. Close #1
    that should dump the whole file into the string
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  5. #5
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    Yes i agree. Apening for Binary access reads the whole file in in one go whereas opening for Input allows u to read line by line (each line terminated by a newline).

    reading the whole file is faster in Binary mode but then you have the problems of finding the things u want from this huge file.
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  6. #6

    Thread Starter
    Hyperactive Member Jareware's Avatar
    Join Date
    Nov 2000
    Location
    Silicon Valley, CA
    Posts
    275
    Cyberius: The code you posted gives me a compile error (expected: end of statement) on the Dim-line. I have never played around with binary access to files, so what could be wrong?

    Blobby: I have already made an xml-parsing engine that takes care of that and it works perfectly, so that is not an issue.

    So how do I input the entire file into a variable using binary access?


    -JR-
    "Blessed are we who can laugh at ourselves for we shall never cease to be amused"
    - Unknown

  7. #7
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    Originally posted by Jareware What would be the fastest way to store the entire file in a string-variable then?
    i was just answering the question
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  8. #8
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    my mistake...try this, it should work now
    VB Code:
    1. Open "file.dat" for Binary As #1
    2. Dim FileSize As Long
    3. FileSize = LOF(1)
    4. Dim FileData As String * FileSize
    5. Get #1, , FileData
    6. Close #1
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  9. #9
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    In Cyberius's post the entire file will be in the variable 'Filedata' which should be dimmed as a string
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  10. #10

    Thread Starter
    Hyperactive Member Jareware's Avatar
    Join Date
    Nov 2000
    Location
    Silicon Valley, CA
    Posts
    275
    Still gives me a compile error, now "Constant expression required", on the same line still.


    -JR-
    "Blessed are we who can laugh at ourselves for we shall never cease to be amused"
    - Unknown

  11. #11
    Frenzied Member macai's Avatar
    Join Date
    Jul 2001
    Location
    Napanoch NY
    Posts
    1,228
    VB Code:
    1. Dim x As String
    2. Open "C:\somefile.txt" For Input As #1
    3. x = Input(LOF(1), 1)
    4. Close #1
    There!
    Luke

  12. #12
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    i DID dim it as a string
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  13. #13

    Thread Starter
    Hyperactive Member Jareware's Avatar
    Join Date
    Nov 2000
    Location
    Silicon Valley, CA
    Posts
    275
    Thank you very much, got it working now, and it's fast as a breeze!


    -JR-
    "Blessed are we who can laugh at ourselves for we shall never cease to be amused"
    - Unknown

  14. #14
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    that what i get for not actually trying the code, lol....someone else comes in and steals the glory
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  15. #15
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    hehehe Cyberius so u did! Im going code blind,....must be this snow we are having
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  16. #16
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    snow...hehe.....i wish.....take a look at my location....hot and humid summer
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  17. #17
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    yes indeedee, bitterly cold here. As this post is now finished and the authors buggered off.....fancy a pint?
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  18. #18
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    dont mind if i do
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  19. #19
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    well isnt that a coincidence, your 'hollow-headed-homer' logo is the same themed image i have on my WinXP logon screen...this can only mean one thing...great, yet incredibly small minds think alike
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

  20. #20
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    hehehe Yeas but ive got a bigger head than you cos my brain is smaller than yours....so there!
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  21. #21
    Frenzied Member Blobby's Avatar
    Join Date
    Oct 2001
    Location
    England
    Posts
    1,512
    ..........and its your round..........
    There are 3 types of people in this world.........those that can count, and those that can't.

    Blobby

  22. #22
    Addicted Member Cyberius's Avatar
    Join Date
    Jan 2003
    Location
    Australia
    Posts
    145
    right you are.....*buys a round of drinks*
    i think we've officially made this a 'Chit Chat' thread now lol.....or more to a point, a thread about big heads and small brains
    -=[Ç¥ßè®Ìú§]=-

    How many microsoft employees does it take to change a lightbulb? None, they simply define darkness as the new industry standard.

    CAUTION: OVERCLOCKING A 386 TO 5Ghz MAY BE HAZARDOUS

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width