Best way to Open a File And Parse It?

[Tec-Nico]

Hello, I am working on my own File Format and then parse it after opening the file... But I am having a couple of issues with the speed of the procedure so I would like to ask the following:

1.- Should I open the whole file and then parse it? Or should I open it part by part and parse it that way? Which would be the best way?
2.- Which will be the best way to save the file so it won't be so big? Right now I am saving it as ASCII, I think.
3.- Which would be the best way to attach strings? I read plenderj's string attachment procedure but something is wrong with it... Should I use an array of Bytes? If so... How can I do that?
4.- Which would be the best way to draw a picture on a form? I use the API set Pixel... Some would say BitBtl but since the File Format is of my own, how could I use it?

Thanks for your attention... Don't answer all questions if you can't, if you are able to help me with just one of them I will be very happy.

[dglienna]

1- Yes, as long as it isn't many megabytes.
2- Nothing wrong with text files
3- Attach to what? Just use

Print #1, myStr

4- the LINE method works, and lets you draw and fill boxes using different colors.

[Tec-Nico]

Thanks for answering, David. What I am doing is to create my own format for saving pictures but they are a little bigger than gifs and it takes longer to load a text file and parse it than doing it from a bitmap and I think it's because of the time it takes to manipulate strings.

The file can turn into MBs if the picture is big and that's why I am asking...

(Added: )
When saving a picture I need to attach a new part to the string that is holding the whole file. I do it this way:

strFile = strFile & strLine

[dglienna]

I don't see why you are creating your own picture format. There are plenty of good ones out there. JPG are compressed, but lose if you zoom in close enough.
RAW is the newest format out there. I saw it demo'd the other night.
It is loseless, and a little bigger than jpg. Newer digital cameras use it.

[Tec-Nico]

I am creating the format for my own program.. so the painting will be faster and the picture would store some features that would be easier to read from the program than saving them into a different file. Am I making sense?

[penagate]

Yep. I would use a UDT header at the start of the file to describe the picture following it, and then read it in two parts, UDT first, and using the descriptive info in the UDT, read the picture itself. You can use Get in binary mode to read bits of files.

[penagate]

Uh, I just read your bit of post about strings. Don't use them. Use byte arrays, much much quicker. You have control over exactly where everything is at all times; something that's hard to achieve with strings unless you know exactly what you're doing, and it is much more complex.

[Tec-Nico]

could you give me an example penagate?

[penagate]

Sure.

Load a UDT file header:

' Something like this
Type FileHeaderThing
    szPictureName As String * 255
    lpPictureData As Long    ' Pointer to the start of the picture data within the file
    cchPictureLen As Long    ' Length of the picture in bytes
    dwWidthPx As Long
    dwHeightPx As Long
    dwBitsppx As Long
End Type
 
' Open the file
Dim lFF As Long: lFF = FreeFile()
Open [filename] For Binary Lock Write As #lFF
 
' Read the header
Dim udtHeader As FileHeaderThing
Get #lFF, , udtHeader
 
' Get the picture data
Dim chPicture() As Byte
ReDim chPicture(udtHeader.cchPictureLen)
Get #lFF, (udtHeader.lpPictureData), chPicture

Hope that gets you started

[Tec-Nico]

and what about using arrays of bytes instead of strings?

[penagate]

Um, under the " 'Get the picture data" heading

How you would display a picture contained in a byte array, I don't know. That is a different matter. Are you creating your own picture format or saving a StdPicture sort of object?

[Tec-Nico]

I see it.. but I meant for when I would have to read every pixel from a picturebox in order to save the file.. Instead of using the string I am putting all into I would like to see how to do something like:

a = a & "b"

Should I use the redim everytime I am going to add a new byte? How can I handle it when I am going to save it?

[penagate]

You are reading each pixel individually?? Oh goodness, that is going to be slow... Just Put the contents of the picture into the file, and then when you want to read it, read it into a StdPicture object.

' Save it
Dim udtHeader As FileHeaderThing
 
' fill the UDT with details
udtHeader.lpPictureData = LenB(udtHeader) + 1
 
Open [filename] For Binary Lock Write As #lFF
Put #lFF, , udtHeader
Put #lFF, (udtHeader.lpPictureData), picPicture.Picture
 
' Read it
Dim objPic As StdPicture
Get #lFF, , udtHeader
Get #lFF, (udtHeader.lpPictureData), objPic
 
Set picPicture = objPic
Set objPic = Nothing

I think that should work.

[Tec-Nico]

yes, because i am translating the picture into my format.. i am going to continue this tomorrow.. i am very tired now. thanks for your answers and i hope i can ask you more if i have some other doubts.

[Tec-Nico]

Hmmm... Penagate, I have another doubt.
The problem is that the size of every line of the picture has a different length.

Example:
Let's say there are 30 red pixels... So my data would look like:

255x30

How would I be able to read that when using Bytes? And how would I be able to store it? Right now I am still playing with the code but I would love to have more pointers so I wouldn't have to spend a lot of time analyzing and trying to understand what I am doing..

[VbMafia]

You can use API to do not process

[Merri]

This might help with the graphics problems

Those all are 8-bit images made in real time, but it shouldn't be impossible to change it to 32-bits. It actually might be easier to handle it in 32-bits as you don't need to worry about "dividable by four bytes per row" thingy. The codes introduce the use of DIBs, Device Independent Bitmaps.

If interested, I also have a Transport Tycoon Deluxe GRF file reader, which reads several sprites from a TTD graphics file and displays the selected one. 8-bit as well, so no many changes in the techinal side...

[Tec-Nico]

Thank you for your help guys.

When I thought about handling byte arrays instead of strings your name came to my mind, Merri. That was also because of the speed issue.

I appreciate every comment made to me regarding this project that is about to be started. After making some tests with get and set pixel API, I developed my own format but translating it was pretty slow and so was reading it from the file even when drawing the picture was faster.

What's special about this format? It might have been already discovered by Merri: The pictures are built out of what could be a description, so the colors might change depending on this. The idea is to have general pictures and then customize them based on small text descriptions.

The problem here is that the files need to be the smallest possible and able to support animations.. They are going to be send through the internet and the transfer should be fast enough so people wouldn't notice when it happens.

I am going to take a look at the MetaBalls project and I am interested in the Transport Tycoon Delux GRF file reader and in everyone's suggestions. Thank you to all

[Merri]

Here is the GRF reader. Originally meant to be a convertor, but I never got as far as coding the editing and saving. TTD community is still alive very well and the game hasn't died. Going strong for a ten years old game

Anyways, you ought to find good tips for handling a file in the project.

[Tec-Nico]

Thanks for the reader... I will keep posting my doubts here if it doesn't bother you, guys. A ten year-old game? is it going to be finished soon?

I hope the tips are in english this time, and again, I really appreciate your help!

[Merri]

A ten year-old game? is it going to be finished soon?

Seems very unlikely, they talk actively about TTD development on IRC, there are several projects going on which include creation of new vehicles, new tools for building them, TTDPatch which edits the original game, OpenTTD which is a custom coded C version of the game... just a lot of stuff