[RESOLVED] philosophy on storing data

[n_minus_one]

There are "text" files, there are databases, I used to use a record (now called a structure) in a ".dat" file, but I'm having trouble finding documentation support in vs2010. So I'm thinking maybe it's time to move up to ... ? databases? ".txt"s? The current app I'm trying to write will only have a couple of hundred "records" at the very most; Old vb6 command like LenB which would return the size of a file in bytes is no longer, and I can't find how len( .."as byte" of file) would actually be implemented. len( <filename>, as byte) isn't the correct syntax - what is?

Is there some thing less dramatic than a full blown database? Or getting used to how a database uses SQL and all that goes along with that shoud be something to get handy with?

[Shaggy Hiker]

If your data all fits into a class or into a series of classes or structures that could be put in a list, then you might look into serialization. Binary serialization would be similar to your .dat solution, but would be pretty simple. After all, the whole thing would be serialized and deserialized with about two lines. Another alternative would be XML serialization which would create an XML file (of course), which has the advantage/disadvantage that you can edit the file with any text editor. With multiple items, this might be easiest if the data fit into a datatable or two, since you could serialize that with one line. XML serialization for classes would take a little more, though not too much.

A database is certainly an option, and a couple hundred records is not so excessive, though if the data will never get larger than that, then I can see why you might not want to go that route.

[David Anton]

I agree - serialization is really an extremely attractive approach.
One warning though - while you can add fields to the classes you're serializing, if you rename or delete fields, then you won't be able to load the previously serialized file again - i.e., you'll lose your data if you're not careful.

[n_minus_one]

This is exactly what I need to consider.

I think I will implement a binary serialization approach. Thanks Shaggy! And the warning about deleting or renaming fields is well heeded. It will stop me from "tweeking". Thanks David.

[Shaggy Hiker]

One other issue about binary serialization which may or may not matter to you is that the information about the assembly is included in the serialized file. What this means is that if you serialize the data in one assembly, you can't deserialize the file in a different assembly. Where this bit me was when I was working on a distributed robot brain. Messages were all in structures that were serialized into a binary string and sent out to other computers via UDP. The programs running on the other computers were not the same program as the one that did the serialization, so they couldn't deserialize the message. The solution was to put the serialization/deserialization into a class library that could be referenced by any program that needed to perform those actions.

[n_minus_one]

Wow, that sounds like a great project!

I'm still in the starting blocks with binary serialization. Do you know of a link to a demo? I have an example of an XML SerializableData class, but not for binary, which would be my first choice. Just a simple "getting started" sort of thing.

[Shaggy Hiker]

Search around on this forum. People seem to post examples about once every two months. In other words, it isn't a very common post, but there are plenty of them. I think I have posted examples twice, but it was well over a year back. Try searching on serializable() or just binary serialization. The thing is that there will be VERY little code. I believe that, once you have the file name, the actual serialization takes two lines, so there isn't much to see. When I have used it, I have spent FAR more code getting the filename using the SaveFileDialog or OpenFileDialog. The actual serialization/deserialization is forgettable.

[David Anton]

Use the 'Serializable' attribute on your class you want to serialize.
e.g.,

<Serializable> _
Public Class Foo
...
End Class

Public var As New Foo()

'writing to the serialized file:
Dim myWriter As New StreamWriter(mySettingsFile, False, System.Text.Encoding.Default)
Dim myFormatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()
myFormatter.Serialize(myWriter.BaseStream, var)
myWriter.Close()

'reading from the serialized file:
Dim myReader As New StreamReader(mySettingsFile, System.Text.Encoding.Default, True)
Dim myFormatter As New System.Runtime.Serialization.Formatters.Binary.BinaryFormatter()
Dim myStream As Stream = myReader.BaseStream
myStream.Seek(0, SeekOrigin.Begin)
var = CType(myFormatter.Deserialize(myStream), Foo)
myReader.Close()

[Shaggy Hiker]

There are other ways to do it, because I am sure I didn't use that code for reading (deserializing), but that's still a good example.

[n_minus_one]

This:

 Private Sub WriteFile()
        Dim bf As New BinaryFormatter

        Using fs As New FileStream(BIN_PATH_FILE, FileMode.Create)
            'D:\VB2008 Programs\GenerateEventList\GenerateEventList\bin\Debug\EventList.dat
            Try
                bf.Serialize(fs, EventArray)
            Catch ex As SerializationException 'Exception
                MsgBox("Failed: " & ex.Message)
                Throw
            End Try
        End Using
    End Sub

creates the error: ... is not marked as serializable.
The ... is a whole whack of text.

EventArray is declared globally and is an array of structures (all the same)
I noticed that <serializable> _ was at the top of your code David - is this a class def'n? And would I need to create a class for my EventArray ? ?? Currently the only class is the form.

[Shaggy Hiker]

Your structures would need to be marked with that serializable attribute, but I would not expect that the collection would need anything further.

[n_minus_one]

   <Serializable()> Public Structure EventRec
...

for the EventRec structure works for writing to a file, I just need to check and see if it also does the trick for reading it back into the array.

Would marking...

 Dim EventArray(1000) As EventRec

with <Serializable> be better, the same, or just wrong?

Do I need all of these imports for binary serialization?

Imports System
Imports System.IO
Imports System.Runtime.Serialization
Imports System.Runtime.Serialization.Formatters.Binary
Imports System.Runtime.Serialization.Formatters.Binary.BinaryFormatter
Imports System.Collections

[n_minus_one]

sorry

[n_minus_one]

Just like you mentioned Shaggy, I am trying to create a library class 'cause I need two programs to use the serialized file.

I'm not sure where the declarations go. Basically I want to fill an array of structures (EventArray) from a serialized file; does the declaration: EventArray(1000) belong in the program: GenerateEvents or in the DLL: EventDBS ? How about the structure for the EventRec declaration?

I've created a EventDBS.dll with the following:

Imports System.IO
Imports System.Runtime.Serialization
Imports System.Runtime.Serialization.Formatters.Binary

Public Class EventDBS

    <Serializable()> Public Structure EventRec
        Dim EventDate As Date
        Dim EventType As Integer    
        <VBFixedString(256)> Dim Desc As String
    End Structure

    Const BIN_FILE_NAME As String = "EventList"
    Public BIN_PATH_FILE As String = Environment.CurrentDirectory() & "\" & BIN_FILE_NAME
    Public EventArray(1000) As EventRec

    Public Sub WriteDatFile()
        Dim bf As New BinaryFormatter

        Using fs As New FileStream(BIN_PATH_FILE, FileMode.Create)
            'D:\VB2008 Programs\GenerateEventList\GenerateEventList\bin\Debug\EventList.dat
            Try
                bf.Serialize(fs, EventArray)
            Catch ex As SerializationException
                MsgBox("Failed: " & ex.Message)
                Throw
            End Try
        End Using

    End Sub

    Public Sub ReadDatFile()
        Dim bf As New BinaryFormatter

        Using fs As New FileStream(BIN_PATH_FILE, FileMode.Open)
            Try
                EventArray = bf.Deserialize(fs)

            Catch ex As Exception
                MsgBox("Failed: " & ex.Message)
                Throw
            End Try
        End Using
    End Sub
End Class