best way to store folder structure?

[rnd me]

i am gathering a list of all the files and folders on my computer. i have found an extremly rapid method for getting the info into listboxes.

http://vbnet.mvps.org/index.html?cod...apicompare.htm

however, i don't want as many listboxes as folders in my app, as the routine puts subitems in a list. i have no probs harvesting a list, but i need somewhere to put all of the information.

can i create something that could be called like:
c.windows.system32.etc.

i worry about the filename "." in that possibility.
an array would be acceptable, but how could i dimension it for an unknown number of files at design time?

would i be better off with a collection?

i can make a FSO collection of all files, but i don't need all of the extra info in the file object, only the filename.

i have about 300,000 files, and FSO takes a long time to build.

any advice?

[bushmobile]

you already have a list all the files and folders on your system in the correct structure - why would you want to create another one?

[rnd me]

you already have a list all the files and folders on your system in the correct structure - why would you want to create another one?

to be honest, i can't legally say what this is for exactly, but it is of the utmost importance for my company.

[Shuja Ali]

How about using the Dir$ function to get all the files and then same them in an array?

[rnd me]

How about using the Dir$ function to get all the files and then same them in an array?

ahh, but what kind of array?
i COULD use a plain 2d one, but that needs all of the extra, redundant path information.

the storage of this information has to be extremely efficient for our servers to manage.

i can't figure how to code an array with several dimensions; the only way i see would be a dimension for each folder level, which once you get deep into documents and settings, likely exceed's vb's dimentional limit, not to mention being a bear to manage.

i wish i could explain it more, but i am not even sure of what i need at this point, other than that i need the most efficient string storage (datasize wise) of a PC's filenames.

any suggestions are very welcome. i don't need code perse, but a nudge in the right direction would really make my day!

thanks for looking!

[Shuja Ali]

A simple one dimensional array would be enough for storing the names/paths of the files.

I would still ask the same question as bush has already asked. What is the need for this kind of program?

You could also do one thing, instead of loading everything at once in an array, you can do it on demand. Say for example, you can first create an array of all the drives and when you need to assess the root folder of some drive (say C you will build the folder/file list of the root folder only. Similarly for all the other sub-folders or files.

[rnd me]

again, i wish i could, but can't go into specifics. i can say that it can't be done on demand, because the data needs to be archived. there are also stringent bandwidth requirements neccesitating the most streamlined transportation possible. we WILL need to get more servers to impliment this project, thats not a question. the question is how MANY servers, and that depends on my ability to efficently move and store the data.
redundancy bad! hehe

it is quite the dilema.
i appriciate your taking the time to help!

does anyone know if UDT's allocate ram to empty stings?
i am thinking perhaps a type array, with a simple parent ref as long, and a filename as string...

[Agilaz]

i am thinking perhaps a type array, with a simple parent ref as long, and a filename as string...

i think that would be (almost) the best solution. i was already working on something like this.

Option Explicit
 
Private Type DirArray
    lParentIdx As Long
    baName() As Byte
End Type
 
Private udtDirs() As DirArray
 
Private Sub whatever()
    
    ReDim udtDirs(5)
    
    udtDirs(0).lParentIdx = -1
    udtDirs(0).baName = StrConv("C:", vbFromUnicode)
    
    udtDirs(1).lParentIdx = 0
    udtDirs(1).baName = StrConv("windows", vbFromUnicode)
    
    udtDirs(2).lParentIdx = 1
    udtDirs(2).baName = StrConv("system32", vbFromUnicode)
    
    udtDirs(3).lParentIdx = 1
    udtDirs(3).baName = StrConv("system", vbFromUnicode)
    
    udtDirs(4).lParentIdx = 0
    udtDirs(4).baName = StrConv("Program Files", vbFromUnicode)
    
    udtDirs(5).lParentIdx = 2
    udtDirs(5).baName = StrConv("etc", vbFromUnicode)
    
End Sub
 
Private Function GetFullPath(ByVal idx As Long) As String
    Dim sTemp As String
    
    Do
        sTemp = StrConv(udtDirs(idx).baName, vbUnicode) & "\" & sTemp
        idx = udtDirs(idx).lParentIdx
    Loop While idx > -1
    
    GetFullPath = sTemp
    
End Function
 
Public Sub test()
 
    whatever
    Debug.Print GetFullPath(5)
    
End Sub

does anyone know if UDT's allocate ram to empty stings?

yes, every empty string needs some RAM. infact every empty variable needs RAM.

you can use a byte array to store the string. that will save you a lot of memory.

[rnd me]

ok, i just spoke with the man, and he allowed me to give some details. please pardon the obfuscation.

a company wants a record of all files on a workstation for a given point in time, a snapshot. this "image" must be made during shift change, giving little more than 5 mins for *several* workstations to report.

the quick harvest is CPU and disk intensive, and as such interfers with the machines usability during the procedure. it cannot be done in the background because, again, they want a "snapshot" at a known time. (after the employee leaves)

this info is then uploaded to a remote (wan) server. the DSL in use at the site should not be brought to a standstill during this upload, so efficiency is paramount.

it seems like a simple enough process, but there are obviously a lot of restrictions and complications to the problem.

i can image the HD in our goal of 2 mins w/ FSO, but sending every workstations data in the remaining 3 proves elusive.
scrapping remote FSO objects, we talked them down to just a file name.

i also said this would be "no problem", so needless to say, i want to solve the problem.

i need to know the most space-efficient, computationally friendly manner in which to store short heirarchical string names and relations.

any advice?

[Agilaz]

compress the data before sending it over the DSL line. text has usually a good compression ratio.

another way would be incremental file lists. for example make a full snapshot once a week and transfer only a list of files/folders added or removed the rest of the week.

[rnd me]

yes, every empty string needs some RAM. infact every empty variable needs RAM.
you can use a byte array to store the string. that will save you a lot of memory.

would using an integer in the UDT half the data of a long? folders dont go deeper than 256. hmm. could i use a byte?

thanks for your post, this is exactly what i am talking about.

[Agilaz]

an Integer uses 2 bytes, a long uses 4 bytes. but you can't use an Integer because the parent index can (and most likely will) be much higher than the max integer value.

[CVMichael]

I have a few questions:

how do you send the data ? using winsock ?
On the other side (the server / the program that receives the data) is it a program made by you (or you will have to make) ?

Do you HAVE to read the whole hard-drive all at once ?

I asked the questions because I think there is a better way to do this.

Look into ReadDirectoryChangesW API, you will find that you can use it to detect exactly when a file is changing (read/write/detele).

How do you think an anti-virus knows when you save a file on the hard-drive to check it for viruses ? It uses ReadDirectoryChangesW API...

If you use this API, you can make a service program that will run all the time on the computer, whenever it detects a file change, it will send the data to the server over the internet using winsock.
The receiving program/server can store the data in a database, and save the time the file changed also.

This way it does not take any noticeble CPU since the changes are done at once are very small, the user won't even know the service is doing what it's doing.
Also, this way, with a quick SQL querry, you can have a list of all the files that existed on the computer at ANY time, it's precise to the millisecond.

The only problem is... this way is more difficult to do.