How to do a directory tree search on a remote computer

[jmsrickland]

I know how to do it on my home PC but how can it be done on my website hosted on a remote computer. I'm making an FTP application that connects to my website. This application has a tree view pane on the left and a folder pane on the right (kind of like windows explorer). When I first connect to my website I would like the treeview to fill up with all the folders and subfolders there are on my website.

[westconn1]

how are you connecting to ftp?

you could do this using APIs or shell object, possibly others
note shell object runs asyncronously so you need to test when each call is completed
shell object works much the same on ftp as local machine

[dee-u]

Perhaps you are looking for these APIs: FtpFindFirstFile, InternetFindNextFile, InternetOpen, InternetConnect. Try searching for them for clues.

[jmsrickland]

how are you connecting to ftp?

you could do this using APIs or shell object, possibly others
note shell object runs asyncronously so you need to test when each call is completed
shell object works much the same on ftp as local machine

I'm not sure that adds anything to my question. How I connect doesn't matter just as long as I'm connected, which I know I am as I have written the entire application to do FTP functions and they work, I just asked how to do a tree-view like display of all the directories.

So, what do you mean by shell objects? Are you saying that some shell object can create a tree-view of all the folders on a remote computer (that isn't even Window's based) the same as it does on your local computer?

[Doogle]

To some extent you're at the mercy of the Server's Operating system and the languages it supports. Irrespective of what you have to use to enumerate the structure you will need to send the data back to the Client so that the TreeView can be populated locally.

If you can use asp there's an example of displaying a directory structure here http://www.brainjar.com/asp/dirlist/. If you can use php there's another one here http://www.codingforums.com/showthread.php?t=71882.

Obviously you'll have to find an appropriate method of sending the structure to the client rather than display it on the server.

[jmsrickland]

To some extent you're at the mercy of the Server's Operating system and the languages it supports. Irrespective of what you have to use to enumerate the structure you will need to send the data back to the Client so that the TreeView can be populated locally.

If you can use asp there's an example of displaying a directory structure here http://www.brainjar.com/asp/dirlist/. If you can use php there's another one here http://www.codingforums.com/showthread.php?t=71882.

Obviously you'll have to find an appropriate method of sending the structure to the client rather than display it on the server.

I believe the directory structure is the same as it is on Windows. I know there are other things applicable to the remote computer that does not apply to Windown and vise versa but the basic tree structure is the same. What is needed I think is a program; like a recursive tree program, that creates all the folders and subfolders on Windows but it runs on the remote host and therefore must run in it's language (ie; not VB) or in Perl and run as a CGI.
Perhaps I will look into those two links you posted and see if I can find a way to use them in my VB application like executing them via a WebBrowser control.

[westconn1]

So, what do you mean by shell objects? Are you saying that some shell object can create a tree-view of all the folders on a remote computer (that isn't even Window's based) the same as it does on your local computer?

no, but you can use a shell object to connect and enumerate the directory contents, the same as if it is enumerating the directory on a local computer, only the url to the folder must include username and password as applicable,
i have successfully a windows shell object to connect to linux server ftp and perform file operations (no i was not filling a treeview of folders)

I know how to do it on my home PC

basically the same recursive search

How I connect doesn't matter just as long as I'm connected,

i would have assumed that whatever method you are using to connect would be an indication of what methods you would want to use to enumerate the directory

[jmsrickland]

I tried that link that Doogle posted in #5 for the PHP samples. They work except that they give the results in the order that the folders and files were created and not in the order that they are actually arranged. It would take me forever to try and figure out what goes where and in what order. Oh, well, so much for that approach.

I created a way to do it but I'm not satified that I have to do it the way I did. I went to my site using CuteFTP and manually copied the name of every folder there is by clicking on each folder and getting all the inner folders etc etc. I then made a list of these folders and arranged them in the order that they are supposed to be. I use this list in my program to create a tree-view of every folder with their subfolders. The big problem is that it only works for my site and I have to make sure I add any new folder I create on the site to the list I have on my PC, otherwise it gives the correct arrangement for the tree-view.

[westconn1]

here is a recursive routine to do what you want using a shell object
call the code from formload or refresh button
in my testing this was not fast, but i am using the slowest internet known to man (well except dial up)
also i had no idea i had so many folders on my ftp site

i have not really used a treeview control before, and got into problem with unique indexes (hence the hack when adding nodes), though i am not quite sure why as i thought i would have avoided duplicates, with the method i was using,
anyway, as you have done directory structure to treeview before, you can set your indexes anyway you like, to suit your own use, and remove my hack

Dim sh as object
Private Sub filltree()
Dim nod As Object, n As Object, initdir as variant
initdir = "ftp://username:[email protected]"
For Each nod In tv.Nodes
    nod.Remove
Next
Set sh = CreateObject("shell.application")
Set n = sh.namespace(initdir)
Set nod = tv.Nodes.Add(, , n.self.Name, n.self.Name)
listdirs n, nod
Me.Caption = "finished"
End Sub
 
Sub listdirs(f As Object, nod As Object)
Dim nod2 As Object, fitem As Object, i As Integer
For Each fitem In f.items
    If fitem.isfolder Then
        i = i + 1
        On Error Resume Next
        Do
        Err.Clear
        Set nod2 = tv.Nodes.Add(nod, tvwChild, nod.Key & i, fitem.Name)
        If Not Err.Number = 0 Then i = i + 1
        DoEvents
        Loop Until Err.Number = 0
        On Error Go To 0
        listdirs sh.namespace(fitem), nod2
    End If
    DoEvents
Next
End Sub

as you might note i did this using late binding, you can add a reference to shell controls and automation, then change your variables to appropriate types, no other APIs or references are required, except treeview (tv) change name to suit,
change ftp host, username and password as required
this same code will work on local machine with no change except initial directory to "c:\" or whatever drive

[jmsrickland]

WOW! That works really neat. But it sure is slow.

[westconn1]

But it sure is slow

any recursive directory search is slow, even on the local machine, as it has to test every file to see if it is a folder

i am sure there would be better ways to do it, you only have to look at the way firefox ftp addin does it

to improve speed, it should be possible to only populate the root, then populate each of the nodes as they are opened, here is modified code, also solves unique key issue

code moved to codebank
http://www.vbforums.com/showthread.p...95#post4006595

i tested on local harddrive runs seemlessly, pretty bloody good on ftp, even with this slow connection

i am certain that similar code can be used with the ftp APIs to produce identical results

edit: you can tell i was bored this morning

[Doogle]

i am certain that similar code can be used with the ftp APIs to produce identical results

If you're up for the challenge, there's a good 'starter for 10' at the AllApi site here: http://allapi.mentalis.org/apilist/6...B16373E9F.html

(I would have a go but don't have an FTP site to test with !)

[westconn1]

If you're up for the challenge,

taken,
see the second post in the codebank thread

without any timer testing i would believe the API version is slower, any comments on this?
of maybe just my internet is busier now

(I would have a go but don't have an FTP site to test with !)

if you want one, to play /test with, pm me

[Doogle]

if you want one, to play /test with, pm me

Thanks for the offer, but I don't think I can better your code

[jmsrickland]

I ran your code that you posted in #11. It's a much better way to do it. Still a little slow but better. You had one little quirks in it that prevented it from working beyond the first pass. When I first started it, it got all the folders on the first level OK. Then I clicked on one of the nodes and it got all the first level folders under that node. But when I clicked on a folder within that node nothing happened. I found that the problem was this:

Private Sub tv_NodeClick(ByVal Node As MSComctlLib.Node)
 If Node.Children > 0 Then Node.Expanded = True: Exit Sub
    
 listdirs sh.namespace(Node.Key), Node
      
 Node.Expanded = True

Node.Key = "ftp://username:[email protected]/wwwcgi-bin"

where it needed to be:

Node.Key = "ftp://username:[email protected]/www/cgi-bin"

So, I took another look at the sub listdirs(f As Object, nod As Object)

1) You had pathsep set to null so I initialized it with the "/". You did it in filltree() but overlooked it in listdirs().

Now the sub looks like this:

Private Sub listdirs(f As Object, nod As Object)
 Dim nod2 As Object, fitem As Object, i As Integer
 
 pathsep = "/"
      
 If f Is Nothing Then Exit Sub
      
 For Each fitem In f.items
   If fitem.isfolder Then
     Set nod2 = tv.Nodes.Add(nod, tvwChild, nod.Key & fitem.Name & pathsep, fitem.Name)
     'listdirs sh.namespace(fitem), nod2
    End If
   
   DoEvents
 Next
End Sub

pathsep would have to be changed to "\" for Windows however like you do in filltree().

The method I am currently using is lighting fast but it requires that I keep a local copy of the folders from my website. I might use your second version because it does get all the folders and subfolders and I don't have to keep a local copy even though it is quite slower it isn't that bad at all. I havent tried the other examples but I will to see how they pan out.

What I really need is your code converted to Perl (minus the tree view stuff) and have it run on the remote computer. I think that might speed up thing a bit but that's another problem I need to solve.

[westconn1]

You had pathsep set to null so I initialized it with the "/". You did it in filltree() but overlooked it in listdirs().

if you look at the code, i commented that pathsep should be declared globally (top of form code, in general section), that would have prevented that problem, and in the API version i had moved the declare to the module

What I really need is your code converted to Perl (minus the tree view stuff) and have it run on the remote computer. I think that might speed up thing a bit but that's another problem I need to solve.

perl or php has to depend on what is available on the each server, not all hosting sites make those available to users
i can certainly run php, but i have never tried anything like this in php

I don't have to keep a local copy even though it is quite slower

you would regularly have to update the local copy, to allow for changes, it should be possible to load from local copy then update the local copy in background, then refresh control, but i doubt i am going to code that as i was quite pleased with the result i achieved, most users expect some delay when using remote sites,
while i posted the code for your use i did not make it specific for one particular ftp site, though all the testing was done on one of my own, it should work on any ftp server, regardless of os


i was much happier with the shell object version than using the ftp APIs, though i did improve the API code slightly (not posted), the shell method seemed much more reliable, as well as easier to work with

of course neither method had any real error handling in case of no connections or time outs, of which i got a large number, or any other possible errors and would need some for a deployable app

[jmsrickland]

if you look at the code, i commented that pathsep should be declared globally (top of form code, in general section), that would have prevented that problem, and in the API version i had moved the declare to the module

I saw that but you didn't comment out pathsep itself and I saw the comment following it: 'declare globally and didn't really let it sink in. My bad.

perl or php has to depend on what is available on the each server, not all hosting sites make those available to users
i can certainly run php, but i have never tried anything like this in php

I realize that but since I'm using this for my own website I am not concerned about any other host on any other remote computer. Perl runs very fast on the remote host as I have programmed in Perl for many of my CGI applications. I'm not an expert in Perl but know enough to do what I have to do but I have never tried to write a Perl program that can do recursive tree searching. I don't know what the Perl translation of the code used in your example would be. Mybe someday I will figure it out.

you would regularly have to update the local copy, to allow for changes, it should be possible to load from local copy then update the local copy in background, then refresh control, but i doubt i am going to code that as i was quite pleased with the result i achieved, most users expect some delay when using remote sites, while i posted the code for your use i did not make it specific for one particular ftp site, though all the testing was done on one of my own, it should work on any ftp server, regardless of os

I am going to use your code from the first example you made as a seperate feature. I do not upgrade my website very often (like the last time was over two years ago when I added a few extra folders. The most I do nowadays is just add files to the existing folders. So, if and when I do add folders I will run your code to produce a complete folder tree structure and then copy that to my local directory listing. Then form that as I already do I read that in and build the tree-view which is like instantaneous after program loads for all further usage of the FTP client program. When I add more folders I will again use your code to re-create the local list. I like the speed of the way I do it but I like the way your sample gets the folders. Very nice and I appreciate your efforts and the code samples you posted. I learned alot from this and that is a big deal. Thanks.

[westconn1]

I saw that but you didn't comment out pathsep itself and I saw the comment following it: 'declare globally and didn't really let it sink in. My bad.

i clarified that in the codebank thread, and also specified sh must be global, as that was only showing as dimensioned in the original post in this thread

the global variables could be dimensioned within a procedurem then passed to each called procedure, though i am not a major fan of global variables they seemed appropriate in this case

[jmsrickland]

i clarified that in the codebank thread

Yeah but I don't read the thread; I just copy and paste the code into my Form and go for run

[westconn1]

Yeah but I don't read the thread; I just copy and paste the code into my Form and go for run

lol
i will remember to leave some easter egg for you

[jmsrickland]

OK, I changed my mind. My first intentions was to use your first version and run a complete tree search and then from that make a local folder fille. However, after implementing your second version and once it was compiled it ran pretty darn good; slightly slower that using the local copy but not enough to make any difference.