"VB Phone Home" With WebDAV/FPSE

One of the things I see programmers wrestling with a lot is a program that needs to be able to communicate back to home base. The need for this can range from product registration to self-updating to actual application purposes. The problem is self-hosting a 24-by-7 server or the expense and headaches of running a paid hosted service.

If your application can use a store-and-forward model of processing (think email), perhaps a thin server approach might serve.

While I'm writing about VB6 here most of the concepts apply equally to other development tools.


The Usual Suspects

There are several ways people do this, with varying success:

• Send an SMTP email.
• FTP a file.
• Use dynamic Web pages and POST data.
• Write a custom service and host it somewhere.

Email has two disadvantages. For one reason or another it won't work on every user's computer, mostly due to anti-spam efforts. Secondly it is a one way proposition: if your application needs to fetch data of some sort email is pretty much worthless.

FTP can be a solution, but unless you can use PASV mode it isn't especially firewall friendly. While there are a couple of secure variations of FTP there isn't much cheap (or free) support for them in VB6, so everything from the logon to your data travels over the 'net in plain text.

ASP, ASP.Net, PHP, CGI, etc. can be used to mediate back at "home base" (your data repository). If you want to write the application code (or scrounge prepackaged samples) and run a Web site this is a very flexible alternative. You also have the easy option of SSL/TLS for securing logons and traffic. Hosting can range from free to cheap and on up.

Developing a custom socket server and hosting it is probably the most flexible option. It is also the most work and probably most expensive to host. The downside is securing logons and traffic, since there are few options unless you want to pay for an SSL socket component or code the necessary encryption at the data level.


Another Option

Something most people seem unaware of is that as of MDAC 2.5 ADO has the ability to act as a connector to Web sites that support either FrontPage Server Extensions (FPSE) or WebDAV. This is accomplished via the Microsoft OLE DB Provider for Internet Publishing.

These are both mechanisms for making a Web site "read/write" though FPSE was primarily intended to support the Web author. WebDAV is intended to open up the Web as a more general read/write medium, but at the base feature level they offer similar capabilities.

It can be hard to find free or cheap WebDAV hosting, but there are offerings out there. Microsoft has deprecated FPSE in favor of WebDAV and Web Services, but it remains popular with cheap hosting providers and is even available in many Unix/Linux based hosting plans.

You can also configure IIS or Apache to support WebDAV and/or FPSE, which can be handy for local development and testing. If your ISP provides personal Web space for you the odds are even good that FPSE is supported.

And depending on your hosting, WebDAV/FPSE over SSL may solve any security issues you face.

However to avoid abuse you'll want to be sure you have control over security on the Web Root (or Virtual Directory) you use for your data. Otherwise people can come along and download what they want using regular anonymous HTTP! So look for the ability to secure your Web directory or subdirectories when choosing hosting for production purposes.


What Does It Get Me?

If we just look at the functionality subset in common among WebDAV, FPSE, and ADO you can:

• Upload a file.
• Download a file.
• Create an empty file.
• Update a file.
• Delete a file.
• Obtain several file properties.

WebDAV can offer more capabilities than FPSE. FPSE can do things that WebDAV does not (depending on the FPSE software). And some WebDAV and FPSE functionality is not available via ADO.

By the way, a full WebDAV server also allows things like:

net use z: https://my.host.com/folder

The resulting mapped drive can be opened via Windows Explorer and even accessed by some programs depending on how they operate on files. The necessary filesystem redirector is in most Windows OSs since late versions of Win95, but might need to be explicitly installed in Vista prior to SP1.


What Can I Do With It?

The possibilities are (ok, maybe not) endless.

Maybe you want your program to trap errors and format and post bug reports to your server. Maybe you want your program to check for new versions and suggest updating to the user. Maybe you want to have your program download updated data such as settings, databases of sales catalog and pricing information, images, contacts, etc. Maybe your program collects data offline and needs an option to post orders when the user has an Internet connection.

For simple retrieval your applications can always fall back on the INet control, VB6's AsyncRead method, WinHTTP, etc. to use straight HTTP. That will probably simplify coding for read-only access needs.

I suppose you could even build a "guestbook" or "feedback" form into your program. However WebDAV is document (whole file) oriented. This means even to append data to a file you must bring the whole thing over the wire, update it, and repost it to the server. But at least there is locking support to assist in handling concurrent updates.

In general I'd think writes to the server should probably be individual new files. These might then be retrieved by another application and merged or individually processed on a periodic basis. For example bug/crash reports might be stored into a local database.


The Demo

I have attached a small demonstration program. It isn't terribly sophisticated but it does demonstrate some of the basics of using ADO with the Internet Publishing Provider.

What it does is:

• Accept your server and folder URL, User, and Password and let you Connect.
• Upon connection it looks for a file named "test.log" there.
• If the log file exists, it displays a message and lets you proceed. if not it allows you to create an empty log file before proceeding.
• Then you can make one or more log entries or retrieve and view the current log. The log entries are simple timestamps in this example.
• You can exit the program, disconnecting if connected.

The demo seems to have a few warts I haven't overcome. Despite attempts at fairly aggressive error handling, failure recovery, and object release the EXE process sometimes seems to linger even after a normal program run. I can only assume that this is due to the way the underlying OLE DB Provider is working with Windows' WebDAV support under the covers. From descriptions I have read it may involve per-process Redirector sessions that may have to time out.

I find this troubling, and so far I don't know how big a problem it really is. However I have seen a normal termination linger for almost a minute and maybe more (I killed the process via Task Manager). Certainly long enough that when I quickly went back, made a small change, and tried to recompile I got a failure because the EXE was still in use!

Maybe somebody else can explain this and tell us whether it is something to worry about - and if it is, how to fix it.


My Conclusions

Using ADO 2.5+ with the OLE DB Provider for Internet Publishing seems to offer some real possibilities for certain classes of applications. For distributed applications that need a way to retrieve and post data to a central respository it offers a firewall-friendly and secure alternative to FTP, and one that supports file level locking.

WebDAV/FPSE may be a path toward distributed computing that avoids the need to run a sophisticated server application somewhere in the "cloud" available 24x7. Hosting is cheap and development and administration costs are low. Unlike the classical distributed processing model, your "central processing site" does not have to be the spider in the middle of the web. You can build any required processing capability into a special "client" at the edge.

So with all of that background in place I can get to my issue.

Aside from the primary point that this particular processing model and the technologies involved are inherently limited to loosely-coupled applications, why don't we see more people using it? It would seem to solve many of the problems people have using other protocols,

We end up with a standard, secure, and surprisingly widely deployed way of using cheap Web hosting as a central data exchange point. Underneath the covers it uses protocols (WebDAV and FPSE) that are basically a RESTFul Web Service with a less painful API on top. Certainly less painful for VB5/6 programmers.

In some ways it resembles Cloud Computing services like Amazon's S3 without the proprietary protocol and API.


Is it partly because of this being a relatively unknown technology? Or does it have "warts" that are difficult to overcome? Or are most people primarily creating isolated desktop software or else Web applications, and little in between?

The attractive thing for me is the way it handles firewall and proxy issues as well as the problem of self-hosting a server and having it up 24 hours a day.

No need to PM guys, reply right here.

One thing that might help is an overview that covers some of the same concepts: Architecting Disconnected Mobile Applications Using a Service Oriented Architecture.


Oh. And yes, to try the demo means you'll need to set up IIS, Apache, etc. with WebDAV or FPSE. Once again: you may have suitable webspace as part of your ISP package too.

There is another good article on the inner workings of WebDAV at Communicating XML Data over the Web with WebDAV.

It even introduces another library you might choose for doing WebDAV: MSXML (and its XMLHTTPRequest and XMLDOMDocument classes). Note however that this approach sacrifices interoperability with FPSE hosts, something extra you get using ADO.

Hmm, I wonder how well ADO.Net works with the Microsoft OLE DB Provider for Internet Publishing?


I see Vista added a WebDAV API too!

From the last two PMs it seems that a lot of VB6 programmers just are not up on using ADO for anything besides database access.

To help people get started I have tried wrapping some of this functionality up in an easy to use component. To make it easier for those who "don't want dependencies" (i.e. can't make a program setup I guess) it is packaged as a set of drop-in .CLS modules instead of a separate DLL.

Many of you will look at the code and conclude you'll just program directly against ADO yourself. It really is that simple.

So the classes are posted in an archive with a sample Project over in the CodeBank: WebSpring Class: Read, Write, Publish to Web from your VB6 Program.


Oh. And I meant to suggest: download the code and look at it before PMing. It really is not very complicated.