XML and ASP

[CiberTHuG]

Okay, I have an ASP that will call a server side DLL to query a DB and do some heavy data washing. The ASP could do it, but the DLL is faster. The DLL will return an XMLDOMDoc to the ASP. The ASP will get the numbers and build the HTML to present them to the user.

I'm not sure if I'm going to mess with XSL and such, but my question is...

Can I keep this XML in memory and not hit the disk? If the user wants to change some parameters and rerun the report, I don't always have to do this data washing over again. He may just be asking me to ignore certain peices. I want the ASP to pass it to itself basicly. Any ideas?

[CMangano]

Well, if you don't mind limiting your users to IE5 + only, then you can store the XML in memory and allow them to manipulate the data on the fly. This is very fast and very effecient, but very incompatible

Netscape does not support XML (although I have heard Netscape 6 does, but have not tested it myself). Therefore, if you want to support users in Netscape 3+ or 4+ or IE 4+, then you will have to use the XML on the server.

But, if you do find a way to do this and support older browsers, by all means let me know!

[CiberTHuG]

Will do, but I was hoping that there was a way that didn't involve passing the data to and from the client.

Kind of like a server side session variable. Make sense?

[CMangano]

I think I see what you mean. You don't mind going back to the server to edit the XML, you just don't want to have to re-create the XML file every time (meaning calling the DLL again).

There should be a way. Just thinking off the top of my head (and I am sure there is a better way), you could create an XML file from that DLL, and have its name correspond to the users SessionID. That way every user has there own XML file. Then you parse the XML file on the server without having to go through the DLL (using the XML DOM, not FSO).

Again, this is just a quick fix, and I am sure there is a better, more effecient way of doing it.

[CiberTHuG]

I think I see what you mean. You don't mind going back to the server to edit the XML, you just don't want to have to re-create the XML file every time (meaning calling the DLL again).

Exactly.

Yeah, I know I could save the XML to a file. I was just trying to avoid that. I want to keep the XML in memory for as long as I can, to cut down on disk IO. After twenty minutes or so of being idle, the client-side session vars expire. Around that same time, I don't mind if the server forgets about the XML.

Another problem I have with saving the XML to a file, is now that file will have to be removed. If the user just abandons the site, then the ASP can't call the delete on that file. Now I'm forced to have a watchdog script that will clean up these files that are older than some specified time.

Anyway, you are on the same train of thought I'm on. I realize I could use a file, I'm just trying to avoid it. And I may not be able to avoid it.

[CMangano]

Not sure if this will help you, but may be worth a read. It sounds like it is similar to what you need to do:

http://www.15seconds.com/Issue/010409.htm

If you figure it out, by all means let me know. I really would like to know!

[Cander]

Put the document as a string into a session variable.
Then you can load it ike you would a xml file with on difference

instead of
xmldoc.load "c:\myfile.xml"

use

xmldoc.loadxml Session("MyXML")

Is this what you want?

[CiberTHuG]

Thanks, the 15 Seconds article looks like it may help with the XML manipulation. I was thinking that if the user asked to ignore some of the data I was going to have to step through each node and see if it should be counted or ignored. Instead, I can kind of do some querying against the XML. Probably much faster.

Cander, that idea is a good one. I can just dump the raw XML into the session variable. That keeps me from having to throw it to disk. Question, do session variables get sent to the client's machine?

If so, then I'm throwing several dozen KB back and forth between the server and the client. Mind you, it may never get big enough to be noticable (though it may be slower than disk caching it doesn't require clean up), but I'm just wondering incase it should become a problem. I don't know, yet, how big this XML will be.

[Cander]

Session variables only stay and get processed on the server. So you will be ok.

[CMangano]

I thought about the idea of storing the XML in a session variable, but thought it would cause more performance issues then the text files. This method would work, but if you have a lot of concurrent users, they are all gonna be lugging around this string of XML data, even when they go to a different application on the site. Also, even if they leave the site, the session remains (20 min is default), and that means there string of data remains 20 more minutes. This is why you hear a lot of people say "Session Variables are Evil."

Now, if you have very few concurrent users, and you somehow get rid of the session variable when they leave the application (meaning they don't need the XML anymore), you can avoid this. But I am not sure it is worth the trouble. Any thoughts?

[Cander]

It actually works very well, I use this technique on a website with up to 300 people at a time and its works with no problems. Now obviously its not a HUGE xml doc, but a fair sized one

[CiberTHuG]

Cool, it stays on the server. This will rock much.

Now... if they abandon, then yes, I have that variable for 20 minutes.

As a matter of clean up, I could set that session variable to "" on any page that doesn't require that I keep it, or any page that indicates that the user is finished with the reports. For example, if the user is done with the report and clicks on the home link, the home asp could clear that variable. And if that variable wasn't around before the user clicked home, no harm done.

I don't imagine having a lot of users at any given time. It is for an intranet application. I do imagine having a lot of DB entries. Just yesterday I was playing with a sample that had 1500 incidents. That is about 3000 DB records that have to be pulled every time and scrubbed to get the open and close times for those 1500 incidents.

Now, that work could be done once and the 1500 incidents could be saved in the XML in memory (and not on the disk).

I feel like I am just playing with the tip of the iceberg of possiblities.

[CMangano]

Like I said, it won't be an issue if the concurrent users are low, and you clear it at first chance. But, be aware, if the user takes the steps necessary to generate this XML string, then clicks home and you empty the string, then if they click back on the browser, they are gonna have to get the xml again (meaning another call to DLL). It may be better to let them lug the thing around, and empty it only when the session ends.

Just some things to think about

[Cander]

1500 record in an XML doc? You might be getting into some trouble there as far as speed when accessing the xml. But try it out and see.

[CiberTHuG]

Okay, I have to rant for a minute...

<rant>Just as soon as I start to get a good grip on one technology, my job puts me in a completely different place. "Oh, he knows MQ/Series, he can do this." Uhm... 'this' is an MFC broker for MQ/Series. Yes, I know C++, but I don't know the MFC. As soon as I start to understand that, boom, that gets trashed and I'm learning Level 8 Geneva. Now I'm learning ASP. Hey, ASP is easy, it uses VBScript. VBScript is about as hard as shell scripts. *yawn* But now I have to write DLLs and find ways to make the DB faster, and figure that XML might be a good idea, all this time I'm playing with JavaScript and browser compatiblity. *grumblegrumble* Why can't I just master one technology and not see how many technologies I can learn just well enough to make a mess</rant>

Okay... so I left out a lot in that rant and it doesn't make any sense, but... onward.

So now I'm thinking... I can just hit the DB, build the incident list and pass that XML (and I'm sold on XML for version maintenance if nothing else, the data is self describing so I don't have to worry about the order if it was a multidemension array or a csv) to the client. If the client wants to make small changes the JavaScript can do all the work, on the client side. The server passes out the information and ignores the client. Saves work on the server, makes the process faster for the client. I think this might be a good idea, but there are compatibility issues (go figure). And we are generating some graphs to put in this report. They are generated with server side DLLs (the MSChart control generates the charts which are dumped to a bmp and converted to a jpeg). How can I make these on the client side? I do not want to have the client install any ActiveX controls. I guess I could start learning Java....

[CMangano]

As I said earlier, the only way XML will work on the client is in IE5+. Even then, there isn't any gaurantee you will get all the functionality. I think the current parser for XML is 3, and IE5 by default has version 1, so you will be missing out on whatever was used by 2 and 3.

I currently use XML on the client for our internal knowledge base (where our providers go in and add items, update items, etc.). It works great, super fast, but our company uses IE 5+. So, if this is an option for you, by all means, use XML on the client and use Javascript to parse it. You will have all kinds of neat things you can do, and it will be super fast.

Of course, if you have Netscape users...

[Cander]

You also should look into XSL. You can use it to parse the XML into HTML and works faster than the ASP engine trying to do all the work.

look for info on this here

http://www.xml101.com

[RSJIG]

Just a note about getting rid of the data.
You could create a global.asa file on the server if it is a server
that supports asp and use something like the following to initialize
the session varriable and set it to "" when the user logs off or leaves the site



<script language="VBScript" runat="Server">

Sub Session_OnStart()
Session("XMLString")="" 'Initialize the session string
End Sub

Sub Session_OnEnd()
Session("XMLString")="" ' clear the xml data from the string
End Sub


</script>

I know this is old, but I thought I'd share