My First APP (PageScanner) - Where to Begin?

**kaanuki** · Jun 8th, 2004, 07:45 AM

Hi.

Im a long-time web builder who has wanted to begin to learn VB, but hasjust decided to begin programming because I now have a reason. And would like just a wee bit of advice, so that I can at least start in the right direction.

I would like to begin my programming journey by building a small app that grabs my game clubs scores and stats from off of a website.

To be more clear...

I am part of an online game club who plays online games on Yahoo'c gaming site. We can click on a link that takes us to a page that shows the detailed scores and stats of games we have played. We then manually enter these scores and stats back into our gaming club's database.

Now, I would like to build a small app that would:
1. Go to my "YAHOO GAMES SCORES & STATS" page.
2. Copy the Scores and stats of (specified) games
3. Paste the scores and stats into a file (text, Excel, etc...), OR EMAIL them in a formatted way.

My question is - where should I begin, what should I read? I Own VB 6 and VB.Net.

Thank you.

-Kaan-

**BukHix** · Jun 8th, 2004, 09:50 AM

Well this is a ASP.Net solution but I think you could make it work for what you are trying to do. All I am doing is Screen Scraping from the Yahoo site. Keep in mind two things. 1) I am not sure whether Screen Scraping is legal or not without permission and 2) It probably will not work on a site where authentications is needed.

First check out this site from Yahoo.
http://classifieds.yahoo.com/

Lets say that you just wanted to grab the section of the page that shows the catagories only. You could use the code below to do so:

VB Code:

<%@ Page Language="VB" %>
<%@ import Namespace="System.IO" %>
<%@ import Namespace="System.Net" %>
<script runat="server">
 
    Sub Page_Load
        Dim strmReader as StreamReader = Nothing
        Dim StrURL as String = "http://classifieds.yahoo.com/"
        Dim objRequest as WebRequest = WebRequest.Create(strUrl)
        Dim objResponse as WebResponse = objRequest.GetResponse()
 
        strmReader = New StreamReader(objResponse.GetResponseStream())
 
        Dim strContent as String = strmreader.ReadToEnd()
        Dim RegEx as Regex = _
            New Regex _
            ("<!-- BODY:END:MAIN:TOP -->((.|\n)*?)<!-- - end FOOT ad call --->", _
            RegexOptions.IgnoreCase)
 
        Dim getMatch as Match = RegEx.Match(strContent)
 
        lblbody.Text = (getMatch.Value)
 
    End Sub
 
</script>
<html>
<head>
</head>
<body>
    <asp:Label id="lblBody" runat="server"></asp:Label>
</body>
</html>

If you cut and paste this code as is and then browse to it you will get a screen like this (attached).

In the sample above everything is being sent to a lable but there is nothing that would stop you from putting the information into a email or a file. By the way you set the content you want to scrape with the RexEx. In the sample above it starts with  and ends with . If you look at the source of the original page you will see those comments in the html.

**kaanuki** · Jun 9th, 2004, 07:05 AM

Thanks, Buk! This looks very promising!

This may be a dumb question, but just so I understand -

The page that you're showing me in the jpg is is beeing stored in the "LABEL" of the ASP code? What's the purpose of the "label"?

How would the info be extracted to a file?

Thanks.

-Kaan-

**BukHix** · Jun 9th, 2004, 08:03 AM

Well the label was just to show you that you could take partial content of a web page store it in a string and then pass it to the browser through the label. You could just as easily send the string in the body of an email or write it to a file.

As far as writing it to a file you are going to have to get that info from someone else because I have never done so. Although I don't think it should be all that hard.

Thread: My First APP (PageScanner) - Where to Begin?

Thread Tools

Display

My First APP (PageScanner) - Where to Begin?

Posting Permissions