Vb.net scrape html table....question...

**Teachme1** · Dec 24th, 2016, 05:10 PM

Say I have a table that will always contain RANDOM DATA (various product titles, prices, & ratings in no particular order). I noticed that sometimes either the "Price:" column or "Rating" column won't always have a value. So when I'm scraping multiple items into an array & sending each column into a listview, the data won't sync up properly if a value is missing in say the "Price" column.

Here is an example of a html table that I'm trying to scrape data from, but notice how row "# 5" is missing the price. This is what's messing up the syncing of the data while it's being added to the listview in VB.NET:

HTML Code:

<html>
<head>
<style>

table {
margin:auto;
margin-top:50px;
font-family: arial, sans-serif;
border-collapse: collapse;
width: 40%;
}

td{
border: 3px solid #000;
text-align: left;
padding: 3px;
}

th {
border: 3px solid #000;
background-color:gold;
text-align: left;
padding: 3px;
}

tr:nth-child(even) {
background-color: #dddddd;
}
</style>
</head>
<body>
    <table>
        <tr><th>#</th><th>Product Title:</th><th width="60">Price:</th><th width="60">Rating:</th></tr>
        <tr><td width="20">1</td><td>Minera Natural Dead Sea Salt, 5lbs Bulk Bag - Fine Grain</td><td>$20.00</td><td>9/10</td></tr>
        <tr><td width="20">2</td><td>Minera Dead Sea Salt 2lb Bag Fine Grain, 100% Pure Mineral Salt Treatment</td><td>$9.99</td><td>6/10</td></tr>
        <tr><td width="20">3</td><td>Minera Pure Dead Sea Salt 10lbs Fine Grain</td><td>$15.95</td><td>8/10</td></tr>
        <tr><td width="20">4</td><td>Dead Sea Warehouse - Amazing Minerals Dead Sea Bath Salts, Temporary Relief from...</td><td>$16.00</td><td>5/10</td></tr>
        <tr><td width="20">5</td><td>Natural Planet Dead Sea Salt, 5lbs Fine Grain - 100% Pure Bath Salt - For Psoriasis...</td><td></td><td>5/10</td></tr>
        <tr><td width="20">6</td><td>Art Naturals Himalayan Salt Body Scrub 20oz -Deep Cleansing Exfoliator With Shea...</td><td>$13.95</td><td>7/10</td></tr>
        <tr><td width="20">7</td><td>Dead Sea Salt 2.2lb try for Psoriasis, Eczema, and Dermatitis (1 x Resealable...</td><td>$9.99</td><td>4/10</td></tr>
        <tr><td width="20">8</td><td>Premier Dead Sea Aromatherapy Mineral Body Treatment, Silver, Salt Scrub, 425...</td><td>$15.95</td><td>8/10</td></tr>
        <tr><td width="20">9</td><td>Dead Sea Warehouse - Amazing Minerals Dead Sea Bath Salts, Temporary Relief from...</td><td>$16.00</td><td>6/10</td></tr>
        <tr><td width="20">10</td><td>Natural Planet Dead Sea Salt, 50lbs Fine Grain - 100% Pure Bath Salt - For Psoriasis...</td><td>$90.25</td><td>10/10</td></tr>

    </table>
</body>
</html>

Now here is an example of what I'm using in VB.NET to collect data from this table:

Code:

Imports System.Text.RegularExpressions
Public Class Form1
    Dim ITEM As New ListViewItem
    Dim ProductTitle As String
    Dim ProductPrice As String

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        ListView1.Items.Clear()
        ProductTitle = ""
        ProductPrice = ""

        Dim keyword As String = TextBox1.Text
        keyword = keyword.Replace(" ", "+")
        Try
    'This is the HTML Table That I'm talking about:
            Dim html As String = "THE HTML TABLE SPECIFIED"
  
            'Product Title:
            '<h5 data-attribute="".+?""
            Dim regx1 As New Regex("<h5 data-attribute="".+?""", RegexOptions.IgnoreCase)
            Dim matches1 As MatchCollection = regx1.Matches(html)
            For Each match1 As Match In matches1
                ProductTitle += match1.Value & "^"
                ProductTitle = ProductTitle.Replace("<h5 data-attribute=""", "").Replace("""", "")
            Next

            'Price:
            Dim regx As New Regex("<span class=""a-size-small a-color-price a-text-bold"">.+?</span>", RegexOptions.IgnoreCase)
            Dim matches As MatchCollection = regx.Matches(html)
            For Each match As Match In matches
                ProductPrice += match.Value & "^"
                ProductPrice = ProductPrice.Replace("<span class=""a-size-small a-color-price a-text-bold"">", "").Replace("</span>", "")
            Next

            'Create the split & add all items to listview:
            Dim split1() As String = ProductTitle.Split("^")
            Dim split2() As String = ProductPrice.Split("^")
            For i = 0 To split1.Count - 2
                ITEM = ListView1.Items.Add(split1(i))
                ITEM.SubItems.Add(split2(i))
            Next


            Label1.Text = "Product Title: " & ListView1.Items.Count
            Label2.Text = "Price: " & ListView1.Items.Count

        Catch ex As Exception

        End Try
    End Sub
End Class

Again, the problem is that sometimes I won't know which table is going to have some elements missing (such as the "Price" column) which causes the data NOT to be synced up in the rows of the ListView. How could I fix this with the code that I've written above? Thanks.

**.paul.** · Dec 25th, 2016, 11:04 AM

You can grab the <td>something</td>

New Regex("/<td/>(.+)/<//td/>")

Match.Groups(1) is the 'something'

Thread: Vb.net scrape html table....question...

Thread Tools

Display

Vb.net scrape html table....question...

Re: Vb.net scrape html table....question...

Tags for this Thread

Posting Permissions