Results 1 to 35 of 35

Thread: Get Information off a page

  1. #1

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Get Information off a page

    This kind of branches of my other post, but has one same thing I need to know.
    1st: I need to know how to figure out what page I am on in my browser (A browser I made from Browser control)
    2nd: How to get information off a page. Like certian information. For the page I am on, there are lists of links, and each link has its own info. Like it will be (Link)>(Item Price)>(Item Quanity). And there will be several rows of links that have that same order. I need to get the Item Price part. How do I search the page, or somehow get this information.

  2. #2
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    1. Will the LocationURL help?
    2. Will .getElementsByTagName help?
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  3. #3

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Well, nevermind the first one. I got that figured out. The second one, its just plain text. I dont think its really an element.

  4. #4
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    Perhaps you could post part of the html source where the information you need is located?
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  5. #5

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    That I dont know of. Since I can barely read Html, I cant pick out the part of the source. Also, It wont be the same source everytime. What I am getting info off of is a search engine basicly. So, the results vary, and will always be different.

  6. #6
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    You could just right click a certain webpage then there will be a View Page Source option, if you will click it then the source of the webpage you are viewing will be shown in notepad...
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  7. #7

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Yea, I know that. I just ment I cant pick out the part I need of the source. Because you wanted the part. Well, here is the full source attached. Remember this is just one result, it can vary.
    Attached Files Attached Files

  8. #8

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    So, is there another way? Looking at the source, theres not much about the table and the parts of it. Like I said its Link>Price>Quanity and I need price. I know it is possible, because I have seen other programs do it before. Made with Vb6 too. So there has to be a way.

  9. #9
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    Could you provide the link you want? Will the link be constant or will it change? You mean you will click/navigate to the link then retrieve the item price and quantity?
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  10. #10

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    I cant give the link. Its one of those things where the link never changes even if you search for something different. So say you search for something and you get your results on the page "www.site.com/results" then you do another and get your results on "www.site.com/results". So no matter what the link stays the same. My program enters the information and submits it, and gets to the results page, and now I want it to retrieve the item price thats all.

  11. #11
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    As I have examined the source you attached I've found out that the data you need are in a table cell, you could try locating them....
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  12. #12

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    oo my head hurts now lol. I dont see it. Is it labled "the table"? I have not read Html in a loong time. I cant find the table though, I found the toolbar and topbar, but not the actual table with the results. Could you excuse me this time

  13. #13
    Fanatic Member
    Join Date
    Jan 2005
    Location
    In front of this pc.
    Posts
    580

    Re: Get Information off a page

    From looking at the html source code you provided I came up with two methods which should work...Personally I'd probably go with using a RegExp but since most VB coders aren't comfortable with RegExps I'll outline the other.

    I assume you have the full source in a string variable - I'll call that htmlStr for this example.

    VB Code:
    1. Dim strParts() As String, i As Integar, j As Integar, itemPrice As String
    2. strParts = Split(htmlStr,"</b></td></tr>")
    3.  
    4. ' strParts(0) will contain all of the syle sheeting, scripts, etc,
    5. ' throught the first row of the table so just ignore it
    6. for i = 1 to Ubound(strParts)
    7.     If Instr(strParts(i),"item you are looking for goes here") then
    8.         j = InstrRev(strParts(i),">")
    9.         itemPrice = Mid$(strParts(i), j)
    10.         Exit For
    11.     End IF
    12. next i

    That should provide you with an itemPrice string like "44,500 NP". Of course, if you're wanting to capture the price for multiple items you would need to modify that code accordingly..

  14. #14
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    Is these what you are looking? I have attached the parsed code....
    Attached Images Attached Images  
    Attached Files Attached Files
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  15. #15

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Alright here is my coding:

    VB Code:
    1. Private Sub cmdPrice_Click()
    2. Load Browser
    3. Browser.Show
    4. Dim itm As ListItem
    5. Set itm = ListView1.SelectedItem
    6. Browser.WebMain.Navigate "http://www.neopets.com/market.phtml?type=wizard"
    7.   Do Until Not Browser.WebMain.Busy
    8.     DoEvents
    9.   Loop
    10. 'Alright, the above loop waits for the page to be done loading. Otherwise we can't examine the HTML
    11.   Dim doc As IHTMLDocument2
    12.   Dim element As IHTMLElement
    13.   Dim inp As IHTMLInputElement
    14.   Dim j As Integer, s As String
    15.   Set doc = Browser.WebMain.Document 'Making the doc variable reference the web browser document; in other words, when doc is used, read it as web.Document.
    16.   For j = 0 To doc.All.length - 1
    17. 'we'll loop through all tags in the HTML to see if it is one of the input boxes.
    18.      Set element = doc.All.Item(j)
    19.     If element.tagName = "INPUT" Then
    20. 'if the tag of the selected element is INPUT, then it may be a textbox.
    21.         Set inp = element
    22.         Select Case inp.Name
    23.         Case "shopwizard": inp.Value = itm.Text
    24.         Case "min_price": inp.Value = "0"
    25.         Case "max_price": inp.Value = ""
    26.         End Select
    27. 'if the name of the element (the 'name' attribute) is 'shopwizard', it's the textbox we're looking for.
    28.     End If
    29.   Next j
    30.   For j = 0 To doc.All.length - 1 ' This submits our data and brings us to the results page.
    31.      Set element = doc.All.Item(j)
    32.      If element.tagName = "INPUT" Then
    33.          Set inp = element
    34.          If inp.Name = "" Then
    35.            inp.Click
    36.          End If
    37.      End If
    38.      Next j
    39.      HtmlStr = "C:\Source.txt"
    40.   Dim strParts() As String, i As Integer, j1 As Integer, itemPrice As String
    41. strParts = Split(HtmlStr, "</b></td></tr>")
    42. For i = 1 To UBound(strParts)
    43.     If InStr(strParts(i), itm.Text) Then
    44.         j1 = InStrRev(strParts(i), ">")
    45.         itemPrice = Mid$(strParts(i), j1)
    46.         Exit For
    47.     End If
    48. Next i
    49. MsgBox itemPrice
    50.  
    51. End Sub

    That should search for the item the user specified, then got the price of the item. Its no working. Also, I wanted the price of the first item on the list. Only the first one and no others. I dont think that coding is doing that. Can I do that?

  16. #16

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Ok, I see what you were trying to do. See, I dont need to search for the item. The item is same, its just different shops that have that item. It goes by cheapest from first to last. I need the cheapest price, so I need the first items price. Searching would not do anything, that explains it I think.

  17. #17
    Admodistrator |2eM!x's Avatar
    Join Date
    Jan 2005
    Posts
    3,900

    Re: Get Information off a page

    couldnt you use the same code i gave you yesterday?

  18. #18

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Which would be? Sorry, I get alot of coding, hard to keep track of what I get. I thought yours didn't work or something. Could you repost it?

  19. #19
    Fanatic Member
    Join Date
    Jan 2005
    Location
    In front of this pc.
    Posts
    580

    Re: Get Information off a page

    Assuming all of the code prior to what I just gave you works as desired, change your code to this and see if that does the trick.

    VB Code:
    1. Private Sub cmdPrice_Click()
    2. Load Browser
    3. Browser.Show
    4. Dim itm As ListItem
    5. Set itm = ListView1.SelectedItem
    6. Browser.WebMain.Navigate "http://www.neopets.com/market.phtml?type=wizard"
    7.   Do Until Not Browser.WebMain.Busy
    8.     DoEvents
    9.   Loop
    10. 'Alright, the above loop waits for the page to be done loading. Otherwise we can't examine the HTML
    11.   Dim doc As IHTMLDocument2
    12.   Dim element As IHTMLElement
    13.   Dim inp As IHTMLInputElement
    14.   Dim j As Integer, s As String
    15.   Set doc = Browser.WebMain.Document 'Making the doc variable reference the web browser document; in other words, when doc is used, read it as web.Document.
    16.   For j = 0 To doc.All.length - 1
    17. 'we'll loop through all tags in the HTML to see if it is one of the input boxes.
    18.      Set element = doc.All.Item(j)
    19.     If element.tagName = "INPUT" Then
    20. 'if the tag of the selected element is INPUT, then it may be a textbox.
    21.         Set inp = element
    22.         Select Case inp.Name
    23.         Case "shopwizard": inp.Value = itm.Text
    24.         Case "min_price": inp.Value = "0"
    25.         Case "max_price": inp.Value = ""
    26.         End Select
    27. 'if the name of the element (the 'name' attribute) is 'shopwizard', it's the textbox we're looking for.
    28.     End If
    29.   Next j
    30.   For j = 0 To doc.All.length - 1 ' This submits our data and brings us to the results page.
    31.      Set element = doc.All.Item(j)
    32.      If element.tagName = "INPUT" Then
    33.          Set inp = element
    34.          If inp.Name = "" Then
    35.            inp.Click
    36.          End If
    37.      End If
    38.      Next j
    39.  
    40.   Do Until Not Browser.WebMain.Busy
    41.     DoEvents
    42.   Loop
    43.   doc = Browser.WebMain.Document
    44.   Dim strParts() As String, itemPrice As String, i as Integer
    45.   strParts = Split(doc, "</b></td></tr>")
    46.   i = InStrRev(strParts(1), ">")
    47.   itemPrice = Mid$(strParts(1), i)
    48.  
    49.   MsgBox itemPrice
    50.  
    51. End Sub

  20. #20

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Im getting an error with:
    "doc = Browser.WebMain.Document"
    Saying the object does not support that method. So thats not a browser control. Mistake?

  21. #21
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    Spending some minutes locating the data you need I came up with this, take a look if its correct!

    VB Code:
    1. Dim oDoc As HTMLDocument
    2. Dim oCell As HTMLTableCell
    3. Set oDoc = WebMain.Document
    4. Set oCell = oDoc.getElementsByTagName("td")(20)
    5. MsgBox oCell.innerText
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  22. #22

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    dee-u, the messagebox gave me "NP ". Im assuming there was supposed to be an amount in there. The first item in the list was 100,000. So shouldn't it have been "NP 100,000"?

  23. #23
    Admodistrator |2eM!x's Avatar
    Join Date
    Jan 2005
    Posts
    3,900

    Re: Get Information off a page

    i actually have no idea where the code is. It found all the links you were looking for on a page, even though they change.

  24. #24

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Well, in the thread where I asked how to get the the first link or certian link you said you could not help me.

    Found the thread:
    http://www.vbforums.com/showthread.php?t=344522

  25. #25
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    Fiddle with it...

    VB Code:
    1. Set oCell = oDoc.getElementsByTagName("td")(19)
    2. MsgBox oCell.innerText
    3. MsgBox oCell.innerHTML

    VB Code:
    1. Set oCell = oDoc.getElementsByTagName("td")(21)
    2. MsgBox oCell.innerText
    3. MsgBox oCell.innerHTML

    I noticed, you are using interface (IHTMLDocument2), try using HTMLDocument, also in your other declarations...
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  26. #26
    Admodistrator |2eM!x's Avatar
    Join Date
    Jan 2005
    Posts
    3,900

    Re: Get Information off a page

    what do you mean i couldnt help you??? did you even try the code i posted???

  27. #27

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Yes, I did, it wont work because I cant specify the url. Remember how I said all that stuff about the link being the same no matter what, so I cant simple go like

    x = Inet1.OpenURL("http://www.neopets.com/market.phtml", icString)

    Thats why it didn't work. Then after that you said you couldn't help me with it anymore.

  28. #28

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Ok, dee-u I kinda got your code working, except I cant use it for two reasons. It is definantly not reliable, as for some reason the vaule will change in odd patterns like sometimes it will be correct and be like "200 NP", other times it will be like "NP ", and "Optional". All those weird results.
    Second reason, I only need numbers, not numbers then NP like "320 NP". So unless I can make it fit around these two standards, I need another way.

  29. #29
    Software Carpenter dee-u's Avatar
    Join Date
    Feb 2005
    Location
    Pinas
    Posts
    11,127

    Re: Get Information off a page

    What you may do is to list all tables and determine if it has Shop Owner, Item, Amount in Stock and Price table cells and go from there... Sorry I have got no time to code it for you....

    To parse the Price only you could use Replace to replace NP with empty string....
    Regards,


    As a gesture of gratitude please consider rating helpful posts. c",)

    Some stuffs: Mouse Hotkey | Compress file using SQL Server! | WPF - Rounded Combobox | WPF - Notify Icon and Balloon | NetVerser - a WPF chatting system

  30. #30

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Yea, I gotts go as well, does someone else have the time to code what dee-u is talking about for me?

  31. #31

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Alright, so how do I replace the "NP" in it? And, could you explain a litte more, Im not sure I know how to do what your talking about in the first part.

  32. #32
    Admodistrator |2eM!x's Avatar
    Join Date
    Jan 2005
    Posts
    3,900

    Re: Get Information off a page

    x = replace(string with NP in it here,"NP",vbnullstring)

  33. #33

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    Dim x As Variant
    x = Replace(Cprice, "NP", vbNullString)

    Not working, still got "NP " In the messagebox.

  34. #34
    Admodistrator |2eM!x's Avatar
    Join Date
    Jan 2005
    Posts
    3,900

    Re: Get Information off a page

    Cprice = Replace(Cprice, "NP", vbNullString)

  35. #35

    Thread Starter
    Frenzied Member Inuyasha1782's Avatar
    Join Date
    May 2005
    Location
    California, USA
    Posts
    1,035

    Re: Get Information off a page

    omg, duh forgot about that part. Anyways, |2eM!X did you have a way? Because I am getting some serious problems with dee-u's. Now I am getting blank every time in the message box. I might have messed up the coding but here is what I have now:

    VB Code:
    1. Private Sub cmdPrice_Click()
    2. Dim Loc As String
    3. Loc = InputBox("What position is the item? (Starting from the first item as 1)")
    4. Load Browser
    5. Browser.Show
    6. Dim itm As ListItem
    7. Set itm = ListView1.SelectedItem
    8. Browser.WebMain.Navigate "http://www.neopets.com/market.phtml?type=wizard"
    9.   Do Until Not Browser.WebMain.Busy
    10.     DoEvents
    11.   Loop
    12. 'Alright, the above loop waits for the page to be done loading. Otherwise we can't examine the HTML
    13.   Dim doc As IHTMLDocument2
    14.   Dim element As IHTMLElement
    15.   Dim inp As IHTMLInputElement
    16.   Dim j As Integer, s As String
    17.   Set doc = Browser.WebMain.Document 'Making the doc variable reference the web browser document; in other words, when doc is used, read it as web.Document.
    18.   For j = 0 To doc.All.length - 1
    19. 'we'll loop through all tags in the HTML to see if it is one of the input boxes.
    20.      Set element = doc.All.Item(j)
    21.     If element.tagName = "INPUT" Then
    22. 'if the tag of the selected element is INPUT, then it may be a textbox.
    23.         Set inp = element
    24.         Select Case inp.Name
    25.         Case "shopwizard": inp.Value = itm.Text
    26.         Case "min_price": inp.Value = "0"
    27.         Case "max_price": inp.Value = ""
    28.         End Select
    29. 'if the name of the element (the 'name' attribute) is 'shopwizard', it's the textbox we're looking for.
    30.     End If
    31.   Next j
    32.   For j = 0 To doc.All.length - 1
    33.      Set element = doc.All.Item(j)
    34.      If element.tagName = "INPUT" Then
    35.          Set inp = element
    36.          If inp.Name = "" Then
    37.            inp.Click
    38.          End If
    39.      End If
    40.   Next j
    41.   Do Until Browser.WebMain.Busy
    42.   DoEvents
    43.   Loop
    44.   HtmlStr = "C:\Source.txt"
    45.   Dim oDoc As HTMLDocument
    46. Dim oCell As HTMLTableCell
    47. Set oDoc = Browser.WebMain.Document
    48. Set oCell = oDoc.getElementsByTagName("td")(20)
    49. Dim Cprice As String
    50. Cprice = oCell.innerText
    51. Cprice = Replace(Cprice, "NP", vbNullString)
    52. Browser.WebMain.Navigate "http://www.neopets.com/market.phtml?type=your"
    53. Do Until Not Browser.WebMain.Busy
    54.     DoEvents
    55.   Loop
    56. Set doc = Browser.WebMain.Document
    57. For j = 0 To doc.All.length - 1
    58. 'we'll loop through all tags in the HTML to see if it is one of the input boxes.
    59.      Set element = doc.All.Item(j)
    60.     If element.tagName = "INPUT" Then
    61. 'if the tag of the selected element is INPUT, then it may be a textbox.
    62.         Set inp = element
    63.         If inp.Name = "cost_" & Loc Then
    64.         inp.Value = Cprice
    65. 'if the name of the element (the 'name' attribute) is 'shopwizard', it's the textbox we're looking for.
    66.     End If
    67. End If
    68.   Next j
    69.   MsgBox Cprice
    70. End Sub

    That should search find the price, and price the item on the pricing page. Except, now I am getting nothing for the item price, on rare occasions ill get a number, but it has not been doing it lately. Which is not good, because I plan on releasing this to people. So is there something wrong? Or another way?

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width