Re: HTML download problem
Try
Debug.Print Data
then look in your immediate (debug) window... see if its all there.
Re: HTML download problem
Ok, tried that and all I got was this...
Code:
<span class="class1">
<p> <a href="http://www.5stardog.com/">5 Star Dog</a>
Online resource for dog owners that includes information on pet selection, training, breeders, and food along with breed descriptions, pictures, forums, and links.<br>
<a href="http://www.purina.com/intl/default.asp">Nestlé Purina PetCare Company</a>
International manufacturer and distributor of food, treats and litter for dogs and cats. Portal to specific national sites.<br>
<a href="http://www.amazon.com/exec/obidos/tg/detail/-/0439110165?v=glance">Amazon.com: Dog Food: Books: Joost Elffers,Saxton Freymann</a>
... Dog Food (Hardcover) by Joost Elffers, Saxton Freymann "Good dog ... a list of ingredients included on the back cover, Dog Food is a captivating and humorous feast for the eyes ...<br>
<a href="http://www.therustydog.com/">The Rusty Dog</a>
Carries natural foods as well as pet furniture and gifts.<br>
<a href="http://www.vegetariandogs.com/">Vegetarian Dog Food</a>
This beautiful and easy to use site is dedicated to vegetarian dog information. Find out where to purchase commercial vegetarian dog food or how you can learn to make your own. ... If you would rather purchase a commerc
Re: HTML download problem
inet and webbrowser are very similar, which is why I am posting this. When I downloaded websites with webbrowser and grab the source HTML from it, I would find that the original source would be missing certain elements that should be there. Now someone ELSE will have to help you here, but try setting HEADER data to be sent when you request the URL. Header data would include shizzle like the browser you're using, the system you're on and all-sorts like that (even cookie data if you were using a site that needed it)...without the browser information, some sites would return partial data...hope that helps :-)
Edit: In fact, I had one site that REFUSED flat out to let me send get data (form stuff) without putting valid header data in to tell it referer and all that :-)
Re: HTML download problem
Debug.Print sometimes doesn't print all data. If you want to see all the data in the string, print it to a file & then open the file manually (use 'Open "C:\debug.txt" For Output As #1...)
Re: HTML download problem
A bit later in my code it updates a mySQL database anyway so I can check. Is webbrowser better to use than inet?
Re: HTML download problem
Quote:
Originally Posted by shirazamod
Debug.Print sometimes doesn't print all data. If you want to see all the data in the string, print it to a file & then open the file manually (use 'Open "C:\debug.txt" For Output As #1...)
Debug.print doesnt print all the data??
Re: HTML download problem
Quote:
Originally Posted by Jon12345
Is webbrowser better to use than inet?
The problem with webbrowser is that (unless you write in code to stop it) it loads everything on a page...images...background music (if you have it)...everything...Inet just grabs the HTML and that's it, and Inet is a hell of a lot faster in that respect because of it. However, it's a lot more work to get it working if you need specialised stuff like cookies or header info :-)
Re: HTML download problem
whats the real URL???.. using Webbrowser & HTML object you can pull any part of the page...
Re: HTML download problem
VB Code:
Dim Data As String
Data = Inet1.OpenURL("http://www.vbforums.com")
While Inet1.StillExecuting = True
DoEvents
Wend
MsgBox Data
Debug.Print Data
i used vbforums as example (couldn't reach your given page). When the messagebox appears, a big part of the text is missing. if you look at debug now, it has the full code.
Re: HTML download problem
Since there seems to be a bug in the inet component, I would like to try webbrowser. So I have the following code:
VB Code:
Dim hDoc
Dim strKeywords as String
Dim HTMLSource As String
strKeywords = "dog%20food"
WebBrowser1.Navigate ("http://mysite.com/cgi-bin/script/script.cgi/?skin=jonclean1&length=10&keywords=" & strKeywords & "'")
hDoc = WebBrowser1.Document
HTMLSource = hDoc.body.innerhtml
But when I run this I get this error:
"Object variable or With block variable not set"
What is wrong with the code?
Re: HTML download problem
VB Code:
Private Sub Form_Load()
strKeywords = "dog%20food"
WebBrowser1.Navigate "http://mysite.com/cgi-bin/script/script.cgi/?skin=jonclean1&length=10&keywords=" & strKeywords & "'"
End Sub
Private Sub WebBrowser1_DocumentComplete(ByVal pDisp As Object, URL As Variant)
If (pDisp Is WebBrowser1.Application) Then
Dim HTMLSource As String
HTMLSource = WebBrowser.Document.documentElement.innerHTML
End If
End Sub
Re: HTML download problem
Static, I am a bit confused on how I would use your code. Currently, my code runs if someone clicks on a command button. It then cycles through a list of urls.
Re: HTML download problem
add the webbrowser control
then paste in the document complete code.
put the navigate to behind your button click
I cant test it because that url is no good
Re: HTML download problem
To make things simpler, I am just using www.yahoo.com
So, my code is as follows:
VB Code:
Private Sub Command1_Click()
strKeywords = "dog%20food"
WebBrowser1.Navigate "www.yahoo.com"
End Sub
Private Sub WebBrowser1_DocumentComplete(ByVal pDisp As Object, URL As Variant)
If (pDisp Is WebBrowser1.Application) Then
Dim HTMLSource As String
HTMLSource = WebBrowser.Document.documentElement.innerHTML
End If
End Sub
When I click the button, it loads Yahoo into the WebBrowser control and then I get the error Object required. Run-time error '424'. The following line of code is then highlighted:
VB Code:
HTMLSource = WebBrowser.Document.documentElement.innerHTML
Re: HTML download problem
oops WebBrowser
WebBrowser1http://www.vbforums.com/
Re: HTML download problem
No error message now. But I can't see how the WebBrowser1_DocumentComplete subroutine is called. From what I can see, you click the button and it just runs the two lines of code.
What am I missing here?
Re: HTML download problem
When the website has been fully loaded (including images etc) it will call the DocumentComplete event
Re: HTML download problem
you are also not displaying it anywhere ;)
add
Re: HTML download problem
What I am getting confused about is that I want to run a loop with changing urls. This will happen when I click the command button. But it seems that "Private Sub Command1_Click()" subroutine loses control and another subroutine is triggered.
Therefore, I cannot loop through the code for changing urls, correct?
Re: HTML download problem
Yes.. I would do it like this
(assuming u have URLs listed in Listbox (List1))
VB Code:
Dim lIndex As Integer
Private Sub Command1_Click()
lIndex = 0
LoopURLS
End Sub
Private Sub WebBrowser1_DocumentComplete(ByVal pDisp As Object, URL As Variant)
If (pDisp Is WebBrowser1.Application) Then
Debug.Print WebBrowser.Document.documentElement.innerHTML
LoopURLS
End If
End Sub
Private Sub LoopURLS()
Dim tmp As String
If lIndex >= List1.ListCount Then Exit Sub
tmp = List1.List(lIndex)
WebBrowser1.Navigate tmp
lIndex = lIndex + 1
End Sub
Re: HTML download problem
Thanks Static! I will give that a whirl.