ASP Remote file size

[Max_aka_NOBODY]

I am trying to get the filesize of a remote file, while acquiring as little as possible data, and failing miserably. The best I can think of is sending a HEAD request(using XMLHTTP), then getting the Content-Length header, but the site I'm contacting does not send Content-Length at all, so this fails.

Can anyone help me with some other way to acomplish this?

Thanks in advance for any help you might offer.

[kazar]

does the site in question belong to you?

[Max_aka_NOBODY]

Neither to me nor to my client.

[kazar]

What method are you using to send the request?

[Max_aka_NOBODY]

I've tried GET, HEAD and OPTIONS(just in case). None return Content-Length.

EDIT: To clarify, it's a text/html page, not a binary file.

[kazar]

it could be the version of IIS it's operating, or it's a weird server

[rory]

PM me the link and ill check its headers ..

[kazar]

I think i have a way using API, i'm just finalising the code

[Max_aka_NOBODY]

Here's an example from their site(automatically generated rss/xml): http://www.buyaphoto.net/rss/rss_gen3.php?&parent=2. They're using Apache BTW.

[kazar]

Right i've cracked it

Tested it on a couple of pages, it works fine.

Option Explicit
 
Const scUserAgent = "IE 6.0"
Const INTERNET_OPEN_TYPE_DIRECT = 1
Const INTERNET_OPEN_TYPE_PROXY = 3
Const INTERNET_FLAG_RELOAD = &H80000000
 
Private Declare Function InternetOpen Lib "wininet" Alias "InternetOpenA" (ByVal sAgent As String, ByVal lAccessType As Long, ByVal sProxyName As String, ByVal sProxyBypass As String, ByVal lFlags As Long) As Long
Private Declare Function InternetCloseHandle Lib "wininet" (ByVal hInet As Long) As Integer
Private Declare Function InternetReadFile Lib "wininet" (ByVal hFile As Long, ByVal sBuffer As String, ByVal lNumBytesToRead As Long, lNumberOfBytesRead As Long) As Integer
Private Declare Function InternetOpenUrl Lib "wininet" Alias "InternetOpenUrlA" (ByVal hInternetSession As Long, ByVal lpszUrl As String, ByVal lpszHeaders As String, ByVal dwHeadersLength As Long, ByVal dwFlags As Long, ByVal dwContext As Long) As Long
 
Function GetFileLength(sUrl As String)
  
 
    Dim hOpen As Long
    Dim hFile As Long
    Dim sBuffer As String
    Dim Ret As Long
    Dim oldret As Long
    Dim oldbuffer As String
    Dim bytesreadint As Long
    'Create a buffer for the file we're going to download
    sBuffer = Space(1000)
    'Create an internet connection
    hOpen = InternetOpen(scUserAgent, INTERNET_OPEN_TYPE_DIRECT, vbNullString, vbNullString, 0)
    'Open the url
    hFile = InternetOpenUrl(hOpen, sUrl, vbNullString, ByVal 0&, INTERNET_FLAG_RELOAD, ByVal 0&)
    'Read a1000 bytes of the file
    oldbuffer = ""
 
    Do
    oldbuffer = sBuffer
       InternetReadFile hFile, sBuffer, 1000, Ret
    
    If Ret <> 1000 Then  'ret will not return 1000 if it cant i.e if there are less bytes than that left, instead, it will return the amount left.
      bytesreadint = bytesreadint + Ret
      Exit Do
    Else
    bytesreadint = bytesreadint + 1000
    End If
    
    
    Loop
      
    
    'clean up
    InternetCloseHandle hFile
    InternetCloseHandle hOpen
    'Show our file
    
    GetFileLength = bytesreadint
End Function

I've checked it against the inet.getheaders one, and the result is the same.

Rate me if it works, Thanks

[Max_aka_NOBODY]

Ah, sorry for making you work for no reason. I can read the file and get its length fine. The goal is getting the length without actually getting the file.

[rory]

Not all servers will supply a content length in the headers ...

This gets the Headers ..
objHTTP.getAllResponseHeaders

This only gets it if the header type exists ..
objHTTP.getResponseHeader("Content-Length")

otherwise you need to read the file then get the length ..

[rory]

<%@ Language="VBScript" %> 
<%
Option Explicit

Response.Buffer = True
Response.Expires = 0
Response.ExpiresAbsolute = Now() - 1
Response.AddHeader "cache-control","private"
Response.AddHeader "pragma", "no-cache"

Dim objHttp
Dim strURL
Dim strStatus
Dim strResults
Dim strLength 

strURL = "http://www.somelink.com"

'// GET A REMOTE WEB PAGE
Function RemoteData()
	Dim strData
	On Error Resume Next
	Set objHttp = Server.CreateObject("Microsoft.XMLHTTP")
	objHttp.Open "GET", strURL, False
	objHttp.setRequestHeader "Content-Type", "text/html"
	If Err = 0 Then
		objHttp.send
		strStatus = objHttp.Status
		strResults = objHttp.StatusText
		If objHTTP.getResponseHeader ("Content-Length") = "" Then
			Do Until objHTTP.readyState = 4: DoEvents: Loop
			strData = objHttp.ResponseText
			strData = Len(strData)
		Else
			strData = objHTTP.getResponseHeader("Content-Length")
		End If
	End If
	Set objHttp = Nothing
	If Err <> 0 Then
		strData = "Error " & Err.Number & "<br>" & _
		Err.Description & vbCrLf
	End If	
	RemoteData = strData
End Function

'// GET PAGE NOW
strLength = RemoteData

'// RESPONSE ERROR
If strStatus >= 400 And strStatus <= 500 Then
	Response.Write "Error " & strStatus & "<br>"
	Response.Write strResults & vbCrLf
	Response.End	
'// SHOW DATA FROM PAGE
Else
	Response.Status = "200 Ok"
	Response.Write "Content Length: " & strLength
End If
%>

[Max_aka_NOBODY]

Well, that still doesn't do anything different from what I was doing. That site isn't sending Content-Length, so the function would get teh data every time which, unfortunately, isn't acceptable in my case.

[rory]

If it doesnt send you the content length .. even if using Winsock .. there is no way to get it .. other than grabbing the resulting HTML/XML and getting the length of that ..

unless you run a remote ASP page on that server to get the File size through FSO ... if its not your server though, then you cant do that ..

Rory

PS. I did add in Error Checking and Status Reponses .. LOL .. ;-)

[rory]

also, you are using MSXML anyway, you have to connect to it .. so why not just use the text it sends back to get the content length ..?? You have to connect to the webpage no matter what ... and with that comes a reply ..

[Max_aka_NOBODY]

Well, I was posting in hopes of there being a way to force the response to contain a Content-Length or somesuch in a HEAD request. Some of the pages I'm getting are quite big, up to half an MB, and I would like to cache them, which is why I need the filesize. And before you ask, If-Modified-Since doesn't work either.

[rory]

i tried it even with Winsock in VB and still no go .. most web pages just aren't sending a content length :-(

Rory

[Max_aka_NOBODY]

Oh well, thanks for trying. I'll see if I can find some workaround.

[rory]

Well, I was posting in hopes of there being a way to force the response to contain a Content-Length or somesuch in a HEAD request. Some of the pages I'm getting are quite big, up to half an MB, and I would like to cache them, which is why I need the filesize. And before you ask, If-Modified-Since doesn't work either.

.. ask them to use a database ..?

[Max_aka_NOBODY]

.. ask them to use a database ..?

"Outside my competence."