Check if website exists

[StrangerInBeijing]

Hi,
I want to create a form, that run throught all url's in my "members" table, and check if the website actually exists.
Reason is that the data capturer's here did not check this when they captured.
My capturing application check if it's a valid url (regex) but that's about it.

I remember when i was working at microsoft (tester) we had this component used to open a url in a browser, and read elements, but forgot what was that.

Anyone know how to check this from a c# desktop app?

[Jumpercables]

You could write a class to ping the website. Just you do in the windows command prompt.

[StrangerInBeijing]

Just got this and tested it, and works fine it's seems:

public static bool UrlIsValid(string smtpHost)
{
	bool br = false;
	try 
	{
		System.Net.IPHostEntry ipHost = System.Net.Dns.Resolve(smtpHost);
		br = true;
	}
	catch 
	{
		br = false;
	}
	return br;
}

Pinging would not been a good idea. It's possible for a host to disable that (not sure what it's called), but experienced that before.

Thanks anyhow for bothering to reply

[Harsh Gupta]

Hey, I used your code, it worked fine for yahoo.com but not working for any other site, microsoft.com, vbforums.com, msn.com, none for!!

Is it working for you for all websites??

[StrangerInBeijing]

Strange, because it's working fine for all the sites you mentioned. I tested it also with sites I know does not exists, like www.dasfljdaskfsadfjsad.com, and then it return false.

Noticed that if you add the "http://" then it always return false, but with or without www it's fine.

Not so sure about this....the reason why I hate having so little time, and end up just re-using code, instead of learning how it works. Gawd, I need a new job!

[RudiVisser]

Noticed that if you add the "http://" then it always return false, but with or without www it's fine.

I'm pretty sure that's because the protocol has nothing to do with Resolving DNS records.

For example:
http://www.example.com/
ftp://www.example.com/

Are the same server (www.example.com).



If you're taking user data, it might be an idea to strip any protocols off the front, and even / at end, possibly even index.php (for example), you know never to trust users with what they enter.

[penagate]

Rudi is correct. The protocol is not part of the DNS record. Neither is the trailing slash, although that doesn't matter so much.

Note that www.example.com and example.com are separate records.

[Harsh Gupta]

Ok, today it's quite working.

Rudi and Penagate stated correctly, http:// has nothing to do with DNS. In fact, if I add http:// to my example, it would return exception "No such host is known".

But adding/subtracting www doesn't make any difference.

This is a simple thing I tried:

try
        {
            System.Net.IPHostEntry host;

            host = System.Net.Dns.GetHostEntry("del.icio.us");

            if (host != null)
                Console.WriteLine("It exists");
        }
        catch (Exception ex)
        {
            Console.WriteLine(ex.Message);
        }

Note that www.example.com and example.com are separate records.

Could you elaborate this please??

Thank you

[penagate]

They're not the same domain. Although most people use both to point to the same site, it's quite possible to point them in different directions (often with quite confusing results).

[RudiVisser]

Yeah, for example, instead of www. and *nothing*.

Think of:
server1.example.com
server2.example.com

They would point to different servers, same with www. and *nothing*.

Another way to possibly clarify it is if example.com wasn't meant to be on the web, just registered, or something, perhaps an intranet domain, you don't want www. infront of it, as it's not on the world wide web (www).

Probably confused things more, but, whatever, hopefully you'll understand.

[penagate]

Personally, I don't see the point of www; I hard redirect the www address of all sites I manage to the non-www form. It looks cleaner that way and you don't have people getting confused with different cookies and so on.