Fetching Text from a Given URL
Basically I want a script to fetch proxys off a given website.
(http://hidemyass.com/free_proxy_lists.php, in this case)
I need it to goto the webpage (url)
and fetch nothing but the Proxy and port like so
127.0.0.1:80, and store it in a Mysql (or any other resource)
(preferably in a colum)
with a refresh rate of about 4 hours?
THANKs!
Re: Fetching Text from a Given URL
i have a basic example set up..
http://lukeidiot.com/display.php will get just the proxy and port from:
http://lukeidiot.com/proxylist.html
Re: Fetching Text from a Given URL
You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).
Re: Fetching Text from a Given URL
Quote:
Originally Posted by penagate
You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).
How would i go about getting just the proxy and port
from: http://lukeidiot.com/proxylist.html ?
Re: Fetching Text from a Given URL
How've you done it on display.php? I assumed you had done that part.
Re: Fetching Text from a Given URL
Quote:
Originally Posted by penagate
How've you done it on display.php? I assumed you had done that part.
Oh, the php code for that is:
php Code:
$dbh=mysql_connect ("localhost", "******", "*******") or die ('I cannot connect to the database because: ' . mysql_error());
mysql_select_db ("********");
$query = "SELECT * FROM proxies";
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_array($result)){
echo $row['Proxy']. ":". $row['Port'];
echo "<br />";
}
?>
The only thing i'm having trouble with is getting "proxycrawler.php" to
go to a given proxylist(url) (.php, or .html) and get nothing but the Proxy and Port, and add it to a mysql. :D
Note: "proxycrawler.php" is an example, it doesnt exist yet.
Re: Fetching Text from a Given URL
Re: Fetching Text from a Given URL
Quote:
Originally Posted by penagate
I think I understand how to get the certain text (proxies)
but how do i go about adding the data found (proxy, port)
to a mysql?
Re: Fetching Text from a Given URL
maybe even a little sample of how you would format it also, im not the best at php, but i understand quickly.