|
-
Sep 11th, 2007, 08:46 PM
#1
Thread Starter
Lively Member
Fetching Text from a Given URL
Basically I want a script to fetch proxys off a given website.
(http://hidemyass.com/free_proxy_lists.php, in this case)
I need it to goto the webpage (url)
and fetch nothing but the Proxy and port like so
127.0.0.1:80, and store it in a Mysql (or any other resource)
(preferably in a colum)
with a refresh rate of about 4 hours?
THANKs!
-
Sep 11th, 2007, 10:39 PM
#2
Thread Starter
Lively Member
Re: Fetching Text from a Given URL
i have a basic example set up..
http://lukeidiot.com/display.php will get just the proxy and port from:
http://lukeidiot.com/proxylist.html
-
Sep 12th, 2007, 01:08 AM
#3
Re: Fetching Text from a Given URL
You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).
-
Sep 12th, 2007, 09:21 AM
#4
Thread Starter
Lively Member
Re: Fetching Text from a Given URL
 Originally Posted by penagate
You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).
How would i go about getting just the proxy and port
from: http://lukeidiot.com/proxylist.html ?
-
Sep 12th, 2007, 09:23 AM
#5
Re: Fetching Text from a Given URL
How've you done it on display.php? I assumed you had done that part.
-
Sep 12th, 2007, 09:34 AM
#6
Thread Starter
Lively Member
Re: Fetching Text from a Given URL
 Originally Posted by penagate
How've you done it on display.php? I assumed you had done that part.
Oh, the php code for that is:
php Code:
$dbh=mysql_connect ("localhost", "******", "*******") or die ('I cannot connect to the database because: ' . mysql_error());
mysql_select_db ("********");
$query = "SELECT * FROM proxies";
$result = mysql_query($query) or die(mysql_error());
while($row = mysql_fetch_array($result)){
echo $row['Proxy']. ":". $row['Port'];
echo "<br />";
}
?>
The only thing i'm having trouble with is getting "proxycrawler.php" to
go to a given proxylist(url) (.php, or .html) and get nothing but the Proxy and Port, and add it to a mysql. 
Note: "proxycrawler.php" is an example, it doesnt exist yet.
-
Sep 12th, 2007, 09:42 AM
#7
Re: Fetching Text from a Given URL
-
Sep 12th, 2007, 09:48 AM
#8
Thread Starter
Lively Member
Re: Fetching Text from a Given URL
 Originally Posted by penagate
I think I understand how to get the certain text (proxies)
but how do i go about adding the data found (proxy, port)
to a mysql?
-
Sep 12th, 2007, 09:56 AM
#9
Thread Starter
Lively Member
Re: Fetching Text from a Given URL
maybe even a little sample of how you would format it also, im not the best at php, but i understand quickly.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|