Results 1 to 9 of 9

Thread: Fetching Text from a Given URL

  1. #1

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Fetching Text from a Given URL

    Basically I want a script to fetch proxys off a given website.
    (http://hidemyass.com/free_proxy_lists.php, in this case)

    I need it to goto the webpage (url)
    and fetch nothing but the Proxy and port like so
    127.0.0.1:80, and store it in a Mysql (or any other resource)
    (preferably in a colum)

    with a refresh rate of about 4 hours?

    THANKs!

  2. #2

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Re: Fetching Text from a Given URL

    i have a basic example set up..
    http://lukeidiot.com/display.php will get just the proxy and port from:
    http://lukeidiot.com/proxylist.html

  3. #3
    I'm about to be a PowerPoster!
    Join Date
    Jan 2005
    Location
    Everywhere
    Posts
    13,647

    Re: Fetching Text from a Given URL

    You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).

  4. #4

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Re: Fetching Text from a Given URL

    Quote Originally Posted by penagate
    You can use file_get_contents to retreive the contents of a a remote URL, and a cronjob/Scheduled Task to run it every 4 hours (*nix/Windows, respectively).
    How would i go about getting just the proxy and port
    from: http://lukeidiot.com/proxylist.html ?

  5. #5
    I'm about to be a PowerPoster!
    Join Date
    Jan 2005
    Location
    Everywhere
    Posts
    13,647

    Re: Fetching Text from a Given URL

    How've you done it on display.php? I assumed you had done that part.

  6. #6

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Re: Fetching Text from a Given URL

    Quote Originally Posted by penagate
    How've you done it on display.php? I assumed you had done that part.
    Oh, the php code for that is:
    php Code:
    1. $dbh=mysql_connect ("localhost", "******", "*******") or die ('I cannot connect to the database because: ' . mysql_error());
    2. mysql_select_db ("********");
    3. $query = "SELECT * FROM proxies";
    4.      
    5. $result = mysql_query($query) or die(mysql_error());
    6.  
    7.  
    8. while($row = mysql_fetch_array($result)){
    9.     echo $row['Proxy']. ":". $row['Port'];
    10.     echo "<br />";
    11. }
    12. ?>

    The only thing i'm having trouble with is getting "proxycrawler.php" to
    go to a given proxylist(url) (.php, or .html) and get nothing but the Proxy and Port, and add it to a mysql.

    Note: "proxycrawler.php" is an example, it doesnt exist yet.

  7. #7
    I'm about to be a PowerPoster!
    Join Date
    Jan 2005
    Location
    Everywhere
    Posts
    13,647

    Re: Fetching Text from a Given URL


  8. #8

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Re: Fetching Text from a Given URL

    Quote Originally Posted by penagate
    I think I understand how to get the certain text (proxies)
    but how do i go about adding the data found (proxy, port)
    to a mysql?

  9. #9

    Thread Starter
    Lively Member
    Join Date
    Aug 2007
    Posts
    95

    Re: Fetching Text from a Given URL

    maybe even a little sample of how you would format it also, im not the best at php, but i understand quickly.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width