How to copy remote html to localhost folder ?

[tony007]

Hi all. could any one show me how i can copy a remote html page to a folder inside localhost folder. For example i want to copy the following url html to folder named copy and name it to whatever value of $id is plus .html (example :23434.html) :


remote URL to copy :

$copyfileurl="http://friends.myspace.com/index.cfm?fuseaction=user.viewfriends2&friendID=". $id; 

Later i want to load the copied html to curl_setopt($curl, CURLOPT_URL, '.....html'

curl_setopt($curl, CURLOPT_URL, '.....html');
curl_setopt($curl, CURLOPT_POST, 0);
$result = curl_exec($curl); 

I hope some one show me how this can be then.Thanks

[RudiVisser]

Well providing you have permissions it's as simple as this:

$copyfileurl="http://friends.myspace.com/index.cfm?fuseaction=user.viewfriends2&friendID=". $id;
file_put_contents($id.".html", file_get_contents($copyfileurl)); 

I don't know what you mean by loading it into cURL. But I guess it would just be 'http://localhost/'.$id.'.html'.

[tony007]

Thanks rudi. What i mean by loading is to use the same saved html file inside this function curl_setopt instead of test.html .

curl_setopt($curl, CURLOPT_URL, 'test.html' 

I tried this but it value of url didn't get loaded properly !!

        curl_setopt($curl, CURLOPT_URL, 'http://localhost/testFolder/'.$id.'.html.'); 

Furthermore, is it possible to save the remove file inside a folder instead of current directory where this script runs ?

[RudiVisser]

Writing:

$copyfileurl="http://friends.myspace.com/index.cfm?fuseaction=user.viewfriends2&friendID=". $id; 
file_put_contents('./testFolder/$id.".html", file_get_contents($copyfileurl)); 

Reading:

curl_setopt($curl, CURLOPT_URL, 'http://localhost/testFolder/'.$id.'.html'); 

[tony007]

I think i got the problem. The following code works if i save the remote page to localhost folder manually!! otherwise it will not work.

i saved the remote file manully and used the following code and boom it gave me the page number .


I think copying the remote file and processing it at same time will not work as the preg_match function has not enough time to process it. Is there any way to put some sort of delay so that preg_match fires ONLY after the remote page copied completely to local drive?

if (preg_match('/of <a href="([^"]+)" class="pagingLink">(\d+)<\/a>/', $result, $matches))
    {
        $pages = $matches[2];
            echo " Number of pages: ".$pages." ";
            unset($matches);

    } 

$copyfileurl="http://friends.myspace.com/index.cfm?fuseaction=user.viewfriends2&friendID=". $id; 
file_put_contents($id.".html", file_get_contents($copyfileurl));

// Retrieve friends page.
    echo 'Checking friends list... ';
        echo " <br>";
       
        curl_setopt($curl, CURLOPT_URL, 'http://localhost/test/'.$id.'.html');
    curl_setopt($curl, CURLOPT_POST, 0);
    $result = curl_exec($curl);

 // i added this 
//print $result;
    $headers = curl_getinfo($curl);
    
    // Find out how many pages there are. 

    if (preg_match('/of <a href="([^"]+)" class="pagingLink">(\d+)<\/a>/', $result, $matches))
    {
        $pages = $matches[2];
            echo " Number of pages: ".$pages." ";
            unset($matches);

    }
else
{

echo "couldn't get the number of pages";

} 

[RudiVisser]

Well you can sleep(int seconds); but that's not practical.

Anyway, file_get_contents should halt execution until it's complete anyway, so I'm not sure what you mean, meaning I don't think that's your issue.

Have you tried updating PHP/Apache as per your other post??

I don't see why you're saving it to disk when you could just grab the contents straight into a string and preg match on the string. Using cURL for a simple GET request seems like overkill to me too.

[tony007]

well i can't update my server at this time as i am worried that my other
program stops working. But any ways after testing many things. i found out that my preg match function prints out the result only if the html file to process is saved localy with .html extention. Otherwise it will not work for some reasons that i don't know why. That is why i decided to save the remote page first them process it!!

I was thinking that preg_match function gets excuted in the middle or just before end of the copy process.

if (preg_match('/of <a href="([^"]+)" class="pagingLink">(\d+)<\/a>/', $result, $matches)) 
    { 
        $pages = $matches[2]; 
            echo " Number of pages: ".$pages." "; 
            unset($matches); 

    } 

[RudiVisser]

Like I said, file_get_contents will block until it is done downloading, and file_put_contents will block until it's done writing too.

It might be worth setting up an upgraded system to run on port 81 (for example) to test if the upgrades have any effect on what you're doing since what you are describing is not happening on my 2 local machines nor either of my servers.