Results 1 to 3 of 3

Thread: Web crawler/scraper

  1. Oct 23rd, 2011, 05:37 PM #1
    baja_yu
    baja_yu is offline

    Thread Starter
    Next Of Kin baja_yu's Avatar
    Join Date
    Aug 2002
    Location
    /dev/root
    Posts
    5,989

    Web crawler/scraper

    Does anyone know of a (free) tool, Windows or Mac OS, that can easily be set to extract all URLs (hyperlinks) from a website based on a pattern, for example they point to google.com etc. I need it to crawl the entire website/domain not just a single html page.
    ---------------------
    IconTextBox Custom Control
    DropDown ListView Custom Control
    Reply With Quote Reply With Quote
  2. Oct 23rd, 2011, 06:56 PM #2
    Justa Lol
    Justa Lol is offline
    Fanatic Member
    Join Date
    Jun 2008
    Posts
    1,023

    Re: Web crawler/scraper

    maybe this will help you? http://www.openwebspider.org/
    Reply With Quote Reply With Quote
  3. Oct 24th, 2011, 06:24 AM #3
    Hack
    Hack is offline
    I'm about to be a PowerPoster! Hack's Avatar
    Join Date
    Aug 2001
    Location
    Searching for mendhak
    Posts
    58,333

    Re: Web crawler/scraper

    Moved To General Developer
    Reply With Quote Reply With Quote
« Previous Thread | Next Thread »

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  

Forum Rules


Click Here to Expand Forum to Full Width