how can i modify this code to search in non-english text !?

**Dode** · Aug 26th, 2000, 09:40 PM

hi

Code:

#!/usr/local/bin/perl
####################################################################
# URL Search Engine
# Copyright 1996 Techno Trade  http://www.technotrade.com
# Written By : Sammy Afifi   [email protected]
# Date Last Modified : Jan 14, 1997
####################################################################
#
# This script is free of charge.
# Please link back to http://technotrade.com, thank you :-)
#######
#
# Jan 14, stops people from entering html tags, converts < to <
#
####################################################################

#  $linktitle, $linkdescrip,  $linkwords, $linkemail, $linkurl
# define some global variables

    $fields = 5;                       # Number of fields in each record
    $filename = "urls.txt";      # The database text file
    $results = 1000;               # maximum number of results to display

    &parse_form;
    
    $searchstring = $FORM{'searchstring'};

    &addrecord if ($searchstring eq "**ADD RECORD**");


    &open_file("FILE1","",$filename);

    print "Content-type: text/html\n\n";
    print "<HTML>\n";
    print "<BODY BGCOLOR=#FFFFFF TEXT=#000000 LINK=#0000FF VLINK=#800040 ALINK=#800040>\n";
    print "<TITLE>Search Results</TITLE>\n";
    print "<CENTER><BR>\n";
    print "<FONT SIZE=5 COLOR=000000 FACE=\"ARIAL,TIMES NEW ROMAN\"><B>Results :</b></FONT></CENTER>\n";

    print "<HR width=80% noshade><BR><UL>\n";
    $counter = 0;

    while (($line = &read_file("FILE1")) && ($counter < $results)) {
         # split the fields at the | character     
         @tabledata = split(/\s*\|\s*/,$line ,$fields);
          &check_record;
          if ($found == 1) {
            $counter++;
            &print_record;
          }

    }
    close(FILE1);
    print "</UL>\n";

    if ($counter == 0) {
       print "<BR><center><B>Sorry ! not found.</B>\n";
    }
    
    
    print "<CENTER>\n";
    print "<HR width=80% noshade>\n";
    print "</CENTER>\n";
    print "</A></BODY></HTML>\n";



#########################################
#
#  Print the matched record
#
#########################################
sub print_record {
       print "<BR>\n";
       print "<LI><A HREF=" . $linkurl . ">" . $linktitle . "</A>  : $linkdescrip</B><BR>\n";    
}


##########################################
#
#  Check to see if record matches search criteria
#
##########################################
sub check_record {
    # get the data from the record read from the file. $tabledata

   $linktitle = $tabledata[0];
   $linkdescrip = $tabledata[1];
   $linkwords = $tabledata[2];
   $linemail   = $tabledata[3];
   $linkurl   = $tabledata[4];
   chop($linkurl);

    #build the search line with all fields we want to search in
    $searchline = $linktitle . " " . $linkdescrip . " " . $linkwords;


   #search by keywords
   # only perform the keyword search if the length of the search string is greater than 2
   # don't think we want people to search for and  or or etc.
   $sfound = 0;
   $found = 0;
   $notfound = 1; 

   $stlen = length($searchstring);
   if ($stlen > 1) {
       @words = split(/ +/,$searchstring);
        foreach $aword (@words) {
           if ($searchline =~ /\b$aword/i) {
                  $sfound = 1;
           } 
           else {
                  $notfound = 0;
            }
         }
     }
    if ($sfound == 1 && $notfound == 1) {
        $found = 1;
     }

    # if search string is too small .. set found to 1
    if ($stlen <= 1) {
        $found = 1;
    }
    #if page doesn't have a title then return not found
    $tlen = length($linktitle);
    if ($tlen < 1) {
        $found = 0;
    }
}


############################################
#
#  Add Record
#
############################################

sub addrecord {

  $linktitle    = $FORM{'linktitle'};
  $linkdescrip    = $FORM{'linkdescrip'};
  $linkwords    = $FORM{'linkwords'};
  $linkemail   = $FORM{'linkemail'};
  $linkurl = $FORM{'linkurl'};

  # Convert < tags to <
  $linktitle =~ s/</\</g;
  $linkdescrip =~ s/</\</g;
  $linkwords =~ s/</\</g;
  $linkemail =~ s/</\</g;
  $linkurl =~ s/</\</g;


  
  &open_file("FILE1",">>",$filename);

  &write_file("FILE1",$linktitle . "|". $linkdescrip. "|" .$linkwords ."|" .$linkemail ."|" .$linkurl ."\n");
   close(FILE1);
   print "Content-type: text/html\n\n";
   print "<html><head><title>Thank You</title></head>\n";
   print "<body background="/images/bground.gif" bgcolor="#ffffff" text="#000000" marginheight="0" topmargin="0"><BR><h3><CENTER>Thank you !</CENTER></h3>\n";
   print "</body></html>\n";
   exit;
}











sub parse_form {

   read(STDIN, $buffer, $ENV{'CONTENT_LENGTH'});
   if (length($buffer) < 5) {
         $buffer = $ENV{QUERY_STRING};
    }
 
  @pairs = split(/&/, $buffer);
   foreach $pair (@pairs) {
      ($name, $value) = split(/=/, $pair);

      $value =~ tr/+/ /;
      $value =~ s/%([a-fA-F0-9][a-fA-F0-9])/pack("C", hex($1))/eg;

      $FORM{$name} = $value;
   }
}



sub open_file {

  local ($filevar, $filemode, $filename) = @_;
  
  open ($filevar,$filemode . $filename) ||
     die ("Can't open $filename");
}

sub read_file {

  local ($filevar) = @_;

  <$filevar>;  
}

sub write_file {

  local ($filevar, $line) = @_;

  print $filevar ($line);
}

i'm waitting ...

**Zvonko** · Aug 27th, 2000, 09:05 AM

As much as I know about Perl there's no need to modify that code. It's just search engine and it will search specified string no matter in what language it is written.

An example for your statement:
If you want to make an Spanish program in VB Microsoft would have to translate all VB keywords, operators, statements... to Spanish that you could make program in Spanish language. No offence, but isn't that stupid?

**Dode** · Aug 27th, 2000, 07:08 PM

wanted : Perl Peogrammer

**Jop** · Aug 27th, 2000, 07:28 PM

Hey, I think Zvonko's right, ofcourse it will search in any language, that has nothing to do with the code, but be sure to put the keywords in the desired language ofcourse.

And BTW, why are you posting a Perl Question in a VB forum?

**Paul282** · Aug 27th, 2000, 09:17 PM

Maybe perl has the same problem as VB does with Double byte characters. A Len() call will give the length in characters as apposed to bytes. "Lenb()"

Not sure how perl works with strings but it'd be worth investigating

Thread: how can i modify this code to search in non-english text !?

Thread Tools

Display

Posting Permissions