[RESOLVED] dynamic search results? like ebay

[Phill64]

Ok,

in google if i was to search for "fishing rod", if ebay has matches to "fishing rod" it will display "great deals on fishing rods at ebay" and when you click on it takes you to a search results page.

i was wondering how one would get something like this to work?

[mendhak]

Which part of this do you need to implement? The google advertisement or the keyword propagation into a search result?

[Phill64]

using the keyword.

I don't understand how this system works, since the search engines bot scans all the links in your page, is there some kind of wild card? or do you create a page for the bot to read containing all the possible keywords(like product names) in your database..

any help or ideas is appreciated

[mendhak]

That link has a parameter which google passes to the search page:

googkw=Fishing%20rod

The search page takes that keyword and does its own search, and then displays it.

[Phill64]

I can't see anywhere on google about this.
It would be an extremely large task if google was to send a query to all its links.. even to just its sponsered links is alot, whenever somebody did a search, surely it doesnt work that way.

When i go into the page for fishing rods it goes to URL "http://search-desc.ebay.com.au/Fishing-Rods_W0QQfclZ4QQfnuZ1QQfsopZ1QQftsZ2QQsaatcZ15"

where as doing the search within ebay takes you to "http://search-desc.ebay.com.au/search/search.dll?sofocus=bs&sbrftog=1&fcl=4&fnu=1&from=R10&satitle=Fishing+Rods&sacat=-1%26catref%3DC6&bs=Search&fts=2&fsop=1%26fsoo%3D1&fgtp=&sargn=-1%26saslc%3D2&sadis=200&fpos=Postcode&ga10244=10425&ftrt=1&ftrv=1&saprclo=&saprchi=&saatc=15&fhlc=1"

and the results are infact different.

I think that makes my idea of the index page likely, could have a hidden link on the homepage that the search engines pickup, taking them to a code generated page marked not to be indexed, but be packed full of links to search result pages to all the different product names.

[sevenhalo]

This may not be quite what you're asking for, but I'll give it a shot:

The first example is before ebay knows anything about you. Since Ebay pays google as a premeire site (or whatever it is you call it), chances are the really weird number and letters are a hash key to validate that you actually visited their site from google (pay per click). The other information is just arguements to be used in the most basic search ebay has to offer.

The second one is where you yourself went to google. By doing a reverse DNS search, they were able to localize you "approximately" and determine a postal code (one of the arguements in that url) from that information. How it uses the rest of them is specific to ebay's code. I'm not sure about those.

If you're curious about how to talk to bots, check out this page. They can give you an idea on how to leave bait for bots.

[Lord_Rat]

Google's entire existance depends on the fact they can perform the search you are talking about.

They didn't write a hundred lines of code to do it, they wrote thousands and probably millions. You are asking now for "How do I build the Taj Mahal?" when your resources are some planks of wood, a few nails and some paint.

Could it eventually be done? Sure, given time, dedication and hard work, but it is not going to be answered in a simple "Do A, B C" manner.


Now, all that said, on a very high level, I am sure that Google gets advertisers that say "Hey, I sell fishing equipment, so any article with the words 'worms' 'bait' 'tacklebox' or 'bass' I want you to show my advertisement" and Google indexes that and thats what you get.

[Phill64]

unfrotunetly no, thats not what i was asking, but thanks for the reply anyway.

my question was how does google know about a search results page for "fishing rods" since its just a database entry, the page doesnt exist until you send the search to it. but i've discovered that infact the page DOES exist, they obviously run a system every now and again that regenerates all these predefined search result pages for google to find.

I think i can mark this one as resolved.
Thanks guys

[Phill64]

Google's entire existance depends on the fact they can perform the search you are talking about.

They didn't write a hundred lines of code to do it, they wrote thousands and probably millions. You are asking now for "How do I build the Taj Mahal?" when your resources are some planks of wood, a few nails and some paint.

Could it eventually be done? Sure, given time, dedication and hard work, but it is not going to be answered in a simple "Do A, B C" manner.


Now, all that said, on a very high level, I am sure that Google gets advertisers that say "Hey, I sell fishing equipment, so any article with the words 'worms' 'bait' 'tacklebox' or 'bass' I want you to show my advertisement" and Google indexes that and thats what you get.

actually i have experimented with search engines before, and the main problem is computer power, rather than code, it was after all written by 3 students to begin with. But that aside I don't think you understood my question at all, i understand how the index works, thats what caused my confusion as to how websites make searches appear dynamic within a search engine, but as per my last post ive discovered they create these search results as actual pages to make the bots index them. although my 1st post was not very detailed, didnt explain it very well

[Lord_Rat]

Umm, those pages are built from their database when you search. They are not static.

But what you have asked is based in Language Theory. Google made their own system for interpreting key words in to results and they do it fast, on the fly, when you want it, where you want it. It is their system.

Other vendors have their own, like KNova and their KnowledgeCenter product, which drives many companies' search engines, like ebay.com.

If you want specifics on some aspect of how a search works, whether it be how the index happens, how the keywords are interpreted or how the two are matched in real time to provide a user details that they can read and search through, you might be better suited to ask a support board for a search engine.

If you want, I can direct you to the one for Google. They have their own Google Search Engine Community full of people who debate this stuff day in and day out.

[Lord_Rat]

it was after all written by 3 students to begin with.

And then updated by a team of programmers since.

Google 1.0 wasnt that complex, it was just fast and clean. That's where they won their market.

But that aside I don't think you understood my question at all, i understand how the index works, thats what caused my confusion as to how websites make searches appear dynamic within a search engine, but as per my last post ive discovered they create these search results as actual pages to make the bots index them. although my 1st post was not very detailed, didnt explain it very well

That being said, you are correct in that I didnt understand the original question, and that the above is more clear, but to answer it for you now, I must tell you that you *almost* got it.

In fact what happens in these cases (the other sites that vie for top results in Google, but are more generic in scope) is that they have a custom 404 handler for when results come to them. That handler checks the referrer and if the referrer is a known indexing bot, then instead of a real 404, they return a keyword sheet with a bunch of words and jargon to make it appear a page exists at the designated URL that might handle the search the user wanted.

And then when the user comes from a search engine, the referrer includes the user's search terms (just look at the URL) and the same 404 invokes, but instead of sending back a junk fact sheet, the user is redirected to whatever page the source Web site wanted them to be redirected to.

[Phill64]

That makes alot of sense.
you are basically turning the filename into a query field for your page, so infact you could have as many of these as you liked and the bot would believe them to be different pages.

thanks for your help

[Lord_Rat]

NP, and sorry bout the earlier confusion.