[RESOLVED] Google image search by code?

[OddGamer]

I'm trying to go through a list of things to search for using Google Images (https://www.google.ca/imghp?hl=en), but I'm having trouble getting it to "click" the button. So far I've tried the following:

            doc.GetElementById("lst-ib").SetAttribute("Value", Current & Environment.NewLine)
            doc.GetElementById("_fZl").InvokeMember("click")
            doc.GetElementById("_fZl").InvokeMember("submit")
            doc.GetElementById("_fZl").InvokeMember("Click")
            doc.GetElementById("_fZl").InvokeMember("Submit")

I tried the "newline" thing in an attempt to make it add an "Enter" key to the end as that triggers the search on a normal web-page. None of the above works. Any ideas?

[PlausiblyDamp]

I'm trying to go through a list of things to search for using Google Images (https://www.google.ca/imghp?hl=en), but I'm having trouble getting it to "click" the button. So far I've tried the following:

            doc.GetElementById("lst-ib").SetAttribute("Value", Current & Environment.NewLine)
            doc.GetElementById("_fZl").InvokeMember("click")
            doc.GetElementById("_fZl").InvokeMember("submit")
            doc.GetElementById("_fZl").InvokeMember("Click")
            doc.GetElementById("_fZl").InvokeMember("Submit")

I tried the "newline" thing in an attempt to make it add an "Enter" key to the end as that triggers the search on a normal web-page. None of the above works. Any ideas?

Rather than trying to automate the browser why not use the supported API? https://cloud.google.com/vision/ might be worth a look or if it doesn't specifically need to be google then https://azure.microsoft.com/en-gb/se...ge-search-api/ is also worth looking at.

[OddGamer]

Rather than trying to automate the browser why not use the supported API? https://cloud.google.com/vision/ might be worth a look or if it doesn't specifically need to be google then https://azure.microsoft.com/en-gb/se...ge-search-api/ is also worth looking at.

Wouldn't that be a lot harder? I mean... all I want to do is quite literally put text in the search field and perform the search to get an overview of the images google comes up with. Using the API sounds like needing to do a lot more than that. Not to mention the Google Vision one requires a credit card to even try for free and I'm currently without one. EDIT: Yes, I could do that by hand easily enough, but I want to look through dozens of searches and say "yes" to some and "no" to most and record which is which. As an example, consider doing this with a list of flowers. Rose, iris, daisy... okay, I don't know flowers. Not the point. The thing is, the first search pulls up a bunch of pictures of roses and I can decide if roses seem like something I want. This is just speeding up that process and making it a little faster. And yes, there's enough of them to go through that writing a program for this is actually going to be faster. ... Plus I'll learn how to press that button!

[PlausiblyDamp]

Wouldn't that be a lot harder? I mean... all I want to do is quite literally put text in the search field and perform the search to get an overview of the images google comes up with. Using the API sounds like needing to do a lot more than that. Not to mention the Google Vision one requires a credit card to even try for free and I'm currently without one. EDIT: Yes, I could do that by hand easily enough, but I want to look through dozens of searches and say "yes" to some and "no" to most and record which is which. As an example, consider doing this with a list of flowers. Rose, iris, daisy... okay, I don't know flowers. Not the point. The thing is, the first search pulls up a bunch of pictures of roses and I can decide if roses seem like something I want. This is just speeding up that process and making it a little faster. And yes, there's enough of them to go through that writing a program for this is actually going to be faster. ... Plus I'll learn how to press that button!

Given what you are doing isn't working, isn't supported and likely to break if anything in the page ever changes and also against Google's T&Cs then I would have thought doing something the correct, supported and legitimate way would have been the preferred option.

As what you are doing is against the google terms of service ( https://www.google.co.uk/intl/en/mob...f_service.html ) you aren't going to get a lot of help here if you carry on with your approach.

[OddGamer]

Given what you are doing isn't working, isn't supported and likely to break if anything in the page ever changes and also against Google's T&Cs then I would have thought doing something the correct, supported and legitimate way would have been the preferred option.

As what you are doing is against the google terms of service ( https://www.google.co.uk/intl/en/mob...f_service.html ) you aren't going to get a lot of help here if you carry on with your approach.

Hadn't realized it was against the TOS. Never mind, then, I'll do it by hand.

[si_the_geek]

It's not entirely clear what you want the program to do, but generally using an API is easier than trying to automate web pages (and it is far more reliable, especially with famous sites), because an API ("Application Programmers Interface") is specifically designed to be used by a program, whereas with a web page that is almost certainly not the case (especially for famous sites, which tend to make it harder on purpose, and change things regularly to break programs that do it).

However, I get the impression this is for a fairly small time saving in a short-term personal use application, in which case just doing the task fully manually would probably be the quicker option.

[Sitten Spynne]

It only SOUNDS harder. Well, at least an image search API. The Vision API is proabably harder because it's doing a different thing. Here's how it breaks down, all said and done.

An API is a contract between you and Google. When they publish an API, they are saying, "This is a way we support for your programs to get images and work with them." They are committing to only change the API if there is a great need, and to warn you long in advance. If you write a program against an API, you generally expect you won't have to tweak it often to keep it working.

The HTML version of Google search is designed to put images in front of human eyeballs and let them click on those images. There is no contract, and they have dramatically changed both the layout and the internals many times this year. Nothing stops them from randomly generating the ID/names for the tags that a program might look for to automate Google Images. When you commit to scraping a web page you understand you might have to update your application several times daily to keep it functioning.

It works this way because automation can be a big drain on Google. They make money from the HTML image search because they can inject ads into the image stream. But a lot of programs that try to automate Google Images either don't display those ads or intentionally remove them. Worse, many of those programs intend to make money off of Google's work. That's why Google takes great pains to make the HTML for Image Search quite difficult to automate.

An API gives them control. Most APIs require you to register and obtain some "developer secrets" which is a fancy word for a handful of tokens that act like usernames and passwords for your program. This lets them cut off your access if you do something suspicious, but more importantly it lets them have control over how much of their resources bots eat up. Usually free APIs are "rate limited", if you try to make more than some number of requests in a time period, they stop responding. Usually if you want to do more, there are ways to pay for it. It usually works out if your usage is crazy high, it's cheaper to write your own image search than to pay for Google's, but for small-time usage it's reasonable.

That said, it looks like Google's deprecated their image search API. It probably wasn't making them enough money to justify it. Or, more specifically, they offer a "Custom Search API" that lets you search a set of images on a site YOU own. Based on what you're saying, though, perhaps the vision API would be useful. Its goal is to "learn" what some things "look like" so you can identify objects in images.

All said and done I didn't quickly find an actual image search API for the public internet. That doesn't mean I think it's a good idea to try and scrape the Google Image Search HTML. If you take that path, you're fighting Google engineers who want to defeat you. It's not fun.

[OddGamer]

Yeah, mostly I was just trying to automate a purely mechanical thing. Not scraping as, honestly, of the over 1000 things I'm going to look up, I'll probably pick less than 50 for later consideration. It has to put the images in front of my human eyeballs since there's no metric to check against that will give me "do I find this appealing". It's not intended for anyone else, or to be published. Certainly I wasn't intending to go up against Google. I'm just checking through a huge list to see which ones I like for a different project that's entirely not programming related.

[Sitten Spynne]

I feel like I've been in scenarios like this before and I think how it usually works out is I get to some point in the project where I decide, "I'd be finished by now if I just did it manually, yet here I am, not sure if I'm 50% finished with a program to do it for me."

If it's a thing you do frequently, there might be some value in automating it. But experts tend to agree automating complex web pages is extremely hard and prone to breaking frequently.