Data Mining?

[Dillinger4]

Does anyone know what is the best way to take data from a website to be used in a program? I have never programmed in this manner before where data is comming from an external source to be used in a program. How would i extract the information. Say i wanted to get the current NAV(net asset value) of a stock from a web site to be used in a program? Anything to lead me on the right track would be greatly appreciated.

[hellswraith]

This doesn't solve your problem, but beware that there is legal things to worry about when doing that. When you are taking information from a web site without the sites permission to be used in your application, it is against copyright laws. I know it isn't what you were looking for, but thought I would throw it out anyway just in case you didn't know.

[hellswraith]

If it is a program for personal use, then it is ok, but if you plan to distribute the program which relies on the web sites content, then you are obligated to get their permission. This is because you are using copyrighted material. Ask a lawyer that specializes with web content if you want to be sure though...

[Tygur]

I don't think that's breaking copyright laws. What of the apps that retrieve information from search engines?

You would probably get in trouble if you claimed that the information was your own, but that's about it.

[hellswraith]

Maybe I am not explaining it well...

All information on my website is copyrighted just by the fact that I created it. If someone was to take information from my site with a program and present it in any other way than I intended, and made a profit from it (or used it for business gain), I would take legal action against said party. It is only fair because I created the content, and it is being used without my expressed permission.

How do you think CNN would feel if I took information from their site every hour to create another news site with their information without their permission? They would sue me in a heart beat...

Even though stock quotes are public information, how you obtain those is not public by any means. To have live updated stock prices on your site (without using a free service that is supported by adds), you will be paying for it.

I didn't mean to get on a rant here, I was just trying to inform him that there COULD be legal implications by doing this. Doesn't mean it WILL happen, just that it could. Kind of a "dig a little before you go mining" type a statment...lol.

[Tygur]

I think there is a big difference between stock quotes and news articles. But even so, if someone makes an app that grabs news headlines off cnn.com and presents them to a user, I don't think that's neccessarily bad (that's what browsers do), unless they present these articles in such a way that they avoid giving cnn.com credit. Otherwise, the only complaint they'd probably bring up is a possible waste of bandwidth and loss of money due to ads not being shown (if there are any ads).

I could be wrong, but those are my views. If I am somehow mistaken, let me know.

[Dillinger4]

The information that i want to get is intended to be used in a personal program. Also there is no need for the data to be real time, just the end of the day values will suffice. With that being said what would be the best way to grab any data?

[hellswraith]

Well, you can download the source for the page. Then use some parsing to find the information your looking for. That is what I would start with. I don't think there is any other way to get information from a page without getting the source first.

[Dillinger4]

Yeah your right. I guess that would be the only way. Thanks for the help guys.