Help Me! - How to extract website's contents using VB - Help Me!

[maxcoder]

Hi there!
I am a vb programmer. I have a problem in my last project and welcome any solution from you.

I am developing a program to connect to internet and extract some information from some websites. I am able to code extraction using winsock, API or inet control. These websites uses login and cookies and check it in each HTTP request. Websites uses POST method for login so I can't include info in URL as a GET action. I also able to code HTTP POSTs using either Winsock or Inet. So I can POST a login username and password.

Problem:
1.Some websites use session, When I login and request next page it lose the session so request login again! Websites I said about are ASP and JSP technologies based websites. ASPs maybe uses some built-in variables or cookies but I see something like "jsession=" in JSP URLs, However I try to include it in next call but problem continues.

2.Some websites use cookies, (maybe problem comes from cookies). I don't know how I can save, get and post cookies for next call.

3.Some websites also check the browser for cookies and java compatiblity, so they reject login request from Winsock and Inet. I don't know how I can set browser-agent-name in Inet but It's OK for Winsock control.

However I thing that my problem based in sessions or cookies. I need a solution (source code, example project or reference about this problem) that show me how I can LIVE a session or cookie during several visits of a website? How I can import session and login info from one page visit to another? Or my solution is completely worng and I have to change the way?

It's a program that have to login to websites, go to next page after login page (a search page) and fill and submit a form (search query) and exract some information from results page's HTML tags. All my problem is for logon and borwser checkups.

Thank you...
Max