Results 1 to 15 of 15

Thread: Vb .net - Operate in another process for mouse and resolution

  1. #1

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Vb .net - Operate in another process for mouse and resolution

    Hi all, i have a problem:
    I have a program in Vb .net that must operate with a specific window of Chrome.
    In my program, trough processes, i am able to search and find the window of chrome (trough the Maintitle) , obtain an handle and an id of the window.
    Now i must "communicate" with this particular window and set his resolution (different from my pc resolution and without changing my pc resolution, only the window), and move the mouse to specific coordinates and click on objects.
    In this moment i am stuck because i have no idea if i can communicate with this window and how, maybe trough Api or something similar?
    Thanks in advance for you help. I'm here waiting for hints.
    Diego.
    I am not a professional, so be easy in explanations
    Last edited by Shaggy Hiker; Jun 1st, 2024 at 01:10 PM.

  2. #2
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    39,258

    Re: Vb .net - Operate in another process for mouse and resolution

    Since you are talking about Chrome, and clicking on specific coordinates, it sounds like you are talking about automating some web process. If that's the case, the best answer is to avoid doing this if there is ANY other way. Even if you can make that work, in most cases it will drive you mad. There is something in the makeup of web masters that drives them to alter their pages frequently. What works today, fails tomorrow, and so forth.

    However, the alternatives are highly dependent on what you are actually doing.
    My usual boring signature: Nothing

  3. #3

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    Thank you Shaggy. I subscribe EVERY single letter you wrote. But unfortunately i must do this job in this way. (maybe i can explain in private better) I am already mad as you told but it is mandatory and so i go on.
    For screen resolution i hoped to use selenium driver to do a workaround, but (of course) unfortunately i am using Vb.net and not Python or Java, so life is hard. I must move mouse on objects, resolution can wait for now.
    I hope somebody will help me in declarations and use. I am reading around but documentation is heavy and sometimes fragmentary.

  4. #4
    PowerPoster PlausiblyDamp's Avatar
    Join Date
    Dec 2016
    Location
    Pontypool, Wales
    Posts
    2,613

    Re: Vb .net - Operate in another process for mouse and resolution

    There is nothing stopping you using selenium with vb.net just follow the c# instructions at https://www.selenium.dev/documentati...stall_library/ to get things set up.

    Once the Nuget package is installed you should be good.

  5. #5

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    @PlausiblyDamp : Thank you very much. Package installed and code working at the moment.
    Now the hardest part : control the other process trough API, for mouse movements and interactions.

  6. #6
    PowerPoster PlausiblyDamp's Avatar
    Join Date
    Dec 2016
    Location
    Pontypool, Wales
    Posts
    2,613

    Re: Vb .net - Operate in another process for mouse and resolution

    Quote Originally Posted by tieko68 View Post
    @PlausiblyDamp : Thank you very much. Package installed and code working at the moment.
    Now the hardest part : control the other process trough API, for mouse movements and interactions.
    Quite often, although not always, instructions for C# are equally valid for VB.Net - especially when it comes to things like Nuget packages.

    Do you need to manipulate an existing Chrome window or can you control the entire process from launching a browser yourself? Also, if using selenium you are more likely to be interacting with elements on the page, rather than screen positions etc.

    Just one more question, is there a reason you need to automate a web browser rather than use some sort of API? If the application has a programmatic API it is always preferable to trying to automate a browser; even if you get the browser automation working it can break without any notice if the designers change things, or even down to screen resolution changes if you are relying on control coordinates.

  7. #7

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    I will explain you completely, if i forgot something important just ask.
    I wrote in Vb.net a Windows Application Forms. Trough my application i start the browser with Selenium, always with Selenium i will go on a site and log in , after that i will launch a game into this site that will appear in a window. Trough processes i catch the handle of the window and now remain the last part, but heavy for me. The first useful thing is to change resoluzion of this window to a more friendly size. In selenium there is this option but actually it is not working. Maybe i can do that trough API. If done, or not, the next work is to "map" every object of interest into that window and when i have the correct coordinates, try to move the mouse and click around from my application vb .net. I need to "see" in the game application, report data to my vb .net application that makes decisions, and after "apply" them in the other process moving the mouse and clicking.
    This is all.

  8. #8
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    39,258

    Re: Vb .net - Operate in another process for mouse and resolution

    I know of no games that allow automating them in this fashion. This site cannot condone an action that goes against the limitations imposed by the game. Therefore, I am closing this thread. If the game in question has a user agreement that allows automating it, then let me know and I will reopen the thread. Otherwise, it is against the rules of this site.
    My usual boring signature: Nothing

  9. #9
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    39,258

    Re: Vb .net - Operate in another process for mouse and resolution

    After having heard from tieko68, I am re-opening the thread. It does not sound like he is violating any agreement.
    My usual boring signature: Nothing

  10. #10

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    thank you. the only thing i am violating is my brain with API

  11. #11

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    After hard working i am here. In the function "FindWindowByTitle" my goal was to obtain the thread of the window (browser) using the title, with the thread obtain the handle of the window.
    and i am stuck (overburning i think). Can you help me to modify it ?
    Code:
    Imports System.Diagnostics
    Imports System.Drawing
    Imports System.Drawing.Imaging
    Imports System.Runtime.InteropServices
    
    Public Class Form1
    
        Private Sub btnCaptureScreen_Click(sender As Object, e As EventArgs) Handles btnCaptureScreen.Click
            CaptureWindowToFile("Titolo della Finestra", "C:\path\to\save\screenshot.png") ' Modifica il titolo della finestra e il percorso del file
        End Sub
    
        Public Function FindWindowByTitle(windowTitle As String) As IntPtr
            Return NativeMethods.FindWindow(Nothing, windowTitle)
        End Function
    
        Public Function GetProcessIdFromWindow(hWnd As IntPtr) As UInteger
            Dim processId As UInteger
            NativeMethods.GetWindowThreadProcessId(hWnd, processId)
            Return processId
        End Function
    
        Public Sub CaptureWindowToFile(windowTitle As String, filePath As String)
            Dim hWnd As IntPtr = FindWindowByTitle(windowTitle)
            If hWnd = IntPtr.Zero Then
                MessageBox.Show("Finestra non trovata.")
                Return
            End If
    
            Dim processId As UInteger = GetProcessIdFromWindow(hWnd)
            MessageBox.Show($"ID del processo: {processId}")
    
            Dim rect As New NativeMethods.RECT()
            NativeMethods.GetWindowRect(hWnd, rect)
    
            Dim width As Integer = rect.Right - rect.Left
            Dim height As Integer = rect.Bottom - rect.Top
    
            Dim bmp As New Bitmap(width, height, PixelFormat.Format32bppArgb)
            Using g As Graphics = Graphics.FromImage(bmp)
                Dim hdc As IntPtr = g.GetHdc()
                NativeMethods.PrintWindow(hWnd, hdc, 0)
                g.ReleaseHdc(hdc)
            End Using
    
            bmp.Save(filePath, ImageFormat.Png)
            MessageBox.Show($"Schermata salvata in {filePath}")
        End Sub
    
        Public Class NativeMethods
            <DllImport("user32.dll", SetLastError := True, CharSet := CharSet.Auto)>
            Public Shared Function FindWindow(lpClassName As String, lpWindowName As String) As IntPtr
            End Function
    
            <DllImport("user32.dll")>
            Public Shared Function GetWindowRect(hWnd As IntPtr, ByRef lpRect As RECT) As Boolean
            End Function
    
            <DllImport("user32.dll")>
            Public Shared Function PrintWindow(hWnd As IntPtr, hdcBlt As IntPtr, nFlags As Integer) As Boolean
            End Function
    
            <DllImport("user32.dll")>
            Public Shared Function GetWindowThreadProcessId(hWnd As IntPtr, ByRef lpdwProcessId As UInteger) As UInteger
            End Function
    
            <StructLayout(LayoutKind.Sequential)>
            Public Structure RECT
                Public Left As Integer
                Public Top As Integer
                Public Right As Integer
                Public Bottom As Integer
            End Structure
        End Class
    End Class
    Last edited by Shaggy Hiker; Jun 2nd, 2024 at 04:53 PM. Reason: Added CODE tags.

  12. #12

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    Solved, there was a mistake in the call. the Dim instruction was not of correct type. Now i obtained the hWnd ptr but if i try to obtain data from the process like in this code, it does not show any error but returns nothing.
    I believed it was for permission problems, so i installed the program into trusted location (c:\Program Files) and started it with admin privileges, but still no answer. GetWindowsRect failed.
    Code:
    Public Sub MoveMouseToProcessWindow(handletavolo As IntPtr, x As Integer, y As Integer)
    
        Dim rect As New NativeMethods.RECT()
    
    
        NativeMethods.GetWindowRect(handletavolo, rect)
    
        If Not NativeMethods.GetWindowRect(handletavolo, rect) Then
            MessageBox.Show("GetWindowRect failed")
            Return
        End If
    
        ' Calcola la posizione assoluta del mouse
        Dim absoluteX As Integer = rect.Left + x
        Dim absoluteY As Integer = rect.Top + y
    
        ' Muovi il cursore del mouse
        NativeMethods.SetCursorPos(absoluteX, absoluteY)
    
    
    
    End Sub
    Last edited by Shaggy Hiker; Jun 2nd, 2024 at 04:54 PM. Reason: Added CODE tags.

  13. #13

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    Maybe we'have another trouble, instead of the permission : i used spy++ and the handle shown bi it is different than mine. I am obtaining something not correct ?
    here you are the routine to obtain handle:
    Code:
     Private Function FindWindowByPartialTitle(partialTitle As String) As IntPtr
    
         Dim handle As IntPtr = IntPtr.Zero
    
         NativeMethods.EnumWindows(New NativeMethods.EnumWindowsProc(Function(hWnd, lParam)
    
         Dim title As New StringBuilder(256)
    
         NativeMethods.GetWindowText(hWnd, title, title.Capacity)
    
         If title.ToString().StartsWith(partialTitle, StringComparison.OrdinalIgnoreCase) Then
                       handle = hWnd
                       Return False ' stop enumerating
         End If
                       Return True ' go on enumerating
                       End Function), IntPtr.Zero)
         Return handle
     End Function
    Last edited by Shaggy Hiker; Jun 2nd, 2024 at 04:54 PM. Reason: Added CODE tags.

  14. #14
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    39,258

    Re: Vb .net - Operate in another process for mouse and resolution

    I edited your posts to add [CODE][/CODE] tags. You can do this by either pressing the # button and pasting the code between the resulting tags, or pasting in the code, then highlighting it and pressing the # button.
    My usual boring signature: Nothing

  15. #15

    Thread Starter
    New Member
    Join Date
    May 2024
    Posts
    9

    Re: Vb .net - Operate in another process for mouse and resolution

    Sorry Shaggy, it's my fault, i forgot the feature.
    In the meantime i solved all problems and now i am controlling with success the other process , moving the mouse, clicking around and recording screen to file.
    If you're interested i can paste code (with tags) for other users that may have the same problem.
    Let me know.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width