[RESOLVED] OpenCV Pose detection from the camera

[LiwisJames]

I'm trying to convert this code to visual basic and use it, I had problems and I couldn't use it in C# I thought it was very interesting and I would like to test it:

https://elbruno.com/2020/11/26/dotne...e-made-kinect/

Let's get to the problem I have another code from the site and I made an adaptation but I saw that it is incompatible so I tried to convert to VB.NET to have a broader view but telerik can't do the conversion does anyone have any idea how I can use this?

The .caffemodel is available at the end of the website and openCV too
and.

Already thanking necesse topic the answer the others worked very well.

[jmcilhinney]

Please don't send us off to other sites to see code that you could have posted right here.

That said, I've looked at the code and I don't see anything that you shouldn't be able to convert yourself. If you can write VB code then you should be able to read most C# if you actually try. How about you do what you can for yourself and then post the C# code, your VB code and an explanation of where you're stuck and then we can address specific issues instead of vague "I can't, you do it" sort of questions.

Regardless, if you want to convert C# code snippets to VB automatically then you should get Instant VB from Tangible Software Solutions. It will handle many conversions that online tools can't and the free version would be perfectly adequate for that snippet.

[LiwisJames]

Please don't send us off to other sites to see code that you could have posted right here.

That said, I've looked at the code and I don't see anything that you shouldn't be able to convert yourself. If you can write VB code then you should be able to read most C# if you actually try. How about you do what you can for yourself and then post the C# code, your VB code and an explanation of where you're stuck and then we can address specific issues instead of vague "I can't, you do it" sort of questions.

Regardless, if you want to convert C# code snippets to VB automatically then you should get Instant VB from Tangible Software Solutions. It will handle many conversions that online tools can't and the free version would be perfectly adequate for that snippet.

It's really right I forgot the code here is my c# code with the implementations based on the other api, but I'm still with error in the pointer coordinates I intend to get it right when translating.

using System;
using System.Collections.Generic;
using System.Diagnostics;
using System.Threading;
using System.Windows.Forms;
using OpenCvSharp;
using OpenCvSharp.Dnn;
using OpenCvSharp.Extensions;
using Point = OpenCvSharp.Point;
using Size = OpenCvSharp.Size;

namespace ShadowMode
{
  
    public partial class Form1 : Form
    {

        private bool _run = true;
        private bool _doFaceDetection = true;
        private bool _doAgeGender = true;

        private VideoCapture _capture;
        private Mat _image;
        private Thread _cameraThread;
        private bool _fps = false;
        private Net _faceNet;
        private Net _ageNet;
        private Net _genderNet;
        private Net _netPose;
        private Object _detectPose;
        private double thresh;
        private int nPoints;

        private const int LineThickness = 2;
        private const int Paddingd = 10;
        public Form1()
        {

            InitializeComponent();
            Load += Form1_Load;
            Closed += Form1_Closed;
        } // end form1()

        private void Form1_Closed(object sender, EventArgs e)
        {
            _cameraThread.Interrupt();
            _capture.Release();
        }

        //begin form load
        private void Form1_Load(object sender, EventArgs e)
        {

            // # detect faces, age and gender using models from https://github.com/spmallick/learnopencv/tree/08e61fe80b8c0244cc4029ac11e44cd0fbb008c3/AgeGender
            const string faceProto = "../../models/deploy.prototxt";
            const string faceModel = "../../models/res10_300x300_ssd_iter_140000_fp16.caffemodel";
           // const string ageProto = @"../../models/age_deploy.prototxt";
            //const string ageModel = @"../../models/age_net.caffemodel";
            //const string genderProto = @"../../models/gender_deploy.prototxt";
            //const string genderModel = @"../../models/gender_net.caffemodel";
            //_ageNet = CvDnn.ReadNetFromCaffe(ageProto, ageModel);
           // _genderNet = CvDnn.ReadNetFromCaffe(genderProto, genderModel);
            _faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel);

            _capture = new VideoCapture(0);
            _image = new Mat();
            _cameraThread = new Thread(new ThreadStart(CaptureCameraCallback));
            _cameraThread.Start();
        }



        private void CaptureCameraCallback()
        {
            while (true)
            {
                if (!_run) continue;
                var startTime = DateTime.Now;

                _capture.Read(_image);
                if (_image.Empty()) return;
                var imageRes = new Mat();
                Cv2.Resize(_image, imageRes, new Size(320, 240));
                if ((bool)_detectPose)
                {

                    var frameWidth = imageRes.Cols;
                    var frameHeight = imageRes.Rows;

                    const int inWidth = 368;
                    const int inHeight = 368;

                    // Convert Mat to batch of images
                    using var inpBlob = CvDnn.BlobFromImage(imageRes, 1.0 / 255, new Size(inWidth, inHeight), new Scalar(0, 0, 0), false, false);

                    _netPose.SetInput(inpBlob);

                    using var output = _netPose.Forward();
                    var H = output.Size(2);
                    var W = output.Size(3);

                    var points = new List<Point>();

                    for (var n = 0; n < nPoints; n++)
                    {
                        // Probability map of corresponding body's part.
                        using var probMap = new Mat(H, W, MatType.CV_32F, output.Ptr(0, n));
                        var p = new Point2f(-1, -1);

                        Cv2.MinMaxLoc(probMap, out _, out var maxVal, out _, out var maxLoc);

                        var x = (frameWidth * maxLoc.X) / W;
                        var y = (frameHeight * maxLoc.Y) / H;

                        if (maxVal > thresh)
                        {
                            p = maxLoc;
                            p.X *= (float)frameWidth / W;
                            p.Y *= (float)frameHeight / H;

                            Cv2.Circle(imageRes, (int)p.X, (int)p.Y, 8, Scalar.Azure, -1);
                            //Cv2.PutText(imageRes, Cv2.Format(n), new Point((int)p.X, (int)p.Y), HersheyFonts.HersheyComplex, 1, new Scalar(0, 0, 255), 1);
                        }

                        points.Add((Point)p);
                    }

                    WriteTextSafe(@$"Joints {nPoints} found");

                    var nPairs = 14; //(POSE_PAIRS).Length / POSE_PAIRS[0].Length;
                    
                    for (var n = 0; n < nPairs; n++)
                    {
                      
                        // lookup 2 connected body/hand parts
                        var partA = points[posePairs[n][0]];
                        var partB = points[posePairs[n][1]];
                        if (partA.X <= 0 || partA.Y <= 0 || partB.X <= 0 || partB.Y <= 0)
                            continue;
                        Cv2.Line(imageRes, partA, partB, new Scalar(0, 255, 255), 8);
                        Cv2.Circle(imageRes, partA.X, partA.Y, 8, new Scalar(0, 0, 255), -1);
                        Cv2.Circle(imageRes, partB.X, partB.Y, 8, new Scalar(0, 0, 255), -1);
                    }

                }
                // rest of the code to calc FPS and display the image
                var diff = DateTime.Now - startTime;
                var fpsInfo = $"FPS: Nan";
                if (diff.Milliseconds > 0)
                {
                    var fpsVal = 1.0 / diff.Milliseconds * 1000;
                    fpsInfo = $"FPS: {fpsVal:00}";
                }

                Cv2.PutText(imageRes, fpsInfo, new Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White);
            }
        }

        private void WriteTextSafe(string v)
        {
            throw new NotImplementedException();
        }

        //Botoes

        //Start
        private void button1_Click(object sender, EventArgs e)
        {
            _run = true;
        }

        //Stop
        private void button2_Click(object sender, EventArgs e)
        {
            _run = false;
        }

        //FPS
    

        //FaceDetection
        

        }
    //end form load
} // end partial class
 // end sub

Here is the part that telerik translated but I'm trying manually but I have to focus on my other projects which are simpler but require perfection so I'm doing it slowly This is just a hobby.

Imports System
Imports System.Collections.Generic
Imports System.Diagnostics
Imports System.Threading
Imports System.Windows.Forms
Imports OpenCvSharp
Imports OpenCvSharp.Dnn
Imports OpenCvSharp.Extensions
Imports Point = OpenCvSharp.Point
Imports Size = OpenCvSharp.Size

N''amespace ShadowMode
  ''  Public Partial Class Form1
      '''  Inherits Form
Public Class Form1

        Private _run As Boolean = True
        Private _doFaceDetection As Boolean = True
        Private _doAgeGender As Boolean = True
        Private _capture As VideoCapture
        Private _image As Mat
        Private _cameraThread As Thread
        Private _fps As Boolean = False
        Private _faceNet As Net
        Private _ageNet As Net
        Private _genderNet As Net
        Private _netPose As Net
        Private _detectPose As Object
        Private thresh As Double
        Private nPoints As Integer
        Private Const LineThickness As Integer = 2
        Private Const Paddingd As Integer = 10

        Public Sub New()
            InitializeComponent()
            Load += AddressOf Form1_Load
            Closed += AddressOf Form1_Closed
        End Sub

        Private Sub Form1_Closed(ByVal sender As Object, ByVal e As EventArgs)
            _cameraThread.Interrupt()
            _capture.Release()
        End Sub

        Private Sub Form1_Load(ByVal sender As Object, ByVal e As EventArgs)
            Const faceProto As String = "../../models/deploy.prototxt"
            Const faceModel As String = "../../models/res10_300x300_ssd_iter_140000_fp16.caffemodel"
            _faceNet = CvDnn.ReadNetFromCaffe(faceProto, faceModel)
            _capture = New VideoCapture(0)
            _image = New Mat()
            _cameraThread = New Thread(New ThreadStart(AddressOf CaptureCameraCallback))
            _cameraThread.Start()
        End Sub

        Private Sub CaptureCameraCallback()
            Dim maxVal = Nothing, maxLoc = Nothing

            While True
                If Not _run Then Continue While
                Dim startTime = DateTime.Now
                _capture.Read(_image)
                If _image.Empty() Then Return
                Dim imageRes = New Mat()
                Cv2.Resize(_image, imageRes, New Size(320, 240))

                If CBool(_detectPose) Then
                    Dim frameWidth = imageRes.Cols
                    Dim frameHeight = imageRes.Rows
                    Const inWidth As Integer = 368
                    Const inHeight As Integer = 368

                    Using inpBlob = CvDnn.BlobFromImage(imageRes, 1.0 / 255, New Size(inWidth, inHeight), New Scalar(0, 0, 0), False, False)
                    End Using

                    _netPose.SetInput(inpBlob)

                    Using output = _netPose.Forward()
                    End Using

                    Dim H = output.Size(2)
                    Dim W = output.Size(3)
                    Dim points = New List(Of Point)()

                    For n = 0 To nPoints - 1

                        Using probMap = New Mat(H, W, MatType.CV_32F, output.Ptr(0, n))
                        End Using

                        Dim p = New Point2f(-1, -1)
                        Cv2.MinMaxLoc(probMap, __, maxVal, __, maxLoc)
                        Dim x = (frameWidth * maxLoc.X) / W
                        Dim y = (frameHeight * maxLoc.Y) / H

                        If maxVal > thresh Then
                            p = maxLoc
                         ''' To Be Here
						 
						 ''' conversion here
						 
						 ''' End Code fallow:
''' 
                            Cv2.Circle(imageRes, CInt(p.X), CInt(p.Y), 8, Scalar.Azure, -1)
                        End If

                        points.Add(CType(p, Point))
                    Next

                    WriteTextSafe($"Joints {nPoints} found")
                    Dim nPairs = 14

                    For n = 0 To nPairs - 1
                        Dim partA = points(posePairs(n)(0))
                        Dim partB = points(posePairs(n)(1))
                        If partA.X <= 0 OrElse partA.Y <= 0 OrElse partB.X <= 0 OrElse partB.Y <= 0 Then Continue For
                        Cv2.Line(imageRes, partA, partB, New Scalar(0, 255, 255), 8)
                        Cv2.Circle(imageRes, partA.X, partA.Y, 8, New Scalar(0, 0, 255), -1)
                        Cv2.Circle(imageRes, partB.X, partB.Y, 8, New Scalar(0, 0, 255), -1)
                    Next
                End If

                Dim diff = DateTime.Now - startTime
                Dim fpsInfo = $"FPS: Nan"

                If diff.Milliseconds > 0 Then
                    Dim fpsVal = 1.0 / diff.Milliseconds * 1000
                    fpsInfo = $"FPS: {fpsVal}"
                End If

                Cv2.PutText(imageRes, fpsInfo, New Point(10, 20), HersheyFonts.HersheyComplexSmall, 1, Scalar.White)
            End While
        End Sub

        Private Sub WriteTextSafe(ByVal v As String)
            Throw New NotImplementedException()
        End Sub

        Private Sub button1_Click(ByVal sender As Object, ByVal e As EventArgs)
            _run = True
        End Sub

        Private Sub button2_Click(ByVal sender As Object, ByVal e As EventArgs)
            _run = False
        End Sub
    End Class
'''End Namespace

Well I'm going to add riches and details, I have 2 problems in the code the first is with pointers the second is that I can't reference it to use it directly in the form I even created the objects and everything logically works but the error in the pointer doesn't allow me to do it with precision I created a fictitious pointer to run and it worked, but I need to fix the conversion and call in form load to work and the site doesn't say how to do it exactly so I did everything as I know and the site hinted, but without pointer I don't have current position then acobo having a drawn shape that does not move. I think I've tried everything to import this Gaskets code now I'm going to try to keep converting it to see if I can fix it in VB.NET

[LiwisJames]

Please don't send us off to other sites to see code that you could have posted right here.

That said, I've looked at the code and I don't see anything that you shouldn't be able to convert yourself. If you can write VB code then you should be able to read most C# if you actually try. How about you do what you can for yourself and then post the C# code, your VB code and an explanation of where you're stuck and then we can address specific issues instead of vague "I can't, you do it" sort of questions.

Regardless, if you want to convert C# code snippets to VB automatically then you should get Instant VB from Tangible Software Solutions. It will handle many conversions that online tools can't and the free version would be perfectly adequate for that snippet.

You're right, I made a post before I could describe my problem in the wait but I saw that I must always do with time I apologize below has the project.

Project Download Full:
https://ufile.io/v8l4k52b

[jmcilhinney]

There are no pointers there. These lines do not use pointers:

p.X *= (float)frameWidth / W;
p.Y *= (float)frameHeight / H;

That is simply a multiplication and an assignment, i.e. this:

x *= y;

is shorthand for this:

x = x * y;

VB provides the same shorthand operators to use a variable and assign the result back to that variable:

x *= y

The only difference in VB would be the removal of the semicolon and the syntax of the cast.

[LiwisJames]

There are no pointers there. These lines do not use pointers:

p.X *= (float)frameWidth / W;
p.Y *= (float)frameHeight / H;

That is simply a multiplication and an assignment, i.e. this:

x *= y;

is shorthand for this:

x = x * y;

VB provides the same shorthand operators to use a variable and assign the result back to that variable:

x *= y

The only difference in VB would be the removal of the semicolon and the syntax of the cast.

var partA = points[posePairs[n][0]];
var partB = points[posePairs[n][1]];

In case the error and pointers are on this line, it says Severity Code Description Project File Line Status of Suppression
Error CS0021 Cannot apply indexing with [] to an expression of type "object" ShadowMode C:\Users\liwis\source\repos\ShadowMode\ShadowMode\Form1.cs 136 Active

And when I use another value for variable other than object it says it doesn't refer to a pointer


Every kind of conversion or error declaration I've tried to use even the references from opencv MAT calc etc... but it won't

[jmcilhinney]

You keep saying "pointer". Do you actually mean "point", as in an (x,y) location in 2D or (x,y,z) location in 3D? A pointer is something else.

[jmcilhinney]

var partA = points[posePairs[n][0]];
var partB = points[posePairs[n][1]];

In case the error and pointers are on this line, it says Severity Code Description Project File Line Status of Suppression
Error CS0021 Cannot apply indexing with [] to an expression of type "object" ShadowMode C:\Users\liwis\source\repos\ShadowMode\ShadowMode\Form1.cs 136 Active

And when I use another value for variable other than object it says it doesn't refer to a pointer


Every kind of conversion or error declaration I've tried to use even the references from opencv MAT calc etc... but it won't

The VB equivalent of that code snippet would be this:

Dim partA = points(posePairs(n)(0))
Dim partB = points(posePairs(n)(1))

You just need to use the same logic and problem-solving skills that you would use outside programming. Don't just look at some code, say "I don't understand" and give up. points is a List<Point> (List(Of Point) in VB) so you obviously need an Integer to index it, so posePairs(n)(0) and posePairs(n)(1) must be Integers, so posePairs must be a jagged array of Integer, i.e. Integer()(). Given the name, each inner array contains a pair of values, which are presumably the X and Y coordinates of the points representing the pose.

[LiwisJames]

The VB equivalent of that code snippet would be this:

Dim partA = points(posePairs(n)(0))
Dim partB = points(posePairs(n)(1))

You just need to use the same logic and problem-solving skills that you would use outside programming. Don't just loo at some code, say "I don't understand" and give up. points is a List<Point> (List(Of Point) in VB) so you obviously need an Integer to index it, so posePairs(n)(0) and posePairs(n)(1) must be Integers, so posePairs must be a jagged array of Integer, i.e. Integer()(). Given the name, each inner array contains a pair of values, which are presumably the X and Y coordinates of the points representing the pose.

Thank you , is very good you're right.