OpenCV - Determine position of wrist - opencv

Question

I need to determine the position of the wrist in a frame with parts of a human under arm & matching hand.
So far I have isolated the hand & arm and I'm able to draw a polygon & hull curve around it:
I achieve this result by simple binary thresholding and automatic contour fitting.
Based on this I want to extract the location of the wrist. This needs to work for all orientations of the hand/wrist.
However, being fairly new to working with OpenCV it is unclear to me what the best way is to determine/isolate the location of the wrist. I have various ideas for this:
The arm section is a fairly straight. Maybe a simple line detection over the contour polygon might do the job to get straight lines for the under arm.
Somehow split the contour polygon into multiple sections. Basically it's fair to assume that the location of the wrist has the smallest distance between the two arms contouring the under arm. Is there a way to find that point along the polygon and then "cut" or "split" the polygon to get two? From there I'd have one polygon representing a rectangle which should be easy to work with.
Use an approach that iterates along the main axis of the polygon fitted using fitLine(), measuring the distance between two opposing points of the polygon, finding the shortest distance.
Unfortunately I lack the experience to make the correct choice here - or even come up with a better idea.
I'd appreciate any kind of ideas & pointers towards achieving this. I could find a lot of valuable research material when it comes to hand detection & tracking and basic body part matching using Haar cascades. Unfortunately, I couldn't find a way to apply those technologies for my use case.
Here's some raw material (images & videos) to work with: (Google Drive Link!): https://drive.google.com/drive/folders/1hU4hGw5dYtVrcXTq8TYWCWfcLWjT-ZJU?usp=sharing

Answer 1

Approach: I used the advantage of arm side. The thickness of arm is almost same until hitting the hand.
Assumption: I coded by assuming the arm will enter the screen vertically. Otherwise my code may not work. I tried all of the images you shared and its working properly for all.
My steps:
Make a simple segmentation methodology for getting only needed part of the image
Start to count the non-black pixel for each column by beginning from the arm side.
Until hitting a column which is different from the previous column countings, you are still on the arm side. When you hit, you reached the wrist.
Note: I decided the threshold experimentally.
Here are the results and code:
Input Image:
After segmentation:
Output after algorithm:
Code:
#include "opencv2/highgui/highgui.hpp"
#include "opencv2/imgproc/imgproc.hpp"
#include <iostream>
#include <cstdlib>
using namespace cv;
using namespace std;
int main() {
Mat src, gray, blur_image, threshold_output;
// take input image
src = imread("/ur/image/directory/image_01.jpg", 1);
// convert to grayscale
cvtColor(src, gray, COLOR_BGR2GRAY);
// add blurring to the input image
medianBlur(gray,gray,9);
// Apply a segmentation to arm
for(int i=0; i<gray.rows; i++)
for(int j=0;j<gray.cols; j++)
if(gray.at<uchar>(Point(j,i))<110)
gray.at<uchar>(Point(j,i)) = 0;
//Creat a bgr mat to show the results clearly
Mat copy_gray = gray;
cvtColor(copy_gray,copy_gray,CV_GRAY2BGR);
double sum = 0;
int loop_cnt = 0,enter = 1;
Point first,second;
for(int j=gray.cols-1; j>=0; j--)
{
loop_cnt++;
int counter = 0,ff=1,enter2 = 1;
for(int i=0;i<gray.rows; i++)
{
if(gray.at<uchar>(Point(j,i))!=0 && enter)
{
if(ff)
first = Point(j,i);
counter++;
ff = 0;
}
if(!ff && gray.at<uchar>(Point(j,i))==0 && enter2)
{
second = Point(j,i);
enter2 = 0;
}
}
sum += (double)counter;
double average = sum/(double)(loop_cnt);
if(abs(average-counter)>20.0 && enter)
{
line(copy_gray,Point(j,0),Point(j,500),Scalar(0,255,0),5);
enter = 0;
}
}
int distance = norm(second-first)/2;
circle(copy_gray,Point(first.x,first.y+distance),20,Scalar(0,0,255),5);
imshow("Result",copy_gray);
waitKey(0);
return 0;
}