OpenCV Camera calibration and SolvePNP translate results - opencv

Question

I am attempting to initially calibrate a sensor using a chessboard.
I make around 50 run, and after I calibrate the camera I proceed in using solvepnp to teach the coordinate system, and since I am using a well defined chessboard to actually learn the real world coordinates.
As input for the SolvePnP I use all corner values with their corresponding real world distances.
My biggest issue is that the translate matrix I calculate from the SolvePvP is a bit strange. It is my understanding that the translate matrix is the actual offset between the camera and the coordinate system, which I define as the the upper part of the chessboard. But I get completely random values, with the Tz gives me a value of around 6000, even though the distance between camera and chessboard is around 1600 mm, and I am not even using depth values as parameters for the solvePnP method anyway.
Any input on what could be off?
Code Sample:
50 x DrawChessboardCorners
Corner result: {X = 1170.45984 Y = 793.002}
{X = 1127.80371 Y = 792.54425}
3d Points: {X = 175 Y = 70, Z = 0}
{X = 140 Y = 70, Z = 0}
total 18 (6x3) results per run for a total of 50 runs.
Afterwards I calibrate the camera:
CalibrateCamera(_objectsList, _pointsList,
new Size(_sensorWrapper.Width, _sensorWrapper.Height),
cameraMatrix, distCoeffs,
CalibType.Default,
new MCvTermCriteria(30, 0.1), out _, out _);
Afterwards, using the cameraMatrix and distCoeffs I use SolverPnP by using the top left, top right, bottom left bottom right corners, with their real world coordinates.
The results i get from the calibration are :
{
"CameraMatrix": [
[
5969.947,
0.0,
959.687256
],
[
0.0,
6809.737,
540.3694
],
[
0.0,
0.0,
1.0
]
],
"DistortionCoefficientsMatrix": [
[
0.141516522,
285.377747,
0.008248664,
0.0280253552,
1.5376302
]
],
"RotationMatrix": [
[
0.9992069,
-0.0270648878,
0.0292078461
],
[
-0.0003847139,
0.726907134,
0.68673563
],
[
-0.0398178138,
-0.6862022,
0.7263202
]
],
"TranslationMatrix": [
[
22.5370159
],
[
-362.535675
],
[
5448.58057
]
],
"SensorName": "BRIO 4K Stream Edition",
"SensorIndex": 0,
"Error": 0.18790249992423425
}