Mahout content-based similarity - mahout

I have created a custom item similarity that simulates content-based similarity based on a product taxonomy. I have a user who likes only two items:
UserId ItemId Preference
7656361 1449133 1.00
7656361 18886199 8.00
My custom itemSimilarity returns values from [-1,1] where 1 should mean strong similarity, and -1 strong dissimilarity. The two items the user liked does not have any lowest common ancestors in the taxonomy tree, so they don't have value of 1. But they have values from 0, 0.20 and 0.25 with some items.
I produce recommendation in the following way:
ItemSimilarity similarity = new CustomItemSimilarity(...);
Recommender recommender = new GenericItemBasedRecommender(model, similarity);
List<RecommendedItem> recommendations = recommender.recommend(7656361, 10);
for (RecommendedItem recommendation : recommendations) {
System.out.println(recommendation);
}
I am getting the following result:
RecommendedItem[item:899604, value:4.5]
RecommendedItem[item:1449081, value:4.5]
RecommendedItem[item:1449274, value:4.5]
RecommendedItem[item:1449259, value:4.5]
RecommendedItem[item:715796, value:4.5]
RecommendedItem[item:3255539, value:4.5]
RecommendedItem[item:333440, value:4.5]
RecommendedItem[item:1450204, value:4.5]
RecommendedItem[item:1209464, value:4.5]
RecommendedItem[item:1448829, value:4.5]
Which at first glance someone will say, ok it produce recommendations. I tried to print the values from the itemSimilarity as it does the comparison between pairwise items, and I got this supprising result:
ItemID1 ItemID2 Similarity
899604 1449133 -1.0
899604 18886199 -1.0
1449081 1449133 -1.0
1449081 18886199 -1.0
1449274 1449133 -1.0
1449274 18886199 -1.0
1449259 1449133 -1.0
1449259 18886199 -1.0
715796 1449133 -1.0
715796 18886199 -1.0
3255539 1449133 -1.0
3255539 18886199 -1.0
333440 1449133 -1.0
333440 18886199 -1.0
1450204 1449133 -1.0
1450204 18886199 -1.0
1209464 1449133 -1.0
1209464 18886199 -1.0
1448829 1449133 -1.0
1448829 18886199 -1.0
228964 1449133 -1.0
228964 18886199 0.25
57648 1449133 -1.0
57648 18886199 0.0
899573 1449133 -1.0
899573 18886199 0.2
950062 1449133 -1.0
950062 18886199 0.25
5554642 1449133 -1.0
5554642 18886199 0.0
...
and there are few more. They are not in the produce order. I just wanted to make a point. All the items that have very strong dissimilarity of -1 are recommended, and those that have some similarity of 0.0, 0.2 and 0.25 are not recommended at all. How is this possible?
The itemSimilarity method of the interface ItemSimilarity have the following explenation:
Implementations of this interface define a notion of similarity
between two items. Implementations should return values in the range
-1.0 to 1.0, with 1.0 representing perfect similarity.
If I use similarity between [0,1] I get the following recommendations:
RecommendedItem[item:228964, value:8.0]
RecommendedItem[item:899573, value:8.0]
RecommendedItem[item:950062, value:8.0]
And pairwise similarity is as follows (only for those tree, for the others is 0):
228964 1449133 0.0
228964 18886199 0.25
950062 1449133 0.0
950062 18886199 0.25
228964 1449133 0.0
228964 18886199 0.25
EDIT: I also printed out the most similar items to 1449133, 18886199 with: (GenericItemBasedRecommender)delegate).mostSimilarItems(new long[]{1449133, 18886199}, 10)
and I got: [RecommendedItem[item:228964, value:0.125], RecommendedItem[item:950062, value:0.125], RecommendedItem[item:899573, value:0.1]]
Only for item 18886199, (GenericItemBasedRecommender)delegate).mostSimilarItems(new long[]{18886199}, 10) I got [RecommendedItem[item:228964, value:0.25]]. For 1449133 only there are no similar items.
I don't understand why it does not work with strong dissimilarity?
Another question is why all the predicted preference values are 8.0 or 4.5. I can see that only the item 18886199 is similar with the the recommended items, but is there a way to multiply the value of 8.0 with the similarity in the case 0.25, and get value of 2.0 instead of 8.0. This I can't do while computing the similarity because I don't know the user yet, but I think it should be done during the recommendation phase. Isn't this how the recommender should work or maybe I should create a custom recommender and do the job in a custom way?
I would really appreciate if someone from the Mahout community can give me directions.

Related

MDLMesh strange buffer layout

I was curious to see the vertex positions of an MDLMesh for a boxWithExtent and I have notices a strange behaviour. When I print out the positions and normals for the first 3 vertices you get the below values. What I find weird is that for the 1st vertex we get 2 0.0s at the end to fit the stride for sims_float4 but weirdly for every subsequent vertex this becomes 1.0 - 0.0. Anyone has any idea why metal does this instead of just filling the last two positions with 0.0s as the first vertex. Thank you 0.5 0.5 0.5 -1.0 -0.0 -0.0 0.0 0.0 0.5 0.5 -0.5 -1.0 -0.0 -0.0 1.0 0.0 0.5 -0.5 0.5 -1.0 -0.0 -0.0 0.0 1.0
This behaviour goes away if I use a custom vertexDescriptor so I'm confused as what's going on here

How to imwrite actual GaussianBlur in OpenCV?

I use the following code to take an original image and blur it:
#include <opencv2/core/core.hpp>
#include <opencv2/imgproc/imgproc.hpp>
#include <opencv2/highgui/highgui.hpp>
int main(int argc, char** argv) {
cv::Mat img = cv::imread("lenna_orig.png");
cv::Mat gray, blurred;
cv::cvtColor(img, gray, CV_BGR2GRAY);
cv::GaussianBlur(gray, blurred, cv::Size(21, 21), 2.0);
cv::imwrite("lenna_blur.png", blurred);
return 0;
}
But is there a way to save the actual image of the Gaussian blur? something like this?
cv::imwrite("gauss.png", cv::GaussianBlur(cv::Size(21, 21), 2.0));
I ask because I eventually want to do a deconvolution problem and compared the computed kernel with the actual Gaussian kernel, so I need to know what the actual Gaussian kernel looks like exactly
EDIT:
I see that if I try
cv::Mat g = cv::getGaussianKernel(15, 2.0, CV_64F);
cv::imshow("g", g);
cv::imwrite("g.bmp", g);
this won't work because this returns a 15x1 matrix as the kernel, according to the docs. But I want a 15x15 kernel
cv::getGaussianKernel returns a 1d Gaussian profile. Since the kernel is symmetric it only needs to calculate a 1d curve.
If you want a 2d version you could stack 15 rows of the 1d ones and then multiply each column by the same profile.
edit: eg. Suppose the Gaussian kernel was 0.2, 0.4, 1.0, 0.4, 0.2 (simplified version for less typing)
Create the square array, with each row equal to the profile.
0.2 0.4 1.0 0.4 0.2
0.2 0.4 1.0 0.4 0.2
0.2 0.4 1.0 0.4 0.2
0.2 0.4 1.0 0.4 0.2
0.2 0.4 1.0 0.4 0.2
Now multiply each column by the same profile
0.2
0.4
1.0
0.4
0.2
To get something like
0.04 0.08 0.2 0.08 0.04
0.08 0.16 0.4 0.16 0.08
0.2 0.4 1.0 0.4 0.2
0.08 0.16 0.4 0.16 0.08
0.04 0.1 0.2 0.08 0.04
Only with the actual Gaussian profile and a 15x15 result.
ps this demonstrates an important feature of these kernels - they are separable. That means you can apply them in the x and y directions independently and then combine the result which makes it a lot more efficient to use.

What does bound values between 0.0 and 1.0 mean?

From the documentation for AVMetadataObject I read:
For video content, the bounding rectangle may be expressed using
scalar values in the range 0.0 to 1.0. Scalar values remain meaningful
even when the original video has been scaled down.
What does that mean?
I'll give you a basic example. Let's say we have two views A and B
A = {0.0, 0.0, 320.0, 568.0}
B = {100.0, 100.0, 100.0, 100.0}
So now we can translate in to our new coordinate system where
A = {0.0, 0.0, 1.0, 1.0}
Let's do some basic calculation for b
the point x : 320 == 1 like 100 == x so 100 / 320 = x = 0.3125
the point y : 568 == 1 like 100 == y so 100 / 568 = y = 0.1760
Do the same calculation for width and height and you will have your new frame translated into the new coordinate system, and obviously you can do the opposite calculation to translate back to your system of coordinates.

Get scale and rotation angle from CGAffineTransform?

I want to get scale factor and rotation angle form view. I've already applied CGAffineTransform to that view.
The current transformation of an UIView is stored in its transform property. This is a CGAffineTransform structure, you can read more about that here: https://developer.apple.com/library/ios/documentation/GraphicsImaging/Reference/CGAffineTransform/Reference/reference.html
You can get the angle in radians from the transform like this:
CGFloat angle = atan2f(yourView.transform.b, yourView.transform.a);
If you want the angle in degrees you need to convert it like this:
angle = angle * (180 / M_PI);
Get the scale like this:
CGFloat scaleX = view.transform.a;
CGFloat scaleY = view.transform.d;
I had the same problem, found this solution, but it only partially solved my problem.
In fact the proposed solution for extracting the scale from the transform:
(all code in swift)
scaleX = view.transform.a
scaleY = view.transform.d
only works when the rotation is 0.
When the rotation is not 0 the transform.a and transform.d are influenced by the rotation. To get the proper values you can use
scaleX = sqrt(pow(transform.a, 2) + pow(transform.c, 2))
scaleY = sqrt(pow(transform.b, 2) + pow(transform.d, 2))
note that the result is always positive. If you are also interested in the sign of the scaling (the view is flipped), then the sign of the scaling is the sign of transform.a for x flip and transform.d for y flip. One way to inherit the sign.
scaleX = (transform.a/abs(transform.a)) * sqrt(pow(transform.a, 2) + pow(transform.c, 2))
scaleY = (transform.d/abs(transform.d)) * sqrt(pow(transform.b, 2) + pow(transform.d, 2))
In Swift 3:
let rotation = atan2(view.transform.b, view.transform.a)

Calculate distance (disparity) OpenCV

-- Update 2 --
The following article is really useful (although it is using Python instead of C++) if you are using a single camera to calculate the distance: Find distance from camera to object/marker using Python and OpenCV
Best link is Stereo Webcam Depth Detection. The implementation of this open source project is really clear.
Below is the original question.
For my project I am using two camera's (stereo vision) to track objects and to calculate the distance. I calibrated them with the sample code of OpenCV and generated a disparity map.
I already implemented a method to track objects based on color (this generates a threshold image).
My question: How can I calculate the distance to the tracked colored objects using the disparity map/ matrix?
Below you can find a code snippet that gets the x,y and z coordinates of each pixel. The question: Is Point.z in cm, pixels, mm?
Can I get the distance to the tracked object with this code?
Thank you in advance!
cvReprojectImageTo3D(disparity, Image3D, _Q);
vector<CvPoint3D32f> PointArray;
CvPoint3D32f Point;
for (int y = 0; y < Image3D->rows; y++) {
float *data = (float *)(Image3D->data.ptr + y * Image3D->step);
for (int x = 0; x < Image3D->cols * 3; x = x + 3)
{
Point.x = data[x];
Point.y = data[x+1];
Point.z = data[x+2];
PointArray.push_back(Point);
//Depth > 10
if(Point.z > 10)
{
printf("%f %f %f", Point.x, Point.y, Point.z);
}
}
}
cvReleaseMat(&Image3D);
--Update 1--
For example I generated this thresholded image (of the left camera). I almost have the same of the right camera.
Besides the above threshold image, the application generates a disparity map. How can I get the Z-coordinates of the pixels of the hand in the disparity map?
I actually want to get all the Z-coordinates of the pixels of the hand to calculate the average Z-value (distance) (using the disparity map).
See this links: OpenCV: How-to calculate distance between camera and object using image?, Finding distance from camera to object of known size, http://answers.opencv.org/question/5188/measure-distance-from-detected-object-using-opencv/
If it won't solve you problem, write more details - why it isn't working, etc.
The math for converting disparity (in pixels or image width percentage) to actual distance is pretty well documented (and not very difficult) but I'll document it here as well.
Below is an example given a disparity image (in pixels) and an input image width of 2K (2048 pixels across) image:
Convergence Distance is determined by the rotation between camera lenses. In this example it will be 5 meters. Convergence distance of 5 (meters) means that the disparity of objects 5 meters away is 0.
CD = 5 (meters)
Inverse of convergence distance is: 1 / CD
IZ = 1/5 = 0.2M
Size of camera's sensor in meters
SS = 0.035 (meters) //35mm camera sensor
The width of a pixel on the sensor in meters
PW = SS/image resolution = 0.035 / 2048(image width) = 0.00001708984
The focal length of your cameras in meters
FL = 0.07 //70mm lens
InterAxial distance: The distance from the center of left lens to the center of right lens
IA = 0.0025 //2.5mm
The combination of the physical parameters of your camera rig
A = FL * IA / PW
Camera Adjusted disparity: (For left view only, right view would use positive [disparity value])
AD = 2 * (-[disparity value] / A)
From here you can compute actual distance using the following equation:
realDistance = 1 / (IZ – AD)
This equation only works for "toe-in" camera systems, parallel camera rigs will use a slightly different equation to avoid infinity values, but I'll leave it at this for now. If you need the parallel stuff just let me know.
if len(puntos) == 2:
x1, y1, w1, h1 = puntos[0]
x2, y2, w2, h2 = puntos[1]
if x1 < x2:
distancia_pixeles = abs(x2 - (x1+w1))
distancia_cm = (distancia_pixeles*29.7)/720
cv2.putText(imagen_A4, "{:.2f} cm".format(distancia_cm), (x1+w1+distancia_pixeles//2, y1-30), 2, 0.8, (0,0,255), 1,
cv2.LINE_AA)
cv2.line(imagen_A4,(x1+w1,y1-20),(x2, y1-20),(0, 0, 255),2)
cv2.line(imagen_A4,(x1+w1,y1-30),(x1+w1, y1-10),(0, 0, 255),2)
cv2.line(imagen_A4,(x2,y1-30),(x2, y1-10),(0, 0, 255),2)
else:
distancia_pixeles = abs(x1 - (x2+w2))
distancia_cm = (distancia_pixeles*29.7)/720
cv2.putText(imagen_A4, "{:.2f} cm".format(distancia_cm), (x2+w2+distancia_pixeles//2, y2-30), 2, 0.8, (0,0,255), 1,
cv2.LINE_AA)
cv2.line(imagen_A4,(x2+w2,y2-20),(x1, y2-20),(0, 0, 255),2)
cv2.line(imagen_A4,(x2+w2,y2-30),(x2+w2, y2-10),(0, 0, 255),2)
cv2.line(imagen_A4,(x1,y2-30),(x1, y2-10),(0, 0, 255),2)
cv2.imshow('imagen_A4',imagen_A4)
cv2.imshow('frame',frame)
k = cv2.waitKey(1) & 0xFF
if k == 27:
break
cap.release()
cv2.destroyAllWindows()
I think this is a good way to measure the distance between two objects

Resources