Please look at this github page. I want to generate heat maps in this way using Python PIL,open cv or matplotlib library. Can somebody help me figure it out?
I could create a heat map for my network at the same size as the input, but I am not able superimpose them. The heatmap shape is (800,800) and the base image shape is (800,800,3)
Updated Answer -- 29th April, 2022.
After the repeated comments I have decided to update this post with a better visualization.
Consider the following image:
img = cv2.imread('image_path')
I obtained a binary image after performing binary threshold on the a-channel of the LAB converted image:
lab = cv2.cvtColor(img, cv2.COLOR_BGR2LAB)
a_component = lab[:,:,1]
th = cv2.threshold(a_component,140,255,cv2.THRESH_BINARY)[1]
Applying Gaussian blur:
blur = cv2.GaussianBlur(th,(13,13), 11)
The resulting heatmap:
heatmap_img = cv2.applyColorMap(blur, cv2.COLORMAP_JET)
Finally, superimposing the heatmap over the original image:
super_imposed_img = cv2.addWeighted(heatmap_img, 0.5, img, 0.5, 0)
Note: You can vary the weight parameters in the function cv2.addWeighted and observe the differences.
My code starts from a heatmap matrix (224,224) called cam, which is applied to the original image called frame, via opencv;
and it seems to work pretty well:
import numpy as np
from cv2 import cv2
from skimage import exposure
...
capture = cv2.VideoCapture(...)
while True:
ret, frame = capture.read()
if ret:
#resize original frame
frame = cv2.resize(frame, (224, 224))
#get color map
cam = getMap(frame)
map_img = exposure.rescale_intensity(cam, out_range=(0, 255))
map_img = np.uint8(map_img)
heatmap_img = cv2.applyColorMap(map_img, cv2.COLORMAP_JET)
#merge map and frame
fin = cv2.addWeighted(heatmap_img, 0.5, frame, 0.5, 0)
#show result
cv2.imshow('frame', fin)
the getMap() function gets the headmap given the frame;
I found some interesting free videos about this topic:
https://www.youtube.com/watch?v=vTY58-51XZA&t=14s
https://www.youtube.com/watch?v=4v9usdvGU50&t=208s
I had some problems with grayscale images at this line
super_imposed_img = cv2.addWeighted(heatmap_img, 0.5, img, 0.5, 0)
but this one worked for me
plt.imshow(binary_classification_result * 0.99 + original_gray_image * 0.01)
Related
Greeting,
I have been trying to extract some regions from the face
In this case (upper lip) using Dlib, the thing is after extracting the ROI (which look perfect) I realized that there is some noise around the ROI
Can't figure out what I'm doing wrong, and how to resolve this issue.
This is the used Python code:
import cv2
import numpy as np
import dlib
import os
from scipy import ndimage, misc
import time
def extract_index_nparray(nparray):
index = None
for num in nparray[0]:
index = num
break
return index
img = cv2.imread( 'input_facial_image.jpg')
img=cv2.resize(img,(512,512))
img_gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
mask = np.zeros_like(img_gray)
detector = dlib.get_frontal_face_detector()
predictor = dlib.shape_predictor("/facial-landmarks-recognition/shape_predictor_68_face_landmarks.dat")
# Face 1
faces = detector(img_gray)
for face in faces:
landmarks = predictor(img_gray, face)
landmarks_points = []
for n in [48,49,50,51,52,53,54,64,63,62,61,60]:
x = landmarks.part(n).x
y = landmarks.part(n).y
landmarks_points.append((x, y))
points = np.array(landmarks_points, np.int32)
convexhull = cv2.convexHull(points)
# cv2.polylines(img, [convexhull], True, (255, 0, 0), 3)
cv2.fillConvexPoly(mask, convexhull, 255)
face_image_1 = cv2.bitwise_or(img, img, mask=mask)
cv2.imwrite('extracted_lips.jpg', face_image_1 )
The extracted image looks like this :
upper lips extracted image
But in further steps in my work, I realized a noise around the upper lip, so I examined and I found unclean_upperlip
Is there any way to get rid of the noise during the ROI extracting or any image processing technique to bypass this issue?
Thanks in advance
For anyone who faced the same issue as me, it's simple. Just change the output format to png. The JPG compressing is the issue here.
I hope that this helps
I'm trying to add faint white diagonal lines (similar to these lines except that they're diagonal) into images for a machine learning task. Does anyone know whether these transformations have a name and how I can replicate them on images (in python/pytorch preferably)?
Check out this fragment of code:
from PIL import Image
import numpy as np
import matplotlib.pyplot as plt
def add_lines(image, line_width = 5, line_intensity = 0.8, row_effect = 0.45):
s_shape = (image.height, image.width)
if row_effect == "aspect":
row_effect = image.width/image.height
lines = Image.fromarray(np.full(s_shape, 255, dtype=np.uint8), "L").convert("RGBA")
alpha = np.zeros(s_shape)
line_fun = lambda x : int(line_intensity*255/2*(np.sin(2*np.pi/line_width*x)+1))
for col_index in range(alpha.shape[1]):
for row_index in range(alpha.shape[0]):
alpha[row_index, col_index] = line_fun(col_index + row_effect*row_index)
alpha = Image.fromarray(np.uint8(alpha), "L")
new_image = image.copy().convert("RGBA")
new_image.paste(lines, (0,0), alpha)
return new_image
source = Image.open("source.jpg")
source_lines = add_lines(source, line_width=5, line_intensity=0.4, row_effect="aspect")
_, axs = plt.subplots(1, 2)
axs[0].imshow(source)
axs[1].imshow(source_lines)
The most important is of course the add_lines function.
To solve Your problem we proceed as follows:
Create a fully white grayscale image with the shape of the source image,
create an array of the same size as the source, but with all zeros,
define a sine function, rescaled so it is in range [0, line_intensity*255], and with period of line_width,
for each column in the alpha array fill it with the value of the sine function with the column index as a parameter, and some percentage of the row index as well, this percentage defines how much "diagonal" will the lines be. If You need a 45 degree angle of the lines, just set row_effect to 1, and if You need it to go exactly diagonally through the image, set it to the aspect ratio of the image.
convert both the source and the lines to RGBA, and the alpha mask to grayscale,
paste the lines image to the source, using alpha as the mask.
This relatively simple code produces the following result:
I have been working on a code where an image is given as shown
I have to place this knife onto some other image. The condition is that I have to crop the knife along its outline and not in a rectangular box.
import numpy as np
import cv2
from matplotlib import pyplot as plt
img = cv2.imread('2.jpg')
img = cv2.cvtColor(img, cv2.COLOR_BGR2RGB)
plt.imshow(img)
img_blur = cv2.bilateralFilter(img, d = 7,
sigmaSpace = 75, sigmaColor =75)
img_gray = cv2.cvtColor(img_blur, cv2.COLOR_RGB2GRAY)
a = img_gray.max()
_, thresh = cv2.threshold(img_gray, a/2+60, a,cv2.THRESH_BINARY_INV)
plt.imshow(thresh, cmap = 'gray')
contours, hierarchy = cv2.findContours(
image = thresh,
mode = cv2.RETR_TREE,
method = cv2.CHAIN_APPROX_SIMPLE)
contours = sorted(contours, key = cv2.contourArea, reverse = True)
img_copy = img.copy()
final = cv2.drawContours(img_copy, contours, contourIdx = -1,
color = (255, 0, 0), thickness = 2)
plt.imshow(img_copy)
This is what I have tried but it doesn't seem to work well.
Input
Output
You can do it starting with bounding box using snake algorithm with balloon force added.
Snake's algo is defined such that it minimizes 3 energies - Continuity, Curvature and Gradient. The first two (together called internal energy) get minimized when points (on curve) are pulled closer and closer i.e. contract. If they expand then energy increases which is not allowed by snake algorithm.
But this initial algo proposed in 1987 has a few problems. One of the problem is that in flat areas (where gradient is zero) algo fails to converge and does nothing. There are several modifications proposed to solve this problem. The solution of interest here is - Balloon Force proposed by LD Cohen in 1989.
Balloon force guides the contour in non-informative areas of the image, i.e., areas where the gradient of the image is too small to push the contour towards a border. A negative value will shrink the contour, while a positive value will expand the contour in these areas. Setting this to zero will disable the balloon force.
Another improvement is - Morphological Snakes which use morphological operators (such as dilation or erosion) over a binary array instead of solving PDEs over a floating point array, which is the standard approach for active contours. This makes Morphological Snakes faster and numerically more stable than their traditional counterpart.
Scikit-image's implementation using the above two improvements is morphological_geodesic_active_contour. It has a parameter balloon
Using your image
import numpy as np
import matplotlib.pyplot as plt
from skimage.segmentation import morphological_geodesic_active_contour, inverse_gaussian_gradient
from skimage.color import rgb2gray
from skimage.util import img_as_float
from PIL import Image, ImageDraw
im = Image.open('knife.jpg')
im = np.array(im)
im = rgb2gray(im)
im = img_as_float(im)
plt.imshow(im, cmap='gray')
Now let us create a function which will help us to store iterations:
def store_evolution_in(lst):
"""Returns a callback function to store the evolution of the level sets in
the given list.
"""
def _store(x):
lst.append(np.copy(x))
return _store
This method needs image to be preprocessed to highlight the contours. This can be done using the function inverse_gaussian_gradient, although the user might want to define their own version. The quality of the MorphGAC segmentation depends greatly on this preprocessing step.
gimage = inverse_gaussian_gradient(im)
Below we define our starting point - a bounding box.
init_ls = np.zeros(im.shape, dtype=np.int8)
init_ls[200:-400, 20:-30] = 1
List with intermediate results for plotting the evolution
evolution = []
callback = store_evolution_in(evolution)
Now required magic line for morphological_geodesic_active_contour with balloon contraction is below:
ls = morphological_geodesic_active_contour(gimage, 200, init_ls,
smoothing=1, balloon=-0.75,
threshold=0.7,
iter_callback=callback)
Now let us plot the results:
fig, axes = plt.subplots(1, 2, figsize=(8, 8))
ax = axes.flatten()
ax[0].imshow(im, cmap="gray")
ax[0].set_axis_off()
ax[0].contour(ls, [0.5], colors='b')
ax[0].set_title("Morphological GAC segmentation", fontsize=12)
ax[1].imshow(ls, cmap="gray")
ax[1].set_axis_off()
contour = ax[1].contour(evolution[0], [0.5], colors='r')
contour.collections[0].set_label("Starting Contour")
contour = ax[1].contour(evolution[25], [0.5], colors='g')
contour.collections[0].set_label("Iteration 25")
contour = ax[1].contour(evolution[-1], [0.5], colors='b')
contour.collections[0].set_label("Last Iteration")
ax[1].legend(loc="upper right")
title = "Morphological GAC Curve evolution"
ax[1].set_title(title, fontsize=12)
plt.show()
With more balloon force you can get only the blade of knife as well.
ls = morphological_geodesic_active_contour(gimage, 100, init_ls,
smoothing=1, balloon=-1,
threshold=0.7,
iter_callback=callback)
Play with these parameters - smoothing, balloon, threshold to get your perfect curve
This is my first experience with image processing. In the jupiter notebook, using scipy , I am trying to convert a gray scale line art image into SVG vector representation. So far I was able to convert the gray scale image to binary (monochrome image) and use sobel filter in x and y axis to get the edges of the drawing. I am getting double lines as edges to account for the both sides of the lines (as shown in below picture and also the code i have used)
I want to replace these double lines with a single one. After that to detect the lines and curves in the drawing and convert them to svg lines and bezier curves. Searching online, i am getting a bit overwhelmed and confused about the proper way forward. It would be of great help if i can get some pointers about how to proceed from here. If possible i want to do this in scipy only and not with opencv.
Rather than simply using the existing scipy functions and algorithms, I also want to learn about the underlying theory so that i can use them efficiently. So please kindly share any helpful theoretical resources.
Thanks in advance
%matplotlib inline
import numpy as np
from scipy import ndimage as nd
import matplotlib.pyplot as plt
from skimage import io
def apply_gradient_threshold(d,thres):
d2 = np.copy(d)
d2[d2 == -thres] = thres
d2[d2 != thres] = 0
return d2
def plot_images(imgs, names):
fig, axes_list = plt.subplots(1, len(imgs), figsize=(20, 20))
for name,axes in zip(names, axes_list):
axes.set_title(name)
for img, axes in zip(imgs, axes_list):
axes.imshow(img, cmap='Greys_r')
plt.show()
img_file = <file_url>
img = plt.imread(img_file)
gray_img = io.imread(img_file, as_gray=True)
if(np.max(gray_img) > 1) :
gray_img = gray_img/255 #normalize
threshold = 0.2
binary = (gray_img > threshold)*1 # convert the grayscale image to binary (monochrome)
im = binary.astype('int32')
dx = nd.sobel(im,1)
dy = nd.sobel(im,0)
dx = apply_gradient_threshold(dx, 4)
dy = apply_gradient_threshold(dy, 4)
mag = np.hypot(dx,dy) #sqrt(dx^2 + dy^2)
mag *= 255.0/np.max(mag)
plot_images([binary, mag ], ['Binary - ' + str(threshold), 'Sobel Filter Result'])
Your image is virtually already made of edges. Use thinning, not an edge filter.
I have an image like this. I wan to use HoughLine detection but the image is too dark to recognize the line. Is there a way that can whiten the background and blacken the grid? Is there any algorithms in openCV or python that I can apply? Thank you
I try to dilate the image first, then medianBlur it, so I get the background. Use the original gray image to sub the background, I get the frontground ( that is the grids). Then do some other steps, I get the result like this.
The code is as follow:
#!/usr/bin/python3
# 2017.10.04 19:37:43 CST
filename = "data/paper.png"
img = cv2.imread(filename)
gray = cv2.cvtColor(img, cv2.COLOR_BGR2GRAY)
## do morph-dilate-op
kernel = cv2.getStructuringElement(cv2.MORPH_ELLIPSE, (5,5))
dilated = cv2.morphologyEx(gray, cv2.MORPH_DILATE, kernel)
diff1 = 255 - cv2.subtract(dilated, gray)
## do medianBlur
median = cv2.medianBlur(dilated, 15)
diff2 = 255 - cv2.subtract(median, gray)
## do normalize
normed = cv2.normalize(diff2,None, 0, 255, cv2.NORM_MINMAX )
## save the result
dst = np.hstack((gray, normed))
cv2.imwrite("result_paper1.png", dst)
res = np.hstack((gray,dilated, diff1, median, diff2, normed))
cv2.imwrite("result_paper2.png", res)
You should try a form of localized adaptive thresholding.
In OpenCV this is called cv2.adaptiveThreshold.
See here: http://docs.opencv.org/master/d7/d4d/tutorial_py_thresholding.html for a python example.
The code (from the source above):
import cv2
import numpy as np
from matplotlib import pyplot as plt
img = cv2.imread('sudoku.png',0)
img = cv2.medianBlur(img,5)
ret,th1 = cv2.threshold(img,127,255,cv2.THRESH_BINARY)
th2 = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_MEAN_C,\
cv2.THRESH_BINARY,11,2)
th3 = cv2.adaptiveThreshold(img,255,cv2.ADAPTIVE_THRESH_GAUSSIAN_C,\
cv2.THRESH_BINARY,11,2)
titles = ['Original Image', 'Global Thresholding (v = 127)',
'Adaptive Mean Thresholding', 'Adaptive Gaussian Thresholding']
images = [img, th1, th2, th3]
for i in range(4):
plt.subplot(2,2,i+1),plt.imshow(images[i],'gray')
plt.title(titles[i])
plt.xticks([]),plt.yticks([])
plt.show()
Your image has poor contrast and inconsistent light. You have to make some preprocessing (c++ code here):
cv::Mat img = cv::imread("E:\\Workspace\\KS\\excercise\\oBwBH.jpg", 0);
cv::Mat workingMat;
cv::GaussianBlur(img, workingMat, cv::Size(101, 101), 31, 31); //high blur to extract background light
img = img - 0.7*work; //adjust light level
cv::normalize(img, img, 0, 255, cv::NORM_MINMAX); \\use whole range
cv::medianBlur(img, img, 5); \\remove noise
cv::Canny(img, work, 100, 200); \\extract lines; you could do hough lines instead since it has canny inside.
Results (from left to right, canny has all lines, in preview there is some compression issue):