EmguCV and Unity3D Interoperation - opencv

I managed integrating EmguCV into Unity3D and wrote a little converter that has some little problems
Step 1 Converting Unity3D Texture to OpenCV Image
public static Image<Bgr, byte> UnityTextureToOpenCVImage(Texture2D tex){
return UnityTextureToOpenCVImage(tex.GetPixels32 (), tex.width, tex.height);
}
public static Image<Bgr, byte> UnityTextureToOpenCVImage(Color32[] data, int width, int height){
byte[,,] imageData = new byte[width, height, 3];
int index = 0;
for (int y = 0; y < height; y++) {
for (int x = 0; x < width; x++) {
imageData[x,y,0] = data[index].b;
imageData[x,y,1] = data[index].g;
imageData[x,y,2] = data[index].r;
index++;
}
}
Image<Bgr, byte> image = new Image<Bgr, byte>(imageData);
return image;
}
Step 2 Converting OpenCV Image back to Unity3D Texture
public static Texture2D OpenCVImageToUnityTexture(Image<Bgr, byte> openCVImage, GameObject check){
return OpenCVImageToUnityTexture(openCVImage.Data, openCVImage.Width, openCVImage.Height, check);
}
public static Texture2D OpenCVImageToUnityTexture(byte[,,] data, int width, int height, GameObject check){
Color32 [] imageData = new Color32[width*height];
int index = 0;
byte alpha = 255;
for (int y = 0; y < width; y++) {
for (int x = 0; x < height; x++) {
imageData[index] = new Color32((data[x,y,2]),
(data[x,y,1]),
(data[x,y,0]),
alpha);
check.SetActive(true);
index++;
}
}
Texture2D toReturn = new Texture2D(width, height, TextureFormat.RGBA32, false);
toReturn.SetPixels32(imageData);
toReturn.Apply ();
toReturn.wrapMode = TextureWrapMode.Clamp;
return toReturn;
}
Compiler throws no errors but some goes wrong all the time. See yourself: cats.
On the left side is the original image, on the right side is the converted one. As you can see there are more cats then it should be...
Has anyone any clues?
Also it is slow as hell because of iterating twice through all pixels. Is there any better solution?
EDIT
This is the code where i draw my GUITextures:
public GameObject catGO;
GUITexture guitex;
Texture catTex;
void Start () {
guitex = GetComponent<GUITexture> ();
catTex = catGO.GetComponent<GUITexture> ().texture;
Image<Bgr, byte> cvImage = EmguCVUnityInterop.UnityTextureToOpenCVImage((Texture2D)catTex);
Texture2D converted = EmguCVUnityInterop.OpenCVImageToUnityTexture(cvImage);
guitex.texture = converted;
}

First of all you wrote in 2 for loops check.SetActive(true) so its width*height times. Enabling GameObject is a costly operation.
Second is that iterating thru every pixel is another costly operation. For examle if u have image 2560x1600 you have over 4 milion iterations.
Try to change TextureWrapMode to Repeat (I know it sounds silly:] )

Related

How to distort an array of images - Processing

I am trying to distort a grid of images that are displayed randomly when the mouse is clicked.
I have the grid and the random when click.
I have accomplished the distorsion I want when I have only one image.
Now I have to merge both codes together, but I can't figure out how to do it when the PImage is an array.
Grid code:
PImage img[];
int nPics;
int w, h;
void loadImages(){
for (int i = 0; i < nPics; i++) {
img[i] = loadImage("img_"+ nf(int(random(0, 687)), 3) + ".jpg");
imageMode(CORNERS);
}
}
void setup() {
size(1500, 500);
nPics=27;
img = new PImage[nPics];
w=width/9;
h=height/3;
loadImages();
}
void mousePressed(){
loadImages();
}
void draw() {
background(0);
for (int i=0; i<nPics; i=i+3) {
int col = i/3;
for (int row=0; row<3; row++)
image(img[i+row], col*w, row*h, (col+1)*w, (row+1)*h);
}
}
Wave code:
PImage myImage;
float speed1 = .01;
float speed2 = 5;
void setup()
{
size(1500,500);
myImage = loadImage("img_001.jpg");
}
void draw()
{
float distortion = map(mouseY, 0,500, 0, 10);
for (int i = 0; i < myImage.width; ++i) {
copy(myImage, i,0,1,height, i, (int) (sin((millis()+i*speed2)*speed1)*distortion),1,height);
}
}
If you want to apply the distortion to the entire canvas, you can just use copy() without the image parameter. This causes it to just copy from what is currently on the canvas.
You can use a very similar for loop to the one that's in your wave code, just omit the myImage parameter and let i go from 0 to width instead of myImage.width. Then you put it at the end of your draw block in the grid code:
//global variables, include speed1 and speed2
void loadImages() {
//as in grid code
}
void setup() {
//as in grid code...
}
void draw() {
//everything from grid code...
float distortion = map(mouseY, 0,500, 0, 10);
for (int i = 0; i < width; ++i) {
copy(i,0,1,height, //the rest...);
}
}

base64 string to image returns only single image in C# MVC

I Am getting Images from Reporting service engine in terms of byte array.
byte[] imageBytes = Convert.FromBase64String(base64);//base64 is an byte array
I am converting this byte array to to image
MemoryStream ms1 = new MemoryStream(imageBytes, 0, imageBytes.Length);
ms1.Write(imageBytes, 0, imageBytes.Length);
System.Drawing.Image images = System.Drawing.Image.FromStream(ms1, true);**
But when I save it it stores only first image
images.Save(Server.MapPath("~/img.png"),System.Drawing.Imaging.ImageFormat.Png);
it must save two images.I am not able to detect base64 first image or second image
You're only saving a single image from that byte array. PNGs don't contain multiple images, so you would have to know where to "split" the base64 so you get two images. See the pseudocode below to get an idea of what I am saying:
int i = 1;
string split = "//**//";
// the string "//**//" was inserted so we know where to split the string to
// extract mutiple images. It can be any string value
foreach (var b64str in base64.Split(split))
{
byte[] imageBytes = Convert.FromBase64String(b64str);//base64 is an byte array
MemoryStream ms1 = new MemoryStream(imageBytes, 0, imageBytes.Length);
ms1.Write(imageBytes, 0, imageBytes.Length);
System.Drawing.Image images = System.Drawing.Image.FromStream(ms1, true);
images.Save(Server.MapPath("~/img" + i.ToString() + ".png"),System.Drawing.Imaging.ImageFormat.Png);
i++;
}
I solved issue from this URL How to show multi paged image in <img/> tag?
At bottom of page they provide solution working fine....
First- Get all the frames from the stream of image as list of Images
public List<Image> GetAllFrames(Stream sm)
{
List<Image> images = new List<Image>();
Bitmap bitmap = new Bitmap(sm);
int count = bitmap.GetFrameCount(FrameDimension.Page);
for (int idx = 0; idx < count; idx++)
{
bitmap.SelectActiveFrame(FrameDimension.Page, idx);
MemoryStream byteStream = new MemoryStream();
bitmap.Save(byteStream, ImageFormat.Tiff);
images.Add(Image.FromStream(byteStream));
}
return images;
}
Second - Combine all frames in to a single bitmap.
public Bitmap CombineAllFrames(List<Image> test)
{
int width = 0;
int height = 0;
Bitmap finalImage = null;
try
{
foreach (Bitmap bitMap in test)
{
height += bitMap.Height;
width = bitMap.Width > width ? bitMap.Width : width;
}
finalImage = new Bitmap(width, height);
using (System.Drawing.Graphics gc = Graphics.FromImage(finalImage))
{
gc.Clear(Color.White);
int offset = 0;
foreach (Bitmap bitmap in test)
{
gc.DrawImage(bitmap, new Rectangle(0, offset, bitmap.Width, bitmap.Height));
offset += bitmap.Width;
}
}
}
catch (Exception)
{
throw;
}
return finalImage;
}
This methods creates a bitmap which will append all the frames into single one vertically. If you want it horizontally make update it to
width += bitmap.Width;
height = bitmap.Height > height ? bitmap.Height : height;
g.DrawImage(image, new System.Drawing.Rectangle(offset, 0, image.Width, image.Height));
Third step - Now if you want a byte array for the created image call the below method
public byte[] GetBytesFromImage(Bitmap finalImage)
{
ImageConverter convertor = new ImageConverter();
return (byte[])convertor.ConvertTo(finalImage, typeof(byte[]));
}

Blackberry how to design the Screen like Profile Screen !! Code for Blackberry 7.0

i am trying to make the Profile screen for my Application and the Requirement is :
I have the Data with me in the Vector and Image in bitmap form.. Only problem is that i am Struggling to figure out how it can be achieved, How to make the Photo Section parallel to the multiple Fields in Right hand Side..
Please help me out..
Thanks in Advance..
You haven't specified everything about how this should work. You may want the profile picture to scale with the screen size. Some of the fields may be editable. I'm not sure if you want backgrounds or borders drawn around the 3 rows of Text/Number. So, I had to make some guesses. But, this should get you started, and help you understand what's needed to create a custom Manager subclass, which is one of the ways to solve this problem.
public class ProfileScreen extends MainScreen {
public ProfileScreen() {
super(MainScreen.NO_VERTICAL_SCROLL | MainScreen.NO_VERTICAL_SCROLLBAR);
add(new ProfileManager());
}
private class ProfileManager extends Manager {
private static final int PADDING = 20; // TODO: might make this depend on screen size!
private static final int ROW_PAD = PADDING / 2;
private static final int NUM_ROWS = 3;
private String _name = "Nate";
private String _place = "USA";
private String _phone = "1-206-123-4567";
private String _email = "nate#stackoverflow.com";
private Bitmap _photo;
private Vector _text;
private Vector _numbers;
private LabelField _nameField;
private LabelField _placeField;
private LabelField _phoneField;
private LabelField _emailField;
private BitmapField _photoField;
private LabelField[] _textFields;
private LabelField[] _numberFields;
private int[] _rowLocations;
public ProfileManager() {
super(Manager.NO_VERTICAL_SCROLL | Manager.NO_VERTICAL_SCROLLBAR);
// Create the data and the fields ... this would probably be more
// dynamic in your production code, so the profile could change.
_nameField = new LabelField(_name);
_placeField = new LabelField(_place);
_phoneField = new LabelField(_phone);
_emailField = new LabelField(_email);
_photo = Bitmap.getBitmapResource("avatar.png");
_photoField = new BitmapField(_photo);
_text = new Vector();
_textFields = new LabelField[NUM_ROWS];
for (int i = 0; i < NUM_ROWS; i++) {
_text.insertElementAt("Text" + (i + 1), i);
_textFields[i] = new LabelField(_text.elementAt(i),
Field.USE_ALL_WIDTH | DrawStyle.HCENTER);
}
_numbers = new Vector();
_numberFields = new LabelField[NUM_ROWS];
for (int i = 0; i < NUM_ROWS; i++) {
_numbers.insertElementAt("Number" + (i + 1), i);
_numberFields[i] = new LabelField(_numbers.elementAt(i),
Field.USE_ALL_WIDTH | DrawStyle.HCENTER);
}
// We will store the bottom 3 row locations for use in paint()
_rowLocations = new int[NUM_ROWS];
// Add the fields to this manager
add(_photoField);
add(_nameField);
add(_placeField);
add(_phoneField);
add(_emailField);
for (int i = 0; i < NUM_ROWS; i++) {
// Add one row of Text and Number fields
add(_textFields[i]);
add(_numberFields[i]);
}
}
public int getPreferredHeight() {
return Display.getHeight(); // full screen
}
public int getPreferredWidth() {
return Display.getWidth(); // full screen
}
protected void sublayout(int width, int height) {
setExtent(width, height);
int x = PADDING;
int y = PADDING;
setPositionChild(_photoField, x, y);
layoutChild(_photoField, _photo.getWidth(), _photo.getHeight());
x += _photo.getWidth() + PADDING;
int widthMinusPhoto = width - 3 * PADDING - _photo.getWidth();
setPositionChild(_nameField, x, y);
layoutChild(_nameField, widthMinusPhoto, _nameField.getPreferredHeight());
y += _nameField.getHeight() + ROW_PAD;
setPositionChild(_placeField, x, y);
layoutChild(_placeField, widthMinusPhoto, _placeField.getPreferredHeight());
y += _placeField.getHeight() + ROW_PAD;
setPositionChild(_phoneField, x, y);
layoutChild(_phoneField, widthMinusPhoto, _phoneField.getPreferredHeight());
y += _phoneField.getHeight() + ROW_PAD;
setPositionChild(_emailField, x, y);
layoutChild(_emailField, widthMinusPhoto, _emailField.getPreferredHeight());
// layout the 3 rows of Text and Numbers (1/3 width for each label field (?)
x = PADDING;
y = PADDING + _photo.getHeight() + PADDING + ROW_PAD;
for (int i = 0; i < NUM_ROWS; i++) {
setPositionChild(_textFields[i], x, y);
// record the y coordinate of this row, to use in paint()
_rowLocations[i] = y;
layoutChild(_textFields[i],
width / 3, _textFields[i].getPreferredHeight());
setPositionChild(_numberFields[i], width - PADDING - width / 3, y);
layoutChild(_numberFields[i],
width / 3, _numberFields[i].getPreferredHeight());
y += _textFields[i].getPreferredHeight() + PADDING + 2 * ROW_PAD;
}
}
// paint overridden to draw gray box behind Text/Number rows
protected void paint(Graphics graphics) {
int oldColor = graphics.getColor();
// paint a light gray background behind each row of Text/Numbers
graphics.setColor(Color.LIGHTGRAY);
for (int i = 0; i < NUM_ROWS; i++) {
// if you want a solid border, use drawRect() instead of fillRect():
graphics.fillRect(PADDING, _rowLocations[i] - ROW_PAD,
getWidth() - 2 * PADDING, _textFields[i].getHeight() + 2 * ROW_PAD);
}
graphics.setColor(oldColor); // reset the color for super.paint()
super.paint(graphics);
}
}
}
It's not always required to override paint() in a custom Manager class. sublayout() is where most of the work is done, to position the fields. In this case, I chose to override paint() in order to provide a custom border around more than one field. This isn't the only way to do that, but there are definitely times when you'll want to be able to add custom drawing code, to improve the look of your UIs.
Results

XNA 4.0 3D Collision

I've been trying to work on a 3d forward-runner game (like temple run) in XNA 4.0.
I'm hitting a bit of a brick wall, so any help would be much appreciated!
Currently, I'm using my own method for collision detection, which requires the dimensions for each model to be hard-coded into the collision method. I've tried using the code from Microsoft, directly below, but it always returns false:
static bool CheckForCollisions(Entity c1, Entity c2)
{
for (int i = 0; i < c1.body.Meshes.Count; i++)
{
// Check whether the bounding boxes of the two cubes intersect.
BoundingSphere c1BoundingSphere = c1.body.Meshes[i].BoundingSphere;
c1BoundingSphere.Center += c1.position;
for (int j = 0; j < c2.body.Meshes.Count; j++)
{
BoundingSphere c2BoundingSphere = c2.body.Meshes[j].BoundingSphere;
c2BoundingSphere.Center += c2.position;
if (c1BoundingSphere.Intersects(c2BoundingSphere))
{
return true;
}
}
}
return false;
}
This was taken and modified very slightly from, Here My Entity class goes as follows.
Code of mine which I think is relevant would be:
public class Entity
{
public int rowID;
public Model body;
public Vector3 position;
public float rotation = 0f;
public float rotatePerFrame = 0f;
protected internal float toRight = 0;
protected internal float toLeft = 0;
protected internal float forward = 0;
protected internal float back = 0;
protected internal float bottom = 0;
protected internal float top = 0;
public void setDimensions(float right, float left, float front, float back, float top, float bottom)
{
this.toRight = right;
this.toLeft = left;
this.forward = front;
this.back = back;
this.top = top;
this.bottom = bottom;
}
public Entity RotateEnt(Entity e,float degrees)//Psuedo-only accurate to 90 degrees.
{
float actual = MathHelper.ToDegrees(degrees);
switch ((int)actual)
{
case 0:
break;
case 90:
// float temp = e.forward;
// e.forward = e.toLeft;
// e.toLeft =e.back ;
// e.back = e.toRight;
// e.toRight = temp;
float temp = e.forward;
e.forward = e.toRight;
e.toRight = e.back;
e.back = e.toLeft;
e.toLeft = temp;
break;
case 180:
e.forward = e.back;
e.back = e.forward;
e.toRight = e.toLeft;
e.toLeft = e.toRight;
break;
default: //case: 270
e.toRight = e.forward;
e.back = e.toRight;
e.toLeft = e.back;
e.forward = e.toLeft;
break;
}
return e;
}
public bool Collides(Entity e)
{
Entity c1 = RotateEnt(this, this.rotation);
Entity c2 = RotateEnt(e, e.rotation);
float myRightest = c1.position.X + c1.toRight;
float myLeftest = c1.position.X - c1.toLeft;
float hisRightest = c2.position.X + c2.toRight;
float hisLeftest = c2.position.X - c2.toLeft;
if(Collides1D(myLeftest, myRightest, hisLeftest, hisRightest))
{
float myTop = c1.position.Y + c1.top;
float myBottom = c1.position.Y - c1.bottom;
float hisTop = c2.position.Y + c2.top;
float hisBottom = c2.position.Y - c2.bottom;
if (Collides1D(myBottom, myTop, hisBottom, hisTop))
{
float myBack = c1.position.Z - c1.forward;
float myForward = c1.position.Z + c1.back;
float hisBack = c2.position.Z - c2.forward;
float hisForward = c2.position.Z + c2.back;
if (Collides1D(myBack, myForward, hisBack, hisForward))
{
return true;
}
}
}
return false;
}
}
static bool Collides1D(float left1, float right1, float left2, float right2)
{
if (left1 >= left2 && left1 <= right2)
return true;
if (right1 >= left2 && right1 <= right2)
return true;
if (left2 >= left1 && left2 <= right1)
return true;
if (right2 >= left1 && right2 <= right1)
return true;
return false;
}
My own method has been screwing up also, when trying to rotate models.
Ideally, it would be good to know what is wrong with the code from Microsoft, so that I can use it wherever, without worrying about hard-coding in object dimensions.
If anyone can see a fast fix to my own basic collision detection method, that would also be great.
I've looked at the Reimers tutorials, but I'm not getting them at the moment, maybe I've been staring at my own code for too long...
Any other information you want, I can try and supply. I can upload the models also, if that's the problem. I'm using models from Maya, exported as FBX.
I'm using MSVS 2010.
Thanks very much!
Jack
I think the problem may not be in the collision code, but in how you are updating the position of your Entitys. I don't see any update code or MoveForward(), etc, type of code. First of all, though, I think it would be much easier for you to use a Matrix instead of a bunch of float values for rotation. For example, you could have:
public class Entity
{
...
public Matrix RotationMatrix = Matrix.Identity;
public Vector3 position;
Public Entity(Vector3 FaceDirection, Vector3 UpDirection, Vector3 Position)
{
...
position = Position
RotationMatrix = Matrix.CreateWorld(Position, FaceDirection, UpDirection)
}
}
Then if you want to rotate, all you have to do is:
public Void RotateEnt(float Degrees)
{
RotationMatrix *= Matrix.CreateFromAxisAngle(RotationMatrix.Up, MathHelper.ToRadians(Degrees));
}
And if you do all this, then you should easily be able to update your Entity's position, which I think is the problem
public Void Update(GameTime gametime)
{
position += RotationMatrix.Forward * gametime.ElapsedRealTime.TotalMilliseconds * x; //x = some number to scale up or down velocity.
}
If you do this, I think your Entity's position will be updated, which then should fix your collision problem. But, not knowing how you update the position of your Entities, I am just speculating. HTH

aparapi start index of getGlobalId()

i use aparapi for parallelize and i wante to convert this java code:
public static void main(String[] args) {
float res = 0;
for (int i = 2; i < 5; i++) {
for (int j = 3; j < 5; j++) {
res += i * j;
}
}
System.out.println(res);
}
to its equivalent in aparapi:
Kernel kernel = new Kernel() {
#Override
public void run() {
int i = getGlobalId();
...
}
};
kernel.execute();
kernel.dispose();
There are a few issues here.
First your code is not data parallel. You have a 'race' condition on 'res' so this code cannot be computed on the GPU.
Secondly the range of execution is way too small. You are trying to execute 6 threads (x [2,3,4] * y [ 3,4]). This will not really gain any benefit from the GPU.
To answer the question regarding how you might implement over the 2 dim grid above.
Range range = Range.create2D(3, 2) ; // A two dimension grid 3x2
Kernel kernel = new Kernel() {
#Override
public void run() {
int x = getGlobalId(0)+2; // x starts at 2
int y = getGlobalId(1)+3; // y starts at 3
...
}
};
kernel.execute(range);
kernel.dispose();

Resources