Konva load stage with images and background video - save

Hey guys so I have this konva js app that has your typical background video just like the demo or close to it. I am able to play the video background and add image objects on top of it etc. Now I want a way to save the stage in its current state. So I do state.stage.toJSON() and as expected it creates a serialized JSON object. Now here's where I am hung up. when I load the stage like so state.stage = Konva.Node.create(data.stage, "container"); the stage data gets loaded (it is the correct size and so forth) yet there is no background video, no images or anything how do I fix this? I don't even know if stage.toJSON is correct but the point is I need to save it, leave the page and load it back up at a future date.
state.backgroundVideo = new Konva.Image({
image: state.video,
draggable: false
});
state.video.addEventListener("loadedmetadata", function(e) {
state.backgroundVideo.width(state.width);
state.backgroundVideo.height(state.height);
});
state.anim = new Konva.Animation(function() {
// do nothing, animation just need to update the layer
}, state.layer);
state.layer.add(state.backgroundVideo);
state.layer.batchDraw();
const canvas = document.getElementsByTagName("canvas")[0];
state.canvas = canvas;

node.toJSON() doesnt serialize image or video objects from Konva.Image attributes.
To solve the issue you can save video or image src into custom attribute:
image.setAttr('source', imageURL);
A simple string will be serialized into JSON. Then after you created a node from the JSON Konva.Node.create(data.stage, "container");
You need to find such nodes and restore image or video manually
// "video-background" is a sample here
// you can define your own selector
const videoNode = stage.findOne('.video-background');
const source = videoNode.getAttr('source');
const video = document.createElement('video');
video.src = source;
videoNode.image(video);
For more information take a look here:
https://konvajs.org/docs/data_and_serialization/Complex_Load.html
https://konvajs.org/docs/data_and_serialization/Best_Practices.html

thanks #lavrton your comments really helped me figure it out. Rather than loading the stage directly however I just saved the data from the stage used the data to rebuild the stage in the exact same way. Heres the code I used to get the data to save.
Heres my code:
const groups = state.stage.find("Group");
const stageData = [];
for (let x = 0; x < groups.length; x++) {
let g = groups[x];
let i = g.findOne("Image").getAttrs();
let group = { x: g.getX(), y: g.getY() };
let image = { x: i.width, y: i.height };
let obj = { image, group, jsonId: g.getAttr("jsonId") };
stageData.push(obj);
}
console.log("stageData", stageData);

Related

Xamarin.Forms Xam.Plugin.Media Take Picture on iOS

I'm using the Xam.Plugin.Media in my Forms app to take pictures.
I take the Image stream (GetStream) and convert to a byte[] and store in my DB.
I then use that as the image source.
On Android and UWP, its working fine.
On iOS, if the picture is taken in portrait mode, the image once, selected is always rotated 90 deg.
I will later, upload this to a server and that image could be used on a different device.
For this, I also tried the GetStreamWithImageRotatedForExternalStorage but in this case, I cant see the image at all.
There are bytes in the stream (I DisplayAlert the length) but the image does not display.
Any idea what I might be doing wrong?
My code:-
private async Task TakePicture(WineDetails details)
{
await CrossMedia.Current.Initialize();
if (CrossMedia.Current.IsCameraAvailable && CrossMedia.Current.IsTakePhotoSupported)
{
var file = await CrossMedia.Current.TakePhotoAsync(new Plugin.Media.Abstractions.StoreCameraMediaOptions
{
AllowCropping = true,
PhotoSize = Plugin.Media.Abstractions.PhotoSize.Medium,
SaveToAlbum = false,
RotateImage = true
});
if (file == null)
return;
using (var ms = new MemoryStream())
{
var stream = file.GetStreamWithImageRotatedForExternalStorage();
stream.CopyTo(ms);
details.LabelImage = ms.ToArray();
details.NotifyChange("ImageSource");
}
}
}
The image is updated in the page via the NotifyChange and looks like this:-
ImageSource.FromStream(() => new MemoryStream(this.LabelImage))
This works fine in all cases on Android and UWP, works on iOS using GetStream (except the image is incorrectly rotated) but does not work using GetStreamWithImageRotatedForExternalStorage on iOS.
Anyone else using this plugin?
Any idea why GetStream returns a rotated image?
Any idea why GetStreamWithImageRotatedForExternalStorage is not working?
Thanks
Update:-
Changed SaveToAlbum = true and when I open the gallery, the image is rotated 90 deg.
Have RotateImage = true which could cause the issue? I'll try setting it to false.
I still can't set the image source to the byte array of the image using GetStreamWithImageRotatedForExternalStorage.
using (var ms = new MemoryStream())
{
file.GetStreamWithImageRotatedForExternalStorage().CopyTo(ms);
details.LabelImage = ms.ToArray();
}
using the byte array for an image
return ImageSource.FromStream(() => new MemoryStream(this.LabelImage));
This does not work for me, GetStream works ok.
Update:-
Ok so, RotateImage = false + GetStreamWithImageRotatedForExternalStorage allows me to display the image but its still incorrectly rotated in my app and the gallery.
I'm using this plugin, which is similar (if not the same thing - I know James Montemagno has recently packaged/bundled his work with Xamarin).
If you check the issues board there, you'll see there are quite a few people that have similar troubles (image rotation on iOS). Almost every 'solution' mentions using GetStreamWithImageRotatedForExternalStorage.
My issue was similar - I was unable to take a photo on iOS in portrait mode, without other (non-ios Devices) rotating the image. I have tried for weeks to solve this issue, but support on the plugin seems to be quite limited.
Ultimately I had to solve this with a huge workaround - using a custom renderer extending from FFImageLoading to display our images and MetadataExtractor. We were then able to extract the EXIF data from the stream and apply a rotation transformation to the FFImageLoding image control.
The rotation information was stored in a sort of weird way, as a string. This is the method I'm using to extract the rotation information, and return the amount it needs to be rotated as an int. Note that for me, iOS was able to display the image correctly still, so it's only returned a rotation change for Android devices.
public static int GetImageRotationCorrection(byte[] image)
{
try
{
var directories = ImageMetadataReader.ReadMetadata(new MemoryStream(image));
if (Device.Android == "Android")
{
foreach (var directory in directories)
{
foreach (var tag in directory.Tags)
{
if (tag.Name == "Orientation")
{
if (tag.Description == "Top, left side(Horizontal / normal)")
return 0;
else if (tag.Description == "Left side, bottom (Rotate 270 CW)")
return 270;
else if (tag.Description == "Right side, top (Rotate 90 CW")
return 90;
}
}
}
}
return 0;
}
catch (Exception ex)
{
return 0;
}
}
Note that there is also a custom renderer for the image for FFImage Loading.
public class RotatedImage : CachedImage
{
public static BindableProperty MyRotationProperty = BindableProperty.Create(nameof(MyRotation), typeof(int), typeof(RotatedImage), 0, propertyChanged: UpdateRotation);
public int MyRotation
{
get { return (int)GetValue(MyRotationProperty); }
set { SetValue(MyRotationProperty, value); }
}
private static void UpdateRotation(BindableObject bindable, object oldRotation, object newRotation)
{
var _oldRotation = (int)oldRotation;
var _newRotation = (int)newRotation;
if (!_oldRotation.Equals(_newRotation))
{
var view = (RotatedImage)bindable;
var transformations = new System.Collections.Generic.List<ITransformation>() {
new RotateTransformation(_newRotation)
};
view.Transformations = transformations;
}
}
}
So, in my XAML - I had declared a RotatedImage instead of the standard Image. With the custom renderer, I'm able to do this and have the image display rotated the correct amount.
image.MyRotation = GetImageRotationCorrection(imageAsBytes)
It's a totally unnecessary workaround - but these are the lengths that I had to go to to get around this issue.
I'll definitely be following along with this question, there might be someone in the community who could help us both!
The SaveMetaData flag is causing the rotation issue.
Setting it to false (default is true) now displays the photo correctly.
One side effect of that, the image no longer appears in the gallery if SaveToAlbum=true.
Still can't use an image take when using GetStreamWithImageRotatedForExternalStorage, even using FFImageLoading.
I found that while using Xam.Plugin.Media v5.0.1 (https://github.com/jamesmontemagno/MediaPlugin), the combination of three different inputs produced different results on Android vs. iOS:
StoreCameraMediaOptions.SaveMetaData
StoreCameraMediaOptions.RotateImage
Using MediaFile.GetStream() vs. MediaFile.GetStreamWithImageRotatedForExternalStorage()
On Android, SaveMetaData = false, RotateImage = true, and using MediaFile.GetStreamWithImageRotatedForExternalStorage() worked for me whether I was saving the result stream externally or processing the result stream locally for display.
On iOS, the combination of RotateImage = true and StreamRotated = true would result in a NullReferenceException coming out of the plugin library. Using MediaFile.GetStreamWithImageRotatedForExternalStorage() appeared to have no impact on behaivor.
--
Before going further, it's important to understand that image orientation in the JPEG format (which Xam.Plugin.Media seems to use) isn't as straightforward as you might think. Rather than rotating the raw image bytes 90 or 180 or 270 degrees, JPEG orientation can be set through embedded EXIF metadata. Orientation issues will happen with JPEGs either if EXIF data is stripped or if downstream consumers don't handle the EXIF data properly.
The approach I landed on was to normalize JPEG image orientation at the point the image is captured without relying on EXIF metadata. This way, downstream consumers shouldn't need to be relied on to properly inspect and handle EXIF orientation metadata.
The basic solution is:
Scan a JPEG for EXIF orientation metadata
Transform the JPEG to rotate/flip as needed
Set the result JPEG's orientation metadata to default
--
Code example compatible with Xamarin, using ExifLib.Standard (1.7.0) and SixLabors.ImageSharp (1.0.4) NuGet packages. Based on (Problem reading JPEG Metadata (Orientation))
using System;
using System.IO;
using ExifLib;
using SixLabors.ImageSharp;
using SixLabors.ImageSharp.Formats.Jpeg;
using SixLabors.ImageSharp.Metadata.Profiles.Exif;
using SixLabors.ImageSharp.Processing;
namespace Your.Namespace
{
public static class ImageOrientationUtility
{
public static Stream NormalizeOrientation(Func<Stream> inputStreamFunc)
{
using Stream exifStream = inputStreamFunc();
using var exifReader = new ExifReader(exifStream);
bool orientationTagExists = exifReader.GetTagValue(ExifTags.Orientation, out ushort orientationTagValue);
if (!orientationTagExists)
// You may wish to do something less aggressive than throw an exception in this case.
throw new InvalidOperationException("Input stream does not contain an orientation EXIF tag.");
using Stream processStream = inputStreamFunc();
using Image image = Image.Load(processStream);
switch (orientationTagValue)
{
case 1:
// No rotation required.
break;
case 2:
image.Mutate(x => x.RotateFlip(RotateMode.None, FlipMode.Horizontal));
break;
case 3:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate180, FlipMode.None));
break;
case 4:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate180, FlipMode.Horizontal));
break;
case 5:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate90, FlipMode.Horizontal));
break;
case 6:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate90, FlipMode.None));
break;
case 7:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate270, FlipMode.Horizontal));
break;
case 8:
image.Mutate(x => x.RotateFlip(RotateMode.Rotate270, FlipMode.None));
break;
}
image.Metadata.ExifProfile.SetValue(ExifTag.Orientation, (ushort)1);
var outStream = new MemoryStream();
image.Save(outStream, new JpegEncoder{Quality = 100});
outStream.Position = 0;
return outStream;
}
}
}
And to use in conjunction with Xam.Plugin.Media:
MediaFile photo = await CrossMedia.Current.TakePhotoAsync(options);
await using Stream stream = ImageOrientationUtility.NormalizeOrientation(photo.GetStream);

Printing with only pdf.js (without the viewer)

There are already two questions about this, but the specific question remains unanswered.
I have a multipage pdf document. I have successfully used pdf.js core to write a page to a canvas. Soon I'm sure I'll have prev / next paging functions so I can see each page. See my existing code at bottom. The reason I am not using the viewer is because my boss doesn't like it. He wants a totally different interface that is extremely minimal and simple, and I have successfully built that minus the full print, ala:
Please note that there is no url to a pdf document. I get the data from a api call that returns base64 data. My code loads the data from an array into the pdf.
My current print function seems like a hack, and I know it isn't how the pdf.js viewer does it. I want to print the pdf as the pdf.js viewer does; meaning, true to form. I'm not sure if converting each page to an image gets it done, but I remain open to suggestions.
The problems are: extra margin not in the original document, and headers / footers. Not to mention pdf documents actually can contain different page layouts and sizes for each page and throwing an image tag into a new window completely ignores that.
I did check the pdf.js examples on Github. None of them are print and they say that using the pdf.js without the viewer is not supported and we are on our own if we do that (is that right?). Anyway, nothing on Google that I could find.
var pdfDocument;
var canvas;
// load from base64 data
loadPdfDocument(file.Content);
function print() {
var doc = document.getElementById('document');
var html = doc.innerHTML;
if (canvas)
html = "<img src='" + canvas.toDataURL() + "'";
var win = window.open('', '', '');
win.document.write(html);
win.document.close();
win.focus();
win.print();
win.close();
};
function loadPdfDocument(base64Data) {
PDFJS.disableWorker = true;
var pdfData = base64ToUint8Array(base64Data);
PDFJS.getDocument(pdfData).then(function(pdf) {
canvas = document.getElementById('pdfPage');
pdfDocument = pdf;
loadPdfPage(1);
});
}
function base64ToUint8Array(base64) {
var raw = atob(base64); // convert base 64 string to raw string
var uint8Array = new Uint8Array(raw.length);
for (var i = 0; i < raw.length; i++) {
uint8Array[i] = raw.charCodeAt(i);
}
return uint8Array;
}
function loadPdfPage(pageIndex) {
pdfDocument.getPage(pageIndex).then(function(page) {
var scale = 1;
var viewport = page.getViewport(scale);
var context = canvas.getContext('2d');
canvas.height = viewport.height;
canvas.width = viewport.width;
page.render({
canvasContext: context,
viewport: viewport
});
});
}
<div id="document">
<canvas id="pdfPage" />
</div>

Three JS select geometry by id

Background: I'm a dev that knows JS, but is relatively new to Three JS. I've done a few small projects that involve static scenes with basic repeating animation.
I'm currently working on a modified version of Google's Globe project http://workshop.chromeexperiments.com/globe/. Looking back, I probably should have just started from scratch, but it was a good tool to see the approach their dev took. I just wish I could now update ThreeJS w/o the whole thing falling apart (too many unsupported methods and some bugs I never could fix, at least not in the hour I attempted it).
In the original, they are merging all of the geometric points into one object to speed up FPS. For my purposes, I'm updating the points on the globe using JSON, and there will never be more than 100 (probably no more than 60 actually), so they need to remain individual. I've removed the "combine" phase so I can now individually assign data to the points and then TWEEN the height change animation.
My question is, how do I manually select a single point (which is a Cube Geometry) so that I can modify the height value? I've looked through Stack Overflow and Three JS on GitHub and I'm not sure I understand the process. I'm assigning an ID to make it directly relate to the data that is being passed into it (I know WebGL adds an individual name/ID for particles, but I need something that is more directly related to what I'm doing for the sake of simplicity). That seems to work fine. But again, as a JS dev I've tried .getElementById(id) and $('#'+id) in jQuery, and neither works. I realize that Geometry objects don't behave the same way as HTML DOM objects, so I guess that's where I'm having struggles.
Code to add a point of data to the globe:
function addPoint(lat, lng, size, color, server) {
geometry = new THREE.Cube(0.75, 0.75, 1, 1, 1, 1, null, false, { px: true,
nx: true, py: true, ny: true, pz: false, nz: true});
for (var i = 0; i < geometry.vertices.length; i++) {
var vertex = geometry.vertices[i];
vertex.position.z += 0.5;
}
var point = new THREE.Mesh(geometry, new THREE.MeshBasicMaterial ({
vertexColors: THREE.FaceColors
}));
var phi = (90 - lat) * Math.PI / 180;
var theta = (180 - lng) * Math.PI / 180;
point.position.x = 200 * Math.sin(phi) * Math.cos(theta);
point.position.y = 200 * Math.cos(phi);
point.position.z = 200 * Math.sin(phi) * Math.sin(theta);
if($('#'+server).length > 0) {
server = server+'b';
}
point.id = server;
point.lookAt(mesh.position);
point.scale.z = -size;
point.updateMatrix();
for (var i = 0; i < point.geometry.faces.length; i++) {
point.geometry.faces[i].color = color;
}
console.log(point.id);
scene.addObject(point);
}
So now to go back, I know I can't use point.id because obviously that will only reference inside the function. But I've tried 'Globe.id', 'Globe.object.id', 'object.id', and nothing seems to work. I know it is possible, I just can't seem to find a method that works.
Okay, I found a method that works for this by playing with the structure.
Essentially, the scene is labeled "globe" and all objects are its children. So treating the scene as an array, we can successfully pass an object into a var using the following structure:
Globe > Scene > Children > [Object]
Using a matching function, we loop through each item and find the desired geometric object and assign it to a temporary var for animation/adjustment:
function updatePoints(server){
var p, lineObject;
$.getJSON('/JSON/'+server+'.json', function(serverdata) {
/* script that sets p to either 0 or 1 depending on dataset */
var pointId = server+p;
//Cycle through all of the child objects and find a patch in
for(var t = 3; t < globe.scene.children.length; t++) {
if(globe.scene.children[t].name === pointId) {
//set temp var "lineObject" to the matched object
lineObject = globe.scene.children[t];
}
}
/* Manipulation based on data here, using lineObject */
});
}
I don't know if this is something that anyone else has had questions on, but I hope it helps someone else! :)
EDIT: Just realized this isn't a keyed array so I can use .length to get total # of objects

Multiple collada scenes

I have two collada files (two different scenes: "01.dae" and "02.dae").
I want to display 01.dae first and right after the animation finishes I want to load and display 02.dae on the same canvas.
(I'm trying to modify "webgl_loader_collada_keyframe.html" to do this but no results so far.)
How could I handle more than one animated collada scenes? A source code or any tips or tricks would be appreciated!
Thank you for your answer. I modified my code based on your idae but unfortunately it's not working.
Could you take a look at my code please?
Here is my code:
loader.load( 'pump.dae', function ( collada ) {
model = collada.scene;
animations = collada.animations;
kfAnimationsLength = animations.length;
model.scale.x = model.scale.y = model.scale.z = 0.125; // 1/8 scale, modeled in cm
init();
start();
animate( lastTimestamp );
setTimeout(loadSecond,3000);
} );
function loadSecond()
{
loader2.load( 'pump2.dae', function ( collada2 )
{
scene.remove( model );
model2 = collada2.scene;
animations2 = collada2.animations;
kfAnimationsLength2 = animations2.length;
model2.scale.x = model2.scale.y = model2.scale.z = 0.125; // 1/8 scale, modeled in cm
init2();
start2();
animate2( lastTimestamp2 );
//alert("second loaded");
} );
}
...
As you can see I extended your code with
scene.remove( model );
to remove the previous scene.
The first scene displays and then disapears properly however the new secene does not load. Do you have an idae why?
(Note: I don't know how long my first scene realy is.)
If you know how long the animation is, you could use a setTimeout to load the second model.
var loader = new THREE.ColladaLoader();
loader.options.convertUpAxis = true;
loader.load('PATH TO MODEL', function colladaReady( collada){
dae = collada.scene;
skin = collada.skins[ 0 ];
dae.scale.x = dae.scale.y = dae.scale.z = 1;
dae.updateMatrix();
scene.add(dae);
render();
setTimeout(loadSecond,3000);
});
function loadSecond(){
var loader = new THREE.ColladaLoader();
loader.options.convertUpAxis = true;
loader.load('PATH TO MODEL', function colladaReady( collada){
///repeat model loading logic
}
Where the time interval in the setTimeout is equal to the length of your animation.

Get image original width & height in actionscript

I use AS3 in Flex 3 to create new image and seem unable to get the exact size of the original image. percentHeight & percentWidth to 100 can do the job, but limitation in ObjectHandlers require me to set the image scale in pixel.
Any solution?
Note: this is also applicable for displaying Image original dimension without ObjectHandler control, just remove those lines that are not applicable.
After struggle hours for solution, I found my own answer thru in actionscript forum, in fact, only one solution, I surprise there was no such topic elsewhere.
private function init():void {
var image:Image = new Image();
image.source = "http://www.colorjack.com/software/media/circle.png";
image.addEventListener(Event.COMPLETE, imageLoaded);
/* wait for completion as Image control is asynchronous,
* which mean ObjectHandler will attempt to load asap
* and you are not able to get the correct dimension for scaling.
* EventListener fixed that.
*/
this.addChild(image);
//whenever you scale ObjectHandler control, the image is always fit by 100%
image.percentHeight = 100;
image.percentWidth = 100;
}
private function imageLoaded(e:Event):void{
var img:Image = e.target as Image;
trace("Height ", img.contentHeight);
trace("Width ", img.contentWidth);
var oh:ObjectHandles = new ObjectHandles();
oh.x = 200;
oh.y = 200;
oh.height = img.contentHeight;
oh.width = img.contentWidth;
oh.allowRotate = true;
oh.autoBringForward = true;
oh.addChild(img);
genericExamples.addChild(oh);
}

Resources