AVCaptureSession CMSampleBuffer issue - ios

I have a problem with releasing memory from CMSampleBuffer in my AVCaptureSession camera.
This is my code for setting capture session. If I dispose imageDataSampleBuffer, app crashes.
using MonoTouch.CoreVideo;
using MonoTouch.CoreMedia;
using MonoTouch.AVFoundation;
using MonoTouch.ImageIO;
using MonoTouch.UIKit;
using MonoTouch.CoreFoundation;
using MonoTouch.Foundation;
using System.Drawing;
using System;
namespace myNamespace
{
public class AVFoundationCamera : UIViewController
{
public AVFoundationCamera (CameraController parView)
{
parentView = parView;
}
NSError error;
AVCaptureSession session;
AVCaptureDevice device;
AVCaptureDeviceInput input;
AVCaptureStillImageOutput output;
AVCaptureVideoPreviewLayer captureVideoPreviewLayer;
NSDictionary outputSettings;
AVCaptureConnection captureConnection;
UIButton buttCaptureImage;
public UIImageView imageV;
NSData imageData;
CameraController parentView;
public override void ViewDidAppear (bool animated)
{
base.ViewDidAppear (animated);
CreateControls();
SetupSession();
}
public override void DidReceiveMemoryWarning ()
{
imageData.Dispose();
session.Dispose();
device.Dispose();
input.Dispose();
output.Dispose();
captureVideoPreviewLayer.Dispose();
base.DidReceiveMemoryWarning ();
}
private void CreateControls()
{
imageV = new UIImageView(new RectangleF(0, 0, UIScreen.MainScreen.ApplicationFrame.Width, UIScreen.MainScreen.ApplicationFrame.Height - UIApplication.SharedApplication.StatusBarFrame.Height));
View.AddSubview(imageV);
buttCaptureImage = UIButton.FromType(UIButtonType.RoundedRect);
buttCaptureImage.Frame = new RectangleF(0, 60, 150, 30);
buttCaptureImage.SetTitle("Take a photo", UIControlState.Normal);
buttCaptureImage.TouchUpInside += HandleButtCaptureImageTouchUpInside;
View.AddSubview(buttCaptureImage);
}
void HandleButtCaptureImageTouchUpInside (object sender, EventArgs e)
{
captureConnection = null;
foreach (AVCaptureConnection capConn in output.Connections)
{
foreach (AVCaptureInputPort port in capConn.inputPorts)
{
if (port.MediaType == AVMediaType.Video)
{
captureConnection = capConn;
break;
}
}
if (captureConnection != null)
break;
}
output.CaptureStillImageAsynchronously(captureConnection, HandleAVCaptureCompletionHandlercompletionHandler);
buttCaptureImage.Enabled = false;
}
void HandleAVCaptureCompletionHandlercompletionHandler (CMSampleBuffer imageDataSampleBuffer, NSError error)
{
try
{
using (var pool = new NSAutoreleasePool ()) {
imageData = AVCaptureStillImageOutput.JpegStillToNSData(imageDataSampleBuffer);
//imageDataSampleBuffer.Dispose();
parentView.DismissModalViewControllerAnimated(true);
parentView.HandlePickedImage(imageData);
session.StopRunning();
}
}
catch (Exception exc)
{
Console.WriteLine(exc);
}
}
private void SetupSession()
{
session = new AVCaptureSession();
session.BeginConfiguration();
session.SessionPreset = AVCaptureSession.PresetPhoto;
captureVideoPreviewLayer = new AVCaptureVideoPreviewLayer(session);
captureVideoPreviewLayer.Frame = imageV.Bounds;
imageV.Layer.AddSublayer(captureVideoPreviewLayer);
device = AVCaptureDevice.DefaultDeviceWithMediaType(AVMediaType.Video);
input = new AVCaptureDeviceInput(device, out error);
session.AddInput(input);
output = new AVCaptureStillImageOutput();
output.OutputSettings = NSDictionary.FromObjectAndKey(new NSString("AVVideoCodecKey"), new NSString("AVVideoCodecJPEG"));
session.AddOutput(output);
session.CommitConfiguration();
session.StartRunning();
}
}
}
This is just a regular camera for taking photos. I tried with UIImagePickerController you posted here: https://github.com/migueldeicaza/TweetStation/blob/master/TweetStation/UI/Camera.cs which eliminates UIImagePickerController bug but whenever I click "Take a photo" button, preview window shows up which allocates memory. If I press "Retake", memory is being released, but in FinishedPiCkingMedia event handler I can't release it. So, after few photos it crashes.
Any solution works for me, but it would be great to see what I'm doing wrong.
Thank you once again.

This was a bug in MonoTouch.
There is a workaround you can use until you get the fix:
[DllImport ("/System/Library/Frameworks/CoreFoundation.framework/CoreFoundation")]
extern static void CFRetain (IntPtr handle);
void HandleAVCaptureCompletionHandlercompletionHandler (CMSampleBuffer imageDataSampleBuffer, NSError error)
{
try {
CFRetain (imageDataSampleBuffer.Handle);
(...)
} finally {
imageDataSampleBuffer.Dispose ();
}
}
I've added a Dispose call, there might be a limited amount of buffers available, and this way you ensure that the app doesn't exhaust them (since it may take a little time before the GC would free it automatically)
Also note that you should remove the workaround once you install a MonoTouch version with the real fix, since you'll leak the buffers otherwise.

Related

ios image picker delegates not triggering in nativescript

I'm writing nativescript plugin for image picker. I'm finished with part of android. Now, I'm writing code for ios. It's showing image picker dialog, but assigned delegates are not getting triggered. Please check my code below.
import * as application from "tns-core-modules/application";
import * as frame from "tns-core-modules/ui/frame"
export class Nativemediapicker extends NSObject implements UIImagePickerControllerDelegate {
public static ObjCProtocols = [UIImagePickerControllerDelegate];
get() {
let version = NSBundle.mainBundle.objectForInfoDictionaryKey("CFBundleShortVersionString");
return version;
}
static new(): Nativemediapicker {
return <Nativemediapicker>super.new();
}
private _callback: (result?) => void;
private _errorCallback: (result?) => void;
public initWithCallbackAndOptions(callback: (result?) => void, errorCallback: (result?) => void, options?): Nativemediapicker {
this._callback = callback;
this._errorCallback = errorCallback;
if (options) {
// collect options
}
console.log('initWithCallbackAndOptions')
return this;
}
static registerFileProvider(provider) { }
static pickFiles(mimeType, onResult, onError) {
onError("ERROR: For ios this feature is comming soon.");
}
static takePicture(onResult, onError) {
// if (!UIImagePickerController.isSourceTypeAvailable(UIImagePickerControllerSourceType.Camera)) {
// onError("ERROR: For ios simulator this feature is not supported.");
// return
// }
let imagePicker = UIImagePickerController.new()
imagePicker.delegate = Nativemediapicker.new().initWithCallbackAndOptions(onResult, onError, null)
imagePicker.sourceType = UIImagePickerControllerSourceType.PhotoLibrary
imagePicker.allowsEditing = false
// imagePicker.showsCameraControls = true
let topMostFrame = frame.topmost();
if (topMostFrame) {
let viewController: UIViewController = topMostFrame.currentPage && topMostFrame.currentPage.ios;
if (viewController) {
while (viewController.parentViewController) {
// find top-most view controler
viewController = viewController.parentViewController;
}
while (viewController.presentedViewController) {
// find last presented modal
viewController = viewController.presentedViewController;
}
viewController.presentViewControllerAnimatedCompletion(imagePicker, true, null);
}
}
}
static recordVideo(onResult, onError) {
onError("ERROR: For ios this feature is comming soon.");
}
static recordAudio(onResult, onError) {
onError("ERROR: For ios this feature is comming soon.");
}
imagePickerControllerDidCancel(picker): void {
console.log("imagePickerControllerDidCancel")
this._errorCallback("ERROR: Image capturing cancelled.");
}
imagePickerControllerDidFinishPickingMediaWithInfo(picker, info): void {
console.log("imagePickerControllerDidCancel")
this._errorCallback("ERROR: Image capturing done.");
}
}
I'm not getting, what I'm doing wrong and where?
Please help me, guys...
I suspect the reason is that your delegate is being cleaned up by garbage collector. One important rule with iOS is, you must always keep a reference of native object in a JS variable to keep it alive.
Try,
private _delegate;
....
this._delegate = Nativemediapicker.new().initWithCallbackAndOptions(onResult, onError, null);
imagePicker.delegate = this._delegate;
After adding this line in takePicture function it worked.
imagePicker.modalPresentationStyle = UIModalPresentationStyle.CurrentContext;

how to control background audio from Dependecy-service in xamarin.ios

I made a Dependency-Service for playing audio and background control in Xamarin.ios .
Audio was executed in background well.
but, Lockscreen controller was not shown.
I don't know what is problem.
I registered "UIBackgroundMode" and "audio" in "info.plist"
and set the AVAudioSession as playback.
I used play_pause(string url) function for playing audio and setting the MPNowPlayingInfo.
[assembly: Dependency(typeof(AudioService))]
namespace BibleHymn.iOS
{
class AudioService : IAudio
{
private bool isPlaying = false;
private bool interrupted = false;
int audioNum = 0;
AVFoundation.AVPlayer player;
public AudioService()
{
var avSession = AVAudioSession.SharedInstance();
avSession.SetCategory(AVAudioSessionCategory.Playback);
NSError activationError = null;
avSession.SetActive(true, out activationError);
}
public bool Play_Pause(string url)
{
if(player == null)
{
this.player = new AVFoundation.AVPlayer();
this.player = AVFoundation.AVPlayer.FromUrl(Foundation.NSUrl.FromString(url));
this.player.Play();
UpdateNotification();
}
isPlaying !=isPlaying;
return isPlaying;
}
public void UpdateNotification()
{
var item = new MPNowPlayingInfo
{
Title = "My Title"
};
MPNowPlayingInfoCenter.DefaultCenter.NowPlaying = item;
Device.BeginInvokeOnMainThread(() => {
UIKit.UIApplication.SharedApplication.BeginReceivingRemoteControlEvents();
});
}
}
I used play_pause from viewModel through button control's command.
init like ...
public IAudio streamService = DependencyService.Get<IAudio>();
and in command ...
streamService.Play_Pause(url);
You need to register background support for Audio in the info.plist by following screenshot
Also did you import Mediaplayer.framework

Gluon iOS audio Player

I've been following various answers to various questions on the subject and I've come to a result and some code which looks like it works. I'm getting stuck with the NSURL part of it. I've got 2 mp3 tracks in the assets folder of my iOS gluon project. I've made the IOSAudioService Class to handle the playback. and I'm passing an argument from the play button in the view to the Play() method. Everything other than the actual file is registering as working. I'm getting an NSError, which from looking at the code is a nil value, so either the argument isn't passing correctly or it can't find the file. Code below.
public AVAudioPlayer backgroundMusic;
private double currentPosition;
NSURL songURL = null;
#Override
public void Play(String filename){
songURL = new NSURL(filename);
try {
if (backgroundMusic != null) {
Resume();
}
else {
//Start the audio at the beginning.
currentPosition = 0;
backgroundMusic = new AVAudioPlayer(songURL);
//create the mendia player and assign it to the audio
backgroundMusic.prepareToPlay();
backgroundMusic.play();}
//catch the audio error
} catch(NSErrorException e) {
System.out.println("error: " + e);
}
}
#Override
public void Stop() {
backgroundMusic.stop();
backgroundMusic = null;
}
#Override
public void Pause() {
currentPosition = backgroundMusic.getCurrentTime();
backgroundMusic.pause();
}
#Override
public void Resume() {
backgroundMusic.setCurrentTime(currentPosition);
backgroundMusic.play();
}
try {
services = (AudioService) Class.forName("com.gluonhq.charm.down.plugins.ios.IOSAudioService").newInstance();
} catch (ClassNotFoundException | InstantiationException | IllegalAccessException ex) {
System.out.println("Error " + ex);
}
I'm getting the error at the catch block for NSExceptionError e.
if (services != null) {
final HBox hBox = new HBox(10,
MaterialDesignIcon.PLAY_ARROW.button(e -> services.Play("/audio.mp3")),
MaterialDesignIcon.PAUSE.button(e -> {
if (!pause) {
services.Pause();
pause = true;
} else {
services.Resume();
pause = false;
}
}),
MaterialDesignIcon.STOP.button(e -> services.Stop()));
//set the HBox alignment
hBox.setAlignment(Pos.CENTER);
hBox.getStyleClass().add("hbox");
//create and set up a vbox to include the image, audio controls and the text then set the alignment
final VBox vBox = new VBox(5, Image(), hBox, text1);
vBox.setAlignment(Pos.CENTER);
setCenter(new StackPane(vBox));
} else {
//start an error if service is null
setCenter(new StackPane(new Label("Only for Android")));
}
Services.get(LifecycleService.class).ifPresent(s -> s.addListener(LifecycleEvent.PAUSE, () -> services.Stop()));
}
I've also follow the advice on creating the service factory class and the interface from Audio performance with Javafx for Android (MediaPlayer and NativeAudioService) taking out the add audio element and I'm intending to do this on a view by view basis if possible.
Problem solved after must fiddling and looking in the Javadocs.Solved by adding/replacing some code with the below.
songURL = new NSURL("file:///" + NSBundle.getMainBundle().findResourcePath(filename, "mp3"));
try {
songURL.checkResourceIsReachable();}
catch(NSErrorException d) {
System.out.println("Song not found!" + d);
}

Showing "Import with Instagram" in UIActivityViewController

I am trying to add Instagram in "Share To" functionality in my app. I have seen the Instagram's iPhone hooks documents. I have created custom UIActivty which works fine but my question is, is there a way to add "Import with Instagram" functionality as it can be seen in iOS's Photos app iOS Photo App:
In my app for some reason, it does not show that "Import with Instagram". my app Share view :
I do not want to share only with Instagram so no ".igo"
EDIT: All of this is specifically for iOS versions < 10. For some reasons Instagram Share Extension works fine (for my app) in devices with iOS >= 10.
EDIT: I am trying to share image and video with ".jpeg" and ".mov" formats respectively
I have seen/read that Instagram added share extension in release 8.2, so technically all the apps should show "Instagram" in share tray, i.e. it can be seen in Google Photos app.
public void NativeShareImage(UIView sourceView, CGRect sourceRect,
UIImage image, string shareCaption, string emailSubject)
{
string filename = Path.Combine(FileSystemUtils.GetTemporaryDataPath(), "Image.jpg");
NSError err = null;
using(var imgData = image.AsJPEG(JpgImageQuality))
{
if(imgData.Save(filename, false, out err))
{
Logger.Information("Image saved before native share as {FileName}", filename);
}
else
{
Logger.Error("Image NOT saved before native share as to path {FileName}. {Error}", filename, err.Description);
return;
}
}
// this are the items that needs to be shared
// Instagram ignores the caption, that is known
var activityItems = new List<NSObject>
{
new NSString(shareCaption),
new NSUrl(new Uri(filename).AbsoluteUri)
};
// Here i add the custom UIActivity for Instagram
UIActivity[] applicationActivities =
{
new InstagramActivity(image, sourceRect, sourceView),
}
var activityViewController = new UIActivityViewController(activityItems.ToArray(), applicationActivities);
activityViewController.SetValueForKey(new NSString(emailSubject), new NSString("subject"));
activityViewController.CompletionWithItemsHandler = (activityType, completed, returnedItems, error) =>
{
UserSharedTo(activityType, completed);
};
// Hide some of the less used activity types so that Instagram shows up in the list. Otherwise it's pushed off the activity view
// and the user has to scroll to see it.
activityViewController.ExcludedActivityTypes = new[] { UIActivityType.AssignToContact, UIActivityType.CopyToPasteboard, UIActivityType.Print };
if(UIDevice.CurrentDevice.UserInterfaceIdiom == UIUserInterfaceIdiom.Phone)
{
PresentViewController(activityViewController, true, null);
}
else
{
activityViewController.ModalPresentationStyle = UIModalPresentationStyle.Popover;
PresentViewController(activityViewController, true, null);
// Get the popover presentation controller and configure it.
UIPopoverPresentationController presentationController = activityViewController.PopoverPresentationController;
presentationController.PermittedArrowDirections = UIPopoverArrowDirection.Down;
presentationController.SourceRect = sourceRect;
presentationController.SourceView = sourceView;
}
}
// when opening custom activity use ".igo" to only show instagram
public class InstagramActivity : UIActivity
{
public InstagramActivity(UIImage imageToShare, CGRect frame, UIView view, string shareCaption = "")
{
_ImageToShare = imageToShare;
_Frame = frame;
_View = view;
}
public override UIImage Image { get { return UIImage.FromBundle("Instagram"); } }
public override string Title { get { return "Instagram"; } }
public override NSString Type { get { return new NSString("PostToInstagram"); } }
public string Caption { get; set; }
public override bool CanPerform(NSObject[] activityItems)
{
return UIApplication.SharedApplication.CanOpenUrl(NSUrl.FromString("instagram://app"));
}
public override void Prepare(NSObject[] activityItems)
{
}
public override void Perform()
{
string filename = Path.Combine(FileSystemUtils.GetTemporaryDataPath(), "Image.igo");
NSError err = null;
using(var imgData = _ImageToShare.AsJPEG(JpgImageQuality))
{
if(imgData.Save(filename, false, out err))
{
Logger.Information("Instagram image saved as {FileName}", filename);
}
else
{
Logger.Error("Instagram image NOT saved as to path {FileName}. {Error}", filename, err.Description);
Finished(false);
return;
}
}
var url = NSUrl.FromFilename(filename);
_DocumentController = UIDocumentInteractionController.FromUrl(url);
_DocumentController.DidEndSendingToApplication += (o, e) => Finished(true);
_DocumentController.Uti = "com.instagram.exclusivegram";
if(!string.IsNullOrEmpty(ShareCaption))
{
_DocumentController.Annotation = NSDictionary.FromObjectAndKey(new NSString(ShareCaption), new NSString("InstagramCaption"));
}
_DocumentController.PresentOpenInMenu(_Frame, _View, true);
}
UIImage _ImageToShare;
CGRect _Frame;
UIView _View;
UIDocumentInteractionController _DocumentController;
}

Presenting UIImagePickerController crashes app

I am trying to present a UIImagePickerController via a Button action. My project crashes with error:
Got a SIGABRT while executing native code. This usually indicates
a fatal error in the mono runtime or one of the native libraries
used by your application.
I only have a ViewController embedded in a NavigationController in the storyboard. Code snippets below:
UIImagePickerController imagePicker;
public override void ViewDidLoad()
{
base.ViewDidLoad();
this.setupImagePicker();
CapturePhotoButton.TouchUpInside += delegate
{
this.AddMedia();
};
}
public void setupImagePicker()
{
imagePicker = new UIImagePickerController();
imagePicker.SourceType = UIImagePickerControllerSourceType.Camera;
imagePicker.ModalPresentationStyle = UIModalPresentationStyle.Popover;
imagePicker.MediaTypes = UIImagePickerController.AvailableMediaTypes(
UIImagePickerControllerSourceType.Camera);
imagePicker.FinishedPickingMedia += HandleFinishedPickingMedia;
imagePicker.Canceled += (sender, e) => {
imagePicker.DismissModalViewController(true);
};
}
public void HandleFinishedPickingMedia(object sender,
UIImagePickerMediaPickedEventArgs e)
{
bool isImage = false;
switch (e.Info[UIImagePickerController.MediaType].ToString())
{
case "public.image":
isImage = true;
break;
case "public.video":
break;
}
if (isImage)
{
UIImage originalImage = e.Info[UIImagePickerController.OriginalImage] as UIImage;
if (originalImage != null)
{
PreviewImageView.Image = originalImage;
imagePicker.DismissViewController(true, null);
}
}
}
public void AddMedia()
{
//Crashes on this line
this.NavigationController.PresentViewController(imagePicker, true, null);
}
Added Privacy - Camera Usage Description to your info.plist and that resolved the issue
Add the following to your info.plist for camera use:
<key>NSCameraUsageDescription</key>
<string>This app needs access to the camera to take photos.</string>
for library:
<key>NSPhotoLibraryUsageDescription</key>
<string>This app needs access to photos.</string>

Resources