How to add UIImage to a Scene in AR Kit - ios

Question

I have AR session which adds SCNText and 3D Objects as well. And now, I want to add UIImage from Image Picker and don't know how to do this. Is there any solutions?
SOLUTION
func insertImage(image: UIImage, width: CGFloat = 0.3, height: CGFloat = 0.3) -> SCNNode {
let plane = SCNPlane(width: width, height: height)
plane.firstMaterial!.diffuse.contents = image
let node = SCNNode(geometry: plane)
node.constraints = [SCNBillboardConstraint()]
return node
}
let image = insertImage(image: addedImage)
node.addChildNode(image)

Answer 1

As I am sure you are aware an SCNGeometry has a materials property which is simply:
A container for the color or texture of one of a material’s visual
properties.
As such you could render a UIImage onto an SCNGeometry using for example the diffuse property.
Here is a fully working and tested example. Which loads a UIImagePickerController after 5 seconds, and then creates an SCNNode with an SCNPlane Geometry which has is contents set to the selected UIImage.
The code is fully commented so it should be easy enough to understand:
//-------------------------------------
//MARK: UIImagePickerControllerDelegate
//-------------------------------------
extension ViewController: UIImagePickerControllerDelegate{
func imagePickerController(_ picker: UIImagePickerController, didFinishPickingMediaWithInfo info: [String : Any]) {
//1. Check We Have A Valid Image
if let selectedImage = info[UIImagePickerControllerOriginalImage] as? UIImage {
//2. We Havent Created Our PlaneNode So Create It
if planeNode == nil{
//d. Dismiss The Picker
picker.dismiss(animated: true) {
//a. Create An SCNPlane Geometry
let planeGeometry = SCNPlane(width: 0.5, height: 0.5)
//b. Set's It's Contents To The Picked Image
planeGeometry.firstMaterial?.diffuse.contents = self.correctlyOrientated(selectedImage)
//c. Set The Geometry & Add It To The Scene
self.planeNode = SCNNode()
self.planeNode?.geometry = planeGeometry
self.augmentedRealityView.scene.rootNode.addChildNode(self.planeNode!)
self.planeNode?.position = SCNVector3(0, 0, -1.5)
}
}
}
picker.dismiss(animated: true, completion: nil)
}
func imagePickerControllerDidCancel(_ picker: UIImagePickerController) { picker.dismiss(animated: true, completion: nil) }
}
class ViewController: UIViewController, UINavigationControllerDelegate {
//1. Create A Reference To Our ARSCNView In Our Storyboard Which Displays The Camera Feed
#IBOutlet weak var augmentedRealityView: ARSCNView!
//2. Create Our ARWorld Tracking Configuration & Session
let configuration = ARWorldTrackingConfiguration()
let augmentedRealitySession = ARSession()
//3. Create A Reference To Our PlaneNode
var planeNode: SCNNode?
var planeGeomeryImage: UIImage?
//---------------
//MARK: LifeCycle
//---------------
override func viewDidLoad() {
super.viewDidLoad()
//1. Setup The Session
setupARSession()
//2. Show The UIImagePicker After 4 Seconds
DispatchQueue.main.asyncAfter(deadline: .now() + 4) {
self.selectPhotoFromGallery()
}
}
override func didReceiveMemoryWarning() { super.didReceiveMemoryWarning() }
//-------------
//MARK: ARSetup
//-------------
func setupARSession(){
//1. Run Our Session
augmentedRealityView.session = augmentedRealitySession
augmentedRealitySession.run(configuration, options: [.resetTracking, .removeExistingAnchors])
}
//---------------------
//MARK: Image Selection
//---------------------
/// Loads The UIImagePicker & Allows Us To Select An Image
func selectPhotoFromGallery(){
if UIImagePickerController.isSourceTypeAvailable(UIImagePickerControllerSourceType.photoLibrary){
let imagePicker = UIImagePickerController()
imagePicker.delegate = self
imagePicker.allowsEditing = true
imagePicker.sourceType = UIImagePickerControllerSourceType.photoLibrary
self.present(imagePicker, animated: true, completion: nil)
}
}
/// Correctly Orientates A UIImage
///
/// - Parameter image: UIImage
/// - Returns: UIImage?
func correctlyOrientated(_ image: UIImage) -> UIImage {
if (image.imageOrientation == .up) { return image }
UIGraphicsBeginImageContextWithOptions(image.size, false, image.scale)
let rect = CGRect(x: 0, y: 0, width: image.size.width, height: image.size.height)
image.draw(in: rect)
let normalizedImage = UIGraphicsGetImageFromCurrentImageContext()!
UIGraphicsEndImageContext()
return normalizedImage
}
}
Don't forget to add the NSPhotoLibraryUsageDescription to your info.plist:
<key>NSPhotoLibraryUsageDescription</key>
<string>For ARkit</string>
This should be more than enough to get you started...