unable to load some images with URLSession.shared.dataTask - ios

hello I have problem loading some image using the following code:
xtension UIImageView {
func downloadImage(from urlRequest: String) {
let urlRequest = URLRequest(url: (URL(string: urlRequest.addingPercentEncoding(withAllowedCharacters: .urlPathAllowed)!)!))
urlRequest.adding
let task = URLSession.shared.dataTask(with: urlRequest) { (data, response, error) in
print(urlRequest)
if error != nil {
print(error!)
return
}
let httpResponse = response as! HTTPURLResponse
let statusCode = httpResponse.statusCode
if (statusCode == 200) {
DispatchQueue.main.async {
self.image = UIImage(data: data!)!
}
}
}
task.resume()
}
}
from what i saw it appears my URL string is being changed from:
https://upload.wikimedia.org/wikipedia/commons/5/5f/Panthera_pardus_%28Leopard_%28Kongo%29%29.jpg
(which is a valid url)
into:
https://upload.wikimedia.org/wikipedia/commons/5/5f/Panthera_pardus_%2528Leopard_%2528Kongo%2529%2529.jpg
perhaps there is something wrong with my parsing from json?
json parse code:
private func readJson() {
do {
self.animals = [Animal]()
if let file = Bundle.main.url(forResource: "animals", withExtension: "json") {
let data = try Data(contentsOf: file)
let json = try JSONSerialization.jsonObject(with: data, options: [])
if let object = json as? [String: Any] {
//parse title:
if (object["title"] as? String) != nil {
}
var isAnimal: Bool
//parse the paragrapphs which is an array
if let paragraphs = object["paragraphs"] as? [[String: AnyObject]] {
for paragraph in paragraphs {
isAnimal = true
let caption = paragraph["caption"]
let text = paragraph["text"]
if let images = paragraph["images"] as? [[String:AnyObject]] {
if images.count == 0 {
isAnimal = false
}
var imageObjects = [[String:String]]()
for image in images {
let imageName = image["name"] as? String
let imageUrl = image["url"] as? String
let imageExplanation = image["explanation"] as? String
var imgDict = [String: String]()
imgDict["name"] = imageName
imgDict["url"] = imageUrl
imgDict["explanation"] = imageExplanation
imageObjects.append(imgDict)
}
if isAnimal == true {
let animal = Animal()
animal.caption = caption as! String?
animal.text = text as! String?
animal.images = imageObjects
self.animals?.append(animal)
}
}
}
}
// json is a dictionary
// print(object)
} else if let object = json as? [Any] {
// json is an array
print(object)
} else {
print("JSON is invalid")
}
} else {
print("no file")
}
DispatchQueue.main.async {
self.tableView.reloadData()
}
} catch {
print(error.localizedDescription)
}
}
and the json file:
{
"title": "Wildlife of South Africa",
"paragraphs": [
{
"caption": "Introduction",
"text": "South Africa has a large variety of wildlife, including snakes, birds, plains animals, and predators. The country has 299 species of mammals and 858 species of birds.",
"images": []
},
{
"caption": "Cape Buffalo",
"text": "The Cape Buffalo, also known as the African cook, is a powerful animal that has few natural enemies. Their power and size means that they are very much able to defend themselves. They have been known to kill lions, hyena, humans, and other wild predators. Because of this they have taken their place in the African big five, elephants, lions, Cape buffalo, rhinoceroses, and leopards. (The big five are known to be some of the most dangerous and aggressive animals in Africa.) Another African name for the Cape buffalo is black death, because of their colour and their aggressive behaviour.",
"images": [
{
"name": "Cape buffalo",
"url": "https://upload.wikimedia.org/wikipedia/commons/c/cb/African_Buffalo.JPG",
"explanation": "The African buffalo or Cape buffalo (Syncerus caffer) is a large African bovine. It is not closely related to the slightly larger wild water buffalo of Asia and its ancestry remains unclear. Syncerus caffer caffer, the Cape buffalo, is the typical subspecies, and the largest one, found in South and East Africa. S. c. nanus (African forest buffalo) is the smallest subspecies, common in forest areas of Central and West Africa, while S. c. brachyceros is in West Africa and S. c. aequinoctialis is in the savannas of Central Africa. The adult buffalo's horns are its characteristic feature; they have fused bases, forming a continuous bone shield across the top of the head referred to as a \"boss\". They are widely regarded as very dangerous animals, as they gore and kill over 200 people every year."
}
]
},
{
"caption": "Gemsbok",
"text": "The African oryx gazelle also known as gemsbuck or gemsbok are African plains animals that travel in groups of 10-45. The gemsbuck's groups are set up with a dominant male and in most cases a few dominant females. Male's horns are straight and pointed at the tip. Because of this they have been known to impale attacking lions. Female horns can be the same but sometimes they are curved backward. There are two different varieties of gemsbok, the southern and the northern. The southern variety have longer horns and the northern have black fringed ears. The northern gemsbok are rarely seen in South Africa.",
"images": [
{
"name": "Gemsbok",
"url": "https://upload.wikimedia.org/wikipedia/commons/c/cd/Oryx_gazella_PICT1415.JPG",
"explanation": "Gemsbok are light brownish-grey to tan in colour, with lighter patches toward the bottom rear of the rump. Their tails are long and black in colour. A blackish stripe extends from the chin down the lower edge of the neck, through the juncture of the shoulder and leg along the lower flank of each side to the blackish section of the rear leg. They have muscular necks and shoulders, and their legs have white 'socks' with a black patch on the front of both the front legs, and both genders have long, straight horns. Comparably, the East African oryx lacks a dark patch at the base of the tail, has less black on the legs (none on the hindlegs), and less black on the lower flanks. One very rare condition is the \"Golden Oryx\", in which the Gemsboks black markings are muted and now appear golden. Gemsbok are the largest species in the Oryx genus. They stand about 1.2 m (3.9 ft) at the shoulder. The body length can vary from 190 to 240 cm (75 to 94 in) and the tail measures 45 to 90 cm (18 to 35 in). Male gemsbok can weigh between 180 and 240 kg (400 and 530 lb), while females weigh 100–210 kg (220–460 lb)."
}
]
},
{
"caption": "Kudu",
"text": "The kudu are split into two different groups, greater kudu and lesser kudu. The greater kudu are regularly found in South Africa. Like the gemsbok, kudu are African antelope. They are fast and stealthy. They are a brown-grey colour with white stripes that go down the centre of their body. For those two facts their African name is grey ghost. The males have tall spiraling horns, females regularly have no horns. Kudu are peaceful and are normally not dangerous.",
"images": [
{
"name": "Greater kudu",
"url": "https://upload.wikimedia.org/wikipedia/commons/c/c8/KuduKrüger.jpg",
"explanation": "Greater kudus have a narrow body with long legs, and their coats can range from brown/bluish grey to reddish brown. They possess between 4 and 12 vertical white stripes along their torso. The head tends to be darker in colour than the rest of the body, and exhibits a small white chevron which runs between the eyes."
},
{
"name": "Lesser kudu",
"url": "https://upload.wikimedia.org/wikipedia/commons/5/5e/Lesser_Kudu.jpg",
"explanation": "The lesser kudu (Tragelaphus imberbis) is a forest antelope found in East Africa. It is placed in the genus Tragelaphus and family Bovidae. It was first described by the English zoologist Edward Blyth in 1869. The head-and-body length is typically 110–140 cm (43–55 in). Males reach about 95–105 cm (37–41 in) at the shoulder, while females reach 90–100 cm (35–39 in). Males typically weigh 92–108 kg (203–238 lb) and females 56–70 kg (123–154 lb). The females and juveniles have a reddish-brown coat, while the males become yellowish grey or darker after the age of two years. Horns are present only on males. The spiral horns are 50–70 cm (20–28 in) long, and have two to two-and-a-half twists."
}
]
},
{
"caption": "Leopards",
"text": "Leopards are the most reclusive of the big cats. They are opportunistic hunters and will prey upon smaller mammals and rodents when other food sources are unavailable. The diet of leopards consists primarily of ungulates such as Thomson's gazelles. Leopards have relatively small physical builds in comparison to lions and therefore choose to hunt nocturnally to prevent the possibility of confrontation. In order to protect themselves and preserve their kills, leopards have developed exceptional climbing skills, allowing them to scale.",
"images": [
{
"name": "African leopard",
"url": "https://upload.wikimedia.org/wikipedia/commons/c/c5/Leopard_africa.jpg",
"explanation": "The leopard (Panthera pardus) /ˈlɛpərd/ is one of the five \"big cats\" in the genus Panthera. It is a member of the family Felidae with a wide range in sub-Saharan Africa and parts of Asia. Fossil records found in Italy suggest that in the Pleistocene it ranged as far west as Europe and as far east as Japan. Compared to other members of Felidae, the leopard has relatively short legs and a long body with a large skull. It is similar in appearance to the jaguar, but has a smaller, lighter physique. Its fur is marked with rosettes similar to those of the jaguar, but the leopard's rosettes are smaller and more densely packed, and do not usually have central spots as the jaguar's do. Both leopards and jaguars that are melanistic are known as black panthers."
},
{
"name": "Leopard skin",
"url": "https://upload.wikimedia.org/wikipedia/commons/5/5f/Panthera_pardus_%28Leopard_%28Kongo%29%29.jpg",
"explanation": "Dark-coloured leopard skin"
}
]
},
{
"caption": "Snakes",
"text": "Lots of frickin' dangerous snakes out there - you have been warned, dudettes!",
"images": [
{
"name": "Eastern green mamba",
"url": "https://upload.wikimedia.org/wikipedia/commons/9/9e/Mamba_Dendroaspis_angusticeps.jpg",
"explanation": "The eastern green mamba (Dendroaspis angusticeps), also known as the common mamba, East African green mamba, green mamba, or white-mouthed mamba, is a large, tree-dwelling, highly venomous snake species of the mamba genus Dendroaspis. This species of mamba was first described by a Scottish surgeon and zoologist in 1849. This snake mostly inhabits the coastal regions of southern East Africa. Adult females average approximately 2.0 metres (6.6 ft) in length, and males are slightly smaller. Eastern green mambas prey on birds, eggs, bats, and rodents such as mice, rats, and gerbils. They are shy and elusive snakes which are rarely seen, making them somewhat unusual among mambas, and elapids in general. This elusiveness is usually attributed to the species' green colouration which blends with its environment, and its arboreal lifestyle. However, eastern green mambas have also been observed to use \"sit-and-wait\" or ambush predation like many vipers, unlike the active foraging style typical of other elapids, which may be a factor in the rarity of sightings."
},
{
"name": "Black mamba",
"url": "https://upload.wikimedia.org/wikipedia/commons/7/76/Dendroaspis_polylepis_%2814%29.jpg",
"explanation": "The black mamba (Dendroaspis polylepis) is a venomous snake endemic to parts of sub-Saharan Africa. Specimens vary in color from grey to dark brown, but not black. Juvenile black mambas tend to be lighter in color than adults and darken with age. It is the longest species of venomous snake indigenous to the African continent; mature specimens generally exceed 2 meters (6.6 ft) and commonly attain 3 meters (9.8 ft). Specimens of 4.3 to 4.5 meters (14.1 to 14.8 ft) have been reported."
},
{
"name": "Cape cobra",
"url": "https://upload.wikimedia.org/wikipedia/commons/3/3c/Naja_nivea.jpg",
"explanation": "The Cape cobra (Naja nivea), also called the yellow cobra is a moderate-sized, highly venomous species of cobra inhabiting a wide variety of biomes across southern Africa including arid savanna, fynbos, bushveld, desert and semi-desert regions. The species is diurnal and is a feeding generalist, preying on a number of different species and carrion. Predators of this species include birds of prey, honey badgers and various species of mongoose. The Cape cobra is also known as the \"geelslang\" (yellow snake) and \"bruinkapel\" (brown cobra) in South Africa. Afrikaans speaking South Africans also refer to the Cape cobra as \"koperkapel\" (\"copper cobra\"), mainly because of a rich yellow colour variation. This species has no known subspecies."
}
]
}
]
}
edit:
code for image to download:
func tableView(_ tableView: UITableView, cellForRowAt indexPath: IndexPath) -> UITableViewCell {
if !(self.animals?[indexPath.item].hasMultipleImages())! {
let cell = tableView.dequeueReusableCell(withIdentifier: "animalCell", for: indexPath) as! AnimalCell
cell.name.text = self.animals?[indexPath.item].caption
cell.info.text = self.animals?[indexPath.item].text
cell.imgView.downloadImage(from: (self.animals?[indexPath.item].images?[0]["url"])!)
cell.imgView.accessibilityIdentifier = self.animals?[indexPath.item].images?[0]["name"]
cell.imgView?.isUserInteractionEnabled = true
cell.imgView?.tag = indexPath.row
let tapped:UITapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(self.TappedOnImage(sender:)))
tapped.numberOfTapsRequired = 1
cell.imgView?.addGestureRecognizer(tapped)
return cell
} else {
let cell = tableView.dequeueReusableCell(withIdentifier: "animalCellTwo", for: indexPath) as! AnimaCellTwo
cell.name.text = self.animals?[indexPath.item].caption
cell.info.text = self.animals?[indexPath.item].text
cell.imgViewLeft.downloadImage(from: (self.animals?[indexPath.item].images?[0]["url"])!)
cell.imgViewLeft.accessibilityIdentifier = self.animals?[indexPath.item].images?[0]["name"]
cell.imgViewLeft?.isUserInteractionEnabled = true
cell.imgViewLeft?.tag = indexPath.row
cell.imgViewRight.downloadImage(from: (self.animals?[indexPath.item].images?[1]["url"])!)
cell.imgViewRight.accessibilityIdentifier = self.animals?[indexPath.item].images?[1]["name"]
cell.imgViewRight?.isUserInteractionEnabled = true
cell.imgViewRight?.tag = indexPath.row
let tapped:UITapGestureRecognizer = UITapGestureRecognizer(target: self, action: #selector(self.TappedOnImage(sender:)))
tapped.numberOfTapsRequired = 1
cell.imgViewLeft?.addGestureRecognizer(tapped)
cell.imgViewRight?.addGestureRecognizer(tapped)
return cell
}
}
basically I have a uitableview with 2 different prototype tableviewcells:
one cell has one image + 2 labes , and the other has 2 images + 2 labels
I am trying to load them to the table I also need to implement caching for images to prevent downloading them everytime upon scrolling, I hope anyone could help I am trying to solve this for 2 days now... :|

Try to encode your image string
let encodedHost = imageUrl.addingPercentEncoding(withAllowedCharacters: .urlHostAllowed)
OR
Replace character
imageUrl = imageUrl.replacingOccurrences(of: "%25", with: "", options: .literal, range: nil)

Related

control over Vertex AI annotation

In the Google Vertex AI platform, the configuration of labelling the data set is decided by the vertex AI itself, ie. for example the color which is given to the bounding box while labelling is decided by vertex ai, what if I want to assign the color as my preference? I couldn't see any option to change the annotation color and also tried to feed externally labelled files to the vertex ai and it shows an error as there is no field named color;
{
"imageGcsUri": "gs://cloud-ai-platform-5100f6e6-d2e6-4966-869e-ba22a09ef85a/bg_MAX_0002.JPG",
"boundingBoxAnnotations": [
{
"displayName": "abc",
"xMin": 0.07510431154381085,
"xMax": 0.34492350486787204,
"yMin": 0.1022964509394572,
"yMax": 0.4384133611691023,
"annotationResourceLabels": {
"aiplatform.googleapis.com/annotation_set_name": "3765893295830466560"
}
},
{
"displayName": "pqr",
"xMin": 0.6801112656467315,
"xMax": 0.9318497913769124,
"yMin": 0.1503131524008351,
"yMax": 0.6367432150313153,
"annotationResourceLabels": {
"aiplatform.googleapis.com/annotation_set_name": "3765893295830466560"
}
},
{
"displayName": "xyz",
"xMin": 0.15438108484005564,
"xMax": 0.6620305980528511,
"yMin": 0.605427974947808,
"yMax": 0.906054279749478,
"annotationResourceLabels": {
"aiplatform.googleapis.com/annotation_set_name": "3765893295830466560"
}
}
],
"dataItemResourceLabels": {
}
}
Given above is an example label generated by vertex ai and it doesn't contain any color information even though I've labelled in multiple colors.
So my question is how can we get control over the color feature of the labelling system in vertex AI?

How to replace parts of Text with TextFields in Jetpack Compose?

For example i have this:
TAKE, O take those lips away
That so sweetly were forsworn,
And those eyes, the break of day,
Lights that do mislead the morn:
But my kisses bring again,
Bring again—
Seals of love, but seal’d in vain,
Seal’d in vain!
– William Shakespeare
And i want to replace some words to TextField(in Compose) or to EditText(in XML).
Example:
TAKE, O take those lips textfield
That so were forsworn,
And those eyes, the break of day,
Lights that do mislead the morn:
But my textfield bring again,
Bring again—
textfield of love, textfield seal’d in vain,
Seal’d in vain!
– William Shakespeare
Can you advise me the way to realise it? Maybe specific libraries?
First of all, I highlighted the words to be replaced by * so that they can be easily found using a regular expression.
val string = """
TAKE, O take those lips *away*
That so sweetly were forsworn,
And those eyes, the break of day,
Lights that do mislead the morn:
But my *kisses* bring again,
Bring again—
*Seals* of love, *but* seal’d in vain,
Seal’d in vain!
– William Shakespeare
""".trimIndent()
val matches = remember(string) { Regex("\\*\\w+\\*").findAll(string) }
Using the onTextLayout argument in Compose Text, you can get a lot of information about the text to be rendered, including the positions of each character. And the indexes of the characters you need to replace are already defined by a regular expression.
All you have to do is place the text fields at the appropriate positions.
I use BasicTextField because it doesn't have the extra padding that TextField has, so the size is easy to match with Text. I set its background to white so that the original text doesn't shine through. If you have an unusual background, a gradient for example, you can also make the text transparent with annotated text as shown in the documentation, then the BasicTextField can be left transparent.
The SubcomposeLayout is a great tool for creating such layouts without waiting a next recomposition to use onTextLayout result.
val textLayoutState = remember { mutableStateOf<TextLayoutResult?>(null) }
val textFieldTexts = remember(matches.count()) { List(matches.count()) { "" }.toMutableStateList() }
val style = MaterialTheme.typography.body1
SubcomposeLayout { constraints ->
val text = subcompose("text") {
Text(
text = string,
style = style,
onTextLayout = {
textLayoutState.value = it
},
)
}[0].measure(constraints)
val textLayout = textLayoutState.value ?: run {
// shouldn't happen as textLayoutState is updated during sub-composition
return#SubcomposeLayout layout(0, 0) {}
}
val wordsBounds = matches.map {
// I expect all selected words to be on a single line
// otherwise path bounds will take both lines
textLayout
.getPathForRange(it.range.first, it.range.last + 1)
.getBounds()
}
val textFields = wordsBounds.mapIndexed { i, wordBounds ->
subcompose("textField$i") {
BasicTextField(
value = textFieldTexts[i],
onValueChange = {
textFieldTexts[i] = it
},
onTextLayout = {
println("${it.size}")
},
textStyle = style,
modifier = Modifier
.border(1.dp, Color.LightGray)
.background(Color.White)
)
}[0].measure(Constraints(
maxWidth = floor(wordBounds.width).toInt(),
)) to wordBounds.topLeft
}
layout(text.width, text.height) {
text.place(0, 0)
textFields.forEach {
val (placeable, position) = it
placeable.place(floor(position.x).toInt(), floor(position.y).toInt())
}
}
}
Result:

How to spatially plot an attribute parameter?

Here is my code below. I want to plot CO2 distribution over the South African map.
Loading packages
library(tidyverse)
theme_set(theme_bw())
library("sf")
library("rnaturalearth")
library("rnaturalearthdata")
Assigning world to the countries of the world data
world <- ne_countries(scale = "medium", returnclass = "sf")
Reading the local file with CO2 and geographic coordinates (Lon and Lat)
SA_CO2 <- read_csv2("C:/Users/Xolile Ncipha/Documents/SA_CO2_DJF_2004_2009.csv")
Converting the data frame to a sf object and the coordinate reference system projection to WGS84, which is the CRS code #4326.
(SA_CO2 <- st_as_sf(SA_CO2, coords = c("Lon", "Lat"), crs = 4326, agr = "constant"))
Plotting the map and overlaying it with CO2 dataThe output of my code/script.
ggplot(data = world) + geom_sf() + geom_sf(data = SA_CO2, aes(fill = CO2)) +
CO2 legend
scale_fill_gradientn(colors = sf.colors(10)) +
Confining the map to South African domain.
coord_sf(xlim = c(15, 35), ylim = c(-36, -22.3), expand = FALSE) +
Axis labels
xlab("Longitude") + ylab("Latitude")
The results is the geographic points on the map. I don't get the overlay of CO2 data and its spatial distribution. I have attached a picture of the resulting map and the spatial data.
This is a good question. But, unfortunately, you did not provide a link to the source of your data ( "SA_CO2_DJF_2004_2009.csv"), so it was necessary for me to create some data that is likely somewhat similar to your data, but probably not exactly the same.
I found a spelling error in the following line of your code. Even after correcting the spelling error, this line of code continued causing an error.
scale_fill_gradientn(colors = sf.colors(10)) +
The data I used included tons of CO2 from SA during the years, 2010-2016. I also selected a few SA cities with populations from another source, and then allocated the SA annual CO2 by the ratio of the combined city populations. Therefore, those cities with smaller populations were allocated smaller proportions of the annual CO2 and the larger cities were allocated larger ratio's of CO2.
https://howsouthafrica.com/major-cities-international-airports-south-africa/
https://data.worldbank.org/indicator/EN.ATM.CO2E.KT?locations=ZA&view=map
The code used to create the plot shown at the link below is:
ggplot(data = world) +
geom_sf() +
geom_sf(data = coor.sf, aes(size = tons),
fill = "blue", color = "blue", alpha = .3) +
coord_sf(xlim = c(15, 35), ylim = c(-36, -22.3),
expand = FALSE) + xlab("Longitude") + ylab("Latitude")
Please email me if you have any questions.
[[![SA CO2]]

How to remove unnecessary symbols from attributedText converted from html?

I am using this code to convert a html-text string into an attributedString.
if let description = article.description {
let data = Data(description.utf8)
if let attributedString = try? NSMutableAttributedString(data: data, options: [.documentType: NSAttributedString.DocumentType.html], documentAttributes: nil) {
attributedString.addAttributes([NSAttributedString.Key.font: UIFont.systemFont(ofSize: 18), NSAttributedString.Key.foregroundColor: UIColor.subtitleColor], range: NSRange(location: 0, length: attributedString.length))
articleView.contentTextView.attributedText = attributedString
}
}
and a weird symbols coming out like here.
How to remove them?
description is a
<p>Greece prepares to end lockdown; Bill Gates vows to fund vaccine production; Australia and New Zealand mark Anzac Day from driveways</p><ul><li>Leader of group peddling bleach as‘cure’ wrote to Trump this week<br></li><li>Global report: Sweden queries lockdowns as Germany keeps guard up</li><li>Coronavirus latest: at a glance</li><li>US coronavirus updates– live<br></li><li>See all our coronavirus coverage</li></ul><p class="block-time published-time"><time datetime="2020-04-25T14:09:45.760Z">3.09pm<span class="timezone">BST</span></time></p><p>Today is World Malaria Day. As we approach 200,000 deaths so far this year from Covid-19, it’s worth remembering that the World Health Organization has warned thatdeaths from malaria could double to 700,000 this yearas a result of the disruption caused by the new disease.</p><p lang="en" dir="ltr">As we work on tackling#COVID19, we can't afford to lose ground on the gains we have made against other diseases like malaria.<br><br>On#WorldMalariaDayI join#WHO's call for maintained malaria prevention& treatment services during the#coronaviruscrisis to save lives.pic.twitter.com/vtm3X6twva</p><p><span>Related:</span>Pandemic could 'turn back the clock' 20 years on malaria deaths, warns WHO</p><p class="block-time published-time"><time datetime="2020-04-25T13:53:37.287Z">2.53pm<span class="timezone">BST</span></time></p><p>Dozens of doctors and nurses in<strong>Pakistan</strong>have launched a hunger strike over a lack of protective masks and other equipment for treating patients with Covid-19.</p><p>More than 150 doctors in Pakistan have tested positive for coronavirus, according to the Young Doctors Association in Punjab, the country’s worst-hit province.<br tabindex="-1"></p>Continue reading...

ARKit Face Tracking SceneKit Object Moves Incorrectly

I'm having trouble understanding SceneKit transforms and anchoring an object to a detected face. I have created a face detection app and have successfully applied masks, with and without texture. I also successfully applied "glasses" made from text ( "00" ) including an occlusion node.
In both cases, the objects move with the face as expected. However, when I create a simple hat made from two cylinders within ScendKit the behavior is totally unexpected.
First, I could not seem to anchor the hat to the face, but had to adjust the transforms which made the hat appear in a different place with almost every face. Even worse, the hat moves in the opposite direction to the face. Rotate the user face to the left, the hat moves to the right. Rotate the face up, the hat moves down.
Clearly, I'm missing something important here about anchoring objects to the face. Any guidance would be appreciated.
Xcode 10 beta 3, iOS 11.4.1 running on an iPhone X.
There is a separate class for hat, glasses, mask:
class Hat : SCNNode {
init(geometry : ARSCNFaceGeometry) {
geometry.firstMaterial?.colorBufferWriteMask = []
super.init()
self.geometry = geometry
guard let url = Bundle.main.url(forResource: "hat", withExtension: "scn", subdirectory: "Models.scnassets") else {fatalError("missing hat resource")}
let node = SCNReferenceNode(url: url)!
node.load()
addChildNode(node)
}//init
func update(withFaceAnchor anchor : ARFaceAnchor) {
let faceGeometry = geometry as! ARSCNFaceGeometry
faceGeometry.update(from: anchor.geometry)
}//upadate
required init?(coder aDecoder: NSCoder) {
fatalError("(#function) has not been implemented")
}//r init
}//class
A couple of the functions in the ViewController:
func renderer(_ renderer: SCNSceneRenderer, didUpdate node: SCNNode, for anchor: ARAnchor) {
guard let faceAnchor = anchor as? ARFaceAnchor else {return}
updateMessage(text: "Tracking your face")
switch contentTypeSelected {
case .none:
break
case .mask:
mask?.update(withFaceAnchor: faceAnchor)
case .glasses:
glasses?.update(withFaceAnchor: faceAnchor)
case .hat:
hat?.update(withFaceAnchor: faceAnchor)
}//switch
}//didUpdate
func createFaceGeometry() {
updateMessage(text: "Creating face geometry")
let device = sceneView.device!
let maskGeometry = ARSCNFaceGeometry(device: device)!
mask = Mask(geometry: maskGeometry, maskType : maskType)
let glassesGeometry = ARSCNFaceGeometry(device: device)!
glasses = Glasses(geometry: glassesGeometry)
let hatGeometry = ARSCNFaceGeometry(device: device)!
hat = Hat(geometry: hatGeometry)
}//createFaceGeometry
The verisimilitude of a hat is going to depend on how well it can be position in relation to the face and how well it can appear situated in the scene (i.e. features that would be in front of the hat should occlude the hat itself). With that in mind, you'll want the face to occlude the hat. So your init for Hat should setup an occlusion node using the face geometry:
let occlusionNode: SCNNode
init(geometry: ARSCNFaceGeometry) {
/*
Taken directly from Apple's sample code https://developer.apple.com/documentation/arkit/creating_face_based_ar_experiences
*/
geometry.firstMaterial!.colorBufferWriteMask = []
occlusionNode = SCNNode(geometry: geometry)
occlusionNode.renderingOrder = -1
super.init()
addChildNode(occlusionNode)
guard let url = Bundle.main.url(forResource: "hat", withExtension: "scn", subdirectory: "Models.scnassets") else {fatalError("missing hat resource")}
let node = SCNReferenceNode(url: url)!
node.load()
addChildNode(node)
}
This will allow the face to appear in front of any virtual objects that have a z depth greater than the face mesh.
You will also want change let hatGeometry = ARSCNFaceGeometry(device: device)! to let hatGeometry = ARSCNFaceGeometry(device: device, fillMesh: true)! otherwise the hat will be visible through the eyes giving an uncanny, undesirable effect.
The next issue is to position the hat so that it appears believably in the scene.
Because we want the face to occlude a large part of the hat, it is best to position it in the y direct at the top of the face geometry. To do that successfully, you'll likely want your hat to have pivot point at the bottom center of the hat geometry and located at x = 0, y = 0 in your .scn file. For example the scene editor and node inspector might look something like:
Then in your func update(withFaceAnchor anchor : ARFaceAnchor) you can say
func update(withFaceAnchor anchor : ARFaceAnchor) {
let faceGeometry = geometry as! ARSCNFaceGeometry
faceGeometry.update(from: anchor.geometry)
hat.position.y = faceGeometry.boundingSphere.radius
}
Finally for the z position of the hat you'll like want a slightly negative value as the bulk of a hat is behind one's face. -0.089 worked well for me.

Resources