I have two strings encoded differently for spacing:
let first = "https://joytst.page.link/zMx3nAj9DxwcE1JC9?title=New+calendar+test"
let second = "https://joytst.page.link/zMx3nAj9DxwcE1JC9?title=New%20calendar%20test"
let firstOutput = first.removingPercentEncoding //https://joytst.page.link/zMx3nAj9DxwcE1JC9?title=New+calendar+test
let secondOutput = second.removingPercentEncoding //https://joytst.page.link/zMx3nAj9DxwcE1JC9?title=New calendar test
Why it doesn't remove encoding correctly, since + is a correct encoding for space?
How can I correctly decode both of them, no matter which one I receive?
“Why” is a difficult question to answer except for the people who had a hand in implementing CFURLCreateStringByReplacingPercentEscapes. The fact is that it doesn't.
I can speculate that it doesn't because the + for space replacement is not part of RFC 3986: Uniform Resource Identifier (URI): Generic Syntax. It should only be used in the query part of the URL, which is of type application/x-www-form-urlencoded. But this is just my guess.
Anyway, if you want to convert + to space, you should do so before performing percent-decoding, lest you decode %2b into + and then further decode it into a space, leaving no way for your URL to contain a genuine + after decoding.
let firstOutput = first
.replacingOccurrences(of: "+", with: " ")
.removingPercentEncoding
If you could decode a whitespace from two different patters, then when you wanted to do the opposite and encode it which one it should take?
That's the reason why removingPercentEncoding only supports one of them.
Related
Here is a weird problem in iOS 14.1/Swift 5 that caused many griefs and consumed hours, until I understood what's going on. I still want to know if this is a bug or "by design". In the latter case, please provide a link describing the behavior and provide a list of other characters that are not counted.
Let us assume that I have a string like below:
let data = "l1\r\nl2\r\nl3"
I need to create an HTTP response manually and replace the Content-Length with the data length. I use a template for that:
static let RESPONSE =
"""
HTTP/1.1 200 OK
Content-Length: LENGTH
Content-Type: text/plain
DATA
""".trimmingCharacters(in: .whitespaces)
Finally, I create a response from a template:
let response = RESPONSE.replacingOccurrences(of: ": LENGTH", with: ": \(data.count)")
.replacingOccurrences(of:"DATA", with:data)
As a result, Content-Length was set to 8, not to 10, and a client didn't receive "l3".
Please note that the string with carriage returns has been generated by Apple's own BASE64 API, so there is nothing "special" that I did here myself:
Data(digest).base64EncodedString(options: [.lineLength64Characters])
Very strange behavior that I didn't see in any other languages.
Swift treats the combination of \r\n as a single newline character (abbreviated in the docs to CR-LF).
let combo = Character('\r\n')
print(combo.isNewline) // true
So when you convert this Character to a String and count it you get the answer one.
print(String(combo).count) // 1
Character has no count because by definition it represents a single user-perceived character even if it is constructed from a number of components.
I guess Swift's developers decided that the count property of String should output the number of user perceived characters, and since \r\n to all intents and purposes has the same effect has a single newline character it is counted as a single character.
Note however that String does not throw away the data from which it was constructed; you can still get the 'raw' count property that is most relevant to your case via the unicodeScalars property.
let data = "l1\r\nl2\r\nl3"
print(data.count) // 8
print(data.unicodeScalars.count) // 10
By the way, it's not just CR-LF that gets this special treatment; national flag emojis are a single user perceived character that are actually composed of two scalars.
let unionJack = Character("🇬🇧")
for scalar in unionJack.unicodeScalars {
print(String(scalar.value, radix: 16).uppercased() )
}
// 1F1EC
// 1F1E7
Change data.count to data.utf16.count in order to get the "outside world" view of how "long" the string is.
(Alternatively you could say (data as NSString).length.)
I am working on string manipulation using LUA and having trouble with the following problem.
Using this as an example of the original data I am given -
"[0;1;36m(Web): You say, "Text here."[0;37m"
I want to keep the string intact except for removing the ANSI codes.
I have been pointed toward using gsub with the LUA pattern matching but I cannot seem to get the pattern correct. I am also unsure how to reference exactly the escape character sent.
text:gsub("[\27\[([\d\;]+)m]", "")
or
text:gsub("%x%[[%d+;+]m", "")
If successful, all I want to be left with, using the above example, would be:
(Web): You say, "Text here."
Your string example is missing the escape character, ASCII 27.
Here's one way:
s = '\x1b[0;1;36m(Web): You say, "Text here."\x1b[0;37m'
s = s:gsub('\x1b%[%d+;%d+;%d+;%d+;%d+m','')
:gsub('\x1b%[%d+;%d+;%d+;%d+m','')
:gsub('\x1b%[%d+;%d+;%d+m','')
:gsub('\x1b%[%d+;%d+m','')
:gsub('\x1b%[%d+m','')
print(s)
I found some weirdest thing in Firebase Database/Storage. The thing is that I don't know if Firebase or Swift is not detecting umlauts e.g(ä, ö, ü).
I did some easy things with Firebase like upload images to Firebase Storage and then download them into tableview. Some of my .png files had umlauts in the title for example(Röda.png).
So the problem occurs now if I download them. The only time my download url is nil is if the file name contains the umlauts I was talking about.
So I tried some alternatives like in HTML ö - ö. But this is not working. Can you guys suggest me something? I can't use ö - o, ü - u etc.
This is the code when url is nil when trying to set some values into Firebase:
FIRStorage.storage().reference()
.child("\(productImageref!).png")
.downloadURLWithCompletion({(url, error)in
FIRDatabase.database().reference()
.child("Snuses").child(productImageref!).child("productUrl")
.setValue(url!.absoluteString)
let resource = Resource(downloadURL: url!, cacheKey: productImageref)
After spending a fair bit of time research your problem, the difference boils down to how the character ö is encoded and I traced it down to Unicode normalization forms.
The letter ö can be written in two ways, and String / NSString considers them equal:
let str1 = "o\u{308}" // decomposed : latin small letter o + combining diaeresis
let str2 = "\u{f6}" // precomposed: latin small letter o with diaeresis
print(str1, str2, str1 == str2) // ö ö true
But when you percent-encode them, they produce different results:
print(str1.stringByAddingPercentEncodingWithAllowedCharacters(.URLPathAllowedCharacterSet())!)
print(str2.stringByAddingPercentEncodingWithAllowedCharacters(.URLPathAllowedCharacterSet())!)
// o%CC%88
// %C3%B6
My guess is that Google / Firebase chooses the decomposed form while Apple prefers the other in its text input system. You can convert the file name to its decomposed form to match Firebase:
let str3 = str2.decomposedStringWithCanonicalMapping
print(str3.stringByAddingPercentEncodingWithAllowedCharacters(.URLPathAllowedCharacterSet()))
// o%CC%88
This is irrelevant for ASCII-ranged characters. Unicode can be very confusing.
References:
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (highly recommended)
Strings in Swift 2
NSString and Unicode
Horray for Unicode!
The short answer is that no, we're actually not doing anything special here. Basically all we do under the hood is:
// This is the list at https://cloud.google.com/storage/docs/json_api/ without the & because query parameters
NSString *const kGCSObjectAllowedCharacterSet =
#"ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-._~!$'()*+,;=:#";
- (nullable NSString *)GCSEscapedString:(NSString *)string {
NSCharacterSet *allowedCharacters =
[NSCharacterSet characterSetWithCharactersInString:kGCSObjectAllowedCharacterSet];
return [string stringByAddingPercentEncodingWithAllowedCharacters:allowedCharacters];
}
What blows my mind is that:
let str1 = "o\u{308}" // decomposed : latin small letter o + combining diaeresis
let str2 = "\u{f6}" // precomposed: latin small letter o with diaeresis
print(str1, str2, str1 == str2) // ö ö true
returns true. In Objective-C (which the Firebase Storage client is built in), it totally shouldn't, as they're two totally different characters (in actuality, the length of str1 is 2 while the length of str2 is 1 in Obj-C, while in Swift I assume the answer is 1 for both).
Apple must be normalizing strings before comparison in Swift (probably a reasonable thing to do, since otherwise it leads to bugs like this where strings are "the same" but compare differently). Turns out, this is exactly what they do (see the "Extended Grapheme Clusters" section of their docs).
So, when you provide two different characters in Swift, they're being propagated to Obj-C as different characters and thus are encoded differently. Not a bug, just one of the many differences between Swift's String type and Obj-C's NSString type. When in doubt, choose a canonical representation you expect and stick with it, but as a library developer, it's very hard for us to choose that representation for you.
Thus, when naming files that contain Unicode characters, make sure to pick a standard representation (C,D,KC, or KD) and always use it when creating references.
let imageName = "smorgasbörd.jpg"
let path = "images/\(imageName)"
let decomposedPath = path.decomposedStringWithCanonicalMapping // Unicode Form D
let ref = FIRStorage.storage().reference().child(decomposedPath)
// use this ref and you'll always get the same objects
decodedData is nil but My base64String contains an extremely long string
Encode
var imgProfile:NSData = UIImagePNGRepresentation(imgUI)
let base64String = imgProfile.base64EncodedStringWithOptions(.allZeros)
Decode
let base64String = prefs.valueForKey("imgDefault") as? String
let decodedData = NSData(base64EncodedString: base64String!, options: NSDataBase64DecodingOptions(rawValue: 0) )
var decodedimage = UIImage(data: decodedData!)
I"m having trouble outputting my image from base64
base64 string ENCODE before inserting into db:
iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAMAAADDpiTIAAAADFBMVEXFxcX////p6enW1tbAmiBwAAAAHGlET1QAAAACAAAAAAAAAQAAAAAoAAABAAAAAQAAAAYAppse6QAABcxJREFUeAHs3et22jAQReEQ3v+di8sikDaxZeGLdPTxpzS202jP9pwRkNWPDw8EEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQiCVwvV4/p8fl9fH3K7cjsau2sBuBqfKvVf/x+ecnDQJtKSn9qw83DQIpDLqktcV/iECCAGFuXf9Rz6o/5UHPEtTe+t9V0Qj6dGCb6t9d4EBvDrzb+b/3gOlvsqAjB97M/f+rf/+KjUEfDuxU/nsb6APBwD/llsn/UycwDTQt1443/9MGSdCqA4eUXxK0Wv6P917xed7hJc8+m6Uw7A922O1/90MOtGXaweWXA8OXnwINKXAtSe1dzvGWcQManND9nzIZBU434MjZ/1n45zP7gVMVOPX2v1ugCZxowHnp/2wBl4tJ4CwFzm7/DwvEwCkGNND+vwzQBI5XoI32/1CAAUcb0Er7fxggBg41oKH2/2WAJnCcAg3Wf/rU4HEABv+Xmqw/Aw6zsq3x7xEB0596wBEStFt/Boxefwbsb0Br27/XBJie2w7u60Dr9WfA6PVnwJ4GtDz/PbPAXmAvB/qov0lw9PozYB8Dern/pyyQAts70Ojrv8/of33mfYHNBeiq/t4X2L
base64 string DECODE when pulling down from db:
Optional("iVBORw0KGgoAAAANSUhEUgAAAgAAAAIACAMAAADDpiTIAAAADFBMVEXFxcX////p6enW1tbAmiBwAAAAHGlET1QAAAACAAAAAAAAAQAAAAAoAAABAAAAAQAAAAYAppse6QAABcxJREFUeAHs3et22jAQReEQ3v
di8sikDaxZeGLdPTxpzS202jP9pwRkNWPDw8EEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQQAABBBBAAAEEEEAAAQQQiCVwvV4/p8fl9fH3K7cjsau2sBuBqfKvVf/x
ecnDQJtKSn9qw83DQIpDLqktcV/iECCAGFuXf9Rz6o/5UHPEtTe
t9V0Qj6dGCb6t9d4EBvDrzb b/3gOlvsqAjB97M/f rf/
KjUEfDuxU/nsb6APBwD/llsn/UycwDTQt1443/9MGSdCqA4eUXxK0Wv6P917xed7hJc8
m6Uw7A922O1/90MOtGXaweWXA8OXnwINKXAtSe1dzvGWcQManND9nzIZBU434MjZ/1n45zP7gVMVOPX2v1ugCZxowHnp/2wBl4tJ4CwFzm7/DwvEwCkGNND
vwzQBI5XoI32/1CAAUcb0Er7fxggBg41oKH2/2WAJnCcAg3Wf/rU4HEABv
Xmqw/Aw6zsq3x7xEB0596wBEStFt/Boxefwbsb0Br27/XBJie2w7u60Dr9WfA6PVnwJ4GtDz/PbPAXmAvB/qov0lw9PozYB8Dern/pyyQAts70Ojrv8/of33mfYHNB
There are two different problems:
It would appear that the + characters have been replaced with spaces. That will happen if you submit an application/x-www-form-urlencoded request without percent escaping the + characters. This probably happened when you first sent the base64 string to be stored in the database.
See https://stackoverflow.com/a/24888789/1271826 for a discussion of some percent encoding patterns. The key point here is to not rely upon stringByAddingPercentEscapesUsingEncoding, because that will allow + characters to go unescaped.
The string is also missing the trailing = characters. (The string's length should be a multiple of four, and in this case, it's two characters short, so there should be == at the end of the rendition with the + characters in it (the "before" string). While that is sometimes a mistake made by poorly designed base64-encoders, this is not a problem that base64EncodedStringWithOptions suffers from.
In this case, it looks like a much longer base64 string must have been truncated somehow. (Your strings are suspiciously close to 1024 characters. lol.) This truncation could happen if you put the parameters in the URL rather than the body of the request. But nothing in this code sample would account for this behavior, so the problem rests elsewhere.
But look at the length of the original NSData. The base64 string should be 1/3 larger than that (plus rounded up to the nearest four characters, once you include the trailing = characters).
And, once you decode the string that you've provided and look at the actual contents, you can also see that the base64 string was truncated. (According to the portion provided, there should be 1484 bytes of IDAT data, and there's not, plus there's no IEND chunk ... don't worry about those details, but rest assured that it's basically saying that the PNG data stream is incomplete.)
If you're getting nil returned, then your base64 string is not valid. NSData(base64EncodedString:options:) requires a base64 string that is padded with = to a multiple of 4.
Here's a similar issue (except in Obj-C).
NSData won't accept valid base64 encoded string
I have a requirement to escape a string with url information but also some special characters such as '<'.
Using cl_http_utility=>escape_url this translates to '%3c'. However due to our backend webserver, it is unable to recognize this as special character and takes the value literally. What it does recognize as special character is '%3C' (C is upper case). Also if one checks http://www.w3schools.com/tags/ref_urlencode.asp it shows the value with all caps as the proper encoding.
I guess my question is is there an alternative to cl_http_utility=>escape_url that does essentially the same thing except outputs the value in upper case?
Thanks.
Use the string function.
l_escaped = escape( val = l_unescaped
format = cl_abap_format=>e_url ).
Other possible formats are e_url_full, e_uri, e_uri_full, and a bunch of xml/json stuff too. The string function escape is documented pretty well, demo programs and all.