Testing for characters in right-to-left languages using STRING MATCH and REGEXP; and encodings? - character-encoding

Question

I'm trying to test for a character at the end of a right-to-left language, Hebrew, in Tcl?
And I think it may be further complicated because the Hebrew was passed in JSON; but I'm not sure because I'm still quite confused by encodings.
I've been trying the following code on some tests strings and, although I think I understand why the regular expressions at the bottom "work", I don't understand why the {*־} in string match provides the desired result.
set hebJoined עַל־הָאָרֶץ
set hebSep עַל־
chan puts stdout [string match {*־} $hebJoined]; #=> 0
chan puts stdout [string match {^*־} $hebJoined]; #=> 0
chan puts stdout [string match {*^־} $hebJoined]; #=> 0
chan puts stdout [string match {*־^} $hebJoined]; #=> 0
chan puts stdout [string match {$*־} $hebJoined]; #=> 0
chan puts stdout [string match {*־$} $hebJoined]; #=> 0
chan puts stdout [string match {*$־} $hebJoined]; #=> 0
chan puts stdout [string match {*־} $hebSep]; #=> 1
chan puts stdout [string match {^*־} $hebSep]; #=> 0
chan puts stdout [string match {*^־} $hebSep]; #=> 0
chan puts stdout [string match {*־^} $hebSep]; #=> 0
chan puts stdout [string match {$*־} $hebSep]; #=> 0
chan puts stdout [string match {*־$} $hebSep]; #=> 0
chan puts stdout [string match {*$־} $hebSep]; #=> 0
chan puts stdout [regexp {(.*־)(.*)} $hebJoined {\1\2} heb1 heb2]
chan puts stdout $heb1; # => עַל־
chan puts stdout $heb2; # => הָאָרֶץ
chan puts stdout [regexp {(.*־)$} $hebJoined]; # 0
chan puts stdout [regexp {(.*־)$} $hebSep]; # 1
Then there is the real larger issue that I am working with data that was passed as JSON and the regular expressions above will not provide the desired result but a modification of the string match does.
string match {*־} [encoding convertto iso8859-1 $hebrew] appears to find all the words that end in a hyphen; that is, on the left-hand side; and does not return results for hyphens in the middle of a string. And I do not understand why it does so. I don't know how to provide an example because the stored data for the Hebrew looks like עַל.
Can string match or regular expression test for a unicode value like \u05BE which is what I think this hyphen is?
Would you please tell me why the code I've used seems to work and how I can correct this to work properly? If the change the encoding to utf-8, then the string match does not provide any matches.
Thank you.
EDIT:
I think this is what is needed. I was confused for awhile partly because I was looking at a file that purposely leaves off the hyphen. This code yields correct results but is ugly and likely not the best approach.
chan puts stdout [regexp {(×¢Ö·×Ö¾)} [encoding convertto utf-8 $hebJoined] {\1} h1]; # => 1
chan puts stdout [regexp {(Ö¾)$} [encoding convertto utf-8 $hebJoined] {\1} h2]; # => 0
chan puts stdout [regexp {(Ö¾)$} [encoding convertto utf-8 $hebSep] {\1} h2]; # => 1
chan configure stdout -encoding iso8859-1 -translation crlf
chan puts stdout $h1; # => עַל־
chan puts stdout $h2; # => ־ the desired hyphen.
ANOTHER EDIT:
I was making a serious mistake in reading this data into Tcl as iso8859-1 instead of utf-8. If change the encoding of the channel receiving the data to utf-8, then most of these issues disappear altogether; and testing with the unicode values like \U05BE works nicely. In this particular case, my error of reading utf-8 as iso8859-1 appears to have resulted in the multi-byte characters being read as individual bytes and that complicated the matching inn string match and regexp.

Answer 1

Can string match or regular expression test for a unicode value like \u05BE which is what I think this hyphen is?
Sure. It is just a conventional character to both of those matching systems. Indeed, they work on the logical sequence of characters, so that's what you should always use; which way they display is an output rendering problem, and Tcl itself says nothing about that. (Your terminal, or a GUI toolkit like Tk, would care more.)