My title may be misunderstood, I describe it:
Can be Chinese/Japanese or any other country's language, such as 你好 or こんにちは
Can be English letters, A-Z or a-z
Can't be a symbol, such as ! or !,, or ,
Can't be special characters such as Emoji or other symbols
Can it be judged by the binary byte number of elixir, or by Unicode?
If I understood your question well, you want to check if a given string contains Chinese/Japanese characters, or alphabetical characters. but not punctuation or emoji?
For the Asian characters you can use the CJK range from Unicode, which might close enough. You can always check more ranges for languages you want to (dis)allow.
So the first step would be to check if a given code point is in the CJK range(s):
def is_in_range?(cp) do
ranges = [
{"\u4E00", "\u9FEF"},
{"\u3400", "\u4DBF"},
{"\u20000", "\u2A6DF"},
{"\u2A700", "\u2B73F"},
{"\u2B740", "\u2B81F"},
{"\u2B820", "\u2CEAF"},
{"\u2CEB0", "\u2EBEF"},
{"\u3007", "\u3007"}
]
# Check if the codepoint is any of the ranges above.
ranges
|> Enum.map(fn {s, e} ->
cp >= s and cp <= e
end)
|> Enum.any?()
end
If we have that function, we can check for any given string if it contains any of these characters:
def contains_cjk(str) do
str |> String.codepoints() |> Enum.map(&is_in_range?/1) |> Enum.any?()
end
If you want to compare alpha characters you can use a regular regex, or just add the range from A-Z, and a-z (\u0061 to \u007A, and \u0041 to \u005A). For example, your second string (こんにちは) its first code point is in the "Hiragana" code block. You could add the range (\u3040 to \u309F) to also allow these characters. A listing of blocks can be found here.
A note on performance is in place here. This code is not linear, as for n characters it will do #amount_of_chars_in_range comparisons.
Related
The Lua manual in section 6.4.1 on Lua Patterns states
A character class is used to represent a set of characters. The
following combinations are allowed in describing a character class:
x: (where x is not one of the magic characters ^$()%.[]*+-?) represents the character x itself.
.: (a dot) represents all characters.
%a: represents all letters.
%c: represents all control characters.
%d: represents all digits.
%g: represents all printable characters except space.
%l: represents all lowercase letters.
%p: represents all punctuation characters.
%s: represents all space characters.
%u: represents all uppercase letters.
%w: represents all alphanumeric characters.
%x: represents all hexadecimal digits.
%x: (where x is any non-alphanumeric character) represents the character x. This is the standard way to escape the magic characters.
Any non-alphanumeric character (including all punctuation characters,
even the non-magical) can be preceded by a % when used to represent
itself in a pattern.
[set]: represents the class which is the union of all characters in set. A range of characters can be specified by separating the end
characters of the range, in ascending order, with a -. All classes
%x described above can also be used as components in set. All other
characters in set represent themselves. For example, [%w_] (or
[_%w]) represents all alphanumeric characters plus the underscore,
[0-7] represents the octal digits, and [0-7%l%-] represents the
octal digits plus the lowercase letters plus the - character.
You can put a closing square bracket in a set by positioning it as the
first character in the set. You can put a hyphen in a set by
positioning it as the first or the last character in the set. (You can
also use an escape for both cases.)
The interaction between ranges and classes is not defined. Therefore, patterns like [%a-z] or [a-%%] have no meaning.
[^set]: represents the complement of set, where set is interpreted
as above.
For all classes represented by single letters (%a, %c, etc.), the
corresponding uppercase letter represents the complement of the class.
For instance, %S represents all non-space characters.
The definitions of letter, space, and other character groups depend on
the current locale. In particular, the class [a-z] may not be
equivalent to %l.
(Highlighting and some formatting added by me)
So, since the "interaction between ranges and classes is not defined.", how do you create a character class set that starts and/or ends with a (magic) character that needs to be escaped?
For example,
[%%-c]
does not define a character class that ranges from % to c and includes all characters in-between but a set that consists only of the three characters %, -, and c.
The interaction between ranges and classes is not defined.
Obviously, this is not a hard and fast rule (of regex character sets in general) but a Lua implementation decision. While using shorthand characters in character sets/ranges work in some (most) regex flavors, it does not in all (like in Python's re module, demo).
However, the second example is misleading:
Therefore, patterns like [%a-z] or [a-%%] have no meaning.
While the first example is fine since %a is a shorthand class (that represents all letters) in a set, [%a-z] is undefined and will return nil if matched against a string.
Escaped range characters in a [set]
In the second example, [a-%%], %% simply defines an escaped % sign and not a shorthand character class. The superficial problem is, the range is defined upsidedown, from high to low (in reference to the US ASCII value of the characters a 61 and % 37), e.g like an erroneous Lua pattern like [f-a]. If the set is defined in reverse order it seems to work: [%%-a] but all it does is matching the three individual characters instead of the range of characters between % and a; credit cyclaminist).
This could be considered a bug and, indeed, means it is not possible to create a range of characters in a [set] if one of the defining range characters need to be escaped.
Possible Solution
Start the character range from the next character that does not need to be escaped - and then add the remaining escaped characters individually, e.g.
[%%&-a]
Sample:
for w in string.gmatch("%&*()-0Aa", "[%%&-a]") do
print(w)
end
This is the answer I have found. Still, maybe somebody else has something better.
please i want to validate the inputs from a user, the format for the inputs would be: 3 uppercase characters, 3 integer numbers, an optional space, a -, an optional space, either a 'LAB or ((EN or ENLH) with 1 interger number ranging from a [1-9]).
The regex i wrote is
/\D{3}\d{3}\s?-\s?(LAB|(EN(LH)?\d{1}))/
am finding it difficult to stop inputs after the LAB so that when EEE333 - LAB1 is inputed it becomes invalid.
If you are asking how to prevent LAB1 at the end, use an end of line anchor $ in your regex test:
/\D{3}\d{3}\s?-\s?(LAB|(EN(LH)?\d{1}))$/
If you are trying to require exactly one digit at the end of the acceptable strings, move the single digit match outside of the optional groups:
/\D{3}\d{3}\s?-\s?(LAB|(EN(LH)?))\d{1}$/
I have wrote for you the following regular expression:
[A-Z]{3}[0-9]{3}\s?-\s?(?:LAB|(?:EN|LH))[1-9]{1}
The regex works a follows:
[A-Z]{3}
MATCH EXACTLY THREE UPPERCASE CHARACTERS RANGING FROM A TO Z
[0-9]{3}
MATCH EXACTLY THREE NUMBERS RANGING FROM 0 TO 9
\s?\-\s?
MATCH a space (optional) or a '-' (required) or a space (optional)
(?:LAB|(?:EN|LH))
MATCH 'LAB' OR ('EN' OR 'LH')?: omits capturing LAB OR EN OR LH
[1-9]{1}
MATCH EXACTLY ONE NUMBERS RANGING FROM 1 TO 9
You could place your regex between word boundaries \b.
You start your regex with \D which is any character that is not a digit. That would for example also match $%^. You could use [A-Z].
You use \d{1} which is a shorhand for [0-9], but you want to match a digit between 1 and 9 [1-9]. You could also omit the {1}.
Maybe this updated will work for you?
\b[A-Z]{3}\d{3} ?- ?(?:LAB|(?:EN(?:LH)?[1-9]))\b
Explanation
A word boundary \b
Match 3 uppercase characters [A-Z]{3}
Match 3 digits \d{3}
Match an optional whitespace, a hyphen and another optional whitespace ?- ?
A non capturing group which for example matches LAB or EN EN1 or ENLH or ENLH9 (?:EN(?:LH)?[1-9]))
A word boundary \b
I have two regular expressions: ^(\\p{L}|[0-9]|_)+$ and #[^[:punct:][:space:]]+ (the first is used in Java, the second on iOS). I want to combine these into one expression, to match either one or the other in iOS.
The first one is for a username so I also need to add a # character to the start of that one. What would that look like?
The ^(\\p{L}|[0-9]|_)+$ pattern in Java matches the same way as in ICU library used in iOS (they are very similar): a whole string consisting of 1 or more Unicode letters, ASCII digits or _. It is poorly written as the alternation group is quantified and that is much less efficient than a character class based solution, ^[\\p{L}0-9_]+$.
The #[^[:punct:][:space:]]+ pattern matches a # followed with 1 or more chars other than punctuation/symbols and whitespace chars (that is, 1 or more letters or digits, or alphanumeric chars).
What you seek can be writtern as
#[\\p{L}0-9_]+|[^[:punct:][:space:]]+
or
#[\\p{L}0-9_]+|#[[:alnum:]]+
or if you want to limit to ASCII digits and not match Unicode digits:
#[\\p{L}0-9_]+|#[\\p{L}0-9]+
It matches
# - a # symbol
[\\p{L}0-9_]+ - 1 or more Unicode letters, ASCII diigts, _
| - or
# - a # char
[[:alnum:]]+ - 1 or more letters or digits.
[^[:punct:][:space:]]+ - any 1+ chars other than punctuation/symbols and whitespace.
Basically, all these expressions match strings like this.
If you want to match #SomeThing_123 in full, just use [##]\\w+, a # or # and then 1 or more letters, digits or _, or to only allow ASCII digits, [##][\\p{L}0-9_]+.
A word boundary may be required at the end of the pattern, [##][\\p{L}0-9_]+\\b.
I am trying to make a Ultraedit regex which allows me to remove all words of a txt file containing a number.
For example:
test
test2
t2est
te2st
and...
get only
test
A case-insensitive search with Perl regular expression search string \<[a-z]+\d\w*\> finds entire words containing at least 1 digit.
\< ... beginning of a word. \b for any word boundary could be also used.
[a-z]+ ... any letter 1 or more times. You can put additional characters into the square brackets like ÄÖÜäöüß also used in language of text file.
\d ... any digit, i.e. 0-9.
\w* ... any word character 0 or more times. Any word character means all word characters according to Unicode table which includes language dependent word characters, all digits and the underscore.
\> ... end of a word. \b for any word boundary could be also used.
A case-insensitive search with UltraEdit regular expression search string [a-z]+[0-9][a-z0-9_]++ finds also entire words containing at least 1 digit if additionally the find option Match whole word is also checked.
[a-z]+ ... any letter 1 or more times. You can put additional characters into the square brackets used in language of text file.
[0-9] ... any digit.
[a-z0-9_]++ ... any letter, digit or underscore 0 or more times.
The UltraEdit regexp search string [a-z]+[0-9][a-z0-9_]++ in Unix/Perl syntax would be [a-z]+[0-9][a-z0-9_]* which could be also used with find option Match whole word checked instead of the Perl regexp search.
Just came across this pattern, which I really don't understand:
^[%w-.]+$
And could you give me some examples to match this expression?
Valid in Lua, where %w is (almost) the equivalent of \w in other languages
^[%w-.]+$ means match a string that is entirely composed of alphanumeric characters (letters and digits), dashes or dots.
Explanation
The ^ anchor asserts that we are at the beginning of the string
The character class [%w-.] matches one character that is a letter or digit (the meaning of %w), or a dash, or a period. This would be the equivalent of [\w-.] in JavaScript
The + quantifier matches such a character one or more times
The $ anchor asserts that we are at the end of the string
Reference
Lua Patterns
Actually it will match nothing. Because there is an error: w- this is a start of a text range and it is out of order. So it should be %w\- instead.
^[%w\-.]+$
Means:
^ assert position at start of the string
[%w\-.]+ match a single character present in the list below
+ Quantifier: Between one and unlimited times, as many times as possible, giving back as needed [greedy]
%w a single character in the list %w literally (case sensitive)
\- matches the character - literally
. the literal character .
$ assert position at end of the string
Edit
As the OP changed the question and the tags this answer no longer fits as a proper answer. It is POSIX based answer.
As #zx81 comment:
%w is \w in Lua which means any alphanumeric characters plus "_"