In ruby, here is how you can check for a substring in a string:
str = "hello world"
str.include?("lo")
=> true
When I am attempting to save an emoji in a text column in a rails application (the text column within a mysql database is utf8), it comes back with this error:
Incorrect string value: \xF0\x9F\x99\x82
For my situation in a rails application, it suffices to see if an emoji is present in the submitted text. If an emoji is present: raise a validation error. Example:
class MyModel < ApplicationRecord
validate :cannot_contain_emojis
private
def cannot_contain_emojis
if my_column.include?("/\xF0")
errors.add(:my_column, 'Cannot include emojis")
end
end
end
Note: The reason I am checking for \xF0 is because according to this site, it appears that all, or most, emoji's begin with this signature.
This however does not work. It continues to return false even when it is true. I'm pretty sure the issue is that my include statement doesn't work because the emoji is not converted to bytes for the comparison.
Question
How can I make a validation to check that an emoji is not passed in?
Example bytes for a smiley face in UTF8: \xF0\x9F\x99\x82
You can use the Emoji Unicode property to test for Emoji using a Regexp, something like this:
def cannot_contain_emojis
if /\p{Emoji}/ =~ my_column
errors.add(:my_column, 'Cannot include emojis')
end
end
Unicode® Technical Standard #51 "UNICODE EMOJI" contains a more sophisticated regex:
\p{RI} \p{RI}
| \p{Emoji}
( \p{EMod}
| \x{FE0F} \x{20E3}?
| [\x{E0020}-\x{E007E}]+ \x{E007F} )?
(\x{200D} \p{Emoji}
( \p{EMod}
| \x{FE0F} \x{20E3}?
| [\x{E0020}-\x{E007E}]+ \x{E007F} )?
)*
[Note: some of those properties are not implemented in Onigmo / Ruby.]
However, checking for Emojis probably not going to be enough. It is pretty clear that your text processing is somehow broken at some point. And if it is broken by an Emoji, then there is a chance it will also be broken by my name, or the name of Ruby's creator 松本 行弘, or by the completely normal English word “naïve”.
Instead of playing a game of whack-a-mole trying to detect every Emoji, mathematical symbol, Arabic letter, typographically correct punctuation mark, etc., it would be much better simply the fix the text processing.
I found Jörg's solution was only working when passing in the string itself and not a variable. Not sure why that is.
/\p{Emoji}/ =~ "🎃"
=> 0
value = "1f383"
=> "1f383"
/\p{Emoji}/ =~ value
=> 0
/\p{Emoji}/ =~ "hello"
=> nil
Regardless I'd recommend using the unicode-emoji gem, as its approach is comprehensive. Its source code and documentation can be found on GitHub.
Related
I'm trying to modify an existing lua script that cleans up subtitle data in Aegisub.
I want to add the ability to delete lines that contain the symbol "♪"
Here is the code I want to modify:
-- delete commented or empty lines
function noemptycom(subs,sel)
progress("Deleting commented/empty lines")
noecom_sel={}
for s=#sel,1,-1 do
line=subs[sel[s]]
if line.comment or line.text=="" then
for z,i in ipairs(noecom_sel) do noecom_sel[z]=i-1 end
subs.delete(sel[s])
else
table.insert(noecom_sel,sel[s])
end
end
return noecom_sel
end
I really have no idea what I'm doing here, but I know a little SQL and LUA apparently uses the IN keyword as well, so I tried modifying the IF line to this
if line.text in (♪) then
Needless to say, it didn't work. Is there a simple way to do this in LUA? I've seen some threads about the string.match() & string.find() functions, but I wouldn't know where to start trying to put that code together. What's the easiest way for someone with zero knowledge of Lua?
in is only used in the generic for loop. Your if line.text in (♪) then is no valid Lua syntax.
Something like
if line.comment or line.text == "" or line.text:find("\u{266A}") then
Should work.
In Lua every string have the string functions as methods attached.
So use gsub() on your string variable in loop like...
('Text with ♪ sign in text'):gsub('(♪)','note')
...thats replace the sign and output is...
Text with note sign in text
...instead of replacing it with 'note' an empty '' deletes it.
gsub() is returning 2 values.
First: The string with or without changes
Second: A number that tells how often the pattern matches
So second return value can be used for conditions or success.
( 0 stands for "pattern not found" )
So lets check above with...
local str,rc=('Text with strange ♪ sign in text'):gsub('(♪)','notation')
if rc~=0 then
print('Replaced ',rc,'times, changed to: ',str)
end
-- output
-- Replaced 1 times, changed to: Text with strange notation sign in text
And finally only detect, no change made...
local str,rc=('Text with strange ♪ sign in text'):gsub('(♪)','%1')
if rc~=0 then
print('Found ',rc,'times, Text is: ',str)
end
-- output is...
-- Found 1 times, Text is: Text with strange ♪ sign in text
The %1 holds what '(♪)' found.
So ♪ is replaced with ♪.
And only rc is used as a condition for further handling.
I got a few string like so:
TFjyg9780878_867978-DGB097908-78679iuhi698_widesky_light_87689uiyhk
AND
TFjyg9780878_867978-DGB097908-78679iuhi698_sky_light_87689uiyhk
AND
TFjyg9780878_867978-DGB097908-78679iuhi698_widesky_dark_87689uiyhk
AND
TFjyg9780878_867978-DGB097908-78679iuhi698_sky_dark_87689uiyhk
I need to check whether the strings above has one of the widesky_light, sky_light, widesky_dark and sky_dark with exactitude so I wrote this:
if my_string.match("widesky_light")
...
end
For each variant, but the problem I'm having is because sky_light and widesky_light are similar, my code is not working properly. I believe the solution to the above would be a regex, but I've spend the afternoon yesterday trying to get it to work without much success.
Any suggestions?
EDIT
A caveat: in this string (as example): TFjyg9780878_867978-DGB097908-78679iuhi698_widesky_light_87689uiyhk, the part after widesky_light, which is _87689uiyhk is optional, meaning that sometimes I have it, sometimes I don't, so a solution would not be able to count on _string_.
Looks like you just need to reorder your if statements
if my_string.match(/widesky_light/)
return 'something'
end
if my_string.match(/sky_light/)
return 'something'
end
Regex
1st regex : extract word for further checking
Here's a regex which only matches the interesting part :
(?<=_)[a-z_]+(?=(?:_|\b))
It means lowercase word with possible underscore inside, between 2 underscores or after 1 underscore and before a word boundary.
If you need some logic depending on the case (widesky, sky, light or dark), you could use this solution.
Here in action.
2nd regex : direct check if one of 4 words is present
If you just want to know if any of the 4 cases is present :
(?<=_)(?:wide)?sky_(?:dark|light)(?=(?:_|\b))
Here in action, with either _something_after or nothing.
Case statement
list = %w(
TFjyg9780878_867978-DGB097908-78679iuhi698_widesky_light_87689uiyhk
TFjyg9780878_867978-DGB097908-78679iuhi698_sky_light_87689uiyhk
TFjyg9780878_867978-DGB097908-78679iuhi698_widesky_dark_87689uiyhk
TFjyg9780878_867978-DGB097908-78679iuhi698_sky_dark_87689uiyhk
TFjyg9780878_867978-DGB097908-78679iuhi698_trash_dark_87689uiyhk
)
list.each do |string|
case string
when /widesky_light/ then puts "widesky light found!"
when /sky_light/ then puts "sky light found!"
when /widesky_dark/ then puts "widesky dark found!"
when /sky_dark/ then puts "sky dark found!"
else puts "Nothing found!"
end
end
In this order, the case statement should be fine. widesky_dark won't match twice, for example.
Maybe something like this:
case my_string
when /_(sky_light)/
# just sky_light
when /sky_light/
# widesky_light
when /_(sky_dark)/
# just sky_dark
when /sky_dark/
# widesky_dark
else
puts "I don't like"
end
Why does Kramdown's autolinking parser break when running it over a gemojified text field?
For [Test](http://google.com "Test") I'm getting:
Test
instead of the expected output:
Test
Live app: http://runnable.com/VAL1VuMjrGFur2yx/forem-gemoji-kramdown (see the Test post)
application_helper.rb:
def add_emojify_and_kramdown(text)
raw(Kramdown::Document.new(emojify(text)).to_html)
end
[...snip...]
def emojify(text)
h(text).to_str.gsub(/:([a-z0-9\+\-_]+):/) do |match|
if emoji = Emoji.find_by_alias($1)
'![' + $1 + '](' + asset_path("emoji/#{emoji.image_filename}") + ')'
else
match
end
end
end
Some additional info:
raw(Kramdown::Document.new(text).to_html) returns the expected output, but without Gemoji
raw(emojify(text)) doesn't change anything seeing as how text contains no emojis
raw(emojify(Kramdown::Document.new(text).to_html)) returns the expected output, but as raw HTML
The first thing your emojify method does is h(text), which HTML escapes the input, converting
[Test](http://google.com "Test")
into
[Test](http://google.com "Test")
Kramdown then operates on this string, and since it no longer contains quote marks it assumes the whole contents of (...) is the URL, producing:
Test
To get it to work you just need to drop the call to h: text.gsub(.... You’ll likely need to think about how to manage your string safety if this is external data.
Quick background: I have a string which contains references to other pages. The pages are linked to using the format: "#12". A hash followed by the ID of the page.
Say I have the following string:
str = 'This string links to the pages #12 and #125'
I already know the IDs of the pages that need linking:
page_ids = str.scan(/#(\d*)/).flatten
=> [12, 125]
How can I loop through the page ids and link the #12 and #125 to their respective pages? The problem I've run into is if I do the following (in rails):
page_ids.each do |id|
str = str.gsub(/##{id}/, link_to("##{id}", page_path(id))
end
This works fine for #12 but it links the "12" part of #125 to the page with ID of 12.
Any help would be awesome.
Instead of extracting the ids first and then replacing them, you can simply find and replace them in one go:
str = str.gsub(/#(\d*)/) { link_to("##{$1}", page_path($1)) }
Even if you can't leave out the extraction step because you need the ids somewhere else as well, this should be much faster, since it doesn't have to go through the entire string for each id.
PS: If str isn't referred to from anywhere else, you can use str.gsub! instead of str = str.gsub
if your indexes always end at word boundaries, you can match that:
page_ids.each do |id|
str = str.gsub(/##{id}\b/, link_to("##{id}", page_path(id))
end
you only need to add the word boundary symbol \b on the search pattern, it is not necessary for the replacement pattern.
I have a Rails site, where the content is written in markdown. I wish to display a snippet of each, with a "Read more.." link.
How do I go about this? Simple truncating the raw text will not work, for example..
>> "This is an [example](http://example.com)"[0..25]
=> "This is an [example](http:"
Ideally I want to allow the author to (optionally) insert a marker to specify what to use as the "snippet", if not it would take 250 words, and append "..." - for example..
This article is an example of something or other.
This segment will be used as the snippet on the index page.
^^^^^^^^^^^^^^^
This text will be visible once clicking the "Read more.." link
The marker could be thought of like an EOF marker (which can be ignored when displaying the full document)
I am using maruku for the Markdown processing (RedCloth is very biased towards Textile, BlueCloth is extremely buggy, and I wanted a native-Ruby parser which ruled out peg-markdown and RDiscount)
Alternatively (since the Markdown is translated to HTML anyway) truncating the HTML correctly would be an option - although it would be preferable to not markdown() the entire document, just to get the first few lines.
So, the options I can think of are (in order of preference)..
Add a "truncate" option to the maruku parser, which will only parse the first x words, or till the "excerpt" marker.
Write/find a parser-agnostic Markdown truncate'r
Write/find an intelligent HTML truncating function
Write/find an intelligent HTML truncating function
The following from http://mikeburnscoder.wordpress.com/2006/11/11/truncating-html-in-ruby/, with some modifications will correctly truncate HTML, and easily allow appending a string before the closing tags.
>> puts "<p><b>Something</p>".truncate_html(5, at_end = "...")
=> <p><b>Someth...</b></p>
The modified code:
require 'rexml/parsers/pullparser'
class String
def truncate_html(len = 30, at_end = nil)
p = REXML::Parsers::PullParser.new(self)
tags = []
new_len = len
results = ''
while p.has_next? && new_len > 0
p_e = p.pull
case p_e.event_type
when :start_element
tags.push p_e[0]
results << "<#{tags.last}#{attrs_to_s(p_e[1])}>"
when :end_element
results << "</#{tags.pop}>"
when :text
results << p_e[0][0..new_len]
new_len -= p_e[0].length
else
results << "<!-- #{p_e.inspect} -->"
end
end
if at_end
results << "..."
end
tags.reverse.each do |tag|
results << "</#{tag}>"
end
results
end
private
def attrs_to_s(attrs)
if attrs.empty?
''
else
' ' + attrs.to_a.map { |attr| %{#{attr[0]}="#{attr[1]}"} }.join(' ')
end
end
end
Here's a solution that works for me with Textile.
Convert it to HTML
Truncate it.
Remove any HTML tags that got cut in half with
html_string.gsub(/<[^>]*$/, "")
Then, uses Hpricot to clean it up and close unclosed tags
html_string = Hpricot( html_string ).to_s
I do this in a helper, and with caching there's no performance issue.
You could use a regular expression to find a line consisting of nothing but "^" characters:
markdown_string = <<-eos
This article is an example of something or other.
This segment will be used as the snippet on the index page.
^^^^^^^^^^^^^^^
This text will be visible once clicking the "Read more.." link
eos
preview = markdown_string[0...(markdown_string =~ /^\^+$/)]
puts preview
Rather than trying to truncate the text, why not have 2 input boxes, one for the "opening blurb" and one for the main "guts". That way your authors will know exactly what is being show when without having to rely on some sort of funkly EOF marker.
I will have to agree with the "two inputs" approach, and the content writer would need not to worry, since you can modify the background logic to mix the two inputs in one when showing the full content.
full_content = input1 + input2 // perhaps with some complementary html, for a better formatting
Not sure if it applies to this case, but adding the solution below for the sake of completeness. You can use strip_tags method if you are truncating Markdown-rendered contents:
truncate(strip_tags(markdown(article.contents)), length: 50)
Sourced from:
http://devblog.boonecommunitynetwork.com/rails-and-markdown/
A simpler option that just works:
truncate(markdown(item.description), length: 100, escape: false)