Cleaning scraped <a href> rails - ruby-on-rails

I have scraped data from a website and entered it into an array using the code below:
def process_course_details(course_details)
details_array =[]
details_link = true
entry_link = true
details_info = {}
# Sets all data in hash
details_info[:url] = clean_link(course_details.search('div.coursedetails_programmeurl a'))
details_array.push(details_info)
print_details_info(details_info)
entry_link = course_details.search('ul.details_tabs').first
end
The code above stores the element being pulled as such:
View course details on provider's website
But I'd like to clean the above to the below:
http://www.abdn.ac.uk/study/courses/undergraduate/C8R1/
or failing that remove the apostrophe and have this:
View course details on providers website`

You can extract the href with Nokogiri like this:
html = Nokogiri::HTML('View course details on provider\'s website')
html.xpath("//a/#href").to_s # => "http://www.abdn.ac.uk/study/courses/undergraduate/C8R1/"

Based on your comment:
When storing other data I've scraped into the database the apostrophe
provided errors and stopped it. Once I had cleaned the apostrophe and
it no longer was part of the array the code worked and the table was
created.
db = SQLite3::Database.open('ahhh.sqlite3')
db.execute "INSERT INTO aahah (uname, cname, duration, qualification, url, entry) VALUES ('#{#uni_name}', #{#course_name}', '#{#course_duration}', '#{#course_qual}', '#{#details_entry}', '#{#requirements}')"
You are inserting the values via string interpolation:
db.execute("INSERT INTO table_name (foo, bar) VALUES ('#{#foo}', '#{#bar}')")
Apparently, if the interpolated strings contain an apostrophe, your SQL string likely becomes invalid. Even worse, this code is prone to SQL injection.
Instead you should use parameter markers and let the SQLite gem handle the escaping:
db.execute("INSERT INTO table_name (foo, bar) VALUES (?, ?)", [#foo, #bar])
This allows you to safely insert apostrophe and other special characters.

Related

manipulating a text datatype representing an array of hashes

The following is a stored value for datatype defined as text
text_field = "[{\"shop_id\"=>110, \"budget\"=>\"040\"}, {\"shop_id\"=>111, \"budget\"=>173}, {\"shop_id\"=>141, \"budget\"=>344}, {\"shop_id\"=>142, \"budget\"=>\"020\"}, {\"shop_id\"=>144, \"budget\"=>\"220\"}]"
Thus defined, it is for all intents and operational purposes a ruby String class. The contents represent an array of hashes.
How can this string become the array of hashes so that one can call
text_field_array.each do |hash|
shop = Shop.find(hash[shop_id])
shop_budget = shop.budget + hash[budget]
[...]
end
Use eval
> eval(text_field)
=> [{"shop_id"=>110, "budget"=>"040"},
{"shop_id"=>111, "budget"=>173},
{"shop_id"=>141, "budget"=>344},
{"shop_id"=>142, "budget"=>"020"},
{"shop_id"=>144, "budget"=>"220"}]
NOTE: Be careful about it. Make sure the text doesn't have malicious code in the string. Read more about it here https://ruby-hacking-guide.github.io/security.html

How do you get strip RTF formatting and get actual string value using DXL in DOORS?

I am trying to get the values in "ID" column of DOORS and I am currently doing this
string ostr=richtext_identifier(o)
When I try to print ostr, in some modules I get just the ID(which is what I want). But in other modules I will get values like "{\rtf1\ansi\ansicpg1256\deff0\nouicompat{\fonttbl{\f0\fnil\fcharset0 Times New Roman;}{\f1\froman\fcharset0 Times New Roman;}} {*\generator Riched20 10.0.17134}\viewkind4\uc1 \pard\f0\fs20\lang1033 SS_\f1\fs24 100\par } " This is the RTF value and I am wondering what the best way is to strip this formatting and get just the value.
Perhaps there is another way to go about this that I am not thinking of as well. Any help would be appreciated.
So the ID column of DOORS is actually a composite- DOORS builds it out of the Module level attribute 'Prefix' and the Object level attribute 'Absolute Number'.
If you wish to grab this value in the future, I would do the following (using your variables)
string ostr = ( module ( o ) )."Prefix" o."Absolute Number" ""
This is opposed to the following, which (despite seeming to be a valid attribute in the insert column dialog) WILL NOT WORK.
string ostr = o."Object Identifier" ""
Hope this helps!
Comment response: You should not need the module name for the code to work. I tested the following successfully on DOORS 9.6.1.10:
Object o = current
string ostr = ( module ( o ) )."Prefix" o."Absolute Number" ""
print ostr
Another solution is to use the identifier function, which takes an Object as input parameter, and returns the identifier as a plain (not RTF) string:
Declaration
string identifier(Object o)
Operation
Returns the identifier, which is a combination of absolute number and module prefix, of object o as a string.
The optimal solution somewhat depends on your underlying requirement for retrieving the object ID.

InfluxDB templating query - referring to measuerments?

I'm trying to make templates for my dahboards, and I have problems when it comes to referring to measuerment names.
My variables:
$space = SHOW MEASUREMENTS
Then I would like a variable that contains only values from a specific $space, which is actually a MEASUREMENT:
$app = SHOW TAG VALUES WITH KEY = "Application" WHERE MEASUREMENT =~ /^$space$/
Here I get a message: Template variables could not be initialized: error parsing query: found MEASUREMENT, expected identifier, string, number, bool at line 1, char 48
In the official example it is like this, though it refers to another tag:
$datacenter = SHOW TAG VALUES WITH KEY = "datacenter"
$host = SHOW TAG VALUES WITH KEY = "hostname" WHERE "datacenter" =~ /^$datacenter$/
I cannot find any info how to refer to MEASUREMENTS which would work. WHERE, WITH, etc.. Maybe is it not possible at all?
I found only this in the official tutorial, but this is for keys, not values.
SHOW TAG KEYS [FROM <measurement_name>]
I actually figured it out:
SHOW TAG VALUES FROM /^$space$/ WITH KEY = "Application"

Generation of table and accessing elements of a Array of Hashes

I have the following Array of hashes in a rails application:
a = ["{\"ROW1\"=>{\"correct\"=>{\"h\"=>\"10\", \"m\"=>\"11\", \"l\"=>
\"12\"}, \"wrong\"=>{\"h\"=>\"2\", \"m\"=>\"2\", \"l\"=>\"4\"}, \"blank
\"=>{\"h\"=>\"2\", \"m\"=>\"4\", \"l\"=>\"3\"}}, \"ROW2\"=>{\"correct
\"=>{\"h\"=>\"2\", \"m\"=>\"4\", \"l\"=>\"4\"}, \"wrong\"=>{\"h
\"=>\"4\", \"m\"=>\"6\", \"l\"=>\"6\"}, \"blank\"=>{\"h\"=>\"7\",
\"m\"=>\"5\", \"l\"=>\"6\"}}, \"ROW3\"=>{\"correct\"=>{\"h\"=>\"4\",
\"m\"=>\"6\", \"l\"=>\"7\"}, \"wrong\"=>{\"h\"=>\"6\", \"m\"=>\"7\",
\"l\"=>\"5\"}, \"blank\"=>{\"h\"=>\"7\", \"m\"=>\"9\", \"l\"=>
\"3\"}}}"]
I want to access its elements and create a database table from it, in the following format
ROW1 correct h=10, m=11,l=12
wrong h=2, m=2,l=4
blank h=2, m=4,l=3
...and similar for ROW2 and ROW3.
How can I do that?
I tried to access a value using
a["ROW1"]["Correct"]["h"]
...but it returns a nil value.
How to access the values of this array of hashes?
you need to first convert the string to hash which can be done as follows:
require 'json'
a = ["{\"ROW1\"=>{\"correct\"=>{\"h\"=>\"10\", \"m\"=>\"11\", \"l\"=>
\"12\"}, \"wrong\"=>{\"h\"=>\"2\", \"m\"=>\"2\", \"l\"=>\"4\"}, \"blank
\"=>{\"h\"=>\"2\", \"m\"=>\"4\", \"l\"=>\"3\"}}, \"ROW2\"=>{\"correct
\"=>{\"h\"=>\"2\", \"m\"=>\"4\", \"l\"=>\"4\"}, \"wrong\"=>{\"h
\"=>\"4\", \"m\"=>\"6\", \"l\"=>\"6\"}, \"blank\"=>{\"h\"=>\"7\",
\"m\"=>\"5\", \"l\"=>\"6\"}}, \"ROW3\"=>{\"correct\"=>{\"h\"=>\"4\",
\"m\"=>\"6\", \"l\"=>\"7\"}, \"wrong\"=>{\"h\"=>\"6\", \"m\"=>\"7\",
\"l\"=>\"5\"}, \"blank\"=>{\"h\"=>\"7\", \"m\"=>\"9\", \"l\"=>
\"3\"}}}"
]
hash_string = a[0]
hash = JSON.parse hash_string.gsub("\n", '').gsub('=>', ':')
# you access the hash now:
hash["ROW1"]["correct"]["h"]
# => 10
Btw, please note that there is a typo. Instead of Correct, the key is correct with small c instead of capital C.
Hope it helps : )

How to store xml parser data stored in sqlite database in IOS?

I want to insert xml data in Sqlite database.
For Example :
UPDATE OBJ_ART SET TITLE = '<?xml version='1.0' encoding='UTF-8'?><root available-locales="en_US,fr_FR," default-locale="en_US"><Title language-id="en_US">Distribution Sing EN</Title><Title language-id="fr_FR">Distribution Bâtiment</Title></root>', CONTENT = ‘ABCD’, STRUCT_ID = 12709, DISPLAY_DATE = 1422890 WHERE ART_ID = 12716
can you please any one guide me how to xml parser data to stored in sqlite database,
When working with SQLite, one should generally avoid building SQL statements with stringWithFormat, but rather use SQL like:
UPDATE OBJ_ART SET TITLE = ?, CONTENT = ?, STRUCT_ID = ?, DISPLAY_DATE = ? WHERE ART_ID = ?
Then, one would use the SQLite functions sqlite3_bind_text, sqlite3_bind_int, etc., to bind values to the ? placeholders in the SQL. That will gracefully handle values that happen to have quotation marks within them.
So, call sqlite3_prepare_v2 to prepare the SQL statement, then call sqlite3_bind_xxx for each of the columns, and then sqlite3_step to perform the SQL and, when done, call sqlite3_finalize to release the memory associated with the statement.
Even easier, use a library like FMDB, which greatly simplifies this process.
If want to insert or update XML data in Sqlite. You just follow some rules.
My example :
UPDATE OBJ_ART SET TITLE ='<?xml version=&apos;1.0&apos; encoding=&apos;UTF-8&apos;?><root available-locales="en_US,fr_FR," default-locale="en_US"><Title language-id="en_US">Distribution Sing EN</Title><Title language-id="fr_FR">Distribution Bâtiment</Title></root>', CONTENT = ‘ABCD’, STRUCT_ID = 12709, DISPLAY_DATE = 1422890 WHERE ART_ID = 12716
It's easy way to do. You just replace XML tags to HTML entities.
HTML entities:
1. < ----- replace to ---- <
2. > ----- replace to ---- >
3. " ----- replace to ---- "
4. ' ----- replace to ---- &apos;
I hope it will work. In every where while sending SOAP action also you have to do same thing.

Resources