Hello I have an string with URL-ENCODING , %3A , %2F ...
http%3A%2F%2Fmydomain.com%2Fimage%2Fflv%2F1%2F8%2Fa%2Fimage_18060.jpg%3Fe%3D13777194
I would like to replace the "special characters" http://www.w3schools.com/tags/ref_urlencode.asp to ASCII characters , I am programing in Rails but i dont have idea from where i can start. Do you have any ideas?
Thank you very much for your help!
hummm
in this way Can I use URI.unescape?
scraper = Scraper.define do
array :items
process "div.mozaique>div", :items => Scraper.define {
process "div.thumb>a", URI.unescape(:link) => "#href"
result :link
}
result :items
end
/Users/jcr/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/uri/common.rb:331:in `unescape': undefined method `gsub' for :link:Symbol (NoMethodError)
from /Users/jcr/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/uri/common.rb:649:in `unescape'
from /Users/jcr/web/sss/app/controllers/cweb.rb:17:in `block (2 levels) in main_web'
from /Users/jcr/.rvm/gems/ruby-1.9.3-p448/gems/scrapi-2.0.0/lib/scraper/base.rb:986:in `module_eval'
from /Users/jcr/.rvm/gems/ruby-1.9.3-p448/gems/scrapi-2.0.0/lib/scraper/base.rb:986:in `define'
from /Users/jcr/web/sss/app/controllers/cweb.rb:15:in `block in main_web'
from /Users/jcr/.rvm/gems/ruby-1.9.3-p448/gems/scrapi-2.0.0/lib/scraper/base.rb:986:in `module_eval'
from /Users/jcr/.rvm/gems/ruby-1.9.3-p448/gems/scrapi-2.0.0/lib/scraper/base.rb:986:in `define'
from /Users/jcr/web/sss/app/controllers/cweb.rb:13:in `main_web'
from /Users/jcr/web/sss/app/controllers/cweb.rb:57:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
You can use URI.unescape:
irb(main):003:0> require 'uri'
=> true
irb(main):006:0> URI.unescape("http%3A%2F%2Fmydomain.com%2Fimage%2Fflv%2F1%2F8%2Fa%2Fimage_18060.jpg%3Fe%3D13777194")
=> "http://mydomain.com/image/flv/1/8/a/image_18060.jpg?e=13777194"
Related
I've got an app with 4 different models which are multisearchable, however they were originally set up with a pg_search_scope; ie:
class Episode < ActiveRecord::Base
include PgSearch
# multisearchable :against => [:title, :description]
pg_search_scope :search_text,
:against => [:title, :description],
:using => {
:tsearch => {:prefix => true}
}
As you can see at one point I tried setting this up the multiserachable way.
Of late there are items we know should be returning that are NOT in the search results, and I thought I'd have to rebuild the pg_search_documents; however that's throwing an error:
irb(main):004:0> PgSearch::Multisearch.rebuild(Show)
PgSearch::Multisearch::ModelNotMultisearchable: PgSearch::Multisearch::ModelNotMultisearchable
from /app/vendor/bundle/ruby/2.0.0/gems/pg_search-0.7.3/lib/pg_search/multisearch/rebuilder.rb:6:in `initialize'
from /app/vendor/bundle/ruby/2.0.0/gems/pg_search-0.7.3/lib/pg_search/multisearch.rb:10:in `new'
from /app/vendor/bundle/ruby/2.0.0/gems/pg_search-0.7.3/lib/pg_search/multisearch.rb:10:in `block in rebuild'
from /app/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:202:in `block in transaction'
from /app/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:210:in `within_new_transaction'
from /app/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.2/lib/active_record/connection_adapters/abstract/database_statements.rb:202:in `transaction'
from /app/vendor/bundle/ruby/2.0.0/gems/activerecord-4.0.2/lib/active_record/transactions.rb:209:in `transaction'
from /app/vendor/bundle/ruby/2.0.0/gems/pg_search-0.7.3/lib/pg_search/multisearch.rb:8:in `rebuild'
from (irb):4
from /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.2/lib/rails/commands/console.rb:90:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.2/lib/rails/commands/console.rb:9:in `start'
from /app/vendor/bundle/ruby/2.0.0/gems/railties-4.0.2/lib/rails/commands.rb:62:in `<top (required)>'
from /app/bin/rails:4:in `require'
from /app/bin/rails:4:in `<main>'
irb(main):005:0>
I thought that it might be because of the setup as a pg_search_scope, so I set one class as above with the multisearch:multisearchable :against => [:title, :description] -- but even then I get rebuild problems.
I tried the manual way of deleting from the pg_search table and rebuilding but even THAT's throwing an error:
2.1.0 :010 > Show.find_each{ |record| record.update_pg_search_document }
Show Load (4.8ms) SELECT "shows".* FROM "shows" ORDER BY "shows"."id" ASC LIMIT 1000
NoMethodError: undefined method `update_pg_search_document' for #<Show:0x007fd945a92358>
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activemodel-4.0.2/lib/active_model/attribute_methods.rb:439:in `method_missing'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/attribute_methods.rb:155:in `method_missing'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/acts_as_follower-0.2.1/lib/acts_as_follower/followable.rb:54:in `method_missing'
from (irb):10:in `block in irb_binding'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/relation/batches.rb:26:in `block (2 levels) in find_each'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/relation/batches.rb:26:in `each'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/relation/batches.rb:26:in `block in find_each'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/relation/batches.rb:75:in `find_in_batches'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-deprecated_finders-1.0.3/lib/active_record/deprecated_finders/relation.rb:70:in `find_in_batches'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/relation/batches.rb:25:in `find_each'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/activerecord-4.0.2/lib/active_record/querying.rb:8:in `find_each'
from (irb):10
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands/console.rb:90:in `start'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands/console.rb:9:in `start'
from /Users/grimm/.rvm/gems/ruby-2.1.0/gems/railties-4.0.2/lib/rails/commands.rb:62:in `<top (required)>'
from bin/rails:4:in `require'
from bin/rails:4
I've been getting the same error while trying to rebuild (for the first time, installing) via the Rails command line with PgSearch::Multisearch.rebuild(Professor). What's worked for me is instead rebuilding from the command line using the alternative approach recommended on the Github:
rake pg_search:multisearch:rebuild[Professor]
This seems to work for me -- it doesn't throw an error, and when I go to the Rails console after and run PgSearch.multisearch("query") it's returning a result, whereas before it was returning an error.
I know this isn't a fix, but it's a solution to the immediate problem. I hope it helps -- posting an issue on their Github might lead to a fuller fix.
I am trying to include a module only if the subclass has a certain column,
and I am doing this in an initializer:
class ActiveRecord::Base
def self.inherited(subclass)
subclass.include(MultiTenancy) if subclass.new.respond_to?(:tenant_id)
end
end
I keep getting this error:
NoMethodError: undefined method `[]' for nil:NilClass
Here is the module that I am importing:
module MultiTenancy
class TenantNotSetError < StandardError ; end
def self.included(model)
model.class_eval do
belongs_to :tenant
validates :tenant_id, presence: true
default_scope -> {
raise TenantNotSetError.new unless Tenant.current_tenant
where(tenant_id: Tenant.current_tenant.id)
}
def multi_tenanted?
true
end
end
end
end
What am I doing wrong. I can include the module on any subclass separate eg. Klass.include(MultiTenancy) successfully, so the problem is with the initializer.
Below is the stack trace:
from -e:1:in `<main>'2.1.1 :002 > Klass.all
NoMethodError: undefined method `[]' for nil:NilClass
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activerecord-4.1.4/lib/active_record/relation/delegation.rb:9:in `relation_delegate_class'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activerecord-4.1.4/lib/active_record/relation/delegation.rb:112:in `relation_class_for'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activerecord-4.1.4/lib/active_record/relation/delegation.rb:106:in `create'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activerecord-4.1.4/lib/active_record/model_schema.rb:133:in `table_name='
from /home/lee/Code/mobifit/app/models/klass.rb:25:in `<class:Klass>'
from /home/lee/Code/mobifit/app/models/klass.rb:22:in `<top (required)>'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:443:in `load'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:443:in `block in load_file'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:633:in `new_constants_in'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:442:in `load_file'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:342:in `require_or_load'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:480:in `load_missing_constant'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:180:in `const_missing'
from (irb):2
from /home/lee/.rvm/gems/ruby-2.1.1/gems/railties-4.1.4/lib/rails/commands/console.rb:90:in `start'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/railties-4.1.4/lib/rails/commands/console.rb:9:in `start'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/railties-4.1.4/lib/rails/commands/commands_tasks.rb:69:in `console'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/railties-4.1.4/lib/rails/commands/commands_tasks.rb:40:in `run_command!'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/railties-4.1.4/lib/rails/commands.rb:17:in `<top (required)>'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `require'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `block in require'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:232:in `load_dependency'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:247:in `require'
from /home/lee/Code/mobifit/bin/rails:8:in `<top (required)>'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `load'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `block in load'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:232:in `load_dependency'
from /home/lee/.rvm/gems/ruby-2.1.1/gems/activesupport-4.1.4/lib/active_support/dependencies.rb:241:in `load'
from /home/lee/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /home/lee/.rvm/rubies/ruby-2.1.1/lib/ruby/2.1.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from -e:1:in `<main>'2.1.1 :003 >
If you follow the stack trace back a bit, you can see that the thing that's nil that isn't supposed to be is is the class's #relation_delegate_cache. Looking at the source, you can see that this normally gets initialized in the initialize_relation_delegate_cache method, which is called in the inherited hook for ActiveRecord::Delegation::DelegateCache.
ActiveRecord::Base extends this module, so normally a class that inherits from ActiveRecord::Base would call this inherited hook. So the problem with your code is that you're monkey-patching inherited on ActiveRecord::Base, which means that the hook in ActiveRecord::Delegation::DelegateCache is no longer being called.
Two comments:
This is why it's a bad idea to monkey-patch, especially something as fundamental and complex as active record. Is it possible to do what you want to do in a more straightforward way?
If it's not, you should take some notes on how ActiveRecord::Delegation::DelegateCache works and rewrite your monkey patch to mix in a module of your own, rather than just over-writing ActiveRecord::Base's inherited method.
The suggestion in (2) would look something like this:
module MultitenancyConcern
def self.inherited(subclass)
subclass.include(MultiTenancy) if subclass.column_names.include?("tenant_id")
super #this is critical for avoiding the error you're getting
end
end
then your monkey-patch becomes just
ActiveRecord::Base.extend(MultitenancyConcern)
I'm trying to run scrapi with rails with this code:
require 'rubygems'
require 'scrapi'
require 'tidy'
scraper = Scraper.define do
process "title", :page_name => :text
result :page_name
end
uri = URI.parse("http://railscasts.com/episodes/173-screen-scraping-with-scrapi")
p scraper.scrape(uri)
but I get the following error:
DL is deprecated, please use Fiddle
/usr/local/rvm/gems/ruby-2.0.0-p353/gems/tidy-1.1.2/lib/tidy/tidybuf.rb:5:in `<class:Tidybuf>': uninitialized constant DL::Importable (NameError)
from /usr/local/rvm/gems/ruby-2.0.0-p353/gems/tidy-1.1.2/lib/tidy/tidybuf.rb:3:in `<top (required)>'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:55:in `require'
from /usr/local/rvm/gems/ruby-2.0.0-p353/gems/tidy-1.1.2/lib/tidy.rb:25:in `<module:Tidy>'
from /usr/local/rvm/gems/ruby-2.0.0-p353/gems/tidy-1.1.2/lib/tidy.rb:21:in `<top (required)>'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:135:in `require'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:135:in `rescue in require'
from /usr/local/rvm/rubies/ruby-2.0.0-p353/lib/ruby/site_ruby/2.0.0/rubygems/core_ext/kernel_require.rb:144:in `require'
from scrapitest.rb:3:in `<main>'
has anybody got an idea why is that and what could I do to fix it?
Thank you.
Try http://github.com/libc/tidy_ffi instead. It seems tidy gem is not mantained anymore.
I'm not new to DynamoDB, but I'm new to DynamoID gem.
I can't achieve to make it work since I get the following error :
/aws-sdk-1.28.1/lib/aws/core/client.rb:366:in `return_or_raise': The action or operation requested is invalid. Verify that the action is typed correctly. (AWS::DynamoDB::Errors::InvalidAction)
/aws-sdk-1.28.1/lib/aws/core/client.rb:467:in `client_request'
(eval):3:in `list_tables'
/aws-sdk-1.28.1/lib/aws/dynamo_db/table_collection.rb:121:in `_each_item'
/aws-sdk-1.28.1/lib/aws/core/collection/with_limit_and_next_token.rb:54:in `_each_batch'
/aws-sdk-1.28.1/lib/aws/core/collection.rb:80:in `each_batch'
/aws-sdk-1.28.1/lib/aws/core/collection.rb:47:in `each'
/dynamoid-0.7.1/lib/dynamoid/adapter/aws_sdk.rb:185:in `collect'
/dynamoid-0.7.1/lib/dynamoid/adapter/aws_sdk.rb:185:in `list_tables'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:146:in `block (3 levels) in <module:Adapter>'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:39:in `benchmark'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:146:in `block (2 levels) in <module:Adapter>'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:25:in `block in reconnect!'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:39:in `benchmark'
/dynamoid-0.7.1/lib/dynamoid/adapter.rb:25:in `reconnect!'
/dynamoid-0.7.1/lib/dynamoid.rb:35:in `configure'
app/config/initializers/dynamoid.rb:1:in `<top (required)>'
It seems that list_tables doesn't exist, but it's a method of the AWS Ruby SDK
My model is very simple :
class Interaction
include Dynamoid::Document
table :name => :interactions, :key => :from, :read_capacity => 1, :write_capacity => 1
end
Could anybody help?
Cheers,
Emmanuel
Hi I am trying to scrape a web page "take the links" go to that links and "to scrape it" too.
require 'rubygems'
require 'scrapi'
require 'uri'
Scraper::Base.parser :html_parser
web = "http://......"
def sub_web(linksubweb)
uri = URI.parse(URI.encode(linksubweb))
end
scraper = Scraper.define do
array :items
process "div.mozaique>div", :items => Scraper.define {
process "p>a", :title => :text
process "div.thumb>a", :link => "#href"
result :title, :link,
}
result :items
end
uri = URI.parse(URI.encode(web))
scraper.scrape(uri).each do |pag|
link_full = uri + pag.link.to_str
puts pag.title
sub_web(link_full)
puts
end
And I have the following error
e $stdout.sync=true;$stderr.sync=true;load($0=ARGV.shift) /Users/sss/web/app/views/admin/topics/webconector.rb
Title 1
http://mydomain/user34/top5
/Users/sss/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/uri/common.rb:304:in `escape': undefined method `gsub' for #<URI::HTTP:0x007fa07cb01e08> (NoMethodError)
from /Users/sss/.rvm/rubies/ruby-1.9.3-p448/lib/ruby/1.9.1/uri/common.rb:623:in `escape'
from ../app/views/admin/topics/conectaweb.rb:11:in `sub_web'
from ../app/views/admin/topics/conectaweb.rb:34:in `block in <top (required)>'
from ../views/admin/topics/conectaweb.rb:29:in `each'
from ../app/views/admin/topics/conectaweb.rb:29:in `<top (required)>'
from -e:1:in `load'
from -e:1:in `<main>'
Process finished with exit code 1
try using uri = URI.parse(URI.encode(linksubweb.to_s)) this should work. The problem is that method requires a string argument so you have to first convert the URI::HTTP object into string.