Update
First of all, there is no method "context". That was a word my brain made up at some point and stuck with. Obviously I should've been running .excerpt(). Second, I was running the command against the returned array, not an individual instance of the PG_Search::Document
Double mistakes, but yes the code does in fact work.
End Update
First some system info:
Ruby 1.9.3p194
Rails 3.2.13
pg_search 0.5.7
Postgres 9.2.3 (with
unaccent enabled)
I'm trying to follow the progress made in this thread: (How to show excerpts from pg-search multisearch results)
Okay so assuming the use of the following query cribbed from that post:
#query = params[:query]
PgSearch.multisearch(#query).select("ts_headline(pg_search_documents.content, plainto_tsquery('english', ''' ' || unaccent('#{#query}') || ' ''' || ':*')) AS excerpt")
returns:
=> [#<PgSearch::Document id: 7, content: "1 <p>You think water moves fast? You should see ice...", searchable_id: 2, searchable_type: "Release", created_at: "2013-03-27 18:58:52", updated_at: "2013-03-27 18:58:52">]
It successfully returns some search results but they don't have the context method at all. It's as if I just called multisearch without the select method.
I'm a newbie when it comes to SQL and Postgres so I'm not exactly sure where to start in debugging that snippet. I would love some help debugging or getting an explanation of what is happening.
Also, an aside that I think is important, I want to thank anyone who works on pg_search or responds to questions like these. You make the world a better place.
You have to select the other columns you need as well.
For example
sanitized = ActionController::Base.helpers.sanitize(params[:q])
#results = PgSearch.multisearch(params[:q])
.select(:id, :content, :searchable_id, :searchable_type)
.select(["ts_headline(pg_search_documents.content, plainto_tsquery('english', ''' ' || '#{sanitized}' || ' ''' || ':*')) AS excerpt"])
Will return the id, content, searchable_id, searchable_type, excerpt
Also notice the sanitizing. Don't want to suffer from an sql injection attack. :]
Related
I am using Elasticsearch 6.2.4 with my RoR application using elasticsearch-rails and elasticsearch-model.
My indexation is runninng without getting any errors. but when I try to perform a search from the application I am getting this error from Elasticsearch
<Elasticsearch::Transport::Transport::Errors::BadRequest: [400] {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"}],"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"},"status":400}>
Everything was working normal prior to the upgrade of Elasticsearch from 1.5 to 6.2.4
I simplified my search query to try narrowing down the problem.
q = { "query" => { "match_all" => {} } }
But I still getting the same error. Probably I am not specifying a type in the query but wouldn't be unnecessary since I have a match_all condition ?
> {"query":{"match_all":{}}}
< {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"}],"type":"illegal_argument_exception","reason":"text is empty (possibly HTTP/0.9)"},"status":400}
I am brand new to Elasticsearch so excuse me in advance if there are some evident stuff that I am missing
Do you have any idea what is causing this error ? If you need more specific info just ask and I'll update this question.
Thanks.
Search request from application is resulting in HTTP 400 Bad Request. Are you able to perform a search request from outside the application i.e. invoking a curl command from your local etc ?
Totally lacking intuition here.
First i thought i only happens when copying and pasting code from editor. Unfortunately it's more common. Only thing I am doing is trying to insert more code somewhere in the middle of current input or modifying it.
Expected behavior:
Modify input without side effects.
What actually happens:
VERY OFTEN when modifying input it gets messed up.
If that happens every key press will copy and insert current input alongside with pressed character.
Vital notes:
Encoding is set to UTF-8 in terminal(s)
Issue persists on different emulators ( Terminator, gnome-terminal )
Issue persists when using different Ruby runtime console ( IRB, Pry )
Issue appears to be related to Ruby runtime, NOT Linux shell (i guess...)
Issue appears since:
Since system install. Didn't appear on my MacBook Air
System:
ArchLinux, although coworker reported same thing happens on his MacBook Pro.
How to reproduce (works for me):
Open rails console
Type example code: Shift.where(name: "som").where(name: "dom").where(name: "pom")
Navigate cursor to modify first where statement.
Change "som" to "SOM"
Should see it break
Press random key repeatedly to see it break even more.
Images:
Good input
Now i will navigate to first where statement to change "som" to uppercase "SOM"
I basically navigated my cursor and pressed SHIFT+S, SHIFT+O, SHIFT+M
Hope it's clear enough :-)
Thank you!
EDIT 1:
Tried using zsh instead of bash, didn't help
Disabled spring gem, didn't help
EDIT 2:
Folks on reddit suggested that i should check if there are any Ruby readline warnings eg. "Readline is not installed". None of them appear anywhere. Also reinstalled ruby 2.4.1, seems like it's not the problem in my case.
EDIT 3:
I cannot reproduce the issue in a different Rails project.
Issue appears on Rails 5.1.1, meanwhile 5.0.3 works flawlessly.
Is it possible that Rails itself (or rather one of its gems) can be the cause?
I had a very similar issue, and I boiled it down to my coloring.
I had the following in my IRBRC:
class String
def _colorize(color_code)
"\e[#{color_code}m#{self}\e[0m"
end
def red
_colorize(31)
end
def yellow
_colorize(33)
end
end
IRB.conf[:PROMPT][:MY_PROMPT] = {
:AUTO_INDENT => true, # enables auto-indent mode
:PROMPT_I => "[ME]".red + " > ".yellow, # normal prompt
:PROMPT_S => "[ME]".red + " ".yellow, # prompt for continuated strings
:PROMPT_C => "[ME]".red + " * ".yellow, # prompt for continuated statement
:RETURN => "[ME]".red + "=> ".yellow + "%s\n".red # format to return value
}
IRB.conf[:PROMPT_MODE] = :MY_PROMPT
And when I removed the .red and .yellow, everything was just fine.
I think it's because my coloring characters like \e[31m was getting counted as a length 5 characters instead of 0 characters.
My solution was to remove my coloring for the time being. Hopefully someone will come up with a better solution than that.
I have an issue with importing a lot of records from a user provided excel file into a database. The logic for this is working fine, and I’m using ActiveRecord-import to cut down on the number of database calls. However, when a file is too large, the processing can take too long and Heroku will return a timeout. Solution: Resque and moving the processing to a background job.
So far, so good. I’ve needed to add CarrierWave to upload the files to S3 because I can’t just hold the file in memory for the background job. The upload portion is also working fine, I created a model for them and am passing the IDs through to the queued job to retrieve the file later as I understand I can’t pass a whole ActiveRecord object through to the job.
I’ve installed Resque and Redis locally, and everything seems to be setup correctly in that regard. I can see the jobs I’m creating being queued and then run without failing. The job seems to run fine, but no records are added to the database. If I run the code from my job line by line in the console, the records are added to the database as I would expect. But when the queued jobs I’m creating run, nothing happens.
I can’t quite work out where the problem might be.
Here’s my upload controller’s create action:
def create
#upload = Upload.new(upload_params)
if #upload.save
Resque.enqueue(ExcelImportJob, #upload.id)
flash[:info] = 'File uploaded.
Data will be processed and added to the database.'
redirect_to root_path
else
flash[:warning] = 'Upload failed. Please try again.'
render :new
end
end
This is a simplified version of the job with fewer sheet columns for clarity:
class ExcelImportJob < ApplicationJob
#queue = :default
def perform(upload_id)
file = Upload.find(upload_id).file.file.file
data = parse_excel(file)
if header_matches? data
# Create a database entry for each row, ignoring the first header row
# using activerecord-import
sales = []
data.drop(1).each_with_index do |row, index|
sales << Sale.new(row)
if index % 2500 == 0
Sale.import sales
sales = []
end
end
Sale.import sales
end
def parse_excel(upload)
# Open the uploaded excel document
doc = Creek::Book.new upload
# Map rows to the hash keys from the database
doc.sheets.first.rows.map do |row|
{ date: row.values[0],
title: row.values[1],
author: row.values[2],
isbn: row.values[3],
release_date: row.values[5],
units_sold: row.values[6],
units_refunded: row.values[7],
net_units_sold: row.values[8],
payment_amount: row.values[9],
payment_amount_currency: row.values[10] }
end
end
# Returns true if header matches the expected format
def header_matches?(data)
data.first == {:date => 'Date',
:title => 'Title',
:author => 'Author',
:isbn => 'ISBN',
:release_date => 'Release Date',
:units_sold => 'Units Sold',
:units_refunded => 'Units Refunded',
:net_units_sold => 'Net Units Sold',
:payment_amount => 'Payment Amount',
:payment_amount_currency => 'Payment Amount Currency'}
end
end
end
I can probably have some improved logic anyway as right now I’m holding the whole file in memory, but that isn’t the issue I’m having – even with a small file that has only 500 or so rows, the job doesn’t add anything to the database.
Like I said my code worked fine when I wasn’t using a background job, and still works if I run it in the console. But for some reason the job is doing nothing.
This is my first time using Resque so I don’t know if I’m missing something obvious? I did create a worker and as I said it does seem to run the job. Here’s the output from Resque’s verbose formatter:
*** resque-1.27.4: Waiting for default
*** Checking default
*** Found job on default
*** resque-1.27.4: Processing default since 1508342426 [ExcelImportJob]
*** got: (Job{default} | ExcelImportJob | [15])
*** Running before_fork hooks with [(Job{default} | ExcelImportJob | [15])]
*** resque-1.27.4: Forked 63706 at 1508342426
*** Running after_fork hooks with [(Job{default} | ExcelImportJob | [15])]
*** done: (Job{default} | ExcelImportJob | [15])
In the Resque dashboard the jobs aren’t logged as failed. They get executed and I can see an increment in the ‘processed’ jobs on the stats page. But as I say the DB remains untouched. What’s going on? How can I debug the job more clearly? Is there a way to get into it with Pry?
It looks like my problem was with Resque.enqueue(ExcelImportJob, #upload.id).
I changed my code to ExcelImportJob.perform_later(#upload.id) and now my code actually runs!
I also added a resque.rake task to lib/tasks as described here: http://bica.co/2015/01/20/active-job-resque/.
That link also notes how to use rails runner to call the job without running the full Rails server and triggering the job, which is useful for debugging.
Strangely, I didn't quite manage to get the job to print anything to STDOUT as suggested by #hoffm but at least it led me down a good avenue of inquiry.
I still don't fully understand the difference between why calling Resqueue.enqueue still added my jobs to the queue and indeed seemed to run them, but the code wasn't executed, so if someone has a better grasp and an explanation, that would be much appreciated.
TL;DR: calling perform_later rather than Resque.enqueue fixed the problem but I don't know why.
It looks like Base CRM has upgraded their API and replaced all of their endpoints/parameters.
Previously I was able to retrieve "Won" deals using this call:
session = BaseCrm::Session.new("<LEGACY_ACCESS_TOKEN>")
session.deals.all(stage: :won, sort_by: :last_activity, sort_order: :desc, page: 1)
This query recently started ignoring my parameters, yet it continued to respond with unfiltered data (that was fun when I realized that was happening).
The new syntax is:
client = BaseCRM::Client.new(access_token: "<YOUR_PERSONAL_ACCESS_TOKEN>")
client.deals.where(organization_id: google.id, hot: true)
yet this does not work:
client.deals.where(stage_name: :won)
client.deals.where(stage_name: "Won")
client.deals.where(stage_id: 8) # specified ID found in Base Docs for "Won"
etc.
I've looked into the most recent updates to the Base CRM Gem as well as the Base CRM API Docs but have not found a solution to searching by specific deal stage.
Has anyone had any luck with the new API and this kind of query?
Is there a way to use the legacy API?
I've left message with Base but I really need to fix this, you know, yesterday.
Thanks for your help!
ADDITIONAL INFO
The legacy API/gem responded with JSON where the v2 API/gem responds with a BaseCRM::Deal object:
$ session.deals.find(123456)
# <BaseCRM::Deal
dropbox_email="dropbox#67890.deals.futuresimple.com",
name="Cool Deal Name",
owner_id=54321,
creator_id=65432,
value=2500,
estimated_close_date=nil,
last_activity_at="2016-04-21T02:29:43Z",
tags=[],
stage_id=84588,
contact_id=098765432,
custom_fields={:"Event Location"=>"New York, NY", :Source=>"Friend"},
last_stage_change_at="2016-04-21T02:08:20Z",
last_stage_change_by_id=559951,
created_at="2016-04-18T22:16:35Z",
id=123456,
updated_at="2016-04-21T02:08:20Z",
organization_id=nil,
hot=false,
currency="USD",
source_id=1466480,
loss_reason_id=nil
>
Checkout stage_id. Is this a bug? According to the Docs stage_id should return an integer between 1 and 10.
One thing I miss about ipython is it has a ? operator which diggs up the docs for a particular function.
I know ruby has a similar command line tool but it is extremely inconvenient to call it while I am in irb.
Does ruby/irb have anything similar?
Pry is a Ruby version of IPython, it supports the ? command to look up documentation on methods, but uses a slightly different syntax:
pry(main)> ? File.dirname
From: file.c in Ruby Core (C Method):
Number of lines: 6
visibility: public
signature: dirname()
Returns all components of the filename given in file_name
except the last one. The filename must be formed using forward
slashes (/'') regardless of the separator used on the
local file system.
File.dirname("/home/gumby/work/ruby.rb") #=> "/home/gumby/work"
You can also look up sourcecode with the $ command:
pry(main)> $ File.link
From: file.c in Ruby Core (C Method):
Number of lines: 14
static VALUE
rb_file_s_link(VALUE klass, VALUE from, VALUE to)
{
rb_secure(2);
FilePathValue(from);
FilePathValue(to);
from = rb_str_encode_ospath(from);
to = rb_str_encode_ospath(to);
if (link(StringValueCStr(from), StringValueCStr(to)) < 0) {
sys_fail2(from, to);
}
return INT2FIX(0);
}
See http://pry.github.com for more information :)
You can start with
irb(main):001:0> `ri Object`
Although the output of this is less than readable. You'd need to filter out some metacharacters.
In fact, someone already made a gem for it
gem install ori
Then in irb
irb(main):001:0> require 'ori'
=> true
irb(main):002:0> Object.ri
Looking up topics [Object] o
= Object < BasicObject
------------------------------------------------------------------------------
= Includes:
Java (from gem activesupport-3.0.9)
(from gem activesupport-3.0.9) [...]
No, it doesn't. Python has docstrings:
def my_method(arg1,arg2):
""" What's inside this string will be made available as the __doc__ attribute """
# some code
So, when the ? is called from ipython, it probably calls the __doc__ attribute on the object. Ruby doesn't have this.