Code: https://github.com/jmopr/job-hunter/blob/master/scraper.rb
So i'm running a scraper on indeed.com but it seems to be unable to find the field q, which is the first field on the left. I'm 100% certain that the code is correct but i'm not sure why it isn't able to view the form. save_and_open_page CONFIRMS that it was able to view indeed.com successfully... yet is unable to locate the form.
Code:
def perform_search
# For indeed0
save_and_open_page
fill_in 'q', :with => #skillset
fill_in 'l', :with => #region
find('#fj').click
sleep(1)
end
Indeed.com HTML:
<input class="input_text" maxlength="512" size="31" aria-labelledby="what_label_top hidden_colon what_label_bot" name="q" autocomplete="off" id="what">
Required portions/redirects of the page are loaded from 'indeed.com' rather than 'www.indeed.com' - Change your capybara-webkit configuration to allow indeed.com.
Additionally, if you want to know what the driver is actually seeing, you're generally better off using save_and_open_screenshot rather that save_and_open_page (The latter will show you the html structure but not how capybara-webkit has rendered it)
Related
I get the error:
Capybara::ElementNotFound:
Unable to find field "user_email"
And this is the test code:
feature 'User' do
given!(:user) { User.new(email: 'testuserid#example.com', encrypted_password: 'test') }
scenario 'opens sign_up page' do
visit new_user_session_path
expect(page).to have_content 'unique text on the page'
end
scenario 'signs in with invalid email' do
visit new_user_session_path
fill_in('user_email',with: 'ssd')
expect(page).to have_content 'unique text on the page'
end
end
My HTML file consists of this code literally:
unique text on the page
<br>
<input type="text" id="user_email">
So this proves that the path is correct because my first scenario runs correctly. It is visiting the right page. But still I get this error for second scenario in fill_in.
I have also tried element = page.find("user_email"), it gives same error.
What am I possibly doing wrong?
I have been scratching my head like hell.
Usually the reason for this is that the input isn't actually visible on the page. You can verify this by doing
fill_in('user_email', with: 'ssd', visible: false)
If that succeeds in finding the element, then you need to change your test to first perform whatever actions make the field visible before attempting to fill it in.
Your code seems right. Maybe you are visiting wrong url or you have used user_email id once more. But you can give a try with alternative syntax like following :
find("input[id$='user_email']").set "ssd"
With Rails, rspec and capybara, I'm trying to test your typical ERB generated form:
<form action="/pages/1" id="edit_page_1" method="post">
<input id="page_title" name="page[title]" type="text">
<input name="commit" type="submit" value="Update Page">
</form>
I run two kinds of feature specs, those that are the same no matter what the language, and those that are I18N specific (for internationalization testing).
The problem is there is no clear way to click that submit button with capybara, unless I'm missing the obvious. I would expect simply click('commit') to do the trick.
Using click_button('Update Page') works but is obviously language specific and can't be used with both the New and Edit templates even though they render the same form template.
Adding an id to the submit button works, but I strongly dislike changing the code exclusively because the test requires it.
Using a css or xml matcher both looks ugly (a user would never know/care about accessing an element that way) and it is overkill.
In the end a macro was the answer I needed as apparently there is nothing native in capybara.
# spec/support/form_helpers.rb
module FormHelpers
def submit_form
find('input[name="commit"]').click
end
end
This gets included in spec_helper.rb
RSpec.configure do |config|
config.include FormHelpers, :type => :feature
...etc...
I used :type => :feature so it gets included only in the integration tests.
In the integration tests you can use it like this:
scenario 'pages can be created' do
visit new_page_path
fill_in 'page_title', with: 'A Tale of Two Cities'
submit_form # Clicks the commit button regardless of id or text
expect(page).to have_content 'The page was created'
...etc..
end
Of course submit_form can also be used inside within blocks and with :js => true.
I usually do:
within 'form#edit_page_1' do
find('input[name="page[title]"]').set "Some Value"
find('input[name="commit"]').click
end
Its tied to the html but to its semantic attributes, so I feel like its fine.
Actually I never use the magical finders.
Btw I dont understand your comment: (a user would never know/care about accessing an element that way).
Integration specs are for you, it mimics a user for sure but its just a matter of giving proper instructions.
Try to use:
find('input[name="commit"]').click
It helps
I'm running Rails 4 using capybara and rspec (OS is Ubuntu 13.10). I'm having a problem -- when I run rspec my specs work, including those that use capybara's fill_in methods. However in one spec I need to use capybara's find_field method, and it is not functioning at all. It gives me the following error:
Failure/Error: page.find_field('From:').set(Date.today - 1.month)
Capybara::ElementNotFound:
Unable to find field "From:"
Now, I have inserted a "puts page.html" line immediately before the page.find_field... line and the html it prints includes the following lines:
<div class="date-range-picker">
<span class="from-date"><label for="from_date">From:</label> <input id="from_date" name="from_date" type="date" value="2013-12-23" /></span>
<span class="to-date"><label for="to_date">To:</label> <input id="to_date" name="to_date" type="date" value="2013-12-30" /></span>
</div>
So the element is there, but not being picked up by the find_field method. Any ideas?
OK, after much meandering through Capybara's source files I found the problem. It seems that the #find_field method doesn't work properly when using Capybara-webkit. The method only failed on examples that had the js: true argument, so that should have been the first clue. Anyway it seems that the cause of this is some method naming conflict between capybara and capybara-webkit, but I didn't analyze it too closely and so I can't be sure.
I changed the find_field('from_date') to find('#from_date') and everything works now. It also worked when changing to the :rack_test driver, but since I need webkit that's what I'll stick too. Is this issue documented anywhere??
As for capybara documentation:
"Find a form field on the page. The field can be found by its name, id or label text."
so use this code instead:
page.find_field('form_date').set(Date.today - 1.month)
So you are selecting the field by it's id.
I placed your HTML fragment within an HTML page and was able to successfully do a find_field('From:') on it, which suggests the problem lies within the containing HTML. My "idea" would be to strip down the page until the find_field succeeds as a means of isolating the offending content.
I found this on another site and it slightly worked for me:
Add this to "support/select_from_chosen.rb":
module SelectFromChosen
# select_from_chosen('Option', from: 'id_of_field')
def select_from_chosen(item_text, options)
field = find_field(options[:from], :visible => false)
find("##{field[:id]}_chosen").click
find("##{field[:id]}_chosen ul.chosen-results li", :text => item_text).click
end
end
RSpec.configure do |config|
config.include SelectFromChosen, type: :feature
end
I was able to do:
select_from_chosen('Blue', from: 'car_colors')
find('#submit-button').click
expect(page).to have_content 'Blue'
select_from_chosen('Red', from: 'car_colors')
find('#submit-button').click
expect(page).to have_content 'Red'
What I was actually trying to do was validate that the options in the chosen results where available for some users and not for others so I wound up using:
find('#car_colors_chosen').click
expect(page).to have_content 'Blue'
expect(page).not_to have_content 'Gold'
Just wanted to add this incase it helped someone else,
When I fill forms using capybara and page opens in browser, the fields are empty. Is it normal?
page.fill_in 'Email', :with => 'test#gmail.com'
save_and_open_page
It seems that filling stuff into a text field doesn't make it part of the DOM (specifically, the value attribute of the <input> element), so it won't show up in the page you get.
So yes, it looks like this is normal indeed.
So I'm doing BDD with Cucumber and have a form with checkboxes populated from a database. The labels for the checkboxes contain hyperlinks. So far, not too exotic (note, this is HAML and not Erb, but it should be readable enough for any Rails person):
I would like my donation to support:
%br
- for podcast in #podcasts
= check_box_tag "donation[podcast_ids][]", podcast.id, true
= donation.label "donation[podcast_ids][]", link_to(podcast.name, podcast.url), :value => podcast.id
%br
The problem is that in my Cucumber features, I can't figure out how to find that checkbox to check it. The relevant part of the story is this:
Scenario: Happy path
Given I am on the home page
When I fill in "My email address" with "john#example.org"
# Skipped for brevity...
And I check the "Escape Pod" podcast
And I check the "PodCastle" podcast
And I press "I'm ready!"
Then I should see "Thank you!"
And there should be 2 podcast donation records
If I'm using the bare webrat_steps.rb file I get the following error:
Could not find field: "Escape Pod" (Webrat::NotFoundError)
I'm quite certain it's because of that link_to() method, which I'm using to make "Escape Pod" a hyperlink to the actual Web site. But I can't easily access link_to from my Cucumber step, and I can't figure out any reasonable way of pointing Webrat at the right checkbox short of kludging up a whole bunch of hyperlink code in my step (which makes it very brittle).
My BDD is stalled at this point. I don't want to take out the link just because it's hard to test. And it feels like it shouldn't be hard to test. Webrat is just limiting what I can pass into the checks() method. Can anyone suggest an elegant answer for this?
The short answer is the to use field_by_xpath or one of the other Webrat::Locators methods to select what element to manipulate in your step:
When(/^I check the "(.+?)" podcast$/) do |name|
check(field_by_xpath("//label/a[.=#{name}]")
end
You might need to play with that xpath a little, or use field_by_id instead. Remember it is looking got the html id of the tag not the id from the database.
Can you post what your HTML looks like in the rendered page near the problematic checkbox(es)? Sometimes you have to play with naming the field... I had all sorts of trouble with a login form... I ended up doing this:
<%= submit_tag 'Enter', {:id => "login_button"} %>
So that the following worked:
Given /^I am logged in as admin$/ do
visit login_path
fill_in "login", :with => "admin"
fill_in "password", :with => "password"
# click_button "login_button"
click_button
end
I know it's not a checkbox example, but maybe fiddling with your name/id/etc will work