Login to amazon partnernet using Ruby Mechanize - ruby-on-rails

Question

I try to login into amazon partnernet, e.g. https://partnernet.amazon.de/ using the Ruby Mechanize gem:
Gemfile:
# https://github.com/sparklemotion/mechanize
gem 'mechanize'
The code below is a rake task. It worked in the past, I think Amazon changed the page html so this code is no longer working, e.g. by changing the submit button of the form name="sign_in" to an image type=input.
desc "Cron Task for Email Notifications"
task :email_amazon_stats => :environment do
puts "Start: Fetch and send Amazon Sales from yesterday (#{Time.now})"
# login to Amazon Partnernet
a = Mechanize.new
a.user_agent_alias = 'Mac Safari'
a.follow_meta_refresh = true
a.redirect_ok = true
a.get('https://partnernet.amazon.de/') do |page|
# Submit the login form
page.form_with(:name => 'sign_in') do |f|
username_field = f.field_with(:id => "username")
username_field.value = "email#example.com"
password_field = f.field_with(:id => "password")
password_field.value = "somepassword"
end.submit
start_date = Time.now - 1.day
end_date = Time.now
my_page2 = a.get("https://partnernet.amazon.de/gp/associates/network/reports/report.html?ie=UTF8&deviceType=all&endDay=#{(end_date.strftime('%d').to_i).to_s}&endMonth=#{((end_date.strftime('%m').to_i)-1).to_s}&endYear=#{end_date.strftime('%Y').to_i.to_s}&periodType=exact&preSelectedPeriod=monthToDate&program=all&reportType=earningsReport&startDay=#{start_date.strftime('%d').to_i.to_s}&startMonth=#{((start_date.strftime('%m').to_i)-1).to_s}&startYear=#{start_date.strftime('%Y').to_s}")
form = my_page2.form_with(:name => 'htmlReport')
button = form.button_with(:name => 'submit.download_XML')
xml = a.submit(form, button)
# ASIN="3423347570"
# Binding="paperback"
# Category="14"
# Date="December 01, 2015"
# DeviceType="BROWSER"
# EDate="1448928000"
# Earnings="0,65"
# LinkType="asn"
# Price="9,25"
# Qty="1"
# Rate="7,03"
# Revenue="9,25"
# Seller="Amazon.de"
# Tag="yx-21"
# Title="Kopf schlägt Kapital: Die ganz andere Art, ein Unternehmen zu gründen Von der Lust, ein Entrepreneur zu sein (dtv Sachbuch)"/>
doc = Nokogiri::XML(xml.body)
#sales = []
doc.xpath("//Item").each do |item|
#sales << {
:sale_itemasin => item['ASIN'],
:sale_itemname => item['Title'].truncate(80),
:sale_date => Time.at(item['EDate'].to_i).strftime("%Y-%m-%d %H:%M:%S").to_s,
:sale_amount => '%.2f' % item['Revenue'].gsub(',','.').to_f,
:sale_commission => '%.2f' % item['Earnings'].gsub(',','.').to_f
}
end
earnings = 0
#sales.each do |s|
earnings += s[:sale_commission].to_f
end
#total_commission = '%.2f' % earnings
end
ReportsMailer.daily_dashboard(#total_commission,#sales).deliver
puts "Done: Fetch and send Amazon Sales from yesterday (#{Time.now})"
end
Can someone help me in this?
--
I looked for similar questions how to restructure the submit, but so far nothing works. Login is not happening. (Yes, PWD is correct :-) )
Similar question, but does not solve the problem above: Cannot Login to Amazon with Ruby Mechanize

Answer 1

So.. I debugged the code, this version is working now as expected:
desc "Cron Task for Email Notifications"
task :email_amazon_stats => :environment do
puts "Start: Fetch and send Amazon Sales from yesterday (#{Time.now})"
agent = Mechanize.new
agent.cookie_jar.clear!
agent.user_agent_alias = 'Mac Firefox'
agent.follow_meta_refresh = true
agent.redirect_ok = true
dashboard_url = "https://partnernet.amazon.de/gp/associates/network/reports/report.html?__mk_de_DE=%C3%85M%C3%85%C5%BD%C3%95%C3%91&tag=&reportType=earningsReport&program=all&deviceType=all&periodType=preSelected&preSelectedPeriod=yesterday&startDay=1&startMonth=11&startYear=2016&endDay=2&endMonth=11&endYear=2016&submit.display.x=87&submit.display.y=16&submit.display=Auf+der+Seite+anzeigen"
agent.get(dashboard_url)
form = agent.page.form_with(:name => 'sign_in')
form.username = ENV['AZON_PARTNER_USR']
form.password = ENV['AZON_PARTNER_KEY']
form.submit
dashboard = agent.get(dashboard_url)
form2 = dashboard.form_with(:name => 'htmlReport')
button = form2.button_with(:name => 'submit.download_XML')
xml = agent.submit(form2, button)
doc = Nokogiri::XML(xml.body)
#sales = []
doc.xpath("//Item").each do |item|
#sales << {
:sale_itemasin => item['ASIN'],
:sale_itemname => item['Title'].truncate(80),
:sale_date => Time.at(item['EDate'].to_i).strftime("%Y-%m-%d %H:%M:%S").to_s,
:sale_amount => '%.2f' % item['Revenue'].gsub(',','.').to_f,
:sale_commission => '%.2f' % item['Earnings'].gsub(',','.').to_f
}
end
earnings = 0
#sales.each do |s|
earnings += s[:sale_commission].to_f
end
#total_commission = '%.2f' % earnings
ReportsMailer.daily_dashboard(#total_commission,#sales).deliver
puts "Done: Fetch and send Amazon Sales from yesterday (#{Time.now})"
end
As you can see yourself this is pretty ugly, because I try to go to the deeplink directly, which redirects me to the login page. There I login and try again to go to the dashboard. This time it works. Why ugly? Because if I try to go to the login page directly the code does not work, I somehow need this redirect. Any idea why? Would be interesting to understand this...