I have the following thor command:
require 'highline'
class Import < Thor
desc "files", "Import files into the database"
method_option "path", :required => true, :desc => "Path to folder containing new files", :aliases => "-p", :type => :string
def files
require './config/environment'
line = HighLine.new
line.say(line.color("Identified files as Version 15 (English)", :green))
if line.agree(line.color("Are you sure you want to import?", :yellow))
line.say(line.color("Finished. Imported 70,114 items", :green))
else
line.say(line.color("Aborting...", :red))
end
end
end
Now, obviously, at the moment this is just outputting some language to the screen. However, what I need to do is write a test for the command that tests the output is as I would expect, and that when I start hooking in the heavy lifting that I can stub that stuff out.
I've had a look at Aruba, but this doesn't appear to like interactivity for some reason, and it's not clear why.
Therefore, does anyone have any ideas on how this might be testable (with RSpec)?
Aruba is a pretty complete set of steps for testing command line apps. If it's not working for you it might be because aruba defaults all file operations into tmp/aruba.
But neimOo is right about how to write the scenario with aruba
When I run `thor import` interactively
And I type "yes"
Here is how you can do this with Aruba
Scenario: Test import
When I run `thor import` interactively
And I type "yes"
Then the stdout should contain "Finished. Imported 70,114 items"
Here you can find a lot of aruba examples
https://github.com/cucumber/aruba/blob/master/features/interactive.feature
And here is implementation itself
https://github.com/cucumber/aruba/blob/master/lib/aruba/cucumber.rb
Related
I have an issue with importing a lot of records from a user provided excel file into a database. The logic for this is working fine, and I’m using ActiveRecord-import to cut down on the number of database calls. However, when a file is too large, the processing can take too long and Heroku will return a timeout. Solution: Resque and moving the processing to a background job.
So far, so good. I’ve needed to add CarrierWave to upload the files to S3 because I can’t just hold the file in memory for the background job. The upload portion is also working fine, I created a model for them and am passing the IDs through to the queued job to retrieve the file later as I understand I can’t pass a whole ActiveRecord object through to the job.
I’ve installed Resque and Redis locally, and everything seems to be setup correctly in that regard. I can see the jobs I’m creating being queued and then run without failing. The job seems to run fine, but no records are added to the database. If I run the code from my job line by line in the console, the records are added to the database as I would expect. But when the queued jobs I’m creating run, nothing happens.
I can’t quite work out where the problem might be.
Here’s my upload controller’s create action:
def create
#upload = Upload.new(upload_params)
if #upload.save
Resque.enqueue(ExcelImportJob, #upload.id)
flash[:info] = 'File uploaded.
Data will be processed and added to the database.'
redirect_to root_path
else
flash[:warning] = 'Upload failed. Please try again.'
render :new
end
end
This is a simplified version of the job with fewer sheet columns for clarity:
class ExcelImportJob < ApplicationJob
#queue = :default
def perform(upload_id)
file = Upload.find(upload_id).file.file.file
data = parse_excel(file)
if header_matches? data
# Create a database entry for each row, ignoring the first header row
# using activerecord-import
sales = []
data.drop(1).each_with_index do |row, index|
sales << Sale.new(row)
if index % 2500 == 0
Sale.import sales
sales = []
end
end
Sale.import sales
end
def parse_excel(upload)
# Open the uploaded excel document
doc = Creek::Book.new upload
# Map rows to the hash keys from the database
doc.sheets.first.rows.map do |row|
{ date: row.values[0],
title: row.values[1],
author: row.values[2],
isbn: row.values[3],
release_date: row.values[5],
units_sold: row.values[6],
units_refunded: row.values[7],
net_units_sold: row.values[8],
payment_amount: row.values[9],
payment_amount_currency: row.values[10] }
end
end
# Returns true if header matches the expected format
def header_matches?(data)
data.first == {:date => 'Date',
:title => 'Title',
:author => 'Author',
:isbn => 'ISBN',
:release_date => 'Release Date',
:units_sold => 'Units Sold',
:units_refunded => 'Units Refunded',
:net_units_sold => 'Net Units Sold',
:payment_amount => 'Payment Amount',
:payment_amount_currency => 'Payment Amount Currency'}
end
end
end
I can probably have some improved logic anyway as right now I’m holding the whole file in memory, but that isn’t the issue I’m having – even with a small file that has only 500 or so rows, the job doesn’t add anything to the database.
Like I said my code worked fine when I wasn’t using a background job, and still works if I run it in the console. But for some reason the job is doing nothing.
This is my first time using Resque so I don’t know if I’m missing something obvious? I did create a worker and as I said it does seem to run the job. Here’s the output from Resque’s verbose formatter:
*** resque-1.27.4: Waiting for default
*** Checking default
*** Found job on default
*** resque-1.27.4: Processing default since 1508342426 [ExcelImportJob]
*** got: (Job{default} | ExcelImportJob | [15])
*** Running before_fork hooks with [(Job{default} | ExcelImportJob | [15])]
*** resque-1.27.4: Forked 63706 at 1508342426
*** Running after_fork hooks with [(Job{default} | ExcelImportJob | [15])]
*** done: (Job{default} | ExcelImportJob | [15])
In the Resque dashboard the jobs aren’t logged as failed. They get executed and I can see an increment in the ‘processed’ jobs on the stats page. But as I say the DB remains untouched. What’s going on? How can I debug the job more clearly? Is there a way to get into it with Pry?
It looks like my problem was with Resque.enqueue(ExcelImportJob, #upload.id).
I changed my code to ExcelImportJob.perform_later(#upload.id) and now my code actually runs!
I also added a resque.rake task to lib/tasks as described here: http://bica.co/2015/01/20/active-job-resque/.
That link also notes how to use rails runner to call the job without running the full Rails server and triggering the job, which is useful for debugging.
Strangely, I didn't quite manage to get the job to print anything to STDOUT as suggested by #hoffm but at least it led me down a good avenue of inquiry.
I still don't fully understand the difference between why calling Resqueue.enqueue still added my jobs to the queue and indeed seemed to run them, but the code wasn't executed, so if someone has a better grasp and an explanation, that would be much appreciated.
TL;DR: calling perform_later rather than Resque.enqueue fixed the problem but I don't know why.
I am having a fully fledged suite for automated tests written using ruby with Appium for mobile automation.
I am running these suites in one simulator in one machine and it takes up a lot of time, around 1 hour to run 56 test cases(We have system test cases where multiple checks like database/Api/functional are integrated). We have more additional test cases adding to our way.
We have implemented running our tests across 3 mac machines currently with running different cucumber tags integrated to Jenkins. However, more addition of tests is only going to take us more time or more mac's
With xcode 9 we can initiate multiple simulators on one machine at the same time, and I had like to know, if there is any sample scenario or documentation on how to implement distributed tests across simulators in one mac machine
I had tried loading two or three different desired capabilities with different platform version, but it only loads the tests in sequential order.
I had gone through a lot of material online that has only the steps to make this possible in android. Does iOS support it?
Or could anyone possibly provide links that would help me? to
1. Implement distributed tests across various simulators in one mac
2. Use cucumber tags to distribute tests creating instance for each desired capability
Update:
I had tried implementing the multithread option and tried to initiate the tests to specific simulator creating instance with each thread. However, I find the tests not running in parallel but sequential.
This is my code:
def start_app(device_id, wdalocalport)
caps_config = {
platformName: "iOS",
wdaLocalPort: wdalocalport,
deviceName: "iPhone Simulator", #update device as per your need
app: (File.join(File.dirname(__FILE__), "Sequoia.app")),
bundleId: "something",
automationName: "XCUITest",
xcodeOrgId: "something",
xcodeSigningId: "iPhone Developer",
#platformVersion: "10.2",
noReset: "true",
fullReset: "false",
showIOSLog: "true",
autoAcceptAlerts: "true",
showXcodeLog: "true",
useNewWDA: "true",
resetOnSessionStartOnly: "true",
udid: device_id }
appium_lib_config={ port: 4723 }
$opts={ caps: caps_config, appium_lib: appium_lib_config }
setup
end
def setup
#appium = Appium::Driver.new($opts)
#appium.start_driver
#Makes all appium_lib methods accessible from steps
#Starts appium driver before the tests begin
end
def test(device1,device2)
threads = []
threads << Thread.new {
start_app(device1, '8100')
}
threads << Thread.new {
start_app(device2, '8200')
}
threads.each(&:join)
end
end
I am calling the launch tests using the test method passing the udid's. The simulators launch at the same time, and also installs the application at the same time, but the tests aren't parallel.
Any help to improvise this case?
I was able to use the rake to parallel run, but I still find this approach runs the tests in sequential manner or doesnt run at all
PFB the code
def run_cucumber(cucumber_options)
Cucumber::Rake::Task.new do |t|
t.cucumber_opts = cucumber_options
end
Rake::Task[:cucumber].invoke
end
task :iphone_7 do |t|
ENV['DEVICE'] = 'iphone7'
run_cucumber('-r features features/test.feature --format pretty --tags #slave1')
end
task :iphone_8 do |t|
ENV['DEVICE'] = 'iphone8'
run_cucumber('-r features features/test.feature --format pretty --tags #slave2')
end
multitask :all => [:iphone_7,:iphone_8]
My hooks.rb
Before do
check
end
def check
if ENV['DEVICE'] == 'iphone7'
start_app('iPhone6','port','udid')
elsif ENV['DEVICE'] == 'iphone8'
start_app('iphone6','port','udid')
else
puts "Device not"
end
end
I have been getting DEVICE NOT. Not sure what am I missing.
You need to create a method which invokes cucumber rake first in the Rakefile as follows:
def run_cucumber(cucumber_options)
Cucumber::Rake::Task.new do |t|
t.cucumber_opts = cucumber_options
end
Rake::Task[:cucumber].invoke
end
This method will need cucumber_options. You can pass same options as you pass on the command line like this:
-r features features/test.feature --format pretty --tags #test1
After this, define rake tasks for a particular configuration in the same Rakefile:
task :iphone_7 do |t|
ENV["DEVICE"] = "iphone 7"
run_cucumber('-r features features/test.feature --format pretty --tags #test1')
end
task :iphone_8 do |t|
ENV["DEVICE"] = "iphone 8"
run_cucumber('-r features features/test.feature --format pretty --tags #test2')
end
Here, ENV["DEVICE"] is used set to device config. You can use this variable in your env.rb in Before block to initialize the test. In Before block you can initialize devices based on this environment variable.
You can create rake tasks like this as many as you need.
Then you can invoke multiple tasks at the same time by using ruby's parallel gem. Examples are in this link -> Ruby Parallel gem
Just pass different types of rake task as the arguments in the Parallel object.
Eg:
task :run_parallel do |t|
Parallel.map([:iphone_7,:iphone_8]) {|task| Rake::Task[task].invoke }
end
And then trigger the run_parallel rake task to execute the whole code.
Eg:
bundle exec rake run_parallel
Iam trying for HOURS now and cant figure out the problem.
Iam new to clojure and try to make a simple flickr client.
But I have this problem with different libraries... I think I make a general mistake.
At the moment I try to use oauth with this lib: https://github.com/mattrepl/clj-oauth
lein new projectname
and my project.clj looks like this:
(defproject flickr "0.1.0-SNAPSHOT"
:description "FIXME: write description"
:url "http://example.com/FIXME"
:license {:name "Eclipse Public License"
:url "http://www.eclipse.org/legal/epl-v10.html"}
:dependencies [[org.clojure/clojure "1.4.0"]
[clj-oauth "1.4.0"]])
After adding the clj-oauth2 I run:
lein deps
And my core.clj:
(ns flickr.core)
(require ['oauth.client :as 'oauth])
(def consumer-key "0000")
(def consumer-secret "0000")
(def consumer (oauth.client/make-consumer <consumer-token>
<consumer-token-secret>
"http://www.flickr.com/services/oauth/request_token"
"http://www.flickr.com/services/oauth/access_token"
"http://www.flickr.com/services/oauth/authorize"
:hmac-sha1))
When I now try to run it:
lein run
I get:
FileNotFoundException Could not locate oauth/client__init.class or oauth/client.clj on classpath: clojure.lang.RT.load (RT.java:432)
Does anyone has an idea where the problem is?
Also also downloaded the oauth source from the github repo, built it and added it to my $PATH variable but still the same error.
Any help would be appreciated!
Thanks!
First, lein run looks for a main namespace, whose name must be specified in project.clj using the :main key; add :main flickr.core there.
Then you need a -main function in flickr.core. Change your namespace declaration and add the function as follows:
(ns flickr.core
(:require [clj-oauth2.client :as oauth]))
(defn -main []
(println oauth/get-access-token))
Then,
$ lein run
;=> #<client$get_access_token clj_oauth2.client$get_access_token#4c9549af>
That worked for me as a sort of "namespace smoke test," and you should be able to go from there.
(As a final note, your development will go much faster if you test these sorts of things in the REPL rather than using 'lein run.')
It appears there's confusion with your dependencies. According to Clojars, the library you're using clj-oauth2 is this GitHub project https://github.com/DerGuteMoritz/clj-oauth2, not the one you link to in the question.
If you want the latest clj-oauth, then the dependency should be [clj-oauth "1.4.0"] (for the latest version). If you need clj-oauth2 then the above GitHub link should be the reference.
I want to somehow ask the user to say their flickr_id, flickr_apikey and that stuff, but id' be most happy to do it under my install command so it dosn't end up being such a long and heavy line because of alle the arguments.
so something like
$ thor PhotoonRails:install
We're about to install your system.. blaa, blaa, blaa...
We have to know you're Flick ID, get i here http://idgettr.com/
Flickr ID: {here you should type your id}
We also has to know you're flick api key, make one here ...
API Key: {here you should type your key}
and so on? Do you get the idea, and can it be done?
Indeed it can!
You are looking for ask.
An example:
class PhotoonRails < Thor
desc "install", "install my cool stuff"
def install
say("We're about to install your system.. blaa, blaa, blaa... We have to know you're Flick ID, get i here http://idgettr.com")
flickr_id = ask("Flickr ID: ")
say("We also has to know you're flick api key, make one here ...")
flickr_api_key = ask("API Key: ")
# validate flickr creds
# do cool stuff
say("Complete!", GREEN)
end
end
It's also possible to set color as a symbol
say "Caution!", :yellow
ask 'Agreed?', :bold
# Choose limit:
ask "We have noticed.", :green, limited_to: ['proceed', 'exit']
# Default value (possible without :blue)
ask 'Type app name', :blue, default: 'blog'
Full list of available colors for Thor, here: http://www.rubydoc.info/github/wycats/thor/Thor/Shell/Color
I'm adding a test task in my Rakefile, similar to this:
namespace :test do
desc "Test lib source"
Rake::TestTask.new(:lib) do |t|
t.libs << "test"
t.pattern = 'test/lib/**/*_test.rb'
t.verbose = true
end
end
and then adding (have also done using "enhance" with the same result:
task :test => [ 'test:lib' ]
My problem is that if there is an error encountered in test:lib, the suite stops running. That's not a terrible thing, but ideally it would go on to run the rest of the suite to let me know that there are more issues later in the suite.
Anyone know how to make it report the errors/failures in test:lib but go on to run the full suite?
Thanks!
Used something along these lines (minus the "on the fly" bit):
http://toolmantim.com/articles/creating_rake_testtasks_on_the_fly