I'm trying to export Windows devices to a CSV from Rapid 7 using the API and Ruby.
When I run the below script it is throwing an error about:
NoMethodError: undefined method 'each' for #<Nexpose::AdhocReportConfig:0x000000
029bf4b8>
from (irb):232:in block in irb_binding
from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:1299:in 'open'
from (irb):231
from C:/Ruby24-x64/bin/irb.cmd:19:in '<main>'
The code that I'm running is:
require 'nexpose'
require 'csv'
include Nexpose
query = "
SELECT da.host_name AS Name, dos.description AS OS
FROM dim_asset da
JOIN dim_operating_system dos USING (operating_system_id)
JOIN dim_host_type dht USING (host_type_id)
JOIN dim_site_asset dsa USING (asset_id)
JOIN dim_site ds USING (site_id)
Where (dos.description LIKE '%Windows%' AND da.host_name IS NOT NULL)"
#nsc = Connection.new('192.168.0.1', 'user', 'pswd')
#nsc.login
report = Nexpose::AdhocReportConfig.new(nil, 'sql')
report.add_filter('version', '1.1.0')
report.add_filter('query', query)
#nsc.logout
headers = ["Name","OS"]
CSV.open('C:\file.csv', 'wb', { force_quotes: true }) do |csv|
report.each do |reports|
if csv.tell() == 0 # file is empty, so write header
csv << headers
end
csv << [report.Name, report.OS]
end
end
You can't call .each on an instance of Nexpose::AdhocReportConfig
Try something like this:
report = Nexpose::AdhocReportConfig.new(nil, 'sql')
report.add_filter('version', '1.1.0')
report.add_filter('query', query)
report_output = report.generate(#nsc)
...
CSV.open('C:\file.csv', 'wb', { force_quotes: true }) do |csv|
report_output.each do |reports|
...
See a full example here: https://community.rapid7.com/docs/DOC-2733
Related
I'm having a strange issue where when I check the File.size of a particular file in Rails console, it returns the correct size. However when I run the same code in a rake task, it returns 0. Here is the code in question (I've tidied it up a bit to help with readability):
def sum_close
daily_closed_tickets = Fst.sum_retrieve_closed_tickets
daily_closed_tickets.each do |ticket|
CSV.open("FILE_NAME_HERE", "w+", {force_quotes: false}) do |csv|
if (FileCopyReceipt.exists?(path: "#{ticket.attributes['TroubleTicketNumber']}_sum.txt"))
csv << ["GENERATE CSV WITH ATTRIBUTES HERE"]
files = Dir.glob("/var/www/html/harmonize/public/close/CLOSED_#{ticket.attributes['TroubleTicketNumber']}_sum.txt")
files.each do |f|
Rails.logger.info "File size (should return non-0): #{File.size(f)}" #returns 0, but not in Rails Console
Rails.logger.info "File size true or false, should be true: #{File.size(f) != 0}" #returns false, should return true
Rails.logger.info "Rails Environment: #{Rails.env}" #returns production
if(!FileCopyReceipt.exists?(path: f) && (File.size(f) != 0))
Rails.logger.info("SUM CLOSE, GOOD => FileUtils.cp_r occurred and FileCopyReceipt object created")
else
Rails.logger.info("SUM CLOSE, WARNING: => no data transfer occurred")
end
end
else
Rails.logger.info("SUM CLOSE => DID NOT make it into initial if ClosedDate.present? if block")
end
end
end
close_tickets.rake
task :close_tickets => :environment do
tickets = FstController.new
tickets.sum_close
tickets.dais_close
end
It is beyond me why this File.size comes back as 0 when this is run as a rake task. I thought it may be a environment issue, but that does not seem to be the case.
Any insight on the matter is appreciated.
The CSV.open block and everything being wrapped in there was causing issues. So I just made CSV generation it's own snippet instead of wrapping everything in there.
daily_closed_tickets.each do |ticket|
CSV.open("generate csv here.txt") do |csv|
#enter ticket.attributes here for the csv
end
#continue on with the rest of the code and File.size() works properly
end
I am trying to write to a CSV but i ran into a problem. I have seen this so I have just applied the solution but I get an error.
This is my code:
require 'csv'
data = Owner.find(2).cats
CSV.open("file.csv", "w") do |csv|
data.each do |cat|
csv << cat.name
end
end
I have checked in console and I am getting data for Owner.find(2).cats.
When trying to write this to my CSV i get the error:
undefined method `map' for 0:Fixnum
and when I try the simple solution from the same question :
require 'csv'
CSV.open("file.csv", "w") do |csv|
csv << "\n"
end
I get this error:
undefined method `map' for "\n":String
Do you know what I am doing wrong?
I am new to ruby so maybe I am doing one of the roockie mistakes
A CSV is a collection of rows, each of which is a collection of columns; it's a two-dimensional array, that gets converted to text form. So the top-level CSV object expects you to append arrays to it, not individual cell values.
Note that in this code:
CSV.open('filename','w') do |csv|
do stuff
end
The do stuff is only run exactly once. It's up to you to create the structure of the CSV, usually with something like this:
CSV.open('filename','w') do |csv|
data.each |item|
row = [item.field1, item.field2, item.field3]
csv << row
end
end
or even a double loop:
CSV.open('filename','w') do |csv|
data.each |item|
row = []
fields.each do |field|
row << item[field]
end
csv << row
end
end
As an example:
$ irb
irb(main):001:0> require 'csv' #=> true
irb(main):002:0> CSV.open("cats.csv", "w") do |csv|
irb(main):003:1* csv << [ "cat1" ] << [ "cat2" ] << [ "cat3 " ]
irb(main):004:1> end
#=> <#CSV io_type:File io_path:"cats.csv" encoding:UTF-8 lineno:3 col_sep:"," row_sep:"\n" quote_char:"\"">
irb(main):005:0>
$ cat cats.csv
cat1
cat2
cat3
$
Notice that the file has no quotation marks or square brackets in it.
require 'csv'
data = Owner.find(2).cats
CSV.open("file.csv", "w") do |csv|
data.each { |cat| csv << [cat.name] }
end
I have just wrote a code where I get a csv file passed in argument and treat it line by line ; so far, everything is okay. Now, I would like to secure my code by making sure that what we receive in argument is a .csv file.
I saw in the Ruby doc that it exist a == "--file" option but using it generate an error : the way I understood it, it seems this option only work for the txt files.
Is there a method specific that allowed to check if my file is a csv ? Here some of my code :
if ARGV.empty?
puts "j'ai rien reçu"
# option to check, don't work
elsif ARGV[0].shift == "--file"
# my code so far, whithout checking
else CSV.foreach(ARGV.shift) do |row|
etc, etc...
I think it is unpossible to make a real safe test without additional information.
Just some notes what you can do:
You get a filename in a variable filename.
First, check if it is a file:
File.exist?
Then you could check, if the encoding is correct:
raise "Wrong encoding" unless content.valid_encoding?
Has your csv always the same number of columns? And do you have only one liner?
This can be a possibility to make the next check:
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
This check can be modified for your case, e.g. if you have always an exact number of rows.
In total you can define something like:
require 'csv'
#columns defines the expected numer of columns per line
def csv?(filename, sep: ';', columns: 3)
return false unless File.exist?(filename) #"No file"
content = File.read(filename, :encoding => 'utf-8')
return false unless content.valid_encoding? #"Wrong encoding"
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
CSV.parse(content, :col_sep => sep)
end
if csv = csv?('test.csv')
csv.each do |row|
p row
end
end
You can use ruby-filemagic gem
gem install ruby-filemagic
Usage:
$ irb
irb(main):001:0> require 'filemagic'
=> true
irb(main):002:0> fm = FileMagic.new
=> #<FileMagic:0x7fd4afb0>
irb(main):003:0> fm.file('foo.zip')
=> "Zip archive data, at least v2.0 to extract"
irb(main):004:0>
https://github.com/ricardochimal/ruby-filemagic
Use File.extname() to check the origin file
File.extname("test.rb") #=> ".rb"
I am trying to get an array of hashes from parsing a CSV file using CSV library.
I currently have this method which works:
def rows
rows = []
CSV.foreach(#csv_file.path, headers: true) do |row|
rows << row.to_hash
end
rows
end
but when I change it to this I get the stream closed error.
def rows
CSV.foreach(#csv_file.path, headers: true).map(&:to_hash)
end
thanks
If you look at the source code of ::foreach :
def self.foreach(path, options = Hash.new, &block)
encoding = options.delete(:encoding)
mode = "rb"
mode << ":#{encoding}" if encoding
open(path, mode, options) do |csv|
csv.each(&block)
end
end
It internally, opening the file using CSV::open, with a block. So, once the block is closed, the IO object got closed, internally. Now as you are trying to access the closed IO object, you are getting the error.
From the doc of CSV::open
This method works like Ruby’s open() call, in that it will pass a CSV object to a provided block and close it when the block terminates,...
The IO object returned by ::foreach is actually returned by the CSV::open, within the method def self.foreach ....
Example :
2.1.0 :016 > require 'csv'
=> true
2.1.0 :017 > CSV.open("Gemfile")
=> <#CSV io_type:File io_path:"Gemfile" encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\n" quote_char:"\"">
2.1.0 :018 > CSV.open("Gemfile") { |c| c }
=> <#CSV io_type:File io_path:"Gemfile" encoding:UTF-8 lineno:0 col_sep:"," row_sep:"\n" quote_char:"\"">
2.1.0 :019 > CSV.open("Gemfile") { |c| c }.read
IOError: closed stream
I am using ruby 2.0.0 and rails 4.0.0. I have something similar to this:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
puts f.gets
end
This opens the file on the FTP site, but it only puts the first line of the csv file. I need to read this file row by row, preferably ignoring the header.
How can I read the file row by row, without downloading the file locally?
I solved this by doing this:
data = sftp.download!("/path/to/remote/file.csv").split(/\r\n/)
data.each do |line|
puts line
end
The proper answer for this would actually be to use the file.eof? value.
The code would look like:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
while !f.eof?
puts f.gets
end
end
Documentation can be found here
In my case something like this worked:
data = sftp.download!("/path/to/remote/file.csv").split(/\n/).map{ |e| e.split(/,/).map{ |x| x.gsub(/"/, "")} }
data.each do |line|
puts line
end
Will also split each row of the .csv into different array columns and remove any excess of "". Note this is for mac where line breaks are \n.