I'm trying to export Windows devices to a CSV from Rapid 7 using the API and Ruby.
When I run the below script it is throwing an error about:
NoMethodError: undefined method 'each' for #<Nexpose::AdhocReportConfig:0x000000
029bf4b8>
from (irb):232:in block in irb_binding
from C:/Ruby24-x64/lib/ruby/2.4.0/csv.rb:1299:in 'open'
from (irb):231
from C:/Ruby24-x64/bin/irb.cmd:19:in '<main>'
The code that I'm running is:
require 'nexpose'
require 'csv'
include Nexpose
query = "
SELECT da.host_name AS Name, dos.description AS OS
FROM dim_asset da
JOIN dim_operating_system dos USING (operating_system_id)
JOIN dim_host_type dht USING (host_type_id)
JOIN dim_site_asset dsa USING (asset_id)
JOIN dim_site ds USING (site_id)
Where (dos.description LIKE '%Windows%' AND da.host_name IS NOT NULL)"
#nsc = Connection.new('192.168.0.1', 'user', 'pswd')
#nsc.login
report = Nexpose::AdhocReportConfig.new(nil, 'sql')
report.add_filter('version', '1.1.0')
report.add_filter('query', query)
#nsc.logout
headers = ["Name","OS"]
CSV.open('C:\file.csv', 'wb', { force_quotes: true }) do |csv|
report.each do |reports|
if csv.tell() == 0 # file is empty, so write header
csv << headers
end
csv << [report.Name, report.OS]
end
end
You can't call .each on an instance of Nexpose::AdhocReportConfig
Try something like this:
report = Nexpose::AdhocReportConfig.new(nil, 'sql')
report.add_filter('version', '1.1.0')
report.add_filter('query', query)
report_output = report.generate(#nsc)
...
CSV.open('C:\file.csv', 'wb', { force_quotes: true }) do |csv|
report_output.each do |reports|
...
See a full example here: https://community.rapid7.com/docs/DOC-2733
I have just wrote a code where I get a csv file passed in argument and treat it line by line ; so far, everything is okay. Now, I would like to secure my code by making sure that what we receive in argument is a .csv file.
I saw in the Ruby doc that it exist a == "--file" option but using it generate an error : the way I understood it, it seems this option only work for the txt files.
Is there a method specific that allowed to check if my file is a csv ? Here some of my code :
if ARGV.empty?
puts "j'ai rien reçu"
# option to check, don't work
elsif ARGV[0].shift == "--file"
# my code so far, whithout checking
else CSV.foreach(ARGV.shift) do |row|
etc, etc...
I think it is unpossible to make a real safe test without additional information.
Just some notes what you can do:
You get a filename in a variable filename.
First, check if it is a file:
File.exist?
Then you could check, if the encoding is correct:
raise "Wrong encoding" unless content.valid_encoding?
Has your csv always the same number of columns? And do you have only one liner?
This can be a possibility to make the next check:
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
This check can be modified for your case, e.g. if you have always an exact number of rows.
In total you can define something like:
require 'csv'
#columns defines the expected numer of columns per line
def csv?(filename, sep: ';', columns: 3)
return false unless File.exist?(filename) #"No file"
content = File.read(filename, :encoding => 'utf-8')
return false unless content.valid_encoding? #"Wrong encoding"
content.each_line{|line|
return false if line.count(sep) < columns - 1
}
CSV.parse(content, :col_sep => sep)
end
if csv = csv?('test.csv')
csv.each do |row|
p row
end
end
You can use ruby-filemagic gem
gem install ruby-filemagic
Usage:
$ irb
irb(main):001:0> require 'filemagic'
=> true
irb(main):002:0> fm = FileMagic.new
=> #<FileMagic:0x7fd4afb0>
irb(main):003:0> fm.file('foo.zip')
=> "Zip archive data, at least v2.0 to extract"
irb(main):004:0>
https://github.com/ricardochimal/ruby-filemagic
Use File.extname() to check the origin file
File.extname("test.rb") #=> ".rb"
I would like my JSON output in Ruby on Rails to be "pretty" or nicely formatted.
Right now, I call to_json and my JSON is all on one line. At times this can be difficult to see if there is a problem in the JSON output stream.
Is there way to configure to make my JSON "pretty" or nicely formatted in Rails?
Use the pretty_generate() function, built into later versions of JSON. For example:
require 'json'
my_object = { :array => [1, 2, 3, { :sample => "hash"} ], :foo => "bar" }
puts JSON.pretty_generate(my_object)
Which gets you:
{
"array": [
1,
2,
3,
{
"sample": "hash"
}
],
"foo": "bar"
}
The <pre> tag in HTML, used with JSON.pretty_generate, will render the JSON pretty in your view. I was so happy when my illustrious boss showed me this:
<% if #data.present? %>
<pre><%= JSON.pretty_generate(#data) %></pre>
<% end %>
Thanks to Rack Middleware and Rails 3 you can output pretty JSON for every request without changing any controller of your app. I have written such middleware snippet and I get nicely printed JSON in browser and curl output.
class PrettyJsonResponse
def initialize(app)
#app = app
end
def call(env)
status, headers, response = #app.call(env)
if headers["Content-Type"] =~ /^application\/json/
obj = JSON.parse(response.body)
pretty_str = JSON.pretty_unparse(obj)
response = [pretty_str]
headers["Content-Length"] = pretty_str.bytesize.to_s
end
[status, headers, response]
end
end
The above code should be placed in app/middleware/pretty_json_response.rb of your Rails project.
And the final step is to register the middleware in config/environments/development.rb:
config.middleware.use PrettyJsonResponse
I don't recommend to use it in production.rb. The JSON reparsing may degrade response time and throughput of your production app. Eventually extra logic such as 'X-Pretty-Json: true' header may be introduced to trigger formatting for manual curl requests on demand.
(Tested with Rails 3.2.8-5.0.0, Ruby 1.9.3-2.2.0, Linux)
If you want to:
Prettify all outgoing JSON responses from your app automatically.
Avoid polluting Object#to_json/#as_json
Avoid parsing/re-rendering JSON using middleware (YUCK!)
Do it the RAILS WAY!
Then ... replace the ActionController::Renderer for JSON! Just add the following code to your ApplicationController:
ActionController::Renderers.add :json do |json, options|
unless json.kind_of?(String)
json = json.as_json(options) if json.respond_to?(:as_json)
json = JSON.pretty_generate(json, options)
end
if options[:callback].present?
self.content_type ||= Mime::JS
"#{options[:callback]}(#{json})"
else
self.content_type ||= Mime::JSON
json
end
end
Check out Awesome Print. Parse the JSON string into a Ruby Hash, then display it with ap like so:
require "awesome_print"
require "json"
json = '{"holy": ["nested", "json"], "batman!": {"a": 1, "b": 2}}'
ap(JSON.parse(json))
With the above, you'll see:
{
"holy" => [
[0] "nested",
[1] "json"
],
"batman!" => {
"a" => 1,
"b" => 2
}
}
Awesome Print will also add some color that Stack Overflow won't show you.
If you find that the pretty_generate option built into Ruby's JSON library is not "pretty" enough, I recommend my own NeatJSON gem for your formatting.
To use it:
gem install neatjson
and then use
JSON.neat_generate
instead of
JSON.pretty_generate
Like Ruby's pp it will keep objects and arrays on one line when they fit, but wrap to multiple as needed. For example:
{
"navigation.createroute.poi":[
{"text":"Lay in a course to the Hilton","params":{"poi":"Hilton"}},
{"text":"Take me to the airport","params":{"poi":"airport"}},
{"text":"Let's go to IHOP","params":{"poi":"IHOP"}},
{"text":"Show me how to get to The Med","params":{"poi":"The Med"}},
{"text":"Create a route to Arby's","params":{"poi":"Arby's"}},
{
"text":"Go to the Hilton by the Airport",
"params":{"poi":"Hilton","location":"Airport"}
},
{
"text":"Take me to the Fry's in Fresno",
"params":{"poi":"Fry's","location":"Fresno"}
}
],
"navigation.eta":[
{"text":"When will we get there?"},
{"text":"When will I arrive?"},
{"text":"What time will I get to the destination?"},
{"text":"What time will I reach the destination?"},
{"text":"What time will it be when I arrive?"}
]
}
It also supports a variety of formatting options to further customize your output. For example, how many spaces before/after colons? Before/after commas? Inside the brackets of arrays and objects? Do you want to sort the keys of your object? Do you want the colons to all be lined up?
Dumping an ActiveRecord object to JSON (in the Rails console):
pp User.first.as_json
# => {
"id" => 1,
"first_name" => "Polar",
"last_name" => "Bear"
}
Using <pre> HTML code and pretty_generate is good trick:
<%
require 'json'
hash = JSON[{hey: "test", num: [{one: 1, two: 2, threes: [{three: 3, tthree: 33}]}]}.to_json]
%>
<pre>
<%= JSON.pretty_generate(hash) %>
</pre>
Here is a middleware solution modified from this excellent answer by #gertas. This solution is not Rails specific--it should work with any Rack application.
The middleware technique used here, using #each, is explained at ASCIIcasts 151: Rack Middleware by Eifion Bedford.
This code goes in app/middleware/pretty_json_response.rb:
class PrettyJsonResponse
def initialize(app)
#app = app
end
def call(env)
#status, #headers, #response = #app.call(env)
[#status, #headers, self]
end
def each(&block)
#response.each do |body|
if #headers["Content-Type"] =~ /^application\/json/
body = pretty_print(body)
end
block.call(body)
end
end
private
def pretty_print(json)
obj = JSON.parse(json)
JSON.pretty_unparse(obj)
end
end
To turn it on, add this to config/environments/test.rb and config/environments/development.rb:
config.middleware.use "PrettyJsonResponse"
As #gertas warns in his version of this solution, avoid using it in production. It's somewhat slow.
Tested with Rails 4.1.6.
#At Controller
def branch
#data = Model.all
render json: JSON.pretty_generate(#data.as_json)
end
If you're looking to quickly implement this in a Rails controller action to send a JSON response:
def index
my_json = '{ "key": "value" }'
render json: JSON.pretty_generate( JSON.parse my_json )
end
Here's my solution which I derived from other posts during my own search.
This allows you to send the pp and jj output to a file as needed.
require "pp"
require "json"
class File
def pp(*objs)
objs.each {|obj|
PP.pp(obj, self)
}
objs.size <= 1 ? objs.first : objs
end
def jj(*objs)
objs.each {|obj|
obj = JSON.parse(obj.to_json)
self.puts JSON.pretty_generate(obj)
}
objs.size <= 1 ? objs.first : objs
end
end
test_object = { :name => { first: "Christopher", last: "Mullins" }, :grades => [ "English" => "B+", "Algebra" => "A+" ] }
test_json_object = JSON.parse(test_object.to_json)
File.open("log/object_dump.txt", "w") do |file|
file.pp(test_object)
end
File.open("log/json_dump.txt", "w") do |file|
file.jj(test_json_object)
end
I have used the gem CodeRay and it works pretty well. The format includes colors and it recognises a lot of different formats.
I have used it on a gem that can be used for debugging rails APIs and it works pretty well.
By the way, the gem is named 'api_explorer' (http://www.github.com/toptierlabs/api_explorer)
if you want to handle active_record object, puts is enough.
for example:
without puts
2.6.0 (main):0 > User.first.to_json
User Load (0.4ms) SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1 [["LIMIT", 1]]
=> "{\"id\":1,\"admin\":true,\"email\":\"admin#gmail.com\",\"password_digest\":\"$2a$10$TQy3P7NT8KrdCzliNUsZzuhmo40LGKoth2hwD3OI.kD0lYiIEwB1y\",\"created_at\":\"2021-07-20T08:34:19.350Z\",\"updated_at\":\"2021-07-20T08:34:19.350Z\",\"name\":\"Arden Stark\"}"
with puts
2.6.0 (main):0 > puts User.first.to_json
User Load (0.3ms) SELECT "users".* FROM "users" ORDER BY "users"."id" ASC LIMIT $1 [["LIMIT", 1]]
{"id":1,"admin":true,"email":"admin#gmail.com","password_digest":"$2a$10$TQy3P7NT8KrdCzliNUsZzuhmo40LGKoth2hwD3OI.kD0lYiIEwB1y","created_at":"2021-07-20T08:34:19.350Z","updated_at":"2021-07-20T08:34:19.350Z","name":"Arden Stark"}
=> nil
if you are handle the json data, JSON.pretty_generate is a good alternative
Example:
obj = {foo: [:bar, :baz], bat: {bam: 0, bad: 1}}
json = JSON.pretty_generate(obj)
puts json
Output:
{
"foo": [
"bar",
"baz"
],
"bat": {
"bam": 0,
"bad": 1
}
}
if it's in the ROR project, I always prefer to use gem pry-rails to format my codes in the rails console rather than awesome_print which is too verbose.
Example of pry-rails:
it also has syntax highlight.
# example of use:
a_hash = {user_info: {type: "query_service", e_mail: "my#email.com", phone: "+79876543322"}, cars_makers: ["bmw", "mitsubishi"], car_models: [bmw: {model: "1er", year_mfc: 2006}, mitsubishi: {model: "pajero", year_mfc: 1997}]}
pretty_html = a_hash.pretty_html
# include this module to your libs:
module MyPrettyPrint
def pretty_html indent = 0
result = ""
if self.class == Hash
self.each do |key, value|
result += "#{key}: #{[Array, Hash].include?(value.class) ? value.pretty_html(indent+1) : value}"
end
elsif self.class == Array
result = "[#{self.join(', ')}]"
end
"#{result}"
end
end
class Hash
include MyPrettyPrint
end
class Array
include MyPrettyPrint
end
Simplest example, I could think of:
my_json = '{ "name":"John", "age":30, "car":null }'
puts JSON.pretty_generate(JSON.parse(my_json))
Rails console example:
core dev 1555:0> my_json = '{ "name":"John", "age":30, "car":null }'
=> "{ \"name\":\"John\", \"age\":30, \"car\":null }"
core dev 1556:0> puts JSON.pretty_generate(JSON.parse(my_json))
{
"name": "John",
"age": 30,
"car": null
}
=> nil
Pretty print variant (Rails):
my_obj = {
'array' => [1, 2, 3, { "sample" => "hash"}, 44455, 677778, nil ],
foo: "bar", rrr: {"pid": 63, "state with nil and \"nil\"": false},
wwww: 'w' * 74
}
require 'pp'
puts my_obj.as_json.pretty_inspect.
gsub('=>', ': ').
gsub(/"(?:[^"\\]|\\.)*"|\bnil\b/) {|m| m == 'nil' ? 'null' : m }.
gsub(/\s+$/, "")
Result:
{"array": [1, 2, 3, {"sample": "hash"}, 44455, 677778, null],
"foo": "bar",
"rrr": {"pid": 63, "state with nil and \"nil\"": false},
"wwww":
"wwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwwww"}
If you are using RABL you can configure it as described here to use JSON.pretty_generate:
class PrettyJson
def self.dump(object)
JSON.pretty_generate(object, {:indent => " "})
end
end
Rabl.configure do |config|
...
config.json_engine = PrettyJson if Rails.env.development?
...
end
A problem with using JSON.pretty_generate is that JSON schema validators will no longer be happy with your datetime strings. You can fix those in your config/initializers/rabl_config.rb with:
ActiveSupport::TimeWithZone.class_eval do
alias_method :orig_to_s, :to_s
def to_s(format = :default)
format == :default ? iso8601 : orig_to_s(format)
end
end
I use the following as I find the headers, status and JSON output useful as
a set. The call routine is broken out on recommendation from a railscasts presentation at: http://railscasts.com/episodes/151-rack-middleware?autoplay=true
class LogJson
def initialize(app)
#app = app
end
def call(env)
dup._call(env)
end
def _call(env)
#status, #headers, #response = #app.call(env)
[#status, #headers, self]
end
def each(&block)
if #headers["Content-Type"] =~ /^application\/json/
obj = JSON.parse(#response.body)
pretty_str = JSON.pretty_unparse(obj)
#headers["Content-Length"] = Rack::Utils.bytesize(pretty_str).to_s
Rails.logger.info ("HTTP Headers: #{ #headers } ")
Rails.logger.info ("HTTP Status: #{ #status } ")
Rails.logger.info ("JSON Response: #{ pretty_str} ")
end
#response.each(&block)
end
end
I had a JSON object in the rails console, and wanted to display it nicely in the console (as opposed to displaying like a massive concatenated string), it was as simple as:
data.as_json