I have 2 locale files en.yml and pt.yml. There are some keys that exist only on pt.yml and other keys that exist only on en.yml
Is there a method or routine to list all these keys? (Just comparing the two files)
Example output:
en.activerecord.attributes.person.hand
pt.activerecord.models.bird
Obs: something other than the i18n-tasks missing task.
I find this solution work perfect. It's from a blog post of Kisko Labs. Reference is here: http://blog.kiskolabs.com/post/908453942/comparing-rails-locale-files-for-missing
LOCALE_1 = "~/Code/project/config/locales/fi.yml"
LOCALE_2 = "~/Code/project/config/locales/en.yml"
require 'yaml'
def flatten_keys(hash, prefix="")
keys = []
hash.keys.each do |key|
if hash[key].is_a? Hash
current_prefix = prefix + "#{key}."
keys << flatten_keys(hash[key], current_prefix)
else
keys << "#{prefix}#{key}"
end
end
prefix == "" ? keys.flatten : keys
end
def compare(locale_1, locale_2)
yaml_1 = YAML.load(File.open(File.expand_path(locale_1)))
yaml_2 = YAML.load(File.open(File.expand_path(locale_2)))
keys_1 = flatten_keys(yaml_1[yaml_1.keys.first])
keys_2 = flatten_keys(yaml_2[yaml_2.keys.first])
missing = keys_2 - keys_1
file = locale_1.split('/').last
if missing.any?
puts "Missing from #{file}:"
missing.each { |key| puts " - #{key}" }
else
puts "Nothing missing from #{file}."
end
end
This would do it:
require 'set'
require 'yaml'
files = ['en.yml', 'pt.yml']
p files.map {| file_path| YAML.load(File.read(file_path))}
.map {|object| Set.new(object.keys) }
.reduce(:^)
Docs: YAML and Set
Related
Would it be possible to access the ActiveStorageBlob or ActiveStorageAttachment like it would be a native model ?
E.g.
I want to do ActiveStorageBlob.first to access the first record of this model/table.
or. ActiveStorageAttachment.all.as_json to generate json formated print.
The background idea is to find a way how to dump the content of these ActiveStorage related tables as json formated files. Then change simething on these files, and load it back.
----Extending this text after got correct answer-----
Thank you very much Sarah Marie.
And I hope you know how to load the JSON data back into these tables ?
I have tried this :
dump_file_path = File.join(Rails.root, "backup", active_storage_blobs_file)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
ActiveStorage::Blob.create(j)
end
But thats not working.
ActiveModel::UnknownAttributeError (unknown attribute
'attachable_sgid' for ActiveStorage::Blob.)
ActiveStorage::Blob.first
ActiveStorage::Attachment.all.as_json
---- For second extended question ----
ActiveStorage::Blob.create_before_direct_upload!(
filename: j[:filename],
content_type: j[:content_type],
byte_size: j[:byte_size],
checksum: j[:checksum]
)
# or
ActiveStorage::Blob.create_before_direct_upload!(**j.symbolize_keys)
Reference: https://github.com/rails/rails/blob/5f3ff60084ab5d5921ca3499814e4697f8350ee7/activestorage/app/controllers/active_storage/direct_uploads_controller.rb#L8-L9
https://github.com/rails/rails/blob/098fd7f9b3d5c6f540911bc0c17207d6b48d5bb3/activestorage/app/models/active_storage/blob.rb#L113-L120
Now I have a complete solution, how to dump and load the ActiveStorage tables as JSON files.
...dump it
active_storage_blobs_file = "active_storage_blob.json"
active_storage_attachments_file = "active_storage_attachment.json"
puts("...dump active_storage_blob")
dump_file_path = File.join(Rails.root, "backup",active_storage_blobs_file)
dump_file = File.open(dump_file_path, "w")
dump_file.write(JSON.pretty_generate(ActiveStorage::Blob.all.as_json))
dump_file.close()
puts("...dump active_storage_attachment")
dump_file_path = File.join(Rails.root, "backup",
active_storage_attachments_file)
dump_file = File.open(dump_file_path, "w")
dump_file.write(JSON.pretty_generate(ActiveStorage::Attachment.all.as_json))
dump_file.close()
...load it back
puts("...load active_storage_blob")
dump_file_path = File.join(Rails.root, "backup", active_storage_blobs_file)
abort("File does not exist (" + dump_file_path + ") > abort <") unless File.exist?(dump_file_path)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
j = j.except("attachable_sgid")
result = ActiveStorage::Blob.create(j)
if (not result.errors.empty?)
puts(result.errors.full_messages.to_s)
puts(j.inspect)
exit(1)
end
end
puts("...load active_storage_attachment")
dump_file_path = File.join(Rails.root, "backup", active_storage_attachments_file)
abort("File does not exist (" + dump_file_path + ") > abort <") unless File.exist?(dump_file_path)
load_json = JSON.parse(File.read(dump_file_path))
load_json.each do |j|
result = ActiveStorage::Attachment.create(j)
if (not result.errors.empty?)
puts(result.errors.full_messages.to_s)
puts(j.inspect)
exit(1)
end
end
I'm trying to create a method that deletes files on an S3 instance, but I am getting a AWS::S3::Errors::NoSuchKey: No Such Key error when I try to call .head or .read on an object.
app/models/file_item.rb
def thumbnail
{
exists: thumbnailable?,
small: "http://#{bucket}.s3.amazonaws.com/images/#{id}/small_thumb.png",
large: "http://#{bucket}.s3.amazonaws.com/images/#{id}/large_thumb.png"
}
end
lib/adapters/amazons3/accessor.rb
module Adapters
module AmazonS3
class Accessor
S3_BUCKET = AWS::S3.new.buckets[ENV['AMAZON_BUCKET']]
...
def self.delete_file(thumbnail)
prefix_pattern = %r{http://[MY-S3-HOST]-[a-z]+.s3.amazonaws.com/}
small_path = thumbnail[:small].sub(prefix_pattern, '')
large_path = thumbnail[:large].sub(prefix_pattern, '')
small = S3_BUCKET.objects[small_path]
large = S3_BUCKET.objects[large_path]
binding.pry
S3_BUCKET.objects.delete([small, large])
end
end
end
end
example url1
"http://projectname-staging.s3.amazonaws.com/images/994/small_thumb.png"
example url2
"http://projectname-production.s3.amazonaws.com/images/994/large_thumb.png"
assuming awssdk v1 for ruby.
small = S3_BUCKET.objects[small_path]
does not actually get any objects.
from: https://docs.aws.amazon.com/AWSRubySDK/latest/AWS/S3/Bucket.html
bucket.objects['key'] #=> makes no request, returns an S3Object
bucket.objects.each do |obj|
puts obj.key
end
so you would need to alter your code to something like:
to_delete = []
S3_BUCKET.objects[small_path].each do |obj|
to_delete << obj.key
end
S3_BUCKET.objects[large_path].each do |obj|
to_delete << obj.key
end
S3_BUCKET.objects.delete(to_delete)
just banged out the code, so the idea is there, you might need to correct/polish it a bit
I was able to come of with a kind of different solution thanks to your answer of #Mircea above.
def self.delete_file(thumbnail)
folder = thumbnail[:small].match(/(\d+)(?!.*\d)/)
to_delete = []
S3_BUCKET.objects.with_prefix("images/#{folder}").each do |thumb|
to_delete << thumb.key
end
# binding.pry
S3_BUCKET.objects.delete(to_delete)
end
input = {"color"=>["red"],"size"=>["s","l"]}
json_obj = [{"color":"red","id":"123","size":"s","name":"test"},
{"color":"yellow","id":"124","size":"s","name":"test"},
{"color":"red","id":"125","size":"l","name":"test"}]
Output should be
output["red_s"] = {"color":"red","id":"123","size":"s","name":"test"}
output["red_l"] = {"color":"red","id":"125","size":"l","name":"test"}
output is the combinations of the input and a find on the json_obj.
How to get the output in rails?
I have the below script to get the combinations ie.red_s and red_l,
ary = input.map {|k,v| [k].product v}
output = ary.shift.product(*ary).map {|a| Hash[a]}
And
output[red_s]=json_obj.find{|h| h["color"] == "red" and h["size"] == "S"}
I don't want to have any hardcodings in code like color and size as above.
I think this should get you close to what you want.
Note the "ticks" around your json array object (what you had is not valid ruby)
The other issue is you would have to figure a better way to create the output hash key.
require 'json'
input = {"color"=>["red"],"size"=>["s","l"]}
output = {}
json_obj = '[{"color":"red","id":"123","size":"s","name":"test"},
{"color":"yellow","id":"124","size":"s","name":"test"},
{"color":"red","id":"125","size":"l","name":"test"}]'
found = JSON.parse json_obj
input.each_key do |key|
found = found.select { |item| input[key].include?(item[key]) }
end
puts found
found.each do |item|
output_key = ""
input.each_key do |key|
output_key = "#{item[key]}_" + output_key
end
output["#{output_key}"] = item.to_json
end
puts output
I encounter a strange problem when trying to alter values from a Hash. I have the following setup:
myHash = {
company_name:"MyCompany",
street:"Mainstreet",
postcode:"1234",
city:"MyCity",
free_seats:"3"
}
def cleanup string
string.titleize
end
def format
output = Hash.new
myHash.each do |item|
item[:company_name] = cleanup(item[:company_name])
item[:street] = cleanup(item[:street])
output << item
end
end
When I execute this code I get: "TypeError: no implicit conversion of Symbol into Integer" although the output of item[:company_name] is the expected string. What am I doing wrong?
Your item variable holds Array instance (in [hash_key, hash_value] format), so it doesn't expect Symbol in [] method.
This is how you could do it using Hash#each:
def format(hash)
output = Hash.new
hash.each do |key, value|
output[key] = cleanup(value)
end
output
end
or, without this:
def format(hash)
output = hash.dup
output[:company_name] = cleanup(output[:company_name])
output[:street] = cleanup(output[:street])
output
end
This error shows up when you are treating an array or string as a Hash. In this line myHash.each do |item| you are assigning item to a two-element array [key, value], so item[:symbol] throws an error.
You probably meant this:
require 'active_support/core_ext' # for titleize
myHash = {company_name:"MyCompany", street:"Mainstreet", postcode:"1234", city:"MyCity", free_seats:"3"}
def cleanup string
string.titleize
end
def format(hash)
output = {}
output[:company_name] = cleanup(hash[:company_name])
output[:street] = cleanup(hash[:street])
output
end
format(myHash) # => {:company_name=>"My Company", :street=>"Mainstreet"}
Please read documentation on Hash#each
myHash.each{|item|..} is returning you array object for item iterative variable like the following :--
[:company_name, "MyCompany"]
[:street, "Mainstreet"]
[:postcode, "1234"]
[:city, "MyCity"]
[:free_seats, "3"]
You should do this:--
def format
output = Hash.new
myHash.each do |k, v|
output[k] = cleanup(v)
end
output
end
Ive come across this many times in my work, an easy work around that I found is to ask if the array element is a Hash by class.
if i.class == Hash
notation like i[:label] will work in this block and not throw that error
end
I have a model that has a method that looks through the filesystem starting at a particular location for files that match a particular regex. This is executed in an after_save callback. I'm not sure how to test this using Rspec and FactoryGirl. I'm not sure how to use something like FakeFS with this because the method is in the model, not the test or the controller. I specify the location to start in my FactoryGirl factory, so I could change that to a fake directory created by the test in a set up clause? I could mock the directory? I think there are probably several different ways I could do this, but which makes the most sense?
Thanks!
def ensure_files_up_to_date
files = find_assembly_files
add_files = check_add_assembly_files(files)
errors = add_assembly_files(add_files)
if errors.size > 0 then
return errors
end
update_files = check_update_assembly_files(files)
errors = update_assembly_files(update_files)
if errors.size > 0 then
return errors
else
return []
end
end
def find_assembly_files
start_dir = self.location
files = Hash.new
if ! File.directory? start_dir then
errors.add(:location, "Directory #{start_dir} does not exist on the system.")
abort("Directory #{start_dir} does not exist on the system for #{self.inspect}")
end
Find.find(start_dir) do |path|
filename = File.basename(path).split("/").last
FILE_TYPES.each { |filepart, filehash|
type = filehash["type"]
vendor = filehash["vendor"]
if filename.match(filepart) then
files[type] = Hash.new
files[type]["path"] = path
files[type]["vendor"] = vendor
end
}
end
return files
end
def check_add_assembly_files(files=self.find_assembly_files)
add = Hash.new
files.each do |file_type, file_hash|
# returns an array
file_path = file_hash["path"]
file_vendor = file_hash["vendor"]
filename = File.basename(file_path)
af = AssemblyFile.where(:name => filename)
if af.size == 0 then
add[file_path] = Hash.new
add[file_path]["type"] = file_type
add[file_path]["vendor"] = file_vendor
end
end
if add.size == 0 then
logger.error("check_add_assembly_files did not find any files to add")
return []
end
return add
end
def check_update_assembly_files(files=self.find_assembly_files)
update = Hash.new
files.each do |file_type, file_hash|
file_path = file_hash["path"]
file_vendor = file_hash["vendor"]
# returns an array
filename = File.basename(file_path)
af = AssemblyFile.find_by_name(filename)
if !af.nil? then
if af.location != file_path or af.file_type != file_type then
update[af.id] = Hash.new
update[af.id]['path'] = file_path
update[af.id]['type'] = file_type
update[af.id]['vendor'] = file_vendor
end
end
end
return update
end
def add_assembly_files(files=self.check_add_assembly_files)
if files.size == 0 then
logger.error("add_assembly_files didn't get any results from check_add_assembly_files")
return []
end
asm_file_errors = Array.new
files.each do |file_path, file_hash|
file_type = file_hash["type"]
file_vendor = file_hash["vendor"]
logger.debug "file type is #{file_type} and path is #{file_path}"
logger.debug FileType.find_by_type_name(file_type)
file_type_id = FileType.find_by_type_name(file_type).id
header = file_header(file_path, file_vendor)
if file_vendor == "TBA" then
check = check_tba_header(header, file_type, file_path)
software = header[TBA_SOFTWARE_PROGRAM]
software_version = header[TBA_SOFTWARE_VERSION]
elsif file_vendor == "TBB" then
check = check_tbb_header(header, file_type, file_path)
if file_type == "TBB-ANNOTATION" then
software = header[TBB_SOURCE]
else
software = "Unified"
end
software_version = "UNKNOWN"
end
if check == 0 then
logger.error("skipping file #{file_path} because it contains incorrect values for this filetype")
asm_file_errors.push("#{file_path} cannot be added to assembly because it contains incorrect values for this filetype")
next
end
if file_vendor == "TBA" then
xml = header.to_xml(:root => "assembly-file")
elsif file_vendor == "TBB" then
xml = header.to_xml
else
xml = ''
end
filename = File.basename(file_path)
if filename.match(/~$/) then
logger.error("Skipping a file with a tilda when adding assembly files. filename #{filename}")
next
end
assembly_file = AssemblyFile.new(
:assembly_id => self.id,
:file_type_id => file_type_id,
:name => filename,
:location => file_path,
:file_date => creation_time(file_path),
:software => software,
:software_version => software_version,
:current => 1,
:metadata => xml
)
assembly_file.save! # exclamation point forces it to raise an error if the save fails
end # end files.each
return asm_file_errors
end
Quick answer: you can stub out model methods like any others. Either stub a specific instance of a model, and then stub find or whatever to return that, or stub out any_instance to if you don't want to worry about which model is involved. Something like:
it "does something" do
foo = Foo.create! some_attributes
foo.should_receive(:some_method).and_return(whatever)
Foo.stub(:find).and_return(foo)
end
The real answer is that your code is too complicated to test effectively. Your models should not even know that a filesystem exists. That behavior should be encapsulated in other classes, which you can test independently. Your model's after_save can then just call a single method on that class, and testing whether or not that single method gets called will be a lot easier.
Your methods are also very difficult to test, because they are trying to do too much. All that conditional logic and external dependencies means you'll have to do a whole lot of mocking to get to the various bits you might want to test.
This is a big topic and a good answer is well beyond the scope of this answer. Start with the Wikipedia article on SOLID and read from there for some of the reasoning behind separating concerns into individual classes and using tiny, composed methods. To give you a ballpark idea, a method with more than one branch or more than 10 lines of code is too big; a class that is more than about 100 lines of code is too big.