It seems that my model loads the wrong file.
The error states that the program is trying to load the wrong architecture: Loading torchvision\models\mobilenet.py, but I trained MobileNetV2.
How to fix this and what did i do wrong?
I used the following to load the model:
model = torch.load("models/MobileNetV2_CLEF_Small10/MobileNetV2_model_29.pt")
This is the creation of the model:
device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")
# Load pretrained MOBILENET Model
model = models.mobilenet_v2(pretrained=True)
model = model.to(device)
# Freeze model parameters
for param in model.parameters():
param.requires_grad = False
# Change the classifier of the model for Transfer Learning
model.classifier = nn.Sequential(
nn.Dropout(0.2),
nn.Linear(model.last_channel, num_classes),
)
Full error code:
Traceback (most recent call last):
File "C:/Users/faxxe/PycharmProjects/NextGrow/MobileNetV2_metrics.py", line 69, in <module>
model = torch.load("models/MobileNetV3_CLEF_Small10/MobileNetV3_model_29.pt")
File "C:\Users\faxxe\anaconda3\envs\NextGrow\lib\site-packages\torch\serialization.py", line 607, in load
return _load(opened_zipfile, map_location, pickle_module, **pickle_load_args)
File "C:\Users\faxxe\anaconda3\envs\NextGrow\lib\site-packages\torch\serialization.py", line 882, in _load
result = unpickler.load()
File "C:\Users\faxxe\anaconda3\envs\NextGrow\lib\site-packages\torch\serialization.py", line 875, in find_class
return super().find_class(mod_name, name)
AttributeError: Can't get attribute 'ConvBNReLU' on <module 'torchvision.models.mobilenet' from 'C:\\Users\\faxxe\\anaconda3\\envs\\NextGrow\\lib\\site-packages\\torchvision\\models\\mobilenet.py'>
Related
I am using Rails Active Storage and I am trying to attach an MP3 file to an object, but I am getting the following error:
ArgumentError (wrong number of arguments (given 0, expected 1)):
Here is the code I currently have:
def perform(streamplay)
text = File.open(streamplay.text_path).read
text = perform_regex(text)
convert = convert_to_audio(text)
audio = convert.audio_content
filename = streamplay.screenplay.title
# The response's audio_content is binary.
File.open "/output.mp3", "wb" do |file|
# Write the response to the output file.
file.write audio
end
streamplay.file.attach(
io: File.open("#{Rails.root}/output.mp3"),
filename: "#{filename}.mp3",
content_type: "audio/mpeg",
identify: false
)
end
I can see that the audio file is getting written to the root directory, and I can listen to it on my computer, so the file is coming through. It's at this bit of the code that things seem to be breaking down:
streamplay.file.attach
Thanks in advance!
Ok, so it looks like the issue was actually in the File.open section here:
File.open "/output.mp3", "wb" do |file|
# Write the response to the output file.
file.write audio
end
I had to remove the preceding "/" before "output.mp3". So instead I now have this and it works:
File.open "output.mp3", "wb" do |file|
# Write the response to the output file.
file.write audio
end
I am trying to use Java's message digest class to calculate an md5 hash for a large amount of files, inside of my rails application. I have written some code in a ruby script with JRuby, but the call to Files.readAllBytes() gives me "undefined method `getFileSystem' for #". Here is the method I've written in ruby:
def calculate_md5_java(zip)
require 'java'
import java.security.MessageDigest
import java.nio.file.Files
import javax.xml.bind.DatatypeConverter
import java.nio.file.FileSystems
md = MessageDigest.getInstance("MD5")
FileUtils.cp(zip, "GODPLEASELETTHISWORK.zip")
Zip::File.open("GODPLEASELETTHISWORK.zip") do |z|
z.each do |entry|
md.update(Files.readAllBytes(entry.get_input_stream))
end
end
digest = md.digest()
DatatypeConverter.printHexBinary(digest).toLowerCase()
end
I've also tried changing my argument to
md.update(entry.get_input_stream.read.bytes.to_a)
Which gives me:
no method 'update' for arguments (org.jruby.RubyArray) on Java::JavaSecurity::MessageDigest::Delegate available overloads: (byte) (java.nio.ByteBuffer) (byte[])
Oh potatoes! That looks a bit complicated... is this acceptable?
def check_please(file)
checksums = {}
Zlib::GzipReader.wrap(file) do |gz|
Gem::Package::TarReader.new(gz) do |tar|
tar.each do |entry|
checksums[entry.full_name] = Digest::MD5.hexdigest(entry.read) if entry.file?
end
end
end
checksums
end
File.open("foo.tgz", "rb") do |file|
puts check_please(file)
end
For what it's worth, I'm using this version of JRuby and Java.
jruby 9.1.9.0 (2.3.3) 2017-05-15 28aa830 Java HotSpot(TM) 64-Bit Server VM 25.40-b25 on 1.8.0_40-b27 +jit [darwin-x86_64]
Also, credit where it's due: http://weblog.jamisbuck.org/2015/7/23/tar-gz-in-ruby.html
I created a java File object based off of the files being passed into here and then passed File.toPath into readAllBytes. readAllBytes doesn't want a string path, but the Path object.
def calculate_md5_java(xmls)
require 'java'
import java.security.MessageDigest
import java.nio.file.Files
import javax.xml.bind.DatatypeConverter
import java.nio.file.FileSystems
import java.io.ByteArrayOutputStream
import java.io.DataOutputStream
md = MessageDigest.getInstance("MD5")
baos = ByteArrayOutputStream.new
out = DataOutputStream.new(baos)
xmls = Hash[xmls.sort_by { |k,v| k.tr(':', '-').to_s }]
xmls.values.each do |xml|
xml_file = java.io.File.new(xml.path)
md.update(Files.readAllBytes(xml_file.toPath()))
end
digest = md.digest()
md5 = DatatypeConverter.printHexBinary(digest)
md5.downcase!
end
I am using ActionMailer in a Ruby on Rails app to read emails (ruby 1.9.3, rails 3.2.13).
I have an email that has a winmail.dat file attached to it (ms-tnef) and I am using the tnef gem to extract its contents.
The problem is that when I read the attachment from the mail, it gets corrupted and tnef can not extract files from it.
$ tnef winmail.dat
ERROR: invalid checksum, input file may be corrupted
Extracting the winmail.dat attachment using any mail app, the extracted winmail.dat works fine with tnef and I got it's content.
Comparing the two files I noticed that:
- original file is bigger (76k against 72k)
- they differ on line breaks: Orginal file has the windows format (0D 0A) and the file saved by rails has the linux format (0A)
I wrote this test:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
That fails with these messages:
WARNING: invalid checksum, input file may be corrupted
Invalid RTF CRC, input file may be corrupted
WARNING: invalid checksum, input file may be corrupted
Assertion failed: ((attr->lvl_type == LVL_MESSAGE) || (attr->lvl_type == LVL_ATTACHMENT)), function attr_read, file attr.c, line 240.
Errno::EPIPE: Broken pipe
anexo = message.attachments[0]
=> #<Mail::Part:2159872060, Multipart: false, Headers: <Content-Type: application/ms-tnef; name="winmail.dat">, <Content-Transfer-Encoding: quoted-printable>, <Content-Disposition: attachment; filename="winmail.dat">>
I tried to save it to disk as bynary, and read it again, but I got the same result
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
How should I read the attachment?
The method anexo.body.decoded calls the decode method of the best suited encoding (Mail::Encodings) for the attachment, in your case quoted_printable.
Some of these encodings (7bit, 8bit and quoted_printable), perform a conversion, changing different types of line breaks to the platform specific line break.
the *quoted_printable" call .to_lf that corrupt the winmail.dat file
# Decode the string from Quoted-Printable. Cope with hard line breaks
# that were incorrectly encoded as hex instead of literal CRLF.
def self.decode(str)
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first.to_lf
end
mail/core_extensions/string.rb:
def to_lf
to_str.gsub(/\n|\r\n|\r/) { "\n" }
end
To solve it you have perform the same encoding without the last .to_lf.
To do that you can create a new encoding that does not corrupt your file and use it to encode you attachment.
create the file:
lib/encodings/tnef_encoding.rb
require 'mail/encodings/7bit'
module Mail
module Encodings
# Encoding to handle Microsoft TNEF format
# It's pretty similar to quoted_printable, except for the 'to_lf' (decode) and 'to_crlf' (encode)
class TnefEncoding < SevenBit
NAME='tnef'
PRIORITY = 2
def self.can_encode?(str)
EightBit.can_encode? str
end
def self.decode(str)
# **difference here** removed '.to_lf'
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first
end
def self.encode(str)
# **difference here** removed '.to_crlf'
[str.to_lf].pack("M")
end
def self.cost(str)
# These bytes probably do not need encoding
c = str.count("\x9\xA\xD\x20-\x3C\x3E-\x7E")
# Everything else turns into =XX where XX is a
# two digit hex number (taking 3 bytes)
total = (str.bytesize - c)*3 + c
total.to_f/str.bytesize
end
private
Encodings.register(NAME, self)
end
end
end
To use your custom encoding you must, first, register it:
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
And then, set it as your preferred encoding for the attachment:
anexo.body.encoding('tnef')
Your test would, then, become:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
anexo.body.encoding('tnef')
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
Hope it helps!
I am trying to setup grape api. While I'm including defaults module:
module API
module V1
module Defaults
extend ActiveSupport::Concern
included do
version 'v1'
format :json
end
end
end
end
the error undefined method ` ' for API::V1::Projects:Class occurs. Also, when I paste
version 'v1'
format :json
to classes without doing mixin, it works. My operating system is Mac OS
The format :json line is indented by both, normal spaces and en spaces.
Inspecting the line's code points reveals them:
line = " format :json"
line.codepoints
#=> [8194, 32, 8194, 32, 8194, 32, 8194, 32, 102, 111, 114, 109, 97, 116, 32, 58, 106, 115, 111, 110]
# ^ ^ ^ ^
Replacing the en spaces with normal spaces should fix the problem.
On attempting to locate a model via rails console
>> f = DBFile.find_by_id(1)
>> NameError: uninitialized constant DBFile
>> D_B_File.all()
LoadError: Expected /home/thrive/rails_projects/tester/app/models/d_b_file.rb to define D_B_File
>> f = DBFile.find_by_id(1)
#<DBFile id: 1, file_name: "output-May-2010.csv">
Now when I rename the model d_b_file.rb >> db_file.rb
>> f = DBFile.find_by_id(1)
#<DBFile id: 1, file_name: "output-May-2010.csv">
So what about the additional underscore in the original model name ...is preventing it from loading normally?
The issue is that the filename is directly related to the class name, since Rails relies on that to determine where it should look for the class.
So, "DBFile".underscore will give you db_file. So that's where Rails expects that class to be found.