I am building an app by python3.8.3 and pyqt5. This is a part of my code:
def logupd(self):
users=open('NeveshtarUsers.txt', 'w')
users.write('Username: ')
users.write(self.user2.text())
users.write(' Password: ')
users.write(self.pass2.text())
users.write('.\n')
users.write('Name: ')
users.write(self.name.text())
users.write('.\n')
users.write('....................\n')
self.no.setText('Loggind up has succesed. Please Login')
with open('NeveshtarUsers.txt') as file:
print('salam')
data = file.readlines()
print(data)
When I call this code, the output is:
salam
[]
Why data is empty? What's the problem?
I think the problem is not closing the file that is open in write mode
def logupd(self):
users=open('NeveshtarUsers.txt', 'w')
users.write('Username: ')
users.write(self.user2.text())
users.write(' Password: ')
users.write(self.pass2.text())
users.write('.\n')
users.write('Name: ')
users.write(self.name.text())
users.write('.\n')
users.write('....................\n')
self.no.setText('Loggind up has succesed. Please Login')
# close the file that is opened in write , the text will get updated to the system's memory
# only when the file is closed
users.close()
with open('NeveshtarUsers.txt') as file:
print('salam')
data = file.readlines()
print(data)
Related
I am using ActionMailer in a Ruby on Rails app to read emails (ruby 1.9.3, rails 3.2.13).
I have an email that has a winmail.dat file attached to it (ms-tnef) and I am using the tnef gem to extract its contents.
The problem is that when I read the attachment from the mail, it gets corrupted and tnef can not extract files from it.
$ tnef winmail.dat
ERROR: invalid checksum, input file may be corrupted
Extracting the winmail.dat attachment using any mail app, the extracted winmail.dat works fine with tnef and I got it's content.
Comparing the two files I noticed that:
- original file is bigger (76k against 72k)
- they differ on line breaks: Orginal file has the windows format (0D 0A) and the file saved by rails has the linux format (0A)
I wrote this test:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
That fails with these messages:
WARNING: invalid checksum, input file may be corrupted
Invalid RTF CRC, input file may be corrupted
WARNING: invalid checksum, input file may be corrupted
Assertion failed: ((attr->lvl_type == LVL_MESSAGE) || (attr->lvl_type == LVL_ATTACHMENT)), function attr_read, file attr.c, line 240.
Errno::EPIPE: Broken pipe
anexo = message.attachments[0]
=> #<Mail::Part:2159872060, Multipart: false, Headers: <Content-Type: application/ms-tnef; name="winmail.dat">, <Content-Transfer-Encoding: quoted-printable>, <Content-Disposition: attachment; filename="winmail.dat">>
I tried to save it to disk as bynary, and read it again, but I got the same result
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
How should I read the attachment?
The method anexo.body.decoded calls the decode method of the best suited encoding (Mail::Encodings) for the attachment, in your case quoted_printable.
Some of these encodings (7bit, 8bit and quoted_printable), perform a conversion, changing different types of line breaks to the platform specific line break.
the *quoted_printable" call .to_lf that corrupt the winmail.dat file
# Decode the string from Quoted-Printable. Cope with hard line breaks
# that were incorrectly encoded as hex instead of literal CRLF.
def self.decode(str)
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first.to_lf
end
mail/core_extensions/string.rb:
def to_lf
to_str.gsub(/\n|\r\n|\r/) { "\n" }
end
To solve it you have perform the same encoding without the last .to_lf.
To do that you can create a new encoding that does not corrupt your file and use it to encode you attachment.
create the file:
lib/encodings/tnef_encoding.rb
require 'mail/encodings/7bit'
module Mail
module Encodings
# Encoding to handle Microsoft TNEF format
# It's pretty similar to quoted_printable, except for the 'to_lf' (decode) and 'to_crlf' (encode)
class TnefEncoding < SevenBit
NAME='tnef'
PRIORITY = 2
def self.can_encode?(str)
EightBit.can_encode? str
end
def self.decode(str)
# **difference here** removed '.to_lf'
str.gsub(/(?:=0D=0A|=0D|=0A)\r\n/, "\r\n").unpack("M*").first
end
def self.encode(str)
# **difference here** removed '.to_crlf'
[str.to_lf].pack("M")
end
def self.cost(str)
# These bytes probably do not need encoding
c = str.count("\x9\xA\xD\x20-\x3C\x3E-\x7E")
# Everything else turns into =XX where XX is a
# two digit hex number (taking 3 bytes)
total = (str.bytesize - c)*3 + c
total.to_f/str.bytesize
end
private
Encodings.register(NAME, self)
end
end
end
To use your custom encoding you must, first, register it:
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
And then, set it as your preferred encoding for the attachment:
anexo.body.encoding('tnef')
Your test would, then, become:
it 'should extract winmail.dat from email and extract its contents' do
file_path = "#{::Rails.root}/spec/files/winmail-dat-001.eml"
message = Mail::Message.new(File.read(file_path))
anexo = message.attachments[0]
tmpfile_name = "#{::Rails.root}/tmp/#{anexo.filename}"
Mail::Encodings.register('tnef', Mail::Encodings::TnefEncoding)
anexo.body.encoding('tnef')
File.open(tmpfile_name, 'w+b', 0644) { |f| f.write anexo.body.decoded }
anexo = File.open(tmpfile_name)
files = []
Tnef.unpack(anexo) do |file|
files << File.basename(file)
end
puts files.inspect
files.size.should == 2
end
Hope it helps!
Ruby 2.0.0, Rails 4.0.3, Windows 8.1 Update, Roo 1.13.2
I am trying to open an Excel spreadsheet with embedded spaces using Roo. So far, I am unable to do that. I don't really know if this problem is restricted to Roo. If I rename it to eliminate the spaces, I have no problem with it. I tried encoding it, but then it simply said the file doesn't exist. Can I open the file while it contains spaces?
Code sample:
exceptions = [URI::InvalidURIError, IOError]
puts "f is #{f}"
puts "f exist? #{File.exist?(f)}"
begin
xls = Roo::Spreadsheet.open(f)
rescue *exceptions => e
puts e.message
end
encoded_f = URI.encode(f).to_s
puts "encoded_f is #{encoded_f}"
puts "encoded_f exist? #{File.exist?(encoded_f)}"
begin
xls = Roo::Spreadsheet.open(encoded_f)
rescue *exceptions => e
puts e.message
end
gsub_f = f.gsub(" ", "") # Rename file without spaces
File.rename(f, gsub_f)
puts "gsub_f is #{gsub_f}"
puts "gsub_f exist? #{File.exist?(gsub_f)}"
begin
xls = Roo::Spreadsheet.open(gsub_f)
rescue *exceptions => e
puts e.message
end
Output sample:
f is Whitt Report 2014-07-28-0803.xls
f exist? true
bad URI(is not URI?): Whitt Report 2014-07-28-0803.xls
encoded_f is Whitt%20Report%202014-07-28-0803.xls
encoded_f exist? false
file Whitt%20Report%202014-07-28-0803.xls does not exist
gsub_f is WhittReport2014-07-28-0803.xls
gsub_f exist? true
No message is given in the end because the file opens successfully.
This is caused by the way in which the URI module is called in the Roo::Spreadsheet#open method.
I posted a fix to this problem which has now been merged. If you update your Roo gem you should no longer have this issue.
I am using ruby 2.0.0 and rails 4.0.0. I have something similar to this:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
puts f.gets
end
This opens the file on the FTP site, but it only puts the first line of the csv file. I need to read this file row by row, preferably ignoring the header.
How can I read the file row by row, without downloading the file locally?
I solved this by doing this:
data = sftp.download!("/path/to/remote/file.csv").split(/\r\n/)
data.each do |line|
puts line
end
The proper answer for this would actually be to use the file.eof? value.
The code would look like:
require 'net/sftp'
sftp = Net::SFTP.start('ftp.app.com','username', :password => 'password')
sftp.file.open("/path/to/remote/file.csv", "r") do |f|
while !f.eof?
puts f.gets
end
end
Documentation can be found here
In my case something like this worked:
data = sftp.download!("/path/to/remote/file.csv").split(/\n/).map{ |e| e.split(/,/).map{ |x| x.gsub(/"/, "")} }
data.each do |line|
puts line
end
Will also split each row of the .csv into different array columns and remove any excess of "". Note this is for mac where line breaks are \n.
I am trying to process some very large tab-separated files. The process is:
begin
Dir["#{#data_path}*.tsv"].each do |file|
begin
CSV.foreach(file, :col_sep => "\t") do |row|
# assign columns to model and save
end
#log.info("Loaded all files into MySQL database illu.datafeeds")
rescue Exception => e
#log.warn("Unable to process the data feed: #{file} because #{e.message}")
next
end
end
However, when I execute this I get the following error:
Unable to process the file: /Users/XXXXX_2013-06-12.tsv because Illegal quoting in line 153.
The files are too big for me to go in and fix the error rows. I would like the process to continue the loop and process the file even if there are error rows.
Any suggestions?
Thanks.
just ... rescue nil the row causing the error
you can even log it with logger
before the loop:
error_log ||= Logger.new("#{Rails.root}/log/my.log")
inside the loop instead of just rescue nil use
rescue error_log.info(row.to_s)
in case you get the error before file begins to parse (before .foreach procedure) you can open it as raw file and read it as CSV later - inside the loop (like mentioned here)
..or just rescue full file parsing procedure
CSV.foreach(file, :col_sep => "\t") do |row|
...
end rescue error_log.info(row.to_s)
The following code that is meant to delete lines that match a regular expression fails
def delete_entry(name)
puts "Deleting #{name}.." if $DEBUG
begin
File.open("#{#file_name}.tmp", 'w') do |out_file|
File.open(#file_name, 'r').each do |line|
unless line =~ /^#{name},/
out_file.print line
else
puts "Deleted #{line}!"
end
end
end
File.delete(#file_name)
File.rename("#{#file_name}.tmp", #file_name)
rescue Exception
puts "Exception thrown in PhoneBook::delete_entry(#{name}): #{$!}"
end
end
The temporary file works just fine. The corresponding entries are deleted properly. But when I try to delete the old file, and rename the tmp to the new file, File.delete throws the following: (line 56 is the call to File#delete)
Exception `Errno::EACCES' at PhoneBook.rb:56 - Permission denied - file-io-sampl
es/phonebooks/test.csv
Exception thrown in PhoneBook::delete_entry(Mike): Permission denied - file-io-s
amples/phonebooks/test.csv
Any help would be appreciated. This is on Windows 7 using a NTFS filesystem if that helps matters.
Edit: As per Az's suggestion in the comments I added:
ObjectSpace.each_object(File) { |f| p f if f.path == #file_name && !f.closed? }
Just before the call to File.delete.
The output is below:
C:\Pickaxe>ruby PhoneBook.rb true
Enter a phonebook!
test.csv
Using test.csv..
Open Called!
Name: Richard Sex: Male Age: 22
Name: Midori Sex: Female Age: 22
Name: Mike Sex: Male Age: 18
Name: Richard Sex: Male Age: 44
Deleting Mike..
Deleted Mike,Male, 18
!
#<File:file-io-samples/phonebooks/test.csv>
#<File:file-io-samples/phonebooks/test.csv>
Exception `Errno::EACCES' at PhoneBook.rb:56 - Permission denied - file-io-sampl
es/phonebooks/test.csv
Exception thrown in PhoneBook::delete_entry(Mike): Permission denied - file-io-s
amples/phonebooks/test.csv
C:\Pickaxe>
The two lines prefixed with # is the output of the ObjectSpace call.
I figured this out as I stated my last comment on the original post. The problem was is I wasn't calling open with a block and therefore did not benefit from the automatic f.close that comes with using File.open with a block.
To remedy this I used File.open do |file| file.each, instead of File.open(..).each
def delete_entry(name)
puts "Deleting #{name}.." if $DEBUG
begin
File.open("#{#file_name}.tmp", 'w') do |out_file|
File.open(#file_name, 'r+') do |file|
file.each do |line|
unless line =~ /^#{name},/
out_file.print line
else
puts "Deleted #{line}!"
end
end
end
end
ObjectSpace.each_object(File) { |f| p f if f.path == #file_name && !f.closed? } if $DEBUG
File.delete(#file_name)
File.rename("#{#file_name}.tmp", #file_name)
rescue Exception
puts "Exception thrown in PhoneBook::delete_entry(#{name}): #{$!}"
end
end
As a general tip to anyone who finds this while trying to diagnose the error themselves, try deleting the file manually via the command line first. That may show you that you have permissions issues on the file.