I'm using Paperclip to upload files directly to s3 for my rails web app.
I'm currently trying to exploit the md5 check integrated in amazon s3 to verify that the upload was carried on successfully. Paperclip offers a s3_headers hash that you can populate with whatever fields you need. Content-Type is automatically filled. Content-MD5 needs to be Base64 encoded so I provide it this way:
:s3_headers => {:content_md5 => Base64.strict_encode64(md5sum)},
I use strict_encode64 because encode64 adds an unnecessary trailing \n.
With this setup I always receive an InvalidDigest error from aws-sdk, even though paperclip correctly shows the calculated header. I also tried to use plain, unencoded md5sum, with similar results.
If md5sum is a string of hex digits, like the std output from the Linux application md5sum, try this:
:s3_headers => {:content_md5 => [[md5sum].pack("H*")].pack("m0") }
For example, from the rails console:
> md5sum = "7d592a3129ab6a867cf6e2eb60f9ef83"
> [[md5sum].pack("H*")].pack("m0")
=> "fVkqMSmraoZ89uLrYPnvgw=="
Take the md5 of your source, convert each (character couple) from string to hex (2 bytes become 1 byte); then base64 encode and you will be fine.
Related
I want to generate pdf file from DB record. Encode it to Base64 string and store it to DB. Which works fine. Now I want reverse action, How can I decode Base64 string and generate pdf file again?
here is what I tried so far.
def data_pdf_base64
begin
# Create Prawn Object
my_pdf = Prawn::Document.new
# write text to pdf
my_pdf.text("Hello Gagan, How are you?")
# Save at tmp folder as pdf file
my_pdf.render_file("#{Rails.root}/tmp/pdf/gagan.pdf")
# Read pdf file and encode to Base64
encoded_string = Base64.encode64(File.open("#{Rails.root}/tmp/pdf/gagan.pdf"){|i| i.read})
# Delete generated pdf file from tmp folder
File.delete("#{Rails.root}/tmp/pdf/gagan.pdf") if File.exist?("#{Rails.root}/tmp/pdf/gagan.pdf")
# Now converting Base64 to pdf again
pdf = Prawn::Document.new
# I have used ttf font because it was giving me below error
# Your document includes text that's not compatible with the Windows-1252 character set. If you need full UTF-8 support, use TTF fonts instead of PDF's built-in fonts.
pdf.font Rails.root.join("app/assets/fonts/fontawesome-webfont.ttf")
pdf.text Base64.decode64 encoded_string
pdf.render_file("#{Rails.root}/tmp/pdf/gagan2.pdf")
rescue => e
return render :text => "Error: #{e}"
end
end
Now I am getting below error:
Encoding ASCII-8BIT can not be transparently converted to UTF-8.
Please ensure the encoding of the string you are attempting to use is
set correctly
I have tried How to convert base64 string to PNG using Prawn without saving on server in Rails but it gives me error:
"\xFF" from ASCII-8BIT to UTF-8
Can anyone point me what I am missing?
The answer is to decode the Base64 encoded string and either send it directly or save it directly to disk (naming it as a PDF file, but without using prawn).
The decoded string is a binary representation of the PDF file data, so there's no need to use Prawn or to re-calculate the content of the PDF data.
i.e.
raw_pdf_str = Base64.decode64 encoded_string
render :text, raw_pdf_str # <= this isn't the correct rendering pattern, but it's good enough as an example.
EDIT
To clarify some of the information given in the comments:
It's possible to send the string as an attachment without saving it to disk, either using render text: raw_pdf_str or the #send_data method (these are 4.x API versions, I don't remember the 5.x API style).
It's possible to encode the string (from the Prawn object) without saving the rendered PDF data to a file (save it to a String object instead). i.e.:
encoded_string = Base64.encode64(my_pdf.render)
The String data could be used directly as an email attachment, similarly to the pattern provided here only using the String directly instead of reading any data from a file. i.e.:
# inside a method in the Mailer class
attachments['my_pdf.pdf'] = { :mime_type => 'application/pdf',
:content => raw_pdf_str }
I would like to know can we convert a image into base64 and save it in a database and access it in the views.
I have searched google and stackoverflow, all of them starts from middle like encoding or displaying the image.
I need to know how can we convert a image url/path(lets say i store image inside my app and its url stored in column)
How to encode it as base64 before saving(should we convert to base64 first and save in db?).
How to display it in the views
You can split this task to three or four steps:
getting the image
encoding to base64
storing it in database (optionaly)
display it in views
Getting the image
From Assets pipeline
If you are using Rails asset pipeline for that, you can use Rails.application.assets hash to get to image: Rails.application.assets['image_name.png'].to_s will give you the content of image_name.png image.
from file - local or by url
Here is the question about that on StackOverflow.
encode
Base64 Ruby module docs tells how to use Base64 encoding in Ruby:
Base64.strict_encode64(your_content_here)
NOTE: in this case strict_encode64 is preferrable over just encode64 because it doesn't add any newlines. (credit goes to Sergey Mell for pointing that out)
From docs:
encode64 - ... Line feeds are added to every 60 encoded characters.
strict_encode64 - ... No line feeds are added.
Store it in database (optionaly)
I suggest you to create a separate ActiveRecord model for that, with field of type text to keep base64 representation of image.
Display it in views
You can provide data-url to src attribute of img tag, so, the browser will decode image from base64 and display it just like regular image:
<img src="data:image/png;base64,YOUR_BASE64_HERE"/>
Don't forget to change image format to whatever format you are using in data:image/png section.
UPDATE (2018-08-22): I have tried to use urlsafe_encode64, as suggested by Xornand, and for me it produces the output that is not recognized as image by the browser.
Tried in both Firefox 61.0.2 and Chromium 68.0.3440.106.
For the sake of reference and to enable experimentation, here are results themselves.
Image used as "original" (resized it to be even more small to reduce the size of base64 output):
encode64:
/9j/4AAQSkZJRgABAQEAYABhAAD/4QBARXhpZgAASUkqAAgAAAABAGmHBAAB
AAAAGgAAAAAAAAACAAKgCQABAAAAZAAAAAOgCQABAAAAfwAAAAAAAAD/2wBD
AAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwc
KDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIy
MjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL/
/gA0T3B0aW1pemVkIGJ5IEpQRUdtaW5pIDMuMTQuMTQuNzI2NzA4NjAgMHhm
ZjIzZjM3OQD/wAARCAB/AGQDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAA
AAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIh
MUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3
ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKT
lJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi
4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQF
BgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMi
MoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZH
SElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJma
oqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq
8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDyNj85+tBbFEvE7j0Y/wA6YxrA2HIx
Jqwp4rS0LwvfayyspWKFj99z1+grvrf4TCSEMLqVzjqu3+VTKSRUYtnmQrS0
WTy9Tib0DfyNdNq/wz1WwUvbAyp6MMH/AArl7eKaw1JUuImjdcgq4x2NS2mt
CopqSuZkLf6Vdn/ZevR/ADgaHAv/AE9E/qteaQn9/de6NXoHw/c/YLVSetye
PxFKrsXT/wAzoPFigaovH/LIfzNYHet/xW2dVHtGP61gfxVqcKLMNXojjFUI
zVyI0AaSN8tFRKw2iigDxi9+S8mX0c1b0mxa6nDNGXUnCr/eNPNgbu5aduIz
g4HU1oaZdvp16JBErIo2hCe1M3SO20uKGwgVZdW0+G9Yjy4XfcR7EjgfSuv8
M6pqc91LZzWitLD1aORQCOxGOoI9K8t1W90vU9k0cM1vMgySuOvpVrwvrM1l
qguXnYkptLE9QOgqJQW5tGfQ92Sa4+YXMIEZ/vNkiud8Q+GdM1C4U3EAKn7r
rwy/jV3Tdfi1G1yrBiv3sH2qw7xOgjBJ5PfNc8tNUax8zx7xb8OJtAt5tU09
3uLIofMB+9HnufUe9R+BDstrTP8Az8/4V7KGla3aEhJIWUgqwzkHqD7Vxh0z
StHuGitrRYvn3hdxIBP16UOfMrMpQszO8TtnVT7ItYWfmrvP7PttQgDSJGc8
cZyKzbvwaWQvZTZcfwMev41pGtF7nJLDSitNTnY2xVqFqptFLbzNDMjJIhwV
YYIqaNua2Odmkr8UVCrfLRQI8fkuZAkTq7KSuCQfSrumTmdzE7Ev1BJ5PtVc
2h8pR6E01LaRHDoSGByCO1O6Ntbm6I8KSfujnJNLp5Ny2YSMZPzH/CobK9md
TBOvDcbgOPxrS0q3FrMzMg2hSMdATSRdzodD1M6ULhgcuy8Z9feuz8K6m7Rs
8nzDOcV5vZ2sjYeT+Lk5712emSER4XACjtUTjc0gzupbw+SZVXI6kKOgrh9V
kj1LV5o7dlMipn3Fbdtq4is5uN+xGZlxyR3rza21Rv7QkugzbgeOe1Z8hpzn
daTPIlqI5wWKjhgBW7pV1NNOqgDHGD2+hFcPp+tQFCJxt3Hsa6fTr2OJCImK
c8NwSKiUS1I1fFehW+sWMl1arjULZMsg6yIO2PUdq82j4Nes6JdFplaQDzF4
LDoR/hXA+KdM/szxDcIi4hlPmx49D2/A5Fa05aWZx14Wd0ZyniimA8UVqYHI
f2YxUAKT9BT10iU/8smrtFt+igKDj0xUgszj+EAjpnvWWp1XRx8eiTHny8fU
1ZXSJY0JbBXqQCa61bMocMB+dP8AsIOFwSPXrRewHPQSIqtnAA/StbSbyJWd
JABjpmsnVITp95sK5VxxVCZ7hIfNAIReorQSN241rZp14DgE5xjqPpXG2MFz
OGkWKVgxyQqkk0GZ76QDkAnkDvWnYtJB8ylyxGAFOD/Opehe4W1o8TgzFkXI
3BgRj2Nd3oFpaRyrKLndkgDL8EHsa4GUzzXn7wSIrDOWFamlXX2Z1UuXj6Z6
YOf8DWctTRHpVsklndgRO5CneoI6gfeXjrxn8q534mzXE8lk9kkxePdv8sEg
A4xnH0NaFnrCSTRqXDsjI69iCDj/AB/OrmmXMRu78yuMRuEXP4nH6ipS1CTS
VzyIavfLwWGR6rRXsk2q6akhDFWPrRV6kc0exzYt125bJ5x171ZjRMBgBn37
f54pflI+RgWxywPQn61Irp8rE9T9AB/XtTuZFTUL2PT41eU5JbCr/WqTa7HK
h+zwOHI6ucD9OtXdW0watbgCURyq2UbHHTvzWNB4ZvlYia6iSNc5cAnFVFxt
qGpl3yfanNxPK3m5zurIvbtlQxlgQRxiu1j0S0Q4Y/aD3yeled6skkGry26c
DzMADsKaak9B7Ghp8IRN38Zq4kRwcdR2FV7dvL24BNaMO1U8xjggfnQ0NMdF
H5u1pZgoTkD+lSGzDMyxyDAP3QOTVAk4AUkY5GBUwvXRspuLduO/+FQ0aJmi
pW3uLeRWAYdQO/HWqUurEu7hyGdiSB64qPVLosgdk2yDAB/z7YrC8w4OR9DT
SIm+hpSamxcnOfq1FZhznjB+uKKqxlc7mbW4wxUEB3PY9PWs648RyqC0UmRj
GCDz9M1z7Ssw6E+h/wDr0EyDCuyKTwBn/wCtT5A5i/J4mvy52yMoI5Kt/wDr
qnJ4jvSmDO6jOcfhVSQBztMnA7VnzWzgMVOfenyhcsy6nqFyCyzOV6fexVBp
5klRnVt3c9ah8+WDCyLj+tWYbxfNUkgnPemlYNzctb1EjVpGwBST6ukjbUbC
9u1Vbm5glRfkAJ61HDFaMwWWIbCeqnBFLQpXNGOZnXhjnHQ1t2WlyahbkrKq
Mv8AfbFQWekaWsSMNQuMH+BcEj8xXQ28XhuBQLiW6fI5Es23H4Lis5M1Rzl/
avCfIlGHBznOQfpWFMJY3MYHOevtXWavf6NLtis4eQNibXJOe3Wsz7EJYVJ+
WTJ4Y8/lTiRM5vzH7LkfQ0VsGylPKqCP9oYNFWZ2LqaRMedwQcA5GBnPT+dT
x+G55QjuoUYJyT64/wDr126m1FuXWDdEyB2O3qehz9MdqmSKBAxODGg4DKRk
k5+o/Gq5hWOCl8MOoARCcn7w7+1V5fDtwgYlDkLnaRyRz/8AW/OvRzCm5I5F
xtIXDAnaep5/EYqZbeP7SGLFcIV5XIwecE9e4HXHFJTHynlf/CO3kilhbrIh
74yaiTwZNMob7MQu4/cznOcYwP5V69BaLIXO5d5GMrk4JPcdccgjJ4H51KsE
ciufOWNVj+Vwo477h+J7envRzhynkcHw91K4d/LMsSocev6fiK04fhlqu8Aa
hApyAQ69M9Mnp+HuK9KmDKy+RC+TglyflHJwQfX6/wD16mgikmMZSaJdoJYk
gsWHGMgf/XqHJlKKPOovhnqzXBR9aiUA4yik8YznH4VswfCiwkPmXWq3kxUE
lUCqT6DGPr+Vd5GrWrKIwXUqxycFwccf1/Sk3MUiXa6MxCsfKJUYIHfvz19v
yV2By8XgvTLBFitLWJSyk72OW47kfhWTc6TdW7yqU/dISCSpPy8dwTnHNegv
GoAizH5oAJBXIkx6jt25z/KmSxGUqhQLGwJYseoBHyj06nii4Hmi2CwqEMg+
uDz+VFehrpNm67sbc9uf6Yopgc2LlYrUSRmQxbifkB6AjBGMce3epLaJkYzf
vGMq/IrEkD2OO+fTPaq9vP5ixIYkMkyEyehVeP6irULLGsZLO4UGRUXAbAA6
9jyfXvVEk6P587xzLtTftUq3PbB9c85okCLbkvviQkBGwxb1HqfQ/nxUT2zz
xqY3O+QCRMSbQo/75PNKpULDtDP5rELliAcj3z2J/OkMekUtvK0cYRkEahGV
dmck9cn0AOetSLd7IxNK25WRdqqBg5ByTnjoCTUkJkkgl+0IglRsAZ47ZGR6
f0qAXm+KGJCFIUAEOxYDjuVPP40ASIjGTz4RDJLLj5mUAEj7v69xU1tciV18
9CSSzHBGBjscY9/z7GkDTm5t5ZZFbBMYC9/XnjuKhUyyWsaxIsJRmXaVVgMk
jHoec+nA96Vh3L3kjc0gmQdAXTGRjv3HrxxxmnXELXksZlOYo1Mu5xtKk579
CMDseuKy7Uu1vb/ZFZImZm25AJHX6etWiyNdC3mQqzyDYoY4Kjk9OnBzz60h
lkyxzweXMpmGMBk3bDkAgkg9MVdF5Daw5TAUL/q26qOg69ecDnFUCUhLkRvJ
wzZUj5SBgcHHBANKpeUCcp5kY3AggHIHPTgYoAtOZWCGWxt7klQVkZl5Htx0
opvn2CRxLDL5ShB8ihgB+GKKBH//2Q==
strict_encode64:
/9j/4AAQSkZJRgABAQEAYABhAAD/4QBARXhpZgAASUkqAAgAAAABAGmHBAABAAAAGgAAAAAAAAACAAKgCQABAAAAZAAAAAOgCQABAAAAfwAAAAAAAAD/2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL/2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL//gA0T3B0aW1pemVkIGJ5IEpQRUdtaW5pIDMuMTQuMTQuNzI2NzA4NjAgMHhmZjIzZjM3OQD/wAARCAB/AGQDASIAAhEBAxEB/8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL/8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4+Tl5ufo6erx8vP09fb3+Pn6/8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL/8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3+Pn6/9oADAMBAAIRAxEAPwDyNj85+tBbFEvE7j0Y/wA6YxrA2HIxJqwp4rS0LwvfayyspWKFj99z1+grvrf4TCSEMLqVzjqu3+VTKSRUYtnmQrS0WTy9Tib0DfyNdNq/wz1WwUvbAyp6MMH/AArl7eKaw1JUuImjdcgq4x2NS2mtCopqSuZkLf6Vdn/ZevR/ADgaHAv/AE9E/qteaQn9/de6NXoHw/c/YLVSetyePxFKrsXT/wAzoPFigaovH/LIfzNYHet/xW2dVHtGP61gfxVqcKLMNXojjFUIzVyI0AaSN8tFRKw2iigDxi9+S8mX0c1b0mxa6nDNGXUnCr/eNPNgbu5aduIzg4HU1oaZdvp16JBErIo2hCe1M3SO20uKGwgVZdW0+G9Yjy4XfcR7EjgfSuv8M6pqc91LZzWitLD1aORQCOxGOoI9K8t1W90vU9k0cM1vMgySuOvpVrwvrM1lqguXnYkptLE9QOgqJQW5tGfQ92Sa4+YXMIEZ/vNkiud8Q+GdM1C4U3EAKn7rrwy/jV3Tdfi1G1yrBiv3sH2qw7xOgjBJ5PfNc8tNUax8zx7xb8OJtAt5tU093uLIofMB+9HnufUe9R+BDstrTP8Az8/4V7KGla3aEhJIWUgqwzkHqD7Vxh0zStHuGitrRYvn3hdxIBP16UOfMrMpQszO8TtnVT7ItYWfmrvP7PttQgDSJGc8cZyKzbvwaWQvZTZcfwMev41pGtF7nJLDSitNTnY2xVqFqptFLbzNDMjJIhwVYYIqaNua2Odmkr8UVCrfLRQI8fkuZAkTq7KSuCQfSrumTmdzE7Ev1BJ5PtVc2h8pR6E01LaRHDoSGByCO1O6Ntbm6I8KSfujnJNLp5Ny2YSMZPzH/CobK9mdTBOvDcbgOPxrS0q3FrMzMg2hSMdATSRdzodD1M6ULhgcuy8Z9feuz8K6m7Rs8nzDOcV5vZ2sjYeT+Lk5712emSER4XACjtUTjc0gzupbw+SZVXI6kKOgrh9Vkj1LV5o7dlMipn3Fbdtq4is5uN+xGZlxyR3rza21Rv7QkugzbgeOe1Z8hpzndaTPIlqI5wWKjhgBW7pV1NNOqgDHGD2+hFcPp+tQFCJxt3Hsa6fTr2OJCImKc8NwSKiUS1I1fFehW+sWMl1arjULZMsg6yIO2PUdq82j4Nes6JdFplaQDzF4LDoR/hXA+KdM/szxDcIi4hlPmx49D2/A5Fa05aWZx14Wd0ZyniimA8UVqYHIf2YxUAKT9BT10iU/8smrtFt+igKDj0xUgszj+EAjpnvWWp1XRx8eiTHny8fU1ZXSJY0JbBXqQCa61bMocMB+dP8AsIOFwSPXrRewHPQSIqtnAA/StbSbyJWdJABjpmsnVITp95sK5VxxVCZ7hIfNAIReorQSN241rZp14DgE5xjqPpXG2MFzOGkWKVgxyQqkk0GZ76QDkAnkDvWnYtJB8ylyxGAFOD/Opehe4W1o8TgzFkXI3BgRj2Nd3oFpaRyrKLndkgDL8EHsa4GUzzXn7wSIrDOWFamlXX2Z1UuXj6Z6YOf8DWctTRHpVsklndgRO5CneoI6gfeXjrxn8q534mzXE8lk9kkxePdv8sEgA4xnH0NaFnrCSTRqXDsjI69iCDj/AB/OrmmXMRu78yuMRuEXP4nH6ipS1CTSVzyIavfLwWGR6rRXsk2q6akhDFWPrRV6kc0exzYt125bJ5x171ZjRMBgBn37f54pflI+RgWxywPQn61Irp8rE9T9AB/XtTuZFTUL2PT41eU5JbCr/WqTa7HKh+zwOHI6ucD9OtXdW0watbgCURyq2UbHHTvzWNB4ZvlYia6iSNc5cAnFVFxtqGpl3yfanNxPK3m5zurIvbtlQxlgQRxiu1j0S0Q4Y/aD3yeled6skkGry26cDzMADsKaak9B7Ghp8IRN38Zq4kRwcdR2FV7dvL24BNaMO1U8xjggfnQ0NMdFH5u1pZgoTkD+lSGzDMyxyDAP3QOTVAk4AUkY5GBUwvXRspuLduO/+FQ0aJmipW3uLeRWAYdQO/HWqUurEu7hyGdiSB64qPVLosgdk2yDAB/z7YrC8w4OR9DTSIm+hpSamxcnOfq1FZhznjB+uKKqxlc7mbW4wxUEB3PY9PWs648RyqC0UmRjGCDz9M1z7Ssw6E+h/wDr0EyDCuyKTwBn/wCtT5A5i/J4mvy52yMoI5Kt/wDrqnJ4jvSmDO6jOcfhVSQBztMnA7VnzWzgMVOfenyhcsy6nqFyCyzOV6fexVBp5klRnVt3c9ah8+WDCyLj+tWYbxfNUkgnPemlYNzctb1EjVpGwBST6ukjbUbC9u1Vbm5glRfkAJ61HDFaMwWWIbCeqnBFLQpXNGOZnXhjnHQ1t2WlyahbkrKqMv8AfbFQWekaWsSMNQuMH+BcEj8xXQ28XhuBQLiW6fI5Es23H4Lis5M1Rzl/avCfIlGHBznOQfpWFMJY3MYHOevtXWavf6NLtis4eQNibXJOe3Wsz7EJYVJ+WTJ4Y8/lTiRM5vzH7LkfQ0VsGylPKqCP9oYNFWZ2LqaRMedwQcA5GBnPT+dTx+G55QjuoUYJyT64/wDr126m1FuXWDdEyB2O3qehz9MdqmSKBAxODGg4DKRkk5+o/Gq5hWOCl8MOoARCcn7w7+1V5fDtwgYlDkLnaRyRz/8AW/OvRzCm5I5FxtIXDAnaep5/EYqZbeP7SGLFcIV5XIwecE9e4HXHFJTHynlf/CO3kilhbrIh74yaiTwZNMob7MQu4/cznOcYwP5V69BaLIXO5d5GMrk4JPcdccgjJ4H51KsEciufOWNVj+Vwo477h+J7envRzhynkcHw91K4d/LMsSocev6fiK04fhlqu8AahApyAQ69M9Mnp+HuK9KmDKy+RC+TglyflHJwQfX6/wD16mgikmMZSaJdoJYkgsWHGMgf/XqHJlKKPOovhnqzXBR9aiUA4yik8YznH4VswfCiwkPmXWq3kxUElUCqT6DGPr+Vd5GrWrKIwXUqxycFwccf1/Sk3MUiXa6MxCsfKJUYIHfvz19vyV2By8XgvTLBFitLWJSyk72OW47kfhWTc6TdW7yqU/dISCSpPy8dwTnHNegvGoAizH5oAJBXIkx6jt25z/KmSxGUqhQLGwJYseoBHyj06nii4Hmi2CwqEMg+uDz+VFehrpNm67sbc9uf6Yopgc2LlYrUSRmQxbifkB6AjBGMce3epLaJkYzfvGMq/IrEkD2OO+fTPaq9vP5ixIYkMkyEyehVeP6irULLGsZLO4UGRUXAbAA69jyfXvVEk6P587xzLtTftUq3PbB9c85okCLbkvviQkBGwxb1HqfQ/nxUT2zzxqY3O+QCRMSbQo/75PNKpULDtDP5rELliAcj3z2J/OkMekUtvK0cYRkEahGVdmck9cn0AOetSLd7IxNK25WRdqqBg5ByTnjoCTUkJkkgl+0IglRsAZ47ZGR6f0qAXm+KGJCFIUAEOxYDjuVPP40ASIjGTz4RDJLLj5mUAEj7v69xU1tciV189CSSzHBGBjscY9/z7GkDTm5t5ZZFbBMYC9/XnjuKhUyyWsaxIsJRmXaVVgMkjHoec+nA96Vh3L3kjc0gmQdAXTGRjv3HrxxxmnXELXksZlOYo1Mu5xtKk579CMDseuKy7Uu1vb/ZFZImZm25AJHX6etWiyNdC3mQqzyDYoY4Kjk9OnBzz60hlkyxzweXMpmGMBk3bDkAgkg9MVdF5Daw5TAUL/q26qOg69ecDnFUCUhLkRvJwzZUj5SBgcHHBANKpeUCcp5kY3AggHIHPTgYoAtOZWCGWxt7klQVkZl5Htx0opvn2CRxLDL5ShB8ihgB+GKKBH//2Q==
urlsafe_encode64:
_9j_4AAQSkZJRgABAQEAYABhAAD_4QBARXhpZgAASUkqAAgAAAABAGmHBAABAAAAGgAAAAAAAAACAAKgCQABAAAAZAAAAAOgCQABAAAAfwAAAAAAAAD_2wBDAAgGBgcGBQgHBwcJCQgKDBQNDAsLDBkSEw8UHRofHh0aHBwgJC4nICIsIxwcKDcpLDAxNDQ0Hyc5PTgyPC4zNDL_2wBDAQkJCQwLDBgNDRgyIRwhMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjIyMjL__gA0T3B0aW1pemVkIGJ5IEpQRUdtaW5pIDMuMTQuMTQuNzI2NzA4NjAgMHhmZjIzZjM3OQD_wAARCAB_AGQDASIAAhEBAxEB_8QAHwAAAQUBAQEBAQEAAAAAAAAAAAECAwQFBgcICQoL_8QAtRAAAgEDAwIEAwUFBAQAAAF9AQIDAAQRBRIhMUEGE1FhByJxFDKBkaEII0KxwRVS0fAkM2JyggkKFhcYGRolJicoKSo0NTY3ODk6Q0RFRkdISUpTVFVWV1hZWmNkZWZnaGlqc3R1dnd4eXqDhIWGh4iJipKTlJWWl5iZmqKjpKWmp6ipqrKztLW2t7i5usLDxMXGx8jJytLT1NXW19jZ2uHi4-Tl5ufo6erx8vP09fb3-Pn6_8QAHwEAAwEBAQEBAQEBAQAAAAAAAAECAwQFBgcICQoL_8QAtREAAgECBAQDBAcFBAQAAQJ3AAECAxEEBSExBhJBUQdhcRMiMoEIFEKRobHBCSMzUvAVYnLRChYkNOEl8RcYGRomJygpKjU2Nzg5OkNERUZHSElKU1RVVldYWVpjZGVmZ2hpanN0dXZ3eHl6goOEhYaHiImKkpOUlZaXmJmaoqOkpaanqKmqsrO0tba3uLm6wsPExcbHyMnK0tPU1dbX2Nna4uPk5ebn6Onq8vP09fb3-Pn6_9oADAMBAAIRAxEAPwDyNj85-tBbFEvE7j0Y_wA6YxrA2HIxJqwp4rS0LwvfayyspWKFj99z1-grvrf4TCSEMLqVzjqu3-VTKSRUYtnmQrS0WTy9Tib0DfyNdNq_wz1WwUvbAyp6MMH_AArl7eKaw1JUuImjdcgq4x2NS2mtCopqSuZkLf6Vdn_ZevR_ADgaHAv_AE9E_qteaQn9_de6NXoHw_c_YLVSetyePxFKrsXT_wAzoPFigaovH_LIfzNYHet_xW2dVHtGP61gfxVqcKLMNXojjFUIzVyI0AaSN8tFRKw2iigDxi9-S8mX0c1b0mxa6nDNGXUnCr_eNPNgbu5aduIzg4HU1oaZdvp16JBErIo2hCe1M3SO20uKGwgVZdW0-G9Yjy4XfcR7EjgfSuv8M6pqc91LZzWitLD1aORQCOxGOoI9K8t1W90vU9k0cM1vMgySuOvpVrwvrM1lqguXnYkptLE9QOgqJQW5tGfQ92Sa4-YXMIEZ_vNkiud8Q-GdM1C4U3EAKn7rrwy_jV3Tdfi1G1yrBiv3sH2qw7xOgjBJ5PfNc8tNUax8zx7xb8OJtAt5tU093uLIofMB-9HnufUe9R-BDstrTP8Az8_4V7KGla3aEhJIWUgqwzkHqD7Vxh0zStHuGitrRYvn3hdxIBP16UOfMrMpQszO8TtnVT7ItYWfmrvP7PttQgDSJGc8cZyKzbvwaWQvZTZcfwMev41pGtF7nJLDSitNTnY2xVqFqptFLbzNDMjJIhwVYYIqaNua2Odmkr8UVCrfLRQI8fkuZAkTq7KSuCQfSrumTmdzE7Ev1BJ5PtVc2h8pR6E01LaRHDoSGByCO1O6Ntbm6I8KSfujnJNLp5Ny2YSMZPzH_CobK9mdTBOvDcbgOPxrS0q3FrMzMg2hSMdATSRdzodD1M6ULhgcuy8Z9feuz8K6m7Rs8nzDOcV5vZ2sjYeT-Lk5712emSER4XACjtUTjc0gzupbw-SZVXI6kKOgrh9Vkj1LV5o7dlMipn3Fbdtq4is5uN-xGZlxyR3rza21Rv7QkugzbgeOe1Z8hpzndaTPIlqI5wWKjhgBW7pV1NNOqgDHGD2-hFcPp-tQFCJxt3Hsa6fTr2OJCImKc8NwSKiUS1I1fFehW-sWMl1arjULZMsg6yIO2PUdq82j4Nes6JdFplaQDzF4LDoR_hXA-KdM_szxDcIi4hlPmx49D2_A5Fa05aWZx14Wd0ZyniimA8UVqYHIf2YxUAKT9BT10iU_8smrtFt-igKDj0xUgszj-EAjpnvWWp1XRx8eiTHny8fU1ZXSJY0JbBXqQCa61bMocMB-dP8AsIOFwSPXrRewHPQSIqtnAA_StbSbyJWdJABjpmsnVITp95sK5VxxVCZ7hIfNAIReorQSN241rZp14DgE5xjqPpXG2MFzOGkWKVgxyQqkk0GZ76QDkAnkDvWnYtJB8ylyxGAFOD_Opehe4W1o8TgzFkXI3BgRj2Nd3oFpaRyrKLndkgDL8EHsa4GUzzXn7wSIrDOWFamlXX2Z1UuXj6Z6YOf8DWctTRHpVsklndgRO5CneoI6gfeXjrxn8q534mzXE8lk9kkxePdv8sEgA4xnH0NaFnrCSTRqXDsjI69iCDj_AB_OrmmXMRu78yuMRuEXP4nH6ipS1CTSVzyIavfLwWGR6rRXsk2q6akhDFWPrRV6kc0exzYt125bJ5x171ZjRMBgBn37f54pflI-RgWxywPQn61Irp8rE9T9AB_XtTuZFTUL2PT41eU5JbCr_WqTa7HKh-zwOHI6ucD9OtXdW0watbgCURyq2UbHHTvzWNB4ZvlYia6iSNc5cAnFVFxtqGpl3yfanNxPK3m5zurIvbtlQxlgQRxiu1j0S0Q4Y_aD3yeled6skkGry26cDzMADsKaak9B7Ghp8IRN38Zq4kRwcdR2FV7dvL24BNaMO1U8xjggfnQ0NMdFH5u1pZgoTkD-lSGzDMyxyDAP3QOTVAk4AUkY5GBUwvXRspuLduO_-FQ0aJmipW3uLeRWAYdQO_HWqUurEu7hyGdiSB64qPVLosgdk2yDAB_z7YrC8w4OR9DTSIm-hpSamxcnOfq1FZhznjB-uKKqxlc7mbW4wxUEB3PY9PWs648RyqC0UmRjGCDz9M1z7Ssw6E-h_wDr0EyDCuyKTwBn_wCtT5A5i_J4mvy52yMoI5Kt_wDrqnJ4jvSmDO6jOcfhVSQBztMnA7VnzWzgMVOfenyhcsy6nqFyCyzOV6fexVBp5klRnVt3c9ah8-WDCyLj-tWYbxfNUkgnPemlYNzctb1EjVpGwBST6ukjbUbC9u1Vbm5glRfkAJ61HDFaMwWWIbCeqnBFLQpXNGOZnXhjnHQ1t2WlyahbkrKqMv8AfbFQWekaWsSMNQuMH-BcEj8xXQ28XhuBQLiW6fI5Es23H4Lis5M1Rzl_avCfIlGHBznOQfpWFMJY3MYHOevtXWavf6NLtis4eQNibXJOe3Wsz7EJYVJ-WTJ4Y8_lTiRM5vzH7LkfQ0VsGylPKqCP9oYNFWZ2LqaRMedwQcA5GBnPT-dTx-G55QjuoUYJyT64_wDr126m1FuXWDdEyB2O3qehz9MdqmSKBAxODGg4DKRkk5-o_Gq5hWOCl8MOoARCcn7w7-1V5fDtwgYlDkLnaRyRz_8AW_OvRzCm5I5FxtIXDAnaep5_EYqZbeP7SGLFcIV5XIwecE9e4HXHFJTHynlf_CO3kilhbrIh74yaiTwZNMob7MQu4_cznOcYwP5V69BaLIXO5d5GMrk4JPcdccgjJ4H51KsEciufOWNVj-Vwo477h-J7envRzhynkcHw91K4d_LMsSocev6fiK04fhlqu8AahApyAQ69M9Mnp-HuK9KmDKy-RC-TglyflHJwQfX6_wD16mgikmMZSaJdoJYkgsWHGMgf_XqHJlKKPOovhnqzXBR9aiUA4yik8YznH4VswfCiwkPmXWq3kxUElUCqT6DGPr-Vd5GrWrKIwXUqxycFwccf1_Sk3MUiXa6MxCsfKJUYIHfvz19vyV2By8XgvTLBFitLWJSyk72OW47kfhWTc6TdW7yqU_dISCSpPy8dwTnHNegvGoAizH5oAJBXIkx6jt25z_KmSxGUqhQLGwJYseoBHyj06nii4Hmi2CwqEMg-uDz-VFehrpNm67sbc9uf6Yopgc2LlYrUSRmQxbifkB6AjBGMce3epLaJkYzfvGMq_IrEkD2OO-fTPaq9vP5ixIYkMkyEyehVeP6irULLGsZLO4UGRUXAbAA69jyfXvVEk6P587xzLtTftUq3PbB9c85okCLbkvviQkBGwxb1HqfQ_nxUT2zzxqY3O-QCRMSbQo_75PNKpULDtDP5rELliAcj3z2J_OkMekUtvK0cYRkEahGVdmck9cn0AOetSLd7IxNK25WRdqqBg5ByTnjoCTUkJkkgl-0IglRsAZ47ZGR6f0qAXm-KGJCFIUAEOxYDjuVPP40ASIjGTz4RDJLLj5mUAEj7v69xU1tciV189CSSzHBGBjscY9_z7GkDTm5t5ZZFbBMYC9_XnjuKhUyyWsaxIsJRmXaVVgMkjHoec-nA96Vh3L3kjc0gmQdAXTGRjv3HrxxxmnXELXksZlOYo1Mu5xtKk579CMDseuKy7Uu1vb_ZFZImZm25AJHX6etWiyNdC3mQqzyDYoY4Kjk9OnBzz60hlkyxzweXMpmGMBk3bDkAgkg9MVdF5Daw5TAUL_q26qOg69ecDnFUCUhLkRvJwzZUj5SBgcHHBANKpeUCcp5kY3AggHIHPTgYoAtOZWCGWxt7klQVkZl5Htx0opvn2CRxLDL5ShB8ihgB-GKKBH__2Q==
I have some issue to allow users download file, which stored in PostgreSQL bytea escaped field (http://www.postgresql.org/docs/current/interactive/datatype-binary.html).
1.9.3p385 :023 > data = PG::Connection.unescape_bytea(m[:data])
=> "JVBERi0xLjMKJcTl8uXrp/Og0MTGCjQgMCBvYmoKPDwgL0xlbmd0aCA1IDAg\r\nUiAvRmlsdGVyIC9GbGF0ZURlY29kZSA+Pgpzd..."
1.9.3p385 :023 > data.encoding.name
=> "ASCII-8BIT"
1.9.3p385 :023 > data.bytesize
=> 3878164
But when I used "send_data" or "send_file" with tempfile, I getting file in invalid format (this is pdf file). It much bigger, than original and not opening by pdf readers.
This data in field is mime part of email. If I build raw email from all this parts (using boundary as separator), this email will contain valid pdf attachment.
How should I convert this data to bytes to allow user download this file separately?
See the following: http://rubyforge.org/tracker/index.php?func=detail&aid=27845&group_id=234&atid=967
The syntax is something like PGConn.unescape_bytea($field);
Depending on your version of pg, you may need to upgrade that gem
I am using MSSQL 2005.
I have StringIO object that contains my zip file content.
Here is how I obtain zip binary data:
stringio = Zip::ZipOutputStream::write_buffer do |zio|
Eclaim.find_by_sql("SET TEXTSIZE 67108864")
zio.put_next_entry("application.xml")
#zio.write #claim_db[:xml]
biblio = Nokogiri::XML('<?xml version="1.0" encoding="utf-8"?>' + #claim_db[:xml], &:noblanks)
zio.write biblio.to_xml
builder = Nokogiri::XML::Builder.new(:encoding => 'utf-8') do |xml|
xml.documents {
docs.where("ext not in (#{PROHIBITED_EXTS.collect{|v| "'#{v}'"}.join(', ')})").each{|doc|
zio.put_next_entry("#{doc[:materialtitle_id]}.#{doc[:ext]}")
zio.write doc[:efile]
xml.document(:id => doc[:materialtitle_id]) {
xml.title doc[:title]
xml.code doc[:code]
xml.filename "#{doc[:materialtitle_id]}.#{doc[:ext]}"
xml.extname doc[:ext]
}
}
}
end
zio.put_next_entry("docs.xml")
zio.write builder.to_xml
end
stringio
In my controller I try:
data.rewind
#claim.docs.create(
:title => 'Some file',
:ext => 'zip',
:size => data.length,
:receive_date => Time.now,
:efile => data.sysread
)
But Rails complains invalid byte sequence in UTF-8
Help me pls with it.
If the stream is configured as UTF-8 stream, you can't write compressed binary (which may contain any value).
I think, setting data as binary stream before write:
data.force_encoding "ASCII-8BIT"
might help.
What is complaining, ruby, ActiveRecord, or SQL Server? My guess is SQL Server. Make sure the data type of the efile field in the database is a binary BLOB.
You should break your problem down in constituate parts and troubleshoot smaller units. Remove complexity till you get to the source of the issue. For instance, have you tried just doing a simple Document.create with the attributes in say the console to remove the possibility that your controller code may be buggy? Something like Document.create :efile => File.read('sometiny.zip') and just go from there.
Assuming that works or break, you have a much simpler support request and less noise to issue ratio. Right now I suspect your controller code, not the SQL Server Adapter or the connection mode, as I have both tested to the hilt for simple binary data. Assuming the above does not work, you can then moving to examining smaller components.
For instance, what is the data type of the efile column? Do this in the console to find out, Document.columns_hash['efile'] and look at the #sql_type. Is it something suitable like varbinary(max)?
Moving on from there, what connection mode are you using with the SQL Server Adapter, TinyTDS? By default TinyTDS will convert everything to UTF8 as needed and is really smart about things. I have it tested with everything from binary to many different encodings. BTW, if you are using TinyTDS, did you make sure that you compiled FreeTDS with libiconv so it can do all this properly? You can easily check by doing the following in the console tsql -C assuming you have FreeTDS's binaries in your path. This should output a few lines, look for "iconv library: yes". Also make sure you are running 0.91 or better too!
Lastly, a bit of advice, that SET TEXTSIZE is so wrong there. You only want to do that once per connection. See here https://github.com/rails-sqlserver/activerecord-sqlserver-adapter#configure-connection--app-name
I'm having some problems reading a file from S3. I want to be able to load the ID3 tags remotely, but using open-URI doesn't work, it gives me the following error:
ruby-1.8.7-p302 > c=TagLib2::File.new(open(URI.parse("http://recordtemple.com.s3.amazonaws.com/music/745/original/The%20Stranger.mp3?1292096514")))
TypeError: can't convert Tempfile into String
from (irb):8:in `initialize'
from (irb):8:in `new'
from (irb):8
However, if i download the same file and put it on my desktop (ie no need for open-URI), it works just fine.
c=TagLib2::File.new("/Users/momofwombie/Desktop/blah.mp3")
is there something else I should be doing to read a remote file?
UPDATE: I just found this link, which may explain a little bit, but surely there must be some way to do this...
Read header data from files on remote server
Might want to check out AWS::S3, a Ruby Library for Amazon's Simple Storage Service
Do an AWS::S3:S3Object.find for the file and then an use about to retrieve the metadata
This solution assumes you have the AWS credentials and permission to access the S3 bucket that contains the files in question.
TagLib2::File.new doesn't take a file handle, which is what you are passing to it when you use open without a read.
Add on read and you'll get the contents of the URL, but TagLib2::File doesn't know what to do with that either, so you are forced to read the contents of the URL, and save it.
I also noticed you are unnecessarily complicating your use of OpenURI. You don't have to parse the URL using URI before passing it to open. Just pass the URL string.
require 'open-uri'
fname = File.basename($0) << '.' << $$.to_s
File.open(fname, 'wb') do |fo|
fo.print open("http://recordtemple.com.s3.amazonaws.com/music/745/original/The%20Stranger.mp3?1292096514").read
end
c = TagLib2::File.new(fname)
# do more processing...
File.delete(fname)
I don't have TagLib2 installed but I ran the rest of the code and the mp3 file downloaded to my disk and is playable. The File.delete would clean up afterwards, which should put you in the state you want to be in.
This solution isn't going to work much longer. Paperclip > 3.0.0 has removed to_file. I'm using S3 & Heroku. What I ended up doing was copying the file to a temporary location and parsing it from there. Here is my code:
dest = Tempfile.new(upload.spreadsheet_file_name)
dest.binmode
upload.spreadsheet.copy_to_local_file(:default_style, dest.path)
file_loc = dest.path
...
CSV.foreach(file_loc, :headers => true, :skip_blanks => true) do |row|}
This seems to work instead of open-URI:
Mp3Info.open(mp3.to_file.path) do |mp3info|
puts mp3info.tag.artist
end
Paperclip has a to_file method that downloads the file from S3.