How to send HttpPostedFileBase to S3 via AWS SDK - asp.net-mvc

I'm having some trouble getting uploaded files to save to S3. My first attempt was:
Result SaveFile(System.Web.HttpPostedFileBase file, string path)
{
//Keys are in web.config
var t = new Amazon.S3.Transfer.TransferUtility(Amazon.RegionEndpoint.USWest2);
try
{
t.Upload(new Amazon.S3.Transfer.TransferUtilityUploadRequest
{
BucketName = Bucket,
InputStream = file.InputStream,
Key = path
});
}
catch (Exception ex)
{
return Result.FailResult(ex.Message);
}
return Result.SuccessResult();
}
This throws an exception with the message: "The request signature we calculated does not match the signature you provided. Check your key and signing method." I also tried copying file.InputStream to a MemoryStream, then uploading that, with the same error.
If I set the InputStream to:
new FileStream(#"c:\folder\file.txt", FileMode.Open)
then the file uploads fine. Do I really need to save the file to disk before uploading it?

This is my working version first the upload method:
public bool Upload(string filePath, Stream inputStream, double contentLength, string contentType)
{
try
{
var request = new PutObjectRequest();
request.WithBucketName(_bucketName)
.WithCannedACL(S3CannedACL.PublicRead)
.WithKey(filePath).InputStream = inputStream;
request.AddHeaders(AmazonS3Util.CreateHeaderEntry("ContentType", contentType));
_amazonS3Client.PutObject(request);
}
catch (Exception exception)
{
// log or throw;
return false;
}
return true;
}
I just get the stream from HttpPostedFileBase.InputStream
(Note, this is on an older version of the Api, the WithBucketName syntax is no longer supported, but just set the properties directly)

Following the comment of shenku, for newer versions of SDK.
public bool Upload(string filePath, Stream inputStream, double contentLength, string contentType)
{
try
{
var request = new PutObjectRequest();
string _bucketName = "";
request.BucketName = _bucketName;
request.CannedACL = S3CannedACL.PublicRead;
request.InputStream = inputStream;
request.Key = filePath;
request.Headers.ContentType = contentType;
PutObjectResponse response = _amazonS3Client.PutObject(request);
return true;
}catch(Exception ex)
{
return false;
}
}

Related

Large File upload to ASP.NET Core 3.0 Web API fails due to Request Body to Large

I have an ASP.NET Core 3.0 Web API endpoint that I have set up to allow me to post large audio files. I have followed the following directions from MS docs to set up the endpoint.
https://learn.microsoft.com/en-us/aspnet/core/mvc/models/file-uploads?view=aspnetcore-3.0#kestrel-maximum-request-body-size
When an audio file is uploaded to the endpoint, it is streamed to an Azure Blob Storage container.
My code works as expected locally.
When I push it to my production server in Azure App Service on Linux, the code does not work and errors with
Unhandled exception in request pipeline: System.Net.Http.HttpRequestException: An error occurred while sending the request. ---> Microsoft.AspNetCore.Server.Kestrel.Core.BadHttpRequestException: Request body too large.
Per advice from the above article, I have configured incrementally updated Kesterl with the following:
.ConfigureWebHostDefaults(webBuilder =>
{
webBuilder.UseKestrel((ctx, options) =>
{
var config = ctx.Configuration;
options.Limits.MaxRequestBodySize = 6000000000;
options.Limits.MinRequestBodyDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.MinResponseDataRate =
new MinDataRate(bytesPerSecond: 100,
gracePeriod: TimeSpan.FromSeconds(10));
options.Limits.RequestHeadersTimeout =
TimeSpan.FromMinutes(2);
}).UseStartup<Startup>();
Also configured FormOptions to accept files up to 6000000000
services.Configure<FormOptions>(options =>
{
options.MultipartBodyLengthLimit = 6000000000;
});
And also set up the API controller with the following attributes, per advice from the article
[HttpPost("audio", Name="UploadAudio")]
[DisableFormValueModelBinding]
[GenerateAntiforgeryTokenCookie]
[RequestSizeLimit(6000000000)]
[RequestFormLimits(MultipartBodyLengthLimit = 6000000000)]
Finally, here is the action itself. This giant block of code is not indicative of how I want the code to be written but I have merged it into one method as part of the debugging exercise.
public async Task<IActionResult> Audio()
{
if (!MultipartRequestHelper.IsMultipartContentType(Request.ContentType))
{
throw new ArgumentException("The media file could not be processed.");
}
string mediaId = string.Empty;
string instructorId = string.Empty;
try
{
// process file first
KeyValueAccumulator formAccumulator = new KeyValueAccumulator();
var streamedFileContent = new byte[0];
var boundary = MultipartRequestHelper.GetBoundary(
MediaTypeHeaderValue.Parse(Request.ContentType),
_defaultFormOptions.MultipartBoundaryLengthLimit
);
var reader = new MultipartReader(boundary, Request.Body);
var section = await reader.ReadNextSectionAsync();
while (section != null)
{
var hasContentDispositionHeader = ContentDispositionHeaderValue.TryParse(
section.ContentDisposition, out var contentDisposition);
if (hasContentDispositionHeader)
{
if (MultipartRequestHelper
.HasFileContentDisposition(contentDisposition))
{
streamedFileContent =
await FileHelpers.ProcessStreamedFile(section, contentDisposition,
_permittedExtensions, _fileSizeLimit);
}
else if (MultipartRequestHelper
.HasFormDataContentDisposition(contentDisposition))
{
var key = HeaderUtilities.RemoveQuotes(contentDisposition.Name).Value;
var encoding = FileHelpers.GetEncoding(section);
if (encoding == null)
{
return BadRequest($"The request could not be processed: Bad Encoding");
}
using (var streamReader = new StreamReader(
section.Body,
encoding,
detectEncodingFromByteOrderMarks: true,
bufferSize: 1024,
leaveOpen: true))
{
// The value length limit is enforced by
// MultipartBodyLengthLimit
var value = await streamReader.ReadToEndAsync();
if (string.Equals(value, "undefined",
StringComparison.OrdinalIgnoreCase))
{
value = string.Empty;
}
formAccumulator.Append(key, value);
if (formAccumulator.ValueCount >
_defaultFormOptions.ValueCountLimit)
{
return BadRequest($"The request could not be processed: Key Count limit exceeded.");
}
}
}
}
// Drain any remaining section body that hasn't been consumed and
// read the headers for the next section.
section = await reader.ReadNextSectionAsync();
}
var form = formAccumulator;
var file = streamedFileContent;
var results = form.GetResults();
instructorId = results["instructorId"];
string title = results["title"];
string firstName = results["firstName"];
string lastName = results["lastName"];
string durationInMinutes = results["durationInMinutes"];
//mediaId = await AddInstructorAudioMedia(instructorId, firstName, lastName, title, Convert.ToInt32(duration), DateTime.UtcNow, DateTime.UtcNow, file);
string fileExtension = "m4a";
// Generate Container Name - InstructorSpecific
string containerName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{instructorId}";
string contentType = "audio/mp4";
FileType fileType = FileType.audio;
string authorName = $"{firstName} {lastName}";
string authorShortName = $"{firstName[0]}{lastName}";
string description = $"{authorShortName} - {title}";
long duration = (Convert.ToInt32(durationInMinutes) * 60000);
// Generate new filename
string fileName = $"{firstName[0].ToString().ToLower()}{lastName.ToLower()}-{Guid.NewGuid()}";
DateTime recordingDate = DateTime.UtcNow;
DateTime uploadDate = DateTime.UtcNow;
long blobSize = long.MinValue;
try
{
// Update file properties in storage
Dictionary<string, string> fileProperties = new Dictionary<string, string>();
fileProperties.Add("ContentType", contentType);
// update file metadata in storage
Dictionary<string, string> metadata = new Dictionary<string, string>();
metadata.Add("author", authorShortName);
metadata.Add("tite", title);
metadata.Add("description", description);
metadata.Add("duration", duration.ToString());
metadata.Add("recordingDate", recordingDate.ToString());
metadata.Add("uploadDate", uploadDate.ToString());
var fileNameWExt = $"{fileName}.{fileExtension}";
var blobContainer = await _cloudStorageService.CreateBlob(containerName, fileNameWExt, "audio");
try
{
MemoryStream fileContent = new MemoryStream(streamedFileContent);
fileContent.Position = 0;
using (fileContent)
{
await blobContainer.UploadFromStreamAsync(fileContent);
}
}
catch (StorageException e)
{
if (e.RequestInformation.HttpStatusCode == 403)
{
return BadRequest(e.Message);
}
else
{
return BadRequest(e.Message);
}
}
try
{
foreach (var key in metadata.Keys.ToList())
{
blobContainer.Metadata.Add(key, metadata[key]);
}
await blobContainer.SetMetadataAsync();
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
blobSize = await StorageUtils.GetBlobSize(blobContainer);
}
catch (StorageException e)
{
return BadRequest(e.Message);
}
Media media = Media.Create(string.Empty, instructorId, authorName, fileName, fileType, fileExtension, recordingDate, uploadDate, ContentDetails.Create(title, description, duration, blobSize, 0, new List<string>()), StateDetails.Create(StatusType.STAGED, DateTime.MinValue, DateTime.UtcNow, DateTime.MaxValue), Manifest.Create(new Dictionary<string, string>()));
// upload to MongoDB
if (media != null)
{
var mapper = new Mapper(_mapperConfiguration);
var dao = mapper.Map<ContentDAO>(media);
try
{
await _db.Content.InsertOneAsync(dao);
}
catch (Exception)
{
mediaId = string.Empty;
}
mediaId = dao.Id.ToString();
}
else
{
// metadata wasn't stored, remove blob
await _cloudStorageService.DeleteBlob(containerName, fileName, "audio");
return BadRequest($"An issue occurred during media upload: rolling back storage change");
}
if (string.IsNullOrEmpty(mediaId))
{
return BadRequest($"Could not add instructor media");
}
}
catch (Exception ex)
{
return BadRequest(ex.Message);
}
var result = new { MediaId = mediaId, InstructorId = instructorId };
return Ok(result);
}
I reiterate, this all works great locally. I do not run it in IISExpress, I run it as a console app.
I submit large audio files via my SPA app and Postman and it works perfectly.
I am deploying this code to an Azure App Service on Linux (as a Basic B1).
Since the code works in my local development environment, I am at a loss of what my next steps are. I have refactored this code a few times but I suspect that it's environment related.
I cannot find anywhere that mentions that the level of App Service Plan is the culprit so before I go out spending more money I wanted to see if anyone here had encountered this challenge and could provide advice.
UPDATE: I attempted upgrading to a Production App Service Plan to see if there was an undocumented gate for incoming traffic. Upgrading didn't work either.
Thanks in advance.
-A
Currently, as of 11/2019, there is a limitation with the Azure App Service for Linux. It's CORS functionality is enabled by default and cannot be disabled AND it has a file size limitation that doesn't appear to get overridden by any of the published Kestrel configurations. The solution is to move the Web API app to a Azure App Service for Windows and it works as expected.
I am sure there is some way to get around it if you know the magic combination of configurations, server settings, and CLI commands but I need to move on with development.

How to Upload a Profile photo in base64 format For Community Users Using ConnectApi.UserProfiles.setPhoto

1Am Uploading Profile Photo for Community Users in base64 format By Using ConnectApi.UserProfiles.setPhoto Method. But am getting "ConnectApi.ConnectApiException: The file you uploaded doesn't appear to be a valid image" This error, Help me to Fix this issue.
Hi you can try the below method:
public PageReference upload() {
Blob b;
document.AuthorId = UserInfo.getUserId();
document.FolderId = UserInfo.getUserId(); // put it in running user's folder
try {
document.type = 'jpg';
document.IsPublic = true;
insert document;
// ImageId = '06990000001HnuB';
b = document.Body;
//ConnectApi.ChatterUsers newPhoto = new ConnectApi.ChatterUsers();
} catch (DMLException e) {
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.ERROR, 'Error uploading file'));
return null;
} finally {
document.body = null; // clears the viewstate
document = new Document();
}
ApexPages.addMessage(new ApexPages.message(ApexPages.severity.INFO, 'File uploaded successfully : ' + b));
String communityId = null;
String userId = UserInfo.getUserId();
//ID fileId = ImageId;
// Set photo
ConnectApi.Photo photo = ConnectApi.ChatterUsers.setPhoto(communityId, userId, new ConnectApi.BinaryInput(b, 'image/jpg', 'userImage.jpg'));
return null;
}
I was getting the same error, there isn't too much detail so I'm not too sure about your problem. I solved the problem using the code below. Modify as needed.
public static Boolean updateUserProfilePic(String userProfilePicString, String userId, String fileType){
Boolean updateSuccessful = true;
System.debug('-------------' + userProfilePicString.length());
try{
Blob blobImage = EncodingUtil.base64Decode(userProfilePicString);
ConnectApi.BinaryInput fileUpload = new ConnectApi.BinaryInput(blobImage, 'image/jpg', 'userImage.jpg');
ConnectApi.Photo photoProfile = ConnectApi.UserProfiles.setPhoto(null, userId, fileUpload);
}
catch(Exception exc){
updateSuccessful = false;
}
return updateSuccessful;
}

JavaMail MIME attachment link by cid

Background
I have banged my head against this for a while and not made much progress. I am generating MPEG_4 / AAC files in Android and sending them by email as .mp3 files. I know they aren't actually .mp3 files, but that allows Hotmail and Gmail to play them in Preview. They don't work on iPhone though, unless they are sent as .m4a files instead which breaks the Outlook / Gmail Preview.
So I have thought of a different approach which is to attach as a .mp3 file but have an HTML link in the email body which allows the attached file to be downloaded and specifies a .m4a file name. Gmail / Outlook users can click the attachment directly whereas iPhone users can use the HTML link.
Issue
I can send an email using JavaMail with HTML in it including a link which should be pointing at the attached file to allow download of that file by the link. Clicking on the link in Gmail (Chrome on PC) gives a 404 page and iPhone just ignores my clicking on the link.
Below is the code in which I generate a multipart message and assign a CID to the attachment which I then try to access using the link in the html part. It feels like I am close, but maybe that is an illusion. I'd be massively grateful if someone could help me fix it or save me the pain if it isn't possible.
private int send_email_temp(){
Properties props = new Properties();
props.put("mail.smtp.auth", "true");
props.put("mail.smtp.host", smtp_host_setting);
//props.put("mail.debug", "true");
props.put("mail.smtp.ssl.enable", "true");
props.put("mail.smtp.starttls.enable", "true");
props.put("mail.smtp.port", smtp_port_setting);
session = Session.getInstance(props);
ActuallySendAsync_temp asy = new ActuallySendAsync_temp(true);
asy.execute();
return 0;
}
class ActuallySendAsync_temp extends AsyncTask<String, String, Void> {
public ActuallySendAsync_temp(boolean boo) {
// something to do before sending email
}
#Override
protected Void doInBackground(String... params) {
try {
Message message = new MimeMessage(session);
message.setFrom(new InternetAddress(username));
message.setRecipients(Message.RecipientType.TO,
InternetAddress.parse(recipient_email_address));
message.setSubject(email_subject);
Multipart multipart = new MimeMultipart();
MimeBodyPart messageBodyPart = new MimeBodyPart();
String file = mFileName;
/**/
DataSource source = new FileDataSource(file);
messageBodyPart.setDataHandler(new DataHandler(source));
/* /
File ff = new File(file);
try {
messageBodyPart.attachFile(ff);
} catch(IOException eio) {
Log.e("Message Error", "Old Macdonald");
}
/* /
messageBodyPart = new PreencodedMimeBodyPart("base64");
byte[] file_bytes = null;
File ff = new File(file);
try {
int length = (int) ff.length();
BufferedInputStream reader = new BufferedInputStream(new FileInputStream(ff));
file_bytes = new byte[length];
reader.read(file_bytes, 0, length);
reader.close();
} catch (IOException eio) {
Log.e("Message Error", "Old Macdonald");
}
messageBodyPart.setText(Base64.encodeToString(file_bytes, Base64.DEFAULT));
messageBodyPart.setHeader("Content-Transfer-Encoding", "base64");
/**/
messageBodyPart.setFileName( DEFAULT_AUDIO_FILENAME );//"AudioClip.mp3");
//messageBodyPart.setContentID("<audio_clip>");
String content_id = UUID.randomUUID().toString();
messageBodyPart.setContentID("<" + content_id + ">");
messageBodyPart.setDisposition(Part.ATTACHMENT);//INLINE);
messageBodyPart.setHeader("Content-Type", "audio/mp4");
multipart.addBodyPart(messageBodyPart);
MimeBodyPart messageBodyText = new MimeBodyPart();
//final String MY_HTML_MESSAGE = "<h1>My HTML</h1><a download=\"AudioClip.m4a\" href=\"cid:audio_clip\">iPhone Download</a>";
final String MY_HTML_MESSAGE = "<h1>My HTML</h1><a download=\"AudioClip.m4a\" href=\"cid:" + content_id + "\">iPhone Download</a>";
messageBodyText.setContent( MY_HTML_MESSAGE, "text/html");
multipart.addBodyPart(messageBodyText);
message.setContent(multipart);
Print_Message_To_Console(message);
Transport transport = session.getTransport("smtp");
transport.connect(smtp_host_setting, username, password);
transport.sendMessage(message, message.getAllRecipients());
transport.close();
} catch (MessagingException e) {
e.printStackTrace();
} finally {
}
return null;
}
#Override
protected void onPostExecute(Void aVoid) {
super.onPostExecute(aVoid);
// something to do after sending email
}
}
int Print_Message_To_Console(Message msg) {
int ret_val = 0;
int line_num = 0;
InputStream in = null;
InputStreamReader inputStreamReader = null;
BufferedReader buff_reader = null;
try {
in = msg.getInputStream();
inputStreamReader = new InputStreamReader(in);
buff_reader = new BufferedReader(inputStreamReader);
String temp = "";
while ((temp = buff_reader.readLine()) != null) {
Log.d("Message Line " + Integer.toString(line_num++), temp);
}
} catch(Exception e) {
Log.d("Message Lines", "------------ OOPS! ------------");
ret_val = 1;
} finally {
try {
if (buff_reader != null) buff_reader.close();
if (inputStreamReader != null) inputStreamReader.close();
if (in != null) in.close();
} catch(Exception e2) {
Log.d("Message Lines", "----------- OOPS! 2 -----------");
ret_val = 2;
}
}
return ret_val;
}
You need to create a multipart/related and set the main text part as the first body part.

How to download file from HTTPResponseMessage

I have included a link on my website to download images. When I click on the link I would like the download to automatically start.
Currently when I click on the link I’m getting back the response message: Example:
StatusCode: 200, ReasonPhrase: 'OK', Version: 1.1, Content: System.Net.Http.PushStreamContent, Headers: { Content-Type: application/octet-stream Content-Disposition: attachment; filename=895621d7-57a4-47a5-8dc5-ae36a2623826Banneraaaaaaaa.jpg }
How do I modify the code below to start the download automatically. I think I might be returning the wrong type:
Here is my code:
public HttpResponseMessage DownloadImageFile(string filepath)
{
filepath = "https://mysite.com/" + filepath;
try
{
var response = new HttpResponseMessage();
response.Content = new PushStreamContent((Stream outputStream, HttpContent content, TransportContext context) =>
{
try
{
DownloadFile(filepath, outputStream);
}
finally
{
outputStream.Close();
}
});
response.Content.Headers.ContentType = new System.Net.Http.Headers.MediaTypeHeaderValue("application/octet-stream");
response.Content.Headers.ContentDisposition = new System.Net.Http.Headers.ContentDispositionHeaderValue("attachment");
response.Content.Headers.ContentDisposition.FileName = Path.GetFileName(filepath);
return response;
}
catch (Exception ex)
{
}
return null;
}
public void DownloadFile(string file, Stream response)
{
var bufferSize = 1024 * 1024;
using (var stream = new FileStream(file, FileMode.Open, FileAccess.Read))
{
var buffer = new byte[bufferSize];
var bytesRead = 0;
while ((bytesRead = stream.Read(buffer, 0, bufferSize)) > 0)
{
response.Write(buffer, 0, bytesRead);
}
response.Flush();
}
}
}
You should use one of the Controller.File overload. The File() helper method provides support for returning the contents of a file. The MediaTypeNames class can be used to get the MIME type for a specific file name extension.
For example:
public FileResult Download(string fileNameWithPath)
{
// Option 1 - Native support for file read
return File(fileNameWithPath, System.Net.Mime.MediaTypeNames.Application.Octet, Path.GetFileName(fileNameWithPath));
// Option 2 - Read byte array and pass to file object
//byte[] fileBytes = System.IO.File.ReadAllBytes(fileName); return
//File(fileBytes, System.Net.Mime.MediaTypeNames.Application.Octet,
//fileName);
}

how to send mail with image attachment in blackberry

I am developing a BlackBerry application that uses the Mail functionality. My problem is
I want to send mail with an image attachment. How can I do that?
You can convert the image to byte array and then use the following method to send the file as attachment.
public synchronized boolean sendMail(final byte []data)
{
Folder[] folders = store.list(4);
Folder sentfolder = folders[0];
// create a new message and store it in the sent folder
msg = new Message(sentfolder);
multipart = new Multipart();
textPart = new TextBodyPart(multipart,"Image");
Address recipients[] = new Address[1];
try {
recipients[0] = new Address(address, "XYZ");
msg.addRecipients(Message.RecipientType.TO, recipients);
msg.setSubject("Image");
try {
Thread thread = new Thread("Send mail") {
public void run() {
try {
attach = new SupportedAttachmentPart(
multipart, "application/octet-stream",
"title",data);
multipart.addBodyPart(textPart);
multipart.addBodyPart(attach);
msg.setContent(multipart);
Transport.send(msg);
}
catch(SendFailedException e)
{
}
catch (final MessagingException e) {
}
catch (final Exception e) {
}
}
};
thread.start();
return true;
}
catch (final Exception e)
{
}
}catch (final Exception e) {
}
return false;
}
This may be help you check it
//create a multipart
Multipart mp = new Multipart();
//data for the content of the file
String fileData = "<html>just a simple test</html>";
String messageData = "Mail Attachment Demo";
//create the file
SupportedAttachmentPart sap = new SupportedAttachmentPart(mp,"text/html","file.html",fileData.getBytes());
TextBodyPart tbp = new TextBodyPart(mp,messageData);
//add the file to the multipart
mp.addBodyPart(tbp);
mp.addBodyPart(sap);
//create a message in the sent items folder
Folder folders[] = Session.getDefaultInstance().getStore().list(Folder.SENT);
Message message = new Message(folders[0]);
//add recipients to the message and send
try {
Address toAdd = new Address("email#company.com","my email");
Address toAdds[] = new Address[1];
toAdds[0] = toAdd;
message.addRecipients(Message.RecipientType.TO,toAdds);
message.setContent(mp);
Transport.send(message);
} catch (Exception e) {
Dialog.inform(e.toString());
}
this is for Image file
InputStream inputStream;
FileConnection fconn = (FileConnection) Connector.open(fName, Connector.READ_WRITE);
if(fconn.exists()){
inputStream=fconn.openInputStream();
byte[] data = IOUtilities.streamToBytes(inputStream);
inputStream.close();
fconn.close();
Multipart multipart = new Multipart();
SupportedAttachmentPart attach = new SupportedAttachmentPart(multipart, ".txt/.jpeg", "attachment1", data);
multipart.addBodyPart(attach);
}

Resources