I am recording audio, sending it as a blob to a nodejs server. The nodejs server then sends it to all connected users that are not currently recording.
Sending the blob:
mediaRecorder.onstop = function(e) {
var blob = new Blob(this.chunks, { 'type' : 'audio/ogg; codecs=opus' });
socket.emit('radio', blob);
};
Server receiving the blob:
socket.on('radio', function(blob) {
socket.broadcast.emit('voice', blob);
});
Listener receiving the blob:
socket.on('voice', function(arrayBuffer) {
var blob = new Blob([arrayBuffer], { 'type' : 'audio/ogg; codecs=opus' });
var audio = document.getElementById('audio');
audio.src = window.URL.createObjectURL(blob);
audio.play();
});
This works in all browsers/devices except safari and any ios device. Taking it further with safari's inspector I found this:
Does safari require something else in its headers for blob objects to be interpreted properly? I've researched accepted audio types, tried aac/mp3/ogg without any success. Upon reading further I've heard references to the fact that there is a bug with streaming blob audio/video data in safari and IOS though I'm not too clear any the details.
Guidance in the rite direction would be very helpful!
EDIT: It looks like this line audio.src = window.URL.createObjectURL(blob); in the receiving blob is what is causing the blob errors (image i linked)
EDIT 2: I tried to see if using another format other than blob would work, opting for base64 encoded string. Looks like this works on all devices and browsers except for IOS and Safari. I'm getting the impression it has something to do with how IOS interprets/loads the data...
For me the solution was to insert a source element into the audio element, and use the sourceElement.src attribute to refer to the blob. I didn't even need to attach the audio-element to the DOM. Example below, hope it helps someone.
var audioElement = document.createElement('audio')
var sourceElement = document.createElement('source')
audioElement.appendChild(sourceElement)
sourceElement.src = '<your blob url>'
sourceElement.type = 'audio/mp3' // or whatever
audioElement.load()
audioElement.play()
I haven't been able to find a solution using an audio element, however the Web Audio Api seems to do the trick: https://developer.mozilla.org/en-US/docs/Web/API/Web_Audio_API
var audioContext = new (window.AudioContext || window.webkitAudioContext);
socket.on('voice', function(arrayBuffer) {
audioContext.decodeAudioData(arrayBuffer, audioData => {
var source = audioContext.createBufferSource();
source.buffer = audioData;
source.connect(audioContext.destination);
source.start()
});
You may still have an issue on iOS as any audio/video must be triggered by a user action.
I had a similar problem this week. I was recording the audio on Safari and the audio blob was being generated just fine. But when I tried to send the blob to server, no data was getting there. The solution bellow reads the blob with File Reader to convert it to base64, and afterwards send it to server. Here is what worked for me:
const reader = new FileReader();
reader.readAsDataURL(audioBlob);
reader.onloadend = () => {
const base64data = reader.result;
const audioName = uuid.v4() + '.mp3';
this.http.post(this.app.cloudStorageEndpoint + '/audios/local?audioName=' + audioName, base64data , header).subscribe(
res => { resolve(audioName); },
err => { reject(err); }
);
};
I've tested with Safari 12 on both iPad and iPhone and it worked fine.
Related
I've downloaded a video to disk but it won't initialise using Video Player (https://pub.dev/packages/video_player).
final future = Downloader.shared
.getVideoPathFor(
url: url,
themeName: themeName,
)
.then(
(value) {
dLog('file path: $value');
final file = File(value);
final videoController = VideoPlayerController.file(file);
return videoController.initialize().then(
(value) {
dLog('video controller initialized');
return videoController;
},
);
},
);
It downloads the file fine and the file path becomes something like:
Application/9E6FD935-A424-4C1E-99CC-D5834448E45E/Library/Caches/videos/Clean water/clean_water_video.mp4
So I know the file exists and I can see it in Finder if I run this on Simulator.
If I use VideoController.contentUri() then it appears to work but it tells me I'm not allowed to use that on iOS and I can only use it on Android.
I know I can use VideoController.network() but I can't keep downloading the same video over and over across multiple screens like this.
What am I doing wrong?
Even when I do load the video like this (which I got from this video: https://youtu.be/uz4xRnE-UIw?t=596):
final filePath =
"<path>/Caches/videos/Clean water/clean_water_video.mp4";
final file = File(filePath);
controller = VideoPlayerController.file(file)
..addListener(() {
setState(() {});
})
..setLooping(true)
..initialize().then((value) {
controller.play();
});
the initialise never happens.
It turns out that it can't load a file if folders have spaces in the name.
We are using the ms graph api to post messages to a teams channel from a internal desktop application. The main purpose is to attach images to the message. We upload the image files into the one-drive folder of the channel as shown below.
var uploadProps = new DriveItemUploadableProperties
{
ODataType = null,
AdditionalData = new Dictionary<string, object>
{
{ "#microsoft.graph.conflictBehavior", "replace" }
}
};
var session = await graphClient.Drives[driveId]
.Items[parentId].ItemWithPath(fileName).CreateUploadSession(uploadProps).Request().PostAsync(token);
int maxSliceSize = 320 * 1024;
var fileUploadTask =
new LargeFileUploadTask<DriveItem>(session, fileStream, maxSliceSize);
// Create a callback that is invoked after each slice is uploaded
IProgress<long> progress = new Progress<long>(reportAsync);
// Upload the file
var uploadResult = await fileUploadTask.UploadAsync(progress);
if (uploadResult.UploadSucceeded)
{
return uploadResult.ItemResponse;
}
We then send a message to the channel and attach the images uploaded previously as reference attachments.
var chatMsg = new ChatMessage();
chatMsg.Body = new ItemBody();
chatMsg.Body.ContentType = BodyType.Html;
chatMsg.Body.Content = msg + " " + string.Join(" ", attachments.Select(d => $"<attachment id=\"{parseEtag(d.ETag)}\"></attachment>"));
chatMsg.Attachments = attachments.Select(d => new ChatMessageAttachment()
{
Id = parseEtag(d.ETag),
ContentType = "reference",
ContentUrl = d.WebUrl,
Name = d.Name
});
return await this.graphClient.Teams[teamId].Channels[channelId].Messages
.Request()
.AddAsync(chatMsg, token);
The problem is that the message only shows the names of the attachments with no preview as seen in the message at the bottom. We want to have a preview as seen (top message) when attaching a file within the teams application.
We've tried to set the thumbnailurl property of the attachment to the thumbnail url fetched from the ms-graph api with no success.
We've uploaded a file using the teams application (with preview) and then created an identical message with the same file (same driveitem id) in our application (show's no preview). Then we fetched both messages using the graph api and could not discern any differences between the two besides the message id's ofc.
We've scoured these forums, the ms documentations and even suggestion pages and found nothing.
We have been able to show previews separately in the body of the message referencing the thumbnail urls and in messagecards but ideally we want the preview directly in the attachments.
EDIT
The thumbnail urls seem to expire after 24 hours and are therefor not a great solution.
We managed to solve exactly this problem using the Simple Upload Api, with the added ?$expand=thumbnails query parameter. I haven't tried but the query param ought to work for the endpoint you're using as well.
Pick a size from the ThumbnailSet in the upload response and add it to the body of your message as an image tag. See below:
// channel, file, extractIdFromEtag, message omitted for brevity.
// PUT /groups/{group-id}/drive/items/{parent-id}:/{filename}:/content
const uploadUrl = `https://graph.microsoft.com/beta/groups/${channel.teamId}/drive/items/root:/${channel.displayName}/${file.name}:/content?$expand=thumbnails`;
const res = await this.http.put(uploadUrl, file).toPromise(); // FYI Using Angular http service
const attachment = {
id: extractIdFromEtag(res.eTag),
contentType: 'reference',
contentUrl: res.webUrl,
name: res.name,
thumbnailUrl: res.webUrl
};
const postBody = {
subject: null,
body: {
contentType: 'html',
content: message
},
};
// This is what makes the image show in the message as if posted from teams
postBody.body.content += `<br><br><img src="${res.thumbnails[0].large.url}" alt="${res.name}"/>`;
const messageUrl = `https://graph.microsoft.com/beta/teams/${channel.teamId}/channels/${channel.id}/messages`;
const result = await this.http.post(messageUrl, postBody).toPromise();
// Done
You can also keep adding the attachment as you already do, if you want the original image attached as a file, as well as showing the image preview in the message.
I'm a relatively new programmer trying to create an ios video-streaming app in swift. I've written the backend in node.js and have a mongoose connection to a mongoDB. I've been able to upload videos to the database using the following code and gridfs
function (req, res, next) {
req.pipe(gfs.createWriteStream({ filename: 'filename'}));
res.send("success");
};
I'm attempting to stream the videos using the following code
gfs.findOne({_id: req.params.id}, function (err, file) {
if (err) {
return res.status(400).send(err);
} else if (!file) {
return res.status(404).send(' ');
} else {
res.header("Content-Type","video/mp4");
res.header("X-Content-Type-Options", "nosniff");
res.header("Accept-Ranges", "bytes");
res.header("Content-Length",file.length);
var readStream = gfs.createReadStream({_id: file._id});
readStream.on('open', function () {
console.log('Starting download...');
});
readStream.on('data', function (chunk) {
console.log('Loading...');
});
readStream.on('end', function () {
console.log('Video is ready to play');
});
readStream.on('error', function (err) {
console.log('There was an error with the download' + err);
res.end();
});
readStream.pipe(res);
}
});
When I run a server on localhost and attempt to access the videos via google chrome, all I get is the default playback screen but no video. However, ultimately I am trying to playback on an ios app. When I try the same thing on swift (using the local host url passed in to mpmovieplayer), there is also no video playback. I know the get request is going through because my console outputs the proper response with the filesize. I have even been fiddling with alamofire on the front end and can see the hexadecimal representation of the video in the response data. Can anybody help with this code? Do I need to update my res.header to fit some IOS specification? Should I even be using alamofire at all for this? Thanks in advance for your responses.
I'm trying desperately to create a Firefox add-on that posts a file with the field name "Filedata" to a particular PHP script which will only work if it sees a JPG in the $_FILE["Filedata"] variable.
I put a web form with a file browser into panel.html, then I take the image and turn it into a canvas which I turn into a blob and send to main.js. I would be happy to send the file directly from panel.js, but nothing at all happens (no error message either) when I attempt to so.
In main.js, I have this code but I get an error message that FormData doesn't exist in main.js. What to do?
function ajupload(mydata) {
var fd = new FormData();
fd.append("Filedata", mydata);
const {XMLHttpRequest} = require("sdk/net/xhr");
var myrequest = new XMLHttpRequest();
myrequest.open('POST', 'MYSITE/image.php?action=upload');myrequest.setRequestHeader("Content-type","application/x-www-form-urlencoded");
myrequest.upload.addEventListener("progress", function(e) {
var percentage = Math.round((e.loaded * 100) / e.total);
}, false);
myrequest.onreadystatechange=function()
{
if (myrequest.readyState==4 && myrequest.status==200)
{
console.log("Response" + myrequest.responseText);
}
}
myrequest.send(fd);
}
I am working on trying to get iOS 6 to use XMLHttpRequest POSTs to upload images. This works on desktop and Android web browsers, but with iOS 6 I am getting an error on the page being posted to: "Request Body Stream Exhausted". (Using iOS Simulator with the Safari Web Inspector).
Here is the basic code of the page:
function fileSelected() {
var file = document.getElementById('fileToUpload').files[0];
if (file) {
var fileSize = 0;
if (file.size > 1024 * 1024)
fileSize = (Math.round(file.size * 100 / (1024 * 1024)) / 100).toString() + 'MB';
else
fileSize = (Math.round(file.size * 100 / 1024) / 100).toString() + 'KB';
document.getElementById('fileName').innerHTML = 'Name: ' + file.name;
document.getElementById('fileSize').innerHTML = 'Size: ' + fileSize;
document.getElementById('fileType').innerHTML = 'Type: ' + file.type;
}
}
function uploadFile() {
var fd = new FormData();
fd.append("fileToUpload", document.getElementById('fileToUpload').files[0]);
var xhr = new XMLHttpRequest();
xhr.upload.addEventListener("progress", uploadProgress, false);
xhr.addEventListener("load", uploadComplete, false);
xhr.addEventListener("error", uploadFailed, false);
xhr.addEventListener("abort", uploadCanceled, false);
xhr.open("POST", "/UploadHandler.ashx");
xhr.send(fd);
}
function uploadProgress(evt) {
if (evt.lengthComputable) {
var percentComplete = Math.round(evt.loaded * 100 / evt.total);
document.getElementById('progressNumber').innerHTML = percentComplete.toString() + '%';
document.getElementById('prog').value = percentComplete;
}
else {
document.getElementById('progressNumber').innerHTML = 'unable to compute';
}
}
function uploadComplete(evt) {
/* This event is raised when the server send back a response */
alert(evt.target.responseText);
}
function uploadFailed(evt) {
alert("There was an error attempting to upload the file.");
}
function uploadCanceled(evt) {
alert("The upload has been canceled by the user or the browser dropped the connection.");
}
When doing this on any other browser, the handler returns correctly and uploads the file. However, with iOS the ashx page has the error "request body stream exhausted".
Here is a screenshot of the inspector:
Any ideas?
UPDATE: This issue only occurs when NTLM/Windows authentication is enabled for the application in IIS. With forms or anonymous authentication, the upload works fine.
Thanks,
John
In iOS 6, Safari sends the file with the initial post, including the file. That means the file stream is at the end, or "exhausted."
However, with NTLM, it will get a 401 challenge in response, and then have to resend the post with the authentication information. Since it does not reset the file stream, it is unable to send the file again with the second post. You can see this in the IIS logs.
As far as I know, there is no particularly good way around it. I am changing my mobile app, so that it uses form authentication. I direct the mobile app to a separate login app on the same server, which is set to use Windows Authentication. The login app can then redirect back to the main app with a form authentication cookie, and all is well again.
You have to set the machine key on both apps in the web.config file, so that both are using the same keys for encryption and validation.
The code on the login app is as simple as
Protected Sub Page_Load(ByVal sender As Object, ByVal e As System.EventArgs) _
Handles Me.Load
With HttpContext.Current.User.Identity
If .IsAuthenticated Then
Dim sUser As String = .Name.ToLower.Trim
FormsAuthentication.RedirectFromLoginPage(s, False)
End If
End With
End Sub
I solved the issue by not setting the self-defined HTTP Authenticate Head on iOS 7 and iOS 8. (At first, our service use the self-defined value for the Authenticate Head). And after the challenge being handled by the delegate, the request will have the "Authenticate: NTLMxxx automatically" header automatically. And the Put and POST works again.
This error Comes on IOS but you can use different approach
like change your code line in formdata where you appending the file
var fd = new FormData();
fd.append("fileToUpload", document.getElementById('fileToUpload').files[0]);
to the below line basically don't append the file control but use the base64 image data
var reader = new FileReader();
reader.readAsDataURL(opmlFile.files[0]);
reader.onload = function () {
var base64DataImg = reader.result;
base64DataImg = base64DataImg.replace('data:'imagetype';base64,', '');
}