With the Chrome 71 changes that start audiocontext's in a suspended state, we need to resume the audio context after a user interaction. We are handling this by displaying a red banner at the top of our website saying we need you to grant us permission to enable calling with a button. The button meets the user engagement so we can resume the audio context.
The issue I am having is that the device which I have already called device.setup() on does not have the audioContext available when the user clicks the button.
I should be able to access the audioContext through device.audio.audioContext and call the resume() method on it, but what I get instead is undefined.
Currently as a workaround I am calling device.audio._audioContext.resume() and this is never undefined and works. But it seems to me like there is some bug where the public accessor for audioContext is not being defined in the twilio-client library. Or am I doing something wrong here?
In the es5 compiled code I see the following in the device.js file I see
Object.defineProperty(Device, "audioContext", {
/**
* The AudioContext to be used by {#link Device} instances.
*/
get: function () {
return Device._audioContext;
},
enumerable: true,
configurable: true
});
I get undefined when calling this. However, the device.audio property is set in this bit of code, where it passes Device.audioContext as an option:
this.audio = new (this.options.AudioHelper || AudioHelper)
(this._updateSinkIds, this._updateInputStream, getUserMedia, {
audioContext: Device.audioContext,
enabledSounds: this._enabledSounds,
logEnabled: !!this.options.debug,
logWarnings: !!this.options.warnings,
});
And then in the audiohelper.js file the constructor calls:
Object.defineProperties(this, {
_audioContext: {
value: audioContext
},
// more stuff
},
so it seems like when the device is initially created the Device.audioContext is in existence and is passed through to audio._audioContext, but later Device.audioContext loses its reference to the audio context.
Twilio developer evangelist here.
This may be a failure of documentation or communication, but I think I know what is going on now.
Device.audioContext is a public method on the Device class. As you discovered in the code it is defined here:
Object.defineProperty(Device, "audioContext", {
/**
* The AudioContext to be used by {#link Device} instances.
*/
get: function () {
return Device._audioContext;
},
enumerable: true,
configurable: true
});
Note that this defines the audioContext property on the Device object itself.
But there is no definition for audioContext on instances of Device. This is why your code in the gist you posted fails when you try to call state.device.audioContext.
As you point out, the audio object on the device object is created using Device.audioContext. So, you can actually drop using the pseudo-private device.audio._audioContext and just use the public Device.audioContext when you want to resume().
I would rewrite to:
resumeTwilioDevice = async () => {
Twilio.Device.audioContext.resume();
};
Related
I'm currently experimenting with Isolates in dart.
I'm trying to create a wrapper around an Isolate to make it more pleasant to use.
The desired interface is something along the lines:
abstract class BgIsolateInterface {
Future<Response> send<Message, Response>(Message message);
}
I want to have a method that sends a message to the background interface and then return the response to the caller.
To achieve this I figured I have to create a new RawReceivePort or ReceivePort in the send function to reliably get the correct response.
But this would mean I'm essentially creating the port and discarding it. Going against the documentations which states
Opens a long-lived port for receiving messages.
So my questions are:
what exactly are ReceivePorts and RawReceivePorts?
would my use case be valid i.e. have them be created only to read a single response?
should I look at another way of doing things?
Note: Please don't suggest the Flutter compute function as an alternative. I'm looking to do this in a long running isolate so I can share services / state between function calls. I'm just not showing this here to keep the question short.
Thank you very much!!!
Edit #1:
When providing the answer I realised there was also an underling question about how to read the Dart source, more specifically how to find external methods' implementations. That question was added to the title. The original question was just: What exactly is a ReceivePort / RawReceivePort?.
Yesterday, I've searched across the source and I think, I now have the answers. If I'm wrong, anyone more involved with the engine please correct me. This is mostly my speculation.
TLDR:
ReceivePort/RawReceivePorts are essentially int ids with a registered message handler. The SendPort knows to which id i.e. ReceivePort/RawReceivePort it should send the data to.
Yes. But for another use case there is better way.
Change the interface, so we react to states / responses coming from the isolate i.e.
abstract class BgIsolateInterface<Message, Response> {
void send(Message message);
void listen(void Function(Response) onData);
}
Long
#1
I've looked at the implementation and I'm including my findings here also to put a note for my future self on how to actually do this if I ever need to.
First, if we look at the implementation of ReceivePort (comments removed):
abstract class ReceivePort implements Stream<dynamic> {
external factory ReceivePort([String debugName = '']);
external factory ReceivePort.fromRawReceivePort(RawReceivePort rawPort);
StreamSubscription<dynamic> listen(void onData(var message)?,
{Function? onError, void onDone()?, bool? cancelOnError});
void close();
SendPort get sendPort;
}
We can see the external keyword. Now, this means implementation is defined somewhere else. Great! Where?
Let's open the SDK source and look. We are looking for a class definition of the same name i.e. ReceivePort with a #patch annotation. Also it seems the Dart team follows the convention of naming the implementation files for these external methods with the suffix _patch.dart.
We then find the three of these patch files. Two for the js runtime, one for development and one for production, and one file for the native? runtime. Since, I'm not using Dart for the web, the latter is the one I'm interested in.
In the file: sdk/lib/_internal/vm/lib/isolate_patch.dart we see:
#patch
class ReceivePort {
#patch
factory ReceivePort([String debugName = '']) =>
new _ReceivePortImpl(debugName);
#patch
factory ReceivePort.fromRawReceivePort(RawReceivePort rawPort) {
return new _ReceivePortImpl.fromRawReceivePort(rawPort);
}
}
Ok, so the implementation for ReceivePort is actually a library private _ReceivePortImpl class.
Note: As you can see factory methods don't have to return the same class the method is defined in. You just have to return an object that implements or extends it. i.e., has the same contract.
class _ReceivePortImpl extends Stream implements ReceivePort {
_ReceivePortImpl([String debugName = ''])
: this.fromRawReceivePort(new RawReceivePort(null, debugName));
_ReceivePortImpl.fromRawReceivePort(this._rawPort)
: _controller = new StreamController(sync: true) {
_controller.onCancel = close;
_rawPort.handler = _controller.add;
}
SendPort get sendPort {
return _rawPort.sendPort;
}
StreamSubscription listen(void onData(var message)?,
{Function? onError, void onDone()?, bool? cancelOnError}) {
return _controller.stream.listen(onData,
onError: onError, onDone: onDone, cancelOnError: cancelOnError);
}
close() {
_rawPort.close();
_controller.close();
}
final RawReceivePort _rawPort;
final StreamController _controller;
}
Which as we can see is really just a wrapper around a RawReceivePort where the handler is a StreamController.add method. So, what about the RawReceivePort?
If we look at initial file where ReceivePort is defined we again see. It's just one external factory method and an interface for others.
abstract class RawReceivePort {
external factory RawReceivePort([Function? handler, String debugName = '']);
void set handler(Function? newHandler);
SendPort get sendPort;
}
Luckily, its #patch version can also be found in the same place as the ReceivePorts.
#patch
class RawReceivePort {
#patch
factory RawReceivePort([Function? handler, String debugName = '']) {
_RawReceivePortImpl result = new _RawReceivePortImpl(debugName);
result.handler = handler;
return result;
}
}
Ok, again the actual implementation is _RawReceivePortImpl class.
#pragma("vm:entry-point")
class _RawReceivePortImpl implements RawReceivePort {
factory _RawReceivePortImpl(String debugName) {
final port = _RawReceivePortImpl._(debugName);
_portMap[port._get_id()] = <String, dynamic>{
'port': port,
};
return port;
}
#pragma("vm:external-name", "RawReceivePortImpl_factory")
external factory _RawReceivePortImpl._(String debugName);
close() {
_portMap.remove(this._closeInternal());
}
SendPort get sendPort {
return _get_sendport();
}
bool operator ==(var other) {
return (other is _RawReceivePortImpl) &&
(this._get_id() == other._get_id());
}
int get hashCode {
return sendPort.hashCode;
}
#pragma("vm:external-name", "RawReceivePortImpl_get_id")
external int _get_id();
#pragma("vm:external-name", "RawReceivePortImpl_get_sendport")
external SendPort _get_sendport();
#pragma("vm:entry-point", "call")
static _lookupHandler(int id) {
var result = _portMap[id]?['handler'];
return result;
}
#pragma("vm:entry-point", "call")
static _lookupOpenPorts() {
return _portMap.values.map((e) => e['port']).toList();
}
#pragma("vm:entry-point", "call")
static _handleMessage(int id, var message) {
final handler = _portMap[id]?['handler'];
if (handler == null) {
return null;
}
handler(message);
_runPendingImmediateCallback();
return handler;
}
#pragma("vm:external-name", "RawReceivePortImpl_closeInternal")
external int _closeInternal();
#pragma("vm:external-name", "RawReceivePortImpl_setActive")
external _setActive(bool active);
void set handler(Function? value) {
final int id = this._get_id();
if (!_portMap.containsKey(id)) {
_portMap[id] = <String, dynamic>{
'port': this,
};
}
_portMap[id]!['handler'] = value;
}
static final _portMap = <int, Map<String, dynamic>>{};
}
OK, now we're getting somewhere. A lot is going on.
First thing to note are the: #pragma("vm:entry-point"), #pragma("vm:entry-point", "call") and #pragma("vm:external-name", "...") annotations. Docs can be found here.
Oversimplified:
vm:entry-point tells the compiler this class / method will be used from native code.
vm:external-name tells the compiler to invoke a native function which is registered to the name provided by the annotation.
For instance to know the implementation of:
#pragma("vm:external-name", "RawReceivePortImpl_factory")
external factory _RawReceivePortImpl._(String debugName);
We have to look for DEFINE_NATIVE_ENTRY(RawReceivePortImpl_factory. And we find the entry in: runtime/lib/isolate.cc.
DEFINE_NATIVE_ENTRY(RawReceivePortImpl_factory, 0, 2) {
ASSERT(TypeArguments::CheckedHandle(zone, arguments->NativeArgAt(0)).IsNull());
GET_NON_NULL_NATIVE_ARGUMENT(String, debug_name, arguments->NativeArgAt(1));
Dart_Port port_id = PortMap::CreatePort(isolate->message_handler());
return ReceivePort::New(port_id, debug_name, false /* not control port */);
}
We see the port_id is created by PortMap::CreatePort and is of type Dart_Port. Hmmm, and what is a the type definition for Dart_Port.
runtime/include/dart_api.h
typedef int64_t Dart_Port;
OK so the actual internal representation of a RawReceivePort is a signed int stored in 64 bits, and some additional information like the type, state, debug names etc.
Most of the work is then being done in PortMap::CreatePort and other of its methods. I won't go in depth, because quite honestly I don't understand everything.
But from the looks of it the PortMap uses the port_id to point to some additional information + objects. It generates it randomly and makes sure the id is not taken. It also does a lot of different things but let's move on.
When sending a message through SendPort.send, the method essentially calls the registered entry SendPortImpl_sendInternal_ which determines which port to send the information to.
Note: SendPort essentially just points to its ReceivePort and also stores the id of the Isolate where it was created. When posting a message this id is used to determine what kind of objects can be sent through.
The a message is created and passed to PortMap::PostMessage which in turn calls MessageHandler::PostMessage.
There the message is enqueued by a call to MessageQueue::Enqueue. Then a MessageHandlerTask is ran on the ThreadPool.
The MessageHandlerTask essentially just calls the MessageHandler::TaskCallback which eventually calls MessageHandler::HandleMessages.
There the MessageHandler::HandleMessage is called, but this function is implemented by a child class of MessageHandler.
Currently there are two:
IsolateMessageHandler and
NativeMessageHandler.
We are interested in the IsolateMessageHandler.
Looking there we see IsolateMessageHandler::HandleMessage eventually calls DartLibraryCalls::HandleMessage which calls object_store->handle_message_function(). full chain: Thread::Current()->isolate_group()->object_store()->handle_message_function()
The function handle_message_function is defined by the (dynamic?) macro LAZY_ISOLATE(Function, handle_message_function) in runtime/vm/object_store.h.
The property + stores created are used in: runtime/vm/object_store.cc by the: ObjectStore::LazyInitIsolateMembers.
_RawReceivePortImpl is registered to lazily load at the isolate_lib.LookupClassAllowPrivate(Symbols::_RawReceivePortImpl()) call.
As well as, the methods marked with #pragma("vm:entry-point", "call"), including static _handleMessage(int id, var message).
Which is the handler that ->handle_message_function() returns.
Later the DartLibraryCalls::HandleMessage invokes it through DartEntry::InvokeFunction with the parameters port_id and the message.
This calls the _handleMessage function which calls the registered _RawReceivePort.handler.
#2
If we compare the Flutter's compute method implementation. It spins up an Isolate and 3 ReceivePorts for every compute call. If I used compute, I would be spending more resources and loose context between multiple message calls I can have with a long-running Isolate. So for my use case I reason, creating a new ReceivePort everytime I pass a message shouldn't be a problem.
#3
I could use a different approache. But I still wish to have a long running Isolate so I have the flexibility to share context between different calls to the Isolate.
Alternative:
Would be following a bloc / stream style interface and have a method to assign a listener and a method to send or add a message event, and have the calling code listen to the responses received and act accordingly.
i.e. an interface like:
abstract class BgIsolateInterface<Message, Response> {
void send(Message message);
void addListener(void Function(Response) onData);
void removeListener(void Function(Response) onData);
}
the down side is the Message and Response have to be determined when creating the class rather than simply when using the send method like the interface in my question. Also now some other part of the code base has to handle the Response. I prefer to handle everything at the send call site.
Note: The source code of the Dart project is put here for presentation purposes. The live source may change with time. Its distribution and use are governed by their LICENSE.
Also: I'm not C/C++ developer so any interpretation of the C/C++ code may be wrong.
While this answer is long side-steps the questions a little bit, I find it useful to include the steps to search through the Dart source. Personally, I found it difficult initially to find where external functions are defined and what some of the annotation values mean. While these steps could be extracted into a separate question, I think it's useful to keep it here where there was a use case to actually dive deep.
Thank you for reading!
I'm converting dart code to nnbd.
I have the following code.
var subscription = response.listen(
(newBytes) async {
/// if we don't pause we get overlapping calls from listen
/// which causes the [writeFrom] to fail as you can't
/// do overlapping io.
subscription.pause();
/// we have new data to save.
await raf.writeFrom(newBytes);
subscription.resume();
});
The problem is I get the following error:
The non-nullable local variable 'subscription' must be assigned before it can be used.
Try giving it an initializer expression, or ensure that it's assigned on every execution path.
I've had a similar problem solved here:
dart - correct coding pattern for subscription when using null saftey?
which was answered by #lrn
However the pattern solution pattern doesn't seem to work in this case.
raf.writeFrom is an async operation so I must use an 'async' method which means I can't use the 'forEach' solution as again I don't have access to the subscription object.
If you really want to use listen, I'd do it as:
var subscription = response.listen(null);
subscription.onData((newBytes) async {
subscription.pause();
await raf.writeFrom(newBytes);
subscription.resume();
});
or, without the async:
var subscription = response.listen(null);
subscription.onData((newBytes) {
subscription.pause(raf.writeFrom(newBytes));
});
which will pause the subscription until the future returned by raf.writeFrom completes (it shouldn't complete with an error, though).
If using listen is not a priority, I'd prefer to use an asynchronous for-in like:
await for (var newBytes in subscription) {
await raf.writeFrom(newBytes);
}
which automatically pauses the implicit subscription at the await and resumes it when you get back to the loop.
Both with stream.listen and the StreamController constructor, null safety has made it nicer to create them first without callbacks, and then add the callbacks later, if the callback needs to refer to the subscription/controller.
(That's basically the same nswer as in the linked question, only applied to onData instead of onDone. You have to pass a default onData argument to listen, but it can be null precisely to support this approach.)
I don't think your code, as written, was legal before null-safety either; you can't reference a variable (subscription) before it's declared, and the declaration isn't complete until after the expression you initialize it with (response.listen(...)) is evaluated. You will need to separate the declaration from the initialization to break the circular dependency:
StreamSubscription<List<int>> subscription;
subscription = response.listen(...);
I was able to follow the instruction on adding data, that part was easy and understandable. But when I tried to follow instructions for editing data, I'm completely lost.
I am following the todo sample, which works quite well, but when I tried to add to my own project using the same principle, nothing works.
in my controller, I have the following:
function listenForPropertyChanged() {
// Listen for property change of ANY entity so we can (optionally) save
var token = dataservice.addPropertyChangeHandler(propertyChanged);
// Arrange to remove the handler when the controller is destroyed
// which won't happen in this app but would in a multi-page app
$scope.$on("$destroy", function () {
dataservice.removePropertyChangeHandler(token);
});
function propertyChanged(changeArgs) {
// propertyChanged triggers save attempt UNLESS the property is the 'Id'
// because THEN the change is actually the post-save Id-fixup
// rather than user data entry so there is actually nothing to save.
if (changeArgs.args.propertyName !== 'Id') { save(); }
}
}
The problem is that any time I change a control on the view, the propertyChanged callback function never gets called.
Here's the code from the service:
function addPropertyChangeHandler(handler) {
// Actually adds any 'entityChanged' event handler
// call handler when an entity property of any entity changes
return manager.entityChanged.subscribe(function (changeArgs) {
var action = changeArgs.entityAction;
if (action === breeze.EntityAction.PropertyChange) {
handler(changeArgs);
}
});
}
If I put a break point on the line:
var action = changeArgs.entityAction;
In my project, it never reaches there; in the todo sample, it does! It completely skips the whole thing and just loads the view afterwards. So none of my callback functions work at all; so really, nothing is subscribed.
Because of this, when I try to save changes, the manager.hasChanges() is always false and nothing happens in the database.
I've been trying for at least 3 days getting this to work, and I'm completely dumbfounded by how complicated this whole issue has been for me.
Note: I'm using JohnPapa's HotTowel template. I tried to follow the Todo editing functionality to a Tee.. and nothing is working the way I'd like it to.
Help would be appreciated.
The whole time I thought the problem was in the javascript client side end of things. Turned out that editing doesn't work when you created projected DTOs.
So in my server side, I created a query:
public IQueryable<PersonDTO> getPerson(){
return (from _person in ContextProvider.Context.Queries
select new PersonDTO
{
Id = _person.Id,
FirstName = _person.FirstName,
LastName = _person.LastName
}).AsQueryable();
}
Which just projected a DTO to send off to the client. This did work with my app in fetching data and populating things. So this is NOT wrong. Using this, I was able to add items and fetch items, but there's no information that allowed the entitymanager to know about the item. When I created an item, the entitymanager has a "createEntity" which allowed me to tell the entitymanager which item to use.. in my case:
manager.createEntity(person, initializeValues);
Maybe if there was a "manager.getEntity" maybe that would help?
Anyways, I changed the above query to get it straight from the source:
public IQueryable<Person> getPeople(){
return ContextProvider.Context.People;
}
Note ContextProvider is:
readonly EFContextProvider<PeopleEntities> ContextProvider =
new EFContextProvider<PeopleEntities>();
So the subscribe method in the javascript checks out the info that's retrieved straight from the contextual object.. interesting. Just wish I didn't spend 4 days on this.
I made a custom component which basically wraps a d3 line chart. Now I want to be able to register a callback for clicks on the lines in the chart.
I gave the component a #NgCallback parameter, which I then send events to:
class NetworkSummaryComponent implements NgShadowRootAware {
#NgCallback('callback')
Function callback;
void onShadowRoot(ShadowRoot shadowRoot) {
...
chart.callMethod('listen', ['line-click', (ev) {
var name = ev.callMethod('getLineName');
print(name);
callback({'name': name});
}]);
}
}
When using the component, I specify a function of my controller as callback:
<network-summary
...
callback="ctrl.lineClicked">
</network-summary>
However, that function is never actually called, put I know the callback arrives from the JS side because the print in the first snippet is executed.
If I instead specify the attribute as callback="ctrl.lineClicked()" I get a strange exception:
Closure call with mismatched arguments: function 'call'
I could not find any official documentation on how to properly do callbacks, so I'm not exactly sure what I'm doing wrong.. Any ideas?
It turns out that I had to explicitly name the expected arguments in the attributes:
<network-summary
...
callback="ctrl.lineClicked(name)">
</network-summary>
Hope this is useful to the next person having this problem.
I'm trying to write a library that will make it easer for dartisans to use the SoundCloud JavaScript SDK (http://developers.soundcloud.com/docs/api/sdks#javascript).
I'm using the 'dart:js' library, and
I'm only using one class to handle the proxy.
class SCproxy {
JsObject proxy = context['SC'];
String client_id;
SCproxy(this.client_id) {}
initialize() {
proxy.callMethod('initialize', [client_id]);
}
stream(String track_id){
var track = new JsObject(proxy.callMethod('stream',[track_id]));
print(track); // track should be the soundmanager2 object that we can call '.play' on.
}
The repo I'm hosting this from is (https://github.com/darkkiero/scproxy)
My problem occurs when I try to run my 'stream' method.
main() {
SCproxy SC = new SCproxy('Your SoundCloud API client_ID');
SC.initialize();
SC.stream('/tracks/111477464');
}
When I try to grab and use the soundmanager2 object returned by the javascript 'SC.stream' method, the dart editor gives me this exception :
Breaking on exception: type 'ScriptElement' is not a subtype of type 'JsFunction' of 'constructor'.
I am under the impression that I should be able to get the dart JsObject for the soundmanager2 object by collecting the callback of the 'SC.stream', But I'm not sure how.However I could be completely misusing 'dart:js' and that would be helpful information as well.
You don't seem to follow the SoundCloud JavaScript SDK documentation. Particularly for the stream method that takes a callback as parameter and doesn't return.
The following Dart code :
context['SC'].callMethod('stream', ['/tracks/293', (sound) {
sound.callMethod('play');
}]);
will do the same as this JS code :
SC.stream("/tracks/293", function(sound){
sound.play();
});
You can have a look at Using JavaScript from Dart for more explanations.