How to render a soundtrack in VexFlow? - vexflow

Using Tone.js and I can play this melody:
const textMeasures = ['rest/4 B4/16 A4/16 G#4/16 A4/16',
'C5/8 rest/8 D5/16 C5/16 B4/16 C5/16',
'E5/8 rest/8 F5/16 E5/16 D#5/16 E5/16',
'B5/16 A5/16 G#5/16 A5/16 B5/16 A5/16 G#5/16 A5/16',
'C6/4 A5/8 C6/8',
'B5/8 A5/8 G5/8 A5/8',
'B5/8 A5/8 G5/8 A5/8',
'B5/8 A5/8 G5/8 F#5/8',
'E5/4'];
Now, I'd like to use VexFlow to render these notes on some staves.
Keeping in mind, the display should be fine on a mobile device, and that there could be multiple voices on the soundtrack.
For now I created a method to have a stave per measure:
private renderSoundtrackVexflow(name: string, soundtrack: Soundtrack) {
const context = this.renderVexflowContext(name, VEXFLOW_WIDTH, VEXFLOW_HEIGHT);
context.setFont('Arial', 10, '').setBackgroundFillStyle('#eed'); // TODO Hard coded values
if (soundtrack.hasTracks()) {
let staveIndex: number = 0;
const voices: Array<vexflow.Flow.Voice> = new Array<vexflow.Flow.Voice>();
for (const track of soundtrack.tracks) {
if (track.hasMeasures()) {
for (const measure of track.measures) {
if (measure.hasNotes()) {
const stave = new this.VF.Stave(0, staveIndex * (VEXFLOW_STAVE_HEIGHT + VEXFLOW_STAVE_MARGIN), VEXFLOW_WIDTH);
staveIndex++;
stave.setContext(context);
stave.addClef(VEXFLOW_CLEF);
stave.addTimeSignature(this.renderTimeSignature(measure));
stave.draw();
const notes = new Array<vexflow.Flow.StaveNote>();
const voice = new this.VF.Voice();
voice.setStrict(false);
voice.setContext(context);
voice.setStave(stave);
for (const placedNote of measure.placedNotes) {
const note: Note = placedNote.note;
notes.push(new this.VF.StaveNote({ keys: [ this.renderAbc(note) ], duration: this.renderDuration(note) }));
}
voice.addTickables(notes);
voices.push(voice);
}
}
const formatter = new this.VF.Formatter();
formatter.joinVoices(voices).format(voices, VEXFLOW_WIDTH);
for (const voice of voices) {
voice.draw();
console.log('Min voice width: ' + formatter.getMinTotalWidth(voice));
}
}
}
}
}
But the notes are displayed a bit on the staves, but some notes are displayed in between staves, on the page.

The question is a bit old, but in case you still need a solution...
The problem is that all notes have their stems oriented upwards. To fix this, add auto_stem: true and clef: "treble" to the option list passed to VF.StaveNote(), see VexFlow: Stem Direction for more information. Here is the corrected line that creates notes:
notes.push(new this.VF.StaveNote({
keys: [ this.renderAbc(note) ],
duration: this.renderDuration(note),
auto_stem: true,
clef: "treble"
}));

Related

Intersection Observer API not working in ios device when opening a webview in it but working perfectly in web, android , safari browser

`Below mentioned is my code for intersection observer
useIntersectionObserver
/* eslint-disable #typescript-eslint/no-shadow */
import { RefObject, useEffect, useState } from 'react';
export default function useIntersectionObserver(
elementRef: RefObject<Element>,
{ threshold = 0, root = null, rootMargin = '0%' }
) {
const [entry, setEntry] = useState<IntersectionObserverEntry>();
const callBackFunction = ([entry]: IntersectionObserverEntry[]): void => {
setEntry(entry);
};
useEffect(() => {
const node = elementRef?.current; // DOM Ref
const isIOSupported = !!window.IntersectionObserver;
if (!isIOSupported || !node) return;
const observerOptions = { threshold, root, rootMargin };
const observer = new IntersectionObserver(
callBackFunction,
observerOptions
);
observer.observe(node);
return () => observer.disconnect();`your text`
}, [elementRef, JSON.stringify(threshold), root, rootMargin]);
return entry;
}
Below is the code mentioned where I am calling useInterSectionObserver hook
const scrollDivElementRef = useRef<null | HTMLDivElement>(null);
const chatDivRef = useRef<null | HTMLDivElement>(null);
const entry = useIntersectionObserver(scrollDivElementRef, {});
const isVisible = !!entry?.isIntersecting;
Here scrollDivElementRef is the ref of div which we are observing for the intersection.It is basically our sentinal element.
Below mentioned is the useEffect hook which is going to perform some action when isVisible will become true.
Below mentioned is a code in react-native webview ,basically we are opening our react web app inside ios app , but our intersection observer is not able to detect changes . We have implemented intersection observer for infinite loading of messages. Every time user will scroll upwards , we will get "isVisible" as true and we will make an API call to fetch more messages.
useEffect(() => {
if (isVisible) {
dispatch(
getPostsForChannel(inputChannelId, true, channelMessages[0].timestamp)
);
}
}, [isVisible]);
<View style={styles.container}>
<WebView
style={{ marginTop: 40 }}
//TODO: Update the url
source={{ uri: 'http://192.168.1.2:3000' }}
onLoad={(syntheticEvent) => {
const { nativeEvent } = syntheticEvent;
}}
javaScriptEnabled={true}
onError={(e) => {
const { nativeEvent } = e;
setUrl(HOME_URL);
}}
onHttpError={() => {
setUrl(HOME_URL);
}}
onMessage={onMessage}
limitsNavigationsToAppBoundDomains={true}
// injectedJavaScript="window.octNativeApp=true"
injectedJavaScriptBeforeContentLoaded={initializeNativeApp}
scalesPageToFit={false}
setBuiltInZoomControls={false}
useWebView2
allowsInlineMediaPlayback={true}
mediaPlaybackRequiresUserAction={false}
></WebView>
</View>`
It will be really very helpful , if you people can help me with this problem.
Thanks & Regards
Mohit
I tried giving the height & width to the webview but it is also not working.Tried almost every piece of advise available in other platforms but not able to fic this`

How do I find a Mattertag sid?

I'm trying to inject some html to a Mattertag as shown in this example here:
http://jsfiddle.net/guillermo_matterport/njhm5aos/
const postMessage = await sdk.Mattertag.injectHTML(tagByTV.sid, htmlString, {
size: {
w: 400,
h: 50,
},
})
However I don't know the sid of the Mattertag, how can I find out?
To get a Mattertag sid, you can set a listener to print out the sid on each click:
sdk.on(sdk.Mattertag.Event.CLICK,
function (tagSid) {
console.log(tagSid + ' was selected');
}
);
Alternatively, you can base the injection on the data from the tags. You'd first need to get the Mattertag data from the space.
Using the Mattertag.data observable map:
const tagData = [];
sdk.Mattertag.data.subscribe({
onAdded: (idx, item) => {
tagData.push(item);
}
});
You can read more about observable maps and when methods are fired here.
You can then identify the Mattertag to inject the HTML, whether it's the title, color, or description. You can also incorporate the injection with the observable map. For example, based on the title:
const tagData = [];
const postMessages = {}; // save postMessage for multiple Mattertag injection references
sdk.Mattertag.data.subscribe({
onAdded: async (idx, item) => {
tagData.push(item);
if(item.label.includes("TV")){
const htmlString = makeHtmlString(item.sid);
const postMessage = await sdk.Mattertag.injectHTML(item.sid, htmlString, {
size: {
w: 400,
h: 50,
},
});
postMessages[item.sid] = postMessage;
}
}
});
See the modified fiddle here.

Basic parsing Flow with Babel not working

I cannot seem to be able to parse Flow using Babel, its not recognising types or "declare" and is coming up with errors on them :-
const fs = require("fs");
const babel = require("#babel/core");
const parser = require('#babel/parser');
const generate = require('#babel/generator').default;
if (process.argv.length == 3) {
const filename = process.argv[2];
const sourceCode = fs.readFileSync(filename).toString();
console.log("sourceCode = ", sourceCode);
var options = {
"sourceType": "module", // parse in strict mode and allow module declarations
"presets": ["#babel/preset-flow"]
};
const parsedAst = parser.parse(sourceCode, options);
console.log("parsedAst = ", parsedAst)
const { codeOutput, map, ast } = babel.transformFromAstSync(parsedAst, sourceCode, { ast: true } );
console.log("ast = ", JSON.stringify(ast, 2, 2))
const output = generate(ast, { /* options */ }, sourceCode);
console.log("codeOutput = ", codeOutput);
console.log("output = ", output);
};
given the following code :-
// #flow strict
const MAX_SUGGESTIONS = 5;
/**
* Given [ A, B, C ] return ' Did you mean A, B, or C?'.
*/
declare function didYouMean(suggestions: $ReadOnlyArray<string>): string;
// eslint-disable-next-line no-redeclare
declare function didYouMean(
subMessage: string,
suggestions: $ReadOnlyArray<string>,
): string;
// eslint-disable-next-line no-redeclare
export default function didYouMean(firstArg, secondArg?) { ... }
I am getting errors on declare also on types :-
C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:730
throw err;
^
SyntaxError: Unexpected token, expected ";" (8:8)
at Parser._raise (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:723:17)
at Parser.raiseWithData (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:716:17)
at Parser.raise (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:710:17)
at Parser.unexpected (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:8610:16)
at Parser.semicolon (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:8592:40)
at Parser.parseExpressionStatement (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11449:10)
at Parser.parseStatementContent (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11050:19)
at Parser.parseStatement (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:10916:17)
at Parser.parseBlockOrModuleBlockBody (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11490:25)
at Parser.parseBlockBody (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11477:10) {
loc: Position { line: 8, column: 8 },
pos: 118
}
Github :-
https://github.com/AaronNGray/babel-flow-parser-test
Tag v0 is without package-lock.json
Need to add some extra details which I cannot think of ???
The following works :-
const fs = require('fs');
const babel = require('#babel/core');
if (process.argv.length == 3) {
const filename = process.argv[2];
const source = fs.readFileSync(filename).toString();
const ast = babel.parseSync(source, {
babelrc: false,
configFile: false,
ast: true,
parserOpts: {
plugins: ['flow', 'jsx'],
},
filename,
});
console.log("ast = ", JSON.stringify(ast, null, 2))
}
I would still like to know if the original will work with modification as its the near enough what is specified in the Babel documentation.

Is it possible to do voice pitch shifting in Twilio group video?

We have built a web application. The application's core is to arrange the meetings/sessions on the web. So User A(Meeting co-ordinator) will arrange a meeting/session and all other participants B, C, D and etc will be joining in the meeting/session. So I have used Twilio group video call to achieve it.
I have the below use case.
We want to do the voice pitch shifting of the User A's(Meeting co-ordinator) voice. So all other participants will be receiving the pitch-shifted voice in group video. We have analyzed the AWS Polly in Twilio but it doesn’t match with our use case.
So please advice is there any services in Twilio to achieve this scenario.
(or)
will it be possible to interrupt Twilio group call and pass the pitch-shifted voice to other participants?
Sample Code Used
initAudio();
function initAudio() {
analyser1 = audioContext.createAnalyser();
analyser1.fftSize = 1024;
analyser2 = audioContext.createAnalyser();
analyser2.fftSize = 1024;
if (!navigator.getUserMedia)
navigator.getUserMedia = navigator.webkitGetUserMedia || navigator.mozGetUserMedia;
if (!navigator.getUserMedia)
return(alert("Error: getUserMedia not supported!"));
navigator.getUserMedia({ audio: true }, function(stream){
gotStream(stream);
}, function(){ console.log('Error getting Microphone stream'); });
if ((typeof MediaStreamTrack === 'undefined')||(!MediaStreamTrack.getSources)){
console.log("This browser does not support MediaStreamTrack, so doesn't support selecting sources.\n\nTry Chrome Canary.");
} else {
MediaStreamTrack.getSources(gotSources);
}
}
function gotStream (stream) {
audioInput = audioContext.createMediaStreamSource(stream);
outputMix = audioContext.createGain();
dryGain = audioContext.createGain();
wetGain = audioContext.createGain();
effectInput = audioContext.createGain();
audioInput.connect(dryGain);
audioInput.connect(effectInput);
dryGain.connect(outputMix);
wetGain.connect(outputMix);
audioOutput = audioContext.createMediaStreamDestination();
outputMix.connect(audioOutput);
outputMix.connect(analyser2);
crossfade(1.0);
changeEffect();
}
function crossfade (value) {
var gain1 = Math.cos(value * 0.5 * Math.PI);
var gain2 = Math.cos((1.0 - value) * 0.5 * Math.PI);
dryGain.gain.value = gain1;
wetGain.gain.value = gain2;
}
function createPitchShifter () {
effect = new Jungle( audioContext );
effect.output.connect( wetGain );
effect.setPitchOffset(1);
return effect.input;
}
function changeEffect () {
if (currentEffectNode)
currentEffectNode.disconnect();
if (effectInput)
effectInput.disconnect();
var effect = 'pitch';
switch (effect) {
case 'pitch':
currentEffectNode = createPitchShifter();
break;
}
audioInput.connect(currentEffectNode);
}
Facing the error while adding the Localaudiotrack to a room
var mediaStream = new Twilio.Video.LocalAudioTrack(audioOutput.stream);
room.localParticipant.publishTrack(mediaStream, {
name: 'adminaudio'
});
ERROR:
Uncaught (in promise) TypeError: Failed to execute 'addTrack' on 'MediaStream': parameter 1 is not of type 'MediaStreamTrack'.
Twilio developer evangelist here.
There is nothing within Twilio itself that pitch shifts voices.
If you are building this in a browser, then you could use the Web Audio API to take the input from the user's microphone and pitch shift it, then provide the resultant audio stream to the Video API instead of the original mic stream.
the comments in the above answer are SO helpful! I've been researching this for a couple of weeks, posted to Twilio-video.js to no avail and finally just the right phrasing pulled this up on S.O!
but to summarize and to add what I've found to work since it's hard to follow all the 27 questions/comments/code excerpts:
when connecting to Twilio:
const room = await Video.connect(twilioToken, {
name: roomName,
tracks: localTracks,
audio: false, // if you don't want to hear the normal voice at all, you can hide this and add the shifted track upon participant connections
video: true,
logLevel: "debug",
}).then((room) => {
return room;
});
upon a new (remote) participant connection:
const stream = new MediaStream([audioTrack.mediaStreamTrack]);
const audioContext = new AudioContext();
const audioInput = audioContext.createMediaStreamSource(stream);
source.disconnect(audioOutput);
console.log("using PitchShift.js");
var pitchShift = PitchShift(audioContext);
if (isFinite(pitchVal)) {
pitchShift.transpose = pitchVal;
console.log("gain is " + pitchVal);
}
pitchShift.wet.value = 1;
pitchShift.dry.value = 0.5;
try {
audioOutput.stream.getAudioTracks()[0]?.applyConstraints({
echoCancellation: true,
noiseSuppression: true,
});
} catch (e) {
console.log("tried to constrain audio track " + e);
}
var biquadFilter = audioContext.createBiquadFilter();
// Create a compressor node
var compressor = audioContext.createDynamicsCompressor();
compressor.threshold.setValueAtTime(-50, audioContext.currentTime);
compressor.knee.setValueAtTime(40, audioContext.currentTime);
compressor.ratio.setValueAtTime(12, audioContext.currentTime);
compressor.attack.setValueAtTime(0, audioContext.currentTime);
compressor.release.setValueAtTime(0.25, audioContext.currentTime);
//biquadFilter.type = "lowpass";
if (isFinite(freqVal)) {
biquadFilter.frequency.value = freqVal;
console.log("gain is " + freqVal);
}
if (isFinite(gainVal)) {
biquadFilter.gain.value = gainVal;
console.log("gain is " + gainVal);
}
source.connect(compressor);
compressor.connect(biquadFilter);
biquadFilter.connect(pitchShift);
pitchShift.connect(audioOutput);
const localAudioWarpedTracks = new Video.LocalAudioTrack(audioOutput.stream.getAudioTracks()[0]);
const audioElement2 = document.createElement("audio");
document.getElementById("audio_div").appendChild(audioElement2);
localAudioWarpedTracks.attach();

Firefox Addon SDK - How to create an about: page

I need to create an about: page, to display addon options. I have seen ti done before, but there seems to be no option in the SDK that allows you to do that.
Is there another way I could let users type about:pagename and get to my page?
I would prefer not to redirect all tabs with a URL of about:pagename to another options page.
Thanks in advance
This is the index.js file for a restartless add-on developed using jpm:
const { Cc, Ci, Cr, Cu, Cm, components } = require("chrome");
Cm.QueryInterface(Ci.nsIComponentRegistrar);
Cu.import("resource://gre/modules/XPCOMUtils.jsm");
Cu.import("resource://gre/modules/Services.jsm");
// globals
var factory;
const aboutPage_description = 'This is my custom about page';
const aboutPage_id = '6c098a80-9e13-11e5-a837-0800200c9a66'; // make sure you generate a unique id from https://www.famkruithof.net/uuid/uuidgen
const aboutPage_word = 'foobar';
const aboutPage_page = Services.io.newChannel('data:text/html,hi this is the page that is shown when navigate to about:foobar', null, null);
function AboutCustom() {};
AboutCustom.prototype = Object.freeze({
classDescription: aboutPage_description,
contractID: '#mozilla.org/network/protocol/about;1?what=' + aboutPage_word,
classID: components.ID('{' + aboutPage_id + '}'),
QueryInterface: XPCOMUtils.generateQI([Ci.nsIAboutModule]),
getURIFlags: function(aURI) {
return Ci.nsIAboutModule.ALLOW_SCRIPT;
},
newChannel: function(aURI) {
let channel = aboutPage_page;
channel.originalURI = aURI;
return channel;
}
});
function Factory(component) {
this.createInstance = function(outer, iid) {
if (outer) {
throw Cr.NS_ERROR_NO_AGGREGATION;
}
return new component();
};
this.register = function() {
Cm.registerFactory(component.prototype.classID, component.prototype.classDescription, component.prototype.contractID, this);
};
this.unregister = function() {
Cm.unregisterFactory(component.prototype.classID, this);
}
Object.freeze(this);
this.register();
}
exports.main = function() {
factory = new Factory(AboutCustom);
};
exports.onUnload = function(reason) {
factory.unregister();
};
Basically it registers a custom about page that will be loaded when you access about:foobar. The loaded page is just a line of text.
This is how it looks like:
You can see a working example here: https://github.com/matagus/about-foobar-addon
I think this is a better solution if you are using the addons-sdk:
Credit goes here:
https://stackoverflow.com/a/9196046/1038866
var pageMod = require("page-mod");
pageMod.PageMod({
include: data.url("options.html"),
...
});
var tabs = require("tabs");
tabs.open(data.url("options.html"));
But there are other ways. You could take a look at the Scroll to Top addon which implements this: https://addons.mozilla.org/firefox/addon/402816

Resources