How to prevent babel.js from encoding string to unicode? - character-encoding

I have a simple function:
const showUTF8Text = text => `Funkcja wyświetla tekst: ${text}`;
and after transpile it with babel I receive:
var showUTF8Text = function showUTF8Text(text) {
return "Funkcja wy\uFFFDwietla tekst: " + text;
};
how to prevent babel from encoding to keep the original string?

As suggested here: https://github.com/babel/babel/issues/9804#issuecomment-480075309
Add this to your babel config:
"generatorOpts": {
"jsescOption": {
"minimal": true
}
}
More info about this jsesc option on their README: https://github.com/mathiasbynens/jsesc#minimal

Related

protobuf post rpc method with string formatting

I have this rpc in my .proto file:
message SetFlagRequest {
string name = 1;
bool enable = 2;
}
rpc SetFeatureFlag(SetFlagRequest) returns (FlagStatus) {
option (google.api.http) = {
post: "/v2/flags/{name}/{enable}"
body: "*"
};
//...
I would like to have post address of:
"/v2/flags/{name}/enable" (for enable = true) instead of "/v2/flags/{name}/true"
"/v2/flags/{name}/disable" (for enable = false) instead of "/v2/flags/{name}/false"
is there a way of string formatting in .proto files?
Nope, standard practice is to make them explicit.
message SetFlagRequest {
string name = 1;
}
rpc EnableFeatureFlag(SetFlagRequest) returns (FlagStatus) {
option (google.api.http) = {
post: "/v2/flags/{name}/enable"
body: "*"
};
rpc DisableFeatureFlag(SetFlagRequest) returns (FlagStatus) {
option (google.api.http) = {
post: "/v2/flags/{name}/disable"
body: "*"
};
Then in your implementation of each you can call a common method if you like.

Basic parsing Flow with Babel not working

I cannot seem to be able to parse Flow using Babel, its not recognising types or "declare" and is coming up with errors on them :-
const fs = require("fs");
const babel = require("#babel/core");
const parser = require('#babel/parser');
const generate = require('#babel/generator').default;
if (process.argv.length == 3) {
const filename = process.argv[2];
const sourceCode = fs.readFileSync(filename).toString();
console.log("sourceCode = ", sourceCode);
var options = {
"sourceType": "module", // parse in strict mode and allow module declarations
"presets": ["#babel/preset-flow"]
};
const parsedAst = parser.parse(sourceCode, options);
console.log("parsedAst = ", parsedAst)
const { codeOutput, map, ast } = babel.transformFromAstSync(parsedAst, sourceCode, { ast: true } );
console.log("ast = ", JSON.stringify(ast, 2, 2))
const output = generate(ast, { /* options */ }, sourceCode);
console.log("codeOutput = ", codeOutput);
console.log("output = ", output);
};
given the following code :-
// #flow strict
const MAX_SUGGESTIONS = 5;
/**
* Given [ A, B, C ] return ' Did you mean A, B, or C?'.
*/
declare function didYouMean(suggestions: $ReadOnlyArray<string>): string;
// eslint-disable-next-line no-redeclare
declare function didYouMean(
subMessage: string,
suggestions: $ReadOnlyArray<string>,
): string;
// eslint-disable-next-line no-redeclare
export default function didYouMean(firstArg, secondArg?) { ... }
I am getting errors on declare also on types :-
C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:730
throw err;
^
SyntaxError: Unexpected token, expected ";" (8:8)
at Parser._raise (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:723:17)
at Parser.raiseWithData (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:716:17)
at Parser.raise (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:710:17)
at Parser.unexpected (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:8610:16)
at Parser.semicolon (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:8592:40)
at Parser.parseExpressionStatement (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11449:10)
at Parser.parseStatementContent (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11050:19)
at Parser.parseStatement (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:10916:17)
at Parser.parseBlockOrModuleBlockBody (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11490:25)
at Parser.parseBlockBody (C:\Users\aaron\Tests\Babel\generator-test\node_modules\#babel\parser\lib\index.js:11477:10) {
loc: Position { line: 8, column: 8 },
pos: 118
}
Github :-
https://github.com/AaronNGray/babel-flow-parser-test
Tag v0 is without package-lock.json
Need to add some extra details which I cannot think of ???
The following works :-
const fs = require('fs');
const babel = require('#babel/core');
if (process.argv.length == 3) {
const filename = process.argv[2];
const source = fs.readFileSync(filename).toString();
const ast = babel.parseSync(source, {
babelrc: false,
configFile: false,
ast: true,
parserOpts: {
plugins: ['flow', 'jsx'],
},
filename,
});
console.log("ast = ", JSON.stringify(ast, null, 2))
}
I would still like to know if the original will work with modification as its the near enough what is specified in the Babel documentation.

How to delete/detect any emoji as a whole from a string in Flutter?

I would like to simulate a keyboard backspace delete event from a string in Flutter (or Dart). Something like:
String str = "hello🇵🇬你们😀😀👨‍👩‍👦"
myBackspace(str) // will return "hello🇵🇬你们😀😀"
myBackspace(str) // will return "hello🇵🇬你们😀"
myBackspace(str) // will return "hello🇵🇬你们"
myBackspace(str) // will return "hello🇵🇬你"
myBackspace(str) // will return "hello🇵🇬"
myBackspace(str) // will return "hello"
myBackspace(str) // will return "hell"
Update
Dart team released a helper package that helps achieving this. String.characters.skipLast(1) should be able to do what you expect.
Old answer
First, let's get to some definitions. According to this page:
Bytes: 8-bit. The number of bytes that a Unicode string will take up in memory or storage depends on the encoding.
Code Units: The smallest bit combination that can be used to express a single unit in text encoding. For example 1 code unit in UTF-8 would be 1 byte, 2 bytes in UTF-16, 4 bytes in UTF-32.
Code Points [or rune]: Unicode character. A single integer value (from U+0000-U+10FFFF) on a Unicode space.
Grapheme clusters: A single character perceived by the user. 1 grapheme cluster consists of several code points.
When you remove the last char using substring, you're actually removing the last code unit. If you run print(newStr.codeUnits) and print(str.codeUnits), you'll see that the rune 128512 is equivalent to the joint of the code units 55357 and 56832, so 55357 is actually valid, but doesn't represent anything without the "help" of another code unit.
In fact, you don't want to use substring() when there's non-ASCII chars in your String (like emojis or arabic letters). It'll never work. What you have to do is remove the last grapheme cluster. Something as simple as that:
str.graphemeClusters.removeLast()
However, Dart doesn't support this yet. There are several issues around this point. Some of those:
https://github.com/dart-lang/language/issues/34
https://github.com/dart-lang/language/issues/49
This lack of support seams to result in some other of issues, like the one you mentioned and this one:
https://github.com/flutter/flutter/issues/31818
String formatText(String str) {
final RegExp regExp = RegExp(r'(?:[\u2700-\u27bf]|(?:\ud83c[\udde6-\uddff]){2}|[\ud800-\udbff][\udc00-\udfff]|[\u0023-\u0039]\ufe0f?\u20e3|\u3299|\u3297|\u303d|\u3030|\u24c2|\ud83c[\udd70-\udd71]|\ud83c[\udd7e-\udd7f]|\ud83c\udd8e|\ud83c[\udd91-\udd9a]|\ud83c[\udde6-\uddff]|\ud83c[\ude01-\ude02]|\ud83c\ude1a|\ud83c\ude2f|\ud83c[\ude32-\ude3a]|\ud83c[\ude50-\ude51]|\u203c|\u2049|[\u25aa-\u25ab]|\u25b6|\u25c0|[\u25fb-\u25fe]|\u00a9|\u00ae|\u2122|\u2139|\ud83c\udc04|[\u2600-\u26FF]|\u2b05|\u2b06|\u2b07|\u2b1b|\u2b1c|\u2b50|\u2b55|\u231a|\u231b|\u2328|\u23cf|[\u23e9-\u23f3]|[\u23f8-\u23fa]|\ud83c\udccf|\u2934|\u2935|[\u2190-\u21ff])');
if(str.contains(regExp)){
str = str.replaceAll(regExp,'');
}
return str; }
Ex: Go to https://dartpad.dev/ to test:
String str = "ThaiKV受け行くけどよね😞😞😍😰😒😜" => ThaiKV受け行くけどよね
This answer still has problem
Since dart does not provide the data type 'Grapheme Cluster', I try to use method channel to do this using kotlin:
Step 1: Create a new 'Flutter Plugin' project, name the project 'gmc01', 2 files will be created automatically, namely 'gmc01.dart' and 'main.dart'.
Step 2: replace the codes in gmc01.dart with the following:
import 'dart:async';
import 'package:flutter/services.dart';
class Gmc01 {
static const MethodChannel _channel =
const MethodChannel('gmc01');
static Future<String> removeLastGMC(String strOriginal) async {
final String version = await _channel.invokeMethod('removeLastGMC', strOriginal);
return version;
}
}
Step 3: Replace the codes in main.dart with the following:
import 'package:gmc01/gmc01.dart';
void main() async {
String strTemp = '12345678我们5🇵🇬你😀👨‍👩‍👦';
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
strTemp = await Gmc01.removeLastGMC(strTemp);
print(strTemp);
}
Step 4: Inside android/build.gradle, change the minSdkVersion from 16 to 24.
Step 5: Inside example/android/app/build.gradle, change the minSdkVersion from 16 to 24.
Step 6: Click File->Open, select gmc01->android, then click 'OK', the kotlin part of the plugin will be opened (In another Window).
Step 7: Replace the codes in Gmc01Plugin.kt with the following: (Replace the first line with your own package name)
package com.example.gmc01 // replace the left with your own package name
import io.flutter.plugin.common.MethodCall
import io.flutter.plugin.common.MethodChannel
import io.flutter.plugin.common.MethodChannel.MethodCallHandler
import io.flutter.plugin.common.MethodChannel.Result
import io.flutter.plugin.common.PluginRegistry.Registrar
import android.icu.text.BreakIterator
class Gmc01Plugin: MethodCallHandler {
companion object {
#JvmStatic
fun registerWith(registrar: Registrar) {
val channel = MethodChannel(registrar.messenger(), gmc01)
channel.setMethodCallHandler(Gmc01Plugin())
}
}
override fun onMethodCall(call: MethodCall, result: Result) {
var strArg: String
strArg = call.arguments.toString()
var boundary = BreakIterator.getWordInstance()
boundary.setText(strArg);
when (call.method) {
removeLastGMC -> {
result.success(removeLastGMC(boundary, strArg))
}
else -> {
result.notImplemented()
}
}
}
fun removeLastGMC(boundary: BreakIterator, source: String):String {
val end = boundary.last()
val start = boundary.previous()
return source.substring(0, start)
}
}
Step 8: Go back to the window of the plugin, and click 'Run'.
Here are the output in the console:
I/flutter (22855): 12345678我们5🇵🇬你😀
I/flutter (22855): 12345678我们5🇵🇬你
I/flutter (22855): 12345678我们5🇵🇬
I/flutter (22855): 12345678我们5
I/flutter (22855): 12345678我们
I/flutter (22855): 12345678
I/flutter (22855):
As you can see, the 'Family Emoji', 'Face Emoji' and 'Country Flag' emoji are removed correctly, but the Chinese 2 chars '我们' and the digits '12345678' are removed by using a single removeLastGMC, so still need to figure out how to distinguish Chinese Double Bytes characters / English Chars / Emojis.
BTW, I don't know how to do the Swift part, can someone help?
Its a bit unclear to what you want to check. I suggest you remove the -1 from the substring because it will break the emoji's code snip
void main() {
var str = "abc😀";
var newStr = str.substring(0, str.length); // i removed it here
print(newStr);
print(newStr.runes);
print(str.runes);
}
This will give the output of
abc😀
(97, 98, 99, 128512)
(97, 98, 99, 128512)
Tested in https://dartpad.dartlang.org/
The code is not working
The code is not working properly. I just put here for reference.
Trial 1
Problem: can not handle 🇵🇬 and 👨‍👩‍👦 properly.
String myBackspace(String str) {
Runes strRunes = str.runes;
str = String.fromCharCodes(strRunes, 0, strRunes.length - 1);
print(str);
return str;
}
Trial 2
Problem: can not handle connected emoji sequence 😀😀 and 👨‍👩‍👦 properly.
Based on the link
String myBackspace(String str) {
int i = 0;
while (str.length > 0) {
i++;
int removedCharCode = str.codeUnitAt(str.length - 1);
if (isWellFormattedUTF16(removedCharCode)) break;
str = str.substring(0, str.length - 1);
}
if (i == 1) str = str.substring(0, str.length - 1);
print(str);
return str;
}
bool isWellFormattedUTF16(int charCode) {
int surrogateLeadingStart = 0xD800;
int surrogateLeadingEnd = 0xDBFF;
int surrogateTrailingStart = 0xDC00;
int surrogateTrailingEnd = 0xDFFF;
if (!(charCode >= surrogateLeadingStart && charCode <= surrogateLeadingEnd) &&
!(charCode >= surrogateTrailingStart && charCode <= surrogateTrailingEnd)) return true;
return false;
}
if someone need simple solution to remove emojies from string try this.
String str = "hello🇵🇬你们😀😀👨‍👩‍👦"İ
final RegExp REGEX_EMOJI = RegExp(r'(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-\udfff]|\ud83e[\ud000-\udfff])');
if(str.contains(REGEX_EMOJI)){
str = str.replaceAll(REGEX_EMOJI,'');
}
With RegExp and replaceAll:
final regex = RegExp(
"(\u00a9|\u00ae|[\u2000-\u3300]|\ud83c[\ud000-\udfff]|\ud83d[\ud000-
\udfff]|\ud83e[\ud000-\udfff])");
final textReplace = String.replaceAll(regex, '');
You can do a method like this one
bool isValid(String prevString, String newString){
if (prevString == newString)
return true;
else return false;
}
then in your keyboard you validate with an onChange property
TextField(
onChanged: (text) {
isValid(varYouHad ,text); //validate
},
);

AngularJS: Creating a directive for converting a string to a Date on the fly

Angular UI Bootstrap changed the way of what the datepicker expects as ng-model in some version after 1.13.0. Before it was fine to give it an ISO date string, now it wants a Date object.
I consume ISO date strings from my API though, so I have to
convert them into Date objects before giving it to the datepicker and
convert them back to an ISO date string when storing it.
In the past I used a directive like this:
function DateObjectDirective() {
const directive = {
restrict: "A",
require: ["ngModel"],
link(scope, element, attributes, controllers) {
const ngModel = controllers[0];
ngModel.$formatters.unshift(value => {
let output = null;
if(value) {
output = moment(value).toDate();
}
return output;
});
ngModel.$parsers.unshift(value => {
let output = null;
if(value) {
output = moment(value).format();
}
return output;
});
},
};
return directive;
}
This no longer works though, as the following error is reported:
this.activeDate.getFullYear is not a function
My guess is that the datepicker still uses the string as reference. Is there any other way I can convert before giving my data to the datepicker?
I found out that the directive I posted does indeed still work. The only problem was the order in which AngularJS evaluated the directives.
For example:
<input ng-model="someDateString" uib-datepicker-popup="yyyy-MM-dd" woo-date-object>
In my case, woo-date-object was always evaluated before uib-datepicker-popup. The result was that the datepicker has always pushed its own formatter on top of ngModel.$formatters, thus eliminating the possibility for me to intervene.
The solution is to give the own directive a higher priority. UI's datepicker doesn't have one set, so anything above 0 (which is the default) works:
{
restrict: "A",
require: "ngModel",
priority: 9999,
link(scope, element, attributes, ngModel) {
ngModel.$formatters.push(value => {
let output = new Date();
if(value) { output = moment(value).toDate(); }
return output;
});
ngModel.$parsers.push(value => {
let output = null;
if(value) { output = moment(value).format(); }
return output;
});
},
}

Error Breeze OData - Metadata query failed for http://localhost:5781/odata/$metadata

I researched questions on forum but not find true result.
Error:
Metadata query failed for //localhost:5781/odata/$metadata; Unable to process returned
metadata: NamingConvention for this server property name does not roundtrip
properly:diagram_id-->Diagram_id Error: Metadata query failed for //localhost:5781/odata/$metadata; Unable to process returned metadata: NamingConvention for this server property name does not roundtrip properly:diagram_id
Code
(function () {
'use strict';
var serviceId = 'entityManagerFactory';
angular.module('myApp')
.factory(serviceId, ['breeze', emFactory]);
function emFactory(breeze) {
configureBreeze();
var serviceRoot = window.location.protocol + '//' + window.location.host + '/';
var serviceName = serviceRoot + 'odata/';
var factory = {
newManager: newManager,
serviceName: serviceName
};
return factory;
function configureBreeze() {
// use Web API OData to query and save
breeze.config.initializeAdapterInstance('dataService', 'webApiOData', true);
// convert between server-side PascalCase and client-side camelCase
breeze.NamingConvention.camelCase.setAsDefault();
}
function newManager() {
var mgr = new breeze.EntityManager(serviceName);
return mgr;
}
}})();
Code other :
(function () {
'use strict';
var serviceId = 'datacontext';
angular.module('myApp')
.factory(serviceId, ['$q', 'logger', 'entityManagerFactory', datacontext]);
function datacontext($q,logger,emFactory) {
logger = logger.forSource(serviceId);
var logError = logger.logError;
var logSuccess = logger.logSuccess;
var logWarning = logger.logWarning;
var manager = emFactory.newManager();
var service = {
getEmployees: getEmployees
};
return service;
/*Hiện thực ở đây*/
function getChangesCount(){
return manager.getChanges().length;
}
function getEmployees(forceRefresh) {
var count;
if (forceRefresh) {
if(manager.hasChanges()){
count = getChangesCount();
manager.rejectChanges();//undo tất cả các thay đổi ko được lưu
logWarning('Số nhân viên' + count + 'bị thay đổi', null, true);
}
}
// Lúc ko có forceRefesh,xem xét nhận bộ nhớ cache hơn từ xa
return breeze.EntityQuery.from('Employees')
.using(manager).execute()
.then(success).catch(failed);
function success(response) {
count = response.results.length;
logSuccess('Đã nhận ' + count + ' nhân viên', response, true);
return response.results;
}
function failed(error) {
var message = error.message || "Truy vấn để bảng nhân viên bị lỗi";
logError(message, error, true);
}
}
}})();
Code other :
(function () {
'use strict';
var controllerId = 'employees';
angular.module('myApp')
.controller(controllerId, ['datacontext', 'logger', employees]);
function employees(datacontext, logger) {
logger = logger.forSource(controllerId);
var logError = logger.logError;
var logSuccess = logger.logSuccess;
var vm = this;
vm.employees = [];
initialize();
/*Hiện thực*/
function initialize() {
getEmployees();
}
function getEmployees(forceRefresh) {
return datacontext.getEmployees(forceRefresh).then(function (data) {
return vm.employees = data;
console.log(data);
});
}
}}());
This problem very likely has to do with the camelCase naming convention and the language and/or property names that you are using. My guess is that if you remove the line that sets camelCase as the default then the error will go away. If so, then you will need to write your own custom naming convention. See http://www.breezejs.com/documentation/naming-convention
The reason that this is occurring, (I'm guessing here), is that the camelCase naming convention is very simplistic and may not work for your property names and/or language. It assumes that all server property names begin with an uppercase character and that this character can be converted to a lowercase character, and further that this process can be reversed. My guess is that one of your property names already has a first character that is lower case or that calling toLower/toUpper on some first character in a property name does not itself roundtrip. (This can occur is some non-latin character sets).
If either of these cases is occuring, its actually rather easy to create your own namingConvention to use instead of 'camelCase'. Again see the docs mentioned above.

Resources