udf map function returns a table with unnamed schema - pyflink

code:
func = udf(log_parser, result_type=DataTypes.ROW(
[DataTypes.FIELD("ts", DataTypes.TIMESTAMP(precision=3)),
DataTypes.FIELD("clientip", DataTypes.STRING()),
DataTypes.FIELD("recordtime", DataTypes.STRING())]))
table = table.map(func)
table.print_schema()
output:
(
`_c0` TIMESTAMP(3),
`_c1` STRING,
`_c2` STRING
)
This looks strange to me, shouldn't it print a schema with defined column names?

This is a known issue and should have be addressed in FLINK-27282. It's fixed recently and so still not released.

Related

Dart compare two strings return false

Im new to dart and have a problem during building my Flutter application.
I have a firestore database as a backend and im getting data from there.
When i want to compare part of the data called status with the text 'CREATED', using == comparator, dart will return false.
Can someone explain why and how to check it properly?
rideObject is a Map
Update:
Here is the function that has the condition in it:
Widget _getPage() {
if (rideObject == null) {
return OrderRidePage(
address: address,
ridesReference: reference,
setRideReference: this._setRideReference);
} else {
print(rideObject['status']);
if (rideObject['status'] == "CREATED") {
return LoadingPage(
removeRideReference: this._removeRideReference,
rideReference: rideReference);
} else {
return RidePage(
address: address,
ridesReference: reference,
setRideReference: _setRideReference);
}
}
}
The print statement returns to output:
I/flutter (15469): CREATED
Here you can see the structure of the rideObject
Funnily enough, the rideObject["status"] is String type as shown in here in console:
rideObject["status"] is String
true
"CREATED" is String
true
rideObject["status"]
"CREATED"
rideObject["status"] == "CREATED"
false
The String you got from your server is probably encoded and contains special character which you can't see, try to compare the hex values of both of the strings, and then replace all the special characters from the String returned by the server.
Using this, you can see the actual non visible difference between the two strings:
var text1 = utf8.encode(hardcodedText).toString();
var text2 = utf8.encode(textFromServer).toString();
If both are really strings, you can use "compareTo" which will return 0 if both are equal.
if(str1.compareTo(str2)==0){
}
It is explained here:
https://www.tutorialspoint.com/dart_programming/dart_programming_string_compareto_method.htm
I don't have a particular solution to this, but I updated to latest Flutter version that came up today, moved the "CREATED" string into constant and resolved an unrelated warning for another part of the application, and it suddenly started to work.
The answer for this problem is in the documentation of flutter:
https://api.flutter.dev/flutter/dart-core/String/compareTo.html
you can do:
(var.compareTo('WORD') == 0)
are equivalent
.compareTo()
Returns a negative value if is ordered before, a positive value if is ordered after, or zero if and are equivalent.thisother
Building off #yonez's answer, the encoding may be different after a string has been passed through a server.
Instead of: String.fromCharCodes(data)
Try using: utf8.decode(data)

Numeric sort in Manual (Legacy) index in Neo4j 3 is not working correctly

I'm using Legacy indexing (now called Manual indexing). After migration from Neo4j 2 to version 3 I have some problems with numeric sorting.
Example of correct statement in Neo4j 2:
queryContext.sort(new Sort(new SortField(AGE, SortField.INT, false)));
This stament should be changed for Neo4j 3 (Lucene 5):
queryContext.sort(new Sort(new SortField(AGE, SortField.Type.INT, false)));
But if you use this sort statement you will get an exception:
java.lang.IllegalStateException: unexpected docvalues type SORTED_SET for field 'firstName' (expected=SORTED). Use UninvertingReader or index with docvalues.
at org.apache.lucene.index.DocValues.checkField(DocValues.java:208)
at org.apache.lucene.index.DocValues.getSorted(DocValues.java:264)
at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getSortedDocValues(FieldComparator.java:762)
at org.apache.lucene.search.FieldComparator$TermOrdValComparator.getLeafComparator(FieldComparator.java:767)
at org.apache.lucene.search.FieldValueHitQueue.getComparators(FieldValueHitQueue.java:183)
at org.apache.lucene.search.TopFieldCollector$SimpleFieldCollector.getLeafCollector(TopFieldCollector.java:164)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.replayTo(DocValuesCollector.java:297)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.getTopDocs(DocValuesCollector.java:275)
at org.neo4j.kernel.api.impl.index.collector.DocValuesCollector.getIndexHits(DocValuesCollector.java:150)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.search(LuceneLegacyIndex.java:346)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:261)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:205)
at org.neo4j.index.impl.lucene.legacy.LuceneLegacyIndex.query(LuceneLegacyIndex.java:217)
at org.neo4j.kernel.impl.api.StateHandlingStatementOperations.nodeLegacyIndexQuery(StateHandlingStatementOperations.java:1440)
at org.neo4j.kernel.impl.api.OperationsFacade.nodeLegacyIndexQuery(OperationsFacade.java:1162)
at org.neo4j.kernel.impl.coreapi.LegacyIndexProxy$Type$1.query(LegacyIndexProxy.java:83)
at org.neo4j.kernel.impl.coreapi.LegacyIndexProxy.query(LegacyIndexProxy.java:365)
I think this is caused by new added statement in Neo4j indexer class (Neo4j is indexing field for sorting automatically now?). See in:
org.neo4j.index.impl.lucene.legacy.IndexType CustomType addToDocument( Document document, String key, Object value )
new line:
document.add( instantiateSortField( key, value ) );
and method instantiateSortField is creating SortedSetDocValuesField
So I changed my code to:
queryContext.sort(new Sort(new SortedSetSortField(AGE, false)));
This runs OK but sorting is not working because numbers are sorted as string. I see that "value" parameter is String every time in method "addToDocument". I think the root cause is explained it this old comment:
see comment in class org.neo4j.index.impl.lucene.legacy.IndexType CustomType
// TODO We should honor ValueContext instead of doing value.toString() here.
// if changing it, also change #get to honor ValueContext.
Am I missing some new way how to index, search and sort data in Neo4j 3 or this is really problem that values are indexed as string in Neo4j?
Simple unit test for Neo4j 2 and Neo4j 3 can be downloaded
Solution added by MishaDemianenko at GH issue

Array.size() returned wrong values (Grails)

I'm developing an app using Grails. I want to get length of array.
I got a wrong value. Here is my code,
def Medias = params.medias
println params.medias // I got [37, 40]
println params.medias.size() // I got 7 but it should be 2
What I did wrong ?
Thanks for help.
What is params.medias (where is it being set)?
If Grials is treating it as a string, then using size() will return the length of the string, rather than an array.
Does:
println params.medias.length
also return 7?
You can check what Grails thinks an object is by using the assert keyword.
If it is indeed a string, you can try the following code to convert it into an array:
def mediasArray = Eval.me(params.medias)
println mediasArray.size()
The downside of this is that Eval presents the possibility of unwanted code execution if the params.medias is provided by an end user, or can be maliciously modified outside of your compiled code.
A good snippet on the "evil (or lack thereof) of eval" is here if you're interested (not mine):
https://javascriptweblog.wordpress.com/2010/04/19/how-evil-is-eval/
I think 7 is result of length of the string : "[37,40]"
Seems your media variable is an array not a collection
Try : params.medias.length
Thanks to everyone. I've found my mistake
First of all, I sent an array from client and my params.medias returned null,so I converted it to string but it is a wrong way.
Finally, I sent and array from client as array and in the grails, I got a params by
params."medias[]"
List medias = params.list('medias')
Documentation: http://grails.github.io/grails-doc/latest/guide/single.html#typeConverters

Attempt to index field 'wikibase' (a nil value)

I have imported to my MediaWiki site the it.Wikipedia Modulo:Bio but I get this error:
Error Lua in Module:Bio line 700: attempt to index field 'wikibase' (a nil value).
In line 700 I have this code:
local entity = mw.wikibase.getEntityObject()
I have multiple wikis that shares the same source code, and the same database, but with its own tables. Then my wikibase is mybase.mywiki.com.
I tried to solve by changing wikibase to mybase:
local entity = mw.mybase.getEntityObject()
But it doesn't work.
The problem is not wikibase: the error simply says that there is no field named wikibase in the mw table, so the problem is that mw doesn't contain what you think it should. You must find the code that puts wikibase field in mw. If it does something like mw.wikibase = something and something is nil, then it is as if that line had not executed (it is not an error to assign nil to a table field, it is like removing the field if it exists already, and doing nothing if it doesn't exist). This is common error when something is an function call, the function may return nil under some circumstances.

How to use Slickgrid Formatters with MVC

I am working on a first Slickgrid MVC application where the column definition and format is to be stored in a database. I can retrieve the list of columns quite happily and populate them until I ran into the issue with formatting of dates. No problem - for each date (or time) column I can store a formatter name in the database so this can be retrieved as well. I'm using the following code which works ok:
CLOP_ViewColumnsDataContext columnDB = new CLOP_ViewColumnsDataContext();
var results = from u in columnDB.CLOP_VIEW_COLUMNs
select u;
List<dynColumns> newColumns = new List<dynColumns>();
foreach(CLOP_VIEW_COLUMN column in results)
{
newColumns.Add(new dynColumns
{
id = column.COLUMN_NUMBER.ToString(),
name = column.HEADING.Trim(),
field = column.VIEW_FIELD.Trim(),
width = column.WIDTH,
formatter = column.FORMATTER.Trim()
});
}
var gridColumns = new JavaScriptSerializer().Serialize(newColumns);
This is all fine apart from the fomatter. An example of the variable gridColumns is:
[{"id":"1","name":"Date","field":"SCHEDULED_DATE","width":100,"formatter":"Slick.Formatters.Date"},{"id":"2","name":"Carrier","field":"CARRIER","width":50,"formatter":null}]
Which doesn't look too bad however the application the fails with the error Microsoft JScript runtime error: Function expected in the slick.grid.js script
Any help much appreciated - even if there is a better way of doing this!
You are assigning a string to the formatter property, wich is expected to be function.
Try:
window["Slick"]["Formatters"]["Date"];
But i really think you should reconsider doing it this way and instead store your values in the db and define your columns through code.
It will be easier to maintain and is less error prone.
What if you decide to use custom editors and formatters, which you later rename?
Then your code will break or you'll have to rename all entries in the db as well as in code.

Resources