Using Pig .12.
I want to use a FOREACH with a block statement; the result, via GENERATE depends on some value. I know that i can use Flatten with a statement in it. I.e.:
grunt> d = foreach j2 {
ord = order j1 by A::a1 desc;
l = limit ord 1;
generate flatten((IsEmpty(l)?'f':'g')); };
This works.
But what i want to do:
grunt> d = foreach j2 {
ord = order j1 by A::a1 desc;
l = limit ord 1;
generate flatten((l.$0=='2') ?'f':'g'); };
2014-05-07 22:28:08,750 [main] ERROR org.apache.pig.tools.grunt.Grunt
- ERROR 1200: mismatched input '?' expecting RIGHT_PAREN
Why does it say this?
I tried:
grunt> d = foreach j2 {
ord = order j1 by A::a1 desc;
l = limit ord 1;
generate flatten(((l.$0=='2') ?'f':'g')); };
2014-05-07 22:40:44,895 [main] ERROR
org.apache.pig.tools.grunt.Grunt - ERROR 1039:
(Name: Equal Type: null Uid: null)incompatible types in Equal Operator left hand side:bag :tuple(A::a1:chararray) right hand side:chararray
And i dont know how to resolve this.
I just want to generate my results in a test case, but first need to flatten the test condition.
Help ?
thanks,
Matt
Related
To create a trigger before insert using Informix database.
When we try to insert a record into the table it should insert random alphanumeric string into one of the field. Are there any built in functions?
The table consists of the following fields:
empid serial NOT NULL
age int
empcode varchar(10)
and I am running
insert into employee(age) values(10);
The expected output should be something as below:
id age empcode
1, 10, asf123*
Any help is appreciated.
As already commented there is no existing function to create a random string however it is possible to generate random numbers and then convert these to characters. To create the random numbers you can either create a UDR wrapper to a C function such as random() or register the excompat datablade and use the dbms_random_random() function.
Here is an example of a user-defined function that uses the dbs_random_random() function to generate a string of ASCII alphanumeric characters:
create function random_string()
returning varchar(10)
define s varchar(10);
define i, n int;
let s = "";
for i = 1 to 10
let n = mod(abs(dbms_random_random()), 62);
if (n < 10)
then
let n = n + 48;
elif (n < 36)
then
let n = n + 55;
else
let n = n + 61;
end if
let s = s || chr(n);
end for
return s;
end function;
This function can then be called from an insert trigger to populate the empcode column of your table.
I have a form with a TPageControl that has two tabs. In each of them there is a TVirtualStringTree and I have defined these two structures:
typedef struct tagTTreeMun
{
AnsiString Municipio;
int Padron;
int Censo;
double Relacion;
int Codigo;
} TTreeMun, *PTreeMun;
typedef struct tagTTreePro
{
AnsiString Proceso;
int Padron;
int Censo;
double Relacion;
int Codigo;
}TTreePro, *PTreePro;
I know: they are almost the same; then I explain it. The first one is loaded from four nested querys and it does it without any problem, but the second one ... There's no way!
To load this second I need two querys:
SELECT DISTINCT Date FROM Elections ORDER BY Date DESC
that field Date contains only the year and runs without any problem.
SELECT A.Codigo, B.Name, SUM (C.Padron) Padron, SUM (C.Censo) Census, A.Closed
FROM Elections A, Process B, HisElec C
WHERE A.CodPrv = (SELECT Literal FROM Installation WHERE Label = 'Province')
AND A.CodPrv = B.CodPrv AND B.Codigo = A.Process AND A.Closed = 1
AND A.CodPrv = C.CodPrv AND A.Codigo = C.Election
AND A. Date =: Date
GROUP BY 1, 2, 5
UNION
SELECT A.Codigo, B.Name, SUM (C.Padron) Padron,
(SELECT SUM (Census) FROM Tables WHERE CodPrv = (SELECT Literal FROM Installation WHERE Label = 'Province')) Census,
A.Closed
FROM Elections A, Process B, Dl01 C
WHERE A.CodPrv = (SELECT Literal FROM Installation WHERE Label = 'Province')
AND A.CodPrv = B.CodPrv AND A.Process = B.Code AND A.Closed = 0
AND A. Date =: Date
GROUP BY 1, 2, 5
ORDER BY 1 DESC, 3
It also runs without problems or errors. The problem comes when trying to pass that data to the corresponding TVirtualStringTree.
PTreePro DatPro;
PVirtualNode Node1, Node2, Node3, Node4;
LisPro->NodeDataSize = sizeof (TTreePro);
LisPro->BeginUpdate ();
LisPro->Clear ();
for (;! qTemp1->Eof; qTemp1->Next ())
{
Node1 = LisPro->AddChild (NULL);
DatPro = (PTreePro) LisPro->GetNodeData (Node1);
DatPro->Process = IntToStr(qTemp1->FieldByName ("Date")->AsInteger);
qTemp2->Close ();
qTemp2->ParamByName ("Date")->AsInteger = qTemp1->FieldByName("Date")->AsInteger;
qTemp2->Open ();
for (;! qTemp1->Eof; qTemp1->Next())
{
Node2 = LisPro->AddChild(Node1);
DatPro = (PTreePro)LisPro->GetNodeData(Node2);
DatPro->Process = qTemp1->FieldByName("Name")
[...]
}
}
When trying to create that Node1 in this query, the lines Node1 = ... and DatPro = (PTreePro) ... are executed without major problem, except that Node1, after running the AddChild, has a NULL value and therefore, from then on, it can only give an error because when trying to give value to Process, it automatically gives an execution error.
I have tried putting the load of each of the trees in a different function, by isolating code; I have tried with the same structure (in the end they are identical) or as in the example with two structures, to change the order of execution. For more tests that I have done I am not able to load both trees, in LisPro I ALWAYS do the same to me.
Input file with two column:
Visit ProductString
101 ;Cross Trainers;1;69.95,;Athletic Socks;10;29.99
102 ;Amplifier;1;120.90,;Headphone;2;59.99;leather wallet;1;99.99;
I am looking for Pig script that can parse "ProductString" value in each row and provide cumulative revenue.
ie.,Output:
69.95+29.99+120.90+59.99+99.99=380.82
I'll assume there should be a , after 59.99 and that there shouldn't be a ; after 99.99. If so, you need to tokenize and flatten on the , to extract products and then split on the ; to get item prices and qty.
Query:
data = LOAD 'db.table';
A = FOREACH data GENERATE visit, FLATTEN(TOKENIZE(product_string, ',')) AS tmp_col;
B = FOREACH A GENERATE visit, STRSPLIT(tmp_col, ';') AS prod;
C = FOREACH B GENERATE visit, prod.$1 AS item:chararray
, (int)prod.$2 AS qty:int, (double)prod.$3 AS revenue:double;
grpd = GROUP C all;
D = FOREACH grpd GENERATE SUM(C.revenue);
DUMP D;
Output:
(380.82)
My graph contains a set of nodes which are enumerated using a dedicated field fid. I want to update this enumeration periodically.
My current approach is to reset the enumeration and execute multiple statements that increase the fid for each node.
1. (f:File) set f.fid = -1
for(int i = 0; i < count ; i++) {
2. (f:File) set f.fid = i where id(f) = nodeId
}
I guess it should be possible to execute this task using a single cypher statement using the foreach clause.
MATCH p=(f:File)
FOREACH (n IN nodes(p)| SET f.fid = -1 )
I was looking for something similar to this statement.
MATCH (f:File)
WITH COLLECT(f) AS fs
WITH fs, i = 0
FOREACH (f in fs, i=i+1| SET f.fid = i ) return f.fid, f.name
Based on the following console set : http://console.neo4j.org/r/447qni
The following query seems to do the trick :
MATCH (f:File)
WITH collect(f) as f, count(f) AS c
UNWIND range(0,c-1) AS x
WITH f[x] AS file,x
SET file.iteration = x+1
Model:
return (from m in meterReadings
group m by new { date = m.ReadDate } into g
select new
{
ReadDate = g.Key.date.ToString("dd.MM.yyyy - HH:mm:ss"),
T1 = from t1 in g
where t1.Name == "T1"
select t1.Value.ToString("0,0.000"),
T2 = from t2 in g
where t2.Name == "T2"
select t2.Value.ToString("0,0.000"),
T3 = from t3 in g
where t3.Name == "T3"
select t3.Value.ToString("0,0.000"),
Total = from total in g
where total.Name == "Toplam"
select total.Value.ToString("0,0.000")
}).AsQueryable<object>();
Query
var table = MeterReadingManager.GetMeterReadingsPivot(meterReadings, 1);
//No Error (in title)
rows = table.OrderBy("ReadDate","desc").Skip((pageIndex) * pageSize).Take(pageSize)
//Error (in title)
rows = table.OrderBy("T1","desc").Skip((pageIndex) * pageSize).Take(pageSize)
When I order by ReadDate, It works. But When I try to order by other fields I get the error : At least one object must implement IComparable
Why I get this error? And How can I fix it?
If you want to sort a list of items of any type, the type must implement IComparable for the sort algorithm to be able to compare items. T1 is an IQueryable, whcih does not implement IComparable. I think you intended to create string values for T1, T2, and T3. If so, you should add FirstOrDefault() to each linq statement creating T1, etc.
Edit
(After your comment)
I mean this:
T1 = (from t1 in g
where t1.Name == "T1"
select t1.Value.ToString("0,0.000")).FirstOrDefault()
Now T1 is a string and, thus, it can be used in sorting.
You could try ThenByDescending:
var rows = table
.OrderByDescending(x => x.ReadDate).Skip((pageIndex) * pageSize).Take(pageSize)
.ThenByDescending(x => x.T1).Skip((pageIndex) * pageSize).Take(pageSize);
UPDATE:
You could use Reflection (a bit slower) if ordering by one field:
var tableInfo = table.GetType().GetProperty("T1");
var sortedRow = table.OrderByDescending(x => tableInfo.GetValue(x, null)).Skip((pageIndex) * pageSize).Take(pageSize);