I have a topic called customers and I have created a stream for it
CREATE STREAM customers_stream (customerId INT, isActive BOOLEAN)
WITH (KAFKA_TOPIC='customers', VALUE_FORMAT='json');
My producer for customers topic is generating a Integer key and a json value. But when I see the row key is being set to a some binary value
ksql> print 'customers';
Format:JSON
{"ROWTIME":1570305904984,"ROWKEY":"\u0000\u0000\u0003�","customerId":1001,"isActive":true}
{"ROWTIME":1570307584257,"ROWKEY":"\u0000\u0000\u0003�","customerId":1002,"isActive":true}
Now if i create a table it results in a single row (maybe because row key is the same??)
CREATE TABLE customers (customerId INT, isActive BOOLEAN)
WITH (KAFKA_TOPIC='customers', KEY='customerId',VALUE_FORMAT='json');
After searching the web I bumped into this article https://www.confluent.io/stream-processing-cookbook/ksql-recipes/setting-kafka-message-key and created a new stream by repartitioning on the key
CREATE STREAM customers_stream2 AS \
SELECT * FROM customers_stream \
PARTITION BY customerId;
So how do I create a table which has the latest values of customers data?
creating a table from stream is resulting in a error
CREATE TABLE customers_2_table_active AS
SELECT CUSTOMERID,ISACTIVE
FROM customers_stream2;
Invalid result type. Your SELECT query produces a STREAM. Please use CREATE STREAM AS SELECT statement instead.
I need the latest value of the various rows so that another microservice can query the new table.
Thank you in advance
Rekeying seems to be the right approach, however, you cannot convert a STREAM into a TABLE directly.
Note, that your rekeyed stream customers_stream2 is written into a corresponding topic. Hence, you should be able to crate a new TABLE from the stream's topic to get the latest value per key.
Related
Is there any way to delete columns of an influx timeseries as we have accidentally injected data using the wrong data type (int instead of float).
Or to change the type of data instead.
Unfortunately, there is no way to delete a "column" (i.e. a tag or a field) from an Influx measurement so far. Here's the feature request for that but there is no ETA yet.
Three workarounds:
use SELECT INTO to copy the desirable data into a different measurement, excluding the undesirable "columns". e.g.:
SELECT desirableTag1, desirableTag2, desirableField1, desirableField2 INTO new_measurement FROM measurement
use CAST operations to "change the data type" from float to int. e.g.:
SELECT desirableTag1, desirableTag2, desirableField1, desirableField2, undesiredableTag3::integer, undesiredableField3::integer INTO new_measurement FROM measurement
"Update" the data with insert statement, which will overwrite the data with the same timestamp, same tags, same field keys. Keep all other things equal, except that the "columns" that you would like to update. To make the data in integer data type, remember to put a trailing i on the number. Example: 42i. e.g.:
insert measurement,desirableTag1=v1 desirableField1=fv1,desirableField2=fv2,undesirableField1=someValueA-i 1505799797664800000
insert measurement,desirableTag1=v21 desirableField1=fv21,desirableField2=fv22,undesirableField1=someValueB-i 1505799797664800000
I have to insert incremental values in a column of the table using Fitnesse. The incremental value I'll get from a stored procedure which returns the last inserted value. So I have to increment the value and store it.
For example: I'll get a value from the stored procedure output. And I have to increment the value by 1 and insert into the table.
Any ideas?
Output from stored procedure is like : ACRDE0001 (PK)
Value to store in table : ACRDE0002, ACRDE0003, .....
Expected output
!|insert|table1|
|col1|col2|col3|
|ACRDE0001|abc|def|
|ACRDE0002|abc|def|
|ACRDE0003|abc|def|
.
.
.
.
As far as I'm aware the only way to change (e.g. increment) a value you get during your test is by writing some code in a fixture. There is a pull request to allow more dynamic Slim expression directly in the wiki, but that has not been merged (let alone released) yet.
Your questions suggests that the value is something you get from a database and that you then want to send back the generated/incremented value with new records you insert. In that case I wonder whether the increment is actually that useful to actually have in your wiki (your test case is not about the generated values, is it?).
Maybe your fixture could just retrieve the initial value (or have it supplied as constructor value) and the fixture could generate the a new value for each row and send them to the database.
I developed an app based on App Inventor and Fusion Tables. When I want to update total money by adding some money to already existing money it is giving some error.
When I use SELECT command to get information from fusion table it is taking then number with column Name. When i am trying to add both of them it is giving following error.
Error message
The result from a fusiontable includes always the header row...
from your example SELECT statement the result is
TotalPaid
5000
Obviously to add any value to that result will result in an error, because you only can add numeric values...
You first have to extract the value (in your example 5000) from the result. Convert the result into a list using the split block, just split at \n (new value) to get a list, then select the 2nd item using the select list item block.
Note: to be able to update something in a table, you need the ROWID, see also the SQL Reference Documentation of the Fusion Tables API.
For UPDATE statements the first step to be done is to get the ROWID of the row to be updated with a SELECT statement. The second step is to do the UPDATE.
In Qlikview, I have an excel sheet that I use to map USERNAME to a TEAM value. But everytime I refresh the dashboard, new USERNAME values come up and since they are not in the excel sheet, these USERNAME values show up as their own value in the TEAM column. How would I make it so that any USERNAME that is not in the excel sheet shows up as 'Unidentified' or another value under the TEAM column instead of showing up as their own separate value?
First of all when posting question here if possible always include the source code so everybody will have more clear picture about your problem. Just saying.
On the topic ...
Use the mapping load in this case with supplying the third parameter. For example:
TeamMapping:
Mapping
Load
UserName,
Team
From
[User_to_Team_Mapping.xlsx] (ooxml, embedded labels, table is [Sheet1])
;
Transactions:
Load
Id,
Amount,
ApplyMap( 'TeamMapping', User, 'Unidentified') as Team
From
Transactions.qvd (qvd)
;
The third parameter in ApplyMap is the default string value when mapping value was not found in the mapping table (TeamMapping)
I am trying to do something like this:
merge MembershipTEST as T
using (select OrganisationID, Name From MembershipPending) as S
on T.OrganisationID = S.OrganisationID
and T.Name = S.Name
when not matched then
insert (MembershipID,OrganisationID, Name)
values(
(EXEC [dbo].[spGetNextIntKeyByTableName]
#PKColName = 'MembershipID',#TableName = 'FWBMembership'),
S.OrganisationID,
S.Name );
Bascially the identitykey is from a sp
Is it possible?
Update 1: Answer is NO
read the online doc http://msdn.microsoft.com/en-us/library/bb510625%28v=sql.105%29.aspx
VALUES ( values_list) Is a comma-separated list of constants,
variables, or expressions that return values to insert into the target
table. Expressions cannot contain an EXECUTE statement.
Build an SSIS package. Just create a new Data Flow task. Use two OLEDB sources make one of them execute the stored procedure and the other one select from the table you are looking to MERGE with. Make sure they are both ordered by the same thing. Go into advanced settings for each OLEDB source and set is sorted to true and then set the sort values for the items you are ORDERing BY. Then move both data flows to to a MERGE JOIN. Then send the Data flow to a Conditional Split and set as INSULL(OrganisationID). Use the resulting data flow to go to an OLEDB destination.
Sorry for the lack of visuals I will add them later when I'm on lunch hour to busy to add them now.