I'm trying to extract data from 2 tables in BigQuery, then join it by CoGroupByKey.
Although the output of BigQuery is PCollection<TableRow>, CoGroupByKey requires PCollection<KV<K,V>>.
How can I convert from PCollection<TableRow> to PCollection<KV<K,V>>?
CoGroupByKey needs to know which key to CoGroup by - this is the K in KV<K, V>, and the V is the value associated with this key in this collection. The result of co-grouping several collections will give you, for each key, all of the values with this key in each collection.
So, you need to convert both of your PCollection<TableRow> to PCollection<KV<YourKey, TableRow>> where YourKey is the type of key on which you want to join them, e.g. in your case perhaps it might be String, or Integer, or something else.
The best transform to do the conversion is probably WithKeys. E.g. here's a code sample converting a PCollection<TableRow> to a PCollection<KV<String, TableRow>> keyed by a hypothetical userId field of type String:
PCollection<TableRow> rows = ...;
PCollection<KV<String, TableRow>> rowsKeyedByUser = rows
.apply(WithKeys.of(new SerializableFunction<TableRow, String>() {
#Override
public String apply(TableRow row) {
return (String)row.get("userId");
}
}));
Related
I am using the Neo4J Java driver
<dependency>
<groupId>org.neo4j.driver</groupId>
<artifactId>neo4j-java-driver</artifactId>
<version>4.2.0</version>
</dependency>
And I am trying to use a DateTime value as value in a parameter map roughly like this:
// we have a timezone, unixTimestamp as epoch seconds and a simple key: "d"
final Instant instant = Instant.ofEpochSecond(unixTimestamp);
final ZoneId zoneId = ZoneId.of(timezone);
final ZonedDateTime zonedValueDate = ZonedDateTime.ofInstant(instant, zoneId);
// creating a DateTimeValue from zonedValueDate does not work either
return Map.of(key, zonedValueDate);
This map is used in an instance of TransactionWork like
public Map<String,Object> execute(Transaction tx) {
Result queryResult = tx.run(query, parameter);
// consuming results ...
// returning them as Map
}
But I am getting an exception:
org.neo4j.driver.exceptions.ClientException: Property values can only be of primitive types or arrays thereof
A DateTime or ZonedDateTime should be allowed as parameter (here is another hint), but how can it be used correctly?
Or do I HAVE to create a DateTime value like this:
datetime({year:1984, month:10, day:11, hour:12, timezone: 'Europe/Stockholm'})
I've got a Realm db containing objects, the primary key is a string containing numbers i.e.
#objc public class Caller : Object
{
#objc dynamic var number = ""
...
override public static func primaryKey() -> String? {
return "number"
}
If I try to retrieve the objects sorted by the key they are not coming out in order i.e.
let results = realm.objects(Caller.self).sorted(byKeyPath: "number")
If the db contains 3 objects with values for number of 12037366517, 12037600287, 67725082. Then if I iterate through results and log "number" the results are:
12037366517
12037600287
67725082
Why are they not coming out in sorted order?
This is expected behaviour if number is a string. When sorting strings in ascending order, it is sorted in lexicographical order. So the character "3" comes after "2", which comes after "1" etc. Your results are sorted correctly, just not as you expected it.
You can fix this by sorting on an actual numeric value. The best way would be to actually store the numbers as integers instead of strings. If that is not possible, you will have to manually sort, but I think that will be less performant.
let results = realm.objects(Caller.self).sorted { (a, b) -> Bool in
if let aInt = Int(a), let bInt = Int(b) {
return aInt < bInt
}
return false
}
During the chaincode init, One can deploy key-value pair such as: ["a","100", "b", "200"]
However, I would like to deploy key-value pairs such as: ["a", "100, v1, v2"]
Such that 100, v1, v2 are values of a. Two notes:
1. The values are non-integer
2. The values are separated by comma ","
Is this possible?
I check the chaincode shim: /home/standards/go/src/github.com/hyperledger/fabric/core/chaincode/shim/chaincode.go
The function:
// PutState writes the specified `value` and `key` into the ledger.
func (stub *ChaincodeStub) PutState(key string, value []byte) error {
return handler.handlePutState(key, value, stub.UUID)
Which invokes handlePutState (key, value, stub.UUID). Any lights on how to modify it so it works as desired? Thanks,
In the chaincode, each state can have only a single value associated with it. However multiple values can be simulated by making that "one value" a list. So you could do something like this
stub.PutState("a",[]byte("100,v1,v2"))
The state for "a" is now a comma separated list. When you want to retrieve these values, do the following:
Avals, err := stub.GetState("a")
AvalsString := string(Avals)
fmt.Println(AvalsString)
which should print the following string
100,v1,v2
If you need the individual arguments from there, just split the string on the commas, and convert to appropriate types. Bam, you can store and retrieve elements now.
Alternatively, if you data structure is more complicated than that, it might be worth it to put your data into a json object. You can then use marshaling and unmarshaling to convert back and forth from []byte (which can be stored in the state directly) and object (which is likely easier to use).
Example with Json, using init method because it's the one you mentioned
type SomeStruct struct {
AVal string `json:"Aval"`
BVal []string `json:"Bval"`
}
func (t *MyChaincode) Init(stub *shim.ChaincodeStub, function string, args []string) ([]byte, error) {
//recieve args ["someStringA","BVal1","Bval2","Bval3"]
//constructing and storing json object
myStruct := SomeStruct{
AVal: args[0],
BVal: []string{args[1],args[2],args[3]},
}
myStructBytes, err := json.Marshal(myStruct)
_ = err //ignore errors for example
stub.PutState("myStructKey",myStructBytes)
//get state back to object
var retrievedStruct SomeStruct
retrievedBytes, err := stub.GetState("myStructKey")
json.Unmarshal(retrievedBytes,retrievedStruct)
//congratulations, retrievedStruct now contains the data you stored earlier
return nil,nil
}
I have two classes
class Phone extends Observable
{
#observable String type = '';
#observable String provider = '';
#observable String num = '';
Map<String, Map<String, String> map = {};
Phone() {}
Phone.build({ this.type,
this.provider,
this.num });
}
I have attempted to use the field values as the key in map like so
Phone phone = new Phone();
phone.map[phone.type] = {'type':'cell', 'provider':'Verizon', 'num':'1234567'};
but it does not work. How can I make the fields value the key for the map?
just remove the quotes
phone.map[phone.type] = {type:'cell', provider:'Verizon', num:'1234567'};
Because you are using strings in your example this may not apply but be aware that if you use instances of custom types as Map key...
The keys of a `HashMap` must have consistent [Object.operator==]
and [Object.hashCode] implementations. This means that the `==` operator
must define a stable equivalence relation on the keys (reflexive,
anti-symmetric, transitive, and consistent over time), and that `hashCode`
must be the same for objects that are considered equal by `==`.
What exactly is the difference between an object and a dictionary in Actionscript?
var obj:Object = new Object();
obj.something = "something";
var dict:Dictionary = new Dictionary();
dict.something = "something";
trace(obj.something, dict.something);
The trace statement seems identical...
I think the example here highlights at least one of the most significant differences, which is strict equality in comparing keys.
In summary, dictionary[key] does NOT necessarily return the same value as dictionary["key"], even if key.toString() equals "key".
However, object[key] will return the same value as object["key"], if key.toString() equals "key".
Object() uses strings as keys, while Dictionary() uses objects as keys.
See http://gskinner.com/blog/archives/2006/07/as3_dictionary_.html