How to refer one avro schema inside another one - avro

I need to refer to this Student schema from inside another schema:
{
"type": "record",
"namespace": "data.add",
"name": "Student",
"fields": [
{
"name": "Name",
"type": "string"
},
{
"name": "Age",
"type": "int"
}
]
}
This is the parent Address schema which needs to refer to Student:
{
"type": "record",
"namespace": "data.add",
"name": "Address",
"fields": [
{
"name": "student",
"type": "Student"
}
]
}
The above throws an error when I build using Gradle with the Avro plugin. Both schemas are located in the same folder.

Sorry if I'm too late to the party, but it looks to me like both the maven avro plugin and the avro-tools compiler do not determine dependency order when loading, but will succeed if you order them yourself on the commandline. I have an example demonstrating this from your sample files in a standard maven directory structure.
When I put the schema with no dependencies first in the commandline, it succeeds:
java -jar /path/to/avro-tools-1.11.0.jar \
compile schema \
src/main/avro/student.avsc \
src/main/avro/address.avsc \
target/generated-sources/avro
ls target/generated-sources/avro/data/add/*
target/generated-sources/avro/data/add/Address.java target/generated-sources/avro/data/add/Student.java
When I put the schema with dependencies first in the commandline, it fails:
java -jar /path/to/avro-tools-1.11.0.jar \
compile schema \
src/main/avro/address.avsc \
src/main/avro/student.avsc \
target/generated-sources/avro
Exception in thread "main" org.apache.avro.SchemaParseException: "data.add.Student" is not a defined name. The type of the "student" field must be a defined name or a {"type": ...} expression.
at org.apache.avro.Schema.parse(Schema.java:1676)
at org.apache.avro.Schema$Parser.parse(Schema.java:1433)
at org.apache.avro.Schema$Parser.parse(Schema.java:1396)
at org.apache.avro.tool.SpecificCompilerTool.run(SpecificCompilerTool.java:154)
at org.apache.avro.tool.Main.run(Main.java:67)
at org.apache.avro.tool.Main.main(Main.java:56)

This was successful:
{
"type" : "record",
"namespace" : "data.add",
"name" : "Address",
"fields" : [
{
"name": "student",
"type": "data.add.Student"
}
]
}

Related

Getting user's organizations from Keyrock

I'm looking for a way to get organizations of a user logged in using Keyrock's OAuth2.
I have Keyrock set up using docker-compose. I've created an organization and assigned users to it, using API. But, when a user logs in, both the JWT token and the /user endpoint return an empty list of organizations.
The user is assigned to an organization, I can see entries in the database and when using API:
mysql> select * from user_organization;
+----+--------+--------------------------------------+--------------------------------------+
| id | role | user_id | organization_id |
+----+--------+--------------------------------------+--------------------------------------+
| 2 | owner | admin | f21f255d-ef4c-4afd-9cc3-261e3e7dc168 |
| 3 | owner | admin | 3cab93ad-83ba-404f-8499-f870d0ab0d02 |
| 4 | member | 42a336da-2321-4258-a4dd-6e447d2ee171 | 3cab93ad-83ba-404f-8499-f870d0ab0d02 |
+----+--------+--------------------------------------+--------------------------------------+
In the API, when logged in as the admin:
curl -iX GET \
'localhost:3000/v1/organizations/3cab93ad-83ba-404f-8499-f870d0ab0d02/users/42a336da-2321-4258-a4dd-6e447d2ee171/organization_roles' \
-H 'Content-Type: application/json' \
-H 'X-Auth-token: 09c30919-ad6c-4eb9-9d90-283f0ae7483e'
{"organization_user":{"user_id":"42a336da-2321-4258-a4dd-6e447d2ee171","organization_id":"3cab93ad-83ba-404f-8499-f870d0ab0d02","role":"member"}}%
When I log in as the 42a336da-2321-4258-a4dd-6e447d2ee171 user using OAuth2, the JWT token has neither roles nor organizations. A part of decoded JWT token:
"organizations": [],
"displayName": "",
"roles": [],
"app_id": "d2e68240-b822-413c-978f-cc9ebdba1600",
"trusted_apps": [],
"isGravatarEnabled": false,
"id": "42a336da-2321-4258-a4dd-6e447d2ee171",
similarly, when instead of JWT I use a bearer token and I get user details using the /user endpoint:
http://localhost:3000/user?access_token=5d50b9b7162fc1cf7890d3b81feb96c5b176f0c8&action=GET&resource=myResource&app_id=d2e68240-b822-413c-978f-cc9ebdba1600
{
"organizations": [],
"displayName": "",
"roles": [],
"app_id": "d2e68240-b822-413c-978f-cc9ebdba1600",
"trusted_apps": [],
"isGravatarEnabled": false,
"id": "42a336da-2321-4258-a4dd-6e447d2ee171",
"authorization_decision": "Deny",
"app_azf_domain": "",
"eidas_profile": {},
"attributes": {},
"shared_attributes": "",
"username": "Test",
"email": "email#example.com",
"given_name": "Test",
"family_name": "Test",
"image": "",
"gravatar": "",
"extra": {
"visible_attributes": [
"username",
"description",
"website",
"identity_attributes",
"image",
"gravatar"
]
},
"sub": "42a336da-2321-4258-a4dd-6e447d2ee171"
}
Did I miss some steps in the organization/user configuration? Is there another way to get the organizations and roles of a current user?

When getting a team using v1.0 API, beta resources are also being returned

I send a GET for a team at least one resource the resources is in beta. The resource teamDiscoverySettings shouldn't be returned when the request is made to v1.0 API
I sent a GET https://graph.microsoft.com/v1.0/teams/{{teamID}} and the server responds with:
{
"#odata.context": "https://graph.microsoft.com/v1.0/$metadata#teams/$entity",
"id": "...",
"displayName": "...",
"description": "...",
"internalId": "...",
"webUrl": "...",
"isArchived": false,
"discoverySettings": {
"showInTeamsSearchAndSuggestions": false
},
"memberSettings": {
"allowCreateUpdateChannels": true,
"allowDeleteChannels": true,
"allowAddRemoveApps": true,
Shouldn't the beta resources only be returned withing the beta version of the APIs?

Loopback Error 401 (Unauthorized) while accessing APIs

I am facing an issue in loopback, 401 (Unauthorized). Initially all APIs are working perfectly.
Now I set ACL for some of the models. After that I got an error 401 (Unauthorized) while accessing APIs
Sample Code
Eg: One of my model.json file
"acls": [
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "$unauthenticated",
"permission": "DENY"
}
I got the accesstoken after login. I think the access token is not getting in in my API headers.
you can also define ACS level. you DENY for all unauthorized user but also set permission for authorized user like.
"acls": [
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "admin",
"permission": "ALLOW"
},
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "$unauthenticated",
"permission": "ALLOW"
}
]
principalId is define in my ROLE collection if you did't define any princepalId you can also use this.
{
"accessType": "*",
"principalType": "ROLE",
"principalId": "$authenticated",
"permission": "ALLOW"
}

devise_token_auth how to access user.created_at

I'm using devise_token_auth on my Rails 5 api
Currently, when I make a POST request to /auth/sign_in everything goes well, I get data looking like this:
"data": {
"id": 3,
"email": "me#user.com",
"name": null,
"nickname": null,
"bio": null,
"description": null,
"url": null,
"image": {
"url": null
},
"provider": "email",
"uid": "me#user.com",
"allow_password_change": false,
"role": null
}
however, I'd like to also be able to access timestamp attributes (created_at, updated_at), which are already enabled:
ActiveRecord::Schema.define(version: 2018_10_18_201621) do
# These are extensions that must be enabled in order to support this database
enable_extension "plpgsql"
create_table "users", force: :cascade do |t|
t.datetime "created_at", null: false
t.datetime "updated_at", null: false
Any ideas? Thanks in advance
Check this issue discussion about overriding the rendering methods on devise_token_auth gem on it's github repo: https://github.com/lynndylanhurley/devise_token_auth/issues/597.
Try the code below on your User model so you can use your custom serializer instead of the default one:
def token_validation_response
UserSerializer.root = false
UserSerializer.new(self).as_json
end

Seed Rails SQLite Database from a JSON file

I'm new to Rails (using 5.0.2). I want to seed my empty SQLite db from a JSON file.
I scaffolded my Product resource, in seed.rb added:
records = JSON.parse(File.read('/home/projects/test1/public/products.json'))
records.each do |record|
ModelName.create!(record)
end
Then launched rake db:seed.
I got an error:
rake aborted!
JSON::ParserError: 409: unexpected token at '{
My JSON consists of products with various properties:
[
{
"model": "Brand",
"brand": "Brand",
"price": 19.99,
"currency": "GBP",
"link1": "https://www.amazon.co.uk/Call-Duty-Ghosts-Hardened-Xbox/dp/B00ECQJ0BS/ref=pd_sim_107_4?_encoding=UTF8&psc=1&refRID=9GE6A8BW61S1FG4H3ZRJ",
"link2": "",
"size1": "{min: "5",max: "90"}",
"net_weight": "263 kg",
"size2": "{w: "19.4",h: "11.7",z: "12.7"}",
"d1": "",
"color": "['Black','White']",
"materials": "plastic+concrete, glass ceiling",
"models": "['model 1', 'model 2', 'model 3', 'model 100']",
"props": "['prop1', 'Prop1', 'Prop1']",
"": "",
"size3": "{w: "100",h: "200",z: "500"}"
},
]
How can I fix this JSON file and correctly seed my db with the products data from it?
Is it the best way to import manually created JSON files into Rails, or it's better to try some other approach?
The problem is with the size1, size2, and size3 properties. For each of those keys, the value contains a double-quote. If you convert those to single quotes or escape them by putting a \ before them, your JSON should parse correctly.

Resources