I am pretty new to rails and have the following problem:
I have an array of serialized hashes stored in a database. Now I want to extend the hash data with a new key-value pair (calculated from stored data), e.g. like this in Ruby:
Data stored database:
coordinates = Array.new
c1 = Hash.new('x' => x1, 'y' => y1)
c2 = Hash.new('x' => x2, 'y' => y2)
c3 = Hash.new('x' => x3, 'y' => y3)
coordinates << c1 << c2 << c3
Extending data:
coordinates.each_with_index do |c, i|
c['z'] = c['x'] + c['y']
Representation in the database looks like:
"coordinates_table": [
{
"id": 1,
"coordinates": [
{ "x": 1.0, "y": 0.5 },
{ "x": 0.5, "y": 0.4 }
],
"created_at": "2015-11-22T00:18:38.592Z",
"updated_at": "2015-11-22T00:18:38.592Z"
}
]
This is the original migration:
class CreateCoordinates < ActiveRecord::Migration
def change
create_table :coordinates do |t|
t.text :coordinates_param
t.timestamps null: false
end
add_index :coordinates, :id
end
end
and Coordinate class:
class Coordinate < ActiveRecord::Base
serialize :coordinates_param
end
Since I do not intend to change the input data, but only extend the data stored in the database, I figured the best way to realize this was a migration, but I did not find any way to access single Hash objects stored in my table in the generated migration model.
Is there any possibility to do this in rails, or do I have to provide a separate extension class where I could store the new data?
Related
I want to use the ActiveRecord model serializer to show results from the primary key table and foreign key table. However, I want the results to be presented grouped by a column in the foreign key table.
cats = Cats.paginate(page: params[:page], per_page: 20)
render json: cats, meta: pagination(cats), adapter: :json
In the ActiveRecord model serializer:
class CatSerializer < ActiveModel::Serializer
attributes :cat, :persons
def persons
object.person
end
def cat
{ id: object.id, cat_name: object.name}
end
Now cat_name is not unique and Persons can share many cat_names. please note that Person => has_many Cats, but cat_name can be similar to multiple Persons. How can I show the data in this format:
"results": [
{
"cat": {
"id": 11,
"cat_name": "Luzi",
...
},
"persons": [
{
"id": 1,
"name": "andy"
},
{
"id": 2,
"name": "david"
}
Please also note that groyp_by(&:cat_name) does not work with pagination.
You can use custom serializer that accepts an already groupby ActiveRecord result
def index
#cats = Cat.joins(:persons).group("persons.name")
render json: #cats, serializer: GroupedCatSerializer
end
And you can define custom serializer like
class GroupedCatSerializer < ActiveModel::Serializer
# method override
def serializable_object(options={})
#object.map do |group_key, models|
[ group_key , serialized_models(models) ]
end.to_h
end
private
def serialized_models models
models.map{ |model| CatSerializer.new(model, root:
false) }
end
end
Using activerecord-postgis-adapter, how can I parse / encode wkt in results from database query?
I've got a simple model of Places:
class CreatePlaces < ActiveRecord::Migration[5.1]
def change
create_table :places do |t|
t.string :name
t.st_point :coords, :geographic => true
t.timestamps
end
change_table :places do |t|
t.index :coords, using: :gist
end
end
end
I get all places in following way:
class PlacesController < ApplicationController
def index
#places = Place.all
render json: #places.to_json
end
end
But my JSON in response contains WKT:
[
{
"id": 1,
"name": "test name 1",
"coords": "POINT (50.324192 19.037805)",
"created_at": "2017-09-07T20:29:19.203Z",
"updated_at": "2017-09-07T20:29:19.203Z"
}
]
I can map #places and encode coords like this:
class PlacesController < ApplicationController
def index
#places = Place.all
#places.map { |k,v| k.coords = RGeo::GeoJSON.encode(k.coords, json_parser: :json) }
render json: #places.to_json
end
end
And then I get what I wanted - encoded/parsed coords in GeoJSON form:
[
{
"id": 1,
"name": "test name 1",
"coords": {
"type": "Point",
"coordinates": [
50.324192,
19.037805
]
},
"created_at": "2017-09-07T20:29:19.203Z",
"updated_at": "2017-09-07T20:29:19.203Z"
}
]
Is it right way to encode POINT?
Add this code to one of your initializers:
RGeo::ActiveRecord::GeometryMixin.set_json_generator(:geojson)
See https://stackoverflow.com/a/6221804
With my current project, I'm receiving a large JSON file that I'm parsing and storing into my database. The problem is I feel like I'm structuring my database in a very inefficient way.
Example of JSON:
{
first_name: "John",
records: {
ids: [110, 725, 2250],
count: [1, 1, 6]
},
items: {
top: {
title: "My top",
values: { value: [51, 50, 70] }
},
middle: {
title: "Middle Stuff",
values: { value: [51] }
},
},
values: {
health: 100,
strength: 250,
mana: 50
}
}
As you can see the JSON is fairly complex, with nested Objects.
While building it, I started with the main Object ( user ), then slowly started adding more objects. Values was easy, so I added that as another table and just with a reference to the user_id.
Then I did records, which is a bit more complex, but works. However, I'm very worried about the most nested parts, that could be 5+ objects deep. I feel like I shouldn't have an entire column row for a simple value.
What would be the best way to improve on this? Should I somehow crunch the data and store it differently?
Thanks for your help.
// JSON response
{
"name": "John Smith",
...
"progression": {
"levels": [{
"name": "Level 1",
"bosses": [
{
"name": "Boss 1",
"difficultyCompleted": "Hard"
},
{
"name": "Boss 2",
"difficultyCompleted": "Hard"
}]
},{
"name": "Level 2",
"bosses": [
{
"name": "Boss 3",
"difficultyCompleted": "Normal"
},
{
"name": "Boss 4",
"difficultyCompleted": "Easy"
}]
}
}
}
In this example JSON, there is a few layers for each boss that the user has completed. What I would have thought to do initially was to create models and tables but that seemed like it would be wasteful not only in memeory, but also would take longer to fetch the current progression for Boss 4.
Example:
User has_one Progression.
Progression has_one Levels.
Levels has_many Dungeons.
Dungeons has_many bosses.
What I did instead was trying to compress the bosses into a single field, and just convert the JSON at runtime.
So, instead my structure would be like this.
User has_one Progression.
Progression has_many Dungeons.
Models:
// Progression.rb
class Progression < ApplicationRecord
has_many :dungeons
def self.initialize(params={})
params = params['levels']
prog = Progression.new()
params.each do |dungeon|
prog.dungeons.append( Dungeon.initialize( dungeon ) )
end
return prog
end
end
// Dungeon.rb
class Dungeon < ApplicationRecord
belongs_to :progression
def self.initialize(params={})
Dungeon.new(params.reject { |k| !Dungeon.attribute_method?(k) }) # Used to ignore any unused parameters that don't exist on the model.
end
# To convert `bosses` from JSON into a hash for easy use.
def get_bosses
JSON.parse bosses.gsub( '=>', ':' )
end
end
Migration:
// xxxxxx_create_progression.rb
class CreateProgression < ActiveRecord::Migration[5.0]
def change
create_table :progressions do |t|
t.integer :character_id
end
create_table :dungeons do |t|
t.integer :character_id
t.integer :progression_id
t.string :name
t.text :bosses
end
add_index :progressions, :character_id
add_index :dungeons, :progression_id
end
end
Now, when a User is updated to fetch their progression, I can set their progression.
// users_controller.rb
def update_progression
progression = ... fetched from the response
#user.progression = Progression.initialize(progression)
end
After that's all saved to the user, you can now fetch the progression back by going:
<% user.progression.dungeons.each do |dungeon| %>
<%= dungeon.name %>
<% end %>
This solution seems like a decent mix, but I'm a bit worried about the parsing of the JSON. It could become too much, but I'll have to keep watching it. Any other ideas or improvements would be greatly appreicated.
Try to calculate nearest points in some distance and one nearest point.
in db/migrate/xxx_create_points.rb":
class CreatePoints < ActiveRecord::Migration
def change
create_table :points do |t|
t.point :location, :geographic => true
t.string :name, :null => false
t.timestamps
end
change_table :points do |t|
t.index :location, :spatial => true
end
end
end
in config/routes.rb:
get 'points/:lat/:lon/:distance', to: 'points#index', :constraints => { :lat => /[^\/]+/, :lon => /[^\/]+/}
in controllers/points_controller.rb:
class PointsController < ApplicationController
def index
#points= Point.all
if params[:distance].present? && params[:lat].present? && params[:lon].present?
#distance = params[:distance].to_i
#latitude = params[:lat].to_f
#longitude = params[:lon].to_f
#points= Point.near(#latitude, #longitude, #distance)
#near = Point.nearest(#latitude, #longitude, 100).first
end
end
In models/point.rb:
class Point < ActiveRecord::Base
set_rgeo_factory_for_column(:location,
RGeo::Geographic.spherical_factory(:srid => 4326))
attr_accessible :location, :name
scope :near, lambda { |latitude, longitude, distance|
where("ST_Distance(location,
"+"'POINT(#{latitude} #{longitude})') < #{distance}")}
scope :nearest, lambda { |latitude, longitude, distance|
where("ST_Distance(location,
"+"'POINT(#{latitude} #{longitude})') < {distance}")
.order("ST_Distance(location, ST_GeographyFromText('POINT(#{latitude} #{longitude})'))").limit(1)}
end
In views/points/index.html.erb:
<script>
$.each(response.data, function(i, point) {
$('#map_canvas').gmap('addMarker', {
'position': new google.maps.LatLng(point.latitude, point.longitude), 'bounds': true });
}
$('#map_canvas').gmap('addShape', 'Circle', {
...
'center': new google.maps.LatLng(<%= "#{#latitude},#{#longitude}" %>),
'radius': 1 });
$('#map_canvas').gmap('addShape', 'Circle', {
...
'center': new google.maps.LatLng(<%= "#{#latitude},#{#longitude}" %>),
'radius': <%= #distance %> });
$('#map_canvas').gmap('addShape', 'Circle', {
...
'center': new google.maps.LatLng(<%= "#{#near.location.x},#{#near.location.y}" %>),
'radius': 2 });
</script>
Result in browser: http://xxx/points?distance=200&lat=55.7632500&lon=52.442000
What am I doing wrong?
PostGIS always uses the coordinate axis order (X Y) or (longitude latitude). I see snippets in your code that has this reversed:
ST_GeographyFromText('POINT(#{latitude} #{longitude})')
This needs to be switched:
ST_MakePoint(#{longitude}, #{latitude})::geography
Google maps uses mercator projections and the difference you see might be caused by the distance calculation differences between spherical projections you use in your model and mercator projections.
Try using RGeo's simple_mercator_factory in your Point class instead of using the spherical_factory.
RGeo's documentation gives you more details about that.
# Many popular visualization technologies, such as Google and Bing
# maps, actually use two coordinate systems. The first is the
# standard WSG84 lat-long system used by the GPS and represented
# by EPSG 4326. Most API calls and input-output in these mapping
# technologies utilize this coordinate system. The second is a
# Mercator projection based on a "sphericalization" of the WGS84
# lat-long system. This projection is the basis of the map's screen
# and tiling coordinates, and has been assigned EPSG 3857.
My RABL template seems to be very un-DRY and over complex. Because of this I think I may be using it wrong, or that there are better ways at generating my desired output.
As you can see from the show.rabl code, I have to turn the plugins_vulnerability.vulnerability association into a JSON hash, explicitly selecting which keys I need, then merge the plugins_vulnerability.fixed_in value into the hash, and finally adding the new hash, which now contains the fixed_in value, to the vulnerabilities_array array.
I'm doing this because I want the fixed_in value to be within the vulnerability node.
plugins_controller.rb
class Api::V1::PluginsController < Api::V1::BaseController
def show
#plugin = Plugin.friendly.includes(:plugins_vulnerability, :vulnerabilities).find(params[:id])
end
end
show.rabl:
object #plugin
cache #plugin if Rails.env == 'production'
attributes :name
# Add the 'vulnerabilities' node.
node :vulnerabilities do |vulnerabilities|
vulnerabilities_array = []
# turn the plugins_vulnerability association into an array
vulnerabilities.plugins_vulnerability.to_a.each do |plugins_vulnerability|
vulnerability = plugins_vulnerability.vulnerability.as_json # turn the plugins_vulnerability.vulnerability association into json
vulnerability = vulnerability.select {|k,v| %w(id title references osvdb cve secunia exploitdb created_at updated_at metasploit fixed_in).include?(k) } # only select needed keys
vulnerabilities_array << {
:vulnerability => vulnerability.merge(:fixed_in => plugins_vulnerability.fixed_in)
} # merge the fixed_in attribute into the vulnerability hash and add them to an array (fixed_in is from plugins_vulnerabilities)
end
vulnerabilities_array
end
output.json
{
"plugin": {
"name": "simple-share-buttons-adder",
"vulnerabilities": [
{
"vulnerability": {
"id": 88157,
"title": "Simple Share Buttons Adder 4.4 - options-general.php Multiple Admin Actions CSRF",
"references": "https:\/\/security.dxw.com\/advisories\/csrf-and-stored-xss-in-simple-share-buttons-adder\/,http:\/\/packetstormsecurity.com\/files\/127238\/",
"osvdb": "108444",
"cve": "2014-4717",
"secunia": "",
"exploitdb": "33896",
"created_at": "2014-07-15T17:16:51.227Z",
"updated_at": "2014-07-15T17:16:51.227Z",
"metasploit": "",
"fixed_in": "4.5"
}
},
{
"vulnerability": {
"id": 88158,
"title": "Simple Share Buttons Adder 4.4 - options-general.php ssba_share_text Parameter Stored XSS Weakness",
"references": "https:\/\/security.dxw.com\/advisories\/csrf-and-stored-xss-in-simple-share-buttons-adder\/,http:\/\/packetstormsecurity.com\/files\/127238\/",
"osvdb": "108445",
"cve": "",
"secunia": "",
"exploitdb": "33896",
"created_at": "2014-07-15T17:16:51.341Z",
"updated_at": "2014-07-15T17:16:51.341Z",
"metasploit": "",
"fixed_in": "4.5"
}
}
]
}
}
I guess you can do something like this:
object #plugin
cache #plugin if Rails.env == 'production'
attributes :name
child(#plugin.vulnerabilities => :vulnerabilities) {
attributes :id, :title, :references, :osvdb, :cve, :secunia, :exploitdb, :created_at, :updated_at, :metasploit
# Add the 'fixed_in' node.
node :fixed_in do |vulnerability|
#plugin.plugins_vulnerability.fixed_in
end
}
This should create the same output that you need. And it doesn't look awefully complex to me.