I would like some help in order to prepare my data gathered from the database for charting.
I have a Browser model where I store all the data.
From each model I want to select the name attribute, I will put a color according an integer attribute in the model (for example if integer is 1, color => "#4572A7") and the y attribute from the model.
Can someone provide an example of the most efficient way to achieve this data format?
Final format of the data:
[
{
:name=> 'Firefox',
:y=> 1,
:color => "#4572A7"
},
{
:name=> 'IE',
:y=> 1,
:color => "#AA4643"
},
{
:name=> 'Chrome',
:y=> 1,
:color => "#89A54E"
},
{
:name=> 'Safari',
:y=> 1,
:color => "#80699B"
},
{
:name=> 'Opera',
:y=> 1,
:color => "#3D96AE"
},
{
:name=> 'Others',
:y=> 1,
:color => "#DB843D"
}
]
You could have a method that does the number-to-color translation and have something like
#browsers.to_json(methods: [:the_method_that_translates_numbers_to_colors], only: [:name, :y])
Hope that helps,
NHI
Related
I am not sure that this is a valid question or not. I have started working on mongodb aggregation. I have to make a graph for the data on daily, weekly, monthly basis.
I am using "$dayOfMonth", "$week", "$month" to group by depending on the date provided. ex if from and to dates difference is less or equal to 6 I am grouping on daily basis using "$dayOfMonth",
If from and to dates difference is greater than 6 and less than 30 grouping is done of "$week" and if differece is greater than 30 then grouping is done on monthly basis "$month".
I am passing date in my "$match". Is it possible to push 0 as keys if the gouping is not present.
example - from_date = "01/01/2018" to_date = "30/6/2018"
so grouping will be done on month. and suppose if I dont have date for 3 and 4th & 5th month. I want to push 0 in the nested keys as the value.
output = [
{"_id": "01/01/2018", "counter":12},
{"_id": "01/02/2018", "counter": 15},
{"_id":"01/06/2018", counter: 10}
]
expected_output =
[
{"_id": "01/01/2018", "counter":12},
{"_id": "01/02/2018", "counter": 15},
{"_id":"01/03/2018", counter: 0},
{"_id":"01/04/2018", counter:0},
{"_id":"01/05/2018", counter: 0},
{"_id":"01/06/2018", counter: 10}
]
I am using Rails and Mongoid Gem.
Query That I am using
converted = Analytics::Conversion::PharmacyPrescription.collection.aggregate([
{ "$match" => {
"organisation_id" => org_id.to_s,
"date" => {
"$gte" => from_date,
"$lte" => to_date
},
"role_ids" => {"$in" => [role_id, "$role_ids"]}
}
},{
"$project" => {
"total_count" => 1,
"converted_count" => 1,
"not_converted_count" => 1,
"total_invoice_amount" => 1,
"user_id" => 1,
"facility_id" => 1,
"organisation_id" => 1,
"date" => 1,
}
},{
"$group" => {
"_id" => { "#{groupby}" => "$date" },
"total_count" => {"$sum" => "$total_count"},
"converted_count" => { "$sum" => "$converted_count" },
"not_converted_count" => { "$sum" => "$not_converted_count"},
}
}
]).to_a
The Aggregation Framework can only aggregate the documents you have. You are actually asking it to add groups for documents that do not exist, but it has no way to "know" which groups to add.
What I would do is run the query as you have it, and afterwards "spread" the date units according to the chosen granularity (in your example it will be 01/01/2018, 01/02/2018, 01/03/2018, 01/04/2018, 01/05/2018, 01/06/2018, and run a simple function which will add an entry for each missing unit.
I have an array of students objects (Student.all) and each object has a name and a class_id.
I want to find students with same class_id and concat their names into the same object, the others remain the same.
I mean, turn this:
[
{
id: 1,
name: 'Joe',
class_id: 55
},
{
id: 2,
name: 'Bill',
class_id: 55
},
{
id: 3,
name: 'Moe',
class_id: 70
},
{
id: 4,
name: 'Larry',
class_id: 80
},
{
id: 5,
name: 'Phill',
class_id: 80
}
]
Into this:
[
{
id: 1,
name: 'Joe/Bill',
class_id: 55
},
{
id: 3,
name: 'Moe',
class_id: 70
},
{
id: 4,
name: 'Larry/Phill',
class_id: 80
}
]
Assuming you're only after creating the array (as opposed to updating the DB) you can do something like this:
arr.group_by{|x| x[:class_id]}
.values.map{|x| x.reduce{|m,v| m[:name] = "#{m[:name]}/#{v[:name]}";m}}
If you care about the original array you can make a small change:
arr.group_by{|x| x[:class_id]}
.values.map{|x| x[1..-1].reduce(x.first){|m,v| m[:name] = "#{m[:name]}/#{v[:name]}";m}}
The result:
[
{:id=>1, :name=>"Joe/Joe/Bill", :class_id=>55},
{:id=>3, :name=>"Moe", :class_id=>70},
{:id=>4, :name=>"Larry/Larry/Phill", :class_id=>80}
]
My solution is to use Enumerable#group_by.
students = [...] # the source array
students.group_by(&:class_id)
# => {
55 => [
{
:id => 1,
:name => "Joe",
:class_id => 55
},
{
:id => 2,
:name => "Bill",
:class_id => 55
}
],
70 => [
{
:id => 3,
:name => "Moe",
:class_id => 70
}
],
80 => [
{
:id => 4,
:name => "Larry",
:class_id => 80
},
{
:id => 5,
:name => "Phil",
:class_id => 80
}
]
}
# Code in reduce block may need to be changed to fit your demand
students.group_by(&:class_id).reduce([]) do |ret, (k,v)|
st = v.first
st.name = v.map(&:name).join('/')
ret << st
end
# => target
I'm having some issues with parsing some JSON data into high charts using the Lazy Highcharts gem. I'm trying to select only the data from the last 7 days, or 1 week ago. At this stage my application just hangs and doesn't load with the code below.
I am loading the JSON data from a link.I have tried the pointStart option, but it doesn't seem to work.
Any help would be appreciated.
JSON
{"status": "ok", "data": [{"2014-06-16 16:00:00": 24.2},{"2014-06-17 12:00:00": 30.2},{"2014-06-18 17:00:00": 42.9}]} etc
Controller
#data = Oj.load(open(#temperature.url).read)
results = []
#data['data'].each do |data|
results << ((7.day.ago.to_i * 1000)..(Date.today.to_i * 1000)).map { |date| [DateTime.parse(data.keys.first).to_i * 1000 == date, data.values.first] }
end
#graph = LazyHighCharts::HighChart.new('graph') do |f|
f.chart(:height => '400', width: '860')
f.yAxis [:title => {:text => "Temperature, :margin => 20, style: { color: '#333'}}]
f.series(:type => 'line', :name => 'Temperature', pointStart: 7.day.ago.to_i * 1000, data: results, marker: {enabled: false}, :color => '#00463f' )
f.xAxis(:type => 'datetime', tickInterval: 1.day.to_i * 1000, :tickmarkPlacement => 'on', :startOnTick => true )
f.legend({:align => 'center', :verticalAlign => 'top', :y => 0, :borderWidth => 0, style: {color: "#333"}})
end
I have solved this. If any one is interested I added;
min: 1.weeks.ago.at_midnight.to_i * 1000
To the xAxis.
I am trying to use the Elasticsearch-Gem in my project. As I understand: By now there is no need for the Tire-Gem anymore, or am I wrong?
In my project I have a search (obivously), which currently applies to one model. Now I am trying to avoid wildcards, since they don't scale well, but I can't seem to get the ngram-Analyzers work properly. If I search for whole words, the search still works, but not for parts of it.
class Pictures < ActiveRecord::Base
include Elasticsearch::Model
include Elasticsearch::Model::Callbacks
settings :analysis => {
:analyzer => {
:my_index_analyzer => {
:tokenizer => "keyword",
:filter => ["lowercase", "substring"]
},
:my_search_analyzer => {
:tokenizer => "keyword",
:filter => ["lowercase", "substring"]
}
},
:filter => {
:substring => {
:type => "nGram",
:min_gram => 2,
:max_gram => 50
}
}
} do
mapping do
indexes :title,
:properties => {
:type => "string",
:index_analyzer => 'my_index_analyzer',
:search_analyzer => "my_search_analyzer"
}
Maybe somebody can give me a hint into the right direction.
I have given up on defining schema in the model class. In fact, it does not make much sense too.
So here is what I have done. A schema/mapping definition the db/ folder and a rake task to build it.
https://gist.github.com/geordee/9313f4867d61ce340a08
In the model
def as_indexed_json(options={})
self.as_json(only: [:id, :name, :description, :price])
end
I'm using an index for suggestions based on edgeNGram (like nGram, but always starting at the left side of the word) with this settings:
{
"en_suggestions": {
"settings": {
"index": {
"analysis": {
"filter": {
"tpNGramFilter": {
"min_gram": "4",
"type": "edgeNGram",
"max_gram": "50"
}
},
"analyzer": {
"tpNGramAnalyzer": {
"type": "custom",
"filter": [
"tpNGramFilter"
],
"tokenizer": "lowercase"
}
}
}
}
}
}
}
and this mapping:
{
"en_suggestions": {
"mappings": {
"suggest": {
"properties": {
"proposal": {
"type": "string",
"analyzer": "tpNGramAnalyzer"
}
}
}
}
}
}
I use the jdbc-river to fill my Elasticsearch instance from a PostgreSQL database. The river's record is created with the following ruby's code (since I query ES from a Rails app):
require 'elasticsearch'
client = Elasticsearch::Client.new
client.create :index => "_river", :type => "ldi", :id => "_meta", :body =>
{
:type => :jdbc,
:jdbc => {
:driver => "org.postgresql.Driver",
:url => "jdbc:postgresql://localhost:5432/" + ENV['DB_NAME'],
:user => ENV['DB_USER'],
:password => ENV['DB_PASS'],
:index => ENV['DB_NAME'],
:type => "ldi",
:sql => "select id as _id, * from ldis"
}
}
I'm using envirnoment variables for the database credentials to avoid showing the actual ones. The problem is that once the record is added to ES, actual credentials are unveiled. Thus, you can query ES and obtain something like this:
"hits": {
"total": 6,
"max_score": 1,
"hits": [
{
"_index": "_river",
"_type": "ldi",
"_id": "_meta",
"_score": 1,
"_source": {
"type": "jdbc",
"jdbc": {
"driver": "org.postgresql.Driver",
"url": "jdbc:postgresql://localhost:5432/any_dbname",
"user": "any_dbuser",
"password": "any_dbpass",
"index": "any_index",
"type": "ldi",
"sql": "select id as _id, * from ldis"
}
}
}
....
Is there any way to keep them in secret?