I am trying to get a list of events from the Eventbrite API, and sort them by date.
However I seem to get an event list that is not ordered by date.
Here is an example:
$ curl "https://www.eventbrite.com/json/event_search?app_key=MY_API_KEY&country=GB&sort_by=date" | jsonlint | grep start_date
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 68722 0 68722 0 0 42483 0 --:--:-- 0:00:01 --:--:-- 61413
"start_date": "2013-03-23 09:00:00",
"start_date": "2013-03-23 09:00:00",
"start_date": "2013-03-31 13:00:00",
"start_date": "2013-01-18 01:45:00",
"start_date": "2013-03-28 09:00:00",
"start_date": "2013-02-01 05:55:00",
"start_date": "2013-02-01 05:55:00",
"start_date": "2013-02-01 05:55:00",
"start_date": "2013-03-23 10:00:00",
"start_date": "2013-01-01 00:00:00",
"start_date": "2012-12-12 00:00:00",
"start_date": "2013-04-09 19:00:00",
"start_date": "2013-04-13 09:00:00",
"start_date": "2013-04-17 18:15:00",
"start_date": "2013-04-17 19:00:00",
"start_date": "2013-04-17 19:00:00",
(The non-indented start_times are those for events)
I know it is accepting the sort_by=date parameter as it is returned in the results summary:
{
"summary": {
"total_items": 14911,
"first_event": 4673700163,
"last_event": 5844441883,
"filters": {
"country": "GB",
"sort_by": "date"
},
"num_showing": 10
}
},
I've assumed the date it sorts by is the start_date, but manually inspecting the other dates (end_date, created, modified) it doesn't seem to be sorting by that either.
Is this a bug in the API or am I doing something wrong?
afaik, Eventbrite allows you to sort by location OR by date, but not both.
It's possible that since you selected a country / region (GB), the date sorting is not being applied.
Related
I am using optuna to tune xgboost model's hyperparameters. I find it stuck at trial 2 (trial_id=3) for a long time(244 minutes). But When I look at the SQLite database which records the trial data, I find all the trial 2 (trial_id=3) hyperparameters has been calculated except the mean squared error value of trial 2. And the optuna trial 2 (trial_id=3) seems stuck at that step. I want to know why this happened? And how to fix the issue?
Here is the code
def xgb_hyperparameter_tuning():
def objective(trial):
params = {
"n_estimators": trial.suggest_int("n_estimators", 1000, 10000, step=100),
"booster": trial.suggest_categorical("booster", ["gbtree", "gblinear", "dart"]),
"max_depth": trial.suggest_int("max_depth", 1, 20, step=1),
"learning_rate": trial.suggest_float("learning_rate", 0.0001, 0.2, step=0.001),
"min_child_weight": trial.suggest_float("min_child_weight", 1.0, 20.0, step=1.0),
"colsample_bytree": trial.suggest_float("colsample_bytree", 0.1, 1.0, step=0.1),
"subsample": trial.suggest_float("subsample",0.1, 1.0, step=0.1),
"reg_alpha": trial.suggest_float("reg_alpha", 0.0, 11.0, step=0.1),
"reg_lambda": trial.suggest_float("reg_lambda", 0.0, 11.0, step=0.1),
"num_parallel_tree": 10,
"random_state": 16,
"n_jobs": 10,
"early_stopping_rounds": 1000,
}
model = XGBRegressor(**params)
mse = make_scorer(mean_squared_error)
cv = cross_val_score(estimator=model, X=X_train, y=log_y_train, cv=20, scoring=mse, n_jobs=-1)
return cv.mean()
study = optuna.create_study(study_name="HousePriceCompetitionXGB", direction="minimize", storage="sqlite:///house_price_competition_xgb.db", load_if_exists=True)
study.optimize(objective, n_trials=100,)
return None
xgb_hyperparameter_tuning()
Here is the output
[I 2021-11-16 10:06:27,522] A new study created in RDB with name: HousePriceCompetitionXGB
[I 2021-11-16 10:08:40,050] Trial 0 finished with value: 0.03599314763859092 and parameters: {'n_estimators': 5800, 'booster': 'gblinear', 'max_depth': 4, 'learning_rate': 0.1641, 'min_child_weight': 17.0, 'colsample_bytree': 0.4, 'subsample': 0.30000000000000004, 'reg_alpha': 10.8, 'reg_lambda': 7.6000000000000005}. Best is trial 0 with value: 0.03599314763859092.
[I 2021-11-16 10:11:55,830] Trial 1 finished with value: 0.028514652199592445 and parameters: {'n_estimators': 6600, 'booster': 'gblinear', 'max_depth': 17, 'learning_rate': 0.0821, 'min_child_weight': 20.0, 'colsample_bytree': 0.7000000000000001, 'subsample': 0.2, 'reg_alpha': 1.2000000000000002, 'reg_lambda': 7.2}. Best is trial 1 with value: 0.028514652199592445.
Here is the sqlite database trial_values table's data
trial_value_id
trial_id
objective
value
1
1
0
0.0359931476385909
2
2
0
0.0285146521995924
Here is the sqlite database trial_params table's data And you can see all the trial 2 (trial_id=3) hyperparameters has been calculated
param_id
trial_id
param_name
param_value
distribution_json
1
1
n_estimators
5800.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
2
1
booster
1.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
3
1
max_depth
4.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
4
1
learning_rate
0.1641
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
5
1
min_child_weight
17.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
6
1
colsample_bytree
0.4
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
7
1
subsample
0.3
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
8
1
reg_alpha
10.8
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
9
1
reg_lambda
7.6
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
10
2
n_estimators
6600.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
11
2
booster
1.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
12
2
max_depth
17.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
13
2
learning_rate
0.0821
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
14
2
min_child_weight
20.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
15
2
colsample_bytree
0.7
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
16
2
subsample
0.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
17
2
reg_alpha
1.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
18
2
reg_lambda
7.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
19
3
n_estimators
7700.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
20
3
booster
2.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
21
3
max_depth
4.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
22
3
learning_rate
0.1221
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
23
3
min_child_weight
3.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
24
3
colsample_bytree
0.5
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
25
3
subsample
0.1
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
26
3
reg_alpha
10.8
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
27
3
reg_lambda
1.1
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
Although I am not 100% sure, I think I know what happened.
This issue happens because some parameters are not suitable for certain booster type and the trial will return nan as result and be stuck at the step - calculating the MSE score.
To solve the problem, you just need to delete the "booster": "dart".
In other words, using "booster": trial.suggest_categorical("booster", ["gbtree", "gblinear"]), rather than "booster": trial.suggest_categorical("booster", ["gbtree", "gblinear", "dart"]), can solve the problem.
I got the idea when I tuned my LightGBMRegressor Model. I found many trials fail because these trials returned nan and they all used the same "boosting_type"="rf". So I deleted the rf and all 100 trials were completed without any error. Then I looked for the XGBRegressor issue which I posted above. I found all the trials which were stuck had the same "booster":"dart" either. So I deleted the dart, and the XGBRegressor run normally.
I'm drawing a stockChart with Highchart, here the last data set:
[1582844400000,32.65863464265468],
[1583449200000,29.571543938725608],
[1584054000000,18.869729784689593]
1584054000000 is Fri 13 March 2020
1583449200000 is Fri 6 March 2020
Now when I plot the graph, the last value is shown as Mon 9 March 2020. Which from what I understand is correct because is the default value for the properties startOfWeek.
So I set this:
xAxis: {
type: 'datetime',
startOnTick: true,
startOfWeek: 5
},
Thinking to set the week correctly to Friday, indeed it works, but it shows me Fri 6 with value 18,86 so it's Fri 13, but for some reason it a week before. If I set startIfWeek at 4, it shows me Th 12.
From docs page
startOfWeek: number
For dateTime axes, this decides where to put the tick between weeks. 0 = Sunday, 1 = Monday.
Defaults to 1.
I was testing with this example from documentation, and it seems that from 0 to 4 it goes up on same week, but for some reason from 5 and 6 it goes back to previous week. Maybe it sets the average distance so the tick is not too distant from origin date.
xAxis: {
startOfWeek: 6,
type: 'datetime',
labels: {
formatter: function () {
return Highcharts.dateFormat('%a %e %b', this.value);
}
}
},
series: [{
data: [29.9, 71.5, 106.4, 129.2, 144.0, 176.0, 135.6, 148.5, 216.4, 194.1, 95.6, 54.4],
pointStart: Date.UTC(2010, 0, 1),
pointInterval: 3 * 24 * 3600 * 1000
}]
We are using KSQL to perform some aggregations / filtering on a real time data.
One of the use case we have is, we need to perform some operation on last N days of a particular activity, this would be continuous operation.
So this needs to be hopping window.
When I tried the query, with hopping duration as M days, KSQL query returned M records instead of 1 (which was hoping for).
Query :
select PROTO,
TIMESTAMPTOSTRING(WindowStart(), 'yyyy-MM-dd''T''HH:mm:ss''Z''', 'UTC') as "timestamp",
TIMESTAMPTOSTRING(WindowEnd(), 'yyyy-MM-dd''T''HH:mm:ss''Z''', 'UTC'),
COUNT(PROTO) AS Count
FROM DATASTREAM
WINDOW HOPPING (SIZE 5 DAYS, ADVANCE BY 1 DAY)
WHERE MSG like '%SOMESTRING%'
AND SPLIT(PROTO, '/')[0] = 'tcp'
GROUP BY PROTO;
tcp/22 | 2020-01-27T00:00:00Z | 2020-02-01T00:00:00Z | 1
tcp/22 | 2020-01-28T00:00:00Z | 2020-02-02T00:00:00Z | 1
tcp/22 | 2020-01-29T00:00:00Z | 2020-02-03T00:00:00Z | 1
tcp/22 | 2020-01-30T00:00:00Z | 2020-02-04T00:00:00Z | 1
tcp/22 | 2020-01-31T00:00:00Z | 2020-02-05T00:00:00Z | 1
Is there any way to get only first record only, or the records for which end time <= current time or any other workaround to get 1 result per window ?
Please consider below data records.
{ "time": "2020-01-25 23:36:37 UTC", "msg": "Error"}
{ "time": "2020-01-25 23:36:38 UTC", "msg": "Error"}
{ "time": "2020-01-25 23:36:40 UTC", "msg": "Error"}
{ "time": "2020-01-26 23:36:37 UTC", "msg": "Error"}
{ "time": "2020-01-26 23:36:38 UTC", "msg": "Error"}
{ "time": "2020-01-26 23:36:39 UTC", "msg": "Error"}
{ "time": "2020-01-26 23:36:40 UTC", "msg": "Error"}
{ "time": "2020-01-27 23:36:37 UTC", "msg": "Error"}
{ "time": "2020-01-27 23:36:38 UTC", "msg": "Error"}
{ "time": "2020-01-27 23:36:39 UTC", "msg": "Error"}
{ "time": "2020-01-28 23:36:37 UTC", "msg": "Error"}
{ "time": "2020-01-28 23:36:38 UTC", "msg": "Error"}
{ "time": "2020-01-29 23:36:37 UTC", "msg": "Error"}
{ "time": "2020-01-29 23:36:38 UTC", "msg": "Error"}
{ "time": "2020-01-29 23:36:39 UTC", "msg": "Error"}
{ "time": "2020-01-29 23:36:40 UTC", "msg": "Error"}
I am looking for count of records which have, msg as Error past 2 days.
If I fire the KSQL query on 25th at 23:36:37, I would be expecting result as :
2020-01-25T23:36:37Z | 1
2020-01-25T23:36:38Z | 2
2020-01-25T23:36:40Z | 3
2020-01-26T23:36:37Z | 4
2020-01-26T23:36:38Z | 5
2020-01-26T23:36:39Z | 6
2020-01-26T23:36:40Z | 7
2020-01-27T23:36:37Z | 5
2020-01-27T23:36:38Z | 6
2020-01-27T23:36:39Z | 7
2020-01-28T23:36:37Z | 4
2020-01-28T23:36:38Z | 5
2020-01-29T23:36:37Z | 3
2020-01-29T23:36:38Z | 4
2020-01-29T23:36:39Z | 5
2020-01-29T23:36:40Z | 6
I think you need a TUMBLING window instead if you want one window for the five day period. You're instead getting five windows because you've used HOPPING with an advance of 1 DAY - see the WINDOWSTART() changes per day.
References:
Hopping windows twitter thread
Tumbling windows twitter thread
https://docs.ksqldb.io/en/latest/concepts/time-and-windows-in-ksqldb-queries/
https://rmoff.net/2020/01/09/exploring-ksqldb-window-start-time/
So I got this array of object from database and I have to sort it in order:
by score (reverse)
by count (reverse)
by name
I've tried all three in reverse but what I need is the last one should not be reverse (DESC) here's my code:
new_data = data.sort_by{ |t| [t.score, t.matches_count, t.name] }.reverse
RESULT
[
{
"id": null,
"team_id": 939,
"name": "DAV",
"matches_count": 2,
"score": 100.0
},
{
"id": null,
"team_id": 964,
"name": "SAN",
"matches_count": 1,
"score": 100.0
},
{
"id": null,
"team_id": 955,
"name": "PAS",
"matches_count": 1,
"score": 100.0
},
{
"id": null,
"team_id": 954,
"name": "PAR",
"matches_count": 1,
"score": 100.0
},
{
"id": null,
"team_id": 952,
"name": "NUE",
"matches_count": 1,
"score": 100.0
}
]
Expected result should be sorted by name in ASC order not DESC I knew my code is wrong because the t.name is inside .reverse but if I will reorder it by name alone after the first 2 I will get the wrong answer it will just sort all by name not by the 3. I've also tried to .order("name DESC") from query so when reverse it will go ASC but no luck. Thank you!
data = [{:score=>100.0, :matches_count=>2, :name=>"DAV"},
{:score=>100.0, :matches_count=>1, :name=>"SAN"},
{:score=>100.0, :matches_count=>1, :name=>"PAS"},
{:score=>110.0, :matches_count=>1, :name=>"PAR"},
{:score=>100.0, :matches_count=>1, :name=>"NUE"}]
data.sort_by{ |h| [-h[:score], -h[:matches_count], h[:name]] }
#=> [{:score=>110.0, :matches_count=>1, :name=>"PAR"},
# {:score=>100.0, :matches_count=>2, :name=>"DAV"},
# {:score=>100.0, :matches_count=>1, :name=>"NUE"},
# {:score=>100.0, :matches_count=>1, :name=>"PAS"},
# {:score=>100.0, :matches_count=>1, :name=>"SAN"}]
You could also use Array#sort, which does not require that values to be sorted on in descending order be numeric, so long as they are comparable, that is, so long as they respond to the method :<=>.
data.sort do |t1, t2|
case t1[:score] <=> t2[:score]
when -1
1
when 1
-1
else
case t1[:matches_count] <=> t2[:matches_count]
when -1
1
when 1
-1
else
t1[:name] <=> t2[:name]
end
end
end
#=> <as above>
Instead of using - one can also try to reverse the sort using ! as demonstrated here: https://www.ruby-forum.com/t/sort-by-multiple-fields-with-reverse-sort/197361/2
So in your case, you could do:
new_data = data.sort_by{ |t| [!t.score, !t.matches_count, t.name] }
which should give you the same result as intended.
I am currently working with Highcharts in combination with the pattern fill module. When I set a pattern for a series in the chart, the pattern is shown but it has a transparent background. I need to set an additional background because the pattern is overlapping with another series which I don't want to see behind it. You can check this fiddle. So basically I don't want to see those three columns on the left behind the pattern. Any ideas how I can do that? I haven't seen any options to set an additional background, but maybe you know some trick. This is the code I am using for the pattern:
"color": {
"pattern": {
"path": {
"d": "M 0 0 L 10 10 M 9 -1 L 11 1 M -1 9 L 1 11"
},
"width": 10,
"height": 10,
"opacity": 1,
"color": "rgb(84,198,232)"
}
}
You need to set fill attribute as a path property:
"color": {
"pattern": {
"path": {
"d": "M 0 0 L 10 10 M 9 -1 L 11 1 M -1 9 L 1 11",
fill: 'red'
},
"width": 10,
"height": 10,
"opacity": 1,
"color": 'rgb(84,198,232)'
}
}
Live demo: https://jsfiddle.net/BlackLabel/m9rxwej5/
I guess there's been an update. backgroundColor should be set at pattern's root level:
"color": {
"pattern": {
"backgroundColor": 'red',
"path": {
"d": "M 0 0 L 10 10 M 9 -1 L 11 1 M -1 9 L 1 11",
},
"width": 10,
"height": 10,
"opacity": 1,
"color": 'rgb(84,198,232)',
}
}
https://jsfiddle.net/vL4fqhao/