Related
I was a bit puzzled to find the GeoJSON
{
"type": "Polygon",
"coordinates": [
[
[-0.8, -0.8],
[ 0.8, -0.8],
[ 0.8, 0.8],
[-0.8, 0.8],
[-0.8, -0.8]
],
[
[-0.6, -0.6],
[-0.6, 0.6],
[ 0.6, 0.6],
[ 0.6, -0.6],
[-0.6, -0.6]
],
[
[-0.4, -0.4],
[ 0.4, -0.4],
[ 0.4, 0.4],
[-0.4, 0.4],
[-0.4, -0.4]
]
]
}
an "island in a lake in an island" if you will, rejected by one online validator, but accepted by another.
Looking again at RFC7846
For Polygons with more than one of these rings, the first MUST be
the exterior ring, and any others MUST be interior rings. The
exterior ring bounds the surface, and the interior rings (if
present) bound holes within the surface.
it looks to me that the first validator was correct, the final ring does not bound a hole in the surface defined by the first ring, so is invalid. So am I right in thinking that a valid GeoJSON Polygon cannot represent an island in a lake in an island? (So one would need to use a MultiPolygon to represent it?)
Correct, the inner rings are holes in the exterior ring so cannot have an island in a lake where an inner ring is not a hole.
A valid representation of an island would be a MultiPolygon where the first polygon is the larger polygon with a hole that forms the lake and the second polygon is the smaller polygon inside the first which is the island.
Here is the GeoJSON:
{
"type": "Feature",
"geometry": {
"type": "MultiPolygon",
"coordinates": [
[
[
[-0.8, -0.8],
[ 0.8, -0.8],
[ 0.8, 0.8],
[-0.8, 0.8],
[-0.8, -0.8]
],
[
[-0.6, -0.6],
[-0.6, 0.6],
[ 0.6, 0.6],
[ 0.6, -0.6],
[-0.6, -0.6]
]
], [
[
[-0.4, -0.4],
[ 0.4, -0.4],
[ 0.4, 0.4],
[-0.4, 0.4],
[-0.4, -0.4]
]
]
]
}
}
I have a project I'm working on that involves rendering 3D models in WebGL, GitHub here. In pulling together several different resources, I've found two different formats for the model data: one with JSON entries like so:
var houseIFS =
{
"vertices": [
[ 2, -1, 2 ],
[ 2, -1, -2 ],
[ 2, 1, -2 ],
[ 2, 1, 2 ],
[ 1.5, 1.5, 0 ],
[ -1.5, 1.5, 0 ],
[ -2, -1, 2 ],
[ -2, 1, 2 ],
[ -2, 1, -2 ],
[ -2, -1, -2 ]
],
"faces": [
[ 0, 1, 2, 3 ],
[ 3, 2, 4 ],
[ 7, 3, 4, 5 ],
[ 2, 8, 5, 4 ],
[ 5, 8, 7 ],
[ 0, 3, 7, 6 ],
[ 0, 6, 9, 1 ],
[ 2, 1, 9, 8 ],
[ 6, 7, 8, 9 ]
],
"normals": [
[ 1, 0, 0 ],
[ 0.7071, 0.7071, 0 ],
[ 0, 0.9701, 0.2425 ],
[ 0, 0.9701, -0.2425 ],
[ -0.7071, 0.7071, 0 ],
[ 0, 0, 1 ],
[ 0, -1, 0 ],
[ 0, 0, -1 ],
[ -1, 0, 0 ]
],
"faceColors": [
[ 1, .8, .8 ],
[ .7, .7, 1 ],
[ 0, 0, 1 ],
[ 0, 0, .7 ],
[ .7, .7, 1 ],
[ 1, 0, 0 ],
[ .4, .4, .4 ],
[ 1, 0, 0 ],
[ 1, .8, .8 ],
]
};
and another with more primitive return types:
/** The return value of each function is an object, model, with properties:
*
* model.vertexPositions -- the vertex coordinates;
* model.vertexNormals -- the normal vectors;
* model.vertexTextureCoords -- the texture coordinates;
* model.indices -- the face indices.
*
* The first three properties are of type Float32Array, while
* model.indices is of type Uint16Array.
*/
I tried to create a method to convert the data from the "modern" version to the "primitive":
function convertPoly(model) {
return {
vertexPositions: new Float32Array(model.vertices),
vertexNormals: new Float32Array(model.normals),
vertexTextureCoords: new Float32Array(model.faces),
indices: new Uint16Array(model.faces)
}
}
but I don't think this is correct, and I don't see anything rendered after trying to render it. How can I compute indices from the vertices or faces? I guess I don't really understand what the indices really represent or how they work (is it triangle vertices of the faces?).
I am new to material UI but I am learning, one thing that I was looking into is breakpoints.
Now, this is what my breakpoints look like.
breakpoints: createBreakpoints({
keys: [
"xs",
"sm",
"md",
"lg",
"xl",
],
values: {
xs: 0,
sm: 600,
md: 960,
lg: 1280,
xl: 1920,
},
}),
I did a console.log(theme.breakpoints.between('xs', 'sm'), and the output I got surprised me since it was 0 to 960, I don't understand. Shouldn't it be 0 to 600?
So, I did console.log(theme.breakpoints.between('xs', 'xs') and the output i got was 0 to 600.
Why is this happening and how is this even logical?
I am so confused.
I am using optuna to tune xgboost model's hyperparameters. I find it stuck at trial 2 (trial_id=3) for a long time(244 minutes). But When I look at the SQLite database which records the trial data, I find all the trial 2 (trial_id=3) hyperparameters has been calculated except the mean squared error value of trial 2. And the optuna trial 2 (trial_id=3) seems stuck at that step. I want to know why this happened? And how to fix the issue?
Here is the code
def xgb_hyperparameter_tuning():
def objective(trial):
params = {
"n_estimators": trial.suggest_int("n_estimators", 1000, 10000, step=100),
"booster": trial.suggest_categorical("booster", ["gbtree", "gblinear", "dart"]),
"max_depth": trial.suggest_int("max_depth", 1, 20, step=1),
"learning_rate": trial.suggest_float("learning_rate", 0.0001, 0.2, step=0.001),
"min_child_weight": trial.suggest_float("min_child_weight", 1.0, 20.0, step=1.0),
"colsample_bytree": trial.suggest_float("colsample_bytree", 0.1, 1.0, step=0.1),
"subsample": trial.suggest_float("subsample",0.1, 1.0, step=0.1),
"reg_alpha": trial.suggest_float("reg_alpha", 0.0, 11.0, step=0.1),
"reg_lambda": trial.suggest_float("reg_lambda", 0.0, 11.0, step=0.1),
"num_parallel_tree": 10,
"random_state": 16,
"n_jobs": 10,
"early_stopping_rounds": 1000,
}
model = XGBRegressor(**params)
mse = make_scorer(mean_squared_error)
cv = cross_val_score(estimator=model, X=X_train, y=log_y_train, cv=20, scoring=mse, n_jobs=-1)
return cv.mean()
study = optuna.create_study(study_name="HousePriceCompetitionXGB", direction="minimize", storage="sqlite:///house_price_competition_xgb.db", load_if_exists=True)
study.optimize(objective, n_trials=100,)
return None
xgb_hyperparameter_tuning()
Here is the output
[I 2021-11-16 10:06:27,522] A new study created in RDB with name: HousePriceCompetitionXGB
[I 2021-11-16 10:08:40,050] Trial 0 finished with value: 0.03599314763859092 and parameters: {'n_estimators': 5800, 'booster': 'gblinear', 'max_depth': 4, 'learning_rate': 0.1641, 'min_child_weight': 17.0, 'colsample_bytree': 0.4, 'subsample': 0.30000000000000004, 'reg_alpha': 10.8, 'reg_lambda': 7.6000000000000005}. Best is trial 0 with value: 0.03599314763859092.
[I 2021-11-16 10:11:55,830] Trial 1 finished with value: 0.028514652199592445 and parameters: {'n_estimators': 6600, 'booster': 'gblinear', 'max_depth': 17, 'learning_rate': 0.0821, 'min_child_weight': 20.0, 'colsample_bytree': 0.7000000000000001, 'subsample': 0.2, 'reg_alpha': 1.2000000000000002, 'reg_lambda': 7.2}. Best is trial 1 with value: 0.028514652199592445.
Here is the sqlite database trial_values table's data
trial_value_id
trial_id
objective
value
1
1
0
0.0359931476385909
2
2
0
0.0285146521995924
Here is the sqlite database trial_params table's data And you can see all the trial 2 (trial_id=3) hyperparameters has been calculated
param_id
trial_id
param_name
param_value
distribution_json
1
1
n_estimators
5800.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
2
1
booster
1.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
3
1
max_depth
4.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
4
1
learning_rate
0.1641
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
5
1
min_child_weight
17.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
6
1
colsample_bytree
0.4
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
7
1
subsample
0.3
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
8
1
reg_alpha
10.8
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
9
1
reg_lambda
7.6
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
10
2
n_estimators
6600.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
11
2
booster
1.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
12
2
max_depth
17.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
13
2
learning_rate
0.0821
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
14
2
min_child_weight
20.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
15
2
colsample_bytree
0.7
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
16
2
subsample
0.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
17
2
reg_alpha
1.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
18
2
reg_lambda
7.2
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
19
3
n_estimators
7700.0
{"name": "IntUniformDistribution", "attributes": {"low": 1000, "high": 10000, "step": 100}}
20
3
booster
2.0
{"name": "CategoricalDistribution", "attributes": {"choices": ["gbtree", "gblinear", "dart"]}}
21
3
max_depth
4.0
{"name": "IntUniformDistribution", "attributes": {"low": 1, "high": 20, "step": 1}}
22
3
learning_rate
0.1221
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0001, "high": 0.1991, "q": 0.001}}
23
3
min_child_weight
3.0
{"name": "DiscreteUniformDistribution", "attributes": {"low": 1.0, "high": 20.0, "q": 1.0}}
24
3
colsample_bytree
0.5
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
25
3
subsample
0.1
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.1, "high": 1.0, "q": 0.1}}
26
3
reg_alpha
10.8
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
27
3
reg_lambda
1.1
{"name": "DiscreteUniformDistribution", "attributes": {"low": 0.0, "high": 11.0, "q": 0.1}}
Although I am not 100% sure, I think I know what happened.
This issue happens because some parameters are not suitable for certain booster type and the trial will return nan as result and be stuck at the step - calculating the MSE score.
To solve the problem, you just need to delete the "booster": "dart".
In other words, using "booster": trial.suggest_categorical("booster", ["gbtree", "gblinear"]), rather than "booster": trial.suggest_categorical("booster", ["gbtree", "gblinear", "dart"]), can solve the problem.
I got the idea when I tuned my LightGBMRegressor Model. I found many trials fail because these trials returned nan and they all used the same "boosting_type"="rf". So I deleted the rf and all 100 trials were completed without any error. Then I looked for the XGBRegressor issue which I posted above. I found all the trials which were stuck had the same "booster":"dart" either. So I deleted the dart, and the XGBRegressor run normally.
I am getting the following error while training my model with more than 1 epoch
[02/06/2019 13:37:08 WARNING 140231582721856] Expected number of batches: 15, did not match the number of batches processed: 16. This may happen when some images or annotations are invalid and cannot be parsed. Please check the dataset and ensure it follows the format in the documentation.
[02/06/2019 13:37:08 INFO 140231582721856] #quality_metric: host=algo-1, epoch=24, batch=16 train cross_entropy <loss>=(nan)
[02/06/2019 13:37:08 INFO 140231582721856] #quality_metric: host=algo-1, epoch=24, batch=16 train smooth_l1 <loss>=(nan)
[02/06/2019 13:37:08 INFO 140231582721856] Round of batches complete
[02/06/2019 13:37:08 INFO 140231582721856] Updated the metrics
[02/06/2019 13:37:08 INFO 140231582721856] #quality_metric: host=algo-1, epoch=24, validation mAP <score>=(0.0)
[02/06/2019 13:37:08 INFO 140231582721856] #progress_metric: host=algo-1, completed 83 % of epochs
#metrics {"Metrics": {"Max Batches Seen Between Resets": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Number of Batches Since Last Reset": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Number of Records Since Last Reset": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Total Batches Seen": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Total Records Seen": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Max Records Seen Between Resets": {"count": 1, "max": 0, "sum": 0.0, "min": 0}, "Reset Count": {"count": 1, "max": 25, "sum": 25.0, "min": 25}}, "EndTime": 1549460228.963195, "Dimensions": {"Host": "algo-1", "Meta": "training_data_iter", "Operation": "training", "Algorithm": "AWS/Object Detection", "epoch": 24}, "StartTime": 1549460224.644808}
Following is the code that I used
for estimator
od_model = sagemaker.estimator.Estimator(training_image,
role,
train_instance_count=1,
train_instance_type='ml.p3.8xlarge',
train_volume_size = 500,
train_max_run = 300000,
input_mode= 'File',
output_path=s3_output_location,
sagemaker_session=sess)
And for hyperparameters
od_model.set_hyperparameters(base_network='resnet-50',
use_pretrained_model=0,
num_classes=1,
mini_batch_size=32,
epochs=30,
learning_rate=0.001,
lr_scheduler_step='3,6',
lr_scheduler_factor=0.1,
optimizer='sgd',
momentum=0.9,
weight_decay=0.0005,
overlap_threshold=0.5,
nms_threshold=0.45,
image_shape=512,
label_width=360,
num_training_samples=500)
But bounding boxes appear fine if I keep epoch to 1 although the output model does not detect properly and creates boxes everywhere
With the above code the final model does not create any bounding boxes
Two training losses are 'nan' and the validation mAP is 0. This means the model was not trained properly. Try to tune the 'learning_rate' and 'batch_size' hyperparameters. For a small dataset (500 images), you can use the transfer learning feature by setting 'use_pretrained_model=1'.