I am running SLIC Superpixel segmentation using skimage. I would like to get the segment results as a .shp file format with associated coordinate NZTM. The segmentation result is numpy.ndarray with 2 dimensions.
My original imagery has the NZTM - New Zealand Transverse Mercator. How to do this?
Below are my script
import arcpy
import skimage
import skimage.io as skio
from skimage.segmentation import slic
from PIL import Image
import matplotlib.pyplot as plt
from shapely.geometry import shape, Point, Polygon, LineString
import numpy as np
from skimage.color import rgb2gray
from skimage.filters import sobel
from skimage.segmentation import slic
from skimage.segmentation import mark_boundaries
from skimage.util import img_as_float
#############Load the image
im1 = skio.imread(r'C:\Data\Deep_Learning\SLICO\kang1019_clip1.png')
segments_slic = slic(im1, n_segments=400, compactness=10, sigma=1,
start_label=1)
Thank you for your help.
Related
import os
import cv2
import numpy as np
from PIL import Image
class training:
def train():
reconizer = cv2.face.LBPHFaceRecognizer_create()
\\ some code part
reconizer.train(faces,Id) \\\\ <----- this line
reconizer.save('reconizer/traindata.yml')
cv2.destroyAllWindows()
What is exactly getting trained in this step. is it some machine learning algorithm going behind or else ?
I am running SLIC Superpixel segmentation using skimage. I would like to get the segment results as a geojson file with associated coordinate NZTM.
My original imagery has the NZTM - New Zealand Transverse Mercator. How to do this?
Below are my script
import arcpy
import skimage
import skimage.io as skio
from skimage.segmentation import slic
import matplotlib.pyplot as plt
import numpy as np
from skimage.color import rgb2gray
from skimage.filters import sobel
from skimage.segmentation import felzenszwalb, slic, quickshift, watershed
from skimage.segmentation import mark_boundaries
from skimage.util import img_as_float
#############Load the image
im1 = skio.imread(r'C:\Data\SLICO\clip1.png')
segments_slic = slic(im1, n_segments=400, compactness=10, sigma=1,
start_label=1)
plt.imshow(mark_boundaries(im1, segments_slic))
plt.show()
Thank you for your help.
How can I do dask_ml preprocessing in a dask distributed cluster? My dataset is about 200GB and Every time I categorize the dataset preparing for OneHotEncoding, it looks like dask is ignoring the client and try to load the dataset in the local machine's memory. Maybe I miss something:
from dask_ml.preprocessing import Categorizer, DummyEncoder
from sklearn.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
import pandas as pd
import dask.dataframe as dd
df = dd.read_csv('s3://some-bucket/files*.csv', dtypes={'column': 'category'})
pipe = make_pipeline(
Categorizer(),
DummyEncoder(),
LogisticRegression(solver='lbfgs')
)
pipe.fit(df, y)
Two immediate things to address:
You have not instantiated a distributed scheduler in your code.
You should probably use the LogisticRegression instance from
dask-ml rather than scikit-learn.
Working Code Example
Below is a minimal code example that works.
Note that the preprocessing functions accept only Dask Dataframes while the LogisticRegression estimator accepts only Dask arrays. You can split the pipeline or use a custom FunctionTransformer (from this answer). See this open Dask issue for more context.
from dask_ml.preprocessing import Categorizer, DummyEncoder
from dask_ml.linear_model import LogisticRegression
from sklearn.pipeline import make_pipeline
from sklearn.preprocessing import FunctionTransformer
import pandas as pd
import dask.dataframe as dd
from dask.distributed import Client
client = Client()
from dask_ml.datasets import make_classification
X, y = make_classification(chunks=50)
# define custom transformers to include in pipeline
def trans_array(array):
return dd.from_array(array)
transform_array = FunctionTransformer(trans_array)
def trans_df(dataframe):
return dataframe.to_dask_array(lengths=True)
transform_df = FunctionTransformer(trans_df)
pipe = make_pipeline(
transform_array,
Categorizer(),
DummyEncoder(),
transform_df,
LogisticRegression(solver='lbfgs')
)
pipe.fit(X,y)
"I have scaled my dataset by using Standard Scaler , Now how to know it has been scaled, I am sure it has been scaled but how to see it"
As #Coderji said you can always find out the mean and standard deviation, which should be equal to 0 and 1 respectively.
However, there is another method to visualize it.
from sklearn import datasets
import numpy as np
from sklearn.preprocessing import StandardScaler
I am using iris dataset for this example.
iris = datasets.load_iris()
X = iris.data
sc = StandardScaler()
sc.fit(X)
x = sc.transform(X)
import matplotlib.pyplot as plt
import seaborn as sns
sns.distplot(x[:,1])
See this Output for sepel length
Similarly you can see for all variables or a simple pairplot will do the job.
This gives an idea that the data is standardised visually.
I'm trying to use Keras' Functional API as I need to add additional input into my recurrent neural net between two layers. This would be fine however I also have input I want to forecast and so I'm using a masking layer to keep the out of sample data from affecting the model. The masking layer throws an error when I try to pass it an Input(...) as it's expecting a list of integers not tensor variables. Is there a specific masking layer for the Functional API? Here is my code:
import numpy
import pandas
import math
from keras.models import Sequential
from keras.layers import Dense
from keras.layers import LSTM
from keras.layers import GRU
from keras.layers import SimpleRNN
from keras.layers.core import Masking
from keras.optimizers import Adam
from keras.optimizers import RMSprop
from keras.optimizers import Nadam
from keras.layers import TimeDistributed
from keras import initializations
from keras.layers import Input, merge
from keras.models import Model
dataframe = pandas.read_csv('C:/Users/RNCZF01/Documents/Cameron-Fen/Economic Ideas/LSTM/LSTM-data/GDP+stockmarket.csv', usecols=[1,3], header=None, engine='python')
dataset = dataframe.values
dataset = dataset.astype('float32')
dataframe1 = pandas.read_csv('C:/Users/RNCZF01/Documents/Cameron-Fen/Economic Ideas/LSTM/LSTM-data/GDP+stockmarket.csv', usecols=[0], header=None, engine='python')
dataset1 = dataframe1.values
dataset1 = dataset1.astype('float32')
train, test = dataset[start:train_size+test_size,:]*mult, dataset1[start:train_size+test_size,:]*mult
#set the masking to 0.0
for each in range(test_size):
train[train_size + each,:] = [0.0,0.0]
train, test = train[:259], test[:259]
validx, validy = dataset[start:train_size+test_size,:]*mult, dataset1[start:train_size+test_size,:]
main_input = Input(shape=(259,2), name='main_input')
m = Masking(mask_value=0.0)(main_input)#error is her because masking expects indices to be integers not a tensor variable
Here is the data. Use columns 0 as the test data and columns 1 and 3 as training data in the code.