Unsupervised learning
(1)In unsupervised learning, data have no labels which means they don't have yi {i=1~m}
(2)Data has its own structure and a algorithm which can find clusters are called clustering.
K means algorithm
It's a algorithm that can separate the data set into several subsets with the same structure. Here're steps to implement it
suppose that you have m training examples.
Randomly initialize the u1,u2,u3... uK. K means you want to have k clusters
Repeat itself //it's a iterate algorithm so it's necessary to set how many times you should repeat
{
for i = 1:m
c(i) = min||x(i) - uk|| //the right part should return the index k
for i = 1:K
ui = the average of the same value c(i)'s x(i)
}
Optimization objective
In unsupervised learning we have the objective :
Randomly initialize u(k) { k = 1,2 , ......K}
(1)We should have K integer numbers { i(1) i(2) ...i(K) } , those numbers ranging from 1 to m.
(2)u(1) u(2) ... u(k) = x(i(1)) x(i(2)) x(i(K))
(3)We can plot the cost function correspond to this randomness and we should choose the least cost's initialization.
How many clusters should we choose?
(1)We can plot the cost function correspond to cluster number, then we can choose the elbow joint as our cluster number.
Dimensionality reduction
Dimensionality reduction can help us visualize the data and can compress the data and also can accelerate the computing process.
PCA algorithm
PCA algorithm can help us reduce the dimension from n to k which is smaller than n. In order to use the PCA, we need to know how we choose number k. Here are the steps:
First the x(i) may be used feature scaling first.
then :
We use PCA only when we have a strong proof that we need a higher computing speed or more disk space.
Remember that PCA can be used on supervised learning, and we can map the input to a lower dimension on the
training set ,
for the new input we can use the same mapping method.
Reconstruct from compressed representation
Remember that the result is only the approximation