PCA algorithm

Principal Component Analysis is a dimensionality reduction technique commonly used in machine learning and data analysis. Its primary goal is to transform high-dimensional data into a lower-dimensional representation, capturing the most important information. This reduction in dimensionality can lead to improved computational efficiency, visualization, and often better model performance.

steps to perform PCA:

  1. Standardize the Data: If the features in the dataset have different scales, it is important to standardize them by subtracting the mean and dividing by the standard deviation to give each feature equal importance.
  2. Compute the Covariance Matrix: Calculate the covariance matrix of the standardized data. The covariance matrix provides information about how variables change together.
  3. Compute Eigenvectors and Eigenvalues: Calculate the eigenvectors and eigenvalues of the covariance matrix. The eigenvectors represent the principal components, and the eigenvalues indicate the amount of variance captured by each principal component.
  4. Sort Eigenvectors by Eigenvalues: Sort the eigenvectors in descending order based on their corresponding eigenvalues. The higher the eigenvalue, the more variance is captured by the corresponding eigenvector.
  5. Select Principal Components: Choose the top k eigenvectors to form the new feature space. Typically, it would select the number of principal components that capture a sufficiently high percentage of the total variance.


Suppose the data on people’s heights and weights. We can find that most of the variation, along a diagonal line, representing a combination of height and weight. PCA helps us focus on this main trend and ignore less important details.

Leave a Reply

Your email address will not be published. Required fields are marked *