Skip to content

diem-ai/datascience-projects

Repository files navigation

  • Text Classification
    • Analyse the data and create new features
    • Transform text data into Term Frequency - Inverse Document Frequency, select the best feature with f_classif and fit the transformed data to Bayesian algorithm
    • Transform text data into Word Embedding, select the best feature with f_classif and fit the transformed data to Bayesian algorithm
    • Word Embedding + Bayes improves 8% accuracy from 86% (baseline) to 94% Meanwhile TF-IDF + Bayes improve 5% accuracy from 86% to 91%
    • Alt Blog Text Classification with TF-IDF, Word Embedding and Naive Bayes

  • Wine Clustering

    • Clustering wine color from their chemical properties. Using KMeans as feature engineering in Classification
      Exploring how hyperparameter tuning and cross validation impact to the improvement
      Alt Blog KMeans - Clustering Method Part 1
  • Customer Segmentation

  • Movie Recommendation

    • Recommend movies to users using collaborative filtering and content based techinques
    • Collaborative filtering: Recommendation list is generated based on the most similar items to a user's already-rated items
    • Content based Model: Recommend movies with similar contents : genres, actors, actresses, crew
    • Techniques: Data Cleaning, Data Visualization, NLP
  • Sentiment Mining

    • Explore Logistic Regression Classifier with postive/negative/neutral product's reviews
    • Extrac topics in customer's review with Laten Dirichlet Allocation (LDA)
    • Generate postitive/negative/neutral reviews by implementing Marko Chain Text Generator
    • Techniques: Data Visualization, Data Cleaning, Classification, Topic Modeling
  • Portfolio Investment Optimization

    • Collect historical data of 20 stocks of S&P 500 in 5 years
    • Use the Principle Component Analysis from Sklearn to structure the eigenvector features of covariance matrix of stocks
    • Calculate the weights of each portfolio in PCA components
    • Compute the sharpe ratio, annual return and annual volatility of each portfolio
    • Techniques: Data scrapping, Data Visualization, Principal Components Analysis