This document explains the steps involved in building a model for data scientists: 1) Data Understanding, 2) Data Preprocessing, and 3) Model Training and Model Evaluation. For more details, please read on
The process of building a model for data scientists
1. Understanding the Data.
Explain suitable data examples for model building, data types, and displaying data in graph format
2.Data Preprocessing
Preparing data for model building involves data cleansing, handling missing data, and managing outliers
Data manipulation to enable analysis, such as transforming nominal data through counting
Converting sequential numerical data to ordinal format divides the data into equal intervals or counts the frequency of each interval
Some data may need to be scaled to fit within a range we want, such as [0, 1] or [-1, +1]
The benefits of scaling data include faster model building and ensuring that each variable has equal importance
3.Model Training and Model Evaluation
The difference between supervised and unsupervised machine learning models lies in the fact that supervised models require real-world data and a predefined predictive model. For example, to predict the outcome of an image of a cat, a large number of cat images would be needed to build the model, identifying each image as a cat and others as not. Then, when new cat images are introduced, the model would be able to predict whether they are cats or not. Unsupervised models, on the other hand, are built using data that doesn't allow for the identification of a specific target
To build a good model, the data should be divided into a training set and a test set for model construction and testing. The training and test sets should be created by randomly selecting data points; using the entire dataset should be avoided as this leads to pattern recognition and results in an unpredictable model
There are two main types of model building: unsupervised machine learning techniques are used if the data has no target, and supervised machine learning techniques are used if the data has a target
There are many techniques for building models. Professor Phaisan selected some techniques to teach, along with introductions using Excel, Weka, Python, and Spark, to demonstrate how to create models using various software programs and to clearly illustrate the data analysis process
Speaker: Dr. Pairsan Padungwiang,
R&D BI Co., Ltd.
Compiled by Nuchanat Rongroang
www.rdbi.co.th
#Consultants
#Data Scientist
#Big Data Analytics
A collection of articles on Data Science Essential.
For more information, please contact us at https://demo.rdbi.co.th/contact/,
visit our Facebook page at http://bit/ly/rdbipage,
on Line Official Account: @rdbi,
or email us at sales@rdbi.co.th.