Simple overview of machine learning

MACHINE LEARNING
Priyadharshini .R
Mtech-(IT)
Reg no:2016246017 1

Evolution of Machine Learning
2

What is machine learning?
A branch of artificial intelligence,
concerned with the design and development
of algorithms that allow computers to evolve
behaviors based on empirical data.
As intelligence requires knowledge, it is
necessary for the computers to acquire
knowledge.
Definition of Machine learning:
Machine learning focuses on the
development of computer programs that can
access data and use it learn for themselves.
3

History of Machine Learning
• 1950s
– Samuel’s checker player
– Selfridge’s Pandemonium
• 1960s:
– Neural networks: Perceptron
– Pattern recognition
– Learning in the limit theory
– Minsky and Papert prove limitations of Perceptron
• 1970s:
– Symbolic concept induction
– Winston’s arch learner
– Expert systems and the knowledge acquisition
bottleneck
– Quinlan’s ID3
4

History of Machine Learning (cont.)
• 1980s:
– Advanced decision tree and rule learning
– Explanation-based Learning (EBL)
– Learning and planning and problem solving
– Utility problem
– Analogy
– Cognitive architectures
– Resurgence of neural networks (connectionism,
backpropagation)
– Valiant’s Learning Theory
– Focus on experimental methodology
5

1990s:
-Data mining
-Adaptive software agents and web applications
-Text learning
-Reinforcement learning (RL)
-Inductive Logic Programming (ILP)
-Ensembles: Bagging, Boosting, and Stacking
-Bayes Net learning
6

• 2000s
– Support vector machines
– Kernel methods
– Graphical models
– Statistical relational learning
– Transfer learning
– Sequence labeling
– Collective classification and structured outputs
– Computer Systems Applications
• Compilers
• Debugging
• Graphics
• Security (intrusion, virus, and worm detection)
7

Traditional Programming
Machine Learning
Computer
Data
Program
Output
Computer
Data
Output
Program
8

Types of Learning
• Supervised (inductive) learning
– Training data includes desired outputs
• Unsupervised learning
– Training data does not include desired outputs
• Semi-supervised learning
– Training data includes a few desired outputs
• Reinforcement learning
– Rewards from sequence of actions
9

Many online software packages & datasets
• onlineData sets
• UC Respirtory
• http://www.kdnuggets.com/datasets/index.html
• Software (much related to data mining)
• JMIR Open Source
• Weka
• Shogun
• RapidMiner
• ODM
• Orange
• CMU
• Several researchers put their software online
10

Training and testing
Training set
(observed)
Universal
set
(unobserved)
Testing set
(unobserved)
Data acquisition Practical usage
11

12
Learning (training): Learn a model using the training data
Testing: Test the model using unseen test data to assess the
model accuracy
Training and testing

Optimization
Combinatorial optimization
– E.g.: Greedy search
Convex optimization
– E.g.: Gradient descent
Constrained optimization
– E.g.: Linear programming
13

 The success of machine learning system also depends on the
algorithms.
 The algorithms control the search to find and build the
knowledge structures.
 The learning algorithms should extract useful information
from training examples.
Algorithms
14

 Supervised learning
– Prediction
– Classification (discrete labels), Regression (real values)
 Unsupervised learning
– Clustering
– Probability
– estimation
– Finding association (in features)
– Dimension reduction
 Semi-supervised learning
 Reinforcement learning
– Decision making (robot, chess machine)
Algorithms
15

Algorithms
Supervised learning Unsupervised learning
Semi-supervised learning 16

 Supervised learning categories and techniques
 Linear classifier (numerical functions)
 Parametric (Probabilistic functions)
• Naïve Bayes, Gaussian discriminant analysis (GDA), Hidden
Markov models (HMM), Probabilistic graphical models
 Non-parametric (Instance-based functions)
• K-nearest neighbors, Kernel regression, Kernel density estimation,
Local regression
 Non-metric (Symbolic functions)
• Classification and regression tree (CART), decision tree
 Aggregation
• Bagging (bootstrap + aggregation), Adaboost, Random forest
Learning techniques
17

 Unsupervised learning categories and techniques
Clustering
• K-means clustering
• Spectral clustering
Density Estimation
• Gaussian mixture model (GMM)
• Graphical models
Dimensionality reduction
• Principal component analysis (PCA)
• Factor analysis
Learning techniques
18

Issues in Machine Learning
• Problem representation / feature extraction
• Intention/independent learning
• Integrating learning with systems
• What are the theoretical limits of learnability
• Transfer learning
• Continuous learning
19

Measuring Performance
• Generalization accuracy
• Solution correctness
• Solution quality (length, efficiency)
• Speed of performance
20

Working Applications of ML
• Electrical power control
• Chemical process control
• Character recognition
• Face recognition
• DNA classification
• Credit card fraud detection
• Cancer cell detection
21

Current Machine Learning Research
• Representation
– data sequences
– spatial/temporal data
– probabilistic relational models
– …
• Approaches
– ensemble methods
– cost-sensitive learning
– active learning
– semi-supervised learning
– collective classification
– …
22

We have a simple overview of some techniques and
algorithms in machine learning. Furthermore, there are more and
more techniques apply machine learning as a solution. In the
future, machine learning will play an important role in our daily
life.
Conclusion
23

[1] W. L. Chao, J. J. Ding, “Integrated Machine
Learning Algorithms for Human Age
Estimation”, NTU, 2011.
Reference
24

Simple overview of machine learning

More Related Content

What's hot

Similar to Simple overview of machine learning

Recently uploaded

Simple overview of machine learning