Machine Learning

Dr. Indu Joshi

Assistant Professor at
Indian Institute of Technology Mandi

4 August 2025
Course Logistics

• Mid-term: 30%
• End-term: 35%
• Assignment: 30%
• Class participation: 5%
Books

• Understanding Machine Learning: From Theory to

Algorithms, by Shai Shalev-Shwartz and Shai Ben-David
• Machine Learning by Tom M. Mitchell
• Machine Learning A Probabilistic Perspective by Murphy
• The Elements of Statistical Learning by Hastie, Tibshirani,
and Friedman
• Pattern Recognition and Machine Learning by Christopher M.
Bishop
Machine Learning

• Machine learning is used to program computers so that they

can “learn” from input available to them.
• Learning is the process of converting experience into expertise
or knowledge.
• The input to a learning algorithm is training data,
representing experience, and the output is some expertise,
which usually takes the form of another computer program
that can perform some task.
• Let us begin by considering a couple of examples from
naturally occurring animal learning.
Learning: Bait Shyness- Rats Learning to Avoid Poisonous
Baits
Learning: Bait Shyness- Rats Learning to Avoid Poisonous
Baits

• When rats encounter novel food items, they will first eat very
small amounts, and subsequent feeding will depend on the
flavor of the food and its physiological effect.
• If the food produces an ill effect, the food will often be
associated with the illness, and subsequently, the rats will not
eat it.
• Clearly, there is a learning mechanism in play here- if past
experience with some food was negatively labeled, the animal
predicts that it will also have a negative label when
encountered in the future.
Example: Spam Filtering

• A naive solution would be seemingly similar to the way rats

learn how to avoid poisonous baits.
• The machine will simply memorize all previous e-mails that
had been labeled as spam e-mails by the human user.
• When a new e-mail arrives, the machine will search for it in
the set of previous spam e-mails. If it matches one of them, it
will be trashed. Otherwise, it will be moved to the user’s
inbox folder.
• This learning by memorization approach is sometimes useful,
but it lacks an important aspect of learning systems – the
ability to label unseen e-mail messages.
Example: Spam Filtering (contd.)

• A successful learner should be able to progress from individual

examples to broader generalization.
• This is also referred to as inductive reasoning or inductive
inference.
• In the bait shyness example, after the rats encounter an
example of a certain type of food, they apply their attitude
toward it on new, unseen examples of food of similar smell
and taste.
• To achieve generalization in the spam filtering task, the
learner can scan the previously seen e-mails, and extract a set
of words whose appearance in an e-mail message is indicative
of spam.
Example: Spam Filtering (contd.)

• Then, when a new e-mail arrives, the machine can check

whether one of the suspicious words appears in it, and predict
its label accordingly.
• Such a system would potentially be able correctly to predict
the label of unseen e-mails.
• However, inductive reasoning might lead us to false
conclusions as explained in the next example.
Pigeon superstition
Learning: Pigeon superstition- Something goes wrong

• The psychologist B.F. Skinner placed a bunch of hungry

pigeons in a cage attached to an automatic mechanism that
delivered food to the pigeons at regular intervals with no
reference whatsoever to the bird’s behavior.
• The hungry pigeons go around the cage pecking at random
objects.
• When food is delivered, it finds each pigeon pecking at some
object. Consequently, each bird tends to spend some more
pecking time at that lucky object.
Learning: Pigeon superstition- Something goes wrong
(contd.)

• That, in turn, increases the chance that the next random food
spraying will find each bird at that location of hers.
• What results is chain of events that reinforces the pigeons’
association of the delivery of the food with whatever chance
actions they had been performing when it was first delivered.
Superstition Vs Useful Learning

• Human learners can rely on common sense to filter out

random meaningless learning conclusions.
• Once we export the task of learning to a program, we must
provide well-defined crisp principles that will protect the
program from reaching senseless/useless conclusions.
Bait Shyness Revisited

• Rats fail to acquire conditioning between food and electric

shock or between sound and nausea.
• In experiments carried out by Garcia, it was demonstrated
that if the unpleasant stimulus that follows food consumption
is replaced by, say, electrical shock (rather than nausea),then
no conditioning occurs.
• That is, even after repeated trials in which the consumption
of some food is followed by the administration of unpleasant
electrical shock, the rats do not tend to avoid that food.
• Similar failure of conditioning occurs when the characteristics
of the food that implies nausea is replaced by a vocal signal
(rather than the taste or smell of that food).
Bait Shyness Revisited (contd.)

• Clearly, the rats have some “built-in” prior knowledge telling

them that, while temporal correlation between food and
nausea can be causal, it is unlikely that there will be a causal
relationship between food consumption and electrical shocks.
• Now, we can observe that one distinguishing feature between
the bait shyness learning and the pigeon superstition is the
incorporation of prior knowledge that biases the learning
mechanism.
• It turns out that the incorporation of prior knowledge, biasing
the learning process, is inevitable for the success of learning
algorithm
Prior Knowledge

• The development of tools for expressing domain expertise,

translating it into a learning bias, and quantifying the effect of
such a bias on the success of learning, is a central theme of
the theory of machine learning.
When Do We Need Machine Learning?

• When do we need machine learning rather than directly

program our computers to carry out the task at hand?
• Two aspects of a given problem may call for the use of
programs that learn and improve on the basis of their
“experience”: the problem’s complexity and the need for
adaptivity.
Tasks That Are Too Complex to Program- Tasks
performed by animals/humans

• There are numerous tasks that, although we perform

routinely, our introspection, concerning how we do them, is
not sufficiently elaborate to extract a well-defined program.
• Examples of such tasks include driving, speech recognition,
and face recognition.
• In all of these tasks, ML programs learn from their experience,
once exposed to sufficiently many training examples.
Tasks That Are Too Complex to Program- Tasks beyond
Human Capabilities

• Another wide family of tasks that benefit from machine

learning techniques are related to the analysis of very large
and complex data sets: Astronomical data, turning medical
archives into medical knowledge, weather prediction, etc.
• Learning to detect meaningful patterns in large and complex
data sets is a promising domain in which the combination of
programs that learn with the almost unlimited memory
capacity and ever increasing processing speed of computers
opens up new horizons
Adaptivity

• One limiting feature of programmed tools is their rigidity -

once the program has been written down and installed, it
stays unchanged.
• However, many tasks change over time or from one user to
another in a way that requires the way we handle them to
adapt.
• Machine learning tools – programs whose behavior adapts to
their input data – offer a solution to such issues; they are, by
nature, adaptive to changes in the environment they interact
with.
• An example of machine learning to such problems include
programs that decode handwritten text, where a fixed
program can adapt to variations between the handwriting of
different users.
Types of Learning

• In the discriminative learning setting, learning can be defined

as the process of using experience to become an expert.
• Discriminative learning is convenient because there is a clear
measure of success – how good the predictions of the learner
on future examples are.
• In non-discriminative learning, the learner aims to
“understand” the data e.g. by learning a generative
probabilistic model that fits the data.
• We describe four parameters along which learning paradigms
can be classified.
Supervised vs. Unsupervised Learning
Supervised vs. Unsupervised Learning

• Viewing learning as a process of using experience to gain

expertise, supervised learning describes a scenario in which the
experience, a training example, contains significant
information that is missing in the test examples to which the
learned expertise is to be applied (say, the Spam/no-Spam
labels).
• In this setting, the acquired expertise is aimed to predict that
missing information for the test data.
• In such cases, we can think of the environment as a teacher
that supervises the learner by providing the extra information
(labels).
Supervised vs. Unsupervised Learning (contd.)

• In unsupervised learning, there is no distinction between

training and test data. The learner processes input data with
the goal of coming up with some summary, or compressed
version of that data.
• Clustering a data set into subsets of similar objets is a typical
example of such a task.
• There is also an intermediate learning setting in which, while
the training examples contain more information than the test
examples, the learner is required to predict even more
information for the test examples.
Supervised vs. Unsupervised Learning (contd.)

• For example, one may try to learn a value function, that

describes for each setting of a chess board the degree by
which White’s position is better than the Black’s.
• Such value functions can be learned based on a data base that
contains positions that occurred in actual chess games,
labeled by who eventually won that game.
• Such learning framework are mainly investigated under the
title of reinforcement learning.
Reinforcement Learning

• Agent learns by interacting with the environment.

• Receives rewards for desired actions and penalties for
undesired ones.
• Goal: Maximize cumulative reward.
• Applications: game playing (e.g., AlphaGo), robotics.
Key Components of Reinforcement Learning

• Agent: The learner.

• Environment: The world the agent interacts with.
• State: Current situation of the agent.
• Action: What the agent can do.
• Reward: Feedback from the environment.
Reinforcement Learning
Types of Learning
Semi-supervised Learning

• Combines labeled and unlabeled data.

• Goal: Improve learning efficiency using limited labeled data.
• Applications: Text classification, Medical diagnosis (limited
labeled data).
Example-Semi-supervised
Active vs. Passive Learning

• An active learner interacts with the environment at training

time, say by posing queries or performing experiments, while a
passive learner only observes the information provided by the
environment (or the teacher) without influencing or directing
it.
• The learner of a spam filter is usually passive- waiting for
users to mark the emails arriving to them.
• In an active setting, one could imagine asking users to label
specific emails chosen by the learner.
Helpfulness of the Teacher

• A helpful teacher tries to feed the learner with the information

most useful for achieving the learning goal.
• In contrast, when a scientist learns about nature, the
environment, playing the role of the teacher, can be best
thought of as passive - apples drop, stars shine and the rain
falls without regards to the needs of the learner.
• We model such learning scenarios by postulating that the
training data (or the learner’s experience) is generated by
some random process.
• Learning also occurs when the learner’s input is generated by
an adversarial teacher.
Online versus Batch Learning

• The last parameter is whether the learner has to respond

online, throughout the learning process.
• Settings in which the learner has to engage the acquired
expertise only after having a chance to process large amounts
of data.
• For example, a stock broker has to make daily decisions,
based on the experience collected so far. He may become an
expert over time, but might have made costly mistakes in the
process.
• In contrast, in many data mining settings, the learner - the
data miner - has large amounts of training data to play with
before having to output conclusions.
Online versus Batch Learning
Thank You

Contact: indujoshi@iitmandi.ac.in