Flashcards on Probabilistic Machine Learning
Probabilistic Machine Learning: An SEO Guide for Students
Tap to flip · Swipe to navigate
Probabilistic Modeling for Machine Learning
92 cards
Card 1
Question: What is the main shift in perspective when moving from the algorithmic viewpoint to the probabilistic viewpoint in machine learning?
Answer: Instead of learning a prediction function f from data, the probabilistic viewpoint learns the underlying data distribution P (joint or marginal) that
Card 2
Question: In the probabilistic model for supervised learning, how are inputs x and targets y formally modeled?
Answer: Inputs x are drawn from a marginal distribution X ~ P_X, and for a given x the target y is drawn from a conditional distribution Y | X = x ~ P_{Y|X}.
Card 3
Question: What is the formal definition of the best possible predictor f* (not restricted to a function class) under a probabilistic model?
Answer: f* = argmin_f L(f; P), where L(f; P) = E_{X,Y~P}[ℓ(f(X),Y)] is the expected pointwise prediction loss under the joint distribution P.
Card 4
Question: When discussing probabilistic models for data, what three distributions are commonly referenced and how are they related?
Answer: The marginal distribution P_X (over inputs), the conditional distribution P_{Y|X} (targets given inputs), and the joint distribution P_{X,Y}, with p_{
Card 5
Question: Why do we model randomness in regression tasks probabilistically (e.g., predicting house prices)?
Answer: Because for a given input x (like house size) many different y values (prices) may occur due to unobserved factors and timing; modeling with a conditi
Card 6
Question: What assumption about data generation underlies prior discussions of generalization and model selection in the lecture?
Answer: That training and test data are i.i.d. samples drawn from some underlying data distribution P (either P_X or P_{X,Y}).
Card 7
Question: Give two tasks/motivations for methods that learn from data mentioned under the algorithmic viewpoint of ML.
Answer: For labeled data (supervised learning): prediction. For unlabeled data (unsupervised learning): finding low-dimensional representations (e.g., PCA, au
Card 8
Question: In classification probabilistic modeling, what two distributions are especially important to consider?
Answer: The marginal distribution P_X over inputs and the conditional distribution P_{Y|X} that gives label probabilities given inputs; together they define t
Card 9
Question: How does the lecture illustrate that inputs x may not be equally likely to occur?
Answer: It notes that inputs (e.g., house sizes) have different probabilities of occurring, so we must average over P_X when assessing predictors rather than
Card 10
Question: What is meant by 'i.i.d. samples' in the context of probabilistic modeling for ML?
Answer: It means the training (and test) examples are independent and identically distributed draws from the underlying distribution P_X or P_{X,Y}.