Unbiased Estimator - StatLect

Home » What Does Unbiased Mean In Math » Unbiased Estimator - StatLect

Maybe your like

Search for probability and statistics terms on Statlect StatLect Index > Glossary Unbiased estimator

by , PhD

An estimator of a given parameter is said to be unbiased if its expected value is equal to the true value of the parameter.

In other words, an estimator is unbiased if it produces parameter estimates that are on average correct.

Table of Contents

Table of contents

  1. Definition

  2. Examples

  3. Biased estimator

  4. Bias

  5. More details

  6. Keep reading the glossary

Definition

Remember that in a parameter estimation problem:

  • we observe some data (a sample, denoted by $\xi $), which has been extracted from an unknown probability distribution;

  • we want to estimate a parameter $\theta _{0}$ (e.g., the mean or the variance) of the distribution that generated our sample;

  • we produce an estimate [eq1]of $\theta _{0}$ (i.e., our best guess of $\theta _{0}$) by using the information provided by the sample $\xi $.

The estimate $\widehat{\theta }$ is usually obtained by using a predefined rule (a function) that associates an estimate $\widehat{\theta }$ to each sample $\xi $ that could possibly be observed

[eq2]

The function [eq3] is called an estimator.

Definition An estimator [eq3] is said to be unbiased if and only if[eq5]where the expected value is calculated with respect to the probability distribution of the sample $\xi $.

Examples

The following table contains examples of unbiased estimators (with links to lectures where unbiasedness is proved).

Estimator Estimated parameter Lecture where proof can be found
Sample mean Expected value Estimation of the mean
Adjusted sample variance Variance Estimation of the variance
OLS estimator Linear regression coefficients Gauss-Markov theorem
Adjusted sample variance of the OLS residuals Variance of the error of a linear regression Normal linear regression model

Biased estimator

An estimator which is not unbiased is said to be biased.

Bias

The bias of an estimator $\widehat{\theta }$ is the expected difference between $\widehat{\theta }$ and the true parameter:[eq6]

Thus, an estimator is unbiased if its bias is equal to zero, and biased otherwise.

More details

Unbiasedness is discussed in more detail in the lecture entitled Point estimation.

Keep reading the glossary

Previous entry: Unadjusted sample variance

Next entry: Variance formula

How to cite

Please cite as:

Taboga, Marco (2021). "Unbiased estimator", Lectures on probability theory and mathematical statistics. Kindle Direct Publishing. Online appendix. https://www.statlect.com/glossary/unbiased-estimator.

The books

Most of the learning materials found on this website are now available in a traditional textbook format.

Probability and statisticsMatrix algebra Featured pages
  • Combinations
  • Maximum likelihood
  • Gamma function
  • Gamma distribution
  • Uniform distribution
  • F distribution
Explore
  • Score test
  • Central Limit Theorem
  • Wald test
Main sections
  • Mathematical tools
  • Fundamentals of probability
  • Probability distributions
  • Asymptotic theory
  • Fundamentals of statistics
  • Glossary
About
  • About Statlect
  • Contacts
  • Cookies, privacy and terms of use
Glossary entries
  • Mean squared error
  • Continuous random variable
  • Precision matrix
  • Probability mass function
  • Posterior probability
  • Distribution function
Share
  • To enhance your privacy,
  • we removed the social buttons,
  • but don't forget to share.

Tag » What Does Unbiased Mean In Math