The negative effects that inbreeding has on a population began to be observed with the emergence of Mendelian genetics. In these early days of genetics the most noted effect of inbreeding was increased homogenization within a plant population. Darwin noted that many plants had specific adaptation to avoid this inbreeding. 1 Although species have adaptation to try and avoid inbreeding when a population is extremely small, such as when a species is introduced to a new habitat, there is a high probability that individuals within the population will reproduce with another member that is closely related. When inbreeding occurs in a population it can result in an inbreeding depression.
The inbreeding depression lowers the overall fitness of the population. This is because in a large population when deleterious recessive alleles express themselves in a member of a population that individual dies off so there is less of that allele in future generations. These deleterious alleles do not always completely disappear but are sometimes masked by a dominant non deleterious allele. In a species native range this is usually not a problem because a sufficient level of outbreeding makes the odds of deleterious allele expressing themselves low. However, when a population suffers a bottleneck and there is no outbreeding there is a much higher chance of the recessive deleterious allele expressing itself in a significant proportion of the population. 2
1.) Charlesworth, D., & Charlesworth, B. (1987, January 1). Inbreeding Depression and its Evolutionary Consequences. Retrieved March 10, 2015, from http://www.annualreviews.org/doi/pdf/10.1146/annurev.es.18.110187.00132
2.) Inbreeding Depression. (n.d.). Retrieved March 10, 2015, from http://evolution.berkeley.edu/evosite/relevance/IIIA1Inbreeding.shtm