Should I use FST, G’ST or D?

I have heard many researchers extolling one estimator over another, saying papers using any other approach should be rejected without review. Yet, although numerous recent papers assess various population genetic parameters and their validity in analyzing population structure, gene flow, and divergence, no clear consensus has been reached. FST is probably the most widely used measure of genetic distance between populations, along with the closely related estimator GST, but there are now also RST, G’ST, ΦST, D and even G”ST, as well as many others. FST has a big advantage in terms of familiarity – it has been around long enough that we have some idea of what its properties are. It can be thought of as the reduction in heterozygosity due to population structure, or (for biallelic markers) the variance in allele frequencies among populations. It behaves in predictable ways in response to particular circumstances.

FST was originally formulated to measure genetic distance using biallelic markers (Wright 1969), but was generalized for multiple alleles (Nei 1973), referred to as GST (although confusingly, the terms GST and FST are often used interchangeably; here I will generally use GST). However, numerous recent papers (e.g. Hedrick 2005, Jost 2008, Jost 2009, Meirmans and Hedrick 2011) have pointed out a difficulty in interpreting GST. With two populations and two alleles, GST ranges from 0.0 to 1.0, as expected, with 0 representing no differences in allele frequencies between two populations and 1.0 indicating that the two populations are fixed for alternate alleles. With more than two alleles, however, GST cannot reach 1.0 even when no alleles are shared between the two populations, as there will always be some heterozygosity within populations. This is not a flaw in GST, but it does indicate that we have to be careful when interpreting GST or comparing GST for different types of markers.

For many situations, certainly, this can be quite problematic – for microsatellites with high heterozygosity, maximum GST is often 0.1-0.2! Clearly, in these cases Wright’s (1978) guidelines are entirely misleading, when he states that values ranging from 0-0.05 indicate “little” genetic differentiation; 0.05-0.15 is “moderate”, 0.15-0.25 is “great”, etc. This is only plausible for biallelic cases, and in other situations we cannot rely on such simple rules of thumb.

To account for the variation in the maximum obtainable GST, Hedrick (2005) proposed a “standardized” measure, G’ST, calculated by dividing GST for a given marker by the maximum theoretical GST based on the heterozygosity at that marker. Additionally, in 2008 Jost introduced another measure of differentiation, D, which measures the fraction of allelic variation among populations. Both G’ST and Jost’s D will be 1 at complete differentiation (even with high variation within populations) and are zero with no differentiation, and both statistics have other intuitively appealing properties.

Figure 1. Effect of migration rate, μ=0.001 from Whitlock 2011, Fig. 1.

Both measures, though, have problems, and can be just as difficult to interpret as GST. G’ST and Jost’s D behave quite well in some cases, e.g. high mutation rates and two populations (Fig. 1, left), closely tracking the “true” divergence as measured by coalescent FST. However, while GST is limited to low values when heterozygosity is high, G’ST or D are in many cases biased upwards, rarely falling much below one even with high migration rates when there are many populations (Fig. 1, right). So, neither G’ST or D should be used as a proxy for migration rates in most situations. Additionally, D is highly affected by mutation rates so it is hard to compare multiple loci, or even to compare the same locus in different species if mutation rates differ. Nevertheless, if allelic differentiation at a particular locus is the value of interest, it appears that D is the best measure.

GST has a fairly straightforward relationship to gene flow and mutation rate, with patterns driven by migration when mutation rates are low relative to migration rates. In many cases gene flow can be safely assumed to be high relative to mutation rate, so in much of the literature GST is used to assess migration rates. This may be true for DNA sequence polymorphisms, where mutation rates are typically on the order of 10-9-10-8. In contrast, however, some markers such as microsatellites, which are widely used in population genetic studies due to their high variability, have mutation rates ranging from 10-6-10-3. As researchers often screen for the most variable markers, many studies are likely relying on microsatellites on the high end of this range. At these high mutation rates, D and G’ST will be very close to one and GST will be close to zero for most markers, regardless of migration rates (Fig. 2). What to do?

Figure 2. Effect of mutation rate. (Whitlock 2011 Fig.2)

It is becoming quite clear that high-mutation markers are not a good choice if one wants to calculate GST or any related measure, except in very particular situations. RST and ΦST can be used to account for mutation rates if the assumption of step-wise mutations is met (see for example Kronhom et al. 2010), but other estimates of population differentiation are either misleading or uninformative. Highly variable microsatellites are great for some things: genetic mapping, looking for recent selective sweeps, or parentage analysis, for example, but not for calculating GST.

When using such markers, if selection or migration are really the population genetic features that one wants to measure, it may be best to evaluate these parameters directly rather than using something like GST. One can use coalescent-based likelihood methods such as IM or MIGRATE to estimate migration rates from the data if migration is of interest. Selection, too, can be assessed more productively using other approaches (e.g. lnRH, which tests for selective sweeps) rather than trying to identify GST outliers. One additional solution frequently recommended for assessing population structure using high-mutation markers is to calculate both GST and D. Markers where GST underestimates divergence should have significantly elevated values of D. Where this pattern is observed, allelic variation can be investigated in greater detail. However, more work needs to be done on the interpretation of these two measures in concert.

Over the past few decades researchers have increasingly used microsatellites, due to their high level of variability and the relative ease of development and scoring in non-model systems. However, now that next-generation sequencing is getting more affordable, sequence-based markers can be assessed throughout the genome (e.g. using RAD sequencing). As we move back towards such low-mutation-rate markers as SNPs, FST becomes easier to assess reliably. On the other hand, FST and other current methods are all designed to assess one or a few markers at a time, and genomic approaches just apply these methods thousands or tens of thousands of times for markers throughout the genome. One can look for outliers, calculate means, etc., without really taking full advantage of the data. For instance, I have seen bi-modal or skewed distributions of FST and other summary statistics; clearly means and standard deviations can be misleading in these cases. My hope is that new methods for assessing divergence will focus not on individual loci but on many markers throughout the genome.

For more on this, the articles listed below are the source of most of the ideas presented here. I particularly recommend the excellent recently-published article Whitlock (2011).


Hedrick PW (2005) A standardized genetic differentiation measure. Evolution, 59, 1633–1638. Link

Jost L (2008) GST and its relatives do not measure differentiation. Molecular Ecology, 17, 4015–4026. Link

Jost L (2009) Reply: D vs G’ST: response to Heller and Siegismund (2009) and Ryman and Leimar (2009). Molecular Ecology, 18, 2088–2091. Link

Kronholm I, Loudet O, de Meaux J (2010) Influence of mutation rate on estimators of genetic differentiation—lessons from Arabidopsis thaliana. BMC Genetics, 11, 33. Link

Meirmans PG, Hedrick PW (2011) Assessing population structure: FST and related measures. Molecular Ecology Resources, 11, 5–18. Link

Michalakis Y, Excoffier L (1996) A generic estimation of population subdivision using distances between alleles with special reference for microsatellite loci. Genetics, 142, 1061–1064. Link

Nei M (1973) Analysis of gene diversity in subdivided populations. Proceedings of the  National Academy of Sciences. 70, 3321–3323. Link

Whitlock M (2011) G’ST and D do not replace FST. Molecular Ecology, 20, DOI: 10.1111/j.1365-294X.2010.04996.x Link

Wright S (1969) Evolution and the Genetics of Populations, Vol. 2. University of Chicago Press, Chicago.

This entry was posted in methods, population genetics and tagged , , . Bookmark the permalink.
  • Lou Jost

    Dear Nolan,

    (Apologies in advance if my graphs don’t post properly–if that happens, I will post them below)

    Thanks for writing about this topic. I think it would be healthy to start a debate about the central assumption of your post, the assumption that Nm determines genome-wide genetic differentiation in subdivided populations. This assumption drives many research programs in population genetics. It also seems to be the basis for most critiques of D. For example, Miermans and Hedrick (2010) say that an ideal measure of population structure would depend only on Nm. Your post also implicitly uses Nm as the “correct” parameter determining differentiation. You use graphs from Whitlock (2011), which claim to show that D is biased or uninformative in certain parameter ranges, because it does not track Fst_coal (which is assumed to track Nm). Ryman and Leimar (2009) made similar graphs and came to similar conclusions.
    The most important result of my mathematical work (Jost 2008) is the proof that this assumption is false, if “differentiation” means “heterozygosity-based relative differentiation of allele frequencies among demes”. Here I will show this by simple counter-examples to the traditional view.

    First I need to be precise about this definition of differentiation, because much of this debate is really about people using different concepts of differentiation (sometimes in the same paper!) First, when we specify a concept of differentiation, we have to say how much weight we give to rare or common alleles, and a “heterozygosity-based” measure of differentiation uses the same weighting scheme as heterozygosity does. The behavior of the most common alleles in each deme count more than the behavior of the rare alleles. Fst_coal, Gst, D, and all their relatives use this weighting scheme. (One important measure which uses a different weighting scheme is based on entropy; see Sherwin 2006, Jost 2006, Jost et al. 2010.) Second, “relative differentiation” has a range that always (regardless of the within-deme allele frequency distribution) can go from zero to unity. Zero always means the allele frequencies of the demes are identical. Unity always means no demes share any alleles. Finally, differentiation can never decrease by adding private alleles to demes in the symmetric way described in Fig. 2 of Jost (2008). We can only have debates about differentiation when the word is defined precisely. I believe most geneticists consciously or unconsciously agree with this definition. I have sometimes presented geneticists with lists of multiple-deme allele frequencies, and asked them to rank the lists in order of increasing differentiation. Their rankings always agree with this definition. (They may have other definitions of differentiation which they would use in other circumstances; and they may wrongly assume that all their definitions are equivalent.)

    Alright, now that we have the fine print out of the way, let’s test the idea that Nm controls differentiation. If Nm controls differentiation, then we can use any marker to estimate Nm, and this value of Nm should tell us something about the amount of differentiation throughout the genome. For my test of this assumption, I’ll make everything perfect for the traditional view- I’ll use a population that obeys the finite island model exactly, and that has reached equilibrium. I will also imagine we have the kind of ideal estimator of population structure that M&H and Whitlock wish for, one that lets us estimate Nm exactly. Let’s see where that gets us. (As in all previous published and posted discussion, we are here assuming samples are so big or so complete that there is no sampling error. We are discussing real conceptual problems, not estimation problems.)

    My first example population has these parameters: d= 5 demes of N=100 haploid reproductive individuals, with migration rate m= 0.0001, and mutation rate u = .000001. Nm is therefore 0.01. Assume a geneticist uses our hypothetical magical estimator and is able to estimate this Nm exactly. Following standard wisdom, geneticists would infer that this marker’s alleles, and the alleles at neutral loci elsewhere in the genome, should show high differentiation. Are they right? We can answer that directly by doing a simulation and looking at the actual allele frequencies, thanks to the work of my colleague Anne Chao and her student T. C. Hsieh.

    At this marker locus, we compared pairwise differentiation between two randomly selected demes at equilibrium, and repeated the simulation 200 times. Of the 200 runs, 189 ended up with both demes fixed FOR THE SAME ALLELE! In other words, 95% of the time there is no differentiation at all, even though Nm=0.01 (with expected Gst = 0.97, expected Fst_coal = approx 0.94. Wow! The classical view is badly wrong. The magic estimator of Nm is worse than useless–it misleads us, because Nm is not the quantity we should be estimating.

    Figure 1 is a scatter plot based on these simulations. After each run, we plot the frequency of each allele in Deme 1 versus its frequency in Deme 2. We repeat this for each of the 200 runs. When points from different runs land on top of each other, we make the points bigger, and the numbers alongside these points are the number of points that are piled on top of each other. These kinds of plots give a visual impression of the amount of differentiation in allele frequencies across demes. When the frequency of an allele is similar in both demes, its point will lie close to the diagonal line x=y in the graph. When an allele is present in one deme but absent from the other, its point will lie along the x- or y- axis. These scatter plots let us judge relative differentiation visually without the need to sue summary statistics.

    Figure 1

    Fig. 1. Here Nm = 0.01, N=100, d=5, m=0.0001, u = 0.000001, and L=200 simulation runs plotted. In almost all runs, a single allele was fixed at each locus. In 189/200 runs, the SAME allele was fixed in both loci. Expected Gst = 0.97, expected and mean observed D = 0.04.

    In this case, Figure 1 shows that fixation is very high in each deme (that is, after all, what Nm really measures–see Gregorius 2010), but pairwise differentiation or population structure –however that is defined—is nonexistent in 95% of the loci (both demes are fixed for the same allele in 95% of the loci). Of course, a real differentiation measure like D always gives the correct value, close to zero in this case (0.04).
    Not only will there be little or no differentiation at this marker, there will be even less differentiation anywhere else in the genome at any neutral locus with lower mutation rate. The idea that Nm controls genome-wide differentiation is not correct. Differentiation is controlled by m/[u(d-1)] as shown in my 2008 paper.

    If people are not convinced by our simulations or our graph, there is another easy way to see that there will be virtually no differentiation in this population. For the parameters in this example, the expected value of within-deme heterozygosity is (under the approximation that u<<m<<1):
    Hs ≈ [2Nmd]/[(m/u) + 2Nmd + (d-1)] = 0.001
    confirming that demes will almost always be fixed for a single allele. The total heterozygosity Ht is
    Ht ≈ [2Nmd + (d-1)2/d]/[(m/u) + 2Nmd + (d-1)] = 0.03.
    The total heterozygosity is also near zero. If the five demes are fixed for a single allele, and their pooled heterozygosity is near zero, then all the demes must almost always be fixed for the same allele, so that there will almost never be any differentiation.

    What does control population structure in these kinds of cases? To answer that, recall that both Gst and D are defined in terms of Hs and Ht. This means we can write Hs and Ht in terms of Gst and D:
    Ht = D/[Gst(d/(d-1) + D(1-Gst)].
    When Gst is close to unity, as in the example just discussed, this expression for Ht simplifies to:
    Ht = D[(d-1)/d].
    Therefore there can be no population structure if D is close to zero. D is close to zero when m/[(d-1)u] is large (as in this example, with D=0.04 and m/[(d-1)u] = 25.
    Now maybe a geneticist has a different definition of “differentiation” than the one I used, and maybe Nm controls his or her kind of differentiation. But if so, he or she will have to explain how a population whose demes are all fixed for the same allele for all neutral loci in the genome somehow shows a high degree of differentiation.

    The last example showed that very low Nm does not necessarily lead to high differentiation of allele frequencies across demes. Now imagine the opposite case, with Nm very high. Suppose there are 100 demes of 2000 haploid reproductive individuals, and suppose the migration rate is 0.01 and the mutation rate is 0.001. Nm is therefore 20. Traditional geneticists would say this high value of Nm will homogenize the demes. Might they be right this time?

    Figure 2 shows that the answer is no. This scatter plot uses the same method as Figure 1. Two demes are chosen for comparison, and the frequency of each allele in Deme 1 is plotted against the frequency of the same allele in Deme 2. If the alleles have the same frequency in both demes, the point for that allele will lie on the diagonal line x=y, and if the allele is present in one deme but absent in the other, the point will lie somewhere along the x-axis or the y-axis. Figure 2 shows the results of 50 runs plotted on the same graph. In this scatter plot, the points representing the frequencies of the most common alleles in each deme in each run are in pink, and the points get redder when they land on top of each other. The less common alleles are plotted in black.

    Figure 2

    Fig 2. Right panel enlarges the lower left corner of the left panel. Here Nm = 20, N=2000, d=100, m=0.01, u = 0.001, and L=50 simulation runs plotted. Points for the most common alleles (those points farthest from the origin) are clustered along the x- and y-axes, indicating that they are common only in one deme, and rare or absent in the other. Expected and mean observed Gst = 0.02, expected D = 0.91, mean observed D = 0.90.

    In Fig 2 the points (especially the pink/red ones from the most common alleles in each run) are concentrated along the x- and y-axes, indicating that the common alleles in one deme are absent or nearly absent from the other. Demes are not homogenized even though Nm is high (Nm = 20 and Gst = 0.02). On the contrary, differentiation is very high. As always, it is m/[(d-1)u] (which equals 0.1) that really controls differentiation (and it is this quantity that controls the expected value of D, which is 0.90 here).

    We had to use a high mutation rate in this simulation, because low mutation rates would have required large numbers of demes in order to get high D, and this would make the simulation drag on for a long time. But it is clear from the math that regardless of mutation rate, we can always find parameters which will falsify the claim that some value of Nm necessarily homogenizes the genome.

    These examples show that Nm does not control differentiation of allele frequencies among demes. Nm (and hence Fst_coal and Gst) has many valid uses, but this is not one of them. The real quantity that determines heterozygosity-based differentiation is m/[(d-1)u], and this is estimated by D. Yes, this varies from locus to locus. It is unfortunate that mutation has an effect, but that is the way nature works. We have to adapt to it and change our research strategies accordingly.

    See Gregorius (2010) for an explanation of the aspect of allele frequency distributions that is actually captured by Gst and its relatives.

    Thanks to Anne Chao and T. C. Hsieh for designing and running the simulations, and to Nolan Kane, Mike Whitlock, H.-R. Gregorius, and Pim Edelaar for helpful discussions.

    Additional references

    Sherwin, W., Jabot, F., Rush, R., Rossetto, M. (2006) Measurement of biological information with applications from genes to landscapes. Molecular Ecology 15: 2857-2869.
    Gregorius, H.-R. (2010) Linking diversity and differentiation. Diversity 2: 370-394.

    • Nolan Kane

      Good points, Lou. I’ll think this over and respond, but for now – I’ve fixed your images so they appear as they should.

    • thank you lou.

  • Lou Jost

    The figures didn’t come through, but they can be seen by plugging these links into your browser:


  • Lou Jost

    Thanks very much, Nolan, for fixing my post so that the Figures appear. I look forward to the discussion on these points. Maybe we can sharpen the various differentiation concepts that are out there, and help people decide which concept (and associated estimator/measure) is right for their work.

  • Nolan Kane


    I agree – if one is interested in measuring differentiation, D is a very good measure in most situations.
    However, if one is interested in migration rates, coalescent-based approaches may be the best approach to directly infer migration rates from the data. And, I still think that FST and GST have some useful properties, as long as one is using low-mutation markers.

    I should expand on my statement that “With more than two alleles, however, GST cannot reach 1.0 even when no alleles are shared between the two populations, as there will always be some heterozygosity within populations.” GST can be as high as 1.0 as long as the number of alleles is less than or equal to the number of populations. In many cases the number of populations is far greater than the number of alleles observed, of course but this is certainly not always the case, particularly for high mutation markers such as microsatellites. For SNPs, though this is almost never a problem.

    In many situations, I imagine researchers will want to calculate several different measures, including GST and D. I would argue that moderate- and low-mutation markers are the most informative for any of these measures, though, particularly if one can assess many markers.

    • Lou Jost

      Nolan, my point here is that absolute migration rate is not what we want to know. It does not control differentiation, so why bother to estimate it in studies investigating population structure?

      Sure, under some circumstances, Fst_coal or Gst can (under ideal conditions) may be useful tools for inferring absolute migration rates. But the examples in my post show that even when we infer these absolute migration rates with perfect precision, this tells us nothing about genetic differentiation between demes. Genetic differentiation is determined by m/[u(d-1)], as my examples prove.

      • Nolan Kane

        I’m sure there are many situations where people want to only know about one thing or another, but for much of my work I’m interested in both gene flow and differentiation, as I’m sure you are as well, as both are key to understanding speciation.
        Similarly, in any case of diverging populations, a key question is often “how much gene flow is occurring?” Knowing that can shed light on what is driving the patterns of differentiation observed.

        • Lou Jost

          You wrote:

          “In any case of diverging populations, a key question is often “how much gene flow is occurring?” Knowing that can shed light on what is driving the patterns of differentiation observed.”

          When you say “gene flow”, I assume you mean absolute number of migrants. Don’t you see that my examples prove that absolute gene flow tells you NOTHING about the pattern of differentiation or the degree of divergence? In fact, it can badly mislead? Look again at my examples. Differentiation can be exactly the opposite of what you would expect based on absolute gene flow.

          Relative gene flow is the relevant quantity (along with mutation rate and degree of population subdivision) for understanding patterns of differentiation. There may be many good reasons for wanting to know absolute gene flow, but this is not one of them. If you want to understand teh kind of genetic divergence between demes that can facilitate speciation, you need to know the relative gene flow m, not Nm.

          Look again at my examples if you doubt this. Show me how, in my first example, knowing that Nm =0.01 helps you to understand that all neutral loci throughout the genome are almost always fixed for the same allele in all demes (no population structure between demes). Show me how, knowing that Nm=20 in my second example, this helps you understand the the common alleles are private to particular demes. That’s incipient speciation if those are neutral or nealy-neutral coding alleles. Yet a value of Nm=20 is so high that you would discard the possibility of divergence if you obtained that number in your study. You would have been mistaken about the whole process, because Nm is not relevant to that process.

          • Nolan Kane

            Hi Lou,

            After thinking it over for some time, I believe I understand what may be going on in your figures, both of which illustrate important points about the measures used. It gets a bit complicated because we are only looking at part of the picture – two demes are illustrated out of many that go into the statistics – and I don’t fully understand the details of the simulations. I don’t mean to say that looking at only two demes is misleading – it just affects how we have to think about these measures. For instance, the case illustrated in figure 1 shows how Gst and D behave with only two alleles and five demes, low mutation rate and low migration. Gst is high in this case because on average most of the variation is between rather than within demes. Just looking at two of the demes, we tend to see fixation for the same allele in this case. However, there are many demes, and most of the time at least one of the demes will end up fixed for an alternate allele. D in this case is low, because it focuses on differentiation between demes: most pairs are fixed for the same allele. This is why it may be quite useful to calculate both measures.

            Figure 2 is in my view less informative, showing a case of high mutation, where D is expected to be high and Gst is expected to be low regardless of the migration rate. Again, though, this does illustrate a case where calculating both statistics is important. When Gst is low and D is high, it indicates that these measures may both be largely driven by high mutation rates.

            To get back to absolute vs. relative gene flow: it depends on the question, but of course both absolute and relative gene flow are important. Absolute gene flow will inform us on whether to expect divergence due to drift, while relative gene flow will affect divergence due to selection.

  • Lou Jost

    Suppose we know N, m, d, and u for various neutral and nearly-neutral loci of our study organism, and suppose we knew that our organisms followed the finite island model exactly. Suppose the current subdivisions of our populations are recently imposed (by historic or prehistoric climate change, habitat fragmentation, etc). Both Nolan and I are interested in the question of whether or not the demes of our populations are on the road to genetic divergence. We would also like to identify the genetic or demographic factors causing or impeding divergence.

    Traditional geneticists will rely on Nm to judge whether the demes will diverge. If they found that Nm was 20, they would say the demes will not diverge. If they found that Nm was 0.01, they would say the demes will show high divergence. The examples in my first comment prove that these conclusions can be wildly wrong; these inferences are invalid. Notice that this is not an estimation problem, nor a problem with Gst’s dependence on mutation rate. I assumed our knowledge of Nm was exact. Mycomment addresses a much more fundamental problem—the myth that Nm controls genetic divergence.

    In contrast, we would get the right answer about the future evolution of divergence at any locus that interested us, if we look at m/[(d-1)u]. Anyone can prove this for themselves just by doing the experiment in a computer simulation, choosing parameters so that m/[(d-1)u] is >>1 while Nm<<1, or vice versa. My examples above are cases like this.


  • Lou Jost

    Hi again Nolan,
    You are right that the graphs only look at two randomly chosen demes, and this only gives partial information about the degree of similarity between demes. We could not figure out a graphical representation of multiple-deme similarity for multiple runs, so we chose this approach which focusses on pairwise similarity. However, for my Fig 1, your concerns are addressed by my argument about total heterozygosity. Expected total heterozygosity for that simulation is 0.03. This shows that, contrary to your explanation, demes are virtually never fixed for different alleles. Recall that there are only five demes. If one or two demes had been fixed for a different allele than the other demes, then total heterozygosity would have been much higher than 0.03. (It would have been at least ten times higher.) The low total heterozygosity proves my point without the need for any graphs: very low Nm (Nm=0.01) does not imply high differentiation.

    Similar arguments can be made for Figure 2, using the mathematics of diversity to show that the most common alleles in all the demes are likely to be private or nearly so.

    Regarding the example in Figure 2, you say that this just shows that high mutation rate drives Gst and D to different extremes. This isn’t quite true of D; D can vary between zero and unity no matter what the mutation rate. It depends on m/[u(d-1)]. But in any case, remember that my point here is not to discuss D vs Gst, but rather to get at something more fundamental (and something that needs to be understood before we can argue about the merits of summary statistics). I am attempting to show which demographic and genetic factors really control differentiation. In this example, I show that even very high Nm (Nm=20) does not homogenize the demes. Far from it, these demes consist mostly of private or nearly-private alleles. So again, as in Example 1, this disproves the thesis that Nm controls differentiation.

    More details of the simulations might be helpful. Anne simulated a single locus and let the simulations run until equilibrium was reached, and plotted the points. She then repeated the process L times (L is given above each graph). She has tested her program against a program I wrote and against EasyPop, and there are no differences, so we are confident in the program. The mean value of D obtained in the simulations agrees to within 1% of the theoretical expected value of D, again showing that the program is behaving.

  • Lou Jost

    Some other important simulation details–we use the infinite allele model, as is usual in the finite island model. In my Example 1, there is no limit to the possible number of alleles. Yet most runs end up with all demes fixed for the same allele.

    Also, I should correct something in my last reply–D is within 0.01 of its expected value, not 1% of its expected value.

  • Lou Jost

    Nolan, I am puzzled by your very last comment, “Absolute gene flow will inform us on whether to expect divergence due to drift, while relative gene flow will affect divergence due to selection.” The simulations in my examples include drift and exclude selection. They show that divergence due to drift is determined by relative rather than absolute gene flow.

  • Lou Jost

    Nolan, I am puzzled by your very last comment, “Absolute gene flow will inform us on whether to expect divergence due to drift, while relative gene flow will affect divergence due to selection.” The simulations I gave above were of neutral loci; the results include the effect of drift and exclude selection. Yet, absolute gene flow did not determine divergence due to drift. Relative gene flow did.

    If it were really true that absolute migration rate Nm determined expected divergence due to drift, there should have been lots of divergence in my Ex 1 and no divergence in Ex 2. The simulations demonstrate exactly the contrary. It is m/[u(d-1)] that gives the correct prediction about the evolution of divergence in these simulations, and in all cases of neutral loci under the finite island model.

    Geneticists often repeat the assertion that divergence due to drift depends on Nm. Isn’t this belief based on the relation between Nm and Gst or Fst? If so, you know it is wrong, because many people have now shown that Gst and Fst do not measure divergence but rather fixation (as Wright originally said).

    • Nolan Kane

      Hi Lou.
      Yes, in this case it looks like my statements would only apply to Gst or Fst, not D.

  • Lou Jost

    Nolan, it is worth noting that my conclusions are not specific to D. My conclusions apply to any measure of relative differentiation that uses alleles as the unit of analysis and that emphasizes common alleles. Any such measure of relative divergence or differentiation will have to give a number near 0 in my Ex 1, because all the demes are identical in most runs. Likewise any such measure of relative divergence or differentiation must give a value near 1 in cases like my Ex 2, because the common alleles are private (or nearly so) to single demes. So in these extreme cases, we can say with great generality that m/[u(d-1)] controls relative differentiation when common alleles are the focus. It is possible to prove this for the intermediate cases too, if we add a few more reasonable mathematical properties to the definition of differentiation.

    These conclusions have an impact on your work, and mine, on the conditions leading to plant speciation. Neutral models provide a sort of stochastic baseline for the evolution of differentiation among demes. We used to think Nm set the baseline. Now we know this is wrong (see simulations above). For any given locus, the baseline is set by m/[u(d-1)]. Population size is not involved in the expected value of differentiation (though it may be involved in the variance of differentiation over evolutionary time, and this could be important). The expected value of differentiation depends on the locus (because of its dependence on u). And if each population receives a proportion m of immigrants every generation, (independent of the number of demes surrounding it), then the number of demes (the degree of population subdivision) greatly promotes differentiation.

    This new set of conditions predicts that differentiation at neutral or nearly neutral loci can evolve even in the face of significant absolute gene flow, as long as the relative migration rate is low. It also predicts that greater population subdivision will itself facilitate the evolution of divergence. Best of all, we get to throw out the “one migrant per generation” rule that appeared to put severe limits on the possibility of divergence. Evolution of divergence is easier than we previously thought.

    I should add a reference to a set of simulations published in the ecological literature by Economo and Keitt (2008), who also found that Nm didn’t matter, and appear to have discovered empirically that m/u was the controlling factor for differentiation, given a particular set of demes and a particular connection scheme between them. (Some of their simulations use the stepping stone model.) Their simulations are made in an ecological context, but it is mathematically the same model as that used in pop gen.
    Economo, E, & Keitt, T. (2008) Species diversity in neutral metacommunities: a network approach. Ecology Letters 11: 52-62

  • Patrick Sullins

    Hi all, I have really enjoyed reading this thread, it has been very useful to me in determining what a) each of these measures mean and b) which to use in my own research. However, I’m still uncertain which measure to use in the following scenario. I am using several chloroplast DNA intergenic spacers (sequences) to compare two similar plant species, suggested for incorporation into the same taxa. I would consider this a “low-mutation-rate” marker, like SNPs. From what I have read above, it seems like I should go with Fst. If it matters, I’ll be using Peakall’s GenAlEx 6.5 to do my calculations, which I believe can do Gst, G’st, D, and several others…your thoughts? Thanks again for your insights and for the bringing up this topic!

    • Patrick Sullins

      Just to clarify, I am trying to determine differences and possible gene flow between these taxa (as evidence of gene flow would suggest that they are the same species). Thanks again!

  • Pingback: Failure of Fst/Gst – eginotes()