Category Archives: Note

How tr2-delimitation over-splits structured populations – a case of unresolved trees

I recently wrote a blog post about the effect of reduced gene flow over the multilocus species delimitation program, tr2. A general pattern is, even if two populations are connected with moderate gene flow, the tr2 erroneously splits them into two “species” when a very large number of loci are used. This result means we need to be cautious when we apply it to a big data set.

One thing I didn’t test is that the effect of unresolved gene trees. In the previous post, I assumed  gene trees are fully resolved. This assumption is rarely met in real data sets as RAD markers or exons of RNAs are not variable enough to give us fully resolved trees. Does this also affect the patterns of oversplits of tr2?

I just tested the effects of less-variable markers on tr2-delimitation. The simulation setting is identical to the previous post except that the branch lengths of gene trees are proportional to mutation rate. As mutation rate gets smaller, gene trees are more unresolved.

The gene trees simulated under Neμ= 0.005 are plotted below. They look realistic, just like gene trees we see in real practices. (You may think that these trees are not informative enough to detect recent speciation. Actually, they are informative.)

genetrees.nemu0.05

So, how does tr2 oversplit populations connected by gene flow, and how does mutation rate affect the pattern of oversplits?

species_tree.svg

The above plot is a species-population tree used in simulations. Pop.1 and pop.2 are connected by gene flow.

The plot below shows the proportions of trials where samples from two populations were assigned to one species. The migration parameter is Nem=1.0, and population mutation parameter, Neμ, ranged from 0.005 to 5.

p.n.vmu

The pattern of oversplits is similar to the previous simulations (see the plot). When Neμ= 0.5 or 5, curves are almost identical to the curve with fully-resolved gene trees. With > 500 loci, populations are always split into two “species”. When you have less mutations, curves diverged from the ideal curve and the overspliting slows down.

This is probably a good result. At least, less informative markers don’t lead you to false positives. However, they are probably less sensitive to the true pattern and you may miss it.

When you use tr2-delimitation, maybe you need to consider two points: how informative your markers are and how many markers you use. As you increase the number and informativeness of loci, you can detect finer scale structure, which may or may not be true species.

Again, it is often hard to determine the “sweet spot” of numbers and variability of markers to detect “true” species since speciation is always continuous. I am not sure if this problem can be solved by the delimitation with full multispecies coalescent model. (tr2 is an approximate method.) Explicit modeling of gene flow or geographic distribution is probably the better approach to tackle this problem, but usually time-consuming.  It may be possible to use resampling of loci to check how the pattern of splits develops, and combine it with locus informativeness to find the best threshold for true species entities.

 

Advertisements

How tr2-delimitation over-splits structured populations

As I test the tr2-delimitation with more data sets and read more papers applying it to  difficult species, I find that it sometimes infers an unrealistically large number of species. These oversplits appear to happen more frequently when tr2 is used with very large data sets (like thousands of loci of RAD or RNA).

A reasonable explanation for this tendency of oversplit is tr2 delimits structured populations as species even if they are connected with gene flow. Whether molecular species delimitation methods actually delimits species or not is still under debate (like this paper). But, it is quite possible that tr2 falsely finds populations connected with weak gene flow as “species”.

The basic idea of tr2 delimitation is that sets of gene tree topology are more similar to each other when multiple species exist in your samples. This concordance of topology results in a skewed distribution of triplets, where one triplet topology is more frequently observed than other two.

Reduced gene flow between populations also creates topological concordance. The effect of reduced gene flow is usually much smaller than the effect of true speciation and often undetectable with a small number of loci. However, even a minimal skew of triplet distribution is detected as a signature of species when the number of  loci is VERY large.

Because the tr2’s triplet distribution model only considers triplets’ topology and  does not include the distribution of branch length, the gene flow likely has more significant effect on its performance than the full multi-species coalescent model (such as BPP).

I checked with simulations how reduced gene flow between populations affects the delimitation results of tr2.

Gene trees were simulated under a model where two populations split but retain genetic exchange with gene flow. The age of split, T, is nearly the same value of effective population size, Ne. For example, if Ne = 50,000 and generation time is 1 year, the time of split is 50,000 years ago. The amount of gene flow, Ne*m = 0.5, 1.0 and 5.0. In this simulation, gene tree’s topology is known without error. (I will consider situations with unresolved gene trees in future posts.)

species_tree.svg

Does tr2 split these two populations, pop.1 and pop.2,  into separate species?

The plot below shows proportions of trials where samples from pop.1 and pop.2  are assigned to the same species and how the proportions change with the number of used loci. Different colors represent different degrees of gene flow.

p.n

As you can see, even under conditions with moderate gene flow (Nem=0.5 or 1), samples from two populations were split into two spurious species. For example, populations connected with gene flow of Nem=1.0 were always split into two species when the number of loci exceeded ~500.

With Nem=5, which means gene flow is very large, the tr2 did not split populations even with 1000 loci (, but I guess it will probably oversplit if much larger number of inputs are used). When Nem=0, that is, the two populations are truely two young species, just about 10 loci was enough to detect them.

The reduced gene flow does have a strong effect on delimitation. So, you need to be careful when interpreting the results of multilocus delimitation with a very large data set. Splits detected only with hundreds or thousands of loci  are probably not species, but population structure.

In the simulation above, only 10 or 20 loci with well-resolved tree topology have enough power to detect young species. Therefore, using different size of inputs and checking how patterns of split appear by increasing loci may help us interpret results.

Ultimately, it is quite hard to decide one threshold of gene flow and an appropriate sample size with which we can confidently say “there are species” since speciation process is continuous. Also, variation of informativeness of loci makes this decision even more difficult (You need more loci when they are less informative).

It seems that explicit modelling and quantification of gene flow is a better way to tackle this problem and should be a new direction of the multilocus delimitation program.

Multilocus delimitation with tr2: Guide tree approach

The second option of delimitation with the “tr2” is a guide tree approach. A guide tree is a tree which specify a hierarchical structure of species grouping. By using it, you can significantly reduce the number of possible delimitation hypotheses to search.

The tr2 implements an algorithm to find a best position of nodes which define species group under a given guide tree. As the algorithm is reasonably fast, you can search the best delimitation through a tree from each tip is distinct species to all tips are from one single species.

Now, acceptable size of the number of taxa on guide tree is around 100. If you exceed 200 taxa, the memory requirement usually becomes huge and normal desktop computers can not handle. The current limitation of the number of input trees, that is, the limit of number of loci, is ~ 1000. Theoretically, it can be larger, but a problem on numerical calculation now limits the number of loci you can use.

To run tr2 with a guide tree, you need a newick formatted guide tree file, and a gene tree file also in newick format.

Only the first line of the guide tree file is used. In the standard analysis, guide tree tips must contain all taxa found in gene trees. Guide trees can be built any methods such as concatenated ML (eg. RAxML) or coalescent-based species tree methods (eg. ASTRAL). Most importantly , it must be properly rooted. Incorrect rooting often results in over-splitting.

The gene tree file must contain one tree per line. They must be rooted too. Missing taxa are allowed.

Once two files are ready, the command below starts a search algorithm.

./run_tr2.py -g guide.tre -t genetree.tre

To test with bundled example files, use files in “sim4sp” directory.

./run_tr2.py -g sim4sp/4sp.nex10.RTC.tre -t sim4sp/simulated.gene.trees.nex10.4sp.tre

After some intermediate outputs, you will see a tree with delimitation results and a table.

write: <stdout>
species sample
1     12.3
1    11.3
1    15.3
1    14.3
1    13.3
2    7.2
2    10.2

If the tree and table are too large on your console screen, use “-o” option to output them into files.

./run_tr2.py -o out -g sim4sp/4sp.nex10.RTC.tre -t sim4sp/simulated.gene.trees.nex10.4sp.tre

This command ouputs a table and a tree into “out.table.txt” and “out.tre” respectively. You can see the results using any programs.

Now, I use R + “ape” package to check a delimitation result.

library(ape)
tr <- read.tree("./out.tre")
plot(tr)
nodelabels(text=substr(tr$node.label,1,6), bg="white")

These R commands plots the guide tree and delimitation like the  below picture.

outdelimit

The numbers on the nodes indicate average differences of posterior probability scores. If they are positive the node has between-species branches. The negative values suggests that nodes are within species. “nan” indicates there are not enough samples to split/merge the node. “*” signs show the best position of delimitation. In this case, there are 4 putative species.

Multilocus delimitation with tr2: Model comparison

The “tr2” currently has two options for species delimitation. One is calculating posterior probability scores for user-specified delimitation hypotheses. Another option is finding the best delimitation under a guide tree, which specifies a hierarchical structure of species grouping.

The first option is probably useful to compare multiple species groupings and find the best one (such as comparing morphological species vs. mtDNA groups) while the second option can be used without any prior assignments and find species only from gene trees.

Let’s start with the first option. (I assume you have already set up an environment for tr2.)

You must have two input files: A gene tree file in Newick format and a tab-delimited text file which specify associations of species and individual samples.

In a tree file, one line must contain one gene tree. Trees can have missing taxa. They must be rooted. (Yes. The program is based on “rooted triplet”. So, trees must be rooted. If you do not have outgroups, midpoint rooting or RAxML’s “-I f” option often works well.)

In an association file, the first column represents the names of samples. They must be identical to the names of the tree tips. The second and so forth columns are species groups. You can write as many columns as you want. Also, you can use any codes to describe species names.

For example, a table below specifies three alternative delimitations of samples 16.4-20.4

19.4     4     B     sp5
17.4     4     B     sp5
18.4     4     B     sp4
16.4     4     A     sp4
20.4     4     A     sp4

Association files must contain all sample names which appear in the tree file.

Once you have a tree file and an association file, simply run the tr2 command as follows.

./run_tr2.py -a sp_association.txt -t genetrees.tre

Some example files are stored in the “sim4sp” folder. If you use them to test tr2, the command is like this.

./run_tr2.py -a sim4sp/sp.assoc.4sp.txt -t sim4sp/simulated.gene.trees.nex10.4sp.tre

The outputs of this command must be like below.

write: <stdout>
model score
null 51391.76
model1 5.73

The score of “model1” looks much smaller than the “null” model (, which assumes all samples are from one single species). So, you can be quite confident that model1 is a better delimitation.

How to install “tr2”

Once the environment is properly set up, the installation of tr2-delimitation is fairly simple. Download the file of tr2-delimitation from the bitbucket repository and decompress it in any place in your computer.

Just click a cloud-like icon on the left side pane and then click “Download repository” to start downloading. A decompressed folder may have a long name  like “tfujisawa-tr2-delimitation-84f248b5fa48”. Simply rename it like “tr2” if you want a handy access to the folder.

To test if the tr2 (and the environment) is correctly installed, let’s run it with a test data set.

$cd /path/to/yourfolder/tr2-delimitation #move to the installed folder
$python run_tr2.py -t sim4sp/simulated.gene.trees.nex10.4sp.tre -g sim4sp/guide.tree.4sp.tre

If you see a table of delimitation like below, the installation is successful.

write: <stdout>

species sample
1 1.1
1 2.1
1 3.1
1 4.1

...

4 17.4
4 18.4
4 19.4
4 20.4

 

In Unix-like OS’s, the program can be run without calling with python.

$./run_tr2.py -t sim4sp/simulated.gene.trees.nex10.4sp.tre -g sim4sp/guide.tree.4sp.tre

 

If you installed python with Anaconda and created a python2 environment, you need to first activate it,

$activate python2
$python run_tr2.py -t sim4sp/simulated.gene.trees.nex10.4sp.tre -g sim4sp/guide.tree.4sp.tre

 

You can integrate delimitation with species tree inference using the rooted triple consensus. To set up the triplec program, download it from its website, and create a folder named “bin” in the same folder as run_tr2.py and put the triplec in the “bin” folder.

$python run_tr2.py -t sim4sp/simulated.gene.trees.nex10.4sp.tre

If the command above returns a similar outputs, the triplec is correctly called.

How to set up an environment for “tr2”

Installing Python and check versions

Python is required to run the tr2-delimitation on your computer. In many modern operating systems, Python is pre-installed and you do not need to install it by yourself.

However, the installed version of Python matters. The tr2 is written in Python 2,  while, in some recent systems, Python 3 is the default version. This is simply because Python 2 was a standard when I started writing the codes, but Python 3 is becoming a new standard now. (I am translating the tr2 codes into Python3.) So, you first need to check the version of Python installed on your system.

Type on your console,

$python --version

If you see a message like below, you are running Python 2.

Python 2.7.6

In some decent OS’s, both Python2 and 3 are pre-installed, and you can call Python 2 by

$python2

even when your default Python is Python3.

If your environment does not have Python2 at all, visit the Python website and install it, or create a Python2 environment as explained in the next section.

Installing dependencies (scipy/numpy and Java)

Python packages called numpy/scipy is required to run tr2. These packages are for numerical calculations. Visit SciPy website and follow the instructions for your operating system to install Scipy libraries. Installing all Scipy related packages following the instruction is just fine though not all Scipy packages are required to run tr2.

An alternative, easy way to install dependencies is installing an all-in-one suite like Anaconda, which includes Python and related packages. As Anaconda allows you to run multiple versions of Python, you can install it with any version of Python.

If you choose to install Anaconda with Python3, you must run Python2 codes by creating Python2 environment.

$conda create --name python2 python=2 numpy scipy

This “conda” command creates an “environment” where the python version is 2 with numpy/scipy installed. You can switch to it by calling the “activate”  command,

$activate python2

and switch back to the default environment by the “deactivate” command.

(python2) $deactivate

 

Checking packages

You can check if the packages are properly installed by loading them on the interactive shell.

$python

(Just by typing “python”, “interactive shell” starts. You can quit this mode by pressing Ctl+d.)

>>>import numpy, scipy

If it does not return errors, packages are ready to use.

Java is also required to run the consensus tree building program, triplec. Almost all modern operating systems have Java as pre-installed software.

Multilocus delimitation with very short sequences

Are very short sequences useful for species tree reconstruction and species delimitation?

Sequence fragments generated by some NGS platforms are often very short (for example, 100bp with RAD) and each locus contains only a few informative sites. Intuitively, they do not look useful for tree reconstruction using gene tree-based method (that is, species tree estimation using estimated gene trees. The SNP-based methods are not included in this category.) because tree of each locus is unresolved and uninformative for subsequent specie s tree inference.

I did a bit of simulation to check if the gene tree-based species tree reconstruction and delimitation work on very short sequences. Results are rather surprising.

The procedure of simulation is simple.

  1. Set up a species tree.
  2. Simulate 500 gene trees within the species tree assuming multispecies coalescent.
  3. Simulate sequence alignments of 100bp along the gene trees.
  4. Reconstruct gene trees.
  5. Reconstruct a species tree.
  6. Delimit species using the species tree as a guide.

The R codes are available here so you can replicate the simulation. You need following programs to replicate: fastsimcoal2, Seq-gen, RAxML, ASTRAL and tr2.

The species tree used for simulation is below. I set the effective population size for each species in between of T3 and T2. So, I expect large amount of incomplete lineage sorting among sp1, sp2 and sp3. Sample size from each species is 5.

sptree.t

The picture below shows simulated gene trees (upper row), simulated alignments (middle row) and reconstructed gene trees (bottom row) for three loci.

tree.alig

The sequences and reconstructed gene trees appear to be barely informative. Although some distantly related species are monophyletic even in reconstructed trees,  closely related ones are unresolved. The bottom left tree is almost “flat”.

The tree below is a species tree reconstructed by ASTRAL with 500 simulated loci, and a result of delimitation. The numbers on the nodes are scores of delimitation (positive values indicate nodes are between-species branchings, and “*” signs show species groups.)

result.delimitation.500One surprising thing is that ASTRAL was able to reconstruct a correct species tree even with apparently uninformative gene trees. It seems delimitation with tr2 can work correctly too once the species tree is correctly estimated.

This is only one simulation run, but I am quite sure that even very short sequences contain information and can be useful with the gene tree-based reconstruction and delimitation, at least, under ideal conditions.

Of cause, real data contain lots of missing sites/loci. There may be gene flows. I guess the ideal conditions of multispecies coalescent are rarely met. I will test how the methods respond to the violations.

Update: 14/Sep/16

Some users of tr2 reported that it crashed when they input thousands of gene trees of RAD. I found that the numerical calculation underflows when the input is very large. I made some small fixes on the program, but it accepts only  about 1000 loci now.

I think the simulation above says even 500 loci can have enough information to delimit some difficult species. But, I am thinking of how to solve this limitation of the number of loci.