Sorting and Removing Duplicates

Data is often redundant, so queries often return redundant information. For example, if we select the quantitites that have been measured from the survey table, we get this:

%load_ext sqlitemagic

%%sqlite survey.db
select quant from Survey;

We can eliminate the redundant output to make the result more readable by adding the distinct keyword to our query:

%%sqlite survey.db
select distinct quant from Survey;

If we select more than one column—for example, both the survey site ID and the quantity measured—then the distinct pairs of values are returned:

%%sqlite survey.db
select distinct taken, quant from Survey;

Notice in both cases that duplicates are removed even if they didn't appear to be adjacent in the database. Again, it's important to remember that rows aren't actually ordered: they're just displayed that way.

Challenges

1. Write a query that selects distinct dates from the Site table.

As we mentioned earlier, database records are not stored in any particular order. This means that query results aren't necessarily sorted, and even if they are, we often want to sort them in a different way, e.g., by the name of the project instead of by the name of the scientist. We can do this in SQL by adding an order by clause to our query:

%%sqlite survey.db
select * from Person order by ident;

By default, results are sorted in ascending order (i.e., from least to greatest). We can sort in the opposite order using desc (for "descending"):

%%sqlite survey.db
select * from person order by ident desc;

(And if we want to make it clear that we're sorting in ascending order, we can use asc instead of desc.)

We can also sort on several fields at once. For example, this query sorts results first in ascending order by taken, and then in descending order by person within each group of equal taken values:

%%sqlite survey.db
select taken, person from Survey order by taken asc, person desc;

This is easier to understand if we also remove duplicates:

%%sqlite survey.db
select distinct taken, person from Survey order by taken asc, person desc;

Challenges

1. Write a query that returns the distinct dates in the Visited table.

2. Write a query that displays the full names of the scientists in the Person table, ordered by family name.