mscroggs.co.uk
mscroggs.co.uk

subscribe

Blog

 2023-02-03 
Imagine a set of 142 points on a two-dimensional graph. The mean of the \(x\)-values of the points is 54.26. The mean of the \(y\)-values of the points is 47.83. The standard deviation of the \(x\)-values is 16.76. The standard deviation of the \(y\)-values is 26.93.
What are you imagining that the data looks like?
Whatever you're thinking of, it's probably not this:
The datasaurus.
This is the datasaurus, a dataset that was created by Alberto Cairo in 2016 to remind people to look beyond the summary statistics when analysing a dataset.

Anscombe's quartet

In 1972, four datasets with a similar aim were publised. Graphs in statistical analysis by Francis J Anscombe [1] contained four datasets that have become known as Anscombe's quartet: they all have the same mean \(x\)-value, mean \(y\)-value, standard deviation of \(x\)-values, standard deviation of \(y\)-values, linear regression line, as well multiple other values related to correlation and variance. But if you plot them, the four datasets look very different:
Plots of the four datasets that make up Anscombe's quartet. For each set of data: the mean of the \(x\)-values is 9; the mean of the \(y\)-values is 7.5; the standard deviation of the \(x\)-values is 3.32; the standard deviation of the \(y\)-values is 2.03; the correlation coefficient between \(x\) and \(y\) is 0.816; the linear regression line is \(y=3+0.5x\); and coefficient of determination of linear regression is 0.667.
Anscombe noted that there were prevalent attitudes that:
The four datasets were designed to counter these by showing that data exhibiting the same statistics can actually be very very different.

The datasaurus dozen

Anscombe's datasets indicate their point well, but the arrangement of the points is very regular and looks a little artificial when compared with real data sets. In 2017, twelve sets of more realistic-looking data were published (in Same stats, different graphs: generating datasets with varied appearance and identical statistics through simulated annealing by Justin Matejka and George Fitzmaurice [2]).
These datasets—known as the datasaurus dozen—all had the same mean \(x\)-value, mean \(y\)-value, standard deviation of \(x\)-values, standard deviation of \(y\)-values, and corellation coefficient (to two decimal places) as the datasaurus.
The twelve datasets that make up the datasaurus dozen. For each set of data (to two decimal places): the mean of the \(x\)-values is 54.26; the mean of the \(y\)-values is 47.83; the standard deviation of the \(x\)-values is 16.76; the standard deviation of the \(y\)-values is 26.93; the correlation coefficient between \(x\) and \(y\) is -0.06.
Creating datasets like this is not trivial: if you have a set of values for the statistical properties of a dataset, it is difficult to create a dataset with those properties—especially one that looks like a certain shape or pattern. But if you already have one dataset with the desired properties, you can make other datasets with the same properties by very slightly moving every point in a random direction then checking that the properties are the same—if you do this a few times, you'll eventually get a second dataset with the right properties.
The datasets in the datasaurus dozen were generated using this method: repeatedly adjusting all the points ever so slightly, checking if the properties were the same, then keeping the updated data if it's closer to a target shape.

The databet

Using the same method, I generated the databet: a collection of datasets that look like the letters of the alphabet. I started with this set of 100 points resembling a star:
My starting dataset
After a long time repeatedly moving points by a very small amount, my computer eventually generated these 26 datasets, all of which have the same means, standard deviations, and correlation coefficient:
The databet. For each set of data (to two decimal places): the mean of the \(x\)-values is 0.50; the mean of the \(y\)-values is 0.52; the standard deviation of the \(x\)-values is 0.17; the standard deviation of the \(y\)-values is 0.18; the correlation coefficient between \(x\) and \(y\) is 0.16.

Words

Now that we have the alphabet, we can write words using the databet. You can enter a word or phrase here to do this:

Given two data sets with the same number of points, we can make a new larger dataset by including all the points in both the smaller sets. It is possible to write the mean and standard deviation of the larger dataset in terms of the means and standard deviations of the smaller sets: in each case, the statistic of the larger set depends only on the statistics of the smaller sets and not on the actual data.
Applying this to the databet, we see that the datasets that spell words of a fixed length will all have the same mean and standard deviation. (The same is not true, sadly, for the correlation coefficient.) For example, the datasets shown in the following plot both have the same means and standard deviations:
Datasets that spell "TRUE☆" and "FALSE". For both sets of (to two decimal places): the mean of the \(x\)-values is 2.50; the mean of the \(y\)-values is 0.52; the standard deviation of the \(x\)-values is 1.42; the standard deviation of the \(y\)-values is 0.18.
Hopefully by now you agree with me that Anscombe was right: it's very important to plot data as well as looking at the summary statistics.
 
If you want to play with the databet yourself, all the letters are available on GitHub in JSON format. The GitHub repo also includes fonts that you can download and install so you can use Databet Sans in your next important document.

Graphs in statistical analysis by Francis J Anscombe. American Statistician, 1973.
Same stats, different graphs: generating datasets with varied appearance and identical statistics through simulated annealing by Justin Matejka and George Fitzmaurice. Proceedings of the 2017 CHI Conference on Human Factors in Computing Systems, 2017.
×9      ×4      ×4      ×4      ×4
(Click on one of these icons to react to this blog post)

You might also enjoy...

Comments

Comments in green were written by me. Comments in blue were not written by me.
Very cool! Thanks for sharing ????
Jessica
×5   ×9   ×5   ×5        Reply
 Add a Comment 


I will only use your email address to reply to your comment (if a reply is needed).

Allowed HTML tags: <br> <a> <small> <b> <i> <s> <sup> <sub> <u> <spoiler> <ul> <ol> <li> <logo>
To prove you are not a spam bot, please type "l" then "i" then "n" then "e" then "a" then "r" in the box below (case sensitive):
 2016-06-05 
The Game of Life is a cellular automaton invented by John Conway in 1970, and popularised by Martin Gardner.
In Life, cells on a square grid are either alive or dead. It begins at generation 0 with some cells alive and some dead. The cells' aliveness in the following generations are defined by the following rules:
Starting positions can be found which lead to all kinds of behaviour: from making gliders to generating prime numbers. The following starting position is one of my favourites:
It looks boring enough, but in the next generation, it will look like this:
If you want to confirm that I'm not lying, I recommend the free Game of Life Software Golly.

Going backwards

You may be wondering how I designed the starting pattern above. A first, it looks like a difficult task: each cell can be dead or alive, so I need to check every possible combination until I find one. The number of combinations will be \(2^\text{number of cells}\). This will be a very large number.
There are simplifications that can be made, however. Each of the letters above (ignoring the gs) is in a 3×3 block, surrounded by dead cells. Only the cells in the 5×5 block around this can affect the letter. These 5×5 blocks do no overlap, so can be calculated seperately. I doesn't take too long to try all the possibilities for these 5×5 blocks. The gs were then made by starting with an o and trying adding cells below.

Can I make my name?

Yes, you can make your name.
I continued the search and found a 5×5 block for each letter. Simply Enter your name in the box below and these will be combined to make a pattern leading to your name!
Enter your name:
×12      ×13      ×11      ×12      ×13
(Click on one of these icons to react to this blog post)

You might also enjoy...

Comments

Comments in green were written by me. Comments in blue were not written by me.
 Add a Comment 


I will only use your email address to reply to your comment (if a reply is needed).

Allowed HTML tags: <br> <a> <small> <b> <i> <s> <sup> <sub> <u> <spoiler> <ul> <ol> <li> <logo>
To prove you are not a spam bot, please type "rotcev" backwards in the box below (case sensitive):

Archive

Show me a random blog post
 2024 

Feb 2024

Zines, pt. 2

Jan 2024

Christmas (2023) is over
 2023 
▼ show ▼
 2022 
▼ show ▼
 2021 
▼ show ▼
 2020 
▼ show ▼
 2019 
▼ show ▼
 2018 
▼ show ▼
 2017 
▼ show ▼
 2016 
▼ show ▼
 2015 
▼ show ▼
 2014 
▼ show ▼
 2013 
▼ show ▼
 2012 
▼ show ▼

Tags

coins turtles chebyshev anscombe's quartet php rhombicuboctahedron determinants pi crossnumber christmas card estimation game of life stickers binary ucl royal institution bempp frobel reuleaux polygons bubble bobble geogebra graphs stirling numbers accuracy big internet math-off recursion braiding squares video games finite element method crochet logs noughts and crosses folding paper phd error bars dragon curves plastic ratio finite group map projections harriss spiral platonic solids sobolev spaces tmip gaussian elimination quadrilaterals final fantasy inverse matrices hexapawn game show probability live stream reddit light games data visualisation the aperiodical correlation simultaneous equations probability sound a gamut of games convergence curvature data news london 24 hour maths ternary fonts hannah fry logo asteroids mathsteroids menace geometry rugby talking maths in public golden spiral sorting newcastle folding tube maps numerical analysis wool countdown errors matrix of minors martin gardner realhats advent calendar matrix multiplication gerry anderson chalkdust magazine pi approximation day manchester bodmas logic signorini conditions runge's phenomenon hats electromagnetic field manchester science festival approximation mathsjam trigonometry flexagons numbers propositional calculus football dates gather town european cup triangles puzzles tennis craft edinburgh latex graph theory books polynomials golden ratio matt parker zines fractals standard deviation cross stitch boundary element methods statistics dataset inline code arithmetic databet programming sport national lottery machine learning dinosaurs royal baby weather station captain scarlet preconditioning weak imposition matrix of cofactors misleading statistics youtube mean exponential growth interpolation pizza cutting go hyperbolic surfaces christmas oeis draughts javascript pascal's triangle raspberry pi fence posts computational complexity guest posts chess people maths radio 4 matrices nine men's morris palindromes mathslogicbot wave scattering datasaurus dozen london underground pac-man cambridge pythagoras python world cup speed

Archive

Show me a random blog post
▼ show ▼
© Matthew Scroggs 2012–2024