228 lines
7.7 KiB
Plaintext
228 lines
7.7 KiB
Plaintext
{
|
|
"cells": [
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"# Visualizing Uncertainties"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"For any scientific measurement, accurate accounting of uncertainties is nearly as important, if not more so, as accurate reporting of the number itself.\n",
|
|
"For example, imagine that I am using some astrophysical observations to estimate the Hubble Constant, the local measurement of the expansion rate of the Universe.\n",
|
|
"I know that the current literature suggests a value of around 70 (km/s)/Mpc, and I measure a value of 74 (km/s)/Mpc with my method. Are the values consistent? The only correct answer, given this information, is this: there is no way to know.\n",
|
|
"\n",
|
|
"Suppose I augment this information with reported uncertainties: the current literature suggests a value of 70 ± 2.5 (km/s)/Mpc, and my method has measured a value of 74 ± 5 (km/s)/Mpc. Now are the values consistent? That is a question that can be quantitatively answered.\n",
|
|
"\n",
|
|
"In visualization of data and results, showing these errors effectively can make a plot convey much more complete information."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Basic Errorbars\n",
|
|
"\n",
|
|
"One standard way to visualize uncertainties is using an errorbar. A basic errorbar can be created with a single Matplotlib function call, as shown in the following figure:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"tags": []
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"%matplotlib inline\n",
|
|
"import matplotlib.pyplot as plt\n",
|
|
"plt.style.use('seaborn-whitegrid')\n",
|
|
"import numpy as np"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"jupyter": {
|
|
"outputs_hidden": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"x = np.linspace(0, 10, 50)\n",
|
|
"dy = 0.8\n",
|
|
"y = np.sin(x) + dy * np.random.randn(50)\n",
|
|
"\n",
|
|
"plt.errorbar(x, y, yerr=dy, fmt='.k');"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Here the `fmt` is a format code controlling the appearance of lines and points, and it has the same syntax as the shorthand used in `plt.plot`, outlined in the previous chapter and earlier in this chapter.\n",
|
|
"\n",
|
|
"In addition to these basic options, the `errorbar` function has many options to fine-tune the outputs.\n",
|
|
"Using these additional options you can easily customize the aesthetics of your errorbar plot.\n",
|
|
"I often find it helpful, especially in crowded plots, to make the errorbars lighter than the points themselves (see the following figure):"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"jupyter": {
|
|
"outputs_hidden": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"plt.errorbar(x, y, yerr=dy, fmt='o', color='black',\n",
|
|
" ecolor='lightgray', elinewidth=3, capsize=0);"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"In addition to these options, you can also specify horizontal errorbars, one-sided errorbars, and many other variants.\n",
|
|
"For more information on the options available, refer to the docstring of `plt.errorbar`."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"## Continuous Errors\n",
|
|
"\n",
|
|
"In some situations it is desirable to show errorbars on continuous quantities.\n",
|
|
"Though Matplotlib does not have a built-in convenience routine for this type of application, it's relatively easy to combine primitives like `plt.plot` and `plt.fill_between` for a useful result.\n",
|
|
"\n",
|
|
"Here we'll perform a simple *Gaussian process regression*, using the Scikit-Learn API (see [Introducing Scikit-Learn](05.02-Introducing-Scikit-Learn.ipynb) for details).\n",
|
|
"This is a method of fitting a very flexible nonparametric function to data with a continuous measure of the uncertainty.\n",
|
|
"We won't delve into the details of Gaussian process regression at this point, but will focus instead on how you might visualize such a continuous error measurement:"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"jupyter": {
|
|
"outputs_hidden": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"from sklearn.gaussian_process import GaussianProcessRegressor\n",
|
|
"\n",
|
|
"# define the model and draw some data\n",
|
|
"model = lambda x: x * np.sin(x)\n",
|
|
"xdata = np.array([1, 3, 5, 6, 8])\n",
|
|
"ydata = model(xdata)\n",
|
|
"\n",
|
|
"# Compute the Gaussian process fit\n",
|
|
"gp = GaussianProcessRegressor()\n",
|
|
"gp.fit(xdata[:, np.newaxis], ydata)\n",
|
|
"\n",
|
|
"xfit = np.linspace(0, 10, 1000)\n",
|
|
"yfit, dyfit = gp.predict(xfit[:, np.newaxis], return_std=True)"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"We now have `xfit`, `yfit`, and `dyfit`, which sample the continuous fit to our data.\n",
|
|
"We could pass these to the `plt.errorbar` function as in the previous section, but we don't really want to plot 1,000 points with 1,000 errorbars.\n",
|
|
"Instead, we can use the `plt.fill_between` function with a light color to visualize this continuous error (see the following figure):"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {
|
|
"collapsed": false,
|
|
"jupyter": {
|
|
"outputs_hidden": false
|
|
}
|
|
},
|
|
"outputs": [],
|
|
"source": [
|
|
"# Visualize the result\n",
|
|
"plt.plot(xdata, ydata, 'or')\n",
|
|
"plt.plot(xfit, yfit, '-', color='gray')\n",
|
|
"plt.fill_between(xfit, yfit - dyfit, yfit + dyfit,\n",
|
|
" color='gray', alpha=0.2)\n",
|
|
"plt.xlim(0, 10);"
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "markdown",
|
|
"metadata": {},
|
|
"source": [
|
|
"Take a look at the `fill_between` call signature: we pass an x value, then the lower *y*-bound, then the upper *y*-bound, and the result is that the area between these regions is filled.\n",
|
|
"\n",
|
|
"The resulting figure gives an intuitive view into what the Gaussian process regression algorithm is doing: in regions near a measured data point, the model is strongly constrained, and this is reflected in the small model uncertainties.\n",
|
|
"In regions far from a measured data point, the model is not strongly constrained, and the model uncertainties increase.\n",
|
|
"\n",
|
|
"For more information on the options available in `plt.fill_between` (and the closely related `plt.fill` function), see the function docstring or the Matplotlib documentation.\n",
|
|
"\n",
|
|
"Finally, if this seems a bit too low-level for your taste, refer to [Visualization With Seaborn](04.14-Visualization-With-Seaborn.ipynb), where we discuss the Seaborn package, which has a more streamlined API for visualizing this type of continuous errorbar."
|
|
]
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
},
|
|
{
|
|
"cell_type": "code",
|
|
"execution_count": null,
|
|
"metadata": {},
|
|
"outputs": [],
|
|
"source": []
|
|
}
|
|
],
|
|
"metadata": {
|
|
"anaconda-cloud": {},
|
|
"jupytext": {
|
|
"formats": "ipynb,md"
|
|
},
|
|
"kernelspec": {
|
|
"display_name": "Python 3",
|
|
"language": "python",
|
|
"name": "python3"
|
|
},
|
|
"language_info": {
|
|
"codemirror_mode": {
|
|
"name": "ipython",
|
|
"version": 3
|
|
},
|
|
"file_extension": ".py",
|
|
"mimetype": "text/x-python",
|
|
"name": "python",
|
|
"nbconvert_exporter": "python",
|
|
"pygments_lexer": "ipython3",
|
|
"version": "3.8.8"
|
|
},
|
|
"widgets": {
|
|
"application/vnd.jupyter.widget-state+json": {
|
|
"state": {},
|
|
"version_major": 2,
|
|
"version_minor": 0
|
|
}
|
|
}
|
|
},
|
|
"nbformat": 4,
|
|
"nbformat_minor": 4
|
|
}
|