Lecture-0-Scientific-Computing-with-Python.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Introduction to scientific computing with Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "J.R. Johansson (jrjohansson at gmail.com)\n",
    "\n",
    "The latest version of this [IPython notebook](http://ipython.org/notebook.html) lecture is available at [http://github.com/jrjohansson/scientific-python-lectures](http://github.com/jrjohansson/scientific-python-lectures).\n",
    "\n",
    "The other notebooks in this lecture series are indexed at [http://jrjohansson.github.io](http://jrjohansson.github.io)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## The role of computing in science"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Science has traditionally been divided into experimental and theoretical disciplines, but during the last several decades computing has emerged as a very important part of science. Scientific computing is often closely related to theory, but it also has many characteristics in common with experimental work. It is therefore often viewed as a new third branch of science. In most fields of science, computational work is an important complement to both experiments and theory, and nowadays a vast majority of both experimental and theoretical papers involve some numerical calculations, simulations or computer modeling.\n",
    "<center>\n",
    "<img src=\"images/theory-experiment-computation.png\" width=\"300\">\n",
    "</center>\n",
    "\n",
    "In experimental and theoretical sciences there are well established codes of conducts for how results and methods are published and made available to other scientists. For example, in theoretical sciences, derivations, proofs and other results are published in full detail, or made available upon request. Likewise, in experimental sciences, the methods used and the results are published, and all experimental data should be available upon request. It is considered unscientific to withhold crucial details in a theoretical proof or experimental method, that would hinder other scientists from replicating and reproducing the results.\n",
    "\n",
    "In computational sciences there are not yet any well established guidelines for how source code and generated data should be handled. For example, it is relatively rare that source code used in simulations for published papers are provided to readers, in contrast to the open nature of experimental and theoretical work. And it is not uncommon that source code for simulation software is withheld and considered a competitive advantage (or unnecessary to publish). \n",
    "\n",
    "However, this issue has recently started to attract increasing attention, and a number of editorials in high-profile journals have called for increased openness in computational sciences. Some prestigious journals, including Science, have even started to demand of authors to provide the source code for simulation software used in publications to readers upon request. \n",
    "\n",
    "Discussions are also ongoing on how to facilitate distribution of scientific software, for example as supplementary materials to scientific papers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### References"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " * [Reproducible Research in Computational Science](http://dx.doi.org/10.1126/science.1213847), Roger D. Peng, Science 334, 1226 (2011).\n",
    "\n",
    " * [Shining Light into Black Boxes](http://dx.doi.org/10.1126/science.1218263), A. Morin et al., Science 336, 159-160 (2012).\n",
    " \n",
    " * [The case for open computer programs](http://dx.doi.org/doi:10.1038/nature10836), D.C. Ince, Nature 482, 485 (2012)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Requirements on scientific computing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Replication** and **reproducibility** are two of the cornerstones in the scientific method. With respect to numerical work, complying with these concepts have the following practical implications:\n",
    "\n",
    "* Replication: An author of a scientific paper that involves numerical calculations should be able to rerun the simulations and replicate the results upon request. Other scientist should also be able to perform the same calculations and obtain the same results, given the information about the methods used in a publication.\n",
    "\n",
    "* Reproducibility: The results obtained from numerical simulations should be reproducible with an independent implementation of the method, or using a different method altogether. \n",
    "\n",
    "\n",
    "In summary: A sound scientific result should be reproducible, and a sound scientific study should be replicable.\n",
    "\n",
    "\n",
    "To achieve these goals, we need to:\n",
    "\n",
    "* Keep and take note of *exactly* which source code and version that was used to produce data and figures in published papers.\n",
    "\n",
    "* Record information of which version of external software that was used. Keep access to the environment that was used.\n",
    "\n",
    "* Make sure that old codes and notes are backed up and kept for future reference. \n",
    "\n",
    "* Be ready to give additional information about the methods used, and perhaps also the simulation codes, to an interested reader who requests it (even years after the paper was published!).\n",
    "\n",
    "* Ideally codes should be published online, to make it easier for other scientists interested in the codes to access it."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Tools for managing source code"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ensuring replicability and reprodicibility of scientific simulations is a *complicated problem*, but there are good tools to help with this:\n",
    "\n",
    "* Revision Control System (RCS) software. \n",
    "    * Good choices include:\n",
    "        * git - http://git-scm.com\n",
    "        * mercurial - http://mercurial.selenic.com. Also known as `hg`.\n",
    "        * subversion - http://subversion.apache.org. Also known as `svn`.\n",
    "\n",
    "* Online repositories for source code. Available as both private and public repositories. \n",
    "    * Some good alternatives are\n",
    "        * Github - http://www.github.com\n",
    "        * Bitbucket - http://www.bitbucket.com\n",
    "        * Privately hosted repositories on the university's or department's servers.\n",
    "\n",
    "#### Note\n",
    "\t\n",
    "Repositories are also excellent for version controlling manuscripts, figures, thesis files, data files, lab logs, etc. Basically for any digital content that must be preserved and is frequently updated. Again, both public and private repositories are readily available. They are also excellent collaboration tools!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What is Python?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Python](http://www.python.org/) is a modern, general-purpose, object-oriented, high-level programming language.\n",
    "\n",
    "General characteristics of Python:\n",
    "\n",
    "* **clean and simple language:** Easy-to-read and intuitive code, easy-to-learn minimalistic syntax, maintainability scales well with size of projects.\n",
    "* **expressive language:** Fewer lines of code, fewer bugs, easier to maintain.\n",
    "\n",
    "Technical details:\n",
    "\n",
    "* **dynamically typed:** No need to define the type of variables, function arguments or return types.\n",
    "* **automatic memory management:** No need to explicitly allocate and deallocate memory for variables and data arrays. No memory leak bugs. \n",
    "* **interpreted:** No need to compile the code. The Python interpreter reads and executes the python code directly.\n",
    "\n",
    "Advantages:\n",
    "\n",
    "* The main advantage is ease of programming, minimizing the time required to develop, debug and maintain the code.\n",
    "* Well designed language that encourage many good programming practices:\n",
    " * Modular and object-oriented programming, good system for packaging and re-use of code. This often results in more transparent, maintainable and bug-free code.\n",
    " * Documentation tightly integrated with the code.\n",
    "* A large standard library, and a large collection of add-on packages.\n",
    "\n",
    "Disadvantages:\n",
    "\n",
    "* Since Python is an interpreted and dynamically typed programming language, the execution of python code can be slow compared to compiled statically typed programming languages, such as C and Fortran. \n",
    "* Somewhat decentralized, with different environment, packages and documentation spread out at different places. Can make it harder to get started."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## What makes python suitable for scientific computing?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<img src=\"images/optimizing-what.png\" width=\"600\">\n",
    "\n",
    "* Python has a strong position in scientific computing: \n",
    "    * Large community of users, easy to find help and documentation.\n",
    "\n",
    "* Extensive ecosystem of scientific libraries and environments\n",
    "    * numpy: http://numpy.scipy.org - Numerical Python\n",
    "    * scipy: http://www.scipy.org -  Scientific Python\n",
    "    * matplotlib: http://www.matplotlib.org - graphics library\n",
    "\n",
    "* Great performance due to close integration with time-tested and highly optimized codes written in C and Fortran:\n",
    "    * blas, atlas blas, lapack, arpack, Intel MKL, ...\n",
    "\n",
    "* Good support for \n",
    "    * Parallel processing with processes and threads\n",
    "    * Interprocess communication (MPI)\n",
    "    * GPU computing (OpenCL and CUDA)\n",
    "\n",
    "* Readily available and suitable for use on high-performance computing clusters. \n",
    "\n",
    "* No license costs, no unnecessary use of research budget.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### The scientific python software stack"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<!-- <img src=\"files/images/scientific-python-stack.svg\" width=\"300\"> -->\n",
    "<img src=\"images/scientific-python-stack.png\" width=\"300\">"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python environments"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Python is not only a programming language, but often also refers to the standard implementation of the interpreter (technically referred to as [CPython](http://en.wikipedia.org/wiki/CPython)) that actually runs the python code on a computer.\n",
    "\n",
    "There are also many different environments through which the python interpreter can be used. Each environment has different advantages and is suitable for different workflows. One strength of python is that it is versatile and can be used in complementary ways, but it can be confusing for beginners so we will start with a brief survey of python environments that are useful for scientific computing."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python interpreter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The standard way to use the Python programming language is to use the Python interpreter to run python code. The python interpreter is a program that reads and execute the python code in files passed to it as arguments. At the command prompt, the command ``python`` is used to invoke the Python interpreter.\n",
    "\n",
    "For example, to run a file ``my-program.py`` that contains python code from the command prompt, use::\n",
    "\n",
    "    $ python my-program.py\n",
    "\n",
    "We can also start the interpreter by simply typing ``python`` at the command line, and interactively type python code into the interpreter. \n",
    "\n",
    "<!-- <img src=\"files/images/python-screenshot.jpg\" width=\"600\"> -->\n",
    "<img src=\"images/python-screenshot.jpg\" width=\"600\">\n",
    "\n",
    "\n",
    "This is often how we want to work when developing scientific applications, or when doing small calculations. But the standard python interpreter is not very convenient for this kind of work, due to a number of limitations."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IPython"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "IPython is an interactive shell that addresses the limitation of the standard python interpreter, and it is a work-horse for scientific use of python. It provides an interactive prompt to the python interpreter with a greatly improved user-friendliness.\n",
    "\n",
    "<!-- <img src=\"files/images/ipython-screenshot.jpg\" width=\"600\"> -->\n",
    "<img src=\"images/ipython-screenshot.jpg\" width=\"600\">\n",
    "\n",
    "Some of the many useful features of IPython includes:\n",
    "\n",
    "* Command history, which can be browsed with the up and down arrows on the keyboard.\n",
    "* Tab auto-completion.\n",
    "* In-line editing of code.\n",
    "* Object introspection, and automatic extract of documentation strings from python objects like classes and functions.\n",
    "* Good interaction with operating system shell.\n",
    "* Support for multiple parallel back-end processes, that can run on computing clusters or cloud services like Amazon EC2.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### IPython notebook"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[IPython notebook](http://ipython.org/notebook.html) is an HTML-based notebook environment for Python, similar to Mathematica or Maple. It is based on the IPython shell, but provides a cell-based environment with great interactivity, where calculations can be organized and documented in a structured way.\n",
    "\n",
    "<!-- <img src=\"files/images/ipython-notebook-screenshot.jpg\" width=\"800\"> -->\n",
    "<img src=\"images/ipython-notebook-screenshot.jpg\" width=\"800\">\n",
    "\n",
    "Although using a web browser as graphical interface, IPython notebooks are usually run locally, from the same computer that run the browser. To start a new IPython notebook session, run the following command:\n",
    "\n",
    "    $ ipython notebook\n",
    "\n",
    "from a directory where you want the notebooks to be stored. This will open a new browser window (or a new tab in an existing window) with an index page where existing notebooks are shown and from which new notebooks can be created."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Spyder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[Spyder](http://code.google.com/p/spyderlib/) is a MATLAB-like IDE for scientific computing with python. It has the many advantages of a traditional IDE environment, for example that everything from code editing, execution and debugging is carried out in a single environment, and work on different calculations can be organized as projects in the IDE environment.\n",
    "\n",
    "<!-- <img src=\"files/images/spyder-screenshot.jpg\" width=\"800\"> -->\n",
    "<img src=\"images/spyder-screenshot.jpg\" width=\"800\">\n",
    "\n",
    "Some advantages of Spyder:\n",
    "\n",
    "* Powerful code editor, with syntax high-lighting, dynamic code introspection and integration with the python debugger.\n",
    "* Variable explorer, IPython command prompt.\n",
    "* Integrated documentation and help."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Versions of Python"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are currently two versions of python: Python 2 and Python 3. Python 3 will eventually supercede Python 2, but it is not backward-compatible with Python 2. A lot of existing python code and packages has been written for Python 2, and it is still the most wide-spread version. For these lectures either version will be fine, but it is probably easier to stick with Python 2 for now, because it is more readily available via prebuilt packages and binary installers.\n",
    "\n",
    "To see which version of Python you have, run\n",
    "    \n",
    "    $ python --version\n",
    "    Python 2.7.3\n",
    "    $ python3.2 --version\n",
    "    Python 3.2.3\n",
    "\n",
    "Several versions of Python can be installed in parallel, as shown above.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Installation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Conda"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The best way set-up an scientific Python environment is to use the cross-platform package manager `conda` from Continuum Analytics. First download and install miniconda http://conda.pydata.org/miniconda.html or Anaconda (see below). Next, to install the required libraries for these notebooks, simply run:\n",
    "\n",
    "    $ conda install ipython ipython-notebook spyder numpy scipy sympy matplotlib cython\n",
    "\n",
    "This should be sufficient to get a working environment on any platform supported by `conda`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Linux"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In Ubuntu Linux, to installing python and all the requirements run:\n",
    "\n",
    "    $ sudo apt-get install python ipython ipython-notebook\n",
    "$ sudo apt-get install python-numpy python-scipy python-matplotlib python-sympy\n",
    "    $ sudo apt-get install spyder"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### MacOS X"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "*Macports*\n",
    "\n",
    "Python is included by default in Mac OS X, but for our purposes it will be useful to install a new python environment using [Macports](http://www.macports.org/), because it makes it much easier to install all the required additional packages. Using Macports, we can install what we need with:\n",
    "\n",
    "    $ sudo port install py27-ipython +pyside+notebook+parallel+scientific\n",
    "    $ sudo port install py27-scipy py27-matplotlib py27-sympy\n",
    "    $ sudo port install py27-spyder\n",
    "\n",
    "These will associate the commands `python` and `ipython` with the versions installed via macports (instead of the one that is shipped with Mac OS X), run the following commands:\n",
    "\n",
    "    $ sudo port select python python27\n",
    "    $ sudo port select ipython ipython27\n",
    "\n",
    "*Fink*\n",
    "\n",
    "Or, alternatively, you can use the [Fink](http://www.finkproject.org/) package manager. After installing Fink, use the following command to install python and the packages that we need:\n",
    "\n",
    "    $ sudo fink install python27 ipython-py27 numpy-py27 matplotlib-py27 scipy-py27 sympy-py27\n",
    "    $ sudo fink install spyder-mac-py27"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Windows"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Windows lacks a good packaging system, so the easiest way to setup a Python environment is to install a pre-packaged distribution. Some good alternatives are:\n",
    "\n",
    " * [Enthought Python Distribution](http://www.enthought.com/products/epd.php). EPD is a commercial product but is available free for academic use.\n",
    " * [Anaconda](http://continuum.io/downloads.html). The Anaconda Python distribution comes with many scientific computing and data science packages and is free, including for commercial use and redistribution. It also has add-on products such as Accelerate, IOPro, and MKL Optimizations, which have free trials and are free for academic use.\n",
    " * [Python(x,y)](http://code.google.com/p/pythonxy/). Fully open source.\n",
    "\n",
    "\n",
    "\n",
    "#### Note\n",
    "\n",
    "EPD and Anaconda are also available for Linux and Max OS X."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Further reading"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    " * [Python](http://www.python.org). The official Python web site.\n",
    " * [Python tutorials](http://docs.python.org/2/tutorial). The official Python tutorials.\n",
    " * [Think Python](http://www.greenteapress.com/thinkpython). A free book on Python."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Python and module versions"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Since there are several different versions of Python and each Python package has its own release cycle and version number (for example scipy, numpy, matplotlib, etc., which we installed above and will discuss in detail in the following lectures), it is important for the reproducibility of an IPython notebook to record the versions of all these different software packages. If this is done properly it will be easy to reproduce the environment that was used to run a notebook, but if not it can be hard to know what was used to produce the results in a notebook.\n",
    "\n",
    "To encourage the practice of recording Python and module versions in notebooks, I've created a simple IPython extension that produces a table with versions numbers of selected software components. I believe that it is a good practice to include this kind of table in every notebook you create. \n",
    "\n",
    "To install this IPython extension, use `pip install version_information`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Collecting version-information\n",
      "Installing collected packages: version-information\n",
      "Successfully installed version-information-1.0.3\n"
     ]
    }
   ],
   "source": [
    "# you only need to do this once\n",
    "!pip install --upgrade version_information"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "or alternatively run (deprecated method):"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "collapsed": false
   },
   "source": [
    "# you only need to do this once\n",
    "%install_ext http://raw.github.com/jrjohansson/version_information/master/version_information.py"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, to load the extension and produce the version table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "data": {
      "application/json": {
       "Software versions": [
        {
         "module": "Python",
         "version": "2.7.10 64bit [GCC 4.2.1 (Apple Inc. build 5577)]"
        },
        {
         "module": "IPython",
         "version": "3.2.1"
        },
        {
         "module": "OS",
         "version": "Darwin 14.1.0 x86_64 i386 64bit"
        },
        {
         "module": "numpy",
         "version": "1.9.2"
        },
        {
         "module": "scipy",
         "version": "0.16.0"
        },
        {
         "module": "matplotlib",
         "version": "1.4.3"
        },
        {
         "module": "sympy",
         "version": "0.7.6"
        },
        {
         "module": "version_information",
         "version": "1.0.3"
        }
       ]
      },
      "text/html": [
       "<table><tr><th>Software</th><th>Version</th></tr><tr><td>Python</td><td>2.7.10 64bit [GCC 4.2.1 (Apple Inc. build 5577)]</td></tr><tr><td>IPython</td><td>3.2.1</td></tr><tr><td>OS</td><td>Darwin 14.1.0 x86_64 i386 64bit</td></tr><tr><td>numpy</td><td>1.9.2</td></tr><tr><td>scipy</td><td>0.16.0</td></tr><tr><td>matplotlib</td><td>1.4.3</td></tr><tr><td>sympy</td><td>0.7.6</td></tr><tr><td>version_information</td><td>1.0.3</td></tr><tr><td colspan='2'>Sat Aug 15 10:38:48 2015 JST</td></tr></table>"
      ],
      "text/latex": [
       "\\begin{tabular}{|l|l|}\\hline\n",
       "{\\bf Software} & {\\bf Version} \\\\ \\hline\\hline\n",
       "Python & 2.7.10 64bit [GCC 4.2.1 (Apple Inc. build 5577)] \\\\ \\hline\n",
       "IPython & 3.2.1 \\\\ \\hline\n",
       "OS & Darwin 14.1.0 x86\\_64 i386 64bit \\\\ \\hline\n",
       "numpy & 1.9.2 \\\\ \\hline\n",
       "scipy & 0.16.0 \\\\ \\hline\n",
       "matplotlib & 1.4.3 \\\\ \\hline\n",
       "sympy & 0.7.6 \\\\ \\hline\n",
       "version_information & 1.0.3 \\\\ \\hline\n",
       "\\hline \\multicolumn{2}{|l|}{Sat Aug 15 10:38:48 2015 JST} \\\\ \\hline\n",
       "\\end{tabular}\n"
      ],
      "text/plain": [
       "Software versions\n",
       "Python 2.7.10 64bit [GCC 4.2.1 (Apple Inc. build 5577)]\n",
       "IPython 3.2.1\n",
       "OS Darwin 14.1.0 x86_64 i386 64bit\n",
       "numpy 1.9.2\n",
       "scipy 0.16.0\n",
       "matplotlib 1.4.3\n",
       "sympy 0.7.6\n",
       "version_information 1.0.3\n",
       "Sat Aug 15 10:38:48 2015 JST"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "%load_ext version_information\n",
    "\n",
    "%version_information numpy, scipy, matplotlib, sympy, version_information"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}