Switch from IDL to Python

Here are a few basic data analysis tasks for scientists and engineers, demonstrated in Python. They all assume you have installed Python.

Installing Python for the first time

Anaconda Python (the compact installer is called Miniconda) is a small download and install–it won’t take several gigabytes of space like Matlab and IDL.

Install Miniconda, then in a new window type

conda create -n py35 python=3.5 pyqt=4

source activate py35

This makes Python 3.5 the default.

You can persistently use Python 3.5 by adding to your ~/.bashrc or ~/.bash_profile the line

source activate py35

Install Data Analysis modules

conda install pandas scipy matplotlib

IDL to Python/Numpy syntax

table of NumPy/Python equivalents to common IDL commands

IDL is a bit distinctive in its syntax. IDL and Matlab syntax remind me a bit of Fortran, while Python is C-based, including that Python has 0-based indexing and C-order array axes.

Plot 3-D data

Using Mayavi, Matplotlib or more advanced modules, one can make very high quality, manipulable volume plots. Install Mayavi in the Python 3.5 environment created above.

conda install -c menpo mayavi

Create a file scalar_field.py with the content

#!/usr/bin/env python
from mayavi import mlab
import numpy as np

x, y, z = np.mgrid[-10:10:20j, -10:10:20j, -10:10:20j]
s = np.sin(x*y*z)/(x*y*z)

mlab.pipeline.volume(mlab.pipeline.scalar_field(x,y,z,s))
mlab.show()

Mayavi Error fix

If you an error about the incorrect number of arguments, try this one-line Mayavi patch fix

Ipyvolume 3-D example

Jupyter Notebook makes an IDE in your web browser, and you can make remarkable animated, interactive 3-D plots as well with Ipyvolume.

conda install jupyter
pip install ipyvolume
web browser 3-D animation flow field
Ipyvolume interactive flow field.

HDF5: read/write

HDF5 can handle enormous datasets quickly and easily. Assume you have an HDF5 file terabyte.h5 with double-precision float variable X of size 100000x2048x2048 (3.4 TB). Let’s load the first frame of the 2048x2048 image data, and write it variable first to image.h5. The with syntax uses Python’s context manager, which closes the file upon exiting the indented code section under with.

import h5py

with h5py.File('terabyte.h5') as f:
    img = f['X'][0,...]

with h5py.File('image.h5') as f:
    f['first'] = img

Call IDL from Python or Python from IDL

IDL 8.5 and newer supports Python 2.7 and 3.5.

Call Matlab from Python and Python from Matlab

In case you or a collaborator uses Matlab or Matlab code, you can:

Leave a Comment