Import Python user modules in Matlab

This page shows how to setup Matlab with user Python install (e.g. Anaconda Python) on Linux, Mac or Windows.

Install Python for Matlab

Matlab is designed to work with specific Python versions for each Matlab version. Matlab will not specifically tell you when you’re using an incompatible Python version, but you may get unstable operation or errors.

In general, we recommend for Python ↔ Matlab interfacing:

  • Matlab ≥ R2018b, as R2018b made exchanging arrays with Python much more efficient and easy.
  • Python 3.6 or 3.7 for significant new functionality and performance
Matlab versionPython version
R2019a3.5, 3.6, 3.7
R2018b3.5, 3.6

You can just type in Matlab

pyversion

If you don’t see your desired Python version or install, here’s how to select a specific Python version.

select Python version to run from Matlab

This section is optional, for example if Matlab pyversion doesn’t find the desired Python version.

  1. Create and activate Python virtual environment (assuming Miniconda Python). This example is for Python 3.7.

    conda create -n py37 python==3.7
    
    conda activate py37
    
  2. the exact location of the Python executable to put in Matlab pyversion() command is determined by:

    python -c "import sys; print(sys.executable)"
    

    This examples assumes a Python 3.7 executable is at ~/miniconda3/envs/py37/bin/python. Setup Matlab to use Python 3.7 by adding to ~/Documents/MATLAB/startup.m

    pyversion('~/miniconda3/envs/py37/bin/python')
    
  3. verify Matlab → Python config by typing pyversion from within Matlab:

    version: '3.7'
    executable: '~/miniconda3/envs/py37/bin/python'
    library: '~/miniconda3/envs/py37/lib/libpython3.7m.so'
      home: '~/miniconda3/envs/py37'
    isloaded: 0
    

Install Python user programs

This example is for the “FindSSH” Python program, which concurrently searches out SSH servers on an IPv4 subnet. It’s useful to find IoT or Raspberry Pi etc. IP addresses quickly.

  1. setup Python user module, from Terminal or Command Prompt:

    git clone https://github.com/scivision/findssh
        
    cd findssh
    
    python -m pip install -e .[tests]   
    
  2. Check the Python code is working by itself from Terminal:

    python -m pytest   
    

    If this makes errors, they will need to be resolved before a Python user library will work in Matlab!

Notes

  • Python module import is implicit in the Matlab → Python module function call. There is no need to import numpy etc. from Matlab
  • Python executable choice persists across Matlab sessions–Matlab “remembers” even after you restart Matlab or reboot the computer.
  • editing imported Python module code requires restarting Matlab to take effect.

reference

Older versions of Matlab back to R2014b can also call Python from Matlab, but is more difficult to use, particularly for arrays / matrices:

Matlab versionPython version
R2018a3.5, 3.6
R2017b3.4, 3.5, 3.6
R2017a3.4, 3.5
R2016b3.3, 3.4, 3.5
R2016a3.3, 3.4
R2015b3.3, 3.4
R2015a3.3, 3.4
R2014b3.3

Troubleshooting

If you get

Error using pyversion Path argument does not specify a valid executable.

Ensure the Python executable path in file matlab.settings matches the desired Python executable. On Linux, this file is at ~/.matlab/R2018b/matlab.settings.

matlab.settings must have a section like:

<settings name="external">
    <settings name="interfaces">
        <settings name="python">
            <key name="Version">
                <string minArraySize="4" maxArraySize="4">
                    <value><![CDATA[3.6]]></value>
                    <value><![CDATA[/home/username/anaconda3/bin/python]]></value>
                    <value><![CDATA[/home/username/anaconda3]]></value>
                    <value><![CDATA[/home/username/anaconda3/lib/libpython3.6m.so]]></value>
                </string>
            </key>
        </settings>
    </settings>
</settings>

reference

Install Matlab Engine API in Python

Related: Call Python from Matlab


Alternative: Call Matlab .m function code using free open-source GNU Octave from Python with oct2py.

Matlab R2019a Engine API works from Python 3.7, 3.6, 3.5 and 2.7. That is, you can call Matlab functions from Python code!

Install

  1. find the directory Matlab is installed in by from Terminal / Command Prompt:

    matlab -nojvm  -r "matlabroot"
    

    then close Matlab: exit

  2. assuming Matlab root is like MATLAB/R2019a/ then do from Terminal / Command Prompt:

    cd MATLAB/R2019a/extern/engines/python
    
  3. This step depends on your operating system. From Terminal / Command Prompt:

    • Mac or Linux:

       python setup.py build --build-base=$(mktemp -d) install
      
    • Windows:

      python setup.py build --build-base=%TEMP% install
      

Permissions Error

If error about needing root permissions, you’re probably installing in system Python. I normally recommend using Anaconda Python instead, but if you must use system Python:

python setup.py build --build-base=$(mktemp -d) install --user

Examples

Python → Matlab

import matlab.engine
eng = matlab.engine.start_matlab('-nojvm')
y=eng.asin(1.)

eng.quit()

The -nojvm Matlab Engine option is essential for not having the Matlab Engine hang forever upon startup. It should take less than 1 second for Matlab Engine to start when called from Python.

matlab.double → Numpy

enclose the matlab.double variable in numpy.asarray()

Python float → Matlab Engine

Python floats pass into Matlab Engine by including a period . after the number.

  • asin(1) fails
  • asin(1.) works

numpy.ndarray → Matlab Engine

Matlab Engine from Python can pass N-dimensional arrays.

Asynchronous Matlab Engine

Matlab Engine provides asynchronous call with background=True option  

Notes

Windows native Meld diff

Git users often use Meld to resolve differences, three-way or two-way, graphically. Meld is also easily used with Git from Windows.

  1. Download/install Meld for Windows.
  2. Download/install Git for Windows.
  3. at Command Prompt type:

    git config --global diff.tool meld
    git config --global merge.tool meld
    
    git config --global mergetool.meld.path "%PROGRAMFILES(X86)%\Meld\meld.exe"
    

Windows with Cygwin or WSL git file permissions

When using native Windows Git client along with WSL or Cygwin Git clients, there can be nuisance dirty repo messages like:

old mode 100755
new mode 100644

This problem is easily resolved as follows, for each Git client you use as below:

  • Cygwin
  • Window Subsystem for Linux
  • native Windows
git config --global core.filemode false

Teaching the discovery process

The usual course instructor was unexpectedly out of town, and they thought I would be an ideal fit for their RF systems engineering lab course. I have a lifelong experience base in all aspects of RF systems design, so I thought I would turn this into a live learning opportunity for the students as well. I did precisely zero preparation for the course.

Upon arriving, I introduced myself as someone who had made a career in RF systems design before college. I said that I wanted to show them how I had to teach myself things as a formally educated engineering student, and we’d learn together the topic at hand. We used intelligently chosen web searches, some of which turned into dead ends. We looked at data sheets of commercial products to get a sense of what performance was reasonable for the RF subsystem we were designing. Then we set about to model the device, including S parameters and other KPI.

The point of this self-induced exercise was, we should not be intimidated by something we haven’t done before, or haven’t done recently. We shouldn’t be afraid to say I don’t know, in fact this is an asset of experience, to know the boundaries of one’s engineering competence. How I approach potential engineering contracts is that I say I will try to find someone with expertise in the needed area. Maybe it will be me, but if not I can help direct the prospective client to the needed expert.

That shows authentic confidence and competence in one’s practice area, and that one isn’t overreaching to get extra income that quarter.

Fortran build systems

We build all our Fortran projects using the CMake build system. We are increasingly using Meson as well as CMake as Fortran support in Meson is growing rapidly (due in part to our code contributions). The most important aspects of a build system are generation speed and build speed.

Generation speed

Regardless of team size, for non-trivial projects in any compiled language, build speed can be a significant pain point. Even for medium projects, the seconds lost repeatedly building add up to annoyance and lost productivity for developers.

In this regard, CMake and Meson are among the fastest generators, while SCons and Autotools are among the slowest.

Build speed

Ninja is known to be generally faster than GNU Make, particularly for the case where only a few out of very many files need to be rebuilt. This is the usual case for developers. So as long as the generator works with Ninja, you can get fast builds.

Meson generates Ninja build files. Meson 0.50 massively increased support for Fortran compilers and features, so check out Meson again if it didn’t work for you on Fortran previously.

using Meson vs. CMake

Meson is a Python ≥ 3.5 program without external dependencies. It’s easy to run the latest development snapshot (master Git branch) as a result.

git clone https://github.com/mesonbuild/meson
cd meson
pip install -e .

Since Meson generates Ninja build files, you will need to download Ninja. With Meson, the code build process goes like:

meson ..
ninja

while with CMake the code build is like:

cmake ..
cmake --build . -j

Modern Fortran submodule vs. preprocessor

Fortran has a long legacy of preprocessing systems, reaching back at least to the 1970s when C was not yet widely known. I would argue that modern Fortran ≥ 2008 has eliminated most preprocessors use cases. It’s usually much cleaner and quicker (build/runtime) to use Fortran submodule to switch functionality such as MKL vs. Atlas vs. Netlib Lapack using CMake, for example.

Numerous Python preprocessors have arisen (and sunk) along with the usual C/C++-based preprocessors. The only thing we use preprocessors for is very simple single-line statements, where adding a submodule would be significantly more cumbersome than a one-liner preprocessor statement.

Historically, very elaborate Fortran preprocessors were developed such that they nearly became a language unto themselves.

These extensive preprocessors have been made obsolete by advances from Fortran 77 onward.

CMake force linking static libs

CMake by default searches for shared libs before static libs. Sometimes a project needs to specifically link external or internal static libs, even if shared libs are present. A simple cross-platform way to do this is to include in CMakeLists.txt, before any find_library() or find_package():

if(NOT BUILD_SHARED_LIBS)
  set(CMAKE_FIND_LIBRARY_SUFFIXES .a .lib)
endif()

This takes advantage of de facto static library extensions for MacOS, Windows and Linux, using CMake system variables.

read huge RINEX files in Python with GeoRINEX

Reading RINEX files in Python or other languages historically required compiling or buying complex software. Several years ago, we created the GeoRINEX Python 3 program and library to be used with RINEX OBS and NAV files, including Hatanaka compressed files or other compression wrappers like ZIP.

Version 1.9 of GeoRINEX added the ability to read only every N seconds from a RINEX file with option georinex.load(..., interval=). This greatly speeds up reading where coarser time intervals than the RINEX file provides is needed. GeoRINEX uses performance-oriented techniques to read RINEX files at speeds approaching compiled languages, using Pure Python + Numpy and Xarray for metadata rich results.

If you use GNSS and RINEX in your work, you will probably like GeoRINEX for Python 3.

GUI programs to view HDF5/NetCDF4 data

Related: Check HDF5 files for corruption


Here are a few options for graphically browsing and editing HDF5 data files from Linux, Mac and Windows.

HDFview

HDFview is maintained by the curators of the HDF5 format, the HDF Group. HDFView enables editing (writing) as well as reading HDF5.

  • Linux: apt install hdfview
  • Mac, Windows download

HDFview needs compiling if running from source, unlike ImageJ, which is a pure platform-independent Java program.

Ubuntu 18.04

Ubuntu 18.04 (still!) has a broken hdfview such that the program opens, but silently fails to read any HDF5/NetCDF4 files. As an alternative, I use HDF-Compass on Ubuntu 18.04.

HDF-Compass

HDF Compass is also by the HDF Group. Unlike HDFView, HDF Compass is read-only.

apt install hdf-compass

Instead of using the menus, I typically open one or more HDF5/NetCDF4 files from Terminal like:

HDFCompass f1.h5 f2.nc

ViTables

ViTables is a Python-based HDF5 viewing program.

apt install libhdf5-dev

python -m pip install pyqt5

python -m pip install vitables

Unfortunately at this time PyQT5 doesn’t have a .whl for armhf, so that means ViTables only works on non-ARM systems (laptops, desktops, NOT Raspberry Pi).

PanoplyJ

PanoplyJ is downloaded for Mac, Linux or Windows. PanoplyJ does not require compiling, just run it using Java. Linux PanoplyJ requires Java 8 or newer, and can also work with Java 9.

apt install openjdk-8-jre

Extract PanoplyJ and make executable:

unzip PanoplyJ-4*.zip -d ~

chmod +x ~/PanoplyJ/panoply.sh

Run PanoplyJ by:

~/PanoplyJ/panoply.sh

Make a command-line shortcut by adding to ~/.bash_aliases

alias panoply='$HOME/PanoplyJ/panoply.sh'

If you get PanoplyJ error

Unknown superblock version=3

You may have an older version of PanoplyJ whose NetCDF-Java library isn’t ready for HDF5 1.10. If you’re writing the HDF5 files from h5py, you can use in your Python program writing the data:

with h5py.File('myfile.h5',libver='earliest') as f:
   ....

which should write in an HDF5 1.8-compatible way.