Tutorial 1: A (very) brief python introduction

This tutorial is meant to provide a very short overview of the basic Python syntax needed for basic phenopype operability, which might be useful if you have never used Python before, but would like to be able to explore phenopype functionality on your own.

If you are familiar with Python, you may want to skip ahead to Tutorial 2: Interacting with images in phenopype.

See also:

• Python resources in the phenopype docs

• CV resources in the phenopype docs

Python modules

At the beginning of our script we import the python modules that we want to work with using import. When we import the module, we can call its methods and functions using what comes after import - for instance, the os module, which provides functions for interacting with the operating system.

import os

We can inspect all the components of this module (i.e., its “namespace”) by using dir(). We can see that os is a comprehensive package with a lots of classes and functions:

print(dir(os))

['DirEntry', 'F_OK', 'MutableMapping', 'O_APPEND', 'O_BINARY', 'O_CREAT', 'O_EXCL', 'O_NOINHERIT', 'O_RANDOM', 'O_RDONLY', 'O_RDWR', 'O_SEQUENTIAL', 'O_SHORT_LIVED', 'O_TEMPORARY', 'O_TEXT', 'O_TRUNC', 'O_WRONLY', 'P_DETACH', 'P_NOWAIT', 'P_NOWAITO', 'P_OVERLAY', 'P_WAIT', 'PathLike', 'R_OK', 'SEEK_CUR', 'SEEK_END', 'SEEK_SET', 'TMP_MAX', 'W_OK', 'X_OK', '_Environ', '__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_check_methods', '_execvpe', '_exists', '_exit', '_fspath', '_get_exports_list', '_putenv', '_unsetenv', '_wrap_close', 'abc', 'abort', 'access', 'altsep', 'chdir', 'chmod', 'close', 'closerange', 'cpu_count', 'curdir', 'defpath', 'device_encoding', 'devnull', 'dup', 'dup2', 'environ', 'error', 'execl', 'execle', 'execlp', 'execlpe', 'execv', 'execve', 'execvp', 'execvpe', 'extsep', 'fdopen', 'fsdecode', 'fsencode', 'fspath', 'fstat', 'fsync', 'ftruncate', 'get_exec_path', 'get_handle_inheritable', 'get_inheritable', 'get_terminal_size', 'getcwd', 'getcwdb', 'getenv', 'getlogin', 'getpid', 'getppid', 'isatty', 'kill', 'linesep', 'link', 'listdir', 'lseek', 'lstat', 'makedirs', 'mkdir', 'name', 'open', 'pardir', 'path', 'pathsep', 'pipe', 'popen', 'putenv', 'read', 'readlink', 'remove', 'removedirs', 'rename', 'renames', 'replace', 'rmdir', 'scandir', 'sep', 'set_handle_inheritable', 'set_inheritable', 'spawnl', 'spawnle', 'spawnv', 'spawnve', 'st', 'startfile', 'stat', 'stat_result', 'statvfs_result', 'strerror', 'supports_bytes_environ', 'supports_dir_fd', 'supports_effective_ids', 'supports_fd', 'supports_follow_symlinks', 'symlink', 'sys', 'system', 'terminal_size', 'times', 'times_result', 'truncate', 'umask', 'uname_result', 'unlink', 'urandom', 'utime', 'waitpid', 'walk', 'write']

The double underscores (“dunders”) before and after the numeric string indicate that these are “special” names reserved for the python namespace (https://dbader.org/blog/meaning-of-underscores-in-python). We cannot tell what type of object is behind each name, e.g., whether it’s a function (like open) or a submodule, with a set of functions (like path). Higher level functions are accessed from the package namespace by joining the module name os with the function open, connected by a dot.

If you are ever unsure about what a package does, simply call help(os) (or any other package) to get some basic information. This also works for its methods and functions, e.g. for the open method:

help(os.open)

Help on built-in function open in module nt:

open(path, flags, mode=511, *, dir_fd=None)
    Open a file for low level IO.  Returns a file descriptor (integer).

    If dir_fd is not None, it should be a file descriptor open to a directory,
      and path should be relative; path will then be relative to that directory.
    dir_fd may not be implemented on your platform.
      If it is unavailable, using it will raise a NotImplementedError.

path on the other hand is a submodule: here we see some functions we can use (they don’t have dunders), for example join, which can join separate character strings to a python-readable path string.

print(dir(os.path))

['__all__', '__builtins__', '__cached__', '__doc__', '__file__', '__loader__', '__name__', '__package__', '__spec__', '_abspath_fallback', '_get_bothseps', '_getfinalpathname', '_getfullpathname', '_getvolumepathname', 'abspath', 'altsep', 'basename', 'commonpath', 'commonprefix', 'curdir', 'defpath', 'devnull', 'dirname', 'exists', 'expanduser', 'expandvars', 'extsep', 'genericpath', 'getatime', 'getctime', 'getmtime', 'getsize', 'isabs', 'isdir', 'isfile', 'islink', 'ismount', 'join', 'lexists', 'normcase', 'normpath', 'os', 'pardir', 'pathsep', 'realpath', 'relpath', 'samefile', 'sameopenfile', 'samestat', 'sep', 'split', 'splitdrive', 'splitext', 'stat', 'supports_unicode_filenames', 'sys']

help(os.path.join)

Help on function join in module ntpath:

join(path, *paths)
    # Join two (or more) paths.

Paths and directories

Time to actually do something with a function. A useful function from the os module is getcwd, which will return the current working directory as a string (all paths in the os module are treated as strings):

os.getcwd()

'D:\\git-repos\\phenopype\\phenopype-tutorials\\tutorials'

Another frequently used function is listdir, which will list all files in a specified directory. If you don’t specify any directory, it will use the current working directory

os.listdir()

['.ipynb_checkpoints',
 'data',
 'tutorial_1.ipynb',
 'tutorial_2.ipynb',
 'tutorial_3.ipynb',
 'tutorial_4.ipynb',
 'tutorial_5.ipynb',
 'tutorial_6.ipynb',
 'tutorial_7.ipynb',
 '_figures']

os.listdir("data")

['isopods_fish.mp4',
 'stickle1.jpg',
 'stickle2.jpg',
 'stickle3.jpg',
 'stickleback_side.jpg',
 'stickleback_top.jpg']

Note that I used the relative path here - full paths of course are also possible.

listdir, as the name suggest, creates a list:

image_list = os.listdir("data")
print(image_list)

['isopods_fish.mp4', 'stickle1.jpg', 'stickle2.jpg', 'stickle3.jpg', 'stickleback_side.jpg', 'stickleback_top.jpg']

We now can access image_list with hard brackets and by giving the position inside you want to have returned:

IMPORTANT - read before continuing

In python, referencing starts with 0 and not with 1.

print(image_list[0])

isopods_fish.mp4

print(image_list[1])

stickle1.jpg

So far we have only returned name-strings of our files, but unless we will always stay in the current working directory, we need the full or relative path of our files. We can get it using the path submodule and the join function to paste the directory path and the names of files within it:

os.path.join("data", image_list[1]) # works

'data\\stickle1.jpg'

Although you can join path strings by simply adding them, this sometimes leads to unexpected results, so better try to avoid manipulating strings directly and instead use the join function:

"data" + image_list[1] # doesn't work

'datastickle1.jpg'

"data" "\\" + image_list[1] # works

'data\\stickle1.jpg'

Putting it all together, let’s try to get the path of all the images in our directory. For this, we need a for loop, and an empty list we can populate:

image_dir = "data" # save image directory to a string variable
filepaths = [] # square-brakets make an empty list
names = os.listdir(image_dir) # making a list of all the files names inside a directory

for i in names: # looping along our list of names
    filepath = os.path.join(image_dir, i) # joining name and path strings
    filepaths.append(filepath) # appending the joint string to the list

filepaths # showing the list content

['data\\isopods_fish.mp4',
 'data\\stickle1.jpg',
 'data\\stickle2.jpg',
 'data\\stickle3.jpg',
 'data\\stickleback_side.jpg',
 'data\\stickleback_top.jpg']

Images in OpenCV

Let’s import another module. In fact the module that phenopype is built around: OpenCV. Note that mopdule bindings in python can be called differently than the package name - e.g., for opencv, we have to call cv2. Once imported, we then can use a basic opencv function: importing an image as an array using imread.

import cv2
img = cv2.imread(r"data/stickle1.jpg")

type(img) ## check loading the image worked: should return numpy.ndarray
## "imread" will silently fail if the path is incorrect, in which case "img" will be "NoneType"

numpy.ndarray

Lets examine our image: cv2.imread converts a digital image file to an array, a stack of three matrices containing pixel wise information on the red, green and blue channel of an image, which - taken together - creates the colour image. It is neither informative nor possible to look at the whole pixel array inside the console, but sometimes it is useful to look at specific features of your image, which we can acess directly from the image-object. For example, the dimensions, using shape:

img.shape

(1600, 2400, 3)

Or the mean pixel intensity:

img.mean()

220.84537473958332

We can look at the picture with imshow. However, because many opencv functions are GUI based, we need to add some more controller functions to open and close a window.

cv2.namedWindow('image', cv2.WINDOW_NORMAL) # open a resizable window
cv2.imshow('image',img) # show the image in that window
cv2.waitKey(0) # do nothing until a keystroke ...
cv2.destroyAllWindows() # ... and then close all open windows

Four lines of code to display an image - this will clutter your code really fast. Luckily, phenopype has some functions of its own to load and display images. Continue with Tutorial 2 to find out how phenopype works with images.