Tutorial 4: The Pype class

The previous tutorial showed how the Pype class operates in comparison to just calling the functions in pure Python. You can make the most of the pype function when using configuration files that are customized to your specific needs or workflow.

A selection of templates for configuration files to be used by the Pype class can be found in the template section of the docs. They can be freely modified, but need to adhere YAML specifications (see below). Also check the phenopype gallery for inspiration and additional templates.

See also:

Modifying configurations

The text inside the configuration files is parsed by phenopype from top to bottom and converted back to Python code in the background, i.e. to phenopype modules and functions. Indentation hierarchy is as follows:

  1. The first level without any indentation, e.g. - preprocessing or - segmentation, denote from the module that a function is part of.

  2. The second level with two-space indentation before the hyphen, e.g. - threshold or - detect_contours are functions that are loaded from the segmentation module.

  3. The third level without hyphens, e.g. method: otsu and blocksize: 99, are arguments passed on to the function.

When running the pype routine, image is automatically loaded and passed to all following functions. You can add or remove functions as you like. Note in the hyphenated first two levels you can specify modules and functions as many times as you want (- is the yaml list notation). When adding or modifying modules and functions, it is important to keep in mind that the function stack is executed sequentially. So, if you want to perform a morphology operation on a binary images, it should come after and not before the main segmentation function (in this case threshold).

Following this notation, the yaml parser in Python interprets


    - threshold:
        method: adaptive
        blocksize: 99
        constant: 5
        channel: red

as


    pp.segmentation.threshold(image, method="adaptive", blocksize = 99, constant=5, channel="red")

Annotation control

In phenopype, functons that generate annotations to images, have an annotation control sequence (ANNOTATION) that control the behavior of the function when the Pype is parsed - for example:


  - create_mask:
      ANNOTATION: {type: mask, id: a, edit: false}

type specifies which type of annotation is created and, together with id (“a-z”), creates a universal identifier for a given configuration. edit controls the overwrite behavior: false will not overwrite an existing annotation of the same Type and ID when the Pype is run again, true will “edit” the annotation, meaning that the previously created masks, landmarks or polygons can be edited or removed. edit: overwrite will simply overwrite the entire annotation. Note that it will be overwritten every time a Pype iteration is completed, until removed.

YAML syntax

The configuration files needed to run the pype are written in YAML (a recursive acronym for “YAML Ain’t Markup Language”). In principle, these are just text files that follows a specific set of rules for indentation and separation.

YAML syntax

Here are the most important rules for YAML syntax (in phenopype and in general):

  • indentation rules:

    • 0 spaces + hyphen + space for modules

    • 4 spaces + hyphen + space in front of functions

    • 8 spaces in front of arguments

  • separation rules:

    • modules and functions with arguments are followed by a colon (:) and a new line

    • functions without specified arguments don’t need a colon

    • arguments are followed by a colon, a space and then the value

  • modules and functions can be emtpy (see- draw_mask above), but function arguments cannot be emtpy (e.g. overwrite: needs to be true or false)

  • as per Python syntax, optional function arguments can, but don’t have to be specified and the functions will just run on default values

  • functions can be added multiple times, but sometimes their output may be overwtritten (e.g. - threshold makes sense only once, but - blur may be used in multiple locations)

Pype operation

These are the most important things to keep in mind during a Pype iteration

Enhanced Window control

In addition to regular GUI window control functions documented in Tutorial 2:

  • Editing and saving the opened configuration file in the text editor will trigger another iteration, i.e. close the image window and run the config file again.

  • Closing the image window manually (with the X button in the upper right), also runs triggers another run.

  • Esc will close all windows and interrupt the pype routine (triggers sys.exit(), which will also end a Python session if run from the command line), as well as any loops.

  • Each step that requires user interaction (e.g. create_mask or landmarks) needs to be confirmed with Enter until the next function in the sequence is executed.

  • At the end of the analysis, when the final steps (visualization and export functions) have run, use Ctrl+Enter to finish and close the window.

Function execution

  • Pype will automatically load the image and execute all functions in sequence, but it will not overwrite overwrite data from past iterations on disk unless specified.

  • To overwrite interactive user input, set the argument edit: true or edit: overwrite in the function’s annotation control arguments.

  • If you forget to remove an overwrite argument and are prompted to overwrite previous input, immediately change to edit: false argument, and save the config file.

  • If a Pype is initialized on a project directory it will attempt to load input data (e.g. masks) that contain the provided tag argument. For example,pp.Pype(path, tag="v1" will attempt to load any files in the directory that contain the suffix "v1" (e.g. "annoations_v1.json").

Visualizing the results

Aspects of visual feedback during a pype run (can be completely suppressed by setting visualize=False:

  • Visual feedback (i.e. output from landmarks, detect_contours or create_mask) are drawn onto a “canvas” (a copy of the original image).

  • Use select_canvas to draw the results either on the raw image, a binary image, or a single colour channel (gray, red, green or blue).

  • If select_canvas is not explicitly specified, it is called automatically and defaults to the raw image as canvas.

  • Output from all functions, needs to be specified manually. For example, after using - landmarks, - draw_landmarks should be called in the visualization module.

  • Visual parameters of interactive tools (e.g. point_size or line_thickness) are specified separately in the respective function, and in the visualization module.

Exporting the results

Saving annotations, canvas and other results:

  • All results are saved automatically, even if the respective functions in export are not specified, with the tag argument in Pype as suffix.

  • If a file already exist in the directory, and the respective function is not listed under export, then it will not be overwritten.

  • If an export function is specified under export:, it will also not overwrite any existing file, unless specified using overwrite: true.

  • The canvas is an exception: it will always be saved and always be overwritten (unless specified with overwrite: False to show the output from the last iteration. However, users can modify the canvas name with file_name in the arguments to save different output side by side. For example, file_name: binary under - save_canvas: save the canvas as canvas_binary.jpg

To learn how to analyze entire datasets by making a project, move on to Tutorial 5.