Project 5: Fluorescence intensity and shape of phytoplankton cells

In this project the goal is to detect phytoplankton cells in gray-channel images and use the obtained coordinates to measure texture traits (intensities and statistical moments) in fluorescence channels of the same images.

Before

Input - Phytoplankton fluorescence. Each sample includes 1 brightfield gray scale image and 3 fluorescence images at different wavelengths.

After

Results - Phenopype thresholding function detects the contours of phytoplankton cells (green) and holes within (red).

Images kindly provided by Irene Gallego and Anita Narwani.

Background

A plate reader creates four images of the same set of objects: bright field gray scale images, and three different fluoresccence channels to represent different photopigments. All four images show the same objects, but with different pixel intensities. We will use the contour outline from the brightfield images, which are processed with the high-throughput workflow, as a stencil to extract texture information from the same coordinates in the fluorescence channels.

53dd9b84010a4fce8ebfc061232ec3b7

Fig. 1: Each sample includes four images: 1 brightfield (top) and 3 fluorescence measurements (black, bottom). Due to different pigments, not all spcecies are visible in each image, because of different emission spectra. For example, the two long string shaped cells marked with the green circles only occur in two of the fluorescence channels, but not the third one or the brightfield.

Preparation

[1]:
import phenopype as pp
import os

## change working dir - will contain project folder
os.chdir(r"D:\git-repos\phenopype\phenopype-gallery\_temp")

## set project name
project_name = "project_5"

## fetch template from downloaded template repo (https://github.com/phenopype/phenopype-templates)
template_path = r"D:\git-repos\phenopype\phenopype-templates\templates\gallery\project_5.yaml"

## set directory with images
image_dir = r"D:\git-repos\phenopype\phenopype-gallery\gallery\data"
phenopype successfully imported the following plugin dependencies:
phenomorph, keras

Project

Find objects in brightfield images

[2]:
proj = pp.Project(project_name)
--------------------------------------------
Creating a new phenopype project directory at:
D:\git-repos\phenopype\phenopype-gallery\_temp\project_5

Proceed? (y/n)
y

Project "project_5" successfully created.
--------------------------------------------
[3]:
## add all phytoplankton images from the data folder, but exclude fluorescence channels
proj.add_files(image_dir = image_dir, include="phyto", exclude="FL")
--------------------------------------------
phenopype will search for image files at

D:\git-repos\phenopype\phenopype-gallery\gallery\data

using the following settings:

filetypes: ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png', 'bmp'], include: phyto, exclude: FL, mode: copy, recursive: False, resize: False, unique: path

Found image phyto1_BF.tif - phenopype-project folder 0__phyto1_BF created
Found image phyto2_BF.tif - phenopype-project folder 0__phyto2_BF created
Found image phyto3_BF.tif - phenopype-project folder 0__phyto3_BF created

Found 3 files - using all
--------------------------------------------
[4]:
## add the config template; provide a tag
proj.add_config(template_path=template_path, tag="v1")
- template saved under D:\git-repos\phenopype\phenopype-gallery\_temp\project_5\data\0__phyto1_BF\pype_config_v1.yaml
- template saved under D:\git-repos\phenopype\phenopype-gallery\_temp\project_5\data\0__phyto2_BF\pype_config_v1.yaml
- template saved under D:\git-repos\phenopype\phenopype-gallery\_temp\project_5\data\0__phyto3_BF\pype_config_v1.yaml

d3cdda244e4e46e69586ac120aa93d1c

Fig. 2: Use the edit_contour function to remove unwanted objects, e.g. cells that are broken, too small, or other unwanted junk and detritus.

[5]:
## run image processing (`window_max_dim` controls the window size of all GUI functions in the Pype config)
for path in proj.dir_paths:
    pp.Pype(path, tag="v1", window_max_dim=1750)

AUTOLOAD
 - nothing to autoload
Stage: add annotation control args
Stage: add annotation control args
Stage: add annotation control args
Updating pype config: applying staged changes


------------+++ new pype iteration 2022-07-29 11:45:32 +++--------------




PREPROCESSING
blur


SEGMENTATION
threshold
- multichannel image supplied, converting to grayscale
- decompose image: using gray channel
morphology
morphology
detect_contour
- found 89 contours that match criteria
edit_contour
An exception has occurred, use %tb to see the full traceback.

SystemExit:

TERMINATE (by user)

C:\Miniconda3\envs\pp-dev\lib\site-packages\IPython\core\interactiveshell.py:3560: UserWarning: To exit: use 'exit', 'quit', or Ctrl-D.
  warn("To exit: use 'exit', 'quit', or Ctrl-D.", stacklevel=1)

Compute shape features of cells in brightfield images

The shape features say something about cell morphology, which is why we only one intact cells. Since the fluorescence channels may not tell us whether a cell is intact or not, we only compute those features in the objects detected in the brightfield images.

[6]:
## use `edit_config`´to inject `compute_shape_features` into the configuration files
## this makes the initial image processing faster, as this step is somehwat computationally intensive
target1 = """    - export:"""
replacement1 = """    - measurement:
        - compute_shape_features:
            features: ["basic","moments","hu_moments"]
    - export:"""
proj.edit_config(tag="v1", target=target1, replacement=replacement1)
# phenopype gallery project 3
# ---------------------------
# detect phytoplankton contours

config_info:
    config_name: pype_config_v1.yaml
    date_created: '2022-01-17 22:16:49'
    date_last_modified:
    template_name: gallery_project_5.yaml
    template_path: D:\workspace\git-repos\phenopype\phenopype-templates\templates\gallery\gallery_project_5.yaml
processing_steps:
    - preprocessing:
        - blur:
            kernel_size: 9
    - segmentation:
        - threshold:
            method: adaptive
            blocksize: 299
            constant: 10
        - morphology:
            operation: open
            shape: cross
            kernel_size: 5
            iterations: 1
        - morphology:
            operation: close
            shape: ellipse
            kernel_size: 3
            iterations: 3
        - detect_contour:
            ANNOTATION: {type: contour, id: a, edit: overwrite}
            min_area: 150
        - edit_contour:
            ANNOTATION: {type: drawing, id: a, edit: false}
            overlay_blend: 0.1
            overlay_line_width: 1
        - detect_contour:
            ANNOTATION: {type: contour, id: b, edit: overwrite}
            min_area: 10
    - visualization:
        - select_canvas:
            canvas: raw
        - draw_contour
    - measurement:
        - compute_shape_features:
            features: ["basic","moments","hu_moments"]
    - export:
        - save_canvas
        - save_annotation:
            overwrite: true

This is what the new config may look like (can differ beteeen files) - proceed?y
New config saved for 0__phyto1_BF
New config saved for 0__phyto2_BF
New config saved for 0__phyto3_BF
[7]:
## run pype again, but without visual feedback to speed things up
## run image processing
for path in proj.dir_paths:
    pp.Pype(path, tag="v1", feedback=False)
Format path to abspath
- no annotation_type selected - returning all annotations

AUTOLOAD
- annotations loaded:
{
"contour": ["a", "b"],
"drawing": ["a"]
}
updating pype config file


------------+++ new pype iteration 2022-01-17 22:18:56 +++--------------




PREPROCESSING
blur


SEGMENTATION
threshold
- multichannel image supplied, converting to grayscale
- decompose image: using gray channel
morphology
morphology
detect_contour
- loaded existing annotation of type "contour" with ID "a": overwriting (edit=overwrite)
- found 89 contours that match criteria
edit_contour
- loaded existing annotation of type "drawing" with ID "a": skipping (edit=False)
detect_contour
- loaded existing annotation of type "contour" with ID "b": overwriting (edit=overwrite)
- found 71 contours that match criteria


VISUALIZATION
select_canvas
- raw image
draw_contour


MEASUREMENT
compute_shape_features


EXPORT
save_canvas
- image saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\data\0__phyto1_BF\canvas_v1.jpg (overwritten).
save_annotation
- loading existing annotation file
- updating annotation of type "contour" with id "a" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "contour" with id "b" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "drawing" with id "a" in "annotations_v1.json" (overwrite="entry")
- writing annotation of type "shape_features" with id "a" to "annotations_v1.json"
updating pype config file


------------+++ finished pype iteration +++--------------
-------(End with Ctrl+Enter or re-run with Enter)--------




TERMINATE

AUTOSAVE
- nothing to autosave
Format path to abspath
- no annotation_type selected - returning all annotations

AUTOLOAD
- annotations loaded:
{
"contour": ["a", "b"],
"drawing": ["a"]
}
updating pype config file


------------+++ new pype iteration 2022-01-17 22:18:57 +++--------------




PREPROCESSING
blur


SEGMENTATION
threshold
- multichannel image supplied, converting to grayscale
- decompose image: using gray channel
morphology
morphology
detect_contour
- loaded existing annotation of type "contour" with ID "a": overwriting (edit=overwrite)
- found 99 contours that match criteria
edit_contour
- loaded existing annotation of type "drawing" with ID "a": skipping (edit=False)
detect_contour
- loaded existing annotation of type "contour" with ID "b": overwriting (edit=overwrite)
- found 85 contours that match criteria


VISUALIZATION
select_canvas
- raw image
draw_contour


MEASUREMENT
compute_shape_features


EXPORT
save_canvas
- image saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\data\0__phyto2_BF\canvas_v1.jpg (overwritten).
save_annotation
- loading existing annotation file
- updating annotation of type "contour" with id "a" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "contour" with id "b" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "drawing" with id "a" in "annotations_v1.json" (overwrite="entry")
- writing annotation of type "shape_features" with id "a" to "annotations_v1.json"
updating pype config file


------------+++ finished pype iteration +++--------------
-------(End with Ctrl+Enter or re-run with Enter)--------




TERMINATE

AUTOSAVE
- nothing to autosave
Format path to abspath
- no annotation_type selected - returning all annotations

AUTOLOAD
- annotations loaded:
{
"contour": ["a", "b"],
"drawing": ["a"]
}
updating pype config file


------------+++ new pype iteration 2022-01-17 22:18:58 +++--------------




PREPROCESSING
blur


SEGMENTATION
threshold
- multichannel image supplied, converting to grayscale
- decompose image: using gray channel
morphology
morphology
detect_contour
- loaded existing annotation of type "contour" with ID "a": overwriting (edit=overwrite)
- found 108 contours that match criteria
edit_contour
- loaded existing annotation of type "drawing" with ID "a": skipping (edit=False)
detect_contour
- loaded existing annotation of type "contour" with ID "b": overwriting (edit=overwrite)
- found 89 contours that match criteria


VISUALIZATION
select_canvas
- raw image
draw_contour


MEASUREMENT
compute_shape_features


EXPORT
save_canvas
- image saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\data\0__phyto3_BF\canvas_v1.jpg (overwritten).
save_annotation
- loading existing annotation file
- updating annotation of type "contour" with id "a" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "contour" with id "b" in "annotations_v1.json" (overwrite="entry")
- updating annotation of type "drawing" with id "a" in "annotations_v1.json" (overwrite="entry")
- writing annotation of type "shape_features" with id "a" to "annotations_v1.json"
updating pype config file


------------+++ finished pype iteration +++--------------
-------(End with Ctrl+Enter or re-run with Enter)--------




TERMINATE

AUTOSAVE
- nothing to autosave

Compute texture featues of cells in fluorescence images

This procedure uses the contour information we collected in the high-throughput workflow above. It provides all object coordinates to the compute_texture_features function, which, if also supplied with the fluorescence channel images, extrace texture featues from those coordinates. This code snippet shows that the low-throughput workflow, i.e., writing phenopype functions in pure Python code, can also have its use.

[8]:
for path in proj.dir_paths:

    ## the _load_yaml function is part of the private API, and used here to load the attributes file to get the image name
    attributes = pp.utils_lowlevel._load_yaml(os.path.join(path, "attributes.yaml"))
    image_stem = attributes["image_original"]["filename"].partition('_')[0]

    ## we load the annotations collection in the high throughput workflow above - we need the contour coordinates of each object
    annotations = pp.export.load_annotation(os.path.join(path, "annotations_v1.json"))

    ## we now loop through the files in the data folder, which are named like the brightfield image, and load those images
    for channel in ["FL1","FL2","FL3"]:
        image_fluorescence_path = os.path.join( r"../../gallery/data", image_stem + "_" + channel + ".tif")
        image_fluorescence = pp.load_image(image_fluorescence_path)

        ## using the fluorescence image and the contours, we can compute texture features for each object. this is somewhat computationally intensive
        annotations = pp.measurement.compute_texture_features(image_fluorescence, contour_id="b", annotations=annotations, annotation_id=channel)

    ## we store the textures back to the annotations file
    pp.export.save_annotation(annotations, dir_path = path, file_name="annotations_v1.json")
- no annotation_type selected - returning all annotations
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 71/71 [00:00<00:00, 143.72it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 71/71 [00:00<00:00, 138.51it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 71/71 [00:00<00:00, 142.57it/s]
- loading existing annotation file
- annotation of type "contour" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "contour" with id "b" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "drawing" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "shape_features" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- writing annotation of type "texture_features" with id "FL1" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL2" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL3" to "annotations_v1.json"
- no annotation_type selected - returning all annotations
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 85/85 [00:00<00:00, 142.17it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 85/85 [00:00<00:00, 139.51it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 85/85 [00:00<00:00, 140.73it/s]
- loading existing annotation file
- annotation of type "contour" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "contour" with id "b" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "drawing" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "shape_features" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- writing annotation of type "texture_features" with id "FL1" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL2" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL3" to "annotations_v1.json"
- no annotation_type selected - returning all annotations
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 89/89 [00:00<00:00, 139.09it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 89/89 [00:00<00:00, 141.95it/s]
- decompose image: using gray channel
Processing gray channel texture features: 100%|███████████████████████████████████████| 89/89 [00:00<00:00, 141.47it/s]
- loading existing annotation file
- annotation of type "contour" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "contour" with id "b" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "drawing" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- annotation of type "shape_features" with id "a" already exists in "annotations_v1.json" (overwrite=False)
- writing annotation of type "texture_features" with id "FL1" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL2" to "annotations_v1.json"
- writing annotation of type "texture_features" with id "FL3" to "annotations_v1.json"

[9]:
## this collects all annotations
proj.collect_results(tag="v1", files="annotations",  folder="annotations", overwrite=True)
Created D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\results\annotations
Search string: ['annotations_v1']
Collected annotations_v1.json from 0__phyto1_BF
0__phyto1_BF_annotations_v1.json saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\results\annotations\0__phyto1_BF_annotations_v1.json.
Collected annotations_v1.json from 0__phyto2_BF
0__phyto2_BF_annotations_v1.json saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\results\annotations\0__phyto2_BF_annotations_v1.json.
Collected annotations_v1.json from 0__phyto3_BF
0__phyto3_BF_annotations_v1.json saved under D:\workspace\git-repos\phenopype\phenopype-gallery\_temp\project_5\project\results\annotations\0__phyto3_BF_annotations_v1.json.
[10]:
## display results
import ipyplot ## install with `pip install ipyplot`

canvas_list = []
for path in proj.dir_paths:
    canvas_list.append(pp.load_image(os.path.join(path, "canvas_v1.jpg"), mode="rgb"))

ipyplot.plot_images(canvas_list, img_width=300)
[ ]: