\n",
"
\n",
"
Hint:
\n",
"
\n",
"Anyone with access to the raw images and the configuration files is able to reproduce the results with a few lines of code, making phenopype project directories the perfect format for code sharing and data sharing after publication through online repositories (such as Dryad, Zenodo or OSF).\n",
" \n",
"
\n",
"
\n",
"\n",
"\n",
"
\n",
"\n",
"
![](\"_figures/luerig_2021_figure_3A.jpg\")
\n",
"
\n",
" \n",
"**Fig 1** Create a phenopype project and organize raw images into separate folders where all relevant data, attributes and results are stored. (Figure from [Lürig 2021 [Fig. 3A]](https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13771)).\n",
" \n",
"
\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"We will use the images from the tutorials folder ([download the tutorials](https://github.com/phenopype/phenopype-tutorials/archive/refs/heads/main.zip) if you have not done so already)."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------\n",
"Found existing project root directory - loading from:\n",
"C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\n",
"\n",
"Project \"tutorial_project\" successfully loaded with 3 images\n",
"--------------------------------------------\n"
]
}
],
"source": [
"import phenopype as pp\n",
"import os\n",
"\n",
"os.chdir(r\"C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\")\n",
"\n",
"myproj = pp.Project(\n",
" root_dir=r\"tutorial_project\", ## pick your project directory\n",
" ## overwrite=True, ## overwrite=True will overwrite the project (after another prompt) \n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next step is to add images to the project. You can do so with the `add_files` method of the created project (a *method* is an executable function that belongs to an existing *object*, in this case \"myproj). The function offers some flexibility in terms of which files to import. Most important arguments here are `include`, `exclude` and `filetypes`. For example, given the following list of images:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"['isopods_fish.mp4',\n",
" 'stickle1.jpg',\n",
" 'stickle2.jpg',\n",
" 'stickle3.jpg',\n",
" 'stickleback_side.jpg',\n",
" 'stickleback_top.jpg']"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"os.listdir(r\"tutorials/data\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If we want to import \"stickle1\", \"stickle2\", and \"stickle3\", we can do a combination `include` and `exclude` (also prints all other default settings):"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------\n",
"phenopype will search for image files at\n",
"\n",
"C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorials\\data\n",
"\n",
"using the following settings:\n",
"\n",
"filetypes: ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png', 'bmp'], include: stickle, exclude: ['side', 'top'], mode: copy, recursive: False, resize: False, unique: path\n",
"\n",
"Found image stickle1.jpg - 0__stickle1 already exists (overwrite=False)\n",
"Found image stickle2.jpg - 0__stickle2 already exists (overwrite=False)\n",
"Found image stickle3.jpg - 0__stickle3 already exists (overwrite=False)\n",
"\n",
"Found 3 files\n",
"--------------------------------------------\n"
]
}
],
"source": [
"image_dir = r\"tutorials/data\"\n",
"\n",
"myproj.add_files(\n",
" image_dir=image_dir,\n",
" include=\"stickle\", ## can be type \"str\" or type \"list\"\n",
" exclude=[\"side\",\"top\"] ## can be type \"str\" or type \"list\"\n",
") "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Thre are more settings to `add_files`: \n",
"- `mode` determines whether raw files should be copied to each folder in the Phenopype directory tree (using `copy` [default]), or just their filepath (using `link`), which can be useful if data sets contain many or very large images. A third option is `mod`, which will open the iages and save them again in TIF format. This mode also allows to resize images.\n",
"- `recursive` indicates whether only the top directory (`False`; default), or also all subdirectories (`True`) should be included in the search. \n",
"- `unique` indicates whether files should be unique by their path (`filepath` [default]) or only by their name (`filename`) - duplicate files will be skipped. "
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Help on function add_files in module phenopype.main:\n",
"\n",
"add_files(self, image_dir, filetypes=['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png', 'bmp'], include=[], include_all=True, exclude=[], mode='copy', ext='tif', recursive=False, overwrite=False, resize_factor=1, unique='path', **kwargs)\n",
" Add files to your project from a directory, can look recursively. \n",
" Specify in- or exclude arguments, filetypes, duplicate-action and copy \n",
" or link raw files to save memory on the harddrive. For each found image,\n",
" a folder will be created in the \"data\" folder within the projects root\n",
" directory. If found images are in subfolders and \"recursive==True\", \n",
" the respective phenopype directories will be created with \n",
" flattened path as prefix. \n",
" \n",
" E.g., with \"raw_files\" as folder with the original image files \n",
" and \"phenopype_proj\" as rootfolder:\n",
" \n",
" - raw_files/file.jpg ==> phenopype_proj/data/file.jpg\n",
" - raw_files/subdir1/file.jpg ==> phenopype_proj/data/1__subdir1__file.jpg\n",
" - raw_files/subdir1/subdir2/file.jpg ==> phenopype_proj/data/2__subdir1__subdir2__file.jpg\n",
" \n",
" Parameters\n",
" ----------\n",
" image_dir: str \n",
" path to directory with images\n",
" filetypes: list or str, optional\n",
" single or multiple string patterns to target files with certain endings.\n",
" \"settings.default_filetypes\" are configured in settings.py: \n",
" ['jpg', 'JPG', 'jpeg', 'JPEG', 'tif', 'png', 'bmp']\n",
" include: list or str, optional\n",
" single or multiple string patterns to target certain files to include\n",
" include_all (optional): bool,\n",
" either all (True) or any (False) of the provided keywords have to match\n",
" exclude: list or str, optional\n",
" single or multiple string patterns to target certain files to exclude - \n",
" can overrule \"include\"\n",
" recursive: (optional): bool,\n",
" \"False\" searches only current directory for valid files; \"True\" walks \n",
" through all subdirectories\n",
" unique: {\"file_path\", \"filename\"}, str, optional:\n",
" how to deal with image duplicates - \"file_path\" is useful if identically \n",
" named files exist in different subfolders (folder structure will be \n",
" collapsed and goes into the filename), whereas filename will ignore \n",
" all similar named files after their first occurrence.\n",
" mode: {\"copy\", \"mod\", \"link\"} str, optional\n",
" how should the raw files be passed on to the phenopype directory tree: \n",
" \"copy\" will make a copy of the original file, \"mod\" will store a \n",
" .tif version of the orginal image that can be resized, and \"link\" \n",
" will only store the link to the original file location to attributes, \n",
" but not copy the actual file (useful for big files, but the orginal \n",
" location needs always to be available)\n",
" overwrite: {\"file\", \"dir\", False} str/bool (optional)\n",
" \"file\" will overwrite the image file and modify the attributes accordingly, \n",
" \"dir\" will overwrite the entire image directory (including all meta-data\n",
" and results!), False will not overwrite anything\n",
" ext: {\".tif\", \".bmp\", \".jpg\", \".png\"}, str, optional\n",
" file extension for \"mod\" mode\n",
" resize_factor: float, optional\n",
" \n",
" kwargs: \n",
" developer options\n",
"\n"
]
}
],
"source": [
"help(pp.Project.add_files)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### B) Adding `Pype` configuration files \n",
"\n",
"In the next step we prepare the files we added for use with the `Pype`class by addding a configuration file with the `add_config` method. Instead of adding the functions one by one we can load presets that are appropriate for the given computer vision analysis.\n",
"\n",
"A selection of templates for configuration files to be used by the `Pype` class can be found in the [template section of the docs](https://www.phenopype.org/docs/templates/). They can be freely modified, but need to adhere YAML specifications (see below). Also check the [phenopype gallery](https://www.phenopype.org/gallery/) for inspiration and additional templates.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"
\n",
"\n",
"
![](\"_figures/luerig_2021_figure_3B.jpg\")
\n",
"
\n",
" \n",
"**Fig 2** Create configuration files and store them alongside the raw images. (Figure from [Lürig 2021 [Fig. 3B]](https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13771)).\n",
" \n",
"
\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"We will use the same template (`quickstart-template.yaml`) as in Tutorial 3: "
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle1\\pype_config_plates-v1.yaml (overwritten)\n",
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle2\\pype_config_plates-v1.yaml (overwritten)\n",
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle3\\pype_config_plates-v1.yaml (overwritten)\n"
]
}
],
"source": [
"template = r\"C:\\Users\\mluerig\\Downloads\\phenopype-quickstart-main\\quickstart-template.yaml\"\n",
"myproj.add_config(\n",
" template_path=template, \n",
" tag=\"plates-v1\",\n",
" overwrite=True, # overwrite if present in image subdirectory\n",
" # interactive=True, # modify the template before distributing it\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now all images folders contain a configuration file.\n",
"\n",
"Note that we could have modified the first template before distributing it to the image subdirectories - either manually, or with `add_config` by setting `interactive=True`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### C) Batch processing \n",
"\n",
"After adding images and configuration, all is set to process your dataset with high throughput. Using a simple `for` loop, we then go through all directories one by one. The `skip` argument will allow to skip files with a given config name you have already analyzed. This allows you to return to the point where you left off. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"
\n",
"\n",
"
![](\"_figures/luerig_2021_figure_3C.jpg\")
\n",
"
\n",
" \n",
"**Fig 3** Apply the Pype class to each folder using a for-loop. Once the Pype class has finished executing all functions from the configuration file, users can decide to either modify the opened configuration file (e.g. either change function parameters or add new functions), which will trigger to run the Pype class again, or to close the GUI window, which will terminate the Pype class instance and save all results to the folder (Figure from [Lürig 2021 [Fig. 3C]](https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13771)).\n",
" \n",
"
\n",
"\n",
"\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"],\n",
"\"reference\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:04:37 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"- excluding pixels from reference\n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 16 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle1\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"reference\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:04:46 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 9 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle2\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:04:51 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 10 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle3\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n"
]
}
],
"source": [
"for path in myproj.dir_paths:\n",
" pp.Pype(path, \n",
" tag=\"plates-v1\", ## loads the config file \"pype_config_lm.yaml\". \"lm\" gets appended to all results files\n",
" skip=False ## skip=True will skip over any directories that already contain results files with \"lm\"\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"
\n",
"\n",
"
\n",
"\n",
"
![](\"_figures/luerig_2021_figure_3D.jpg\")
\n",
"
\n",
" \n",
"**Fig 3** Each folder contains all information necessary to reproduce the collected phenopytic data. Ouput from different `pype` runs can be stored side by side in the same folders (Figure from [Lürig 2021 [Fig. 3D]](https://besjournals.onlinelibrary.wiley.com/doi/10.1111/2041-210X.13771)).\n",
" \n",
"
\n",
"\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### D) Collecting results\n",
"\n",
"After finishing all analyses, you can use the `project`-method `collect_results` to copy all results (e.g. the annotation json files or canvas images) to a \"results\" folder in the root directory."
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Search string: ['annotations_plates-v1']\n",
"Collected annotations_plates-v1.json from 0__stickle1\n",
"0__stickle1_annotations_plates-v1.json saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\results\\annotations-v1\\0__stickle1_annotations_plates-v1.json (overwritten).\n",
"Collected annotations_plates-v1.json from 0__stickle2\n",
"0__stickle2_annotations_plates-v1.json saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\results\\annotations-v1\\0__stickle2_annotations_plates-v1.json (overwritten).\n",
"Collected annotations_plates-v1.json from 0__stickle3\n",
"0__stickle3_annotations_plates-v1.json saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\results\\annotations-v1\\0__stickle3_annotations_plates-v1.json (overwritten).\n"
]
}
],
"source": [
"myproj.collect_results(tag=\"plates-v1\", \n",
" files=\"annotations\", # \n",
" folder=\"annotations-v1\",\n",
" overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Saving and loading a project"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The `Project` class does not require a specific save argument, as results, annotations, etc. are saved in the image-specific subfolders in the data-directory. \n",
"\n",
"To load a `Project`, simply run it on the root path of your project: "
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"--------------------------------------------\n",
"Found existing project root directory - loading from:\n",
"C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\n",
"\n",
"Project \"tutorial_project\" successfully loaded with 3 images\n",
"--------------------------------------------\n"
]
},
{
"data": {
"text/plain": [
"['C:\\\\Users\\\\mluerig\\\\Downloads\\\\phenopype-tutorials-main\\\\tutorial_project\\\\data\\\\0__stickle1',\n",
" 'C:\\\\Users\\\\mluerig\\\\Downloads\\\\phenopype-tutorials-main\\\\tutorial_project\\\\data\\\\0__stickle2',\n",
" 'C:\\\\Users\\\\mluerig\\\\Downloads\\\\phenopype-tutorials-main\\\\tutorial_project\\\\data\\\\0__stickle3']"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import phenopype as pp\n",
"\n",
"myproj = pp.Project(\"tutorial_project\")\n",
"myproj.dir_paths"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Adding another configuration\n",
"\n",
"As mentioned above, it's possible to have multiple configuration files side by side in phenopype folders. First, [download the template repo](https://github.com/phenopype/phenopype-templates/archive/refs/heads/main.zip), and unzip it. Now, for example, if we also want to set some landmarks, we can supply this template:"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle1\\pype_config_lm-v1.yaml (overwritten)\n",
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle2\\pype_config_lm-v1.yaml (overwritten)\n",
"- template saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle3\\pype_config_lm-v1.yaml (overwritten)\n"
]
}
],
"source": [
"template = r\"C:\\Users\\mluerig\\Downloads\\phenopype-templates-main\\templates\\landmarks\\landmarks_plain.yaml\"\n",
"\n",
"myproj.add_config(\n",
" tag=\"lm-v1\",\n",
" template_path=template,\n",
" overwrite=True,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now simply run another loop:"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"landmark\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:05 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"MEASUREMENT\n",
"set_landmark\n",
"- loaded existing annotation of type \"landmark\" with ID \"a\": skipping (edit=False)\n",
"\n",
"\n",
"VISUALIZATION\n",
"draw_landmark\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle1\\canvas_lm-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"landmark\" with id \"a\" in \"annotations_lm-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"landmark\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:08 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"MEASUREMENT\n",
"set_landmark\n",
"- loaded existing annotation of type \"landmark\" with ID \"a\": skipping (edit=False)\n",
"\n",
"\n",
"VISUALIZATION\n",
"draw_landmark\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle2\\canvas_lm-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"landmark\" with id \"a\" in \"annotations_lm-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"landmark\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"Stage: add annotation control args\n",
"Updating pype config: applying staged changes\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:10 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"MEASUREMENT\n",
"set_landmark\n",
"- loaded existing annotation of type \"landmark\" with ID \"a\": skipping (edit=False)\n",
"\n",
"\n",
"VISUALIZATION\n",
"draw_landmark\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle3\\canvas_lm-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"landmark\" with id \"a\" in \"annotations_lm-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"AUTOSHOW\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n"
]
}
],
"source": [
"for path in myproj.dir_paths:\n",
" pp.Pype(\n",
" path, \n",
" tag=\"lm-v1\",\n",
" )"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that we added a second template with landmarking steps for the purpose of demonstrating how to have multiple configurations side by side in the image folders, which might also make sense if you want to separate the two work steps. However, it is possible and potentially also more time efficient, to have both, thep plate segmentation step *and* the landmarking step in one configuration file. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Modify configurations in all subdirectories"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In case you would like to modify your configuration after it has already been distributed, use the `edit_config` class method. This is a simple string-replacement function that allows you do remove or add elements to all existing configuration files. \n",
"\n",
"For instance, if we wanted to add the landmarking steps to our `plates-v1` configuration, we can pecifiy replacement and target strings - they have to **exactly** match the patterns in the config files, including spaces and indentations. We will modify the template in two places: the measurement, and the visualization step."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"# phenopype quickstart template\n",
"# -----------------------------\n",
"# This template is intended to go with the phenopype quickstart materials \n",
"# - for details see https://www.phenopype.org/docs/quickstart/ and refer to\n",
"# Figure 2 in Luerig 2021 (https://doi.org/10.1111/2041-210X.13771) or \n",
"# phenopype tutorial 3 (https://www.phenopype.org/docs/tutorials/tutorial_3).\n",
"# For a better job of measuring individual plate area see the phenopype \n",
"# gallery (https://www.phenopype.org/gallery/example_5/).\n",
"\n",
"config_info:\n",
" config_name: pype_config_plates-v1.yaml\n",
" date_created: '2022-05-06 15:04:33'\n",
" date_last_modified:\n",
" template_name: quickstart-template.yaml\n",
" template_path: C:\\Users\\mluerig\\Downloads\\phenopype-quickstart-main\\quickstart-template.yaml\n",
"processing_steps:\n",
" - preprocessing:\n",
" - create_mask:\n",
" ANNOTATION: {type: mask, id: a, edit: false}\n",
" tool: polygon\n",
" label: plates\n",
" - blur:\n",
" kernel_size: 9\n",
" - segmentation:\n",
" - threshold:\n",
" method: adaptive\n",
" blocksize: 99\n",
" constant: 5\n",
" - detect_contour:\n",
" ANNOTATION: {type: contour, id: a, edit: overwrite}\n",
" retrieval: ext\n",
" min_area: 150\n",
" - measurement:\n",
" - compute_shape_features:\n",
" ANNOTATION: {type: shape_features, id: a, edit: overwrite}\n",
" - visualization:\n",
" - select_canvas:\n",
" canvas: raw\n",
" - draw_contour\n",
" - draw_mask:\n",
" label: true\n",
" - export:\n",
" - save_canvas\n",
" - save_annotation:\n",
" overwrite: true\n",
"\n",
"This is what the new config may look like (can differ beteeen files) - proceed?y\n",
"New config saved for 0__stickle1\n",
"New config saved for 0__stickle2\n",
"New config saved for 0__stickle3\n"
]
}
],
"source": [
"## \"measurement\" modification to add `set_landmarks`:\n",
"\n",
"target1 = \"\"\" - measurement:\n",
" - compute_shape_features:\n",
" ANNOTATION: {type: morphology, id: a, edit: overwrite}\"\"\"\n",
"replacement1 = \"\"\" - measurement:\n",
" - compute_shape_features:\n",
" ANNOTATION: {type: morphology, id: a, edit: overwrite}\n",
" - set_landmark:\n",
" point_size: 8\n",
" point_colour: lawngreen\n",
" label_size: 2\n",
" label_width: 2\"\"\"\n",
" \n",
"myproj.edit_config(\n",
" tag=\"plates-v1\",\n",
" target=target1,\n",
" replacement=replacement1,\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now run the loop again - note that since we don't have to do any new manual annotations (we already set the polygon mask), we can turn off the visual feedback (`feedback=False`) to let the whole project run from beginning to end without interrupting:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"],\n",
"\"reference\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:19 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"- excluding pixels from reference\n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 16 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle1\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"reference\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:19 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 9 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle2\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n",
"- no annotation_type selected - returning all annotations\n",
"\n",
"AUTOLOAD\n",
"- annotations loaded:\n",
"{\n",
"\"mask\": [\"a\"],\n",
"\"contour\": [\"a\"],\n",
"\"shape_features\": [\"a\"]\n",
"}\n",
"- reference template image loaded from root directory\n",
"\n",
"\n",
"------------+++ new pype iteration 2022-05-06 15:05:20 +++--------------\n",
"\n",
"\n",
"\n",
"\n",
"PREPROCESSING\n",
"create_mask\n",
"- loaded existing annotation of type \"mask\" with ID \"a\": skipping (edit=False)\n",
"blur\n",
"\n",
"\n",
"SEGMENTATION\n",
"threshold\n",
"- multichannel image supplied, converting to grayscale\n",
"- decompose image: using gray channel\n",
"- including pixels from 1 drawn masks \n",
"detect_contour\n",
"- loaded existing annotation of type \"contour\" with ID \"a\": overwriting (edit=overwrite)\n",
"- found 10 contours that match criteria\n",
"\n",
"\n",
"MEASUREMENT\n",
"compute_shape_features\n",
"- loaded existing annotation of type \"shape_features\" with ID \"a\": overwriting (edit=overwrite)\n",
"\n",
"\n",
"VISUALIZATION\n",
"select_canvas\n",
"- raw image\n",
"draw_contour\n",
"draw_mask\n",
"\n",
"\n",
"EXPORT\n",
"save_canvas\n",
"- image saved under C:\\Users\\mluerig\\Downloads\\phenopype-tutorials-main\\tutorial_project\\data\\0__stickle3\\canvas_plates-v1.jpg (overwritten).\n",
"save_annotation\n",
"- loading existing annotation file\n",
"- no annotation_type selected - exporting all annotations\n",
"- updating annotations of type \"mask\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"contour\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"- updating annotations of type \"shape_features\" with id \"a\" in \"annotations_plates-v1.json\" (overwrite=\"entry\")\n",
"\n",
"\n",
"------------+++ finished pype iteration +++--------------\n",
"-------(End with Ctrl+Enter or re-run with Enter)--------\n",
"\n",
"\n",
"\n",
"\n",
"TERMINATE\n",
"\n",
"AUTOSAVE\n",
"- nothing to autosave\n"
]
}
],
"source": [
"for path in myproj.dir_paths:\n",
" pp.Pype(path, tag=\"plates-v1\", feedback=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Move on to [Tutorial 6](tutorial_6.ipynb) to learn how to set and detect a project-wide size reference."
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.7.13"
}
},
"nbformat": 4,
"nbformat_minor": 2
}