Classification: inference

In this notebook we demonstrate how to use the script scripts/classification/main_classification_inference.py to identify organisms in clips to taxonomic groups.

We first import the necessary packages, and specify the path where our main scripts are found:

import os, sys
from pathlib import Path
import yaml

from mzbsuite.utils import cfg_to_arguments

The most important function in this task is inference_classifier, and we specifically import it here for ease:

from scripts.classification.main_classification_inference import main as inference_classifier
?inference_classifier

Signature: inference_classifier(args, cfg)
Docstring:
Function to run inference on macrozoobenthos images clips, using a trained model.

Parameters
----------
args : argparse.Namespace
    Namespace containing the arguments passed to the script. Notably:

        - input_dir: path to the directory containing the images to be classified
        - taxonomy_file: path to the csv file containing the taxonomy of the classes
        - input_model: path to the directory containing the model to be used for inference
        - output_dir: path to the directory where the results will be saved
        - config_file: path to the config file with train / inference parameters

cfg : dict
    Dictionary containing the configuration parameters.

Returns
-------
None. Saves the results in the specified folder.
File:      d:\mzb-workflow\scripts\classification\main_classification_inference.py
Type:      function

Next we declare the running parameters, telling the script where to find the input data, the model and where to put the outputs. In this notebook, we pass these as a Python dictionary, but a shell (.sh) and Windows batch (.bat) files achieving the same result can be found in workflows.

ROOT_DIR = Path("D:\mzb-workflow") #Path("/data/shared/mzb-workflow")
MODEL = "efficientnet-b2-v0"

arguments = {
    "input_dir": ROOT_DIR / "data/mzb_example_data/training_dataset/val_set",
    "input_model": ROOT_DIR / f"models/mzb-classification-models/{MODEL}",
    "output_dir": ROOT_DIR / "results/mzb_example_data/classification",
    "config_file": ROOT_DIR / "configs/mzb_example_config.yaml",
    "taxonomy_file": ROOT_DIR / "data/mzb_example_data/MZB_taxonomy.csv"
}

with open(str(arguments["config_file"]), "r") as f:
    cfg = yaml.load(f, Loader=yaml.FullLoader)

cfg["trcl_gpu_ids"] = None
cfg

{'glob_random_seed': 222,
 'glob_root_folder': '/home/jovyan/work/mzb-workflow/',
 'glob_blobs_folder': '/home/jovyan/work/mzb-workflow/data/derived/blobs/',
 'glob_local_format': 'pdf',
 'model_logger': 'wandb',
 'impa_image_format': 'jpg',
 'impa_clip_areas': [2700, 4700, -1, -1],
 'impa_area_threshold': 5000,
 'impa_gaussian_blur': [21, 21],
 'impa_gaussian_blur_passes': 3,
 'impa_adaptive_threshold_block_size': 351,
 'impa_mask_postprocess_kernel': [11, 11],
 'impa_mask_postprocess_passes': 5,
 'impa_bounding_box_buffer': 200,
 'impa_save_clips_plus_features': True,
 'lset_class_cut': 'order',
 'lset_val_size': 0.1,
 'trcl_learning_rate': 0.0001,
 'trcl_batch_size': 8,
 'trcl_weight_decay': 0,
 'trcl_step_size_decay': 5,
 'trcl_number_epochs': 75,
 'trcl_save_topk': 1,
 'trcl_num_classes': 8,
 'trcl_model_pretrarch': 'convnext-small',
 'trcl_num_workers': 16,
 'trcl_wandb_project_name': 'mzb-classifiers',
 'trcl_logger': 'wandb',
 'trsk_learning_rate': 0.001,
 'trsk_batch_size': 32,
 'trsk_weight_decay': 0,
 'trsk_step_size_decay': 25,
 'trsk_number_epochs': 400,
 'trsk_save_topk': 1,
 'trsk_num_classes': 2,
 'trsk_model_pretrarch': 'mit_b2',
 'trsk_num_workers': 16,
 'trsk_wandb_project_name': 'mzb-skeletons',
 'trsk_logger': 'wandb',
 'infe_model_ckpt': 'last',
 'infe_num_classes': 8,
 'infe_image_glob': '*_rgb.jpg',
 'skel_class_exclude': 'errors',
 'skel_conv_rate': 131.6625,
 'skel_label_thickness': 3,
 'skel_label_buffer_on_preds': 25,
 'skel_label_clip_with_mask': False,
 'trcl_gpu_ids': None}

We now need to transform parameters in the configuration file in a format that Python can understand (a dictionary in this case), we can use the provided helper function cfg_to_arguments:

# Transforms configurations dicts to argparse arguments
args_p = cfg_to_arguments(arguments)
cfg_p = cfg_to_arguments(cfg)
print(str(cfg_p))

{'glob_random_seed': 222, 'glob_root_folder': '/home/jovyan/work/mzb-workflow/', 'glob_blobs_folder': '/home/jovyan/work/mzb-workflow/data/derived/blobs/', 'glob_local_format': 'pdf', 'model_logger': 'wandb', 'impa_image_format': 'jpg', 'impa_clip_areas': [2700, 4700, -1, -1], 'impa_area_threshold': 5000, 'impa_gaussian_blur': [21, 21], 'impa_gaussian_blur_passes': 3, 'impa_adaptive_threshold_block_size': 351, 'impa_mask_postprocess_kernel': [11, 11], 'impa_mask_postprocess_passes': 5, 'impa_bounding_box_buffer': 200, 'impa_save_clips_plus_features': True, 'lset_class_cut': 'order', 'lset_val_size': 0.1, 'trcl_learning_rate': 0.0001, 'trcl_batch_size': 8, 'trcl_weight_decay': 0, 'trcl_step_size_decay': 5, 'trcl_number_epochs': 75, 'trcl_save_topk': 1, 'trcl_num_classes': 8, 'trcl_model_pretrarch': 'convnext-small', 'trcl_num_workers': 16, 'trcl_wandb_project_name': 'mzb-classifiers', 'trcl_logger': 'wandb', 'trsk_learning_rate': 0.001, 'trsk_batch_size': 32, 'trsk_weight_decay': 0, 'trsk_step_size_decay': 25, 'trsk_number_epochs': 400, 'trsk_save_topk': 1, 'trsk_num_classes': 2, 'trsk_model_pretrarch': 'mit_b2', 'trsk_num_workers': 16, 'trsk_wandb_project_name': 'mzb-skeletons', 'trsk_logger': 'wandb', 'infe_model_ckpt': 'last', 'infe_num_classes': 8, 'infe_image_glob': '*_rgb.jpg', 'skel_class_exclude': 'errors', 'skel_conv_rate': 131.6625, 'skel_label_thickness': 3, 'skel_label_buffer_on_preds': 25, 'skel_label_clip_with_mask': False, 'trcl_gpu_ids': None}

Once we have all set up, we can simply call the inference_classifier function, which will use the provided parameters to produce predictions. If input_dir is a validation set (i.e. called val_set and is structured hierarchically with classes’ names), then a classification accuracy report and confusion matrix are also produced.

inference_classifier(args_p, cfg_p)

Validation set size: 52

GPU available: True (cuda), used: True
TPU available: False, using: 0 TPU cores
IPU available: False, using: 0 IPUs
HPU available: False, using: 0 HPUs
LOCAL_RANK: 0 - CUDA_VISIBLE_DEVICES: [0]

Predicting DataLoader 0: 100%|██████████| 7/7 [00:01<00:00,  5.14it/s]

The produced confusion matrix show how many predictions the model made that corresponded to the real identity of the organism (on the diagonal), and also when the model was wrong what it guessed instead (off diagonal).