TrackMate Python Class in Memory Processing¶

This document outlines the workflow of the trackmate_in_mem.py script, which performs cell tracking using TrackMate via pyimagej. The script is designed to work with pre-segmented label images or user-provided spot data in a pandas DataFrame, providing a flexible in-memory tracking pipeline.

References¶

py-imagej Documentation: https://py.imagej.net/en/latest/index.html
TrackMate Scripting: https://imagej.net/plugins/trackmate/scripting/scripting
scyjava Forum Discussion: https://forum.image.sc/t/importing-cellpose-via-scyjava-for-trackmate-automation-in-python-3/84176

Core Logic Flow¶

The script follows these main steps:

Initialization: Sets up the ImageJ/Fiji environment using pyimagej.
Data Input: Accepts either a pre-segmented label image or a pandas DataFrame with cell centroids (spot data in Trackmate).
Spot Extraction: If a label image is provided, it extracts spot features (like position, size, and intensity) for each detected object in each frame.
TrackMate Configuration: A TrackMate model is configured in memory. Since spots are already detected, it uses the ManualDetectorFactory.
Tracking: The tracking algorithm (e.g., Sparse LAP Tracker) is executed on the spots.
Results Export: The tracking results (tracks and spots) are returned as pandas DataFrames.

Mermaid UML Diagram¶

        graph TD
    A[Start] --> B{Input Type?};
    B -- Label Image --> C[Load Image and Labels];
    C --> D[Extract Spots to DataFrame];
    B -- DataFrame --> D;
    D --> E[Create TrackMate Model];
    E --> F[Configure Detector and Tracker];
    F --> G[Add Spots to Model];
    G --> H[Run Tracking];
    H --> I[Export Results to DataFrame];
    I --> J[End];

Input and Output Format¶

The TrackMateInMemoryProcessor class will be adapted to handle dictionary-based inputs and outputs, similar to cyto/tracking/trackmate.py.

Input `dict`¶

The input will be a dictionary containing either image/label data or a pre-computed spots DataFrame.

input_data = {
    "image": image_array,  # (Optional) numpy array of the original image in XYT form
    "labels": label_array, # (Optional) numpy array of segmented labels in XYT form
    "features": features_df   # (Optional) pandas DataFrame with features data that maps to trackmate spots
}

If features is not provided, it will be generated from labels and image.

Input DataFrame Columns¶

The input DataFrame should contain information about detected spots (cells or objects) in each frame. Each row represents a single spot, and columns provide details required for TrackMate to perform tracking. Below is a detailed description of each expected column:

Column Name	Description	TrackMate Mapping
`label`	Unique identifier for each spot within a frame. This is typically an integer assigned to each segmented object. It helps TrackMate distinguish between different spots in the same frame.	Spot ID
`y`, `x`	Centroid coordinates of the spot in pixel units. These specify the spatial location of the spot in the image. Both coordinates are required for accurate placement in TrackMate.	`POSITION_Y`, `POSITION_X`
`frame`	The time point (frame index) at which the spot appears. This is a zero-based integer indicating the temporal position of the spot in the sequence.	`FRAME`
`feret_radius`	A size metric for the spot, often calculated as the radius of the smallest enclosing circle (Feret diameter/2). This value is used by TrackMate to estimate the spot’s physical size.	`RADIUS`
`mean`	The mean intensity value of the spot, typically computed from the original image. This can be used as a quality metric and is mapped to TrackMate’s `QUALITY` feature, as well as being available for intensity-based analyses.	`QUALITY` (and intensity features)

This DataFrame structure allows TrackMate to reconstruct the spatial and temporal context of each spot, enabling robust tracking and downstream analysis.

Output `dict`¶

The output will be a dictionary containing the tracking results.

output_data = {
    "features": tracks_dataframe, # DataFrame with combined spot and track information. This will map back to pyCyto dataframe header standard
}

`scyjava` and Fiji Plugin Management¶

scyjava is the bridge that allows Python to interact with Java objects from the JVM, which is how pyimagej works. This is crucial for controlling TrackMate, which is a Fiji plugin written in Java.

Sideloading Fiji Plugins with TrackMate¶

The TrackMateInMemoryProcessor class demonstrates how to use scyjava to import and use Java classes from TrackMate directly in Python.

Initialization: imagej.init('sc.fiji:fiji') automatically downloads a local Fiji distribution that includes TrackMate and its dependencies.

Importing Java Classes: The _import_java_classes method uses scyjava.jimport to make TrackMate’s Java classes available in the Python environment.

# Example of importing a Java class
self.Model = scyjava.jimport('fiji.plugin.trackmate.Model')
self.SparseLAPTrackerFactory = scyjava.jimport('fiji.plugin.trackmate.tracking.jaqaman.SparseLAPTrackerFactory')

This in-memory approach avoids the overhead of using command-line scripts for communication and allows for a more seamless and efficient integration of TrackMate into Python workflows. By managing the Fiji instance and its plugins through pyimagej and scyjava, we can build powerful and flexible image analysis pipelines.