Complex Label Class Maps

For many tasks, a simple tuple of three values is enough to capture everything needed to visualize and interpret a dataset's labels and inferences:

class_entry = al.ClassMapEntry(
    name="car",
    class_id=0,
    color=(200,200,100)
)

Other times, you might have more complex requirements, such as regions of images that you don't want to punish a model for poor semantic segmentation on, or child classes that get rolled up at inference time.

Presented below are some example scenarios and how to implement them in Aquarium. If you have a setup that isn't covered here, reach out and let us know!

Finer-grained labels than inferences

Often, you may end up with a model that predicts fewer classes than you have labels. As an example, let's imagine we're training a self-driving car and want to detect pedestrians and vehicles.

At inference time, our model only returns two classes: person and vehicle.

For whatever reason (more precise performance metrics, future-looking labeling, etc.) we have five finer-grained label classes: child, adult, car, truck, and van.

We can represent that with the following configuration:

# A LabelClassMap can represent class re-mapping between
# labeling time and inference/train time.
label_class_map = al.LabelClassMap(
    entries=[

        # All of our ground truth label classes. They contain a 'train_name'
        # and 'train_id' field, which are used to reference the class used
        # at training/inference time.

        al.ClassMapEntry(
            name="child",
            class_id=0,
            color=color_child,

            train_name="person",
            train_id=5,
            train_color=color_person,
        ),
        al.ClassMapEntry(
            name="adult",
            class_id=1,
            color=color_adult,

            train_name="person",
            train_id=5,
            train_color=color_person,
        ),
        al.ClassMapEntry(
            name="car",
            class_id=2,
            color=color_car,

            train_name="vehicle",
            train_id=6,
            train_color=color_vehicle,
        ),
        al.ClassMapEntry(
            name="truck",
            class_id=3,
            color=color_truck,

            train_name="vehicle",
            train_id=6,
            train_color=color_vehicle,
        ),
        al.ClassMapEntry(
            name="van",
            class_id=4,
            color=color_van,

            train_name="vehicle",
            train_id=6,
            train_color=color_vehicle,
        ),

        # All of our inference-time only classes.
        
        al.ClassMapEntry(
            name="person",
            class_id=5,
            color=color_person,
        ),
        al.ClassMapEntry(
            name="vehicle",
            class_id=6,
            color=color_vehicle,
        ),
    ]
)

In detail for one specific entry:

al.ClassMapEntry(
    # String id of the label class
    name="child",
    # Integer id of the label class
    class_id=0,
    # Color to render corresponding ground truth
    color=color_child,

    # String id of the class this should be treated as
    # when comparing against inferences
    train_name="person",
    
    # Int id of the class this should be treated as
    # when comparing against inferences
    train_id=5,

    # Color this should be rendered as when comparing
    # against inferences
    train_color=color_person,
),

After creating a project with this configuration, we should be able to provide labels and inferences with classes from their corresponding sets, and Aquarium will handle resolving them together.