Organizing Your Data
Organize your data using Aquarium Segments
Introduction to Segments
A Segment is a grouping of datapoints in Aquarium. Use segments to organize and manage subsets of your dataset at either the frame or crop level. Segments are available in multiple types each of which is tied to a core workflow in Aquarium.
Previously called Issues, Segments was renamed to reflect the expanded Data Collection and Model Performance feature set.
The Python SDK (calls and associated payloads) will continue to use the Issues naming and won't be renamed to avoid breaking existing integrations.
Data Quality
Frame Issue - Use frame issue segments to track and manage issues with your underlying frame data. This can be particularly useful when data collection systems are feeding high-scale, unlabeled data into Aquarium and you need to prioritize where to label or run inference.
Data Collection
Collection Campaign - Use collection campaign segments to define a search seed and then obtain relevant examples from large-scale, unlabeled data holdings - without needing to manually review the entire dataset. Learn more about implementing Collection Campaigns in Aquarium.
Model Performance
Model Performance Segments are an all-new way to evaluate the performance of your models in Aquarium. Learn more about setting up model performance segments and integrating regression tests into your model evaluation process.
Split - Use split segments to track and report model performance on the Test, Training and Validation sets within your dataset.
Regression Test - Use regression test segments to define subsets of the dataset that mimic your domain's hardest problems, set target model performance thresholds and then easily assess pass-fail for every model experiment or release candidate.
Scenario - Use scenarios to evaluate model performance against subsets of your data
Data Organization
Bucket - Use bucket segments as a catch-all for data organization or an intermediary step in the process of moving to a more defined workflow.
Creating and Populating Segments
Frame or Crop Level Segments
The most important thing to remember when creating a segment is that you can create a segment comprised of frame level data or crop level data. Depending on how you add elements to a segment, the segment will default to be created as one specific type.
Depending on the kind ML task you are working with, you may only be able to create segments of one type.
For example:
With a Classification task, you'll only have frame/image level segments created because there are no bounding box/polygon labels
With an Object Detection task, you can create segments at the frame/image level and at the object/bounding box level
Note that Model Performance Segments are only available at the Frame level, so if you'd like to create a Split, Regression Test or Scenario, be sure the Frame / Crop toggle is set to Frame.
When working through a data curation flow, creating a Collection Campaign segment of type frame or crop will impact if you can run similarity searches between the entire image or specific objects, respectively. So make sure you create the right type of segment depending on what you're looking for.
The kind of segment (frame vs crop) is determined when you initially create and add data to the segment. For example, If you are creating a segment from a label or inference bounding box, you will create a crop level segment.
However, if you start the segment creation process with a crop, when you choose to create a Segment you can use the Frame / Crop toggle to decide if you want to create a segment with the selected crop or if you want to instead add the whole image to a frame level segment.
In the section below, we will point out how to create both frame and crop level segments for each main view in Aquarium.
How to Add Element to Segment
Anywhere you can select a group of frames or crops in Aquarium, you can create a Segment.
In the Grid View and the Embedding View, you also have the option to toggle between frames and crops to influence what kind of segment you will create.
From the Grid View:
From the Embedding View:
From the Frame Detail View:
To add a frame to a segment, right click within the image to add the Frame to a segment.
To add a crop to a segment, select a label or inference and either right click the label overlay or click the 'Add to Segment' button near the label metadata on the right hand side of the screen.
Managing Segments
The Segments page is accessible from the top navigation bar and holds all of the Segments your team has created, organized by primary type.
Click in to a segment to access the segment details page. From here you can:
Provide comments about the segment, visible to the rest of your team,
Manage the elements assigned to the segment,
Start an intra-dataset similarity search to find additional relevant elements,
Move elements between segments
Edit the segment's metadata.
Filtering Based on Issues
You can use the "in_segment" and "not_in_segment" filters in the query bar to include or exclude datapoints. This is particularly useful for looking at the queue of failure datapoints that you have not yet triaged into an issue.
For more information on how to use the query bar, see this page!
The following filter filters for datapoints that are not in any issue:
Last updated
