Overview
We will construct workflows designed to enforce safety equipment compliance in construction areas. Specifically, we will show a few possible approaches related to how you could construct aWorkflow
to detect if workers are wearing PPE.
We will be using this image to test our results:

Approach 1: Using ClassificationAgent
ClassificationAgent
relies on CLIP
embedding models for zero-shot classification, which, while fast, does not offer the same level of robust image understanding as models like GPTVision
.

Prerequisite: The following approaches leverage the
Instructor
library to generate structured data. If youβre not familiar with the library we recommend reading the Docs.Approach 2: Using InstructorImageAgent
InstructorImageAgent
generates structured outputs directly from images, streamlining the process with fewer steps while maintaining robustness. The BoundingBoxSelectAgent
allows the InstructorImageAgent
to focus its analysis on person detections.
In this example, we were able to split our image at the person level, instead of having to select for heads. This is possible because the InstructorImageAgent
is able to leverage GPTVision
to provide additional robustness.

Approach 3: Using DenseCaptioningAgent
InstructorTextAgent
can leverage a much cheaper model like GPT-3.5, or even a local LLM.
We also can leverage robust models like GPTVision
to improve the robustness of our model in DenseCaptioningAgent
if necessary.
This approach provides the flexibility in choosing between robustness and speed.