Overview
We will construct workflows designed to enforce safety equipment compliance in construction areas. Specifically, we will show a few possible approaches related to how you could construct aWorkflow to detect if workers are wearing PPE.
We will be using this image to test our results:

Approach 1: Using ClassificationAgent
ClassificationAgent relies on CLIP embedding models for zero-shot classification, which, while fast, does not offer the same level of robust image understanding as models like GPTVision.

Prerequisite: The following approaches leverage the
Instructor library to generate structured data. If youβre not familiar with the library we recommend reading the Docs.Approach 2: Using InstructorImageAgent
InstructorImageAgent generates structured outputs directly from images, streamlining the process with fewer steps while maintaining robustness. The BoundingBoxSelectAgent allows the InstructorImageAgent to focus its analysis on person detections.
In this example, we were able to split our image at the person level, instead of having to select for heads. This is possible because the InstructorImageAgent is able to leverage GPTVision to provide additional robustness.

Approach 3: Using DenseCaptioningAgent
InstructorTextAgent can leverage a much cheaper model like GPT-3.5, or even a local LLM.
We also can leverage robust models like GPTVision to improve the robustness of our model in DenseCaptioningAgent if necessary.
This approach provides the flexibility in choosing between robustness and speed.