Workflow
to detect if workers are wearing PPE.
We will be using this image to test our results:
ClassificationAgent
relies on CLIP
embedding models for zero-shot classification, which, while fast, does not offer the same level of robust image understanding as models like GPTVision
.
Instructor
library to generate structured data. If youβre not familiar with the library we recommend reading the Docs.InstructorImageAgent
generates structured outputs directly from images, streamlining the process with fewer steps while maintaining robustness. The BoundingBoxSelectAgent
allows the InstructorImageAgent
to focus its analysis on person detections.
In this example, we were able to split our image at the person level, instead of having to select for heads. This is possible because the InstructorImageAgent
is able to leverage GPTVision
to provide additional robustness.
InstructorTextAgent
can leverage a much cheaper model like GPT-3.5, or even a local LLM.
We also can leverage robust models like GPTVision
to improve the robustness of our model in DenseCaptioningAgent
if necessary.
This approach provides the flexibility in choosing between robustness and speed.