Expert Insights

Oji Udezu makes a compelling argument for treating AI prompts as code and the implications of this for version control. The adjustment is critical for maintaining the reliability and consistency of AI responses in dynamic user environments.

Oji explores the inherent unpredictability of AI models and the need for prompt versioning for traces and control. He underscores dealing with the fluctuating outputs of AI models, which can produce different results for similar prompts across different users.

Hear Oji explain:

  • Why treating AI-generated prompts as code enhances version control.
  • How the unpredictability of AI models presents challenges for achieving consistent responses.
  • How dealing with indeterminacy in AI outputs affects user experience and requires careful handling.

Quote

quotation-marks icon
I realized very early on that prompts were code, right? And I think there are products now that will help you, version prompts, but we have to version it ourselves.” How do you deal with, uh, Insta, you know, like on deterministic output from these models? Because you can ask it the same thing for 10 different people and you get different answers.quotation-marks icon
Oji Udezu ,
Co-Author, ProductMind

THE NEW DEFAULT angle

Here are key steps for teams looking to implement AI-assisted workflows:

    • Treat AI prompts as code: Recognize that AI-generated prompts are part and parcel of the source code and need a similar level of control and tracking.

    • Implement version control for prompts: Just as with code, keep track of modifications and iterations of AI prompts to control the different outputs from the AI models.

    • Address indeterminacy in model output: Put mechanisms in place to handle varied responses from models. This could involve developing a standardized method for interpreting non-deterministic output or providing prompts tailored to individual user attributes.

    • Train AI models for consistency: Ensure that your AI model is trained sufficiently to provide consistent results.

    • Take user experience into account: Be prepared to address varied responses from the AI system to user prompts in a way that augments rather than hinders the user experience.