Product(s): Portkey
Overall Comment:"Portkey has been a crucial part of our success with LLM observability, guardrails, and cost management. The Portkey team has also been there every step of the way to make sure our implementation was successful and met our needs. Not only was the implementation smooth, but if we did find ourselves with questions, the Portkey team was very responsive and eager to help support us. Now that we are in production with Portkey we are seeing the true value of the product for not only our original needs (LLM observability, guardrails, and cost management), but also for other aspects such as model agnostic development and analytic insights to how the application is being used by users."
- Observability - the ability to see the details of prompts, responses, latency, user info, etc. - Guardrails - ability to prevent PII, toxicity, bias, etc. - Cost Management - setting rate/budget limits, seeing what each call/app is costing - Redundancy/Failover - allowing retry logic or to failover to another LLM provider if one is available without any code change - Model agnostic development - being able to change LLMs on the back-end of the application without having to change any code.
It's hard to find much to dislike, but if I had to really pick Portkey apart I would say: - Constant changing - because of the AI landscape changing right now, Portkey has to try to keep up with new processes/platforms/features. Because of this, the product seems to have updates almost every other day. -Workspace level security settings - at the time of writing this, there is only org level security settings. It would be nice to be able to assign these at a workspace level