Perfecto Platform

The support we get from the Customer Success Manager is second to none and the 24x7 support team helps resolve our issues faster. We also test on beta OS and the latest GA version of Android and iOS, and Perfecto usually supports these new versions quickly when they are released. The AI feature analyses the failed tests and assigns the appropriate failure category.eg. script failure, back end error, authentication issue etc.

In my experience, automated testing of native mobile applications has always been a pain point. Sure, testing responsive web pages is easy enough, but native mobile applications are an entirely different hurdle. Perfecto lab allows a generous suite of approaches to interacting with native mobile virtual machines, and is executable on a large breadth of various devices, ranging from mobile phones to tablets, for both Android and iOS, and spanning a large number of OS versions.

The device farms are always available and accessible 24/7. Customer support is highly responsive and proactive in addressing issues. The partnership between our organizations has been strong since the beginning.

Incase you need to swap devices, you will have to wait up to 24 hours for the devices to be physically swapped. A few Chinese model devices like Oppo sometimes have stability issues.

Perfecto can be very slow, and often tests will fail intermittently with very little feedback. Often, I can execute a test script and it will work, walk away for a while and rerun the same script and it will fail. There is usually a good reason, but the reason is often not obvious to the user. Another pain point is that often the running script will get hung up, and the live stream will continue to run without explanation; this can cause further confusion, because if one is to rerun the test, it will fail because the device is already in use, but no feedback is given. This required constant vigilance in monitoring the cloud platform to monitor exactly what is going on. Lastly, Perfecto times out frequently and logs the user out. I do appreciate the intention of keeping the user session safe, but I feel that the timeout is too frequent. Perhaps a user-defined option for this time would be a good solution.

One challenge has been initial script debugging, especially when test failures are intermittent. While Perfecto provides video logs and screenshots, digging into mobile network logs or deeper device-level issues still takes time. We'd also like to see more streamlined integrations with our existing test frameworks (e.g., Appium and Cypress for web), as most of our team is more familiar with those.