Both the draft and the test kept crashing the system. I could never even set one up. They clearly are not ready.
UPDATE This team reached out to me right after I left a one star review offering to help. That's why I gave them as second chance. I respect that kind of commitment.
I have used them extensively since then. It works well.
Things I would like them to add:
1) Factor in cost of goods and create an ROI calculator
2) Make it work on all mobile devices. It's too inconsistent on mobile so I have to use it solely from desktop.
When I first began using this app, it was newly released (I was the first person to review it). I encountered some significant issues that impacted use and told the developers, and they were EXTREMELY responsive and began making modifications to resolve issues that same day. Some of the things that were immediately improved:
-Ability to create a draft test (you can draft an alternative product page, for instance, and save it to continue working on it later)
-Ability to draft test in both html & rich text
-Better instructions on how to stop test / select a winner
-Improvements to workflow and UX
It still feels a little rudimentary and could benefit from some better user directives (guiding a beginner through the steps with some more direction rather than relying on the help section to be used as an instruction book) but I think it's going to work ok. If you run into any issues, rest assured that the developers are VERY responsive. There is also a pretty well written Help section that will provide answers and coach on setting up tests, achieving statistical significance, and the theory behind A/B testing.
All in all, Neat AB currently fills a gap for performing split testing within the Shopify system. I'm sure the developers will continue working on it. In the meantime, I'm running some tests and will probably set up a few more, and hope to update this review after more time spent with the app.