ABsolutely: A/B Testing

tl;dr site is now unreachable for returning customers.

I try to avoid leaving negative reviews unless there's something egregious.

A test was having 2 issues. 1. not aggregating sales results correctly. 2. Showing data "A" on site but data "B" at checkout, resulting in customer confusion. Not my first and only issue with the app, nor an issue bad enough to result in 1 star (though it's a pretty big customer impacting issue).

1 star because after ending this test in question, users can no longer reach our site (ERR_TOO_MANY_REDIRECTS). The only way to reach our site now is if the user clears cache, but realistically there's no way to share that information with a customer. The conclusion customers reach is that the website it down.

I've had suspicions that how they handled A/B testing was a bit of a hack for awhile now, but it wasn't a large enough issue to act on.

I messaged them via their support system on Sunday, got an auto response that I will be emailed back due to it being the weekend. It is now mid-day Tuesday with no response.

Arguably I could allow for more time, but imo a site being unreachable is a priority 1 issue for immediate response.

Uninstalling the application did not result in a fix.

Really easy to use app, quite intuiative and you can have tests up and running in minutes. The customer service is also great. I had a big issue blocking me from getting the test live and they quickly jumped on it, listen to my (very poorly explained!) issues and help deliver a fix within 24 hours. I have been very impressed with so far.

We have a heavily customized store with some non-standard checkout logic, and running a price test on top of it turned out to be technically complicated. Absolutely handled it far better than I expected.

When we hit a conflict, they dug in and found the actual root cause. They came back within 24 hours with a working fix and had tested it themselves before telling us it would work.

They'd also already gone out of their way earlier to accommodate our catalog size so we could test everything we needed to, which is why we picked them over several alternatives.

Fast, responsive, and they actually understand what's happening under the hood. If you've got a setup that isn't cookie-cutter, these are the people you want.

One of the best customer service experiences I had with a Shopify App provider. Highly recommend!