Supersonic: A/B Testing & CRO

The idea is great, but...

Analytics and testing has a lot of errors, at first I was excited by the functionalities it they offer, but very quickly I've learned that app is super underdeveloped. I created my first test, and on the first try it seemed like it worked, but the next day, when I checked the analytics it showed random analytics like 100% conversion rate and 2 visits, even if we had like 200 visits on the product page. Also the same day I tested if it really works, I opened the page on a few different devices while the split test was live and I couldn't see the split test working, It was still original variant, even when refreshing multiple times and using private browser.

I contacted them and the problem wasn't solved, they told me that I need to wait 1 day, I waited, but errors are still here and I've got no reply for 3 days now.

here's the chat transcript

"It seems your app has errors

Profile image for Nataliya
Can you please specify what you mean? Could you let me know where you encountered the errors?

wrong visitor count
with that said, I even doubt the analytics are correct

Profile image for Nataliya
Thank you for sharing this information! Let us check that

only today we've got 100+ visits

Profile image for Nataliya
Hi! The metrics are being updated on a daily basis. You need to wait when your test is "live" for at least 1 full calendar day. Today's stats should be updated tomorrow (general GA statement: 24 to 48 hours). Thanks!

wow, can i change that?
btw i checked the test, i refreshed multiple times the page, I'm only getting my original variant
it should be 50/50 no?
I doubt it shows modified variant for other people, since i cannot see it myself
also is it possible to run test based on time, not 50/50 visitor count
for example change between variants every 1 hour

Profile image for Nataliya
Make sure that you saved your changes in the widget editor
2. You have a preview option link that should preview the needed variant with customization
3. It's not possible to setup a test based on a time
4. Additionally, the visitors count is being calculated based on a user's interaction with a test. If a user won't go through a test it won't be counted in your stat.

it worked yesterday
today, doesn't
now i just paused the test

December 17
well, with that kind of responsibility and concern I won't become your long term client which is what you should be aiming for as a business

data is still incorrect, I've sent you a screenshot of how many visitors we already captured in the first half of the day while the test was still active.

I turned it off, because there was an error already, which I've told you about: It worked the first moment, when i created the test and tested if it works. Then this day when I messaged you, I didn't work, I refreshed and went on the product page from multiple devices, I was only shown the org variant"