Shogun ‑ A/B Testing

---UPDATE - July 13, 2026---

Hey there, Hamad.

I just wanted to check if you received the last update and if that has helped.

I am also reaching out via the email thread, and we'd appreciate it if you could reply back to us.

Thanks for your patience and understanding.

Sweetan | Client Services
Shogun

-----------------------------------------------

---UPDATE - July 8, 2026---

Hi Hamad,

Thank you again for taking the time to investigate this so thoroughly. We shared your findings with our engineering team, and I wanted to address each of the points you raised.

Regarding `snippets/shogun-optimizer.liquid`, you're correct that this file can remain in the theme after the app is uninstalled. Once the app is uninstalled, Shopify immediately invalidates our API tokens, which means we're no longer able to modify the theme or remove files as part of a cleanup process.

That said, the optimizer includes a safeguard where, if it hasn't received a refreshed configuration for approximately 24 hours, it automatically stands down. This is intended to prevent visitors from continuing to receive experiments after the app has been uninstalled.

With respect to `snippets/product-media-gallery.liquid`, this is an existing theme snippet that's responsible for rendering your product image gallery. During an AI Theme Modification, this is one of the files that is commonly updated so that Shogun can swap in the appropriate media for products participating in a test. The behavior you described- reading the media metafields and displaying the corresponding images for the active variation is expected.

You also asked whether a syntax error could have been introduced. While our AI-generated modifications go through validation, no automated process is perfect, and it's possible for an issue to slip through in certain scenarios. This is why we recommend reviewing the modified theme and verifying that key pages are functioning as expected before publishing it. We'd be happy to review your theme with you or help rerun the theme modification if necessary to check this better.

Regarding the product metafields, our expectation is generally that Shopify manages app-owned data on uninstall. However, we've seen reports from other developers that, in some cases, app-owned metafields can remain after an app is removed due to Shopify platform behavior.

To answer your remaining questions:

* Reinstalling the app **does not** automatically restart or resume paused tests. It performs the necessary setup, but any previously paused optimization remains paused unless you explicitly start it again.

* The `cartCreate` mutation does **not** modify the price charged at checkout. It simply passes the selected variant to the cart. It has no effect on the final checkout pricing.

* The cart transform function is managed by Shopify and is automatically removed when the app is uninstalled, so it is no longer active after uninstall.

It is possible you were seeing the cached version on your end.

We really appreciate the effort you've put into documenting what you found. Collaborator access would still be valuable because it would allow our engineering team to inspect the current state of your theme and configuration directly, verify whether there are any lingering issues specific to your store, and determine whether there's anything we can improve on our side. If you're open to it, we'd be happy to continue the investigation together.

Thanks again for your detailed report, and I look forward to hearing from you.

Sweetan
Shogun Support

-------------------------------------------
Hi Hamad, we're sorry to hear about your experience.

I see you reinstalled our app on July 2, ran the test for a few hours, stopped the test, and then uninstalled the app.

We weren't able to locate any support conversations under the email address associated with the account. If you contacted us using a different email address, please let us know; we'd be happy to look into that correspondence. That should help us to analyze this better for you.

Regarding the pricing, when a pricing test is started, paused, or stopped, Shogun updates the related data to ensure pricing remains consistent across your storefront. That said, we'd like to investigate your specific experience thoroughly to understand what happened.

I've reached out to you via email to gather a few additional details, and I'll also be sending a Shopify Collaborator access request so our technical team can investigate. We'd really appreciate your reply so we can get to the bottom of this and work toward a resolution.

Thanks.

Sweetan | Client Services
Shogun

Hey there, Shenghai. Thanks for sharing your feedback.

We apologize for the inconvenience you faced that has led to this feedback.

I was looking at the use case and your account, and I want to let you know that when you start/run a product details test, we set up a duplicate theme that has all the required adjustments for you to make the changes to the variant.

We then publish this modified theme. You can treat the modified theme as the new main theme moving forward, and modify that in case you wish to modify anything in the theme. However, if you want, once the test ends, you can republish the old theme again.

Also, about the product details, I wanted to check if you had made changes to the description on the original. When you change the product description, you may want to re-run theme modifications if you have already run it on a previous version. This ensures the theme is set correctly and you can make changes to the variant B of your test.

I see you had run a few product details tests previously, and they were all for the same product.

Having said that, we would love to analyze the issue for you and help you better. I have reached out to you via the same chat thread and we would appreciate if you could reply back to us

We appreciate your support.

Sweetan | Client Services
Shogun

Hi Tim,

Thank you for taking the time to share your feedback. I’m really sorry to hear how frustrating this experience has been.

I’d like to clarify one important point: we do not publish a theme without your approval. Can you please confirm which theme you would refer to, or would you be referring to the activity log?

Regarding the traffic split you experienced, we completely understand how this impacted your confidence in the test results. As mentioned previously in the email thread by my colleague, James, we have identified cases where bot traffic can skew experiment distribution by behaving like normal visitors. This is something our team is actively working on improving, so test data remains reliable.

We truly appreciate you raising both of these concerns and giving us the chance to investigate and also process a refund for both your invoices.

We'll look forward to hearing back from you on the same email thread so that we can look at the other concerns for you.

Thanks for your support.

Sweetan | Client Services
Shogun