Shopify Knowledge Base

Knowledge Base still needs a lot of foundational work, especially to be used reliably with Inbox.

Content hallucination. There's no control over AI hallucination, and the tool sometimes invents answers.
Test-and-learn scenarios should feed Knowledge Base more broadly. Right now they seem limited to a single conversation, when they should let the AI learn to adapt to the e-commerce store in question, for wider question coverage and training.
No control over the data source. We have no way to select or correct the data source used for a FAQ, which is surprising for a tool meant to accurately represent our brand. During testing, a URL was invented, along with other incorrect data. The FAQ creation system is a good idea, but it's missing key configuration options (data source, reliability, hallucination level...) to make the tool trustworthy.

Excellent so far. Perfect as it logs all the questions and queries so you can see what customers are asking. Then you can add your custom "FAQ's" to help the Agentic system fill in its knowledge gaps.

It really simplifies the process for the front end Ai to help answer store relative and sensitive questions such as returns process, warranty, services and shipping information.

I think it could expand into some better training fields and add more but my sneaky feeling is Shopify is already building most of this into their features as they go.

The pre-generated FAQ answers are wildly inaccurate! Things like the brands sold aren't pulled from the product listings. One answer even claimed we offer worldwide shipping though we only have two countries active!
Furthermore, the questions and answers are always in English, even if you have a different default language, and lastly, why does this app even exist? Why do we have a bunch of JSON FAQs implemented, if none of them are used?