With similarity_search_with_score, the most relevant document has the lowest score. In another setup, all scores are high, even for unrelated text.
Why this happens
There are two separate issues here.
The first is distances versus similarities. Many tools return a distance, where lower means more similar, rather than a similarity, where higher means more similar. FAISS IndexFlatL2 returns squared L2 distances. Chroma uses L2 distance by default.
LangChain's similarity_search_with_score passes through whatever the underlying store returns.
The second is that many embedding models place all texts in a narrow region of the space. As a result, even unrelated texts can have cosine similarities around 0.6–0.8. The ranking is still meaningful, but the absolute values are not intuitive.
Answer 1: Find out what your tool returns
Check whether you're getting a distance or a similarity. Search for a document using its own text: if the top score is 0 (or close to it), you're getting a distance. If it's 1, you're getting a cosine similarity.
