Reproduced on Qwen3-Embedding-0.6B · TEI 1.9.3

A healthy server.
Different vectors.

The model, revision, dimensions, and health endpoint all matched. Normal requests passed. Client batches passed. Concurrent router batching collapsed cosine similarity to 0.1586.

MODELQwen3-Embedding-0.6B

COMPARISONSentenceTransformers ↔ TEI 1.9.3

Normal request1.0000PASS

Client list batch1.0000PASS

Concurrent router batch0.1586FAIL

Same model · Same dimensions · Healthy endpoint

embed-parity is a differential test for catching silent embedding changes across runtimes, batching modes, prompts, pooling, and production workloads.

brew install kraftaa/tap/embed-parity

pip install 'embed-parity[sentence-transformers]'

View source ↗

Interactive failure lab

The same text should not care who arrived beside it.

Switch request patterns and runtime builds. These are measured cosine similarities from the pinned Qwen3 reproduction of open TEI issue #882.

Minimum cosine 1.0000

Result Stable

One request at a time produces the isolated reference embedding.

What actually broke

“No padding” accidentally became “no causal mask.”

Padding only fills shorter rows. A causal mask is separate: it controls which token positions may look ahead. Equal-length requests need no padding, but Qwen3 still needs its causal mask.

databaseindexesspeedqueries

oceancurrentsshapeweather

Padding maskNot needed

Causal maskSkipped by bug

Reproduced, not simulated

A current upstream regression ordinary checks miss.

Pinned model revision. Pinned container digests. Identical probes. The proposed patch changes only the broken condition.

Request path Official 1.9.3 PR #883 build

Client list batch1.00001.0000

Concurrent router batch0.15861.0000

Unequal-length control1.00001.0000

Open TEI issue #882 ↗ Proposed fix PR #883 ↗ Reproduction artifacts ↗

One command, layered evidence

Check the embedding behavior you actually deploy.

terminal

$ embed-parity compare \
  --model Qwen/Qwen3-Embedding-0.6B \
  --revision 97b0c614... \
  --tei http://localhost:8080 \
  --concurrent-requests 4 \
  --concurrency-trials 3

Concurrent server batching
  responses compared               12
  minimum cosine                   0.266183
  divergent responses              6
                                     FAIL

Migration evidence

Compare the search results, not only the vectors.

Bring representative queries and documents. The workload report compares each runtime independently and shows exactly which rankings changed—without claiming retrieval quality when no relevance labels exist.

Matching configuration 100%

Top-1, top-5, and top-10 parity on the deterministic SciFact sample.

Unintended prompt 86.4%

Top-5 overlap, despite mean vector cosine remaining at 0.9959.