My embeddings are 1536 or 3072 dimensions. Storage and search are expensive, and some indexes have dimension limits. How can I make them smaller?
Answer 1: Use the provider's built-in dimension option
OpenAI's text-embedding-3 models are trained so that shorter versions still work well. Request the size you want:
resp = client.embeddings.create(model="text-embedding-3-large",
input=texts, dimensions=1024)
The documents and queries must use the same dimensions value.
