An embedding turns a string into a vector, so you compare meaning by distance instead of keywords. That powers semantic search, deduplication and the retrieval step in RAG. EmbeddingGemma 2 is Google’s open embedding model: 740M parameters, 768-dimensional output, and it runs locally through Sentence Transformers.
1. What embeddings are for
If vectors are new to you, read embeddings explained with code first. The short version: similar texts land near each other, so you embed a corpus once, store the vectors, then embed a query and return the nearest matches. EmbeddingGemma expects you to label which side is the query and which is the document, which the next steps handle.
2. Install and accept the license
pip install -U sentence-transformers
Gemma models are gated: open the model page, accept the licence once, then authenticate with hf auth login (see the hf CLI guide).
3. Encode queries and documents
from sentence_transformers import SentenceTransformer
model = SentenceTransformer("google/embeddinggemma-2")
query = "How do I cache Hugging Face models?"
documents = [
"Downloaded weights live in the Hugging Face cache under HF_HOME.",
"A tokenizer splits text into model-sized pieces.",
]
q = model.encode(query, prompt_name="SearchQuery")
d = model.encode(documents, prompt_name="Document")
prompt_name applies the prefixes the model was trained on, task: search result | query: … for the query and title: none | text: … for each document; skip them and retrieval quality drops.
4. Score cosine similarity
scores = model.similarity(q, d)
print(scores) # e.g. tensor([[0.79, 0.31]])
similarity returns cosine similarity by default, so the higher score matches the query to the cache sentence rather than the tokenizer one. Keep the load precision at float32 or bfloat16, because EmbeddingGemma’s activations do not support float16.
5. Feed a vector database
At scale you do not loop in Python: you store vectors in a vector database such as Chroma, FAISS or pgvector and let it do nearest-neighbour search. Embed and insert documents with prompt_name="Document", embed each incoming query with prompt_name="SearchQuery", and wire the results into a generator; minimal RAG in Python shows the full loop. Need something smaller? google/embeddinggemma-300m is the 308M text-only version and uses model.encode_query() and model.encode_document() instead.
If it fails
OSError: ... is a gated repoor401 Client ErrorYou have not accepted the licence or logged in. Open the model page, accept it, then runhf auth loginwith a read token.ValueError: Prompt name 'query' is not in the configurationThe names are case-sensitive. Use exactly"SearchQuery"and"Document".- All-
nanvectors or a float16 error You loaded in float16. Load withSentenceTransformer("google/embeddinggemma-2", model_kwargs={"torch_dtype": "float32"})or use bfloat16.
Next: embeddings explained with code · minimal RAG in Python · RAG explained simply.