Embedding Cache (historical)

Embedding Cache is a mechanism that stores computed embeddings in the data/embeddings directory, preventing repeated API calls for phrases that have already been vectorized. If the embedding for a phrase like 'garden pools' already exists in cache, the pipeline skips the API call and uses the stored vector—saving both cost and time.

In clustering pipelines, cache becomes especially important because the same phrase can appear across multiple cycles of topical map expansion. Cache format: JSON or CSV file mapping phrases to vectors. The cache is persistent (on disk, not in memory), so it survives session restarts and notebook changes.

Critical rule: cache is valid ONLY with the same embedding model and task_type—changing either requires complete regeneration. In practice, implement cache from day one of embedding work—with regular pipeline use, savings reach 50–80% of API costs.