Preprint, not peer reviewed · Version 1 · October 6, 2026
A few vectors for a mixed folder: a pre-registered measurement of pooled directory embeddings
Ali Basheer, Craftpine Inc., Toronto. ali@basheerali.com
Abstract
A folder's metadata is read before its files and holds a few kilobytes. A vector stored there can route a query without opening the folder, and the cheapest is the mean of its files' embeddings. We ask when the mean is enough and what to store when it is not. Under our main vision-language embedder, the mean does at least as well as a few representatives on GitHub directories that do not mix media, about 95 percent of those eligible. Representatives for mixed folders only are level with the mean over all directories (both post hoc). Where a folder mixes images and texts, the mean loses its minority modality. On Zenodo record folders a held-out minority file finds its folder 0.16 to 0.25 recall@5 less often with the mean than with a few per-modality representatives under two vision-language embedders, and 0.35 to 0.50 less often under two CLIP-family dual towers. There, four representatives per label are not inferior to searching every file, at a 0.02 margin, over all queries and in every cell under all four encoders (post hoc). On queries that a language model wrote as a simulated searcher, the loss under the main embedder is 0.14 (interval 0.02 to 0.28) for pictures in text-heavy folders and 0.42 (0.28 to 0.56) for documents in image-heavy ones. At three per label every folder's representatives fit in 8 KiB at a further cost of at most 0.012 in any cell. In 2 KiB a per-kind sample of the folder's files keeps the minority files the mean loses, and on ext4 a summary of 2,048 or 4,000 bytes costs 1.2 blocks per cold directory read. With every creator weighted equally the main embedder's file-query loss is 0.13 and 0.14, and 0.06 and 0.12 on queries whose names do not give the folder away. Every hypothesis was fixed in a brief before its test. Five briefs are timestamped externally, the rest dated by the working repository's history. Code, manifests and briefs are released.
Keywordsdirectory embeddings · pooled embeddings · modality gap · mixed-modality retrieval · vision-language embeddings · collection representation · semantic file systems · information retrieval · pre-registration
The question
A file system reaches a folder before it reaches the folder’s files. The folder’s own metadata can be read without opening anything inside it, but it holds little. On ext4 a file’s extended attributes, the named values attached to a file or a directory, normally have to fit in its inode and one block of at most 4,096 bytes.
The cheapest vector to keep there is the mean of the folder’s file embeddings. It is the dense counterpart of the single bag of words by which early federated search summarised a collection, and one router for retrieval-augmented generation already takes each source’s centroid as an input.
Folders mix media, and in contrastive models image and text embeddings occupy offset regions of one space. The study asks when the mean is enough, what it loses in a mixed folder, and what a folder’s few kilobytes can keep instead.
Data and models
The Zenodo folders are public records of 3 to 60 files that hold both an image type and a document type, under CC0 or CC-BY, from 2017 to 2026. Their images are mostly scientific figures.
The GitHub folders are directories of public repositories with at least five stars and a permissive licence. Their tests take the repository’s owner as the independent unit. One of the study’s 27 sessions, session 23, also walks 261 whole repository trees.
| Draw | Source | Folders | Files | Role |
|---|---|---|---|---|
| S1 | Zenodo | 400 | 4,773 | Pilot |
| S3 | Zenodo | 400 | 4,891 | Pre-registered retest |
| S5 | Zenodo | 2,808 | 27,545 | Main set |
| S11 | Zenodo | 2,597 | 25,990 | Confirmation, four encoders |
| S19 | GitHub | 1,600 | 12,656 | First draw |
| S24 | GitHub | 2,600 | 21,854 | Second draw |
Four encoders from three vendors read the same files. Two are vision-language embedders, jina-embeddings-v4 (E1, the main one, 2,048 dimensions) and gme-Qwen2-VL-2B-Instruct (E3, 1,536). The other two are CLIP-family dual towers, jina-clip-v2 (E2, 1,024) and Nomic Embed v1.5 (E4, 768).
Images are read as images. Text, tables, text PDFs and office documents give their first 2,000 tokens of extracted text.
Experimental design
The task is directory retrieval. A held-out file is a simulated known-item query, and its own folder is the one relevant item. Recall@5 is the fraction of queries whose own folder ranks in the top five of all folders in the set. It is reported with 95 percent paired bootstrap intervals over directories, and the GitHub tests resample owners.
Each folder is represented from its remaining files, with the query file left out, and scored by the maximum cosine over its vectors. The pooled centroid (a) is the mean of the file vectors. Per-label centroids (b) keep one mean per modality label. Per-modality representatives (c) run k-means within each label, up to three vectors per label. The flat walk (d) keeps every file vector and is the reference.
Queries are read in cells by modality and by the image fraction of their folder, so that images in text-heavy folders and texts in image-heavy ones are measured apart. Later sessions add record titles, commit subjects and queries written by a simulated searcher.
Findings
The paper asks seven questions. Its answers follow, with differences in recall@5. The paper’s appendix lists every pre-registered hypothesis with its verdict.
When is the mean enough?
On GitHub directories that do not mix media, about 95 percent of those eligible, the mean does at least as well as a few representatives under E1 (post hoc).
What does the mean lose in a mixed folder?
Its minority modality. On Zenodo folders the loss is 0.16 to 0.25 under the two vision-language embedders and 0.35 to 0.50 under the two dual towers. On GitHub directories it is about 0.2 under E1, for files with a sibling of their kind.
The pooled vector loses most in the minority cells, P1 and P2, and less in the majority cells. Points are recall@5 of c minus a with paired 95 percent intervals, and the shaded bands span the ranges quoted above. Source: Table 2 and Figure 3 of the paper. Does the loss reach a simulated searcher’s queries?
Under E1 the loss is 0.14 for pictures in text-heavy folders and 0.42 for documents in image-heavy ones, on queries a language model wrote as a simulated searcher.
Why?
A mixed folder is two clusters, its images and its texts, and the mean sits between them. Per-modality mean-centering recovers 8 to 62 percent of the loss, depending on the encoder and the cell.
Under every encoder an image and a text from the same folder are less alike than two images or two texts, even after centering has closed most of the gap between the modality means (right-hand numbers). Mean cosine over same-folder file pairs on S11. Source: Table 4 and Figure 5 of the paper. What fixes it, and at what budget?
A few k-means representatives per modality label. None of the single vectors tried recovers the loss without giving up the majority. Four per label are not inferior to searching every file, at a 0.02 margin, over all queries and in every cell under all four encoders (post hoc). At three per label every folder’s representatives fit in 8 KiB, at a further cost of at most 0.012 in any cell. On ext4 a summary of 2,048 or 4,000 bytes costs 1.2 blocks per cold directory read.
Most of what searching every file (d) gains over the mean (a) is already there at c, the per-modality representatives. Recall@5 under E1 on S11, with tb2c for two label-blind centered clusters. Source: Table 3 and Figure 7 of the paper. Stored at one bit per dimension, the representatives of c take less room on average than the single float32 vector of a. Every S11 folder’s c fits in 8 KiB at one or four bits per dimension, depending on the encoder. Source: Section 12 of the paper. Does a folder summary guide a walk down a tree?
Not yet at 2 KiB. On 261 repository trees the guided walk finds the target 0.069 less often than reading every directory’s own summary, though it reads 0.39 as many.
Does a written folder abstract do better?
Not for the folder’s images.
Limitations
The Zenodo folders are researchers’ deposits, and the GitHub folders come from public repositories. Personal working directories, mailboxes and agent memories were not measured. Relevance is structural. A file’s one relevant folder is its own, so the test measures whether a folder vector represents its files. It does not measure topical relevance across folders.
No person wrote queries. The searcher queries cover 100 folders of one set and were written by a model, whose queries may be more specific than a person’s. The two vision-language embedders share a Qwen lineage. The two dual towers are the weakest encoders here, so part of their larger loss is the encoders.
Each Zenodo set ranks 400 to 2,808 folders, and summaries were written to one file system, ext4. Larger folders, deeper trees and other file systems were not measured.
Pre-registration, code and data
The study ran as 27 sessions, each with a brief written before it. Twenty-two tested a hypothesis whose kill criterion was fixed before the numbers that decide it were computed.
The briefs of sessions 19, 21, 23, 24 and 27 are timestamped with OpenTimestamps, each anchored in a Bitcoin block, and the repository holds the proofs. The others are dated by the working repository’s history. Analyses added after review are marked post hoc, and the hypotheses that died or stayed inconclusive are reported beside the ones that survived.
The repository holds the embedding and evaluation scripts, a second evaluation written from a specification, and the script and driver of every session. The outputs of sessions 25 to 27 rebuild from per-query files without a GPU.
The corpora are not redistributed. The manifests give each file’s path, hash, licence and source URL, and two fetch scripts rebuild the corpora from them. Ground truth, selections and per-query ranks are included.
Every brief is in BRIEF.md, and every session’s verdicts, readings and deviations are in results/. The script scripts/verify_prereg.py checks each timestamped commit from its git objects against the briefs.
Integrity
To check a copy of the PDF, compare its SHA-256 hash with this one.
SHA-256 of the PDF
8bef21aed187c640b44ebeb35f17b89704fc0df508e0e21186e8dbb82a9d5068
Cite as
APABasheer, A. (2026). A few vectors for a mixed folder: a pre-registered measurement of pooled directory embeddings (Version 1) [Preprint]. Zenodo. https://doi.org/10.5281/zenodo.23178122
IEEEA. Basheer, "A few vectors for a mixed folder: a pre-registered measurement of pooled directory embeddings," Zenodo, preprint, Oct. 6, 2026, doi: 10.5281/zenodo.23178122.
ChicagoBasheer, Ali. 2026. "A Few Vectors for a Mixed Folder: A Pre-Registered Measurement of Pooled Directory Embeddings." Version 1. Preprint, Zenodo, October 6. https://doi.org/10.5281/zenodo.23178122.
@misc{basheer2026fewvectors, author = {Basheer, Ali}, title = {A few vectors for a mixed folder: a pre-registered measurement of pooled directory embeddings}, year = {2026}, month = oct, date = {2026-10-06}, publisher = {Zenodo}, version = {1}, doi = {10.5281/zenodo.23178122}, url = {https://doi.org/10.5281/zenodo.23178122}, note = {Preprint, not peer reviewed. Code and data: https://github.com/ali-basheer/dirvec-study}}
Related research
Work the paper builds on, grouped as the paper discusses it. The full bibliography has 69 entries.
The modality gap
- Liang et al. (2022). Mind the Gap: Understanding the Modality Gap in Multi-modal Contrastive Representation Learning. NeurIPS. doi:10.52202/068431-1280
- Li et al. (2025). Closing the Modality Gap for Mixed Modality Search. doi:10.48550/arXiv.2507.19054
Representing a collection
- Callan, Lu and Croft (1995). Searching distributed collections with inference networks. SIGIR. doi:10.1145/215206.215328
- Si and Callan (2003). Relevant document distribution estimation method for resource selection. SIGIR. doi:10.1145/860435.860490
- Shokouhi and Si (2011). Federated Search. Foundations and Trends in Information Retrieval. doi:10.1561/1500000010
Folders and sources as units of retrieval
- Gifford et al. (1991). Semantic file systems. SOSP. doi:10.1145/121132.121138
- Dhasade et al. (2026). Efficient Federated Search for Retrieval-Augmented Generation Using Lightweight Routing. DAIS. doi:10.1007/978-3-032-27358-1_1
- Lee et al. (2025). RouterRetriever: Routing over a Mixture of Expert Embedding Models. doi:10.48550/arXiv.2409.02685
- Wang et al. (2026). Directory-Aware Query and Maintenance in Vector Databases. doi:10.48550/arXiv.2606.16903
Several vectors per item
- Weston, Weiss and Yee (2013). Nonlinear latent factorization by embedding multiple user interests. RecSys. doi:10.1145/2507157.2507209
- Pal et al. (2020). PinnerSage: Multi-Modal User Embedding Framework for Recommendations at Pinterest. KDD. doi:10.1145/3394486.3403280
The four encoders
- Günther et al. (2025). jina-embeddings-v4: Universal Embeddings for Multimodal Multilingual Retrieval. MRL Workshop. doi:10.18653/v1/2025.mrl-main.36
- Koukounas et al. (2024). jina-clip-v2: Multilingual Multimodal Embeddings for Text and Images. doi:10.48550/arXiv.2412.08802
- Zhang et al. (2025). Bridging Modalities: Improving Universal Multimodal Retrieval by Multimodal Large Language Models. CVPR. doi:10.48550/arXiv.2412.16855
- Nussbaum, Duderstadt and Mulyar (2024). Nomic Embed Vision: Expanding the Latent Space. doi:10.48550/arXiv.2406.18587
Corrections
Send corrections to ali@basheerali.com.