In human lupus nephritis, tubulointerstitial inflammation (TII) is associated with in situ expansion of B cells expressing anti-vimentin antibodies (AVAs). The mechanism by which AVAs are selected is unclear. Herein, we demonstrate that AVA somatic hypermutation and selection increase affinity for vimentin. However, enzyme-linked immunosorbent assays (ELISAs) suggested that affinity maturation might be a non-specific consequence of increasing polyreactivity. Subsequent multi-color confocal microscopy studies indicated that while TII AVAs often appeared polyreactive by ELISA, they bound selectively to vimentin fibrils in whole cells or inflamed renal tissue. Using a novel machine learning pipeline (CytoSkaler) to quantify the cellular distribution of antibody staining, we demonstrated that TII AVAs were selected for both enhanced binding and specificity in situ . These data suggest a new approach to assess and define antibody polyreactivity based on quantifying the distribution of binding to native and contextually relevant antigens. ### Competing Interest Statement The authors have declared no competing interest.