Improving Scene Text Recognition in Rainy Weather Conditions with Controlled Rain Realism and Text Readability

Scene text recognition (STR) is crucial for traffic monitoring, surveillance, and autonomous driving.It is even more challenging to read texts in rainy weather conditions. Rain-induced distortions, suchas raindrops, streaks, and rain accumulation, can obscure essential visual features like signboards,traffic signs, and license plates, making meaningful data extraction difficult for recognition models.Recent deep learning methods to add realistic rain to scene text images include no control over thesynthesis of rain and the readability of texts. This work addresses the problem of STR in rainy scenesby proposing a controllable rain synthesis and refinement pipeline that controls rain realism and text-readability in rainy images generated from clean images. The pipeline uses an alternating-projectionrefinement technique by introducing two interpretable hyperparameters: readability suppression (α)and rain realism (w). The optimal setup help improve results on the real rainy dataset. Overall, weobserve the reduction in word error rate (WER) and character error rate (CER) by 8.44% and 9.21%respectively over state-of-the-art benchmarks. We also present the ablations of tuning the parameters(α and w) on STR performance