Update README

README.md

@@ -1,3 +1,45 @@
 ---
 license: apache-2.0
 ---
+
+
+
+# Imperfect Generative Models are Secretly Strong Vision-Centric Enhancers
+
+Code: https://github.com/mashijie1028/Gen4Rep
+
+Paper: TBD
+
+Project Page: https://mashijie1028.github.io/gen4rep/
+
+
+
+## Introduction
+
+The synergy between generative and discriminative models receives growing attention. While discriminative Contrastive Language-Image Pre-Training (CLIP) excels in high-level semantics, it struggles with perceiving fine-grained visual details. Generally, to enhance representations, generative models take CLIP's visual features as conditions for reconstruction. However, the underlying principle remains underexplored.
+
+In this work, we empirically found that **visually** perfect generations are not always optimal for representation enhancement. The essence lies in effectively extracting fine-grained knowledge from generative models while mitigating irrelevant information. To explore critical factors, we delve into three aspects: (1) Conditioning mechanisms: We found that even a small number of local tokens can drastically reduce the difficulty of reconstruction, leading to collapsed training. We thus conclude that utilizing **only** global visual tokens as conditions is the most effective strategy. (2) Denoising configurations: We observed that end-to-end training introduces extraneous information. To address this, we propose a two-stage training strategy to prioritize learning useful visual knowledge. Additionally, we demonstrate that lightweight denoisers can yield remarkable improvements. (3) Generation paradigms: We explore both continuous and discrete denoisers with desirable outcomes, validating the versatility of our method.
+
+Through our in-depth exploration, we have finally arrived at an effective method that consistently outperforms prior arts on the MMVP-VLM benchmark, *e.g.*, 6.0% on OpenAICLIP. The enhanced CLIP can be plugged into multimodal large language models for better vision-centric performance.
+
+
+
+## This repo
+
+The proposed two-stage post-training scheme serves as a *plug-and-play* method to enhance pre-trained CLIPs' fine-grained representations, and here we release the enhanced model weights of [OpenAICLIP](https://huggingface.co/openai/clip-vit-large-patch14-336), [MetaCLIP](https://huggingface.co/facebook/metaclip-h14-fullcc2.5b) and [SigLIP](https://huggingface.co/google/siglip-so400m-patch14-384).
+
+We also attach the evaluation codes in `evaluation/`.
+
+
+
+## Citation
+
+```
+@article{ma2025genrep,
+      title={Imperfect Generative Models are Secretly Strong Vision-Centric Enhancers},
+      author={Ma, Shijie and Ge, Yuying and Wang, Teng and Guo, Yuxin and Ge, Yixiao and Shan, Ying},
+      journal={arXiv preprint arXiv:2503.TODO},
+      year={2025}
+}
+```
+