We propose a self-organized and self-adapting system for content-based search in multimedia data. In particular, we build a semantic overlay over an existing peer-to-peer network. The self-organization of the overlay is obtained by using the social-network paradigm. The connections between peers are formed on the basis of a query-answer principle. The knowledge of answers to previous queries is exploited to route queries efficiently. At the same time, a randomized mechanism is used to explore new and unvisited parts of the network. In this way, the self-adaptable and robust system is built. Moreover, the metric space data model is used to achieve extensibility. The proposed concepts are verified on a network consisting of 2,000 peers and indexing 10 million images.