Roman K. Abramovich, Viacheslav Yu. Dobrynin, Alexey V. Platonov

ITMO university

Traditional search methods based on sparse vector representations are characterized by high efficiency but limited quality due to their inability to capture semantic relationships in the data. On the other hand, dense vector representations can improve quality by capturing semantic relationships. However, these methods face scalability issues and require significant computational resources. With the development of deep neural networks, including transformer-based architectures, there is a growing interest in combining these two approaches. The purpose of this review paper is to review existing works that use deep models to generate sparse representations.

deep neural networks, semantic search, computational resources, sparse representations, inverted index