Cross-attention is a mechanism in transformer architectures that enables one sequence to focus on another, facilitating the integration of information from different sources. In tasks like machine translation, the decoder attends to the encoder’s output, aligning the generated sequence with the input sequence. This interaction allows the model to capture complex relationships between different data modalities, enhancing performance in various applications.
« Back to Glossary Index
« Back to Glossary Index