Implement Cross-Attention Mechanism between Encoder and Decoder

Implement a cross-attention mechanism that allows a decoder to attend over the encoder's output. Assume that the encoder's output and the decoder's hidden state are already provided as inputs to your function. Your task is to implement the cross-attention logic that connects these two components.

Constraints

• You should use PyTorch for tensor operations.
• Your implementation should efficiently handle the batch processing of sequences.

Examples

{
  "input": {
    "encoder_outputs": "A tensor of shape (batch_size, sequence_length, hidden_size) representing the encoder's output.",
    "decoder_hidden": "A tensor of shape (batch_size, hidden_size) representing the current hidden state of the decoder."
  },
  "output": "A tensor of shape (batch_size, hidden_size) representing the updated hidden state of the decoder after applying cross-attention."
}

{
  "input": {
    "encoder_outputs": "A tensor of shape (batch_size, sequence_length, hidden_size) with all zeros, simulating a case where the encoder output is not informative.",
    "decoder_hidden": "A tensor of shape (batch_size, hidden_size) with all ones, simulating a uniform decoder hidden state."
  },
  "output": "A tensor of shape (batch_size, hidden_size) with values reflecting the uniform attention over non-informative encoder output."
}