LSTM Model with Attention

class lstm_attention_model.AttentionLayer(*args, **kwargs)

Bases: Layer

Custom attention layer compatible with LSTM outputs. Outputs a weighted sum across the time dimension.

Source: https://www.geeksforgeeks.org/adding-attention-layer-to-a-bi-lstm/?

build(input_shape)

Build the attention layer.

Args

  • input_shape: Shape of the input tensor.

Returns

None

call(x)

Apply the attention mechanism to the input tensor.

Args

  • x: Input tensor of shape (batch_size, timesteps, features).

Returns

  • output: Weighted sum of the input tensor across the time dimension.

compute_output_shape(input_shape)

Compute the output shape of the attention layer.

Args

  • input_shape: Shape of the input tensor.

Returns

  • output_shape: Shape of the output tensor.

lstm_attention_model.build_attention_model(input_shape, num_classes)

Build and return an attention-based LSTM model.

Args

  • input_shape: tuple, shape of the input data (timesteps, features)

  • num_classes: int, number of output classes

Returns

  • Keras Model instance

lstm_attention_model.train_attention_model()

LSTM model with custom attention mechanism for multi-class classification of time-series data.

This module defines a deep learning model using TensorFlow and Keras, integrates a custom attention mechanism, and trains the model on preprocessed input data with categorical labels.

Expected data format:

  • Input: X_train.npy, X_test.npy (shape: [samples, 60, features])

  • Labels: y_train.npy, y_test.npy (categorical class indices)

  • Class weights: class_weights.json

The trained model is saved as HDF5 and Keras formats.