Back to projects
Epitech Hub · Solo
WorldVideo
A ConvLSTM that watches 10 frames of a video and predicts the next 10.
- Python
- PyTorch
- ConvLSTM
- Period
- Jun – Jul 2026
- Team
- Solo project
- Code
- View the code
The project
WorldVideo predicts what happens next in a video. From the first 10 frames of a sequence, the model generates the next 10.
The model is a ConvLSTM with 4 layers of 128 channels, trained on Moving MNIST: 10,000 sequences of 20 frames at 64 × 64. I evaluate it with MSE and SSIM.
What was hard
- Training a ConvLSTM with 4 layers of 128 channels needs a GPU: training ran on another machine, with no network between the two.
- The further the prediction gets from the 10 input frames, the more it blurs, especially when two digits cross.
What I learned
- Starting from a reference architecture, OpenSTL’s, adapting it and crediting it, rather than rewriting everything.
- Evaluating image prediction with two complementary metrics: MSE, pixel by pixel, and SSIM, which looks at structure.
- Tracking a training run from end to end with Weights & Biases.
Try it.
Real against predicted
On the left the real sequence, on the right what the model generates. Pause and step frame by frame: the prediction blurs as it moves away from the starting frames.
Credits
- ConvLSTM architecture adapted from OpenSTL (Apache 2.0 licence).
- Data: Moving MNIST, Srivastava et al., 2015.




