Anomaly detection in time-series data

Authors:

Tommy S. Alstrøm (Associate Professor, CogSys, DTU Compute - bldg. 321, room 007 - tsal@dtu.dk)

Milena Bajic (PostDoc in CogSys, DTU Compute - bldg. 321, room 120 - mibaj@dtu.dk)

Maximillian F. Vording (PhD Student in CogSys, DTU Compute - bldg. 321, room 106 - maxvo@dtu.dk)

Introducing and motivating the problem

Maybe with some plots of the data and figures illustrating the vehicles+sensors collecting it

Proposed projects outline:

Variational autoencoders (VAE):

Standard VAE

Blogpost Tutorials:

Variational autoencoders.

Tutorial - What is a variational autoencoder? How to build them in Keras: A Tutorial on Variational Autoencoders with a Concise Keras Implementation

Window convolutions

Blogpost Tutorials:

2D conv: An Intuitive Explanation of Convolutional Neural Networks Conv Nets: A Modular Perspective

Dilated convolutions

Blogpost Tutorials:

2D: Understanding 2D Dilated Convolution Operation with Examples in Numpy and Tensorflow with… How they are used for auto-regressive models over tagged time-series: WaveNet: A generative model for raw audio

Recurrent units (GRU or LSTM)

Understanding LSTM Networks A Beginner's Guide to LSTMs and Recurrent Neural Networks The Unreasonable Effectiveness of Recurrent Neural Networks

Deep weakly supervised learning with attention mechanisms:

Attention mechanisms: Attention and Augmented Recurrent Neural Networks

Papers:

[1803.02353] Multi-level Attention Model for Weakly Supervised Audio Classification [1903.00765] Weakly Labelled AudioSet Tagging with Attention Neural Networks

Git repos:

qiuqiangkong/audioset_classification

Convolutional Gaussian processes (CGP):

Blogpost Tutorials:

Understanding Gaussian processes

Papers:

• Introducing CGP:
  • [1709.01894] Convolutional Gaussian Processes
• GPs for time-series:
  • Gaussian processes for time-series modelling
• GP regression for anomaly detection in time-series:
  • Anomaly detection based on data stream monitoring and prediction with improved Gaussian process regression algorithm - IEEE Conference Publication
• GP regression on binned data:
  • [1809.02010] Gaussian Process Regression for Binned Data
  • Binned Kernels for Anomaly Detection in Multi-timescale Data using Gaussian Processes

Git repos:

markvdw/convgp: Convolutional Gaussian processes based on GPflow. kekeblom/DeepCGP: Deep convolutional gaussian processes.

Libraries w. GPU acceleration in Python:

GPFlow: Convolutional Gaussian Processes GPyTorch

General Software Links:

PyTorch

Online courses and lectures

• Free introductory course on Deep Learning in Pytorch: Intro to Deep Learning with PyTorch.
  Especially useful lessons are:
  • Lesson 4: Introduction to PyTorch where you learn how to build a first simple deep learning model using PyTorch.
  • Lesson 5: Convolutional Neural Networks CNN in PyTorch
  • Lesson 6: Recurrent Neural Networks RNN in PyTorch

Blogpost Tutorials:

PyTorch 101: Building Your First Neural Network
PyTorch 101, more advanced: Going Deep with PyTorch

Git repos:

PyTorch-101-Tutorial