4.6 Review

Continual Deep Learning for Time Series Modeling

相关参考文献

注意:仅列出部分参考文献,下载原文获取全部文献信息。
Article Computer Science, Artificial Intelligence

A wholistic view of continual learning with deep neural networks: Forgotten lessons and the bridge to active and open world learning

Martin Mundt et al.

Summary: Current deep learning methods are evaluated based on their performance on dedicated test sets, without considering real-world scenarios. This has led to the overlook of important lessons from open set recognition and active learning. In this work, a consolidated view is proposed to bridge the gap between continual learning, active learning, and deep neural networks in open set recognition. Empirical results show that this approach improves catastrophic forgetting, data querying, task ordering, and robust open-world application.

NEURAL NETWORKS (2023)

Article Computer Science, Artificial Intelligence

Online continual learning in image classification: An empirical survey

Zheda Mai et al.

Summary: Online continual learning for image classification aims to address the challenges of learning to classify images from an online stream of data and tasks. State-of-the-art methods such as MIR, iCaRL, and GDumb have been compared to understand their relative advantages under different settings. The study concludes that MIR is a strong and versatile method across various settings, bringing online continual learning closer to its ultimate goal of matching offline training.

NEUROCOMPUTING (2022)

Article Computer Science, Artificial Intelligence

Continual learning of neural networks for quality prediction in production using memory aware synapses and weight transfer

Hasan Tercan et al.

Summary: This paper explores deep learning-based predictive quality and proposes a method for continual learning to adapt to the ongoing changes in manufacturing processes. The evaluation demonstrates that the approach successfully prevents the neural network from forgetting previous tasks, improves training efficiency for new tasks, and enhances performance on sparse data by transferring network weights from similar previous tasks. Our publicly available code allows for result reproducibility and further development.

JOURNAL OF INTELLIGENT MANUFACTURING (2022)

Article Computer Science, Artificial Intelligence

Continual Learning for Real-World Autonomous Systems: Algorithms, Challenges and Frameworks

Khadija Shaheen et al.

Summary: This article reviews the state-of-the-art methods for continuous learning of computational models over time and critically analyzes the key challenges associated with continual learning for autonomous real-world systems. The article also compares current methods in terms of computations, memory, and network/model complexity and briefly describes the implementations of continuous learning algorithms in autonomous systems such as self-driving vehicles, unmanned aerial vehicles, and urban robots.

JOURNAL OF INTELLIGENT & ROBOTIC SYSTEMS (2022)

Article Computer Science, Artificial Intelligence

Towards Continual Reinforcement Learning: A Review and Perspectives

Khimya Khetarpal et al.

JOURNAL OF ARTIFICIAL INTELLIGENCE RESEARCH (2022)

Review Chemistry, Multidisciplinary

Advances in Machine Learning for Sensing and Condition Monitoring

Sio-Iong Ao et al.

Summary: This paper presents a comprehensive survey on the advances in machine learning algorithms and their recent applications in the sensing and condition monitoring fields. It carefully selects and discusses case studies of developing tailor-made data mining and deep learning algorithms from practical aspects. The characteristics and contributions of these algorithms to the sensing and monitoring fields are elaborated.

APPLIED SCIENCES-BASEL (2022)

Letter Multidisciplinary Sciences

Deep learning for tipping points: Preprocessing matters

Fabian Dablander et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2022)

Article Environmental Sciences

A Deep Learning Time Series Approach for Leaf and Wood Classification from Terrestrial LiDAR Point Clouds

Tao Han et al.

Summary: The study utilizes deep learning to separate leaf and woody components from terrestrial laser scanning data, comparing the performance of different neural network models. The results indicate that the multivariable time series (MTS) method has a higher accuracy than the univariable time series (UTS), while ResNet takes longer in model development.

REMOTE SENSING (2022)

Review Neurosciences

Contributions by metaplasticity to solving the Catastrophic Forgetting Problem

Peter Jedlicka et al.

Summary: Catastrophic forgetting refers to the severe loss of prior information when learning new information, which is a challenge for artificial neural networks. The brain has solved this problem during evolution, and now modellers are using various strategies, including drawing on metaplasticity phenomena, to overcome catastrophic forgetting.

TRENDS IN NEUROSCIENCES (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Streaming Traffic Flow Prediction Based on Continuous Reinforcement Learning

Yanan Xiao et al.

Summary: Traffic flow prediction is crucial in smart transportation for predicting future traffic conditions based on historical data. This study presents a streaming traffic flow prediction model based on continuous reinforcement learning, which can continuously learn and predict traffic flow as the network expands and evolves, while also providing data privacy protection to some extent.

2022 IEEE INTERNATIONAL CONFERENCE ON DATA MINING WORKSHOPS, ICDMW (2022)

Proceedings Paper Computer Science, Artificial Intelligence

Rethinking Video Anomaly Detection - A Continual Learning Approach

Keval Doshi et al.

Summary: This study takes a comprehensive look at the problem of video anomaly detection, considering practical challenges such as continual learning and few-shot learning. A novel algorithm is proposed and evaluated using a new dataset and performance metric, which shows significant improvements over existing methods in continual learning and few-shot learning tasks.

2022 IEEE WINTER CONFERENCE ON APPLICATIONS OF COMPUTER VISION (WACV 2022) (2022)

Article Computer Science, Information Systems

Robust Unsupervised Anomaly Detection With Variational Autoencoder in Multivariate Time Series Data

Umaporn Yokkampon et al.

Summary: This article discusses the importance of accurately detecting anomalies in multivariate time series data and proposes a new unsupervised anomaly detection algorithm. The algorithm uses a Multi Scale Convolutional Variational Autoencoder (MSCVAE) to capture the inter-correlations between time series and an attention-based ConvLSTM network to capture temporal patterns. Experimental results show that the proposed framework outperforms competing algorithms in terms of model performance and robustness.

IEEE ACCESS (2022)

Review Computer Science, Artificial Intelligence

Biological underpinnings for lifelong learning machines

Dhireesha Kudithipudi et al.

Summary: Biological organisms learn from interactions with their environment, and artificial systems also need the ability to learn throughout their lifetime. This article introduces biological mechanisms and artificial models and mechanisms for lifelong learning, and discusses opportunities to bridge the gap between natural and artificial intelligence.

NATURE MACHINE INTELLIGENCE (2022)

Article Engineering, Civil

Deep learning-based detection of structural damage using time-series data

Hung V. Dang et al.

Summary: This study extends the applicability of four prominent deep learning algorithms to structural damage detection using raw data, showing insights on their performance in various structures through quantitative evaluations of detection accuracy, time complexity, and required data storage in multi-damage scenarios.

STRUCTURE AND INFRASTRUCTURE ENGINEERING (2021)

Review Computer Science, Artificial Intelligence

An Experimental Review on Deep Learning Architectures for Time Series Forecasting

Pedro Lara-Benitez et al.

Summary: Research demonstrates that long short-term memory (LSTM) and convolutional networks (CNN) are the best options for time series forecasting, with LSTMs yielding the most accurate predictions; CNNs show more stable performance under different parameter configurations and are also more efficient.

INTERNATIONAL JOURNAL OF NEURAL SYSTEMS (2021)

Article Environmental Sciences

Predicting Plant Growth from Time-Series Data Using Deep Learning

Robail Yasrab et al.

Summary: Phenotyping involves quantitative assessment of plant traits, but natural growth cycles can be slow. Deep learning and machine learning-based high-throughput phenotyping offer solutions to automate and accelerate experimental processes. The study demonstrates predicting plant growth using segmentation masks and shows strong performance on public datasets.

REMOTE SENSING (2021)

Article Computer Science, Interdisciplinary Applications

Deep Learning for Time Series Forecasting: A Survey

Jose F. Torres et al.

Summary: Time series forecasting is a research field where deep neural networks play a significant role in solving big data problems. This work explores the application of deep learning methods in time series prediction, discussing the advantages and limitations of common architectures. Practical recommendations on parameter settings and framework selection are provided, along with a review of successful cases in various research fields and identification of research gaps.

BIG DATA (2021)

Article Automation & Control Systems

Processes soft modeling based on stacked autoencoders and wavelet extreme learning machine for aluminum plant-wide application

Yongxiang Lei et al.

Summary: The paper presents an effective and efficient soft modeling method (SAE-WELM) for industrial processes, which outperforms existing state-of-the-art methods in capturing deep features with faster speed and higher accuracy.

CONTROL ENGINEERING PRACTICE (2021)

Article Computer Science, Artificial Intelligence

Continual learning for recurrent neural networks: An empirical evaluation

Andrea Cossu et al.

Summary: Continuous learning is fundamental for deploying machine learning solutions and recurrent neural networks have the potential for a wide range of applications. Current research on Continual Learning (CL) for sequential data processing is fragmented and lacks clear evaluation criteria. Results show that sequence length and clear definition of CL scenario play a key role in mitigating forgetting in class-incremental scenarios.

NEURAL NETWORKS (2021)

Article Computer Science, Artificial Intelligence

Pseudo-rehearsal: Achieving deep reinforcement learning without catastrophic forgetting

Craig Atkinson et al.

Summary: The proposed model overcomes catastrophic forgetting in sequential reinforcement learning by combining ideas from continual learning in both the image classification domain and the reinforcement learning domain. It features a dual memory system and a pseudo-rehearsal system, and successfully learns Atari 2600 games without forgetting previous tasks.

NEUROCOMPUTING (2021)

Review Multidisciplinary Sciences

Time-series forecasting with deep learning: a survey

Bryan Lim et al.

Summary: This article explores the diversity of time-series datasets across different domains, discussing common encoder and decoder designs, and how deep learning models incorporate temporal information into predictions. Additionally, recent developments in hybrid deep learning models are highlighted, along with ways in which deep learning can facilitate decision support.

PHILOSOPHICAL TRANSACTIONS OF THE ROYAL SOCIETY A-MATHEMATICAL PHYSICAL AND ENGINEERING SCIENCES (2021)

Article Computer Science, Artificial Intelligence

Smoothing and stationarity enforcement framework for deep learning time-series forecasting

Ioannis E. Livieris et al.

Summary: This study introduces a new framework for enhancing deep learning time-series models based on data preprocessing techniques. The framework focuses on generating high-quality time-series data through a series of transformations, significantly improving forecasting performance as demonstrated in comprehensive numerical experiments and statistical analysis.

NEURAL COMPUTING & APPLICATIONS (2021)

Article Multidisciplinary Sciences

A clinical deep learning framework for continually learning from cardiac signals across diseases, time, modalities, and institutions

Dani Kiyasseh et al.

Summary: The authors propose a continual learning strategy called CLOPS to alleviate the performance degradation of deep learning algorithms trained on data streamed temporally from different clinical sites and physiological sensors. Experimental results show that CLOPS outperforms state-of-the-art methods in various continual learning scenarios, highlighting its potential to pave the way for robust diagnostic systems over time.

NATURE COMMUNICATIONS (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Continual Learning for Multivariate Time Series Tasks with Variable Input Dimensions

Vibhor Gupta et al.

Summary: This study addresses specific challenges in multivariate time series learning tasks, proposing solutions for different sensor subsets and data storage restrictions, and improving task performance by learning task-specific generative models and classifiers.

2021 21ST IEEE INTERNATIONAL CONFERENCE ON DATA MINING (ICDM 2021) (2021)

Proceedings Paper Computer Science, Artificial Intelligence

Toward Application of Continuous Power Forecasts in a Regional Flexibility Market

Yujiang He et al.

Summary: This article discusses the use of deep neural network models in a continual learning framework to improve the accuracy of forecasting non-stationary power data and applies it to the regional flexibility market in Germany, effectively enhancing the accuracy of electricity forecasts and the precision of grid operator analysis.

2021 INTERNATIONAL JOINT CONFERENCE ON NEURAL NETWORKS (IJCNN) (2021)

Review Computer Science, Information Systems

Deep Learning for Anomaly Detection in Time-Series Data: Review, Analysis, and Guidelines

Kukjin Choi et al.

Summary: With the advancement of industrial automation and connectivity technologies, systems are generating massive amounts of data. Detecting anomalies in time series data is crucial for preventing accidents and economic losses. Recent research in deep learning has made significant progress in this field, although most models require domain knowledge for appropriate deployment.

IEEE ACCESS (2021)

Article Computer Science, Artificial Intelligence

Prediction Error-Driven Memory Consolidation for Continual Learning: On the Case of Adaptive Greenhouse Models

Guido Schillaci et al.

Summary: This research presents an adaptive architecture that addresses catastrophic forgetting issues through an episodic memory system and prediction-error driven memory consolidation. The congruence of memories with prior knowledge is estimated in terms of prediction error from a deep neural model. The proposed AI system is successfully applied to the learning and transfer of greenhouse models in the horticulture industry.

KUNSTLICHE INTELLIGENZ (2021)

Article Biochemical Research Methods

Scaling tree-based automated machine learning to biomedical big data with a feature set selector

Trang T. Le et al.

BIOINFORMATICS (2020)

Article Computer Science, Artificial Intelligence

A spatio-temporal decomposition based deep neural network for time series forecasting

Reza Asadi et al.

APPLIED SOFT COMPUTING (2020)

Article Computer Science, Artificial Intelligence

Robust IoT time series classification with data compression and deep learning

Joseph Azar et al.

NEUROCOMPUTING (2020)

Article Computer Science, Artificial Intelligence

A novel validation framework to enhance deep learning models in time-series forecasting

Ioannis E. Livieris et al.

NEURAL COMPUTING & APPLICATIONS (2020)

Article Computer Science, Artificial Intelligence

Multimodal multitask deep learning model for Alzheimer's disease progression detection based on time series data

Shaker El-Sappagh et al.

NEUROCOMPUTING (2020)

Article Multidisciplinary Sciences

Understanding deep learning in land use classification based on Sentinel-2 time series

Manuel Campos-Taberner et al.

SCIENTIFIC REPORTS (2020)

Article Environmental Sciences

Rice Crop Detection Using LSTM, Bi-LSTM, and Machine Learning Models from Sentinel-1 Time Series

Hugo Crisostomo de Castro Filho et al.

REMOTE SENSING (2020)

Review Behavioral Sciences

Embracing Change: Continual Learning in Deep Neural Networks

Raia Hadsell et al.

TRENDS IN COGNITIVE SCIENCES (2020)

Article Geography, Physical

Self-attention for raw optical Satellite Time Series Classification

Marc Russwurm et al.

ISPRS JOURNAL OF PHOTOGRAMMETRY AND REMOTE SENSING (2020)

Article Multidisciplinary Sciences

Time series classification: nearest neighbor versus deep learning models

Weiwei Jiang

SN APPLIED SCIENCES (2020)

Article Computer Science, Information Systems

Traffic Flow Forecast Through Time Series Analysis Based on Deep Learning

Jianhu Zheng et al.

IEEE ACCESS (2020)

Article Computer Science, Information Systems

Time Series Data for Equipment Reliability Analysis With Deep Learning

Baotong Chen et al.

IEEE ACCESS (2020)

Review Computer Science, Artificial Intelligence

Deep learning for time series classification: a review

Hassan Ismail Fawaz et al.

DATA MINING AND KNOWLEDGE DISCOVERY (2019)

Review Chemistry, Multidisciplinary

Wavelet Transform Application for/in Non-Stationary Time-Series Analysis: A Review

Manel Rhif et al.

APPLIED SCIENCES-BASEL (2019)

Article Engineering, Electrical & Electronic

Deep Learning with Long Short-Term Memory for Time Series Prediction

Yuxiu Hua et al.

IEEE COMMUNICATIONS MAGAZINE (2019)

Review Genetics & Heredity

Review of Causal Discovery Methods Based on Graphical Models

Clark Glymour et al.

FRONTIERS IN GENETICS (2019)

Article Computer Science, Interdisciplinary Applications

A spatiotemporal deep learning model for sea surface temperature field prediction using time-series satellite data

Changjiang Xiao et al.

ENVIRONMENTAL MODELLING & SOFTWARE (2019)

Article Computer Science, Artificial Intelligence

Learning without Forgetting

Zhizhong Li et al.

IEEE TRANSACTIONS ON PATTERN ANALYSIS AND MACHINE INTELLIGENCE (2018)

Article Computer Science, Theory & Methods

Efficient IoT-based sensor BIG Data collection-processing and analysis in smart buildings

Andreas P. Plageras et al.

FUTURE GENERATION COMPUTER SYSTEMS-THE INTERNATIONAL JOURNAL OF ESCIENCE (2018)

Article Automation & Control Systems

Deep Coupling Autoencoder for Fault Diagnosis With Multimodal Sensory Data

Meng Ma et al.

IEEE TRANSACTIONS ON INDUSTRIAL INFORMATICS (2018)

Article Computer Science, Artificial Intelligence

ASCERTAIN: Emotion and Personality Recognition Using Commercial Sensors

Ramanathan Subramanian et al.

IEEE TRANSACTIONS ON AFFECTIVE COMPUTING (2018)

Article Multidisciplinary Sciences

Comparing continual task learning in minds and machines

Timo Flesch et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2018)

Proceedings Paper Computer Science, Cybernetics

Introducing WESAD, a Multimodal Dataset for Wearable Stress and Affect Detection

Philip Schmidt et al.

ICMI'18: PROCEEDINGS OF THE 20TH ACM INTERNATIONAL CONFERENCE ON MULTIMODAL INTERACTION (2018)

Review Radiology, Nuclear Medicine & Medical Imaging

Convolutional neural networks: an overview and application in radiology

Rikiya Yamashita et al.

INSIGHTS INTO IMAGING (2018)

Review Computer Science, Artificial Intelligence

The great time series classification bake off: a review and experimental evaluation of recent algorithmic advances

Anthony Bagnall et al.

DATA MINING AND KNOWLEDGE DISCOVERY (2017)

Article Multidisciplinary Sciences

Overcoming catastrophic forgetting in neural networks

James Kirkpatricka et al.

PROCEEDINGS OF THE NATIONAL ACADEMY OF SCIENCES OF THE UNITED STATES OF AMERICA (2017)

Article Multidisciplinary Sciences

Human-level control through deep reinforcement learning

Volodymyr Mnih et al.

NATURE (2015)

Review Multidisciplinary Sciences

Deep learning

Yann LeCun et al.

NATURE (2015)

Article Computer Science, Artificial Intelligence

The Graph Neural Network Model

Franco Scarselli et al.

IEEE TRANSACTIONS ON NEURAL NETWORKS (2009)

Article Computer Science, Artificial Intelligence

A fast learning algorithm for deep belief nets

Geoffrey E. Hinton et al.

NEURAL COMPUTATION (2006)

Article Computer Science, Information Systems

Providing public intradomain traffic matrices to the research community

S Uhlig et al.

ACM SIGCOMM COMPUTER COMMUNICATION REVIEW (2006)