Kaicheng (Kevin) Guo

• Advisor: Prof. Eric Eaton

• Overall GPA: 3.92/4.00
• Selected courses: Robust Algorithms for ML, Deep Learning, Computer Vision, Optimization, Learning & Sequential Decision Making, Systems for ML, Probabilistic Methods in CS, Functional Analysis, PDE, Honors Statistical Inference, Graph Theory, Abstract Algebra

Spectral Collapse Drives Loss of Plasticity in Deep Continual Learning

Arjun Prakash*, Naicheng He*, Kaicheng Guo*, Saket Tiwari, Tyrone Serapio, Ruo Yu Tao, Amy Greenwald, George Konidaris

International Conference on Machine Learning (ICML 2026)

We investigate why deep neural networks suffer from loss of plasticity in continual learning, and thus fail to learn new tasks without reinitializing parameters. We show that this failure is preceded by Hessian spectral collapse at new-task initialization, where meaningful curvature directions vanish and gradient descent becomes ineffective. Analyzing a linearized ReLU network, we derive explicit ε-rank conditions for successful training and prove that the loss-weighted Gram matrix is spectrally equivalent to the Generalized Gauss-Newton approximation, thereby relating NTK dynamics to Hessian curvature. Targeting spectral collapse directly, we then discuss the Kronecker factored approximation of the Hessian, which motivates two regularization enhancements: maintaining high effective feature rank and applying L2 penalties. Experiments on continual supervised and reinforcement learning tasks confirm that combining these two regularizers effectively preserves plasticity.

Exploring Non-Markov Environments Using Random Recurrent Memories

Ruo Yu Tao*, Kaicheng Guo*, Cameron Allen, Ronald Parr, George Konidaris

RL in Big Worlds Workshop, Reinforcement Learning Conference (RLC 2026)

Exploring an environment requires a reinforcement learning agent to keep track of what it has already explored, which is difficult when observations do not fully reveal environment state. In the Markov setting, exploration algorithms have focused on achieving systematic coverage of observed states. These same methods have frequently been applied to the non-Markov setting, where they aim to achieve coverage over observations. However, decision-making in the non-Markov setting often depends on the agent's entire history, as opposed to only single observations. Unfortunately, achieving systematic coverage over the space of all trajectories is untenable: exploration over histories is exponentially expensive due to the dependence of the search space on the horizon. We therefore propose a new family of methods to featurize the agent's history with random recurrences. This produces finitely-sized random statistics, or random recurrent memories, over an agent's history, and we aim for coverage over these memories. We describe desirable properties for efficient history compression with random recurrences and propose a new architecture type, the tangent recurrent unit (TRU). We show that in a diverse suite of partially observable exploration tasks, tangent recurrent units, as well as other structured random recurrences, outperform popular methods that aim for observation coverage.

Benchmarking Partial Observability in Reinforcement Learning with a Suite of Memory-Improvable Domains

Ruo Yu Tao*, Kaicheng Guo*, Cameron Allen, George Konidaris

Reinforcement Learning Conference (RLC 2025)

Mitigating partial observability is a necessary but challenging task for general reinforcement learning algorithms. To improve an algorithm's ability to mitigate partial observability, researchers need comprehensive benchmarks to gauge progress. Most algorithms tackling partial observability are only evaluated on benchmarks with simple forms of state aliasing, such as feature masking and Gaussian noise. Such benchmarks do not represent the many forms of partial observability seen in real domains, like visual occlusion or unknown opponent intent. We argue that a partially observable benchmark should have two key properties. The first is coverage in its forms of partial observability, to ensure an algorithm's generalizability. The second is a large gap between the performance of agents with more or less state information, all other factors roughly equal. This gap implies that an environment is memory improvable: where performance gains in a domain are from an algorithm's ability to cope with partial observability as opposed to other factors. We introduce best-practice guidelines for empirically benchmarking reinforcement learning under partial observability, as well as the open-source library POBAX: Partially Observable Benchmarks in JAX.

• Investigating trajectory-based exploration strategies for partially observable domains where traditional state-based exploration fails.
• Developed the Lock environment, where rewards depend on modular properties of full trajectories rather than single observations.
• Studying how memory traces and recurrent mechanisms enable agents to infer hidden structures and recover near-optimal memory function.

• Investigating why RNNs outperform MLPs even in fully observable environments.
• Demonstrated that MLPs break the Markov property after embedding and show empirically RNN exhibit greater expressiveness.
• Proposed RNN-Skip, a novel architecture that removes explicit recurrence while retaining the representational benefits of recurrent models.

• Developed TOLD-ZERO agent for model-based reinforcement learning with improved sample efficiency.
• Implemented and evaluated performance on Cartpole environment, achieving stable returns of 200.
• Analyzed training dynamics showing high variability in early phases followed by stable convergence.

• Conducted comprehensive survey of fast approximate maximum flow algorithms including theoretical analysis and practical implementations.
• Implemented and compared performance of various maxflow algorithms across different graph types and problem sizes.
• Analyzed trade-offs between accuracy and computational efficiency in approximate solutions to network flow problems.