Complex activity recognition is challenging since a complex activity can be performed in different ways, with each having its own configuration of primitive events and their temporal dependencies. To address such temporal relational variabilities in complex activity recognition, we propose a Bayesian network- based probabilistic generative framework that employs Allen’s interval relation network to represent local temporal dependencies in a generative way. By employing the Chinese restaurant process and introducing relation generation constraints, our framework can characterize these unique internal configurations of a particular complex activity as a joint distribution. Three concrete models are implemented based on our framework. Specifically, in this paper we improve two of our previous models and provide an enhanced model to handle temporal relational variabilities in complex activities more efficiently. Empirical evalu- ations on three benchmark datasets demonstrate the competitiveness of our framework. In particular, it is shown that our models are rather robust against errors caused by the low-level predictions from raw signals.