DECT: Distributed Evolving Context Tree for Understanding User Behavior Pattern Evolution

Mar 7, 2016

Internet user behavior models characterize user browsing dynamics or the transitions among web pages. The mod- els help Internet companies improve their services by accu- rately targeting customers and providing them the informa- tion they want. For instance, specific web pages can be cus- tomized and prefetched for individuals based on sequences of web pages they have visited. Existing user behavior models abstracted as time-homogeneous Markov models cannot efficiently model user behavior variation through time. This paper presents DECT, a scalable time-inhomogeneous variable-order Markov model. DECT digests terabytes of user session data and yields user behavior patterns through time. We realize DECT using Apache Spark. Our imple- mentation is being open-sourced and we deploy DECT on top of Yahoo! infrastructure. We demonstrate the benefits of DECT with anomaly detection and ad click rate prediction applications. DECT enables the detection of higher-order path anomalies that are masked out by existing models. DECT also provides deep insights into ad click rates with respect to user visiting paths. 

  • EDBT (EDBT 2016)
  • Conference/Workshop Paper