Dealbreaker - Lan et al

Machine Learning based education problem is student-response modeling i.e., developing principled statistical models that (i) accurately predict unobserved student responses to questions and (ii) identify the latent concepts that govern correct or incorrect responses. The Rasch Model: simple yet effective for analyzing student-response data. This model characterizes the probability of a correct response as a function of two scalar parameters: the student's ability and the question's difficulty. Affine functions - characterize a student's probability of success on a question as an affine function of the student's knowledge on underlying concepts. Affine models allow weak knowledge of a concept to be erroneously covered up by strong knowledge of other potentially unrelated concepts. Affine models also fail to capture more complicated nonlinear dynamics underlying student responses.

We read the DQN paper - Playing Atari with Deep Reinforcement Learning by Min et. al and a variation of the DQN algorithm, DQRN - Deep Recurrent Q-Learning for Partially Observable MDPs - by Hausknecht & Stone.

Here is an annotated version of the paper with some useful side notes and external references DQN and DQRN.

Some of the questions that came up while discussing these papers:

• How does the \(\gamma \) affect the policy that is learnt? Also, is there an optimal \(\gamma \) for every particular game?
• Further, how does one decide on the correct in a more strategic game like Dota versus in Cartpole. The main argument being that in cartpole, you always want to give your immediate rewards a lot of weight given the nature of the game. However, in a more strategic game, that is not necessarily the case.

Here's a cartpole demo with help from external implementations here. I have added some experiments surrounding the questions that came up during the reading group.

This is a reading of the paper - "Personalized Thread Recommendation for MOOC Discussion Forums" by Andrew Lan

Point Processes

A point process, the discretization of a Poisson process, is characterized by a rate function \( \lambda(t) \) that models the probability that an event will happen in an infinitesimal time window \( dt \). The rate function at time \( t \) is given by: $$ \lambda(t) = \mathbb{P}(\text{event in} [t, t + dt]) = \lim_{dt \rightarrow 0} \frac{N(t + dt) - N(t)}{dt}$$. Where, \( N(t)\) denotes the number of events up to time $t$. Assuming the time period of interest is \( [0, T)\), the likelihood of a series of events at times \( t_1, \dots, t_N \le T\) is given b: $$ \mathcal{L}(\{ t_i\}^{N}_{i=1}) = (\prod_{i=1}^{N} \lambda(t_i)) \exp^{- \int_{0}^{T} \lambda(\tau) d\tau}$$ Here, we care about rate functions that are affected by excitations of past events (e.g., forum posts in the same thread). This leads us to Hawkes processes, which characterize the rate function at time \( t\) given a series of past events at \( t_1, \dots, t_N \le t \) as: $$ \lambda(t) = \mu + a \sum_{i=1}^{N'} \kappa(t - t_i)$$