Posts

The Euler-Lagrange Equation In the previous post, we introduced the variational calculus and took the first steps towards deriving necessary and sufficient conditions for establishing local extrema of functionals. In this post, we will build on our previous work and derive the Euler-Lagrange equation, then subsequently apply it to some physics problems.

In introductory calculus, we learn to take derivatives of functions of a single variable. In vector calculus, we learn to do the same for vector-valued functions, although here the analysis becomes more complex as there is no longer a single, canonical notion of a derivative.

An explanation using Sard’s Theorem.

If you’ve read my article about the REINFORCE algorithm, you should be familiar with the update that’s typically used in policy gradient methods. $$\nabla_{\theta} J(\theta) = \mathbb{E}_{\tau \sim \pi_{\theta}(\tau)} \left[ \left( \sum_{t} \nabla_\theta \log{\pi_\theta}(a_t \mid s_t)\right) \left(\sum_t r(s_t, a_t)\right)\right]$$

Identifying variants in genomic sequences and an introduction to Google’s DeepVariant model.

The setup for the general reinforcement learning problem is as follows. We’re given an environment $\mathcal{E}$ with a specified state space $\mathcal{S}$ and an action space $\mathcal{A}$ giving the allowable actions in each of those states.

Any continuous function can be approximated to an arbitrary degree of accuracy by some neural network.