[Numpy-discussion] automatic differentiation with PyAutoDiff
Thu Jun 14 17:59:50 CDT 2012
> Of course, maybe you were pointing out that if your derivative
> calculation depends in some intrinsic way on the topology of some
> graph, then your best bet is to have an automatic way to recompute it
> from scratch for each new graph you see. In that case, fair enough!
That is indeed what I had in mind. In neural networks, Markov random
fields, Bayesian networks, graph regularization etc it is something
that has to be dealt with all the time.
> Right, and what I want is to do those correctness checks once, and
> then save the validated derivative function somewhere and know that it
> won't break the next time I upgrade some library or make some
> seemingly-irrelevant change to the original code.
What I was getting at is: even if it is not feasible to get a pretty
printed python output, the byte code can still be validated (somewhat)
with a few numeric sanity checks. So, yes the derivatives
needn't/shouldn't be re-computed in runtime all the time and an API
that that returns even some opaque but computable representation of
the derivative that can be validated and then "frozen" would be
I think one can go further and formally prove the correctness of the
derivative computing engine. I dont know if anyone has done it. Maybe
Theano does it. Should be possible for a statically typed sublanguage.
More information about the NumPy-Discussion