# MXNet设计笔记：深度学习的编程模式比较

2018-03-13 11:40:20         来源：Hunt Counter

【编者按】继xgboost,cxxnet,minerva之后，DMLC在9月29日发布了新的Project:dmlc/MXNet，MXNet是cxxnet的进化，在设计上经过成熟的思考，文档也很清楚。尤为难得的是，MXNet开发团队把设计笔记也做了分享。笔记的思想不局限于MXNet，也不局限于深度学习，无论对深度学习初学入门还是对高阶提升，都具有很好的参考价值。本文是第一篇设计笔记的译文，深入讨论了不同深度学习库的接口对深度学习编程的性能和灵活性产生的影响。

## 符号式编程vs命令式编程

```import numpy as np
a = np.ones(10)
b = np.ones(10) * 2
c = b * a
d = c + 1```

```A = Variable('A')
B = Variable('B')
C = B * A
D = C + Constant(1)
# compiles the function
f = compile(D)
d = f(A=np.ones(10), B=np.ones(10)*2)```

### 命令式程序更加灵活

```a = 2
b = a + 1
d = np.zeros(10)
for i in range(d):
d += np.zeros(10)```

### 符号式程序更高效

```import numpy as np
a = np.ones(10)
b = np.ones(10) * 2
c = b * a
d = c + 1
...```

### Backprop和AutoDiff的案例分析

```class array(object) :
"""Simple Array object that support autodiff."""
def __init__(self, value, name=None):
self.value = value
if name:
self.grad = lambda g : {name : g}

assert isinstance(other, int)
ret = array(self.value + other)
return ret

def __mul__(self, other):
assert isinstance(other, array)
ret = array(self.value * other.value)
return x
return ret

# some examples
a = array(1, 'a')
b = array(2, 'b')
c = b * a
d = c + 1
print d.value
# Results
# 3
# {'a': 2, 'b': 1}```

```A = Variable('A')
B = Variable('B')
C = B * A
D = C + Constant(1)
f = compile([gA, gB])

```with context.NoGradient():
a = array(1, 'a')
b = array(2, 'b')
c = b * a
d = c + 1```

### 模型检查点

```A = Variable('A')
B = Variable('B')
C = B * A
D = C + Constant(1)
D.save('mygraph')
...
f = compile([D2])
# more operations
...```

## 大操作vs小操作

### 更小的操作更灵活

`sigmoid(x)=1.0/(1.0+exp(-x))`

`SigmoidLayer(x) = EWiseDivisionLayer(1.0, AddScalarLayer(ExpLayer(-x), 1.0))`

### 大操作更高效

`SigmoidLayer(x)=EWiseDivisionLayer(1.0,AddScalarLayer(ExpLayer(-x),1.0))`

## 混合各种风格

### 符号式和命令式程序

```A = Variable('A')
B = Variable('B')
C = B * A
D = C + Constant(1)
# compiles the function
f = compile(D)
d = f(A=np.ones(10), B=np.ones(10)*2)
d = d + 1.0```