Skip to content

Conversation

@QiJune
Copy link
Member

@QiJune QiJune commented Jan 10, 2018

Memory optimization result(by bytes):

Model Before After Saving
VGG 1729499136 1132920832 34.5%
Resnet 1277345792 666173440 47.8%

Correctness check in training loss seems good.

  • vgg

loss:[ 2.52220297]
loss:[ 2.66482925]
loss:[ 2.6781323]
loss:[ 2.75176835]
loss:[ 2.91183567]
loss:[ 2.63675451]
loss:[ 2.71298695]
loss:[ 2.71268129]
loss:[ 2.51224518]
loss:[ 2.57749677]
loss:[ 2.58871031]
loss:[ 2.62577391]
loss:[ 2.46702719]
loss:[ 2.47233582]
loss:[ 2.47583246]
loss:[ 2.43947959]
loss:[ 2.50619984]
loss:[ 2.48155594]
loss:[ 2.37081432]
loss:[ 2.5065763]
loss:[ 2.495049]

  • resnet

loss:[ 3.23119116]
loss:[ 3.46338725]
loss:[ 2.85568714]
loss:[ 2.86154795]
loss:[ 2.74208355]
loss:[ 2.61007118]
loss:[ 2.38463306]
loss:[ 2.55949521]
loss:[ 2.4749732]
loss:[ 2.32410693]
loss:[ 2.17808747]
loss:[ 2.14268446]
loss:[ 2.09680271]
loss:[ 2.13581395]
loss:[ 2.00303197]
loss:[ 2.09924126]
loss:[ 2.04527116]
loss:[ 1.99301922]
loss:[ 1.99249315]
loss:[ 1.92059994]
loss:[ 1.88696837]

@tonyyang-svail
Copy link

@QiJune Have you tested it on GPU? What is the largest batch size we can have?

@QiJune
Copy link
Member Author

QiJune commented Jan 10, 2018

@tonyyang-svail Not yet. I am testing the demo on CPU, I will test on GPU in next step.

@QiJune
Copy link
Member Author

QiJune commented Jan 11, 2018

The clean implementation version is in #7443 , so close this PR

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

2 participants