Don't Learn here. I think Everything here is exact opposite of what you do in a real job. I am not a profession AI guy. I am a noob.
Forward pass was simple.
I need to pass the input value of one layer to another layers.
But Backpropagation has multiple steps.
I think backpropagation is when you make your network aware of loss. By Aware I mean updating value based on loss of network. I just realized this today(17-03-2024). So Below is the methods that I wrote to train and get loss of network.
def train(self,train_X,train_Y): # pass the train_X to Network to get loss of network or the error in prediction. self.inputL_out = self.inputL.forward(train_X) for l in range(0,len(self.neuronsLayers)): if l == 0: self.neuronsLayers[l].forward(self.inputL_out) else: self.neuronsLayers[l].forward(self.neuronsLayers[l-1].f_out) if l == len(self.neuronsLayers) -1: self.outL_f_out = self.outL.forward(self.neuronsLayers[l].f_out) #self.outL_f_out is output of network. # now calculate the loss. # I am calculating two type for losses. 1. loss on actual answer 2. Loss on wrong answer. losses = [] for i in range(0,self.out_dim): if train_Y != i: if abs(self.outL_f_out[i]) > 0.6: losses.append(abs(self.outL_f_out[i])-0.6) #prediction for other digits other then actual answer should be less then 0.6 else: losses.append(0) else: losses.append(0.99 - abs(self.outL_f_out[train_Y])) #loss for actual answer. print(losses) print(self.outL_f_out) Until today I was stuck on derivative and how to update weights.
After wasting two days(2 hours per day) I realized I need to update weight based on loss.
But still how do I update weight & by how much.
And why my network in predicting negative probability.(wrong choice of activation function I think.)
I am weak at math
Top comments (0)