Convert batch normalization from Tensorlayer tf1.x to TF2 keras

Issue

Trying to convert the batch normalization layer from Tensorlayer version 1.11.1 to Tensorflow 2 and getting different outputs from this layer during inference using the same pretrained model.

Tensorlayer 1.11.1

tensorlayer.layers.BatchNormLayer(network, is_train=False, name="batch_norm")

Tensorflow 2.8.0

tf.keras.layers.BatchNormalization(trainable=False, momentum=0.9, axis=3, epsilon=1e-05, gamma_initializer=tf.random_normal_initializer(mean=1.0, stdev=0.002))(network)

What am I missing to get the BatchNorm output to match?

Solution

The TF1 model I had was in NPZ format.
The weights from Tensorlayer are saved in the order of:
beta, gamma, moving mean, variance.
In TF2, the batch norm layer is in the order of:
gamma, beta, moving mean, variance.

If the order of the weights for beta and gamma are reversed when moving from TF1 to TF2 it solves the issue.

Answered By – ro5423

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0

Leave a Reply

(*) Required, Your email will not be published