The model_lib_tf2.py file contains all the training and evaluation loops.
In the function:
def eager_train_step(detection_model,
features,
labels,
unpad_groundtruth_tensors,
optimizer,
learning_rate,
add_regularization_loss=True,
clip_gradients_value=None,
global_step=None,
num_replicas=1.0):
We can see that every training iteration it saves a few training dataset images.
tf.compat.v2.summary.image(
name='train_input_images',
step=global_step,
data=features[fields.InputDataFields.image],
max_outputs=5)
There are three problems with this:
1. It takes A LOT of space
2. It actually slows up training
3. Images look saturated.
We can fix this easily by replacing the above snippet with this:
if global_step % 100 == 0:
# --- get images and normalize them
images_normalized = \
(features[fields.InputDataFields.image] + 128.0) / 255.0
tf.compat.v2.summary.image(
name='train_input_images',
step=global_step,
data=images_normalized,
max_outputs=5)