How can I modify ModelCheckPoint in keras to monitor both val_acc and val_loss and save accordingly the best model?

Issue

ModelCheckPoint gives options to save both for val_Acc and val_loss separately.
I want to modify this in a way so that if val_acc is improving -> save model. if val_acc is equal to previous best val_acc then check for val_loss, if val_loss is less than previous best val_loss then save the model.

    if val_acc(epoch i)> best_val_acc:
        save model
    else if val_acc(epoch i) == best_val_acc:
        if val_loss(epoch i) < best_val_loss:
           save model
        else
           do not save model

Solution

You can just add two callbacks:

callbacks = [ModelCheckpoint(filepathAcc, monitor='val_acc', ...),
             ModelCheckpoint(filepathLoss, monitor='val_loss', ...)]

model.fit(......., callbacks=callbacks)

Using custom callbacks

You can do anything you want in a LambdaCallback(on_epoch_end=saveModel).

best_val_acc = 0
best_val_loss = sys.float_info.max 

def saveModel(epoch,logs):
    val_acc = logs['val_acc']
    val_loss = logs['val_loss']

    if val_acc > best_val_acc:
        best_val_acc = val_acc
        model.save(...)
    elif val_acc == best_val_acc:
        if val_loss < best_val_loss:
            best_val_loss=val_loss
            model.save(...)
    
callbacks = [LambdaCallback(on_epoch_end=saveModel)]

But this is nothing different from a single ModelCheckpoint with val_acc. You won’t really be getting identical accuracies unless you’re using very few samples, or you have a custom accuracy that doesn’t vary much.

Answered By – Daniel Möller

This Answer collected from stackoverflow, is licensed under cc by-sa 2.5 , cc by-sa 3.0 and cc by-sa 4.0