Keras validation loss and metric inconsistent - validation

Question

RNN model used for regression, cf. Chollet, Deep Learning with Python, 6.3.1 A temperature-forecasting problem
In this example I used random data, both regressors and regressand
I have used the mean absolute error, both as loss function and as a metric
I do not understand the values I get for val_loss and val_mean_absolute_error. Neither of them make sense to me.
code:
import tensorflow as tf
import numpy as np
from keras.models import Sequential
from keras import layers
from keras.optimizers import Adam
import keras
I use random input data:
data_np = np.random.rand(6400,10)
target_np = np.random.rand(6400,)
Normalizing the data:
mean1 = data_np[:].mean(axis=0)
std1 = data_np[:].std(axis=0)
data_np -= mean1
data_np /= std1
mean2 = target_np.mean(axis=0)
std2 = target_np.std(axis=0)
target_np -= mean2
target_np /= std2
Create RNN input with lookback:
lookback = 7
train_data = np.array([data_np[(i-lookback):i,:] for i in range(lookback,len(data_np))])
target_data = target_np[lookback:len(data_np)]
And then set up a simple RNN:
model = Sequential()
model.add(layers.SimpleRNN(64,
activation = 'relu',
return_sequences=False,
input_shape=(train_data.shape[1], train_data.shape[2])))
model.add(layers.Dense(1))
opt = Adam(learning_rate=0.1)
mae = tensorflow.keras.losses.MeanAbsoluteError()
model.compile(optimizer=opt, loss=mae, metrics=[mae])
history = model.fit(train_data, target_data,
steps_per_epoch=round(0.7*len(train_data))//64,
epochs=10,
shuffle=False,
validation_split=0.3,
validation_steps = round(0.3*len(train_data))//64,
verbose=1)
The output then looks like this:
Train on 3495 samples, validate on 1498 samples Epoch 1/10 54/54
[==============================] - 2s 38ms/step - loss: 0.7955 -
mean_absolute_error: 0.7955 - val_loss: 0.0428 -
val_mean_absolute_error: 22.6301 Epoch 2/10 54/54
[==============================] - 2s 30ms/step - loss: 0.7152 -
mean_absolute_error: 0.7152 - val_loss: 0.0421 -
val_mean_absolute_error: 22.2968
I would expect val_loss and val_mean_absolute_error to be the same. Moreover, the levels don't make much sense either. After 10 epochs, I get
Epoch 10/10 54/54 [==============================] - 2s 32ms/step -
loss: 0.7747 - mean_absolute_error: 0.7747 - val_loss: 0.0409 -
val_mean_absolute_error: 21.6337
If I calculate the mean absolute error manually:
N=len(data_np)
val_data = np.array([data_np[(i-lookback):i,:] for i in range(round(0.7*N),N)])
val_target = target_np[round(0.7*N):N]
model_output = model.predict(val_data)
model_output=[output[0] for output in model_output]
np.mean(abs(model_output-val_target))
0.940300949276649
This looks like a result that one could expect. However, it is not even close to either val_loss or val_mean_absolute_error. What is wrong here?

Answer 1

OK. I managed to solve the issue by consistently using tensorflow.keras. So, replacing
import tensorflow as tf
import numpy as np
from keras.models import Sequential
from keras import layers
from keras.optimizers import Adam
import keras
with
import tensorflow as tf
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras import layers
from tensorflow.keras.optimizers import Adam
import tensorflow.keras
(and corrected a couple of details in the original question)