My loss is validation is changed suddenly by a huge amount and I don't know why?
Do you have any suggestions to fix this problem?
I used for training 9533 images and for validation 2724 images
Model: "model"
_________________________________________________________________
Layer (type) Output Shape Param #
=================================================================
input_1 (InputLayer) [(None, 224, 224, 3)] 0
_________________________________________________________________
sequential (Sequential) (None, 7, 7, 1024) 56487040
_________________________________________________________________
flatten (Flatten) (None, 50176) 0
_________________________________________________________________
dense (Dense) (None, 512) 25690624
_________________________________________________________________
batch_normalization (BatchNo (None, 512) 2048
_________________________________________________________________
dropout_2 (Dropout) (None, 512) 0
_________________________________________________________________
batch_normalization_1 (Batch (None, 512) 2048
_________________________________________________________________
dense_2 (Dense) (None, 1) 513
=================================================================
Total params: 82,182,273
Trainable params: 82,180,225
Non-trainable params: 2,048
from tensorflow.keras.callbacks import EarlyStopping
es = EarlyStopping(monitor='val_acc', mode='max', verbose=1, patience=60)
model.compile(optimizer=tf.keras.optimizers.Adam(learning_rate=0.0001),
loss = 'binary_crossentropy', metrics = ['acc'])
history =model.fit(train_set, epochs=150,steps_per_epoch=149, validation_data=test_set, validation_steps=43,callbacks=[es])
2022-12-24 09:44:53.627442: I tensorflow/compiler/mlir/mlir_graph_optimization_pass.cc:185] None of the MLIR Optimization Passes are enabled (registered 2)
Epoch 1/150
2022-12-24 09:44:56.776701: I tensorflow/stream_executor/cuda/cuda_dnn.cc:369] Loaded cuDNN version 8005
149/149 [==============================] - 316s 2s/step - loss: 0.6654 - acc: 0.6526 - val_loss: 0.7226 - val_acc: 0.4596
Epoch 2/150
149/149 [==============================] - 214s 1s/step - loss: 0.5438 - acc: 0.7849 - val_loss: 0.7408 - val_acc: 0.8513
Epoch 3/150
149/149 [==============================] - 213s 1s/step - loss: 0.4704 - acc: 0.8266 - val_loss: 0.4078 - val_acc: 0.8590
Epoch 4/150
149/149 [==============================] - 212s 1s/step - loss: 0.4360 - acc: 0.8393 - val_loss: 0.7902 - val_acc: 0.8513
Epoch 5/150
149/149 [==============================] - 213s 1s/step - loss: 0.4080 - acc: 0.8526 - val_loss: 4.9581 - val_acc: 0.1858
Epoch 6/150
149/149 [==============================] - 213s 1s/step - loss: 0.4054 - acc: 0.8510 - val_loss: 0.7537 - val_acc: 0.4725
Epoch 7/150
149/149 [==============================] - 212s 1s/step - loss: 0.3959 - acc: 0.8573 - val_loss: 1.7509 - val_acc: 0.2034
Epoch 8/150
149/149 [==============================] - 212s 1s/step - loss: 0.3922 - acc: 0.8579 - val_loss: 2.4338 - val_acc: 0.5808
Epoch 9/150
149/149 [==============================] - 214s 1s/step - loss: 0.3901 - acc: 0.8621 - val_loss: 0.4251 - val_acc: 0.8656
Epoch 10/150
149/149 [==============================] - 213s 1s/step - loss: 0.4003 - acc: 0.8596 - val_loss: 323.0921 - val_acc: 0.8374
Epoch 11/150
149/149 [==============================] - 213s 1s/step - loss: 0.4406 - acc: 0.8475 - val_loss: 2.6854 - val_acc: 0.2048
Epoch 12/150
149/149 [==============================] - 213s 1s/step - loss: 0.4364 - acc: 0.8478 - val_loss: 0.4173 - val_acc: 0.8502
Epoch 13/150
149/149 [==============================] - 213s 1s/step - loss: 0.4328 - acc: 0.8465 - val_loss: 0.7263 - val_acc: 0.8513
Epoch 14/150
149/149 [==============================] - 215s 1s/step - loss: 0.4331 - acc: 0.8479 - val_loss: 0.7787 - val_acc: 0.8484
Epoch 15/150
149/149 [==============================] - 216s 1s/step - loss: 0.4294 - acc: 0.8489 - val_loss: 0.6814 - val_acc: 0.8488
Epoch 16/150
149/149 [==============================] - 214s 1s/step - loss: 0.4269 - acc: 0.8497 - val_loss: 0.5430 - val_acc: 0.8488
Epoch 17/150
149/149 [==============================] - 213s 1s/step - loss: 0.4233 - acc: 0.8495 - val_loss: 1.3373 - val_acc: 0.8484
Epoch 18/150
149/149 [==============================] - 214s 1s/step - loss: 0.4209 - acc: 0.8491 - val_loss: 0.4197 - val_acc: 0.8502
Epoch 19/150
149/149 [==============================] - 213s 1s/step - loss: 0.4127 - acc: 0.8495 - val_loss: 0.5153 - val_acc: 0.8480
Epoch 20/150
149/149 [==============================] - 214s 1s/step - loss: 0.4207 - acc: 0.8504 - val_loss: 1.9701 - val_acc: 0.1773
Epoch 21/150
149/149 [==============================] - 213s 1s/step - loss: 0.4253 - acc: 0.8479 - val_loss: 0.4600 - val_acc: 0.8488
Epoch 22/150
149/149 [==============================] - 214s 1s/step - loss: 0.4256 - acc: 0.8482 - val_loss: 0.4191 - val_acc: 0.8510
Epoch 23/150
149/149 [==============================] - 213s 1s/step - loss: 0.4227 - acc: 0.8474 - val_loss: 0.4114 - val_acc: 0.8513
Epoch 24/150
149/149 [==============================] - 212s 1s/step - loss: 0.4227 - acc: 0.8474 - val_loss: 0.4148 - val_acc: 0.8506
Epoch 25/150
149/149 [==============================] - 214s 1s/step - loss: 0.4187 - acc: 0.8484 - val_loss: 0.4060 - val_acc: 0.8513
Epoch 26/150
149/149 [==============================] - 213s 1s/step - loss: 0.4158 - acc: 0.8461 - val_loss: 11608.2793 - val_acc: 0.8513
Epoch 27/150
149/149 [==============================] - 214s 1s/step - loss: 0.4072 - acc: 0.8507 - val_loss: 0.5867 - val_acc: 0.8510
Epoch 28/150
149/149 [==============================] - 213s 1s/step - loss: 0.3938 - acc: 0.8561 - val_loss: 0.5836 - val_acc: 0.8513
Epoch 29/150
149/149 [==============================] - 213s 1s/step - loss: 0.3926 - acc: 0.8566 - val_loss: 0.4901 - val_acc: 0.8499
Epoch 30/150
149/149 [==============================] - 214s 1s/step - loss: 0.3847 - acc: 0.8581 - val_loss: 0.4888 - val_acc: 0.8528
Epoch 31/150
149/149 [==============================] - 213s 1s/step - loss: 0.4060 - acc: 0.8508 - val_loss: 0.4505 - val_acc: 0.8513
Epoch 32/150
149/149 [==============================] - 213s 1s/step - loss: 0.3811 - acc: 0.8603 - val_loss: 0.4195 - val_acc: 0.8671
Epoch 33/150
149/149 [==============================] - 214s 1s/step - loss: 0.3852 - acc: 0.8595 - val_loss: 0.5208 - val_acc: 0.7988
Epoch 34/150
149/149 [==============================] - 213s 1s/step - loss: 0.3798 - acc: 0.8607 - val_loss: 17.5676 - val_acc: 0.8634
Epoch 35/150
149/149 [==============================] - 213s 1s/step - loss: 0.3774 - acc: 0.8618 - val_loss: 0.3762 - val_acc: 0.8704
Epoch 36/150
149/149 [==============================] - 212s 1s/step - loss: 0.3773 - acc: 0.8613 - val_loss: 0.7999 - val_acc: 0.8506
Epoch 37/150
149/149 [==============================] - 213s 1s/step - loss: 0.3753 - acc: 0.8625 - val_loss: 386.2837 - val_acc: 0.8649
Epoch 38/150
149/149 [==============================] - 213s 1s/step - loss: 0.3711 - acc: 0.8675 - val_loss: 28.4553 - val_acc: 0.8330
Epoch 39/150
149/149 [==============================] - 212s 1s/step - loss: 0.3770 - acc: 0.8622 - val_loss: 1.6037 - val_acc: 0.8568
Epoch 40/150
149/149 [==============================] - 213s 1s/step - loss: 0.3690 - acc: 0.8659 - val_loss: 1.7776 - val_acc: 0.8513
Epoch 41/150
149/149 [==============================] - 213s 1s/step - loss: 0.3710 - acc: 0.8653 - val_loss: 153.6027 - val_acc: 0.8711
Epoch 42/150
149/149 [==============================] - 213s 1s/step - loss: 0.3649 - acc: 0.8667 - val_loss: 22623.5195 - val_acc: 0.8458
Epoch 43/150
149/149 [==============================] - 212s 1s/step - loss: 0.4131 - acc: 0.8556 - val_loss: 0.5640 - val_acc: 0.8502
Epoch 44/150
149/149 [==============================] - 212s 1s/step - loss: 0.4263 - acc: 0.8506 - val_loss: 1.8865 - val_acc: 0.8480
Epoch 45/150
149/149 [==============================] - 212s 1s/step - loss: 0.4274 - acc: 0.8505 - val_loss: 2032.7307 - val_acc: 0.8462
Epoch 46/150
149/149 [==============================] - 215s 1s/step - loss: 0.4274 - acc: 0.8502 - val_loss: 634.5857 - val_acc: 0.8477
Epoch 47/150
149/149 [==============================] - 212s 1s/step - loss: 0.4133 - acc: 0.8540 - val_loss: 2.7397 - val_acc: 0.1806
Epoch 48/150
149/149 [==============================] - 212s 1s/step - loss: 0.3606 - acc: 0.8697 - val_loss: 0.9382 - val_acc: 0.8550
Epoch 49/150
149/149 [==============================] - 212s 1s/step - loss: 0.3639 - acc: 0.8689 - val_loss: 17156.6055 - val_acc: 0.8623
Epoch 50/150
149/149 [==============================] - 212s 1s/step - loss: 0.3661 - acc: 0.8661 - val_loss: 2815.0986 - val_acc: 0.8554
Epoch 51/150
149/149 [==============================] - 213s 1s/step - loss: 0.3576 - acc: 0.8714 - val_loss: 0.7997 - val_acc: 0.8521
Epoch 52/150
149/149 [==============================] - 212s 1s/step - loss: 0.3673 - acc: 0.8706 - val_loss: 0.3928 - val_acc: 0.8620
Epoch 53/150
149/149 [==============================] - 212s 1s/step - loss: 0.3525 - acc: 0.8717 - val_loss: 0.4565 - val_acc: 0.8605
Epoch 54/150
149/149 [==============================] - 213s 1s/step - loss: 0.3524 - acc: 0.8750 - val_loss: 2680.2422 - val_acc: 0.8667
Epoch 55/150
149/149 [==============================] - 211s 1s/step - loss: 0.3575 - acc: 0.8735 - val_loss: 372.1012 - val_acc: 0.8535
Epoch 56/150
149/149 [==============================] - 212s 1s/step - loss: 0.3510 - acc: 0.8720 - val_loss: 6.8838 - val_acc: 0.8678
Epoch 57/150
149/149 [==============================] - 212s 1s/step - loss: 0.3489 - acc: 0.8738 - val_loss: 0.5691 - val_acc: 0.8554
Epoch 58/150
149/149 [==============================] - 212s 1s/step - loss: 0.3460 - acc: 0.8772 - val_loss: 0.3476 - val_acc: 0.8759
Epoch 59/150
149/149 [==============================] - 212s 1s/step - loss: 0.3529 - acc: 0.8716 - val_loss: 0.5015 - val_acc: 0.8557
Epoch 60/150
149/149 [==============================] - 214s 1s/step - loss: 0.3531 - acc: 0.8730 - val_loss: 0.3856 - val_acc: 0.8645
Epoch 61/150
149/149 [==============================] - 212s 1s/step - loss: 0.3405 - acc: 0.8773 - val_loss: 0.3940 - val_acc: 0.8767
Epoch 62/150
149/149 [==============================] - 212s 1s/step - loss: 0.3440 - acc: 0.8774 - val_loss: 0.3669 - val_acc: 0.8752
Epoch 63/150
149/149 [==============================] - 213s 1s/step - loss: 0.3418 - acc: 0.8776 - val_loss: 900.9280 - val_acc: 0.1935
Epoch 64/150
149/149 [==============================] - 213s 1s/step - loss: 0.3452 - acc: 0.8760 - val_loss: 175.3406 - val_acc: 0.2651
Epoch 65/150
149/149 [==============================] - 211s 1s/step - loss: 0.3447 - acc: 0.8766 - val_loss: 485.4781 - val_acc: 0.8524
Epoch 66/150
149/149 [==============================] - 212s 1s/step - loss: 0.3414 - acc: 0.8796 - val_loss: 0.5346 - val_acc: 0.8550
Epoch 67/150
149/149 [==============================] - 213s 1s/step - loss: 0.3478 - acc: 0.8745 - val_loss: 4.4924 - val_acc: 0.3025
Epoch 68/150
149/149 [==============================] - 213s 1s/step - loss: 0.3410 - acc: 0.8775 - val_loss: 0.3990 - val_acc: 0.8763
Epoch 69/150
149/149 [==============================] - 213s 1s/step - loss: 0.3417 - acc: 0.8775 - val_loss: 0.4408 - val_acc: 0.8616
Epoch 70/150
149/149 [==============================] - 212s 1s/step - loss: 0.3440 - acc: 0.8755 - val_loss: 0.7289 - val_acc: 0.8814
Epoch 71/150
149/149 [==============================] - 212s 1s/step - loss: 0.3379 - acc: 0.8787 - val_loss: 0.4772 - val_acc: 0.8660
Epoch 72/150
149/149 [==============================] - 213s 1s/step - loss: 0.3394 - acc: 0.8780 - val_loss: 26.1834 - val_acc: 0.7192
Epoch 73/150
149/149 [==============================] - 213s 1s/step - loss: 0.3356 - acc: 0.8806 - val_loss: 0.3749 - val_acc: 0.8781
Epoch 74/150
149/149 [==============================] - 212s 1s/step - loss: 0.3378 - acc: 0.8791 - val_loss: 0.4133 - val_acc: 0.8741
Epoch 75/150
149/149 [==============================] - 213s 1s/step - loss: 0.3365 - acc: 0.8775 - val_loss: 0.6339 - val_acc: 0.8517
Epoch 76/150
149/149 [==============================] - 212s 1s/step - loss: 0.3417 - acc: 0.8801 - val_loss: 1.1401 - val_acc: 0.3363
Epoch 77/150
149/149 [==============================] - 213s 1s/step - loss: 0.3395 - acc: 0.8772 - val_loss: 1.0921 - val_acc: 0.8767
Epoch 78/150
149/149 [==============================] - 216s 1s/step - loss: 0.3322 - acc: 0.8792 - val_loss: 0.3860 - val_acc: 0.8796
Epoch 79/150
149/149 [==============================] - 218s 1s/step - loss: 0.3314 - acc: 0.8797 - val_loss: 0.4313 - val_acc: 0.8634
Epoch 80/150
149/149 [==============================] - 214s 1s/step - loss: 0.3271 - acc: 0.8839 - val_loss: 0.4355 - val_acc: 0.8513
Epoch 81/150
149/149 [==============================] - 212s 1s/step - loss: 0.3341 - acc: 0.8807 - val_loss: 0.5978 - val_acc: 0.7640
Epoch 82/150
149/149 [==============================] - 213s 1s/step - loss: 0.3352 - acc: 0.8785 - val_loss: 0.3702 - val_acc: 0.8737
Epoch 83/150
149/149 [==============================] - 212s 1s/step - loss: 0.3474 - acc: 0.8766 - val_loss: 1867.5101 - val_acc: 0.8557
Epoch 84/150
149/149 [==============================] - 212s 1s/step - loss: 0.3607 - acc: 0.8706 - val_loss: 0.3549 - val_acc: 0.8756
Epoch 85/150
149/149 [==============================] - 213s 1s/step - loss: 0.3415 - acc: 0.8761 - val_loss: 0.3836 - val_acc: 0.8840
Epoch 86/150
149/149 [==============================] - 211s 1s/step - loss: 0.3442 - acc: 0.8764 - val_loss: 0.3777 - val_acc: 0.8682
Epoch 87/150
149/149 [==============================] - 212s 1s/step - loss: 0.3360 - acc: 0.8802 - val_loss: 5225.8945 - val_acc: 0.8612
Epoch 88/150
149/149 [==============================] - 211s 1s/step - loss: 0.3419 - acc: 0.8763 - val_loss: 0.4597 - val_acc: 0.8638
Epoch 89/150
149/149 [==============================] - 212s 1s/step - loss: 0.3334 - acc: 0.8814 - val_loss: 0.5231 - val_acc: 0.8627
Epoch 90/150
149/149 [==============================] - 212s 1s/step - loss: 0.3298 - acc: 0.8813 - val_loss: 0.3845 - val_acc: 0.8807
Epoch 91/150
149/149 [==============================] - 212s 1s/step - loss: 0.3310 - acc: 0.8818 - val_loss: 0.3812 - val_acc: 0.8730
Epoch 92/150
149/149 [==============================] - 212s 1s/step - loss: 0.3281 - acc: 0.8826 - val_loss: 0.3276 - val_acc: 0.8880
Epoch 93/150
149/149 [==============================] - 212s 1s/step - loss: 0.3252 - acc: 0.8857 - val_loss: 0.3554 - val_acc: 0.8833
Epoch 94/150
149/149 [==============================] - 214s 1s/step - loss: 0.3286 - acc: 0.8845 - val_loss: 0.3449 - val_acc: 0.8811
Epoch 95/150
149/149 [==============================] - 214s 1s/step - loss: 0.3224 - acc: 0.8864 - val_loss: 0.4516 - val_acc: 0.8733
Epoch 96/150
149/149 [==============================] - 213s 1s/step - loss: 0.3222 - acc: 0.8871 - val_loss: 0.3827 - val_acc: 0.8737
Epoch 97/150
149/149 [==============================] - 214s 1s/step - loss: 0.3152 - acc: 0.8893 - val_loss: 0.3400 - val_acc: 0.8807
Epoch 98/150
149/149 [==============================] - 213s 1s/step - loss: 0.3244 - acc: 0.8865 - val_loss: 0.4585 - val_acc: 0.8825
Epoch 99/150
149/149 [==============================] - 213s 1s/step - loss: 0.3189 - acc: 0.8870 - val_loss: 0.3198 - val_acc: 0.8866
Epoch 100/150
149/149 [==============================] - 214s 1s/step - loss: 0.3181 - acc: 0.8875 - val_loss: 0.4341 - val_acc: 0.8763
Epoch 101/150
149/149 [==============================] - 213s 1s/step - loss: 0.3169 - acc: 0.8887 - val_loss: 3.6571 - val_acc: 0.8807
Epoch 102/150
149/149 [==============================] - 215s 1s/step - loss: 0.3134 - acc: 0.8886 - val_loss: 0.3532 - val_acc: 0.8888
Epoch 103/150
149/149 [==============================] - 215s 1s/step - loss: 0.3130 - acc: 0.8903 - val_loss: 3.1462 - val_acc: 0.8939
Epoch 104/150
149/149 [==============================] - 213s 1s/step - loss: 0.3118 - acc: 0.8916 - val_loss: 557.1219 - val_acc: 0.8748
Epoch 105/150
149/149 [==============================] - 213s 1s/step - loss: 0.3118 - acc: 0.8910 - val_loss: 0.4914 - val_acc: 0.8822
Epoch 106/150
149/149 [==============================] - 213s 1s/step - loss: 0.3161 - acc: 0.8870 - val_loss: 0.5733 - val_acc: 0.8689
Epoch 107/150
149/149 [==============================] - 213s 1s/step - loss: 0.3112 - acc: 0.8908 - val_loss: 0.3187 - val_acc: 0.8968
Epoch 108/150
149/149 [==============================] - 214s 1s/step - loss: 0.3093 - acc: 0.8913 - val_loss: 6.7099 - val_acc: 0.8906
Epoch 109/150
149/149 [==============================] - 212s 1s/step - loss: 0.3027 - acc: 0.8957 - val_loss: 0.3419 - val_acc: 0.8910
Epoch 110/150
149/149 [==============================] - 214s 1s/step - loss: 0.3038 - acc: 0.8962 - val_loss: 0.4641 - val_acc: 0.8814
Epoch 111/150
149/149 [==============================] - 214s 1s/step - loss: 0.3020 - acc: 0.8955 - val_loss: 1886.6735 - val_acc: 0.8899
Epoch 112/150
149/149 [==============================] - 213s 1s/step - loss: 0.3024 - acc: 0.8930 - val_loss: 0.3290 - val_acc: 0.8891
Epoch 113/150
149/149 [==============================] - 213s 1s/step - loss: 0.2965 - acc: 0.8954 - val_loss: 673.0439 - val_acc: 0.8906
Epoch 114/150
149/149 [==============================] - 213s 1s/step - loss: 0.3046 - acc: 0.8939 - val_loss: 0.3333 - val_acc: 0.8895
Epoch 115/150
149/149 [==============================] - 213s 1s/step - loss: 0.2936 - acc: 0.8988 - val_loss: 918.0665 - val_acc: 0.8822
Epoch 116/150
149/149 [==============================] - 214s 1s/step - loss: 0.2975 - acc: 0.8954 - val_loss: 417.7471 - val_acc: 0.8866
Epoch 117/150
149/149 [==============================] - 213s 1s/step - loss: 0.2984 - acc: 0.8984 - val_loss: 411.4460 - val_acc: 0.8833
Epoch 118/150
149/149 [==============================] - 213s 1s/step - loss: 0.2989 - acc: 0.8965 - val_loss: 0.7020 - val_acc: 0.8851
Epoch 119/150
149/149 [==============================] - 214s 1s/step - loss: 0.2909 - acc: 0.8991 - val_loss: 0.4214 - val_acc: 0.8935
Epoch 120/150
149/149 [==============================] - 214s 1s/step - loss: 0.2950 - acc: 0.8975 - val_loss: 0.5487 - val_acc: 0.8965
Epoch 121/150
149/149 [==============================] - 214s 1s/step - loss: 0.2904 - acc: 0.8997 - val_loss: 0.5133 - val_acc: 0.8704
Epoch 122/150
149/149 [==============================] - 212s 1s/step - loss: 0.2879 - acc: 0.9027 - val_loss: 0.3872 - val_acc: 0.8987
Epoch 123/150
149/149 [==============================] - 213s 1s/step - loss: 0.2876 - acc: 0.9011 - val_loss: 0.4144 - val_acc: 0.8774
Epoch 124/150
149/149 [==============================] - 216s 1s/step - loss: 0.2908 - acc: 0.9001 - val_loss: 0.3101 - val_acc: 0.8924
Epoch 125/150
149/149 [==============================] - 214s 1s/step - loss: 0.2881 - acc: 0.9011 - val_loss: 0.3160 - val_acc: 0.9027
Epoch 126/150
149/149 [==============================] - 212s 1s/step - loss: 0.2862 - acc: 0.9016 - val_loss: 1158.0825 - val_acc: 0.5000
Epoch 127/150
149/149 [==============================] - 213s 1s/step - loss: 0.2865 - acc: 0.9002 - val_loss: 0.9562 - val_acc: 0.8979
Epoch 128/150
149/149 [==============================] - 213s 1s/step - loss: 0.2840 - acc: 0.9049 - val_loss: 0.5856 - val_acc: 0.8899
Epoch 129/150
149/149 [==============================] - 213s 1s/step - loss: 0.2839 - acc: 0.9022 - val_loss: 0.6770 - val_acc: 0.8932
Epoch 130/150
149/149 [==============================] - 212s 1s/step - loss: 0.2773 - acc: 0.9043 - val_loss: 0.2974 - val_acc: 0.8943
Epoch 131/150
149/149 [==============================] - 213s 1s/step - loss: 0.2862 - acc: 0.9019 - val_loss: 317.9427 - val_acc: 0.8774
Epoch 132/150
149/149 [==============================] - 212s 1s/step - loss: 0.2872 - acc: 0.9000 - val_loss: 0.5752 - val_acc: 0.8572
Epoch 133/150
149/149 [==============================] - 212s 1s/step - loss: 0.2857 - acc: 0.8991 - val_loss: 0.3966 - val_acc: 0.9020
Epoch 134/150
149/149 [==============================] - 213s 1s/step - loss: 0.2829 - acc: 0.9030 - val_loss: 2.7376 - val_acc: 0.8983
Epoch 135/150
149/149 [==============================] - 213s 1s/step - loss: 0.2812 - acc: 0.9044 - val_loss: 0.3372 - val_acc: 0.9101
Epoch 136/150
149/149 [==============================] - 213s 1s/step - loss: 0.2758 - acc: 0.9058 - val_loss: 0.3804 - val_acc: 0.8998
Epoch 137/150
149/149 [==============================] - 213s 1s/step - loss: 0.2752 - acc: 0.9045 - val_loss: 0.3713 - val_acc: 0.8950
Epoch 138/150
149/149 [==============================] - 214s 1s/step - loss: 0.2766 - acc: 0.9071 - val_loss: 0.3311 - val_acc: 0.9049
Epoch 139/150
149/149 [==============================] - 214s 1s/step - loss: 0.2777 - acc: 0.9045 - val_loss: 0.2921 - val_acc: 0.9068
Epoch 140/150
149/149 [==============================] - 215s 1s/step - loss: 0.2772 - acc: 0.9071 - val_loss: 10.6979 - val_acc: 0.7346
Epoch 141/150
149/149 [==============================] - 213s 1s/step - loss: 0.2786 - acc: 0.9056 - val_loss: 0.3320 - val_acc: 0.8972
Epoch 142/150
149/149 [==============================] - 214s 1s/step - loss: 0.2743 - acc: 0.9058 - val_loss: 39.9653 - val_acc: 0.8976
Epoch 143/150
149/149 [==============================] - 213s 1s/step - loss: 0.2814 - acc: 0.9042 - val_loss: 167.1936 - val_acc: 0.8726
Epoch 144/150
149/149 [==============================] - 212s 1s/step - loss: 0.2708 - acc: 0.9064 - val_loss: 5.8791 - val_acc: 0.8836
Epoch 145/150
149/149 [==============================] - 212s 1s/step - loss: 0.2785 - acc: 0.9041 - val_loss: 23.8933 - val_acc: 0.8968
Epoch 146/150
149/149 [==============================] - 213s 1s/step - loss: 0.2766 - acc: 0.9033 - val_loss: 0.3173 - val_acc: 0.9053
Epoch 147/150
149/149 [==============================] - 212s 1s/step - loss: 0.2690 - acc: 0.9095 - val_loss: 0.3940 - val_acc: 0.8814
Epoch 148/150
149/149 [==============================] - 211s 1s/step - loss: 0.2745 - acc: 0.9078 - val_loss: 1669.4307 - val_acc: 0.8968
Epoch 149/150
149/149 [==============================] - 212s 1s/step - loss: 0.2758 - acc: 0.9062 - val_loss: 0.3801 - val_acc: 0.8990
Epoch 150/150
149/149 [==============================] - 213s 1s/step - loss: 0.2725 - acc: 0.9061 - val_loss: 0.5109 - val_acc: 0.9009
Related
I am trying to build elasticsearch multi-node containers using docker. It's not communicating both nodes, we are using ES OSS version (elasticsearch-oss-7.9.3-no-jdk-linux-x86_64.tar.gz) for multi-node instances. The following docker-compose.yml file executing properly, but it's not listing the nodes, always showing a single node (refer the attached image here) and also getting the following exception message while accessing node endpoint..
Docker-compose.yml,
version: '3.7'
services:
es01:
image: es-oss:latest
container_name: es01
environment:
- jvm_args_memory=512m
- node.name=es01
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02
- es_network_host=es01
- es.cgroups.hierarchy.override=/
- bootstrap.memory_lock=true
- log_level_root=debug
- log_level_sample=debug
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- /home/centos/dev/dockerdata/es01:/var/lib/esdata
ports:
- "9200:9200"
- "9300:9300"
es02:
image: es-oss:latest
container_name: es02
environment:
- jvm_args_memory=512m
- node.name=es02
- cluster.name=es-docker-cluster
- discovery.seed_hosts=es01,es02
- cluster.initial_master_nodes=es01,es02
- es_network_host=es02
- es.cgroups.hierarchy.override=/
- bootstrap.memory_lock=true
- log_level_root=debug
- log_level_sample=debug
ulimits:
memlock:
soft: -1
hard: -1
nofile:
soft: 65536
hard: 65536
volumes:
- /home/centos/dev/dockerdata/es02:/var/lib/esdata
Error logs:
[![java.nio.file.NoSuchFileException: /sys/fs/cgroup/cpuacct/docker/7a11d2fe8a5fbbda8bbc41fd9b2a185875a1e78dd9b3dc5ea62b922be9d35999/cpuacct.usage
at sun.nio.fs.UnixException.translateToIOException(UnixException.java:92) ~\[?:?\]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:106) ~\[?:?\]
at sun.nio.fs.UnixException.rethrowAsIOException(UnixException.java:111) ~\[?:?\]
at sun.nio.fs.UnixFileSystemProvider.newByteChannel(UnixFileSystemProvider.java:219) ~\[?:?\]
at java.nio.file.Files.newByteChannel(Files.java:375) ~\[?:?\]
at java.nio.file.Files.newByteChannel(Files.java:426) ~\[?:?\]
at java.nio.file.spi.FileSystemProvider.newInputStream(FileSystemProvider.java:420) ~\[?:?\]
at java.nio.file.Files.newInputStream(Files.java:160) ~\[?:?\]
at java.nio.file.Files.newBufferedReader(Files.java:2916) ~\[?:?\]
at java.nio.file.Files.readAllLines(Files.java:3396) ~\[?:?\]
at java.nio.file.Files.readAllLines(Files.java:3436) ~\[?:?\]
at org.elasticsearch.monitor.os.OsProbe.readSingleLine(OsProbe.java:235) ~\[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.os.OsProbe.readSysFsCgroupCpuAcctCpuAcctUsage(OsProbe.java:324) ~\[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.os.OsProbe.getCgroupCpuAcctUsageNanos(OsProbe.java:311) ~\[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.os.OsProbe.getCgroup(OsProbe.java:533) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.os.OsProbe.osStats(OsProbe.java:659) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.os.OsService.<init>(OsService.java:50) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.monitor.MonitorService.<init>(MonitorService.java:46) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.node.Node.<init>(Node.java:409) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.node.Node.<init>(Node.java:277) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Bootstrap$5.<init>(Bootstrap.java:227) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Bootstrap.setup(Bootstrap.java:227) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Bootstrap.init(Bootstrap.java:393) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Elasticsearch.init(Elasticsearch.java:170) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Elasticsearch.execute(Elasticsearch.java:161) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:86) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:127) \[elasticsearch-cli-7.9.3.jar:7.9.3\]
at org.Elasticsearch. cli.Command.main(Command.java:90) \[elasticsearch-cli-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:126) \[elasticsearch-7.9.3.jar:7.9.3\]
at org.elasticsearch.bootstrap.Elasticsearch.main(Elasticsearch.java:92) \[elasticsearch-7.9.3.jar:7.9.3\]][1]][1]
Any inputs here are really appreciated.
Starting at epoch 0. LR=0.002
Checkpoint Path: /content/Mask_RCNN/logs/crack_images20200707T0451/mask_rcnn_crack_images_{epoch:04d}.h5
Selecting layers to train
fpn_c5p5 (Conv2D)
fpn_c4p4 (Conv2D)
fpn_c3p3 (Conv2D)
fpn_c2p2 (Conv2D)
fpn_p5 (Conv2D)
fpn_p2 (Conv2D)
fpn_p3 (Conv2D)
fpn_p4 (Conv2D)
In model: rpn_model
rpn_conv_shared (Conv2D)
rpn_class_raw (Conv2D)
rpn_bbox_pred (Conv2D)
mrcnn_mask_conv1 (TimeDistributed)
mrcnn_mask_bn1 (TimeDistributed)
mrcnn_mask_conv2 (TimeDistributed)
mrcnn_mask_bn2 (TimeDistributed)
mrcnn_class_conv1 (TimeDistributed)
mrcnn_class_bn1 (TimeDistributed)
mrcnn_mask_conv3 (TimeDistributed)
mrcnn_mask_bn3 (TimeDistributed)
mrcnn_class_conv2 (TimeDistributed)
mrcnn_class_bn2 (TimeDistributed)
mrcnn_mask_conv4 (TimeDistributed)
mrcnn_mask_bn4 (TimeDistributed)
mrcnn_bbox_fc (TimeDistributed)
mrcnn_mask_deconv (TimeDistributed)
mrcnn_class_logits (TimeDistributed)
mrcnn_mask (TimeDistributed)
Epoch 1/20
100/100 [==============================] - 2546s - loss: 59.9573 - rpn_class_loss: 4.9158 - rpn_bbox_loss: 55.0416 - mrcnn_class_loss: 2.5773e-06 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 42.9541 - val_rpn_class_loss: 8.3271 - val_rpn_bbox_loss: 34.6270 - val_mrcnn_class_loss: 4.1246e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 2/20
100/100 [==============================] - 2466s - loss: 47.9174 - rpn_class_loss: 3.4403 - rpn_bbox_loss: 44.4434 - mrcnn_class_loss: 0.0044 - mrcnn_bbox_loss: 0.0198 - mrcnn_mask_loss: 0.0096 - val_loss: 40.8780 - val_rpn_class_loss: 1.7095 - val_rpn_bbox_loss: 39.1685 - val_mrcnn_class_loss: 4.3463e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 3/20
100/100 [==============================] - 2432s - loss: 45.2084 - rpn_class_loss: 2.1342 - rpn_bbox_loss: 43.0547 - mrcnn_class_loss: 0.0030 - mrcnn_bbox_loss: 0.0062 - mrcnn_mask_loss: 0.0103 - val_loss: 40.2231 - val_rpn_class_loss: 1.9367 - val_rpn_bbox_loss: 38.2864 - val_mrcnn_class_loss: 7.5409e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 4/20
100/100 [==============================] - 2402s - loss: 44.3991 - rpn_class_loss: 2.0594 - rpn_bbox_loss: 42.3396 - mrcnn_class_loss: 7.5638e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 42.4247 - val_rpn_class_loss: 3.2007 - val_rpn_bbox_loss: 39.2239 - val_mrcnn_class_loss: 7.1904e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 5/20
100/100 [==============================] - 2404s - loss: 47.2541 - rpn_class_loss: 3.9966 - rpn_bbox_loss: 43.2575 - mrcnn_class_loss: 4.8513e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 36.8668 - val_rpn_class_loss: 0.6245 - val_rpn_bbox_loss: 36.2422 - val_mrcnn_class_loss: 3.1065e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 6/20
100/100 [==============================] - 2402s - loss: 41.3224 - rpn_class_loss: 3.4068 - rpn_bbox_loss: 37.9156 - mrcnn_class_loss: 2.7582e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 30.5534 - val_rpn_class_loss: 1.2449 - val_rpn_bbox_loss: 29.3085 - val_mrcnn_class_loss: 2.4438e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 7/20
100/100 [==============================] - 2391s - loss: 45.0313 - rpn_class_loss: 3.8534 - rpn_bbox_loss: 41.1548 - mrcnn_class_loss: 0.0021 - mrcnn_bbox_loss: 0.0122 - mrcnn_mask_loss: 0.0089 - val_loss: 41.7707 - val_rpn_class_loss: 0.9706 - val_rpn_bbox_loss: 40.8001 - val_mrcnn_class_loss: 3.5309e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 8/20
100/100 [==============================] - 2435s - loss: 41.4883 - rpn_class_loss: 2.7715 - rpn_bbox_loss: 38.6841 - mrcnn_class_loss: 0.0016 - mrcnn_bbox_loss: 0.0224 - mrcnn_mask_loss: 0.0086 - val_loss: 42.0558 - val_rpn_class_loss: 2.9680 - val_rpn_bbox_loss: 39.0877 - val_mrcnn_class_loss: 2.6321e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 9/20
100/100 [==============================] - 2473s - loss: 41.6691 - rpn_class_loss: 2.3281 - rpn_bbox_loss: 39.3410 - mrcnn_class_loss: 2.8393e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 38.0360 - val_rpn_class_loss: 1.7419 - val_rpn_bbox_loss: 36.2941 - val_mrcnn_class_loss: 2.2530e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 10/20
100/100 [==============================] - 2450s - loss: 39.7236 - rpn_class_loss: 2.0216 - rpn_bbox_loss: 37.6904 - mrcnn_class_loss: 0.0010 - mrcnn_bbox_loss: 0.0026 - mrcnn_mask_loss: 0.0080 - val_loss: 33.0290 - val_rpn_class_loss: 6.8746 - val_rpn_bbox_loss: 26.1544 - val_mrcnn_class_loss: 2.5153e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 11/20
100/100 [==============================] - 2397s - loss: 40.1832 - rpn_class_loss: 3.3822 - rpn_bbox_loss: 36.8010 - mrcnn_class_loss: 2.4559e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 36.0607 - val_rpn_class_loss: 4.1198 - val_rpn_bbox_loss: 31.9409 - val_mrcnn_class_loss: 2.4557e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 12/20
100/100 [==============================] - 2391s - loss: 31.5999 - rpn_class_loss: 2.2898 - rpn_bbox_loss: 29.3101 - mrcnn_class_loss: 2.5380e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 28.0113 - val_rpn_class_loss: 0.6510 - val_rpn_bbox_loss: 27.3603 - val_mrcnn_class_loss: 2.1243e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 13/20
100/100 [==============================] - 2441s - loss: 29.5633 - rpn_class_loss: 2.0158 - rpn_bbox_loss: 27.5475 - mrcnn_class_loss: 2.8165e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 24.3744 - val_rpn_class_loss: 1.0115 - val_rpn_bbox_loss: 23.3629 - val_mrcnn_class_loss: 2.1410e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 14/20
100/100 [==============================] - 2424s - loss: 29.5249 - rpn_class_loss: 2.8759 - rpn_bbox_loss: 26.6115 - mrcnn_class_loss: 0.0013 - mrcnn_bbox_loss: 0.0202 - mrcnn_mask_loss: 0.0160 - val_loss: 21.7251 - val_rpn_class_loss: 1.2835 - val_rpn_bbox_loss: 20.4416 - val_mrcnn_class_loss: 1.4686e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 15/20
100/100 [==============================] - 2441s - loss: 25.1142 - rpn_class_loss: 2.5004 - rpn_bbox_loss: 22.6137 - mrcnn_class_loss: 1.3815e-05 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 31.8563 - val_rpn_class_loss: 0.5517 - val_rpn_bbox_loss: 31.3045 - val_mrcnn_class_loss: 1.5712e-05 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 16/20
100/100 [==============================] - 2371s - loss: 28.6811 - rpn_class_loss: 2.5408 - rpn_bbox_loss: 26.1402 - mrcnn_class_loss: 9.1934e-06 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 20.2755 - val_rpn_class_loss: 0.6740 - val_rpn_bbox_loss: 19.6015 - val_mrcnn_class_loss: 6.0320e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 17/20
100/100 [==============================] - 2427s - loss: 26.1218 - rpn_class_loss: 4.1050 - rpn_bbox_loss: 21.9986 - mrcnn_class_loss: 0.0018 - mrcnn_bbox_loss: 0.0074 - mrcnn_mask_loss: 0.0091 - val_loss: 23.0992 - val_rpn_class_loss: 3.4643 - val_rpn_bbox_loss: 19.6348 - val_mrcnn_class_loss: 5.0068e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 18/20
100/100 [==============================] - 2487s - loss: 27.5055 - rpn_class_loss: 2.8223 - rpn_bbox_loss: 24.6514 - mrcnn_class_loss: 0.0017 - mrcnn_bbox_loss: 0.0155 - mrcnn_mask_loss: 0.0145 - val_loss: 20.8848 - val_rpn_class_loss: 2.3465 - val_rpn_bbox_loss: 18.5383 - val_mrcnn_class_loss: 2.8133e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 19/20
100/100 [==============================] - 2537s - loss: 21.7046 - rpn_class_loss: 1.6853 - rpn_bbox_loss: 20.0193 - mrcnn_class_loss: 3.7229e-06 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 27.4155 - val_rpn_class_loss: 8.5852 - val_rpn_bbox_loss: 18.8303 - val_mrcnn_class_loss: 4.4346e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Epoch 20/20
100/100 [==============================] - 2537s - loss: 25.0851 - rpn_class_loss: 2.9776 - rpn_bbox_loss: 22.1076 - mrcnn_class_loss: 2.8539e-06 - mrcnn_bbox_loss: 0.0000e+00 - mrcnn_mask_loss: 0.0000e+00 - val_loss: 17.3773 - val_rpn_class_loss: 2.7509 - val_rpn_bbox_loss: 14.6263 - val_mrcnn_class_loss: 2.5272e-06 - val_mrcnn_bbox_loss: 0.0000e+00 - val_mrcnn_mask_loss: 0.0000e+00
Fine tune Resnet stage 4 and up
Training took 814.59 minutes
I ran 20 epochs and noticed that the val_mrcnn_bbox_loss and val_mrcnn_mask_loss were 0 right from the beginning and I would like to ask:
What does each loss mean? Is there perhaps a documentation explaining them?
Why are the two losses 0? Is there something wrong?
What I have done was:
Draw labels using a website called Labelbox
Export the labels as a JSON file
Run a script to convert the JSON file to suit the COCO format
Run MASK-RCNN
So far I'm unable to produce any segmentations for my predictions but I am guessing it is due to the low number of epochs I have ran.
I am new to classification through segmentation and I would appreciate any advice. Thanks!
There is no straight docs, but you may find some comment on github repo: https://github.com/matterport/Mask_RCNN/issues/1112
rpn_class_loss: How well the Region Proposal Network separates background with objects
rpn_bbox_loss : How well the RPN localize objects
mrcnn_bbox_loss : How well the Mask RCNN localize objects
mrcnn_class_loss : How well the Mask RCNN recognize each class of object
mrcnn_mask_loss : How well the Mask RCNN segment objects
That makes a bigger loss:
loss : A combination (surely an addition) of all the smaller losses.
Extremely small val loss (0 or close to) may happen when val data is just (partly) copies of train data. Don't mix this sets.
I want to use the allow_growth configuration with Keras and tensorflow backend.
As suggested f. e. in use tensorflow.GPUOptions within Keras when using tensorflow backend I implemented my code as follows:
import keras.backend.tensorflow_backend as K
import tensorflow as tf
...
gpu_options = tf.GPUOptions(allow_growth=True)
config = tf.ConfigProto(gpu_options=gpu_options)
sess = tf.Session(config = config)
K.set_session(sess)
model = Sequential()
model.add(...)
...
model.compile(...)
model.fit(...)
With nvidia-smi I can see, that dynamic allocation works. But with this adaption applied, I'm not able to achieve any improvements in the classification accuracy, which means that the validation error remains the same over multiple epochs.
Hope you can help me.
EDIT:
Output of the first 10 epochs:
Epoch 1/30
307/307 [==============================] - 45s 147ms/step - loss: 3.7766 - acc: 0.0558 - val_loss: 3.7457 - val_acc: 0.0420
Epoch 2/30
307/307 [==============================] - 43s 140ms/step - loss: 3.7372 - acc: 0.0566 - val_loss: 3.7309 - val_acc: 0.0420
Epoch 3/30
307/307 [==============================] - 44s 143ms/step - loss: 3.7222 - acc: 0.0566 - val_loss: 3.7170 - val_acc: 0.0420
Epoch 4/30
307/307 [==============================] - 45s 146ms/step - loss: 3.7079 - acc: 0.0566 - val_loss: 3.7037 - val_acc: 0.0420
Epoch 5/30
307/307 [==============================] - 46s 150ms/step - loss: 3.6944 - acc: 0.0566 - val_loss: 3.6911 - val_acc: 0.0420
Epoch 6/30
307/307 [==============================] - 45s 147ms/step - loss: 3.6815 - acc: 0.0565 - val_loss: 3.6793 - val_acc: 0.0420
Epoch 7/30
307/307 [==============================] - 44s 144ms/step - loss: 3.6693 - acc: 0.0566 - val_loss: 3.6680 - val_acc: 0.0420
Epoch 8/30
307/307 [==============================] - 44s 143ms/step - loss: 3.6577 - acc: 0.0566 - val_loss: 3.6573 - val_acc: 0.0420
Epoch 9/30
307/307 [==============================] - 45s 147ms/step - loss: 3.6467 - acc: 0.0566 - val_loss: 3.6473 - val_acc: 0.0420
Epoch 10/30
307/307 [==============================] - 44s 143ms/step - loss: 3.6363 - acc: 0.0565 - val_loss: 3.6378 - val_acc: 0.0420
I need to execute my bash script (output.sh) as piped script.
see below.
echo "Dec 10 03:39:13 cgnat2.dd.com 1 2015 Dec 9 14:39:11 01-g3-adsl - - NAT44 - [UserbasedW - 100.70.24.236 vrf-testnet - 222.222.34.65 - 3072 4095 - - ][UserbasedW - 100.70.25.9 vrf-testnet - 222.222.34.65 - 16384 17407 - - ][UserbasedW - 100.70.25.142 vrf-testnet - 222.222.34.69 - 9216 10239 - - ]" | ./output.sh
how can i get echoing text in to my output.sh file and I need to split echoing text using [
output should be
[UserbasedW - 100.70.24.236 vrf-testnet - 222.222.34.65 - 3072 4095 - - ]
[UserbasedW - 100.70.25.9 vrf-testnet - 222.222.34.65 - 16384 17407 - - ]
[UserbasedW - 100.70.25.142 vrf-testnet - 222.222.34.69 - 9216 10239 - - ]
please help me. i have no idea.. :(
With grep:
| grep -o '\[[^]]*\]'
or with GNU grep:
| grep -oP '\[.*?\]'
Output:
[UserbasedW - 100.70.24.236 vrf-testnet - 222.222.34.65 - 3072 4095 - - ]
[UserbasedW - 100.70.25.9 vrf-testnet - 222.222.34.65 - 16384 17407 - - ]
[UserbasedW - 100.70.25.142 vrf-testnet - 222.222.34.69 - 9216 10239 - - ]
With a bash script (e.g. output.sh):
#!/bin/bash
grep -o '\[[^]]*\]'
Usage:
echo "... your string ..." | ./output.sh
See: The Stack Overflow Regular Expressions FAQ
If it is removing header before first [, use a sed between before your pipe (assuming your echo is a sample of other source)
echo "Dec 10 03:39:13 cgnat2.dd.com 1 2015 Dec 9 14:39:11 01-g3-adsl - - NAT44 - [UserbasedW - 100.70.24.236 vrf-testnet - 222.222.34.65 - 3072 4095 - - ][UserbasedW - 100.70.25.9 vrf-testnet - 222.222.34.65 - 16384 17407 - - ][UserbasedW - 100.70.25.142 vrf-testnet - 222.222.34.69 - 9216 10239 - - ]" \
| sed 's/^[^[]*//' \
| ./output.sh
I have a file 'matrix.dat' looks like this:
10584 179888 115816 16768 91440 79928 50656 23624 21712 51776 89670 21815 13536 18984 11997 16221 10336 432 632 2024 - - - - - - - - - - - - - 408 - - - - - - - - - - - - - - - B - - - B - - B - - - - - - - - - - - - 3672 - - 4480 - - - - - - - - 17600 11632 1008 4384 144 - 216 72 - - - - - 768 336 - 384 - - 408 5312 - - - 72 3648 - - - - - - - - - - - - 1088 - - 224 - - - - - - - - - - - 1696 2040 2664 216 - B 344 - - - - - 336 296 248 88 88 616 - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - - 2840 - - 128 16 - 112 - - - - - 1904 2776 24 B
I want to plot numbers using palette, '-' using white color and 'B' using black color.
In gnuplot, I use log2 palette (blue -> cyan -> green -> orange -> red) and set '-' as missing data:
set palette model HSV functions 0.666*(1-gray), 1, 1
set logscale cb 2
set datafile missing "-"
plot 'matrix.dat' matrix with image
Now I can only plot numbers and '-' in desired colors. How can I plot 'B' in black color?
I solved the problem using a piecewise function. Just a small change:
set palette model HSV functions gray>0 ? 0.666*(1-gray):0, 1, gray>0 ? 1:0
then change all 'B' of the file into '0'. The idea is that using black color for 0 and using colors in palette for non-zero values. Thanks!