VGG16遷移學習(他山之石,可以攻玉)_分辨人和馬_使用 TensorFlow Keras 實作 VGG16 遷移學習_part2.用VGG16預訓練模型來訓練自己的圖片集



這邊補安裝pyyaml套件,後續可用來將神經網路模型保存到.yaml檔案中。
可用pip install pyyaml 來裝
[更新]pyyaml後來也用不到了,除非你用tensorflow早期版本。
TensorFlow 2.5之前版本都還能夠用model.to_yaml()將模型結構存出來,到後期就被移除了。




記得在欲執行的py檔案相對路徑層級創建models目錄

程式碼1.利用VGG16 遷移學習模型完成 5 個 Epoch 訓練


import os
from pathlib import Path
import json
import yaml
#os.environ['TF_CPP_MIN_LOG_LEVEL']='2'
# 在匯入 TensorFlow 前設定
os.environ["CUDA_VISIBLE_DEVICES"] = "-1"
# 除錯期間保留警告訊息
os.environ["TF_CPP_MIN_LOG_LEVEL"] = "1"
import tensorflow as tf
print("TensorFlow:", tf.__version__)
print("Available GPU:", tf.config.list_physical_devices("GPU"))

def transfer_model(input_shape=(224, 224, 3)):
    vgg16 = tf.keras.applications.vgg16.VGG16(
        include_top=False,  # 不包含頂層
        weights='imagenet',
        input_shape=input_shape
    )

    for layer in vgg16.layers:
        layer.trainable = False  # 設置VGG16預訓練模型不可訓練

    pre_model = vgg16.output

    # 添加自定義全連接層
    x = tf.keras.layers.Flatten()(pre_model)
    x = tf.keras.layers.Dense(128, activation='relu')(x)
    x = tf.keras.layers.Dropout(0.3)(x)
    x = tf.keras.layers.Dense(32, activation='relu')(x)
    x = tf.keras.layers.Dropout(0.3)(x)
    x = tf.keras.layers.Dense(2, activation='softmax')(x)

    # 生成遷移模型
    model = tf.keras.models.Model(inputs=vgg16.input, outputs=x)
    model.summary()

    return model

def read_images(path):
    all_images = []
    # 讀取 horses 影像
    horse_path = path + "/horses/"
    for f in os.listdir(horse_path):
        img_string = tf.io.read_file(horse_path + f)  # 讀取原始檔案
        img_graph = tf.image.decode_png(img_string)  # 解碼 PNG 圖片
        # 調整影像大小,與後續模型的輸入要求一致,並進行標準化
        img_final = tf.image.resize(img_graph, [224, 224]) / 255.0
        all_images += [img_final]
    horse_files_num = len(all_images)
    # 讀取 humans 影像
    human_path = path + "/humans/"
    for f in os.listdir(human_path):
        img_string = tf.io.read_file(human_path + f)  # 讀取原始檔案
        img_graph = tf.image.decode_png(img_string)  # 解碼 PNG 圖片
        # 調整影像大小,並進行標準化
        img_final = tf.image.resize(img_graph, [224, 224]) / 255.0
        all_images += [img_final]
    human_files_num = len(all_images) - horse_files_num

    print('read files:', len(all_images))
    print(horse_files_num, 'horses files in', 'files')
    print(human_files_num, 'humans files in', 'files')
    # 取得所有標籤。標籤使用 One-hot 編碼表示, horse 標籤為 [1, 0],human 標籤為 [0, 1]
    all_labels = [[1,0]] * horse_files_num + [[0,1]] * human_files_num
    return all_images, all_labels

def generate_dataset(data_path,batch_size=8):
    images,labels = read_images(data_path)
    dataset = tf.data.Dataset.from_tensor_slices((images, labels))
    dataset = dataset.shuffle(len(images))#->洗牌打亂,避免前面固定都馬,後面都人。
    dataset = dataset.batch(batch_size)
    dataset = dataset.prefetch(tf.data.experimental.AUTOTUNE)
    return dataset

model = transfer_model()


model.compile(optimizer='adam',
              loss=tf.keras.losses.CategoricalCrossentropy(from_logits=False),
              metrics=['categorical_accuracy'])

data_path = r"E:\論文研究\遷移學習\horse-or-human\horse-or-human\train2"
dataset_train = generate_dataset(data_path)
#訓練
train_history=model.fit(dataset_train,epochs=5,verbose=1)
model_dir = Path(__file__).resolve().parent / "models"
yaml_path = model_dir / "horse-or-human_model.yaml"
weights_path = model_dir / "horse-or-human_weights.h5"

# 將 Keras 模型架構轉換成 YAML
#底下寫法已經不適用
#str_yaml = model.to_yaml()
#RuntimeError: Method `model.to_yaml()` has been removed due to security risk of arbitrary code execution. Please use `model.to_json()` instead.
# with open('./models/horse-or-human_model.yaml', 'w') as model_file:
#     model_file.write(str_yaml)
model_json = model.to_json()
model_config = json.loads(model_json)

with open(yaml_path, "w", encoding="utf-8") as model_file:
    yaml.safe_dump(
        model_config,
        model_file,
        allow_unicode=True,
        sort_keys=False
    )

# 儲存訓練完成的權重
model.save_weights(str(weights_path))

print("\n模型儲存完成!")
print("YAML:", yaml_path)

運行結果



預設Kaggle提供了驗證資料集
horses:128張
humans:128張



這邊圖片也是RGBA的四個通道





Ref:
TensorFlow 2.7 drops YAML support #22
딥러닝 텐서플로우 이미지 10개로 분류 Saving architecture & Saving network weights, Dropout, validation_data, Flatten안쓰기
How to Save and Load Your Keras Deep Learning Model


留言

這個網誌中的熱門文章

何謂淨重(Net Weight)、皮重(Tare Weight)與毛重(Gross Weight)

外貿Payment Term 付款條件(方式)常見的英文縮寫與定義

Architecture(架構) 和 Framework(框架) 有何不同?_軟體設計前的事前規劃的藍圖概念