Generate instances and prediction schema files for Vertex AI model evaluation (Google Cloud pipeline components) - google-cloud-vertex-ai

Question

I build a custom model and try to use Google Cloud pipeline components for model evaluation on Vertex AI. According to the model_upload_predict_evaluate notebook sample, I need to prepare instance_schema.yaml and prediction_schema.yaml to import unmanaged model.
So how to generate the instance and prediction schema files programmatically?

Answer 1

For custom models, instances and prediction schemas are not required:
"predictSchemata": {
"predictionSchemaUri": MODEL_URI + "/prediction_schema.yaml",
"instanceSchemaUri": MODEL_URI + "/instance.yaml",
},

Answer 2

In short:
the content of your yaml file must be following the OpenAPI 3.0.2 schema object (https://github.com/OAI/OpenAPI-Specification/blob/main/versions/3.0.2.md#schemaObject) -> found from the [python sdk](https://github.com/googleapis/python-aiplatform/blob/e21762b703e3f6f45a40e42395b3387eee72141d/google/cloud/aiplatform_v1/types/model.py#L468
your yaml file must be suffixed with .yaml. I do not fully understand why, but it gave me some headaches to find this problem knowing that kfp OutputPath do not suffix the paths...
Now to build programmatically, it depends on your use case and what you start from. I'm sure it is feasible to build yourself the yaml content from your source data. In my case, I built my custom image to run my predictor using Pydantic models, from which you can automatically generate its openapi schema in a json format, which can then be transformed into a yaml file. Using a different dataset than yours (I'm using the sushi dataset), my code example is:
import yaml
from pydantic import BaseModel, Field
class InstanceModel(BaseModel):
age: int = Field(..., ge=0, le=5)
east_west_id_now: int = Field(..., ge=0, le=1)
east_west_id_until_15yo: int = Field(..., ge=0, le=1)
gender: int = Field(..., ge=0, le=1)
prefecture_id_now: int = Field(..., ge=0, le=47)
prefecture_id_until_15yo: int = Field(..., ge=0, le=47)
region_id_now: int = Field(..., ge=0, le=11)
region_id_until_15yo: int = Field(..., ge=0, le=11)
same_prefecture_id_over_time: int = Field(..., ge=0, le=1)
time_fill_form: int = Field(..., ge=0)
schema_file_path = "..." # where you want your yaml file
model_json_string = InstanceModel.schema_json()
model_dict = json.loads(model_json_string)
example = {}
for key in model_dict['properties']:
example[key] = 0
model_dict['example'] = example
with open(schema_file_path, 'w', encoding='utf8') as f:
yaml.dump(model_dict, f)
which creates the following:
example:
age: 0
east_west_id_now: 0
east_west_id_until_15yo: 0
gender: 0
prefecture_id_now: 0
prefecture_id_until_15yo: 0
region_id_now: 0
region_id_until_15yo: 0
same_prefecture_id_over_time: 0
time_fill_form: 0
properties:
age:
maximum: 5
minimum: 0
title: Age
type: integer
east_west_id_now:
maximum: 1
minimum: 0
title: East West Id Now
type: integer
east_west_id_until_15yo:
maximum: 1
minimum: 0
title: East West Id Until 15Yo
type: integer
gender:
maximum: 1
minimum: 0
title: Gender
type: integer
prefecture_id_now:
maximum: 47
minimum: 0
title: Prefecture Id Now
type: integer
prefecture_id_until_15yo:
maximum: 47
minimum: 0
title: Prefecture Id Until 15Yo
type: integer
region_id_now:
maximum: 11
minimum: 0
title: Region Id Now
type: integer
region_id_until_15yo:
maximum: 11
minimum: 0
title: Region Id Until 15Yo
type: integer
same_prefecture_id_over_time:
maximum: 1
minimum: 0
title: Same Prefecture Id Over Time
type: integer
time_fill_form:
minimum: 0
title: Time Fill Form
type: integer
required:
- age
- east_west_id_now
- east_west_id_until_15yo
- gender
- prefecture_id_now
- prefecture_id_until_15yo
- region_id_now
- region_id_until_15yo
- same_prefecture_id_over_time
- time_fill_form
title: InstanceModel
type: object
Now my issue is that I have no clue where it is actually used... I cannot see it in the model version/details/request example from the console... so I may have done something wrong after that lol.