AI model training fails with an unspecific internal error. What can I do to narrow down possible reasons for this behavior?
Screenshot of error message
Regards,
Christoph
I uploaded a training dataset and started training.
Related
My use case is related to multiclass image classification. Deployed CNN Model in production and enabled Model Monitoring for prediction drift detection only which does not require training data. It automatically gets created two buckets- analysis and predict in storage bucket. Then I created and run 1000 instances for model testing purpose(Same request 1000 times through Apache Bench) as it was prerequisite. I kept monitoring job to run for every hour and 100% sampling rate. I am not getting any output or logs in newly created buckets?
What's the error here?
Is Model Monitoring(Prediction Drift Detection) not enabled for Image Data by Vertex AI?
What steps do I need to take in order to check the Model Monitoring is working fine for Image Classification Model. We need evidence in the form of logs generated in two buckets.
Model monitoring is only supported for tabular AutoML and tabular custom-trained models at the moment. It is not support for custom-trained image classification models.
For a more proactive approach that should minimize prediction drift in image classification models, Vertex AI Team would recommend the following:
• Augmenting your data such that you have a more diverse set of samples. This set should match your business needs, and has meaningful transformations given your context. Please refer to [2] for more information about data augmentation.
• Utilizing Vertex Explainable AI to identify the features which are contributing the most to your model's classification decisions. This would help you to augment your data in a more educated manner. Please refer to [3] for more information about Vertex Explainable AI.
[1] https://cloud.google.com/vertex-ai/docs/model-monitoring/overview
[2] https://www.tensorflow.org/tutorials/images/data_augmentation
[3] https://cloud.google.com/vertex-ai/docs/explainable-ai/overview
We are trying to understand the underlying model of Rasa - the forums there still didnt get us an answer - on two main questions:
we understand that Rasa model is a transformer-based architecture. Was it
pre-trained on any data set? (eg wikipedia, etc)
then, if we
understand correctly, the intent classification is a fine tuning task
on top of that transformer. How come it works with such small
training sets?
appreciate any insights!
thanks
Lior
the transformer model is not pre-trained on any dataset. We use quite a shallow stack of transformer which is not as data hungry as deeper stacks of transformers used in large pre-trained language models.
Having said that, there isn't an exact number of data points that will be sufficient for training your assistant as it varies by the domain and your problem. Usually a good estimate is 30-40 examples per intent.
I'm training a yolov3 neural network (https://github.com/ultralytics/yolov3/) to recognize objects in an image and was able to get some metrics out.
I was just wondering if anyone knew how to interpret the following metrics (i.e. definition of what these metrics measure).
Objectness
Classification.
yoloV3 Training Metrics Plots
I'm assuming the val Objectness and val Classification are the scores for the validation set.
Thanks!
Sorry for the late reply. Anyways, hope this useful for somebody.
Objectness: measures how well the model is at identifying that an object exists in a proposed region of interest
Classification: measures how well the model is at labeling those objects by their corresponding associated class
Both are usually calculated by nn.BCEWithLogitsLoss as both are classification tasks
I am trying a multilayer perceptron with 5 hidden nodes. However the testing error is lower than the validation error and higher than the training error. The coefficient of correlation on the test set is also higher than that on the validation set. Is that acceptable? I've included the regression and performance plots. The generalization is okay according to me; not the best but adequate.
This is an ANN-GARCH-type model for volatility.
I was using AlchemyAPI for text analysis. I want to know if there is way to influence the API results or fine-tune it as per the requirement.
I was trying to analyse different call center conversations available on internet. To understand the sentiments i.e. whether customer was unsatisfied/angry and hence conversation is negative.
For 9 out of 10 conversations it gave sentiment as positive and for 1 it was negative. That conversation was about emergency response system (#911 in US). It seems that words shooting, fear, panic, police, siren could have cause this result.
But actually the whole conversation was fruitful. Caller was not angry with the service instead call center person solved the caller's problem and caller was relaxed. So logically this should not be treated as negative.
What is the way ahead to customize the AlchemyAPI behavior ?
We are currently looking at the tools that would be required to allow customization of the AlchemyAPI services. Our current service is entirely pre-trained on billions of web pages, but customization is on the road map. I can't give you any timelines this early, but keep checking back!
Zach, Dev Evangelist AlchemyAPI