I am using Microsoft form-recognizer service. My forms are bit complex and I tried training a model for them. The performance I achieved is not really good. Is there anyway I can improve this accuracy? Is there anyway to tune this model? I have trained the model using 5 different populated forms of the same type.
Not sure if you're still interested in it. I've been using the labeling tool in the v2 You can manually label area to create tags (features). This can be done without too much effort. I've been using it and get a very accurate results.
IC
Related
Say I want to train a model for sequence classification. And so I define my model to be:
model = DistilBertForSequenceClassification.from_pretrained("bert-base-uncased")
My question is - what would be the optimal way if I want to pre-train this model with masked language modeling task? After pre-training I would like to model to train on the down-stream task of sequence classification.
My understanding is that I can somehow switch the heads of my model and a DistilBertForMaskedLM for pre-training, and then switch it back to the original downstream task. Although I haven't figured out if this is indeed optimal or how to write it.
Does hugging face offer any built in function that accepts the input ids, a percentage of tokens to masked (which aren't pad tokens) and simply trains the model?
Thank you in advance
I've tried to implement this myself, and while it does seem to work it is extremely slow. I figured there could already be implemented solutions instead of trying to optimize my code.
I'm working on a data mining project when I need to be able to predict chances of success on a Kickstarter project funding.
I've used a kickstarter dataset which i've found on Kaggle, and i've cleaned all the noisy data, deleted irrelevant attributes and added another useful attributes.
Now I have about 320K instances and 6 attributes.
After running J48 algorithm, I'm getting 65.07% correctly classified instances and 68.7% average roc area.
I have to get this performance improved but I dont know how.
It's a college project so I have specific rules: I can only change the Confidence Factor and NumMinObj of the algorithm.
I've spending a lot of time trying every combination.
What can I do else? Maybe something in my dataset is problematic?
You have a lot of instances, but few attributes. If you can't add more attributes then probably you already got the best result you can have with J48 trees, and feature selection is useless. You probably have to use a more complex classification algorithm such as RandomForest.
I used AutoML Vision to train a model to predict cancer based on images. It works quite well. I want to know what the model is, whether it is CNN, how many layers.
Thank you!
We don't normally release the exact details of the model because we want to, and continue to, change it under the hood as newer better models come out.
It is a relatively deep CNN.
I followed this post and first made it work on the dataset «Cats vs dogs». Then I substituted this set with my own images, which show the presence of an object vs the absence of that object. My dataset is even smaller than the one in the post. I only have 496 images containing that object for training and 160 images with that object for validation. For the «absent» class I have numerous samples (without that object in an image).
So far I didn't try class_weight to tackle the imbalanced data problem. I just randomly choose 496 and 160 images without that object for training and validation, respectively. Basically, I do a two class image classification with a smaller dataset using the techniques in this post. Thus I expected a worse performance in comparison due to the insufficient data. But the actual problem is that the performance is not convergent as shown in the figures.
Could you tell me possible reasons that lead to the unconvergence? I guess the problem is related to my dataset as the model works perfectly for «cats vs dogs». But I don't know how to address it. Are there any good techniques to make it convergent?
Thank you.
This performance plot is based on VGG16, keeping all layers up to fully connected layer and training a small fully connected layer with 256 neurons.
This performance plot is also based on VGG16, but using 128 neurons instead of 256 neurons. Also I set epochs to 80.
Based on the suggestions provided so far, I'm thinking to have a customized convnet model to fight the overfitting problem. But how to do this? One of my worries is that a model with fewer layers will downgrade the performance for training. Any guidelines to customize a good model for little data? Thank you.
Updates:
Now I think I know the half reason that leads to the unconvergent problem. You know, Actually I only have 100+ images. The rest images are downloaded from Flickr. I thought those images having centric objects and better quality will work for the model. But later on I found they can not contribute to the accuracy and even worse the output class probabilities. After removing these downloaded images, the performance is bumping upward a little and the uncovergency is gone. Note I only use 64*2 images for training and 48*2 images for testing. Also I found the image augmentation could not improve the performance for my dataset. Without image augmentation, the training accuracy could reach 1. But if I add some image augmentation, the training accuracy is only around 85%. Did somebody have such experience? Why doesn't data augmentation always work? Because our specific dataset? Thank you very much.
Your model is working great, but it's "overfitting". It means it's capable of memorizing all your training data without really "thinking". That leads to great training results and bad test results.
Common ways to avoid overfitting are:
More data - If you have little data, the chance of overfitting increases
Less units/layers - Make the model less capable, so it will stop memorizing and start thinking.
Add "dropouts" to your layers (something that randomly discards part of the results to prevent the model from being too powerful)
Do more layers mean more power and performance?
If by performance you mean capability of learning, yes. (If you mean "speed", no)
Yes, more layers mean more power. But too much power leads to overfitting: the model is so capable that it can memorize training data.
So there is an optimal point:
A model that is not very capable will not give you the proper results (both training and test results will be bad)
A model that is too capable will memorize the training data (excellent training results, but bad test results)
A balanced model will learn the right things (good training and test results)
That's exactly why we use test data, it's data that is not presented for training, so the model doesn't learn from the test data.
I am very beginner in h2o and I want to know if there is any attribute selection capabilities in h2o framework so to be applied in h2oframes?
No there are not currently feature selection functions in H2O -- my advice would be to use Lasso regression (in H2O this means use GLM with alpha = 1.0) to do the feature selection, or simply allow whatever machine learning algorithm (e.g. GBM) you are planning to use to use all the features (they'll tend to ignore the bad ones, but it could still degrade performance of the algorithm to have bad features in the training data).
If you'd like, you can make a feature request by filling out a ticket on the H2O-3 JIRA. This seems like a nice feature to have.
In my opinion, Yes
My way is use automl to train your data.
after training, you can get a lot of model.
use h2o.get_model method or H2O server page to watch some model you like.
you can get VARIABLE IMPORTANCES frame.
then pick your features.