I want to add a layer in tensorflow model that preprocess the input data before its feeded to the neural network, and my goal is when i save the model with model.save, i can call it again give it the base64 and it automatically converts it to image and give prediction
I have images saved after they were converted to base 64 with this code
import cv2
im = cv2.imread("anyimage.jpg")
_, buffer = cv2.imencode('.jpg', im)
jpg_as_text = base64.b64encode(buffer)
value=jpg_as_text.decode('utf-8'
Example
value = '/9j/4AAQSkZJRgABAQEAYABgAAD/4QBGRXhpZgAATU0AKgAAAAgABwESAAMAAAABAAEAAAEaAAUAAAABAAAAYgEbAAUAAAABAAAAagEoAAMAAAABAAIAAAExAAIAAAAQAAAAcgEyAAIAAAAUAAAAlIdpAAQAAAABAAAApAAAANAACvyAAAAnEAAK/IAAACcQQWRvYmUgUGhvdG9zaG9wIENTMyBXaW5kb3dzADIwMTY6MDg6MzAgMTM6Mzc6MjA="
So i need to build the decoder of that value, but i need it inside the model, i model.save it with architecture of the model
I already have decoder in a function but as i mentioned i want to know if i can somehow add it to the model as a layer
Related
I am trying to use the huggingface implementation of the vision transformer to get the feature vector of the last but one dense layer
In order to get information from the second last layer, you need to output_hidden_states=True. Here is an example in my context:
configBert = BertConfig.from_pretrained('bert-base-uncased', output_hidden_states=True, num_labels=NUM_LABELS)
modelBert = TFBertModel.from_pretrained('bert-base-uncased', config=configBert)
I would like to train a encoder decoder model as configured below for a translation task. Could someone guide me as to how I can set-up a training pipeline for such a model? Any links or code snippets would be appreciated to understand.
from transformers import BertConfig, EncoderDecoderConfig, EncoderDecoderModel
# Initializing a BERT bert-base-uncased style configuration
config_encoder = BertConfig()
config_decoder = BertConfig()
config = EncoderDecoderConfig.from_encoder_decoder_configs(config_encoder, config_decoder)
# Initializing a Bert2Bert model from the bert-base-uncased style configurations
model = EncoderDecoderModel(config=config)
The encoder-decoder models are used in the same as any other models in Transformers. It accepts batches of tokenized text as vocabulary indices (i.e., you need a tokenizer that is suitable for your sequence-to-sequence task). When you feed the model with the input (input_ids) and the desired output (decoder_input_ids and labels), you will get the loss value that you can optimize during training. Note that if the sentences in the batch have different lengths, you need to do masking too. This is a minimum example for the EncoderDecoderModel documentation:
from transformers import EncoderDecoderModel, BertTokenizer
import torch
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = EncoderDecoderModel.from_encoder_decoder_pretrained(
'bert-base-uncased', 'bert-base-uncased')
input_ids = torch.tensor(
tokenizer.encode("Hello, my dog is cute", add_special_tokens=True)).unsqueeze(0)
outputs = model(
input_ids=input_ids, decoder_input_ids=input_ids, labels=input_ids,
return_dict=True)
loss = outputs.loss
If you do not want to write the training loop yourself, you can use dataset processing (DataCollatorForSeq2Seq) and training (Seq2SeqTrainer) utilities from Transformers. You can follow the Seq2Seq example on GitHub.
In Gensims word2vec api, I trained a model where I initialized the model with max_final_vocab = 100000 and saved the model using model.save()
(This gives me one .model file, one .model.trainables.syn1neg.npy and one .model.wv.vectors.npy file).
I do not need to train model any further, so I'm fine with using just
model = gensim.models.Word2Vec.load("train.fr.model")
kv = model.wv
del model
the kv variable shown here. I now want to use only the top N (N=40000 in my case) vocabulary items instead of the entire vocabulary. The only way to even attempt cutting down the vocabulary I could find was
import numpy as np
emb_matrix = np.load("train.fr.model.wv.vectors.npy")
emb_matrix.shape
# (100000, 300)
new_emb_matrix = emb_matrix[:40000]
np.save("train.fr.model.wv.vectors.npy", new_emb_matrix)
If I load this model again though, the vocabulary still has length 100000.
I want to reduce the vocabulary of the model or model.wv while retaining a working model. Retraining is not an option.
from gensim.models import KeyedVectors
model = KeyedVectors.load_word2vec_format('train.fr.model', limit=1000)
Use optional limitparameter to reduce number of vectors that will be loaded from Word2Vec model file.
ImageProcessor ImageFactory supports Load method with argument System.Drawing.Image. But Save method does not support similar argument. Is there a way with ImageProcessor ImageFactory to Load a System.Drawing.Image, manipulate the image, e.g., Brightness, then Save the modified image to System.Drawing.Image?
You can't use a reference to the original image since that is tightly bound to the input stream.
You can either.
1. Save to the output stream and use Image.FromStream to create a new image.
2. Use a binary formatter to perform a deep clone of the ImageFactory.Image property https://stackoverflow.com/a/43042865/427899
I am generating images on fly from DICOM files using:
public ActionResult Generatemage()
{
FileContentResult data;
.....
objImage = im.Bitmap(outputSize, PixelFormat.Format24bppRgb, m);
using (var memStream = new MemoryStream())
{
objImage.Save(memStream, ImageFormat.Png);
data = this.File(memStream.GetBuffer(), "image/png");
}
return data;
}
Can I store the image as a session variable so I can modify it using Point3D?
I tried to use:
Bitmap data = (Bitmap)Session["newimage"];
Got these two errors:
Cannot implicitly convert type 'System.Drawing.Bitmap' to 'System.Web.Mvc.FileContentResult' and
A local variable named 'data' is already defined in this scope
I would appreciate your suggestions, thanks in advance.
Can I store the image as a session variable so I can modify it using
Point3D?
I suggest to not do that. If you have not read Nathanael's post on image resizing pitfalls then I suggest you do so now. It may be talking about resizing but it also give hints on working with images in general. On point #3 it says:
Serving a file from disk by loading it into memory. Think about how
much RAM your server has, how large a single image is, how long it has
to stay in memory before users fi downloading it, and how many
users you have requesting images.
In your particular case you can replace "before users finish downloading it" with "before Point3D finish processing the image". So, what I suggest is that you get a handle to that file, say maybe there's an Id that uniquely identifies a file per user, use that Id to retrieve the file when it's time to process it with Point3D, load it into a MemoryStream (assuming Point3D can work with mem. stream), process it, then dispose of it. In that manner you are only holding on to the image for the duration of "Point3D processing".
Cannot implicitly convert type 'System.Drawing.Bitmap' to
'System.Web.Mvc.FileContentResult' and A local variable named 'data'
is already defined in this scope
That is most probably because you have defined data as such:
FileContentResult data;
and then you are doing a:
Bitmap data = (Bitmap)Session["newimage"];
same variable of two different types within the same scope.