When I fine tune a pretrained resnet152 model, I seem to lose all the named layers I’d like access to. I’ve include the simple fine tuned model code, and the print out of named layers of both pretrained and fine tuned.I'd like to maintain the layer names so I can visualize their output in a Class Activation Map.
Code
class ConvNet3(nn.Module):
def init(self):
super().init()
model = models.resnet152(pretrained=True)
model.fc = nn.Linear(2048, 10)
self.model = model
def forward(self, x):
return self.model(x) # [batch_size, 10]
import torchvision.models as models
model = ConvNet3().eval()
print([n for n, _ in model.named_children()])
model = models.resnet152(pretrained=True).eval()
print([n for n, _ in model.named_children()])
Output
[‘model’]
[‘conv1’, ‘bn1’, ‘relu’, ‘maxpool’, ‘layer1’, ‘layer2’, ‘layer3’, ‘layer4’, ‘avgpool’, ‘fc’]
The layers are not lost, you are encapsulating the original Resnet model in your own class. If you use:
print([n for n, _ in model.model.named_children()])
since the Resnet model is stored under the model attribute of the ConvNet3 class.
Unless you need it for another reason, the wrapper class seems unnecessary, a simpler approach would be to do something as follows:
model = models.resnet152(pretrained=True)
model.fc = nn.Linear(2048,10)
model.eval()
print([n for n, _ in model.named_children()])
Related
I want to use .generate() functionality of hugging face in my model's predictions.
My model is a custom model inehriting from "TFPreTrainedModel" class and has a custom transformer inheriting from tf.keras.layers followed by few hidden layers and a final dense layer (inherited from tf.keras.layers).
I am not able to use .generate() inspite of adding get_lm_head() function (as given here https://huggingface.co/docs/transformers/main_classes/model) and returning my last dense layer in it.
When I call .generate() it throws
TypeError: The current model class (NextCateModel) is not compatible with.generate(), as it doesn't have a language model head.
Can anyone suggest on how to use .generate() functionality of huggingface in our custom transformer based models without using the huggingface's list of pre-trained models?
PS: It checks for models among huggingface pretrained ones which are defined in their generation_tf_utils.py
generate_compatible_mappings = [
TF_MODEL_FOR_CAUSAL_LM_MAPPING,
TF_MODEL_FOR_VISION_2_SEQ_MAPPING,
TF_MODEL_FOR_SEQ_TO_SEQ_CAUSAL_LM_MAPPING,
TF_MODEL_FOR_SPEECH_SEQ_2_SEQ_MAPPING,
]
I donot intend to use their pretrained models given in above mappings (one of them is shown below)
TF_MODEL_FOR_CAUSAL_LM_MAPPING=
("bert", "TFBertLMHeadModel"),
("camembert", "TFCamembertForCausalLM"),
("ctrl", "TFCTRLLMHeadModel"),
("gpt2", "TFGPT2LMHeadModel"),
("gptj", "TFGPTJForCausalLM"),
("openai-gpt", "TFOpenAIGPTLMHeadModel"),
("opt", "TFOPTForCausalLM"),
("rembert", "TFRemBertForCausalLM"),
("roberta", "TFRobertaForCausalLM"),
("roformer", "TFRoFormerForCausalLM"),
("transfo-xl", "TFTransfoXLLMHeadModel"),
("xglm", "TFXGLMForCausalLM"),
("xlm", "TFXLMWithLMHeadModel"),
("xlnet", "TFXLNetLMHeadModel"),
1340 if generate_compatible_classes:
1341 exception_message += f" Please use one of the following classes instead: {generate_compatible_classes}"
-> 1342 raise TypeError(exception_message)
I'm trying to figure out if there's a way to return one attribute of a nested object when the attribute is addressed using the 'dot' notation, but to return different attributes of that object when subsequent attributes are requested.
ex)
class MyAttributeClass:
def __init__(self, value):
self.value = value
from datetime.datetime import now
self.timestamp = now()
class MyOuterClass:
def __init__(self, value):
self._value = MyAttributeClass(value)
test = MyOuterClass(5)
test.value (should return test._value.value)
test.value.timestamp (should return test._value.timestamp)
Is there any way to accomplish this? I imagine, if there is one, it involves defining the __getattr__ method of MyOuterClass, but I've been searching around and I haven't found any way to do this. Is it just impossible? It's not a big deal if it can't be done, but I've wanted to do this many times and I'd just like to know if there's a way.
It seems obvious now, but inheritance was the answer. Defining attributes as instances of a custom class which inherits from the datatype I wanted for ordinary attribute access (i.e. object.attr) and giving this subclass the desired attributes for subsequent attribute requests (i.e. object.attr.subattr).
I am unable to send object to direct view workers.
Here is what I want to do:
class Test:
def __init__(self):
self.id = 'ddfdf'
from IPython.parallel import Client
rc = Client()
dv = rc[:]
t = Test()
dv['t'] = t
print dv['t']
NameError: name 't' is not defined
This would work if I try to push pandas object or any of the build in objects.
What is the way to do it with custom object?
I tried:
dv['Test'] = Test
dv['t'] = t
print dv['t']
UnpicklingError: NEWOBJ class argument isn't a type object
For interactively defined classes (in __main__), you do need to push the class definition, or use dill. But even this doesn't appear to work for old-style classes, which is a bug in IPython's handling of old-style classes[1]. This code works fine if you use a new-style class:
class Test(object):
...
instead of on old-style class. Note that old-style classes are not available in Python 3.
It's generally a good idea to always use new-style classes anyway.
I'm pretty sure the answer to this question is obviously "NO", since Django mixins are supposed to
inherit "object"s, but I can't find an alternative solution to my problem :(
To make the question as simple as possible,,,
views.py
class JSONResponseMixin(object):
def render_to_response(self, context):
"Returns a JSON response containing 'context' as payload"
return self.get_json_response(self.convert_context_to_json(context))
def get_json_response(self, content, **httpresponse_kwargs):
"Construct an `HttpResponse` object."
return http.HttpResponse(content,
content_type='application/json',
**httpresponse_kwargs)
def convert_context_to_json(self, context):
"Convert the context dictionary into a JSON object"
# Note: This is *EXTREMELY* naive; in reality, you'll need
# to do much more complex handling to ensure that arbitrary
# objects -- such as Django model instances or querysets
# -- can be serialized as JSON.
return json.dumps(context)
class HandlingAJAXPostMixin(JSONResponseMixin):
def post(self, request, *args, **kwargs):
.....
data = {'somedata': somedata}
return JSONResponseMixin.render_json_response(data)
class UserDetailView(HandlingAJAXPostMixin, DetailView):
model = MyUser
.....
So the problem I have is that, for multiple Views, I want to respond to their "post" request with the same
JSON Response. That is why I defined the HandlingAJAXPostMixin so that I could reuse it for
other Views. Since the HandlingAJAXPostMixin returns a JSON response,
it requires a render_json_response method, which is defined in the JSONResponseMixin.
This is the reason why I am making my HandlingAJAXPostMixin inherit the JSONResponseMixin, but this obviously seems wrong :(..
Any suggestions..?
Thanks!!!
It's perfectly valid for a mixin to inherit from another mixin - in fact, this is how most of Django's more advanced mixins are made.
However, the idea of mixins is that they are reusable parts that, together with other classes, build a complete, usable class. Right now, your JSONResponseMixin might as well be a separate class that you don't inherit from, or the methods might just be module-wide methods. It definitely works, there's nothing wrong with it, but that's not the idea of a mixin.
If you look at Django's BaseDetailView, you see the following get() method:
def get(self, request, *args, **kwargs):
self.object = self.get_object()
context = self.get_context_data(object=self.object)
return self.render_to_response(context)
get_object() and get_context_data() are defined in the subclasses of BaseDetailView, but render_to_response() isn't. It's okay for mixins to rely on methods that it's superclasses don't define, this allows different classes that inherit from BaseDetailView to supply their own implementation of render_to_response(). Right now, in Django, there's only one subclass, though.
However, logic is delegated as much as possible to those small, reusable methods that the mixins supply. That's what you want to aim for. If/else logic is avoided as much as possible - the most advanced logic in Django's default views is:
if form.is_valid():
return self.form_valid(form)
else:
return self.form_invalid(form)
That's why very similar views, like CreateView and UpdateView are in fact two separate views, while they could easily be a single view with some additional if/else logic. The only difference is that CreateView does self.object = None, while UpdateView does self.object = self.get_object().
Right now you are using a DetailView that defines a get() method that returns the result of self.render_to_response(). However, you override render_to_response() to return a JSON response instead of a template-based HTML response. You're using a mixin that you don't what to use (SingleObjectTemplateResponseMixin) and then override it's behavior to do something that you don't want to do either, just to get the view doing what you want it to do. A better idea would be to write an alternative for DetailView who's only job is to supply a JSON response based on a single object. To do this, I would create a SingleObjectJSONResponseMixin, similar to the SingleObjectTemplateResponseMixin, and create a class JSONDetailView that combines all needed mixins into a single object:
class SingleObjectJSONResponseMixin(object):
def to_json(context):
return json.dumps(context)
def render_to_response(context, **httpresponse_kwargs):
return HttpResponse(self.to_json(context),
context_type='application/json',
**httpresponse_kwargs)
class BaseJSONDetailView(SingleObjectMixin, View):
# if you want to do the same for get, inherit just from BaseDetailView
def post(self, request, *args, **kwargs):
self.object = self.get_object()
context = self.get_context_data(object=self.object)
return render_to_response(context)
class JSONDetailView(SingleObjectJSONResponseMixin, BaseJSONDetailView):
"""
Return JSON detail data of a single object.
"""
Notice that this is almost exactly the same as the BaseDetailView and the SingleObjectTemplateResponseMixin provided by Django. The difference is that you define a post() method and that the rendering is much more simple with just a conversion to JSON of the context data, not a complete template rendering. However, logic is deliberately kept simple as much as possible, and methods that don't depend on each other are separated as much as possible. This way, SingleObjectJSONResponseMixin can e.g. be mixed with BaseUpdateView to easily create an AJAX/JSON-based UpdateView. Subclasses can easily override the different parts of the mixins, like overriding to_json() to supply a certain data structure. Rendering logic is where it belongs (in render_to_response()).
Now all you need to do to create a specific JSONDetailView is to subclass and define which model to use:
class UserJSONDetailView(JSONDetailView):
model = MyUser
I would like to write a subclass of pandas.core.index.Index. I'm following the guide to subclassing ndarrays which can be found in the numpy documentation. Here's my code:
import numpy as np
import pandas as pd
class InfoIndex(pd.core.index.Index):
def __new__(subtype, data, info=None):
# Create the ndarray instance of our type, given the usual
# ndarray input arguments. This will call the standard
# ndarray constructor, but return an object of our type.
# It also triggers a call to InfoArray.__array_finalize__
obj = pd.core.index.Index.__new__(subtype, data)
# set the new 'info' attribute to the value passed
obj.info = info
# Finally, we must return the newly created object:
return obj
However, it doesn't work; I only get a Index object:
In [2]: I = InfoIndex((3,))
In [3]: I
Out[3]: Int64Index([3])
What am I doing wrong?
Index constructor tries to be clever when the inputs are special (all ints or datetimes for example) and skips to calls to view at the end. So you need to put that in explicitly:
In [150]: class InfoIndex(pd.Index):
.....: def __new__(cls, data, info=None):
.....: obj = pd.Index.__new__(cls, data)
.....: obj.info = info
.....: obj = obj.view(cls)
.....: return obj
.....:
In [151]: I = InfoIndex((3,))
In [152]: I
Out[152]: InfoIndex([3])
Caveat emptor: be careful subclassing pandas objects as many methods will explicitly return Index as opposed to the subclass. And there are also features in sub-classes of Index that you'll lose if you're not careful.
If you implement the __array_finalize__ method you can ensure that metadata is preserved in many operations. For some index methods you'll need to provide implementations in your subclass. See http://docs.scipy.org/doc/numpy/user/basics.subclassing.html for a bit more help
To expand on the previous answers. You can also preserve most index methods, if you use the _constructor property and set _infer_as_myclass = True.