How to provide the "result.has_errors()" and "result.has_validation_errors()" attributes when using import_data method for Django import_export - django-import-export

I need to make changes to a csv file being imported using the module import_Export for Django. I implement the import_data method for this but get the error 'Dataset' object has no attribute 'has_errors'
Traceback (most recent call last):
File "C:\Program Files\Python36\lib\site-packages\django\core\handlers\exception.py", line 34, in inner
response = get_response(request)
File "C:\Program Files\Python36\lib\site-packages\django\core\handlers\base.py", line 115, in _get_response
response = self.process_exception_by_middleware(e, request)
File "C:\Program Files\Python36\lib\site-packages\django\core\handlers\base.py", line 113, in _get_response
response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "C:\Program Files\Python36\lib\site-packages\django\utils\decorators.py", line 130, in _wrapped_view
response = view_func(request, *args, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\django\views\decorators\cache.py", line 44, in _wrapped_view_func
response = view_func(request, *args, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\django\contrib\admin\sites.py", line 231, in inner
return view(request, *args, **kwargs)
File "C:\Program Files\Python36\lib\site-packages\import_export\admin.py", line 316, in import_action
if not result.has_errors() and not result.has_validation_errors():
Exception Type: AttributeError at /admin/engine/mtg/import/
Exception Value: 'Dataset' object has no attribute 'has_errors'
How can I provide the result.has_error() and result.has_validation_errors() when I return my customized dataset to avoid this error? Here is where I implement the method
Admin
class ModelResource(resources.ModelResource):
def import_data(self, dataset, dry_run=False, raise_errors=True,use_transactions=None, collect_failed_rows=False, **kwargs):
new_dataset = do_stuff(dataset)
return new_dataset
The documentation states that the first thing that the import_data method does is create a "result" instance containing error information. I'm assuming this is what I need maybe? But I don't know how to gain access to it or return it with my new dataset[Import Data workflow][1]
[1]: https://django-import-export.readthedocs.io/en/latest/import_workflow.html?highlight=before)import

You shouldn't need to override has_errors() and has_validation_errors() because the logic should be handled for you.
Generally you shouldn't be over-riding import_data() because this is where the import logic happens. Pass a valid Dataset object into import_data() as the first arg. I suggest having a quick look at the source, because this will clarify what is going on.
If you need to modify the imported data, then there are several hooks you can use. This is where you subclass the base Resource and add your own logic.
This example is based on the django-import-export example app:
class BookResource(resources.ModelResource):
def before_import_row(self, row, row_number=None, **kwargs):
"""
Override to add additional logic.
"""
pass
def before_save_instance(self, instance, using_transactions, dry_run):
"""
Override to add additional logic.
"""
pass
class Meta:
model = Book
fields = ('id', 'author_email', 'price')
Then you can call this with:
rows = [
('book1', 'email#example.com', '10.25'),
('book2', 'email#example.com', '10.25'),
('book1', 'email#example.com', '10.25'),
]
dataset = tablib.Dataset(*rows, headers=['name', 'author_email', 'price'])
book_resource = BookResource()
result = book_resource.import_data(dataset)
print(result.totals)

Related

Django-rest-framework-social-oauth2

Exception happened during testing request -
curl -X POST -d "client_id=<client_id>&client_secret=
<client_secret>&grant_type=password&username=<user_name>&password=
<password>" http://localhost:8000/api/v1/auth/token
Traceback:
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/core/handlers/exception.py" in inner
41. response = get_response(request)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
187. response = self.process_exception_by_middleware(e, request)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/core/handlers/base.py" in _get_response
185. response = wrapped_callback(request, *callback_args, **callback_kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/views/generic/base.py" in view
68. return self.dispatch(request, *args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/utils/decorators.py" in _wrapper
67. return bound_func(*args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/views/decorators/csrf.py" in wrapped_view
58. return view_func(*args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/utils/decorators.py" in bound_func
63. return func.__get__(self, type(self))(*args2, **kwargs2)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/braces/views/_forms.py" in dispatch
24. return super(CsrfExemptMixin, self).dispatch(*args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
489. response = self.handle_exception(exc)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework/views.py" in handle_exception
449. self.raise_uncaught_exception(exc)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework/views.py" in dispatch
486. response = handler(request, *args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework_social_oauth2/views.py" in post
42. url, headers, body, status = self.create_token_response(request._request)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauth2_provider/views/mixins.py" in create_token_response
125. return core.create_token_response(request)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauth2_provider/oauth2_backends.py" in create_token_response
139. headers, extra_credentials)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/endpoints/base.py" in wrapper
64. return f(endpoint, uri, *args, **kwargs)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/endpoints/token.py" in create_token_response
117. request, self.default_token_type)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/grant_types/resource_owner_password_credentials.py" in create_token_response
100. self.validate_token_request(request)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauthlib/oauth2/rfc6749/grant_types/resource_owner_password_credentials.py" in validate_token_request
178. request.password, request.client, request):
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/oauth2_provider/oauth2_validators.py" in validate_user
520. u = authenticate(username=username, password=password)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/contrib/auth/__init__.py" in authenticate
68. for backend, backend_path in _get_backends(return_tuples=True):
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/contrib/auth/__init__.py" in _get_backends
29. backend = load_backend(backend_path)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/contrib/auth/__init__.py" in load_backend
23. return import_string(path)()
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/utils/module_loading.py" in import_string
20. module = import_module(module_path)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/importlib/__init__.py" in import_module
126. return _bootstrap._gcd_import(name[level:], package, level)
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework_social_oauth2/backends.py" in <module>
7. class DjangoOAuth2(BaseOAuth2):
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/rest_framework_social_oauth2/backends.py" in DjangoOAuth2
11. if DRFSO2_URL_NAMESPACE else 'authorize')
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/urls/base.py" in reverse
91. return force_text(iri_to_uri(resolver._reverse_with_prefix(view, prefix, *args, **kwargs)))
File "/Users/d.matveyev/.virtualenvs/dp/lib/python3.6/site-packages/django/urls/resolvers.py" in _reverse_with_prefix
497. raise NoReverseMatch(msg)
Exception Type: NoReverseMatch at /api/v1/auth/token
Exception Value: Reverse for 'authorize' not found. 'authorize' is not a valid view function or pattern name.
I'd like to point to several probably important facts:
I work with custom user model -
https://pastebin.com/yWSvVUbe
I work with rest-framework-jwt in my usual authentication (register and login) and I wanted to use drfso2 for the possibility of social authentication (facebook and google).
settings.py -
https://pastebin.com/nqnbNusT
v1/urls.py -
https://pastebin.com/vu9Lghrq
Thank you very much!!!
You have to set the DRFSO2_URL_NAMESPACEparameter on your setting.py file.
The URL namespace is the one where you placed your auth API.
Check your main urls.py file:
from django.contrib import admin
from django.urls import path, include
urlpatterns = [
path('admin/', admin.site.urls),
path('api/auth/', include('myproject.myauthapp.api.urls', namespace='auth-api')),
...
]
With my example, you have to set it to 'auth-api':
DRFSO2_URL_NAMESPACE='auth-api'
I am not sure what are the configurations of your settings and urls as the files with the code you provided as links to pastebin are apparently removed and no longer available, but I tackled and resolved the similar issue the following way.
If you use default urlconf setting for endpoints handling social authentication, this issue should not appear at all, so you can just use:
urlpatterns = patterns(
...
(r'^auth/', include('rest_framework_social_oauth2.urls')),
)
If you have some custom urlconf settings and you wish to place the endpoints for social authentication elsewhere, do not set the variable DRFSO2_URL_NAMESPACE in settings, just add the name parameter to your urls.py:
path('oauth/login/', SocialLoginView.as_view(), **name='authorize'**).
And the issue should disappear.
Hope it could help someone.

Unique Key Error when exporting Scrapy data to Elasticsearch

I'm attempting to use the scrapy elasticsearch pipeline (here: https://github.com/knockrentals/scrapy-elasticsearch) to put data into elasticsearch. however i get the following error, i'm aware that it's related to the ELASTICSEARCH_UNIQ_KEY value that is currently set at 'url' but i have no idea what it should be set to.
Similar posts on here recommend solutions that involve creating a field for the unique key but i don't understand what this means.
Here's my error message:
2015-08-05 11:34:40 [scrapy] ERROR: Error processing {'link': [u'http://www.meetup.com/Search-Meetup-Karlsruhe/events/192357732/'],
'title': [u'Suchen in der vernetzten Welt']}
Traceback (most recent call last):
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/twisted/internet/defer.py", line 588, in _runCallbacks
current.result = callback(current.result, *args, **kw)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapyelasticsearch/scrapyelasticsearch.py", line 70, in process_item
self.index_item(item)
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapyelasticsearch/scrapyelasticsearch.py", line 52, in index_item
local_id = hashlib.sha1(item[uniq_key]).hexdigest()
File "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/scrapy/item.py", line 56, in __getitem__
return self._values[key]
KeyError: 'url'
Similar posts on here recommend solutions that involve creating a
field for the unique key but i don't understand what this means.
Declare a field in your Item with the name you configured in the ELASTICSEARCH_UNIQ_KEY.
import scrapy
class DemoItem(scrapy.Item):
url = scrapy.Field() # ELASTICSEARCH_UNIQ_KEY
class DemoSpider(scrapy.Spider):
name = 'demo'
start_urls = ['http://www.example.com']
def parse(self, response):
demoItem = DemoItem()
demoItem['url'] = response.url
yield demoItem

Error when I Update Module List in OpenERP

I am trying to update my modulo list using the Update Module List menu item, but I get the follwing error:
OpenERP Server Error
Client Traceback (most recent call last):
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/http.py", line 204, in dispatch
response["result"] = method(self, **self.params)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/controllers/main.py", line 1132, in call_button
action = self._call_kw(req, model, method, args, {})
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/controllers/main.py", line 1120, in _call_kw
return getattr(req.session.model(model), method)(*args, **kwargs)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/session.py", line 42, in proxy
result = self.proxy.execute_kw(self.session._db, self.session._uid, self.session._password, self.model, method, args, kw)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/session.py", line 30, in proxy_method
result = self.session.send(self.service_name, method, *args)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/session.py", line 103, in send
raise xmlrpclib.Fault(openerp.tools.ustr(e), formatted_info)
Server Traceback (most recent call last):
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/session.py", line 89, in send
return openerp.netsvc.dispatch_rpc(service_name, method, args)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/netsvc.py", line 296, in dispatch_rpc
result = ExportService.getService(service_name).dispatch(method, params)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/service/web_services.py", line 626, in dispatch
res = fn(db, uid, *params)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/osv/osv.py", line 190, in execute_kw
return self.execute(db, uid, obj, method, *args, **kw or {})
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/osv/osv.py", line 132, in wrapper
return f(self, dbname, *args, **kwargs)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/osv/osv.py", line 199, in execute
res = self.execute_cr(cr, uid, obj, method, *args, **kw)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/osv/osv.py", line 187, in execute_cr
return getattr(object, method)(cr, uid, *args, **kw)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/base/module/wizard/base_module_update.py", line 42, in update_module
update, add = module_obj.update_list(cr, uid,)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/base/module/module.py", line 617, in update_list
handler.load_addons()
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/web/http.py", line 580, in load_addons
m = __import__('openerp.addons.' + module)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/modules/module.py", line 133, in load_module
mod = imp.load_module('openerp.addons.' + module_part, f, path, descr)
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/magento_integration-develop/__init__.py", line 9, in <module>
import magento_
File "/opt/bitnami/apps/openerp/lib/openerp-7.0_20140330_231328-py2.7.egg/openerp/addons/magento_integration-develop/magento_.py", line 17, in <module>
import magento
ImportError: No module named magento
I am trying to install a Magento OpenERP connector, but in order to to that I must locate it in the Installed Module list.
Thanks
Please check your file magento_.py, and see whether the class has been called.And if it has been called correctly check whether it has been correctly specified in the import line in your init.py file.

TypeError: hasattr(): attribute name must be string in pymc

I have looked at the following links but none of them provide the solution I am looking for
https://github.com/pymc-devs/pymc/issues/125
PyMC error : hasattr(): attribute name must be string
I have to write a function which given the priors (and other stuff like data etc) returns a pymc model.
eg
m = pym.Model([fittable_params.values(), rv])
return m
and the in the calling function, when I do mcmc = pymc.MCMC(model)
It gives a long error
Traceback (most recent call last):
File "model_constructor.py", line 81, in <module>
mcmc = pm.MCMC(model)
File "/usr/local/lib/python2.7/dist-packages/pymc-2.3.2-py2.7-linux-i686.egg/pymc/MCMC.py", line 81, in __init__
**kwds)
File "/usr/local/lib/python2.7/dist-packages/pymc-2.3.2-py2.7-linux-i686.egg/pymc/Model.py", line 195, in __init__
Model.__init__(self, input, name, verbose)
File "/usr/local/lib/python2.7/dist-packages/pymc-2.3.2-py2.7-linux-i686.egg/pymc/Model.py", line 98, in __init__
ObjectContainer.__init__(self, input)
File "/usr/local/lib/python2.7/dist-packages/pymc-2.3.2-py2.7-linux-i686.egg/pymc/Container.py", line 605, in __init__
conservative_update(self, input_to_file)
File "/usr/local/lib/python2.7/dist-packages/pymc-2.3.2-py2.7-linux-i686.egg/pymc/Container.py", line 548, in conservative_update
if not hasattr(obj, k):
TypeError: hasattr(): attribute name must be string
On the other hand , if in the function (which returns a model), if I do
m = pm.MCMC([fittable_params.values(), rv])
it is running fine, but the function should return a model so that the user can do whatever he wants with the model in other parts of code.
If the linked solutions don't work for you then as a last resort you can just delete the non-string attributes from the model, since they don't seem to be used anyway.
for key in m.__dict__.keys():
if not isinstance(key, basestring):
del m.__dict__[key]

gdata.docs.client.DocsClient

I have the following code, reads oauth2 token form file, then try's to perform a doc's list query to find a specific spreadsheet that I want to copy, however no matter what I try the code either errors out or returns with an object containing no document data.
I am using gdata.docs.client.DocsClient which as far as I can tell is version 3 of the API
def CreateClient():
"""Create a Documents List Client."""
client = gdata.docs.client.DocsClient(source=config.APP_NAME)
client.http_client.debug = config.DEBUG
# Authenticate the user with CLientLogin, OAuth, or AuthSub.
if os.path.exists(config.CONFIG_FILE):
f = open(config.CONFIG_FILE)
tok = pickle.load(f)
f.close()
client.auth_token = tok.auth_token
return client
1st query attempt
def get_doc():
new_api_query = gdata.docs.client.DocsQuery(title='RichSheet', title_exact=True, show_collections=True)
d = client.GetResources(q = new_api_query)
this fails with the following stack trace
Traceback (most recent call last):
File "/Users/richard/PycharmProjects/reportone/make_my_report.py", line 83, in <module>
get_doc()
File "/Users/richard/PycharmProjects/reportone/make_my_report.py", line 57, in get_doc
d = client.GetResources(q = new_api_query)
File "/Users/richard/PycharmProjects/reportone/gdata/docs/client.py", line 151, in get_resources
**kwargs)
File "/Users/richard/PycharmProjects/reportone/gdata/client.py", line 640, in get_feed
**kwargs)
File "/Users/richard/PycharmProjects/reportone/gdata/docs/client.py", line 66, in request
return super(DocsClient, self).request(method=method, uri=uri, **kwargs)
File "/Users/richard/PycharmProjects/reportone/gdata/client.py", line 267, in request
uri=uri, auth_token=auth_token, http_request=http_request, **kwargs)
File "/Users/richard/PycharmProjects/reportone/atom/client.py", line 115, in request
self.auth_token.modify_request(http_request)
File "/Users/richard/PycharmProjects/reportone/gdata/gauth.py", line 1047, in modify_request
token_secret=self.token_secret, verifier=self.verifier)
File "/Users/richard/PycharmProjects/reportone/gdata/gauth.py", line 668, in generate_hmac_signature
next, token, verifier=verifier)
File "/Users/richard/PycharmProjects/reportone/gdata/gauth.py", line 629, in build_oauth_base_string
urllib.quote(params[key], safe='~')))
File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/urllib.py", line 1266, in quote
if not s.rstrip(safe):
AttributeError: 'bool' object has no attribute 'rstrip'
Process finished with exit code 1
then my second attempt
def get_doc():
other = gdata.docs.service.DocumentQuery(text_query='RichSheet')
d = client.GetResources(q = other)
this returns an ResourceFeed object, but has no content. I have been through the source code for these function but thing are not any obvious.
Have i missed something ? or should i go back to version 2 of the api ?

Resources