python 3 suds cache not working - caching

Question

I'm trying to write a script for accessing Sharepoint via Python.
The following libraries have been installed: suds.jurko, ntlm.
The following code succeeds, but takes close to 20 seconds:
#!/usr/bin/env python3
from suds.client import Client
from suds.transport.https import WindowsHttpAuthenticated
from suds.cache import ObjectCache
url = 'http://blah/_vti_bin/Lists.asmx?WSDL'
user = "blah"
passwd = "blah"
ntlm = WindowsHttpAuthenticated(username=user, password=passwd)
client = Client(url, transport=ntlm)
I tried adding cache:
oc = ObjectCache()
oc.setduration(days=10)
client = Client(url, transport=ntlm, cache=oc)
I see /tmp/suds created and I see cached files under there, but it looks like it just creates more files on every run, instead of using the cached files:
-rw-r--r-- 1 pchernik smsvcs 3 Feb 5 13:27 version
-rw-r--r-- 1 pchernik smsvcs 309572 Feb 5 13:27 suds-536283349122900148-document.px
-rw-r--r-- 1 pchernik smsvcs 207647 Feb 5 13:27 suds-4765026134651708722-document.px
-rw-r--r-- 1 pchernik smsvcs 21097 Feb 5 13:27 suds-1421279777216033364-document.px
-rw-r--r-- 1 pchernik smsvcs 207644 Feb 5 13:27 suds-6437332842122298485-document.px
-rw-r--r-- 1 pchernik smsvcs 309572 Feb 5 13:27 suds-3510377615213316246-document.px
-rw-r--r-- 1 pchernik smsvcs 21097 Feb 5 13:28 suds-7540886319990993060-document.px
-rw-r--r-- 1 pchernik smsvcs 207617 Feb 5 13:30 suds-1166110448227246785-document.px
-rw-r--r-- 1 pchernik smsvcs 309548 Feb 5 13:30 suds-2848176348666425151-document.px
-rw-r--r-- 1 pchernik smsvcs 21076 Feb 5 13:31 suds-6077994449274214633-document.px
Is suds normally this slow?
Any ideas on fixing the caching issues?
Are there any other python 3 libraries I can use for this instead of suds?
Any ideas / suggestions are appreciated.
Thanks,
-Pavel

Answer 1

I had the same issue, try setting your cachingpolicy to 1:
client = Client(url, transport=ntlm, cache=oc, cachingpolicy=1)
This will cache your WSDL objects instead of your XML files.
From suds documentation:
cachingpolicy
The caching policy, determines how data is cached. The default is 0. version 0.4+
0 = XML documents such as WSDL & XSD.
1 = WSDL object graph.
Edit: I re-read your question and realized I am missing something important; your cache is getting re-generated. I believe this is due to not specifying a location for the cache. This is from the documentation of the FileCache class in cache.py:
If no cache location is specified, a temporary default location will be
used. Such default cache location will be shared by all FileCache
instances with no explicitly specified location within the same
process. The default cache location will be removed automatically on
process exit unless user sets the remove_default_location_on_exit
FileCache class attribute to False.
So, even if you want to use the default cache location, you will need to explicitly define it when you create your cache object. This is what I've done in my code:
# Configure cache location and duration ('days=0' = infinite)
cache_dir = os.path.join(os.path.abspath(os.sep), r'tmp\suds')
self.cache = ObjectCache(cache_dir, days=0)
You could also try setting the remove_default_location_on_exit attribute as suggested in the FileCache documentation, but I have not tried this method.

Answer 2

I had the same issue, but I noticed the version of suds-jurko in pypi has the following function in reader.py that generates the name of the cache file:
def mangle(self, name, x):
"""
Mangle the name by hashing the I{name} and appending I{x}.
#return: the mangled name.
"""
h = abs(hash(name))
return '%s-%s' % (h, x)
In python3 hash adds a random seed to the string. This has been fixed in the current version of suds-jurko at https://bitbucket.org/jurko/suds/ by using hashlib/md5 instead. You could either install it from there instead of pypi or just edit your reader.py file and change it to
h = hashlib.md5(name.encode()).hexdigest()

Answer 3

Are you sure you are using suds-jourko? It resembles very much the issue described here:
Suds is not reusing cached WSDLs and XSDs, although I expect it to
You could profile your application or run it with logging enabled (like suggested in the linked question).
As an alternative you could try osa: https://pypi.python.org/pypi/osa/
Edit: Did not see you already had installed suds-jourko