I have a Python script that accesses some files, form example one line of code appends some text to a .txt file.
It runs correctly in a normal cmd.exe, with or without Admnistrator access. But while using a Anaconda Prompt or a cmd.exe opened from Anaconda Navigator it gives a PermissionError: [Errno 13] Permission denied: in the open('path/to/file', 'a+') line.
I'm looking for a solution for why Anaconda's python cannot access files. Another line of code using the function handlers.TimedRotatingFileHandler() from the logging module also throws the same error.
Some info that might be useful:
I have two Anaconda environments, one with python 2.7 and another one with python 3.6.6, both throw the error;
I have installed Anaconda 3 without adding anaconda to my PATH, because it was recommended in the installer;
My PATH's python, the version that works in the non-anaconda cmd.exe, is also 3.6.6;
I use Windows 10.
Edit:
Okay, so there seems to be some confusion about what I'm trying to accomplish. I'm making a text file with every .jpg file in a folder recursively, followed by the split of the file name. The text file should have: "path/to/XY_file_XY.jpg file" for every file found. Those splits represent the labels of the images for a neural network I'm training.
Here is my code for creating this text file:
import glob
class DBSetter(object):
def __init__(self, rootPath = 'C:\\Users\\Victor.Lundgren\\Google Drive\\Mestrado\\VC\\Projeto\\data\\mnt'):
self.path = rootPath
return
def setDB(self, extension = '.jpg', splitFileName = True, splitDilimiter = '_', splitPosition = 1):
query = self.path+'\\**\\*'+extension
print(query)
fileList = glob.glob(query, recursive = True)
sampleFileText = ''
for file in fileList:
word = file.split('\\')[-1]
if splitFileName:
word = word.split(splitDilimiter)[splitPosition]
sampleFileText += file+' '+word+'\n'
print('Sample File Text created.')
sampleFile = open(self.path+'\\sample.txt', 'a+')
sampleFile.write(sampleFileText)
sampleFile.close()
return
For instance, let's create a DBSetter object in a main.py and test ir with the default Python (3.6.6) in a simple cmd.exe and with Anaconda's Python (3.6.6) in an Anaconda Prompt:
from utils import database_setter
setter = database_setter.DBSetter('C:\\Users\\Victor.Lundgren\\Pictures\\Trabalho\\ASA') #choosing this folder for this example because it has few files in it.
setter.setDB(extension = '.png', splitFileName = False)
Here we have our cmd history in a simple cmd.exe calling main.py:
> Microsoft Windows [versão 10.0.16299.492] (c) 2017 Microsoft
> Corporation. Todos os direitos reservados. Clink v0.4.9 [git:2fd2c2]
> Copyright (c) 2012-2016 Martin Ridgers http://mridgers.github.io/clink
>
>
> C:\Users\Victor.Lundgren>cd C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto
>
> C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto>Python
> __main__.py C:\Users\Victor.Lundgren\Pictures\Trabalho\ASA\**\*.png Sample File Text created.
>
> C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto>
Now executing Anaconda Promp from the windows menu and runnning the same thing with my Python 3.6.6 env:
> Clink v0.4.9 [git:2fd2c2] Copyright (c) 2012-2016 Martin Ridgers
> http://mridgers.github.io/clink
>
>
> (base) C:\Users\Victor.Lundgren>cd C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto
>
> (base) C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto>activate Py3
>
> (Py3) C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto>Python
> __main__.py C:\Users\Victor.Lundgren\Pictures\Trabalho\ASA\**\*.png Sample File Text created. Traceback (most recent call last): File
> "__main__.py", line 6, in <module>
> setter.setDB(extension = '.png', splitFileName = False) File "C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto\utils\database_setter.py", line 19, in setDB
> sampleFile = open(self.path+'\\sample.txt', 'a+') PermissionError: [Errno 13] Permission denied:
> 'C:\\Users\\Victor.Lundgren\\Pictures\\Trabalho\\ASA\\sample.txt'
>
> (Py3) C:\Users\Victor.Lundgren\Google Drive\Mestrado\VC\Projeto>
After multiple attempts on reinstalling Anaconda I discovered the issue. My antivirus (Bitdefender) was exclusively blocking the Python version installed by Anaconda and for some reason not showing any notifications of it being blocked, removing the block made it run correctly.
Related
Im really new to python so if anyone could explain this issue to me with a fix it is much appreciated!!
I have created a web scraping bot using python and selenium, however, I am faced with this error and I think it has got to do with my Path and chrome driver.
This is the error I'm faced with:
/Users/jonathan/venvs/env310/bin/python3.10 "/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py" --multiproc --qt-support=auto --client 127.0.0.1 --port 64298 --file /Users/jonathan/PycharmProjects/Seleniumlearning/BBDC_fullauto.py
/Applications/PyCharm CE.app/Contents/plugins/python-ce/helpers/pydev/pydevd.py:1844: DeprecationWarning: currentThread() is deprecated, use current_thread() instead
dummy_thread = threading.currentThread()
Connected to pydev debugger (build 212.5284.44)
Traceback (most recent call last):
File "/Users/jonathan/venvs/env310/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 74, in start
self.process = subprocess.Popen(cmd, env=self.env,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 966, in __init__
self._execute_child(args, executable, preexec_fn, close_fds,
File "/Library/Frameworks/Python.framework/Versions/3.10/lib/python3.10/subprocess.py", line 1842, in _execute_child
raise child_exception_type(errno_num, err_msg, err_filename)
FileNotFoundError: [Errno 2] No such file or directory: 'chromedriver'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/jonathan/venvs/env310/lib/python3.10/site-packages/selenium/webdriver/common/service.py", line 84, in start
raise WebDriverException(
selenium.common.exceptions.WebDriverException: Message: 'chromedriver' executable needs to be in PATH. Please see https://chromedriver.chromium.org/home
python-BaseException
Thus, in order to try and fix it...
I read an online tutorial on how to change your path using vi ~/.bash_profile, and have added the file containing my chrome driver (~/drivers) using the code export PATH=$PATH:~/drivers.
> export PATH=$PATH:~/drivers Setting PATH for Python 3.10
> # The original version is saved in .bash_profile.pysave PATH="/Library/Frameworks/Python.framework/Versions/3.10/bin:${PATH}"
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> ~
> "~/.bash_profile" 4L, 184B
After that, I proceeded to source ~/.bash_profile but it gave me this message - /Users/jonathan/.bash_profile:2: command not found: Setting. Apparently, in the tutorial, this message isn't supposed to come up and its to refresh the page
However, I could still see the path to my driver being added when I echo $PATH:
/Library/Frameworks/Python.framework/Versions/3.10/bin:/Library/Frameworks/Python.framework/Versions/3.10/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/Users/jonathan/drivers
So, since I added it already, why does it still give me the error mentioned above?
You need a # before your comment: # Setting PATH for..
You were seeing directories in your path from when you previously sourced this file. Try exiting and restarting the terminal emulator.
CEF :
Branch : 2987
Terminal Commands for Generating binary distrib files after all the data gets downloaded
$ export GYP_DEFINES=proprietary_codecs=1 ffmpeg_branding=Chrome
$ python /Users/imfinity/Documents/CEF_2987/automate/automate-git.py --download-dir=/Users/imfinity/Documents/CEF_2987/download --branch=2987 --x64-build --force-config --force-build
$ cd /Users/imfinity/-dir/chromium/src/cef/tools
$ ./make_distrib.sh --ninja-build
ERROR : Traceback (most recent call last):
File "make_distrib.py", line 468, in
raise Exception('Missing generated header file: %s' % include)
Exception: Missing generated header file: cef_pack_resources.h
This leads to creation of : Incomplete Folder : cef_binary_3.2987.1574.g4232c4c_macosx32
Any help is appreciated!!
I tried this command
$ export GYP_DEFINES=proprietary_codecs=1 ffmpeg_branding=Chrome
$ python /Users/imfinity/Documents/CEF_20March/automate/automate-git.py --download-dir /Users/imfinity/Documents/CEF_20March/download --branch=2987 --x64-build --force-config
and finally it worked and 64 bit binaries were generated
still surprised it worked after 5 different attempts!!!
I created the following Python 3.5 script:
import sys
from pathlib import Path
def test_unlink():
log = Path('log.csv')
fails = 0
num_tries = int(sys.argv[1])
for i in range(num_tries):
try:
log.write_text('asdfasdfasdfasdfasdfasdfasdfasdf')
with log.open('r') as logfile:
lines = logfile.readlines()
# Check the second line to account for the log file header
assert len(lines) == 1
log.unlink()
not log.exists()
except PermissionError:
sys.stdout.write('! ')
sys.stdout.flush()
fails += 1
assert fails == 0, '{:%}'.format(fails / num_tries)
test_unlink()
and run it like this: python test.py 10000. On Windows 7 Pro 64 with Python 3.5.2, the failure rate is not 0: it is small, but non-zero. Sometimes, it is not even that small: 5%! If you printout the exception, it will be this:
PermissionError: [WinError 5] Access is denied: 'C:\\...\\log.csv'
but it will sometimes occur at the exists(), other times at the write_text(), and I wouldn't be surprised if it happens at the unlink() and the open() too.
Note that the same script, same Python (3.5.2), but on linux (through http://repl.it/), does not have this issue: failure rate is 0.
I realize that a possible workaround could be:
while True:
try: log.unlink()
except PermissionError: pass
else: break
but this is tedious and error prone (several methods on Path instance would need this, easy to forget), and should (IMHO) not be necessary, so I don't think it is practical solution.
So, does anyone have an explanation for this, and a practical workaround, maybe a mode flag somewhere that can be set when Python starts?
I'm trying to work through getting started with ansible. When I try to run ad hoc commands with ansible, such as ansible all -m ping I have to either be in the same folder as my hosts file ~/Development/ansible/ or use sudo. I'm getting ready to move forward with learning playbooks but I feel like this is going to come back as a problem at some point.
If I try to run the command outside of that folder without sudo I get this message
<XXX.XXX.XXX.XXX> ESTABLISH CONNECTION FOR USER: Joe
<XXX.XXX.XXX.XXX> REMOTE_MODULE ping
server | FAILED => Traceback (most recent call last):
File "/Library/Python/2.7/site-packages/ansible/runner/__init__.py", line 586, in _executor
exec_rc = self._executor_internal(host, new_stdin)
File "/Library/Python/2.7/site-packages/ansible/runner/__init__.py", line 789, in _executor_internal
return self._executor_internal_inner(host, self.module_name, self.module_args, inject, port, complex_args=complex_args)
File "/Library/Python/2.7/site-packages/ansible/runner/__init__.py", line 1036, in _executor_internal_inner
result = handler.run(conn, tmp, module_name, module_args, inject, complex_args)
File "/Library/Python/2.7/site-packages/ansible/runner/action_plugins/normal.py", line 62, in run
return self.runner._execute_module(conn, tmp, module_name, module_args, inject=inject, complex_args=complex_args)
File "/Library/Python/2.7/site-packages/ansible/runner/__init__.py", line 470, in _execute_module
) = self._configure_module(conn, module_name, args, inject, complex_args)
File "/Library/Python/2.7/site-packages/ansible/runner/__init__.py", line 1351, in _configure_module
module_path = utils.plugins.module_finder.find_plugin(module_name, module_suffixes)
File "/Library/Python/2.7/site-packages/ansible/utils/plugins.py", line 176, in find_plugin
full_paths = (os.path.join(path, f) for f in os.listdir(path))
OSError: [Errno 13] Permission denied: '/Users/Joe/library/Saved Application State/jp.co.trendy.sdformatter.savedState'
I've done a lot of searching around but haven't had much luck finding a solution.
Here is my ~/.ansible.cfg file
# config file for ansible -- http://ansible.com/
# ==============================================
# nearly all parameters can be overridden in ansible-playbook
# or with command line flags. ansible will read ANSIBLE_CONFIG,
# ansible.cfg in the current working directory, .ansible.cfg in
# the home directory or /etc/ansible/ansible.cfg, whichever it
# finds first
[defaults]
# some basic default values...
inventory = ~/Development/ansible/hosts
And, here is my hosts file
server ansible_ssh_host=XXX.XXX.XXX.XXX ansible_ssh_user=Joe ansible_ssh_private_key_file=~/.ssh/id_rsa
This appears to be not an Ansible problem but a permission issue with a software called SD Card Formatter from Trendy Corp. I don't know why this would be triggered when you run Ansible though.
The permissions of all the files in your library should get fixed by "repair permissions" in the Disk Utility Tool. Alternatively make sure the files and folders in /Users/Joe/library/Saved Application State/jp.co.trendy.sdformatter.savedState belong to user Joe: chown -R Joe:Joe /Users/Joe/library/Saved Application State/jp.co.trendy.sdformatter.savedState. If you don't care about this application, deleting the mentioned folder completely should do it too.
In my org we have issues of not having enough Arc licenses. I would like to be able to trap this, rather than have the python script blow up on me. My script bombs right at the import arcpy line when all licenses are in use. When there are licenses available, this works likes a charm.
here is the beginning of my script:
import os
import sys
import shutil
import arcpy
pathName = str(sys.argv[1])
print pathName
Here is the response:
Traceback (most recent call last):
File "C:\Dropbox\Work\EclipseSrc\src\arcMapScripts\dluProc\arcPyNad27toNad83_Auto.py", line 4, in <module>
import arcpy
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\__init__.py", line 17, in <module>
from geoprocessing import gp
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\__init__.py", line 14, in <module>
from _base import *
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 568, in <module>
env = GPEnvironments(gp)
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 565, in GPEnvironments
return GPEnvironment(geoprocessor)
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 521, in __init__
self._refresh()
File "C:\Program Files (x86)\ArcGIS\Desktop10.0\arcpy\arcpy\geoprocessing\_base.py", line 523, in _refresh
envset = (set(env for env in self._gp.listEnvironments()))
RuntimeError: NotInitialized
I was experiencing this issue and the resolution for me revolved around scheduled tasks that were also executing at the time. Apparently with a Desktop Advanced License you cannot have two scripts running at the same time that both import arcpy. Once the scheduled task finished, I was able to execute my script with no problems.
You can use the CheckProduct function to check if licences are available. See this example copy/pasted from the Help page:
import sys
import arcpy
arcpy.env.workspace = "c:/data/world.gdb"
if arcpy.CheckProduct("ArcInfo") == "Available":
arcpy.PolygonToLine_management("Lakes", "LakeLines")
else:
msg = 'ArcGIS for Desktop Advanced license not available'
print(msg)
sys.exit(msg)