torch write video throws IndexError - torchvision

torchvision write_video throws this error without any information for all videos in the dataset.
write_video(filename=str(output_path), video_array=clip_video, fps=video_fps, audio_array=clip_audio, audio_fps=audio_fps, audio_codec='aac')
File "/home/team/miniconda3/envs/gm_torch/lib/python3.8/site-packages/torchvision/io/video.py", line 117, in write_video
audio_sample_fmt = container.streams.audio[0].format.name
IndexError: tuple index out of range

I changed the line at File "/home/team/miniconda3/envs/gm_torch/lib/python3.8/site-packages/torchvision/io/video.py", line 117, in write_video:
audio_sample_fmt = container.streams.audio[0].format.name
with:
audio_sample_fmt = "s16p"
"s16p" works but "s16" does not.
container.streams.audio is an empty tuple this makes sense because it is empty file created from scratch. This code worked before i changed my ubuntu version and from source installation to conda installation of torchvision. I don't what is causing this.

Related

Scipy.io.mmread Throws Value Error in Streamlit Heroku App

I have been trying to read a scipy matrix file in my streamlit app.
The app runs on local machine without any errors in app or console.
While deployed on Heroku it raises ValueError: not enough values to unpack (expected 5, got 2)
on the line co_occurrence_matrix = scipy.io.mmread("./database/matrix.mtx").
I've crosschecked the following points, and not sure where to look for the problem.
Matrix is created with
smatrix = scipy.sparse.csr_matrix(matrix)
scipy.isspmatrix(smatrix) #-> returns True
scipy.io.mmwrite("./database/matrix.mtx\",smatrix)
All library versions including python itself are identical between the two apps. Both checked on consoles with pip list. And requirements file is created with pip freeze.
Files are synchronized on git, git status returns 'up to date. Heroku uses the file on git, the local app runs on the synchronized file.
If it makes any difference, the .mtx file is uploaded via git lfs.
Heroku deploys the app successfully, yet inside the streamlit app, it gives the error.
Full Error:
File "/app/.heroku/python/lib/python3.9/site-packages/streamlit/runtime/scriptrunner/script_runner.py", line 565, in _run_script
exec(code, module.__dict__)File "/app/main.py", line 4, in <module>
from gorsellestirme_util import set_bg, Plotter, matrix, word_list, main_dfFile "/app/gorsellestirme_util.py", line 30, in <module>
matrix, word_list, main_df = matrix_reader()File "/app/.heroku/python/lib/python3.9/site-packages/streamlit/runtime/legacy_caching/caching.py", line 625, in wrapped_func
return get_or_create_cached_value()File "/app/.heroku/python/lib/python3.9/site-packages/streamlit/runtime/legacy_caching/caching.py", line 609, in get_or_create_cached_value
return_value = non_optional_func(*args, **kwargs)File "/app/gorsellestirme_util.py", line 12, in matrix_reader
co_occurrence_matrix = mmread("./database/matrix.mtx")File "/app/.heroku/python/lib/python3.9/site-packages/scipy/io/_mmio.py", line 77, in mmread
return MMFile().read(source)File "/app/.heroku/python/lib/python3.9/site-packages/scipy/io/_mmio.py", line 438, in read
self._parse_header(stream)File "/app/.heroku/python/lib/python3.9/site-packages/scipy/io/_mmio.py", line 502, in _parse_header
self.__class__.info(stream)File "/app/.heroku/python/lib/python3.9/site-packages/scipy/io/_mmio.py", line 234, in info
mmid, matrix, format, field, symmetry = \
As for the source of the problem: It is clearly stated in the Heroku docs that Heroku does not support git-lfs files. I've missed that point.
As a workaround, there are multiple build packs in Heroku elements. FYI, those buildpacks are also limited with Heroku's 500Mb filesize cap. And security has to be considered as an issue as those buildpacks require third-party access to your git.

OSError: [WinError 87] The parameter is incorrect: Jupyter & Anaconda

I am using Jupyter notebooks via anaconda. When saving a new notebook to a specific directory, the whole directory becomes inaccessible, showing this error (image and text below). Other directories and the notebooks inside them are still accessible and able to be run. I am able to save new notebooks and run them in other directories no problem.
Error:
Server error: Traceback (most recent call last): File "C:\Users\u1252652.conda\envs\DS011_loss_ratio\lib\site-packages\tornado\web.py", line 1704, in _execute result = await result File "C:\Users\u1252652.conda\envs\DS011_loss_ratio\lib\site-packages\tornado\gen.py", line 234, in wrapper yielded = ctx_run(next, result) File "C:\Users\u1252652.conda\envs\DS011_loss_ratio\lib\site-packages\notebook\services\contents\handlers.py", line 116, in get model = yield maybe_future(self.contents_manager.get( File "C:\Users\u1252652.conda\envs\DS011_loss_ratio\lib\site-packages\notebook\services\contents\filemanager.py", line 435, in get model = self._dir_model(path, content=content) File "C:\Users\u1252652.conda\envs\DS011_loss_ratio\lib\site-packages\notebook\services\contents\filemanager.py", line 309, in _dir_model for name in os.listdir(os_dir): OSError: [WinError 87] The parameter is incorrect: 'C:\Users\u1252652\OneDrive - MMC\Documents\DS011_loss_ratio'
This is strange as i have been working in this directory and saving numerous notebooks to it without error until today. I can delete the notebook that caused it to crash and then it returns to normal and i can run notebooks that were already saved before today.
I am unsure if this is related or not but i also have an issue whereby i have to run python -m ipykernel install --user every time i change to a new environment in anaconda, otherwise the executable remains as that of the last environment i ran that command with. Perhaps this is useful i'm not sure, if anyone had advice with this too i'd be grateful.
I have also tried making a new directory, copying over the contents of the dysfunctional directory, and then making a new notebook from there, however this causes the same error.

YFinance JSONDecodeError--latest version

When I tried to dowload historic data, I get the following error message:
Exception in thread Thread-6:
Traceback (most recent call last):
File "E:\Anaconda\envs\finrl\lib\threading.py", line 932, in _bootstrap_inner
self.run()
File "E:\Anaconda\envs\finrl\lib\threading.py", line 870, in run
self._target(*self._args, **self._kwargs)
File "E:\Anaconda\envs\finrl\lib\site-packages\multitasking\__init__.py", line 104, in _run_via_pool
return callee(*args, **kwargs)
File "E:\Anaconda\envs\finrl\lib\site-packages\yfinance\multi.py", line 169, in _download_one_threaded
data = _download_one(ticker, start, end, auto_adjust, back_adjust,
File "E:\Anaconda\envs\finrl\lib\site-packages\yfinance\multi.py", line 181, in _download_one
return Ticker(ticker).history(period=period, interval=interval,
File "E:\Anaconda\envs\finrl\lib\site-packages\yfinance\base.py", line 162, in history
data = data.json()
File "E:\Anaconda\envs\finrl\lib\site-packages\requests\models.py", line 910, in json
return complexjson.loads(self.text, **kwargs)
File "E:\Anaconda\envs\finrl\lib\json\__init__.py", line 357, in loads
return _default_decoder.decode(s)
File "E:\Anaconda\envs\finrl\lib\json\decoder.py", line 337, in decode
obj, end = self.raw_decode(s, idx=_w(s, 0).end())
File "E:\Anaconda\envs\finrl\lib\json\decoder.py", line 355, in raw_decode
raise JSONDecodeError("Expecting value", s, err.value) from None
json.decoder.JSONDecodeError: Expecting value: line 1 column 1 (char 0)
here is my code:
import yfinance as yf
data = yf.download("SPY AAPL", start="2017-01-01", end="2017-04-30")
I have tried many methods:
install the latest version of yfinance
install the old version of yfinance
revise the code
In fact, when I use yahoo_fin, the same error occurs too.
Wish someone to help me solve this problem
your code is correct.
This is happening because Anaconda is used to simulate an environment so packets have determined versions so that everything is compatible. The thing that happens in your case is that yfinance is older and your version of JSON is not compatible with some functions of yfinance since yfinance is no anymore updated.
I recommend you to just install on your computer with pip the packet yfinance as follows instead of using the environment of anaconda:
pip install yfinance
and then it should try to get the correct versions of the libraries that it needs. After that, you can create a python file and then import the packet
import yfinance as yf
and then execute your download code
data = yf.download("SPY AAPL", start="2017-01-01", end="2017-04-30")
As a recommendation, try to pass different values as a list instead of a string, because it will be hard to debug later. Instead of "SPY AAPL" -> ['SPY', 'AAPL']
Edit: Also it can be that your finance version is YEARS old (but I don't think so), is this is the case then you should try to update inside of the anaconda updater. Otherwise, with the terminal, it would be something like:
pip install yfinance --upgrade

cqlsh will not run on OS X

I installed Cassandra on Big Sur 11.4 using homebrew, and when I tried to run cqlsh, I received this error.
isaackim#Isaacs-MacBook-Pro bin % cqlsh
Traceback (most recent call last):
File "/usr/local/Cellar/cassandra/3.11.10/libexec/bin/cqlsh.py", line 2458, in <module>
main(*read_options(sys.argv[1:], os.environ))
File "/usr/local/Cellar/cassandra/3.11.10/libexec/bin/cqlsh.py", line 2232, in read_options
configs.read(CONFIG_FILE)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 305, in read
self._read(fp, filename)
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/ConfigParser.py", line 512, in _read
raise MissingSectionHeaderError(fpname, lineno, line)
ConfigParser.MissingSectionHeaderError: File contains no section headers.
I previously had Anaconda3 installed, and received the same error (the only difference was the path to cqlsh.py). I thought it might be an issue with Anaconda so I uninstalled Anaconda entirely, then I reinstalled Cassandra using homebrew.
I would appreciate any and all suggestions. Thanks in advance.
Thanks to Alex Ott's suggestion, I managed to replicate the issue you posted by creating a bad ~/.cassandra/cqlshrc configuration file which only contains the following line:
username = alice
As this error suggests:
ConfigParser.MissingSectionHeaderError: File contains no section headers.
the configuration parser detected that there are no section headers in the file. Section headers are line entries enclosed in square brackets ([header_name]). For example:
[authentication]
username = alice
password = Som3Comp7exPa$$word
Review your cqlshrc and make the necessary corrections. If necessary, use the example cqlshrc.sample file in conf/. Cheers!

NLTK v3.2: Unable to nltk.pos_tag()

Hi text mining champions,
I'm using Anaconda with NLTK v3.2 on Windows 10.(client's environment)
When I try to POS tag, I keep getting a URLLIB2 error:
URLError: <urlopen error unknown url type: c>
It seems urllib2 is unable to recognize windows paths? How can I work around this?
The command is simple as:
nltk.pos_tag(nltk.word_tokenize("Hello World"))
edit:
There is a duplicate question, however I think the answers obtained here by manan and alvas are a better fix.
EDITED
This issue has been resolved from NLTK v3.2.1. Upgrading your NLTK version would resolve the issue, e.g. pip install -U nltk.
I faced the same issue and the error encountered was as follows;
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\__init__.py", line 110, in pos_tag
tagger = PerceptronTagger()
File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py", line 141, in __init__
self.load(AP_MODEL_LOC)
File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py", line 209, in load
self.model.weights, self.tagdict, self.classes = load(loc)
File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\data.py", line 801, in load
opened_resource = _open(resource_url)
File "C:\Python27\lib\site-packages\nltk-3.2-py2.7.egg\nltk\data.py", line 924, in _open
return urlopen(resource_url)
File "C:\Python27\lib\urllib2.py", line 126, in urlopen
return _opener.open(url, data, timeout)
File "C:\Python27\lib\urllib2.py", line 391, in open
response = self._open(req, data)
File "C:\Python27\lib\urllib2.py", line 414, in _open
'unknown_open', req)
File "C:\Python27\lib\urllib2.py", line 369, in _call_chain
result = func(*args)
File "C:\Python27\lib\urllib2.py", line 1206, in unknown_open
raise URLError('unknown url type: %s' % type)
urllib2.URLError: <urlopen error unknown url type: c>
The URLError that you mentioned was due to a bug in the perceptron.py file within the NLTK library for Windows.
In my machine, the file is at this location
C:\Python27\Lib\site-packages\nltk-3.2-py2.7.egg\nltk\tag\perceptron.py
(Basically look at an equivalent location within yours wherever you have the Python27 folder)
The bug was basically in the code to find the corresponding location for the averaged_perceptron_tagger within your machine. One can have a look at the line 801 and 924 mentioned in the data.py file regarding this.
I think the NLTK developer community recently fixed this bug in the code. Have a look at this commit made to their code a few days back.
https://github.com/nltk/nltk/commit/d3de14e58215beebdccc7b76c044109f6197d1d9#diff-26b258372e0d13c2543de8dbb1841252
The snippet where the change was made is as follows;
self.tagdict = {}
self.classes = set()
if load:
AP_MODEL_LOC = 'file:'+str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
self.load(AP_MODEL_LOC)
# Initially it was:AP_MODEL_LOC = str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
def tag(self, tokens):
Updating the file to the most recent commit worked for me and was able to use the nltk.pos_tag command. I believe this would resolve your problem as well (assuming you have everything else set up).
EDITED
This issue has been resolved from NLTK v3.2.1. Please upgrade your NLTK!
First read #MananVyas answer for the why:
https://stackoverflow.com/a/35902494/610569
Here's the how, without downgrading to NLTK v3.1, using NLTK 3.2, you can use this "hack":
>>> from nltk.tag import PerceptronTagger
>>> from nltk.data import find
>>> PICKLE = "averaged_perceptron_tagger.pickle"
>>> AP_MODEL_LOC = 'file:'+str(find('taggers/averaged_perceptron_tagger/'+PICKLE))
>>> tagger = PerceptronTagger(load=False)
>>> tagger.load(AP_MODEL_LOC)
>>> pos_tag = tagger.tag
>>> pos_tag('The quick brown fox jumps over the lazy dog'.split())
[('The', 'DT'), ('quick', 'JJ'), ('brown', 'NN'), ('fox', 'NN'), ('jumps', 'VBZ'), ('over', 'IN'), ('the', 'DT'), ('lazy', 'JJ'), ('dog', 'NN')]
I faced the same issue a while back.
Solution:
nltk.download('averaged_perceptron_tagger')

Resources