I am trying to align 4 difference sequences using MuscleCommandLine. This code works perfectly on Anaconda and Mac but I am trying to make it work on Windows and I am having several issues.
muscle_exe = r'../muscle3.8.31_i86darwin64.exe'
in_file = r'../infile.fasta'
out_file = r'../aln_out.fasta'
muscle_cline = MuscleCommandline(muscle_exe, input=in_file, out=out_file)
muscle_cline()
When I try this using Anaconda on Mac, I get the result. However, when running the same code on Window, I get the following error:
---------------------------------------------------------------------------
ApplicationError Traceback (most recent call last)
<ipython-input-3-70ad09443dc6> in <module>
3 genome = genome_location(before, after, alignment)
4
----> 5 filename = custom_fasta(genome, alignment)
6
7 info = AlignIO.read(r'../aln_out.fasta','fasta')
<ipython-input-1-655071c8d454> in custom_fasta(locations_dict, names)
185
186 muscle_cline = MuscleCommandline(muscle_exe, input=in_file, out=out_file)
--> 187 muscle_cline()
188
189 return(filename)
~\Anaconda3\lib\site-packages\Bio\Application\__init__.py in __call__(self, stdin, stdout, stderr, cwd, env)
567
568 if return_code:
--> 569 raise ApplicationError(return_code, str(self), stdout_str, stderr_str)
570 return stdout_str, stderr_str
571
ApplicationError: Non-zero return code 1 from '../muscle3.8.31_i86darwin64.exe -in ../infile.fasta -out ../aln_out.fasta', message "'..' is not recognized as an internal or external command,"
I have read the documentation but this is not listed as an issue. This same question has been asked before but no solution has been offered. People keep asking to install muscle, but the way it works with Biopython and Anaconda is that you do not need to install it.
So why does this keep happening?
Related
I'm trying to reproduce the result in a paper and its GitHub is this.
What is wired is, after I download the code and run the sample "small simulation.ipynb" in VScode with conda environment python 3.9.16, it shows
Output exceeds the size limit. Open the full output data in a text editor
---------------------------------------------------------------------------
RemoteTraceback Traceback (most recent call last)
RemoteTraceback:
"""
Traceback (most recent call last):
File "/Users/yanghongguo/anaconda3/envs/py3.7/lib/python3.8/multiprocessing/pool.py", line 125, in worker
result = (True, func(*args, **kwds))
File "/Users/yanghongguo/anaconda3/envs/py3.7/lib/python3.8/multiprocessing/pool.py", line 48, in mapstar
return list(map(*args))
File "/Users/yanghongguo/Downloads/SINC-master 2/small_example/../SINC_functions.py", line 249, in VI_VS_parallel
sig = np.ones((Q,1)) / 2500
NameError: name 'Q' is not defined
"""
The above exception was the direct cause of the following exception:
NameError Traceback (most recent call last)
Cell In[5], line 15
13 tol_elbo = 10.0
14 cpus = 1
---> 15 omega, EZ, phi,B,iters_total, elbo, elbo_score = SINC_update_tau(x, m, v0, v1, lamb, vB,a_gamma,b_gamma,a_pi,b_pi,a_tau,b_tau, max_iters, tol_prec, tol_elbo, cpus)
File ~/Downloads/SINC-master 2/small_example/../SINC_functions.py:495, in SINC_update_tau(x, m, v0, v1, lamb, vB, a_gamma, b_gamma, a_pi, b_pi, a_tau, b_tau, max_iters, tol_prec, tol_elbo, cpus)
493 pool = Pool(cpus)
494 args = [(Z[:,i] - B0[i],m,vB,sigs_j[i],B[i,],phi[i,],theta[i],a_gamma,b_gamma) for i in range(P)]
...
769 return self._value
770 else:
--> 771 raise self._value
NameError: name 'Q' is not defined
Q is defined as a global variable in the function. I asked my friend to run it in his computer and it was successful.
I tried different version of python but still got this error. Seems global variable is not identified in my environment.
My labtop is MAC M1 chip. What could be the reason that caused the error???
In this Notebook, we use Explainable AI SDK from Google to load a model, right after saving it. This fails with a message that the model is missing.
But note
the info message saying that the model was saved
checking working/model shows that the model is there.
However, working/model/assets is empty.
Why do we get this error message? How can we avoid it?
model_path = "working/model"
model.save(model_path)
builder = SavedModelMetadataBuilder(model_path)
builder.set_numeric_metadata(
"numpy_inputs",
input_baselines=[X_train.median().tolist()], # attributions relative to the median of the target
index_feature_mapping=X_train.columns.tolist(), # the names of each feature
)
builder.save_metadata(model_path)
explainer = explainable_ai_sdk.load_model_from_local_path(
model_path=model_path,
config=configs.SampledShapleyConfig(path_count=20),
)
INFO:tensorflow:Assets written to: working/model/assets
---------------------------------------------------------------------------
NotImplementedError Traceback (most recent call last)
/tmp/ipykernel_26061/1928503840.py in <module>
18 explainer = explainable_ai_sdk.load_model_from_local_path(
19 model_path=model_path,
---> 20 config=configs.SampledShapleyConfig(path_count=20),
21 )
22
/opt/conda/lib/python3.7/site-packages/explainable_ai_sdk/model/model_factory.py in load_model_from_local_path(model_path, config)
128 """
129 if _LOCAL_MODEL_KEY not in _MODEL_REGISTRY:
--> 130 raise NotImplementedError('There are no implementations of local model.')
131 return _MODEL_REGISTRY[_LOCAL_MODEL_KEY](model_path, config)
132
NotImplementedError: There are no implementations of local model.
I'm trying to run language model finetuning script (run_language_modeling.py) from huggingface examples with my own tokenizer(just added in several tokens, see the comments). I have problem loading the tokenizer. I think the problem is with AutoTokenizer.from_pretrained('local/path/to/directory').
Code:
from transformers import *
tokenizer = AutoTokenizer.from_pretrained('bert-base-uncased')
# special_tokens = ['<HASHTAG>', '<URL>', '<AT_USER>', '<EMOTICON-HAPPY>', '<EMOTICON-SAD>']
# tokenizer.add_tokens(special_tokens)
tokenizer.save_pretrained('../twitter/twittertokenizer/')
tmp = AutoTokenizer.from_pretrained('../twitter/twittertokenizer/')
Error Message:
OSError Traceback (most recent call last)
/z/huggingface_venv/lib/python3.7/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, pretrained_config_archive_map, **kwargs)
248 resume_download=resume_download,
--> 249 local_files_only=local_files_only,
250 )
/z/huggingface_venv/lib/python3.7/site-packages/transformers/file_utils.py in cached_path(url_or_filename, cache_dir, force_download, proxies, resume_download, user_agent, extract_compressed_file, force_extract, local_files_only)
265 # File, but it doesn't exist.
--> 266 raise EnvironmentError("file {} not found".format(url_or_filename))
267 else:
OSError: file ../twitter/twittertokenizer/config.json not found
During handling of the above exception, another exception occurred:
OSError Traceback (most recent call last)
<ipython-input-32-662067cb1297> in <module>
----> 1 tmp = AutoTokenizer.from_pretrained('../twitter/twittertokenizer/')
/z/huggingface_venv/lib/python3.7/site-packages/transformers/tokenization_auto.py in from_pretrained(cls, pretrained_model_name_or_path, *inputs, **kwargs)
190 config = kwargs.pop("config", None)
191 if not isinstance(config, PretrainedConfig):
--> 192 config = AutoConfig.from_pretrained(pretrained_model_name_or_path, **kwargs)
193
194 if "bert-base-japanese" in pretrained_model_name_or_path:
/z/huggingface_venv/lib/python3.7/site-packages/transformers/configuration_auto.py in from_pretrained(cls, pretrained_model_name_or_path, **kwargs)
192 """
193 config_dict, _ = PretrainedConfig.get_config_dict(
--> 194 pretrained_model_name_or_path, pretrained_config_archive_map=ALL_PRETRAINED_CONFIG_ARCHIVE_MAP, **kwargs
195 )
196
/z/huggingface_venv/lib/python3.7/site-packages/transformers/configuration_utils.py in get_config_dict(cls, pretrained_model_name_or_path, pretrained_config_archive_map, **kwargs)
270 )
271 )
--> 272 raise EnvironmentError(msg)
273
274 except json.JSONDecodeError:
OSError: Can't load '../twitter/twittertokenizer/'. Make sure that:
- '../twitter/twittertokenizer/' is a correct model identifier listed on 'https://huggingface.co/models'
- or '../twitter/twittertokenizer/' is the correct path to a directory containing a 'config.json' file
If I change AutoTokenizer to BertTokenizer, the code above can work. Also I can run the script without any problem is I load by shortcut name instead of path. But in the script run_language_modeling.py it uses AutoTokenizer. I'm looking for a way to get it running.
Any idea? Thanks!
The problem is that you are using nothing that would indicate the correct tokenizer to instantiate.
For reference, see the rules defined in the Huggingface docs. Specifically, since you are using BERT:
contains bert: BertTokenizer (Bert model)
Otherwise, you have to specify the exact type yourself, as you mentioned.
AutoTokenizer.from_pretrained fails if the specified path does not contain the model configuration files, which are required solely for the tokenizer class instantiation.
In the context of run_language_modeling.py the usage of AutoTokenizer is buggy (or at least leaky).
There is no point to specify the (optional) tokenizer_name parameter if it's identical to the model name or path. Therefore, to my understanding, it supposes to support exactly the case of a modified tokenizer. I also found this issue very confusing.
The best workaround that I have found is to add config.json to the tokenizer directory with only the "missing" configuration:
{
"model_type": "bert"
}
when loading modified tokenizer or pretrained tokenizer you should load it as follows:
tokenizer = AutoTokenizer.from_pretrained(path_to_json_file_of_tokenizer, config=AutoConfig.from_pretrained('path to thefolderthat contains the config file of the model'))
I am getting the following error while executing the mentioned code. This is the first time I am using StackOverflow so pardon my mistakes.
---------------------------------------------------------------------------
NoBackendError Traceback (most recent call last)
<ipython-input-29-2eac49b48cf3> in <module>
1
2 audio_path = 'C:/Users/Abhishek Duhan/Desktop/EP.mp3'
----> 3 y, sr = librosa.load(audio_path)
~\Anaconda3\lib\site-packages\librosa\core\audio.py in load(path, sr, mono, offset, duration, dtype, res_type)
117
118 y = []
--> 119 with audioread.audio_open(os.path.realpath(path)) as input_file:
120 sr_native = input_file.samplerate
121 n_channels = input_file.channels
~\Anaconda3\lib\site-packages\audioread\__init__.py in audio_open(path)
114
115 # All backends failed!
--> 116 raise NoBackendError()
NoBackendError:
On Ubuntu,you can solve the problem by installing libav-toools or ffmpeg:
sudo apt-get install libav-tools
Or
sudo apt-get install -y ffmpeg
See here.
I'm facing the same error too. For importing the audio files, you can use:
import scipy.io as sio
samplerate, data = sio.wavfile.read(location_of_file)
Imagine that I am manipulating a shapefile in geopandas. I then want to load it using another library (like networkx) but since my file is large I dont want to have to save and reload it. Is there a way I can save it in memory? I imagine it would look something like this:
import geopandas
from io import BytesIO
writeBytes = BytesIO()
### load the demo
world = geopandas.read_file(geopandas.datasets.get_path('naturalearth_lowres'))
### do something trivial to the demo
world['geometry'] = world['geometry'].buffer(0.05)
### save to bytes IO so that I can do something else with it without having to save and read a file
world.to_file(writeBytes)
Running the above yields a TypeError: expected str, bytes or os.PathLike object, not _io.BytesIO
This is the full traceback:
TypeError Traceback (most recent call last)
<ipython-input-1-1ba22f23181a> in <module>
8 world['geometry'] = world['geometry'].buffer(0.05)
9 ### save to bytes IO so that I can do something else with it without having to save and read a file
---> 10 world.to_file(writeBytes)
~/.conda/envs/geopandas/lib/python3.7/site-packages/geopandas/geodataframe.py in to_file(self, filename, driver, schema, **kwargs)
427 """
428 from geopandas.io.file import to_file
--> 429 to_file(self, filename, driver, schema, **kwargs)
430
431 def to_crs(self, crs=None, epsg=None, inplace=False):
~/.conda/envs/geopandas/lib/python3.7/site-packages/geopandas/io/file.py in to_file(df, filename, driver, schema, **kwargs)
125 if schema is None:
126 schema = infer_schema(df)
--> 127 filename = os.path.abspath(os.path.expanduser(filename))
128 with fiona_env():
129 with fiona.open(filename, 'w', driver=driver, crs=df.crs,
~/.conda/envs/geopandas/lib/python3.7/posixpath.py in expanduser(path)
233 """Expand ~ and ~user constructions. If user or $HOME is unknown,
234 do nothing."""
--> 235 path = os.fspath(path)
236 if isinstance(path, bytes):
237 tilde = b'~'
Any assistance is appreciated, Thank You
geopandas.to_file() requires a file path, not a BytesIO object.
Use a temporary file (or folder for shape files)
For example: https://stackoverflow.com/a/70254174/2023941