Based on this, I wrote the following code on Python file, which I want to run on my server
model_name = 'xlm-roberta-base'
access_token = "........................"
self.tokenizer = transformers.AutoTokenizer.from_pretrained(model_name, use_auth_token=access_token)
self.model = transformers.AutoModel.from_pretrained(model_name, use_auth_token=access_token)
I provided my access_token properly and I checked with both read and write roles. Always, I am getting the following error
File "/home/.../anaconda3/envs/test/lib/python3.8/site-packages/transformers/utils/hub.py", line 424, in cached_file
raise EnvironmentError(
OSError: 5450 is not a local folder and is not a valid model identifier listed on 'https://huggingface.co/models'
If this is a private repository, make sure to pass a token having permission to this repo with `use_auth_token` or log in with `huggingface-cli login` and pass `use_auth_token=True`.
I did every step possible to me but still got the error. Where do I need to modify the code?
Related
I am able to save all my files in the bucket but having difficulties with download.
My code is:
$url = Storage::disk('s3')->temporaryUrl(
$request->file, now()->addMinutes(10)
);
return Storage::disk('s3')->download($url);
Full file path stored in $request->file
Example path: https://bucket_name.privacy_region_info/folder_inside_bucket/cTymyY2gzakfczO3j3H2TtbJX4eeRW4Uj073CZUW
I am getting the fallowing https://prnt.sc/1ip4g77
Did I not understand the purpose od generating temporaryUrl? How can I download files from S3 non public bucket?
BTW I am using Laravel 8 and league/flysystem-aws-s3-v3 1.0.29.
The error message you have shown suggests your user does not have the correct permissions, or that the file does not exist.
If you are sure the file exists, i would suspect a permissions issue.
In AWS IAM, make sure the user has a policy attached to it that grants the correct permissions.
In this case from the comments, i can see the user only has "Write" permissions. You will need explicit "Read" permissions too.
I am trying to save the tokenizer in huggingface so that I can load it later from a container where I don't need access to the internet.
BASE_MODEL = "distilbert-base-multilingual-cased"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.save_vocabulary("./models/tokenizer/")
tokenizer2 = AutoTokenizer.from_pretrained("./models/tokenizer/")
However, the last line is giving the error:
OSError: Can't load config for './models/tokenizer3/'. Make sure that:
- './models/tokenizer3/' is a correct model identifier listed on 'https://huggingface.co/models'
- or './models/tokenizer3/' is the correct path to a directory containing a config.json file
transformers version: 3.1.0
How to load the saved tokenizer from pretrained model in Pytorch didn't help unfortunately.
Edit 1
Thanks to #ashwin's answer below I tried save_pretrained instead, and I get the following error:
OSError: Can't load config for './models/tokenizer/'. Make sure that:
- './models/tokenizer/' is a correct model identifier listed on 'https://huggingface.co/models'
- or './models/tokenizer/' is the correct path to a directory containing a config.json file
the contents of the tokenizer folder is below:
I tried renaming tokenizer_config.json to config.json and then I got the error:
ValueError: Unrecognized model in ./models/tokenizer/. Should have a `model_type` key in its config.json, or contain one of the following strings in its name: retribert, t5, mobilebert, distilbert, albert, camembert, xlm-roberta, pegasus, marian, mbart, bart, reformer, longformer, roberta, flaubert, bert, openai-gpt, gpt2, transfo-xl, xlnet, xlm, ctrl, electra, encoder-decoder
save_vocabulary(), saves only the vocabulary file of the tokenizer (List of BPE tokens).
To save the entire tokenizer, you should use save_pretrained()
Thus, as follows:
BASE_MODEL = "distilbert-base-multilingual-cased"
tokenizer = AutoTokenizer.from_pretrained(BASE_MODEL)
tokenizer.save_pretrained("./models/tokenizer/")
tokenizer2 = DistilBertTokenizer.from_pretrained("./models/tokenizer/")
Edit:
for some unknown reason:
instead of
tokenizer2 = AutoTokenizer.from_pretrained("./models/tokenizer/")
using
tokenizer2 = DistilBertTokenizer.from_pretrained("./models/tokenizer/")
works.
Renaming "tokenizer_config.json" file -- the one created by save_pretrained() function -- to "config.json" solved the same issue on my environment.
You need to save both your model and tokenizer in the same directory. HuggingFace is actually looking for the config.json file of your model, so renaming the tokenizer_config.json would not solve the issue
I have a new computer and I'm trying to set up my AWS CLI environment so that I can run a management console I've created.
This is the code I'm running:
def create_bucket(bucket_args)
AWS_S3 = Aws::S3::Client.new(signature_version: 'v4')
AWS_S3.create_bucket(bucket_args)
end
Which raises this error:
Aws::S3::Errors::SignatureDoesNotMatch - The request signature we calculated does not match the signature you provided. Check your key and signing method.:
This was working properly on my other computer, which I no longer have access to. I remember debugging this same error on the other computer, and I thought I had resolved it by adding signature_version = s3v4 to my ~/.aws/config file. But this fix is not working on my new computer, and I'm not sure why.
To give some more context: I am using aws-sdk (2.5.5) and these aws cli specs: aws-cli/1.11.2 Python/2.7.12 Linux/4.4.0-38-generic botocore/1.4.60
In this case the issue was that my aws credentials (in ~/.aws/credentials) - specifically my secret token - were invalid.
The original had a slash in it:
xx/xxxxxxxxxxxxxxxxxxxxxxxxxx
which I didn't notice at first, so when I double clicked the token to select the word, it didn't include the first three characters. I then pasted this into the terminal when running aws configure.
To fix this, I found the correct, original secret acceess token and set the correct value in ~/.aws/credentials.
Trying to save a file locally from an app running in the iOS 8 Simulator and I'm continually getting access denied exceptions.
In previous apps I've used the following code to get a valid file path:
Environment.GetFolderPath(Environment.SpecialFolder.Personal)
or
Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)
But I've read that with iOS 8 this has now got to be written as:
NSFileManager.DefaultManager.GetUrls(NSSearchPathDirectory.DocumentDirectory,
NSSearchPathDomain.User)[0]
So I'm using the following code to generate a file path for a .txt file and receiving an access denied exception when trying to save with it:
public void SaveMyFile(string content)
{
NSUrl[] urls;
string filePath;
//
urls = NSFileManager.DefaultManager.GetUrls(NSSearchPathDirectory.DocumentDirectory,
NSSearchPathDomain.User);
filePath = Path.Combine(urls[0].Path, "MyApp", "myFile.txt");
File.WriteAllText(filePath, content);
}
So the file path that it gives me and also denies access to is /Users/Idox/Library/Developer/CoreSimulator/Devices/92498E38-7D50-4081-8A64-83061DC00A86/data/Containers/Data/Application/C35B3E98-C9E3-4ABA-AA7F-CD8419FA0EA5/Documents/MyApp/myFile.txt.
I'm wondering if there's some setting that needs to be toggled to give the app write access to this directory or if the directory itself is invalid.
I've also done a call to Directory.Exists(string path) to check if the directory is there, which it is.
You're missing the Path property on urls[0].Path
filePath = Path.Combine(urls[0].Path, "MyApp", "myFile.txt");
This was fixed in Xamarin.iOS 8.4, so if you're using a recent version of Xamarin you can use Environment.GetFolderPath without problems (which is useful if you want to share code across platforms).
I am trying to write a script for using with op5/nagios.
What it tries to do is to connect to database an get a value there, and then send a request to a webservice and match the response from the webservice with the value from the database.
Now writing the script wasn't the problem. The problem arrises when trying to run it from OP5.
When including the interface to script I use the following syntax
use MyInterfaces::PortalServicesImplService::PortalServicesImplPort;
This works just fine when I execute the script located on the same level as "MyInterfaces".
The problem is that the script itself is located in the folder /opt/plugins/custom/SOAP where both the script and the folders generated by wsdl2perl.pl is located. However when OP5 executes the script is does so from the folder /opt/monitor.
So I tried to include the "PortalServicesImplPort" using this syntax
use lib "/opt/plugins/custom/SOAP/MyInterfaces/PortalservicesImplServices/PortalServicesImplPort
The code compiles but when trying to run it I get an error message saying
Can't locate object method "new" via package "MyInterfaces::PortalServicesImplService::PortalServicesImplPort" (perhaps you forgot to load "MyInterfaces::PortalServicesImplService::PortalServicesImplPort"?)
Am I doing something wrong when importing methods from another location or doesn't SOAP::WSDL support placing the auto generated files in a different directory from where you are executing the script?
Seems I managed to solve this myself by adding use lib "/opt/plugins/custom/SOAP