I've been trying to use args in a function so that it can accept an unspecified amount of parameters but during testing I seem to be lumped with this error. Which seems to be produced by the choice module itself not the file i'm writing.
File "C:\ProgramFiles\WindowsApps\PythonSoftwareFoundation.Python.3.10_3.10.1520.0_x64__qbz5n2kfra8p0\lib\random.py", line 378, in choice
return seq[self._randbelow(len(seq))]
IndexError: list index out of range
Here is the code i'm trying to run:
def key_word_choice(key, *otherkeys):
key_choice = []
for items in sauces:
if key and otherkeys in items:
key_choice.append(items[0])
print("You should choose:", choice(key_choice))
key_word_choice('hot', 'garlic', 'buffalo')
Python 3.9.7, Numba 0.54.0
I wrote this simple code, that works well:
import numpy as np
import numba
from collections import namedtuple
Deal = namedtuple('Deal' , ['DateTime', 'Price', 'Quantity'])
dtype_deals = [('DateTime', 'datetime64[s]'), ('Price', 'float64'), ('Quantity', 'uint64')]
# #numba.njit
def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
deals.append(Deal(np.datetime64('2015-03-17T15:08:36'), 1.15, 100))
deals.append(Deal(np.datetime64('2015-04-02T15:14:32'), 11.01, 20))
return np.array(deals, dtype=dtype_deals)
print(str(my_func()))
But when I remove the comment before #numba.njit, I get a mistake:
numba.core.errors.TypingError: Failed in nopython mode pipeline (step:
nopython frontend) No implementation of function
Function(datetime64[]) found for signature:
(Literalstr) There are 2 candidate implementations:
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(unicode_type)': Rejected as the implementation raised a specific
error:
TypingError: Casting unicode_type to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(Literalstr)': Rejected as the
implementation raised a specific error:
TypingError: Casting Literalstr to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
During: resolving callee type: class(datetime64[])
During: typing of call at /home/ivan/eclipse-workspace/MarketAnalysis/Experiment.py (11)
File "Experiment.py", line 11: def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
^
I'm working in a Python 3.8+ Django/Rest-Framework environment enforcing types in new code but built on a lot of untyped legacy code and data. We are using TypedDicts extensively for ensuring that data we are generating passes to our TypeScript front-end with the proper data type.
MyPy/PyCharm/etc. does a great job of checking that our new code spits out data that conforms, but we want to test that the output of our many RestSerializers/ModelSerializers fits the TypeDict. If I have a serializer and typed dict like:
class PersonSerializer(ModelSerializer):
class Meta:
model = Person
fields = ['first', 'last']
class PersonData(TypedDict):
first: str
last: str
email: str
and then run code like:
person_dict: PersonData = PersonSerializer(Person.objects.first()).data
Static type checkers don't be able to figure out that person_dict is missing the required email key, because (by design of PEP-589) it is just a normal dict. But I can write something like:
annotations = PersonData.__annotations__
for k in annotations:
assert k in person_dict # or something more complex.
assert isinstance(person_dict[k], annotations[k])
and it will find that email is missing from the data of the serializer. This is well and good in this case, where I don't have any changes introduced by from __future__ import annotations (not sure if this would break it), and all my type annotations are bare types. But if PersonData were defined like:
class PersonData(TypedDict):
email: Optional[str]
affiliations: Union[List[str], Dict[int, str]]
then isinstance is not good enough to check if the data passes (since "Subscripted generics cannot be used with class and instance checks").
What I'm wondering is if there already exists a callable function/method (in mypy or another checker) that would allow me to validate a TypedDict (or even a single variable, since I can iterate a dict myself) against an annotation and see if it validates?
I'm not concerned about speed, etc., since the point of this is to check all our data/methods/functions once and then remove the checks later once we're happy that our current data validates.
The simplest solution I found works using pydantic.
from typing import cast, TypedDict
import pydantic
class SomeDict(TypedDict):
val: int
name: str
# this could be a valid/invalid declaration
obj: SomeDict = {
'val': 12,
'name': 'John',
}
# validate with pydantic
try:
obj = cast(SomeDict, pydantic.create_model_from_typeddict(SomeDict)(**obj).dict())
except pydantic.ValidationError as exc:
print(f"ERROR: Invalid schema: {exc}")
EDIT: When type checking this, it currently returns an error, but works as expected. See here: https://github.com/samuelcolvin/pydantic/issues/3008
You may want to have a look at https://pypi.org/project/strongtyping/. This may help.
In the docs you can find this example:
from typing import List, TypedDict
from strongtyping.strong_typing import match_class_typing
#match_class_typing
class SalesSummary(TypedDict):
sales: int
country: str
product_codes: List[str]
# works like expected
SalesSummary({"sales": 10, "country": "Foo", "product_codes": ["1", "2", "3"]})
# will raise a TypeMisMatch
SalesSummary({"sales": "Foo", "country": 10, "product_codes": [1, 2, 3]})
A little bit of a hack, but you can check two types using mypy command line -c options. Just wrap it in a python function:
import subprocess
def is_assignable(type_to, type_from) -> bool:
"""
Returns true if `type_from` can be assigned to `type_to`,
e. g. type_to := type_from
Example:
>>> is_assignable(bool, str)
False
>>> from typing import *
>>> is_assignable(Union[List[str], Dict[int, str]], List[str])
True
"""
code = "\n".join((
f"import typing",
f"type_to: {type_to}",
f"type_from: {type_from}",
f"type_to = type_from",
))
return subprocess.call(("mypy", "-c", code)) == 0
You could do something like this:
def validate(typ: Any, instance: Any) -> bool:
for property_name, property_type in typ.__annotations__.items():
value = instance.get(property_name, None)
if value is None:
# Check for missing keys
print(f"Missing key: {property_name}")
return False
elif property_type not in (int, float, bool, str):
# check if property_type is object (e.g. not a primitive)
result = validate(property_type, value)
if result is False:
return False
elif not isinstance(value, property_type):
# Check for type equality
print(f"Wrong type: {property_name}. Expected {property_type}, got {type(value)}")
return False
return True
And then test some object, e.g. one that was passed to your REST endpoint:
class MySubModel(TypedDict):
subfield: bool
class MyModel(TypedDict):
first: str
last: str
email: str
sub: MySubModel
m = {
'email': 'JohnDoeAtDoeishDotCom',
'first': 'John'
}
assert validate(MyModel, m) is False
This one prints the first error and returns bool, you could change that to exceptions, possibly with all the missing keys. You could also extend it to fail on additional keys than defined by the model.
I like your solution!. In order to avoid iteration fixes for some user, I added some code to your solution :D
def validate_custom_typed_dict(instance: Any, custom_typed_dict:TypedDict) -> bool|Exception:
key_errors = []
type_errors = []
for property_name, type_ in my_typed_dict.__annotations__.items():
value = instance.get(property_name, None)
if value is None:
# Check for missing keys
key_errors.append(f"\t- Missing property: '{property_name}' \n")
elif type_ not in (int, float, bool, str):
# check if type is object (e.g. not a primitive)
result = validate_custom_typed_dict(type_, value)
if result is False:
type_errors.append(f"\t- '{property_name}' expected {type_}, got {type(value)}\n")
elif not isinstance(value, type_):
# Check for type equality
type_errors.append(f"\t- '{property_name}' expected {type_}, got {type(value)}\n")
if len(key_errors) > 0 or len(type_errors) > 0:
error_message = f'\n{"".join(key_errors)}{"".join(type_errors)}'
raise Exception(error_message)
return True
some console output:
Exception:
- Missing property: 'Combined_cycle'
- Missing property: 'Solar_PV'
- Missing property: 'Hydro'
- 'timestamp' expected <class 'str'>, got <class 'int'>
- 'Diesel_engines' expected <class 'float'>, got <class 'int'>
Unexpected keyword argument 'queryset' in constructor call [E:unexpected-keyword-arg]
Tried using form_kwargs as shown on stack overflow here:
# How to use the new form_kwargs on an inline formset?
if request.method == "POST":
ctx['formset'] = project_comparison_form_set(
data=request.POST, files=request.FILES, queryset=ctx['projects'])
ctx['data1'] = request.POST.copy
if ctx['formset'].is_valid():
instances = ctx['formset'].save(commit=False)
for project in instances:
project.save()
Getting both a warning and error message in pylint Unexpected keyword argument 'queryset' in constructor call [E:unexpected-keyword-arg]
You don't show where project_comparison_form_set is defined, but i assume that it is a modelformset_factory.
Here, you do not have the queryset argument. If you do want to pass a queryset, you can pass it to the formset and the formset than to the modelformset_factory.
Check the documentation https://docs.djangoproject.com/en/4.1/topics/forms/modelforms/#changing-the-queryset.
I am trying to do some text analysis:
def cleaning_text(sentence):
sentence=sentence.lower()
sentence=re.sub('\'','',sentence.strip())
sentence=re.sub('^\d+\/\d+|\s\d+\/\d+|\d+\-\d+\-\d+|\d+\-\w+\-\d+\s\d+\:\d+|\d+\-\w+\-\d+|\d+\/\d+\/\d+\s\d+\:\d+',' ',sentence.strip())# dates removed
sentence=re.sub(r'(.)(\/)(.)',r'\1\3',sentence.strip())
sentence=re.sub("(.*?\//)|(.*?\\\\)|(.*?\\\)|(.*?\/)",' ',sentence.strip())
sentence=re.sub('^\d+','',sentence.strip())
sentence = re.sub('[%s]' % re.escape(string.punctuation),'',sentence.strip())
cleaned=' '.join([w for w in sentence.split() if not len(w)<2 and w not in ('no', 'sc','ln') ])
cleaned=cleaned.strip()
if(len(cleaned)<=1):
return "NA"
else:
return cleaned
org_val=udf(cleaning_text,StringType())
df_new =df.withColumn("cleaned_short_desc", org_val(df["symptom_short_description_"]))
df_new =df_new.withColumn("cleaned_long_desc", org_val(df_new["long_description"]))
longWordsDF = (df_new.select(explode(split('cleaned_long_desc',' ')).alias('word'))
longWordsDF.count()
I get the following error.
File "<stdin>", line 2, in cleaning_text
AttributeError: 'NoneType' object has no attribute 'lower'
I want to perform word counts but any kind of aggregation function is giving me an error.
I tried following things:
sentence=sentence.encode("ascii", "ignore")
Added this statement in the cleaning_text function
df.dropna()
Its still giving the same issue, I do not know how to resolve this issue.
It looks like you have null values in some columns. Add an if at the beginning of cleaning_text function and the error will disappear:
if sentence is None:
return "NA"