Julia iterates over `Base.Iterators.Reverse` - for-loop
for i in Iterators.reverse(Iterators.drop([1,2], 1))
println(i)
end
MethodError: no method matching iterate(::Base.Iterators.Reverse{Base.Iterators.Drop{Array{Int64,1}}})
Closest candidates are:
iterate(!Matched::LibGit2.GitRevWalker) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/LibGit2/src/walker.jl:29
iterate(!Matched::LibGit2.GitRevWalker, !Matched::Any) at /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.5/LibGit2/src/walker.jl:29
iterate(!Matched::Base.EnvDict) at env.jl:119
...
Stacktrace:
[1] top-level scope at ./In[1]:1
[2] include_string(::Function, ::Module, ::String, ::String) at ./loading.jl:1091
Below version runs correctly
for i in Iterators.reverse(collect(Iterators.drop([1,2], 1)))
println(i)
end
Why is collect required?
My julia version is 1.5.3, running on WSL2.
The answer is the the error message: collect is required because there is no method matching iterate(::Base.Iterators.Reverse{Base.Iterators.Drop{Array{Int64,1}}}). From the documentation of reverse:
Not all iterator types T support reverse-order iteration. If T doesn't, then iterating over Iterators.reverse(itr::T) will
throw a MethodError because of the missing iterate methods for Iterators.Reverse{T}. (To implement these methods, the
original iterator itr::T can be obtained from r = Iterators.reverse(itr) by r.itr.)
Related
How to create and fill structured array with numba?
Python 3.9.7, Numba 0.54.0
I wrote this simple code, that works well:
import numpy as np
import numba
from collections import namedtuple
Deal = namedtuple('Deal' , ['DateTime', 'Price', 'Quantity'])
dtype_deals = [('DateTime', 'datetime64[s]'), ('Price', 'float64'), ('Quantity', 'uint64')]
# #numba.njit
def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
deals.append(Deal(np.datetime64('2015-03-17T15:08:36'), 1.15, 100))
deals.append(Deal(np.datetime64('2015-04-02T15:14:32'), 11.01, 20))
return np.array(deals, dtype=dtype_deals)
print(str(my_func()))
But when I remove the comment before #numba.njit, I get a mistake:
numba.core.errors.TypingError: Failed in nopython mode pipeline (step:
nopython frontend) No implementation of function
Function(datetime64[]) found for signature:
(Literalstr) There are 2 candidate implementations:
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(unicode_type)': Rejected as the implementation raised a specific
error:
TypingError: Casting unicode_type to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
Of which 1 did not match due to: Overload in function 'make_callable_template..generic': File:
numba/core/typing/templates.py: Line 174. With argument(s):
'(Literalstr)': Rejected as the
implementation raised a specific error:
TypingError: Casting Literalstr to datetime64[] directly is unsupported. raised from
/home/ivan/.local/lib/python3.9/site-packages/numba/core/typing/builtins.py:818
During: resolving callee type: class(datetime64[])
During: typing of call at /home/ivan/eclipse-workspace/MarketAnalysis/Experiment.py (11)
File "Experiment.py", line 11: def my_func():
deals = []
deals.append(Deal(np.datetime64('2015-02-27T15:12:12'), 12.48, 10))
^
Prolog error arguments are not instantiated
When running this code: signcheck(Sign1,Sign2):-(Sign1==clubs,Sign2==hearts);(Sign1==clubs,Sign2==spades) ;(Sign1==clubs,Sign2==diamonds);(Sign1==hearts,Sign2==spades) ;(Sign1==hearts,Sign2==diamonds);(Sign1==spades,Sign2==diamonds). lowercard(card(Num1,Sign1),card(Num2,_),card(Num1,Sign1)):- Num1<Num2. lowercard(card(Num1,Sign1),card(Num2,Sign2),card(Num1,Sign1)):- Num1==Num2,signcheck(Sign1,Sign2). lowercard(card(Num1,Sign1),card(Num2,Sign2),card(Num2,Sign2)):- Num1==Num2,signcheck(Sign2,Sign1). lowercard(card(Num1,_),card(Num2,Sign2),card(Num2,Sign2)):- Num1>Num2. lowest([card(X,Y)],card(X,Y)). lowest([X|XS],Z):- lowercard(X,Y,Z), lowest(XS,Y). when running the query: lowest([card(5, hearts),card(4,diamonds)], X) I get this error message: Arguments are not sufficiently instantiated In: [3] 5<_1600 [2] lowercard(card(5,hearts),card(_1668,_1670),card(5,hearts)) at line 4 [1] lowest([card(5,hearts),...],card(5,hearts)) at line 8 Why is this occouring?
Arithmetic operations (such as <) cannot be evaluated unless all arguments are instantiated. If your case, the right-hand argument (Num2, it looks like) hasn't been instantiated.
Using addprocs() and pmap() inside a function in Julia
In Julia, I want to use addprocs and pmap inside a function that is defined inside a module. Here's a silly example:
module test
using Distributions
export g, f
function g(a, b)
a + rand(Normal(0, b))
end
function f(A, b)
close = false
if length(procs()) == 1 # If there are already extra workers,
addprocs() # use them, otherwise, create your own.
close = true
end
W = pmap(x -> g(x, b), A)
if close == true
rmprocs(workers()) # Remove the workers you created.
end
return W
end
end
test.f(randn(5), 1)
This returns a very long error
WARNING: Module test not defined on process 4
WARNING: Module test not defined on process 3
fatal error on fatal error on WARNING: Module test not defined on process 2
43: : WARNING: Module test not defined on process 5
fatal error on fatal error on 5: 2: ERROR: UndefVarError: test not defined
in deserialize at serialize.jl:504
in handle_deserialize at serialize.jl:477
in deserialize at serialize.jl:696
...
in message_handler_loop at multi.jl:878
in process_tcp_streams at multi.jl:867
in anonymous at task.jl:63
Worker 3 terminated.
Worker 2 terminated.ERROR (unhandled task failure): EOFError: read end of file
WARNING: rmprocs: process 1 not removed
Worker 5 terminated.ERROR (unhandled task failure): EOFError: read end of file
4-element Array{Any,1}:Worker 4 terminated.ERROR (unhandled task failure): EOFError: read end of file
ERROR (unhandled task failure): EOFError: read end of file
ProcessExitedException()
ProcessExitedException()
ProcessExitedException()
ProcessExitedException()
What I'm trying to do is write a package that contains functions that perform operations that can be optionally parallelized at the user's discretion. So a function like f might take an argument par::Bool that does something like I've shown above if the user calls f with par = true and loops otherwise. So from within the definition of f (and within the definition of the module test), I want to create workers and broadcast the Distributions package and the function g to them.
What's wrong with using #everywhere in your function? The following, for example, works fine on my computer. function f(A, b) close = false if length(procs()) == 1 # If there are already extra workers, addprocs() # use them, otherwise, create your own. #everywhere begin using Distributions function g(a, b) a + rand(Normal(0, b)) end end close = true end W = pmap(x -> g(x, b), A) if close == true rmprocs(workers()) # Remove the workers you created. end return W end f(randn(5), 1) Note: when I first ran this, I needed to recompile the Distributions package since it had been updated since I had last used it. When I first tried the above script right after the recompiling, it failed. But, I then quit Julia and reopened it and it worked fine. Perhaps that was what is causing your error?
Rscript : Why is Error in UseMethod("extract_") : being indicated when attempting to use raster::extract?
I attempting to use raster package's extract method to extract values from a Raster* object.
RStudioPrompt> jpnpe <- extract(jpnp, jpnb, fun = mean, na.rm = T)
where jpnp is the raster object and jpnb is SpatialPolygonsDataFrame
However the following error is indicated:
Error in UseMethod("extract_") :
no applicable method for 'extract_' applied to an object of class "c('RasterStack', 'Raster', 'RasterStackBrick', 'BasicRaster')"
How can I get passed this error?
Issue may be due to having another package with the same method name, obfuscating the raster extract method.
The tidyr package has an extract method which may conflict with raster's extract method.
Confirm by checking libraries loaded by doing:
>search()
[1] ".GlobalEnv" **"package:tidyr"** "package:dplyr"
[4] "package:rgeos" "package:ggplot2" "package:RColorBrewer"
[7] "package:animation" "package:rgdal" "package:maptools"
[10] **"package:raster"** "package:sp" "tools:rstudio"
[13] "package:stats" "package:graphics" "package:grDevices"
[16] "package:utils" "package:datasets" "package:methods"
[19] "Autoloads" "package:base"
you can also check which extract method is being loaded by typing name of function without brackets (as below, the environment will tell you which package is being used):
> extract
function (data, col, into, regex = "([[:alnum:]]+)", remove = TRUE,
convert = FALSE, ...)
{
col <- col_name(substitute(col))
extract_(data, col, into, regex = regex, remove = remove,
convert = convert, ...)
}
<environment: namespace:tidyr>
To resolve the error just unload the offending package, in RStudio you can use the following command:
>.rs.unloadPackage("tidyr")
and re-execute the raster extract method:
>jpnpe <- extract(jpnp, jpnb, fun = mean, na.rm = T)
Forcing a package's function to use user-provided function
I'm running into a problem with the MNP package which I've traced to an unfortunate call to deparse (whose maximum width is limited to 500 characters).
Background (easily skippable if you're bored)
Because mnp uses a somewhat idiosyncratic syntax to allow for varying choice sets (you include cbind(choiceA,choiceB,...) in the formula definition), the left hand side of my formula call is 1700 characters or so when model.matrix.default calls deparse on it. Since deparse supports a maximum width.cutoff of 500 characters, the sapply(attr(t, "variables"), deparse, width.cutoff = 500)[-1L] line in model.matrix.default has as its first element:
[1] "cbind(plan1, plan2, plan3, plan4, plan5, plan6, plan7, plan8, plan9, plan10, plan11, plan12, plan13, plan14, plan15, plan16, plan17, plan18, plan19, plan20, plan21, plan22, plan23, plan24, plan25, plan26, plan27, plan28, plan29, plan30, plan31, plan32, plan33, plan34, plan35, plan36, plan37, plan38, plan39, plan40, plan41, plan42, plan43, plan44, plan45, plan46, plan47, plan48, plan49, plan50, plan51, plan52, plan53, plan54, plan55, plan56, plan57, plan58, plan59, plan60, plan61, plan62, plan63, "
[2] " plan64, plan65, plan66, plan67, plan68, plan69, plan70, plan71, plan72, plan73, plan74, plan75, plan76, plan77, plan78, plan79, plan80, plan81, plan82, plan83, plan84, plan85, plan86, plan87, plan88, plan89, plan90, plan91, plan92, plan93, plan94, plan95, plan96, plan97, plan98, plan99, plan100, plan101, plan102, plan103, plan104, plan105, plan106, plan107, plan108, plan109, plan110, plan111, plan112, plan113, plan114, plan115, plan116, plan117, plan118, plan119, plan120, plan121, plan122, plan123, "
[3] " plan124, plan125, plan126, plan127, plan128, plan129, plan130, plan131, plan132, plan133, plan134, plan135, plan136, plan137, plan138, plan139, plan140, plan141, plan142, plan143, plan144, plan145, plan146, plan147, plan148, plan149, plan150, plan151, plan152, plan153, plan154, plan155, plan156, plan157, plan158, plan159, plan160, plan161, plan162, plan163, plan164, plan165, plan166, plan167, plan168, plan169, plan170, plan171, plan172, plan173, plan174, plan175, plan176, plan177, plan178, plan179, "
[4] " plan180, plan181, plan182, plan183, plan184, plan185, plan186, plan187, plan188, plan189, plan190, plan191, plan192, plan193, plan194, plan195, plan196, plan197, plan198, plan199, plan200, plan201, plan202, plan203, plan204, plan205, plan206, plan207, plan208, plan209, plan210, plan211, plan212, plan213, plan214, plan215, plan216, plan217, plan218, plan219, plan220, plan221, plan222, plan223, plan224, plan225, plan226, plan227, plan228, plan229, plan230, plan231, plan232, plan233, plan234, plan235, "
[5] " plan236, plan237, plan238, plan239, plan240, plan241, plan242, plan243, plan244, plan245, plan246, plan247, plan248, plan249, plan250, plan251, plan252, plan253, plan254, plan255, plan256, plan257, plan258, plan259, plan260, plan261, plan262, plan263, plan264, plan265, plan266, plan267, plan268, plan269, plan270, plan271, plan272, plan273, plan274, plan275, plan276, plan277, plan278, plan279, plan280, plan281, plan282, plan283, plan284, plan285, plan286, plan287, plan288, plan289, plan290, plan291, "
[6] " plan292, plan293, plan294, plan295, plan296, plan297, plan298, plan299, plan300, plan301, plan302, plan303, plan304, plan305, plan306, plan307, plan308, plan309, plan310, plan311, plan312, plan313)"
When model.matrix.default tests this against the variables in the data.frame, it returns an error.
The problem
To get around this, I've written a new deparse function:
deparse <- function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
}
However, when I run mnp again and step through, it returns the same error for the same reason (base::deparse is being run, not my deparse).
This is somewhat surprising to me, as what I expect is more typified by this example, where the user-defined function temporarily over-writes the base function:
> print <- function() {
+ cat("user-defined print ran\n")
+ }
> print()
user-defined print ran
I realize the right way to solve this problem is to rewrite model.matrix.default, but as a tool for debugging I'm curious how to force it to use my deparse and why the anticipated (by me) behavior is not happening here.
The functions fixInNamespace and assignInNamespace are provided to allow editing of existing functions. You could try ... but I will not since mucking with deparse looks too dangerous:
assignInNamespace("deparse",
function (expr, width.cutoff = 60L, backtick = mode(expr) %in%
c("call", "expression", "(", "function"), control = c("keepInteger",
"showAttributes", "keepNA"), nlines = -1L) {
ret <- .Internal(deparse(expr, width.cutoff, backtick, .deparseOpts(control), nlines))
paste0(ret,collapse="")
} , "base")
There is an indication on the help page that the use of such functions has restrictions and I would not be surprised that such core function might have additional layers of protection. Since it works via side-effect, you should not need to assign the result.
This is how packages with namespaces search for functions, as described in Section 1.6, Package Namespaces of Writing R Extensions Namespaces are sealed once they are loaded. Sealing means that imports and exports cannot be changed and that internal variable bindings cannot be changed. Sealing allows a simpler implementation strategy for the namespace mechanism. Sealing also allows code analysis and compilation tools to accurately identify the definition corresponding to a global variable reference in a function body. The namespace controls the search strategy for variables used by functions in the package. If not found locally, R searches the package namespace first, then the imports, then the base namespace and then the normal search path.