Parallel processing in Julia throws errors - parallel-processing

My understanding is that parallelization is included by default in a base Julia installation.
However, when I try to use it, I am getting errors that the functions and macros are not defined. For example:
nprocs()
Throws an error:
ERROR: UndefVarError: nprocs not defined
Stacktrace:
[1] top-level scope at none:0
Nowhere in any Julia documentation can I find mention of any packages that need to be included in order to use these functions. Am I missing something here?
I am using Julia version 1.0.5 inside the JuliaPro/Atom IDE

I figured it out. I'll leave this up for anyone else who is having this problem.
The solution is to import the Distributed package using:
using Distributed
Why this is not included in the documentation I do not know.

Once you know that nproc needs to be used, there exist a couple of options to find where it is defined.
A search through the documentation can help: https://docs.julialang.org/en/v1/search/?q=nprocs
Without leaving the Julia REPL, and even before nprocs gets imported in your session, you can use apropos in order to find more about it and determine that it is needed to import the Distributed package:
julia> apropos("nprocs")
Distributed.nprocs
Distributed.addprocs
Distributed.nworkers
An other way of using apropos is via the help REPL mode:
julia> # type `?` when the cursor is right after the prompt to enter help REPL mode
# note the use of double quotes to trigger "apropos" instead of a regular help query
help?> "nprocs"
Distributed.nprocs
Distributed.addprocs
Distributed.nworkers
Previous options work well in the case of nprocs because it is part of the standard library. JuliaHub is another option which allows looking for things more broadly, in the entire Julia ecosystem. As an example, looking for nprocs in JuliaHub's "Doc Search" tool also returns relevant results: https://juliahub.com/ui/Documentation?q=nprocs

Related

What is `ac_cv_func_malloc_0_nonnull` as provided to ./configure?

I'm cross-compling with mingw and got this error:
undefined reference to `rpl_realloc'
After some searching I found this can be resolved as follows in configure.ac or as environment variables set prior to calling ./mingw64-configure:
ac_cv_func_malloc_0_nonnull=yes
ac_cv_func_realloc_0_nonnull=yes
What defines these macros, and as there any documentation on the subject? I couldn't find any...
What defines these macros, and as there any documentation on the subject?
Autoconf uses the ac_cv_ prefix for its "cache variables", in which it records the results of configuration tests it has performed. In the event that the same check is requested multiple times, these allow it to use the previously-determined result instead of performing the check again.
The general naming convention for these is documented in the Autoconf manual. The particular cache variable names you ask about are documented to cache the results of the Autoconf's AC_FUNC_MALLOC and AC_FUNC_REALLOC macros, respectively. That documentation also speaks to the rpl_realloc name.
It is allowed to use these variables in configure.ac to programmatically determine the results of those checks, but it is a relatively nasty hack to assign values to those variables directly. In this particular case, however, the error suggests that whoever prepared the autotooling for the project you're trying to build did a sloppy job of it. If fudging the cache variables gets you a successful build and a working program then that's a tempting and much easier alternative to actually fixing the project.

Sourcing data into rstudio [duplicate]

This is meant to be a FAQ question, so please be as complete as possible. The answer is a community answer, so feel free to edit if you think something is missing.
This question was discussed and approved on meta.
I am using R and tried some.function but I got following error message:
Error: could not find function "some.function"
This question comes up very regularly. When you get this type of error in R, how can you solve it?
There are a few things you should check :
Did you write the name of your function correctly? Names are case sensitive.
Did you install the package that contains the function? install.packages("thePackage") (this only needs to be done once)
Did you attach that package to the workspace ?
require(thePackage) (and check its return value) or library(thePackage) (this should be done every time you start a new R session)
Are you using an older R version where this function didn't exist yet?
Are you using a different version of the specific package? This could be in either direction: functions are added and removed over time, and it's possible the code you're referencing is expecting a newer or older version of the package than what you have installed.
If you're not sure in which package that function is situated, you can do a few things.
If you're sure you installed and attached/loaded the right package, type help.search("some.function") or ??some.function to get an information box that can tell you in which package it is contained.
find and getAnywhere can also be used to locate functions.
If you have no clue about the package, you can use findFn in the sos package as explained in this answer.
RSiteSearch("some.function") or searching with rdocumentation or rseek are alternative ways to find the function.
Sometimes you need to use an older version of R, but run code created for a newer version. Newly added functions (eg hasName in R 3.4.0) won't be found then. If you use an older R version and want to use a newer function, you can use the package backports to make such functions available. You also find a list of functions that need to be backported on the git repo of backports. Keep in mind that R versions older than R3.0.0 are incompatible with packages built for R3.0.0 and later versions.
Another problem, in the presence of a NAMESPACE, is that you are trying to run an unexported function from package foo.
For example (contrived, I know, but):
> mod <- prcomp(USArrests, scale = TRUE)
> plot.prcomp(mod)
Error: could not find function "plot.prcomp"
Firstly, you shouldn't be calling S3 methods directly, but lets assume plot.prcomp was actually some useful internal function in package foo. To call such function if you know what you are doing requires the use of :::. You also need to know the namespace in which the function is found. Using getAnywhere() we find that the function is in package stats:
> getAnywhere(plot.prcomp)
A single object matching ‘plot.prcomp’ was found
It was found in the following places
registered S3 method for plot from namespace stats
namespace:stats
with value
function (x, main = deparse(substitute(x)), ...)
screeplot.default(x, main = main, ...)
<environment: namespace:stats>
So we can now call it directly using:
> stats:::plot.prcomp(mod)
I've used plot.prcomp just as an example to illustrate the purpose. In normal use you shouldn't be calling S3 methods like this. But as I said, if the function you want to call exists (it might be a hidden utility function for example), but is in a namespace, R will report that it can't find the function unless you tell it which namespace to look in.
Compare this to the following:
stats::plot.prcomp
The above fails because while stats uses plot.prcomp, it is not exported from stats as the error rightly tells us:
Error: 'plot.prcomp' is not an exported object from 'namespace:stats'
This is documented as follows:
pkg::name returns the value of the exported variable name in namespace pkg, whereas pkg:::name returns the value of the internal variable name.
I can usually resolve this problem when a computer is under my control, but it's more of a nuisance when working with a grid. When a grid is not homogenous, not all libraries may be installed, and my experience has often been that a package wasn't installed because a dependency wasn't installed. To address this, I check the following:
Is Fortran installed? (Look for 'gfortran'.) This affects several major packages in R.
Is Java installed? Are the Java class paths correct?
Check that the package was installed by the admin and available for use by the appropriate user. Sometimes users will install packages in the wrong places or run without appropriate access to the right libraries. .libPaths() is a good check.
Check ldd results for R, to be sure about shared libraries
It's good to periodically run a script that just loads every package needed and does some little test. This catches the package issue as early as possible in the workflow. This is akin to build testing or unit testing, except it's more like a smoke test to make sure that the very basic stuff works.
If packages can be stored in a network-accessible location, are they? If they cannot, is there a way to ensure consistent versions across the machines? (This may seem OT, but correct package installation includes availability of the right version.)
Is the package available for the given OS? Unfortunately, not all packages are available across platforms. This goes back to step 5. If possible, try to find a way to handle a different OS by switching to an appropriate flavor of a package or switch off the dependency in certain cases.
Having encountered this quite a bit, some of these steps become fairly routine. Although #7 might seem like a good starting point, these are listed in approximate order of the frequency that I use them.
If this occurs while you check your package (R CMD check), take a look at your NAMESPACE.
You can solve this by adding the following statement to the NAMESPACE:
exportPattern("^[^\\\\.]")
This exports everything that doesn't start with a dot ("."). This allows you to have your hidden functions, starting with a dot:
.myHiddenFunction <- function(x) cat("my hidden function")
I had the error
Error: could not find function some.function
happen when doing R CMD check of a package I was making with RStudio. I found adding
exportPattern(".")
to the NAMESPACE file did the trick. As a sidenote, I had initially configured RStudio to use ROxygen to make the documentation -- and selected the configuration where ROxygen would write my NAMESPACE file for me, which kept erasing my edits. So, in my instance I unchecked NAMESPACE from the Roxygen configuration and added exportPattern(".") to NAMESPACE to solve this error.
This error can occur even if the name of the function is valid if some mandatory arguments are missing (i.e you did not provide enough arguments).
I got this in an Rcpp context, where I wrote a C++ function with optionnal arguments, and did not provided those arguments in R. It appeared that optionnal arguments from the C++ were seen as mandatory by R. As a result, R could not find a matching function for the correct name but an incorrect number of arguments.
Rcpp Function : SEXP RcppFunction(arg1, arg2=0) {}
R Calls :
RcppFunction(0) raises the error
RcppFunction(0, 0) does not
Rdocumentation.org has a very handy search function that - among other things - lets you find functions - from all the packages on CRAN, as well as from packages from Bioconductor and GitHub.
If you are using parallelMap you'll need to export custom functions to the slave jobs, otherwise you get an error "could not find function ".
If you set a non-missing level on parallelStart the same argument should be passed to parallelExport, else you get the same error. So this should be strictly followed:
parallelStart(mode = "<your mode here>", N, level = "<task.level>")
parallelExport("<myfun>", level = "<task.level>")
You may be able to fix this error by name spacing :: the function call
comparison.cloud(colors = c("red", "green"), max.words = 100)
to
wordcloud::comparison.cloud(colors = c("red", "green"), max.words = 100)
I got the same, error, I was running version .99xxx, I checked for updates from help menu and updated My RStudio to 1.0x, then the error did not come
So simple solution, just update your R Studio

Can't list source using debug in ruby 1.8 [duplicate]

With this minimal ruby code:
require 'debug'
puts
in a file called, e.g. script.rb
if I launch it like so: ruby -rdebug script.rb
and then press l on the debug prompt, I get the listing, as expected
if I instead run it normally as ruby script.rb
when pressing l I get:
(rdb:1) l
[-3, 6] in script.rb
No sourcefile available for script.rb
The error message seems misleading at best: the working directory hasn't changed, and the file is definitely still there!
I'm unable to find documentation on this behavior (I tried it on both jruby and mri, and the behavior is the same)
I know about 'debugger' and 'pry', but they serve a different use case:
I'm used to other scripting languages with a builtin debug module, that can let me put a statement anywhere in the code to drop me in a debug shell, inspect code, variables and such... the advantage of having it builtin it's that it is available everywhere, without having to set up an environment for it, or even when I'm on a machine that's not my own
I could obviously workaround this by always calling the interpreter with -rdebug and manually setting the breakpoint, but I find this more work than the alternative
After looking into the debug source code, I found a workaround and a fix:
the workaround can be:
trace on
next
trace off
list
this will let you get the listing without restarting the interpreter with -rdebug, with the disadvantage that you'll get some otherwise unwanted output from the tracing, and you'll be able to see it only after moving by one statement
for the fix: the problem is that the SCRIPT_LINES__ Hash lacks a value for the current file... apparently it's only set inside tracer.rb
I've changed line 161, and changed the Hash with a subclass that tracks where []= has been called from, and I wasn't able to dig up the actual code that does the work when stepping into a function that comes from a different file
Also: I haven't found a single person yet who actively uses this module (I asked both on #ruby, #jruby and #pry on freenode), and together with the fact that it uses a function that is now obsolete it leads me to be a bit pessimistic about the maintenance state of this module
nonetheless, I submitted a pull request for the fix (it's actually quite dumb and simple, but to do otherwise I'd need a deeper understanding of this module, and possibly to refactor some parts of it... but if this module isn't actively maintaned, I'm not sure that's a good thing to put effort on)
I suspect the Ruby interpreter doesn't have the ability to load the sourcefile without the components in the debug module. So, no -rdebug, no access to the sourcefile. And I agree it is a misleading error. "Debugging features not loaded" might be better.

Sequences in Maple [Unable to Execute]

I've been working on a programme to solve any maximisation LPP using the Revised Simplex Method. I have a problem with it though as I'm trying to input a sequence to solve the problem of non-basic variables.
My code is as follows:
matmax:=proc(tableau,basic)
local pivot,T,nbv,n,m,b;
T:=evalm(tableau);
n:=coldim(T); m:=rowdim(T);
b:=evalm(basic);
print(evalm(T));
nbv:={seq(i,i=2..n-1)}minus{seq(b[i],i=1..m)};
pivot:=getpiv(T,nbv);
while not pivot=FAIL do
b[pivot[1]]:=pivot[2];
T:=evalm(gauss(col(T,pivot[2]),pivot[1])&*T);
print(evalm(T));
nbv:={seq(i,i=2,..n-1)}minus{seq(b[i],i=1..m)};
pivot:=getpiv(T,nbv);
od;
[evalm(T),evalm(b)];
end;
The gauss and getpiv commands are procedures written to work in this programme, these work fine. However upon executing this procedure Maple returns with the message "Error, (in matmax) unable to execute seq" If anyone can give me any help on how to fix this problem it would be greatly appreciated.
If you have not loaded the linalg package before calling your matxmax then commands like coldim will simplify not work and not produce the integer results for n and m that are expected when using those in the bound of the seq calls. I believe that is why your seq error occurs, because n and m are not being assigned integer intermediate results like you expect.
You could try to remedy this by either loading the package before calling matmax, with with(linalg). But that is not so robust, and there are scenarios where it might not work. The with command won't work within a procedure body, so you can't put that inside the defn of proc matmax.
You could insert a line into matmax above, say, the local declaration line, like,
uses linalg;
That would make coldim and friends from the linalg package work. Unfortunately you've used the name pivot as a local variable, and that clashes with the pivot export from linalg package. So that simple fix via uses is not quite enough. You could use some name other than pivot, and that simple uses line, sure.
My own preference would be to make everything fully explicit, so that later on you or anyone else reading the code can understand it more clearly, even if it's longer. So I would use linalg[coldim] instead of coldim, and so on for the other exports uses from the linalg package within matmax.
Having said all the above, you should know that the linalg package is deprecated in modern Maple and that LinearAlgebra is the new package that offers the functionality you seem to be using. The command names are longer, but using the newer package means that you don't need all those evalm calls (or anything like it).
The problem could lie in your gauss and getpiv commands as they may not work with your procedure, could you expand on what they do?

Reading in Packages in Mathematica

I've created a package in Mathematica but I can't seem to get Mathematica to read it in. The Package is of the form:
BeginPackage["name`"]
(function[]::usage)
Begin["`Private'"]
(functions)
End[]
EndPackage[]
I saved this file as a .m. The problem is that after I quit the kernel and then try to read in the package using Needs["name`"], I always get a no::cont error. I've tried saving the file in $BaseUserDirectory and $BaseDirectory, but it still give me a no::cont error:
Needs["name"]
Needs::nocont: "Context \!\(\"name\") was not created when Needs was evaluated."
I've also tried using the built-in File->Install function in Mathematica but it still gives me the same error. Is there something that I need to change with regards to the context?
Any help is much appreciated.
Thank you,
jm
Write the definitions in "Initialization Cells" when you create the package notebook. Otherwise they will be ignored. This worked for me with Version 9 on OS X. In previous versions it was probably not necessary, but I don't remember any more... :-)
It's because the directory of your package is not in $Path. Needs only searches the packages in $Path, while Get can search subdirectories.

Resources