Error when setting SCIP as solver with PYOMO - anaconda

I am trying to solve a MILP problem set up in PYOMO whith SCIP as solver.
I run the problem from within PyDev using the python ANACONDA interpreter.
I can run and solve the problem with other solvers, namely CBC, GLPK, and IPOPT.
However, it does not work when using SCIP as solver. It seems like there is something wrong with the SCIP/AMPL interface... Anybody can help out?
Below are some details on the error prompt and system configuration.
I tried with 'scip' and 'scipampl'.
With 'scip'
opt = SolverFactory('scip')
instance = model.create_instance("test1.dat")
results = opt.solve(instance)
instance.display()
WARNING: "[base]/site-packages/pyomo/solvers/plugins/solvers/SCIPAMPL.py", 68, _default_executable
Could not locate the 'scipampl' executable, which is required for solver scip
Traceback (most recent call last):
File "/home/alessandro/Documents/Eclipse workspace/test1/src/test1.py", line 48, in
results = opt.solve(instance) # solves and updates instance
File "/home/alessandro/anaconda3/lib/python3.6/site-packages/pyomo/opt/base/solvers.py", line 539, in solve
self.available(exception_flag=True)
File "/home/alessandro/anaconda3/lib/python3.6/site-packages/pyomo/opt/solver/shellcmd.py", line 122, in available
raise ApplicationError(msg % self.name)
pyutilib.common._exceptions.ApplicationError: No executable found for solver 'scip'
With 'scipampl'
opt = SolverFactory('scipampl')
instance = model.create_instance("test1.dat")
results = opt.solve(instance)
instance.display()
WARNING: "[base]/site-packages/pyomo/opt/base/solvers.py", 202, solver_call
Failed to create solver with name 'scipampl':
Failed to set executable for solver asl. File with name=scipampl either does not exist or it is not executable. To skip this validation, call set_executable with validate=False.
Traceback (most recent call last):
File "/home/alessandro/Documents/Eclipse workspace/test1/src/test1.py", line 48, in
results = opt.solve(instance)
File "/home/alessandro/anaconda3/lib/python3.6/site-packages/pyomo/opt/base/solvers.py", line 127, in solve
self._solver_error('solve')
File "/home/alessandro/anaconda3/lib/python3.6/site-packages/pyomo/opt/base/solvers.py", line 155, in _solver_error
+ "\n\toptions: %s" % ( self.options, ) )
RuntimeError: Attempting to use an unavailable solver.
The SolverFactory was unable to create the solver "scipampl"
and returned an UnknownSolver object. This error is raised at the point
where the UnknownSolver object was used as if it were valid (by calling
method "solve").
The original solver was created with the following parameters:
executable: scipampl
type: scipampl
_args: ()
options: {}
The Eclipse version is "Oxygen.1 (4.7.1)" and the PyDev version is "6.0.0".
The python, pyomo, and solvers versions are as follows:
~$ python
Python 3.6.2 |Anaconda custom (64-bit)| (default, Sep 30 2017, 18:42:57) [GCC 7.2.0] on linux
~$ pyomo --version
Pyomo 5.2 (CPython 3.6.2 on Linux 4.10.0-35-generic)
~$ cbc
Welcome to the CBC MILP Solver
Version: 2.9.9
Build Date: Jul 6 2017
~$ glpsol
GLPSOL: GLPK LP/MIP Solver, v4.63
~$ ipopt
No stub!
usage: ipopt [options] stub [-AMPL] [<assignment> ...]
~$ scip
SCIP version 4.0.1 [precision: 8 byte] [memory: block] [mode: optimized] [LP solver: SoPlex 3.0.1] [GitHash: 8a04b84]
Copyright (C) 2002-2017 Konrad-Zuse-Zentrum fuer Informationstechnik Berlin (ZIB)
External codes:
SoPlex 3.0.1 Linear Programming Solver developed at Zuse Institute Berlin (soplex.zib.de) [GitHash: 24cffa9]
CppAD 20160000.1 Algorithmic Differentiation of C++ algorithms developed by B. Bell (www.coin-or.org/CppAD)
ZLIB 1.2.8 General purpose compression library by J. Gailly and M. Adler (zlib.net)
GMP 6.1.0 GNU Multiple Precision Arithmetic Library developed by T. Granlund (gmplib.org)
ZIMPL 3.3.4 Zuse Institute Mathematical Programming Language developed by T. Koch (zimpl.zib.de)
Ipopt 3.12.8 Interior Point Optimizer developed by A. Waechter et.al. (www.coin-or.org/Ipopt)
user parameter file <scip.set> not found - using default parameters
SCIP> ^C
I installed the CBC, GLPK and IPOT packages as follows:
conda install -c conda-forge coincbc
conda install -c conda-forge glpk
conda install -c conda-forge ipopt
I installed the latest SCIPOPTSUITE version "4.0.1" with the Linux-Debian package "SCIPOptSuite-4.0.1-Linux.deb"
I also installed AMPL-MP SCIPOPTLIB and PYSCIPOPT as follows:
conda install -c conda-forge ampl-mp
conda install -c leethargo scipoptlib
conda install -c leethargo pyscipopt

EDIT:
follow these instructions: http://zverovich.net/2012/08/07/using-scip-with-ampl.html
Several things that might help:
the conda package at leethargo is not the official conda package (because there is none)
try installing and running PySCIPOpt via pip first
PySCIPOpt is not interfaced to PYOMO, so you most likely need to rely on AMPL for the SCIP support
does PYOMO even support SCIP? I couldn't figure that out from their documentation
Ah, look what I found after googling "pyomo scip":
https://groups.google.com/forum/#!topic/pyomo-forum/peBjIvkVwtQ
https://github.com/Pyomo/pyomo/issues/48
So, PYOMO only supports SCIP through the AMPL interface (ASL).

Related

Installing IPOPT nonlinear solvers

I am trying to install ipopt with anaconda on Windows10 x64.
I already installed ipopt via "conda install -c conda-forge ipopt" in the anaconda prompt (version=3.13.4).
I built a small test-script in python to check my installation:
import pyomo.environ as pe
import pyomo.opt as po
M = pe.ConcreteModel()
M.x = pe.Var()
M.y = pe.Var(bounds=(0, None))
M.obj = pe.Objective(expr=M.x+M.y, sense=pe.minimize)
M.c1 = pe.Constraint(expr=(M.x >=4)) // this constraint expr will be replaced by M.x**2 >= 4 to test nonlinear problems
M.c2 = pe.Constraint(expr=(M.y >= M.x))
solver = po.SolverFactory('ipopt')
results = solver.solve(M, tee=True)
print(pe.value(M.x), pe.value(M.y))
Running the script, I received the following error message:
ApplicationError: No executable found for solver 'ipopt'
I could solve this problem downloading the zip-folder for 64-bit windows from https://ampl.com/products/solvers/open-source/ and extracting the files into the "Library"-folder of the anaconda environment I am working on.
Running the test script for the linear problem again, the problem is solved but I get the following information:
This is Ipopt version 3.12.13, running with linear solver mumps.
NOTE: Other linear solvers might be more efficient (see Ipopt documentation).
The nonlinear test problem, however, cannot be solved as still the linear mumps solver is applied.
EXIT: Maximum Number of Iterations Exceeded.
WARNING: Loading a SolverResults object with a warning status into
model.name="unknown";
- termination condition: maxIterations
- message from solver: Ipopt 3.12.13\x3a Maximum Number of Iterations
Exceeded.
M.x=-281240.49215541 M.y=13.01195537079578
How can I make sure that nonlinear problems are solved by ipopt as well? I am aware that there are a few similar questions on stackoverflow, but I wasn't able to solve my problem yet.
(I also tried to follow the installation guide from https://coin-or.github.io/Ipopt/INSTALL.html, but encountered the message "configure: error: no acceptable C compiler found in $PATH""" in the MSYS2 console when I tried to install ASL or HSL, even though I installed gcc as described and included the folder which contains gcc.exe into my PATH-variable.)
Probably the conda-forge package doesn't include the AMPL interface and the ipopt executable that is required by Pyomo (at least, that is the case for https://anaconda.org/conda-forge/ipopt/3.13.4/download/win-64/ipopt-3.13.4-hf6be2e5_0.tar.bz2).
MUMPS should not be the reason if Ipopt failed to solve min x+y s.t. x>=4, y>=x. One would need to inspect the Ipopt log to see what is going wrong.
If you feel that the executable from ampl.com is too old, you can try the ones that are attached to the Ipopt releases, e.g., https://github.com/coin-or/Ipopt/releases/download/releases%2F3.14.1/Ipopt-3.14.1-win64-msvs2019-md.zip for the latest.
They also include Pardiso from MKL in addition to MUMPS.
To check why the compiler check in configure failed under msys2, have a look into file config.log that is generated by configure.

Installing Octave Symbolic Package using SymPy (macOS) - Can't get SymPy to work

So, I've installed the Octave Symbolic Package from here:
https://sourceforge.net/p/octave/symbolic/ci/master/tree/
In the Octave command window I typed:
pkg install -forge symbolic
After the install I ran a test (as shown in link above) by typing:
pkg load symbolic
syms x
However, I got an following error with syms x.
Symbolic pkg v2.8.0: Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named sympy
error: Python cannot import SymPy: have you installed SymPy?
Try "sympref diagnose" for more information.
error: called from
assert_have_python_and_sympy at line 123 column 7
python_ipc_popen2 at line 79 column 5
python_ipc_driver at line 59 column 13
python_cmd at line 163 column 9
valid_sym_assumptions at line 38 column 10
assumptions at line 82 column 7
syms at line 97 column 13
Next, I visited the following site to install SymPy (which says to install Anaconda).
https://docs.sympy.org/latest/install.html#anaconda
So, then I go to the following site and download the graphical installer for macOS (Python 2.7 version).
https://www.anaconda.com/distribution/#download-section
Then I tested from the macOS terminal to see if Anaconda is installed using this command
conda --version
which returns conda 4.7.11.
I also typed
conda list
which returns (among a long list). sympy 1.4 py27_0.
So, it appears that SymPy is installed (at least at the terminal).
Next, I go back to the command line of Octave and typed:
sympref diagnose
Here is the output:
Symbolic package diagnostics
============================
Python and SymPy are needed for most features of the Symbolic package.
The Python interpreter is currently: "python".
Computers may have more than one Python interpreter installed. If you
need to, you can select a different one using the PYTHON environment
variable (see "help sympref"). For example, to use Python 3, try
setenv PYTHON python3
sympref reset
Attempting to run python -c "print(\"Python says hello\")"
status = 0
output = Python says hello
Good, Python ran correctly.
Python version
--------------
Let's check what version of Python we are calling...
Attempting to run python -c "import sys; print(sys.version)"
status = 0
output = 2.7.15 (default, Feb 5 2019, 12:31:36)
[GCC 4.2.1 Compatible Apple LLVM 8.0.0 (clang-800.0.42.1)]
SymPy Python Library
--------------------
SymPy is a Python library used by Symbolic for almost all features.
Attempting to run python -c "import sympy; print(sympy.__version__)"
Traceback (most recent call last):
File "<string>", line 1, in <module>
ImportError: No module named sympy
status = 1
output =
Unfortunately status was non-zero: probably Python cannot import sympy.
* Is there an error message above?
* Do you have SymPy installed? If not, please try to install it and
try again.
* If you do have SymPy installed, maybe it's installed for a different
Python interpreter than the one we found? Please try "setenv" as
described above to change your python interpreter.
Octave doesn't have access to SymPy.
Any idea what I did wrong?
Try running these in octave...
setenv PYTHON /usr/local/anaconda3/bin/python
pkg load symbolic
/usr/local/anaconda3/bin was where all my anaconda bits were installed,
including a version of python. Yours may differ.
sympref diagnose then passes for me.
You will have to do this every time you start 'octave' before using the symbolic package, but it should be safe.

ipython can't load a module which python does

I have recently updated to OSX El Capitain upgrading an exhisting working osx installation. Now I have a module, installed via pip on an anaconda distribution which is now broken. Or better the module can be correctly imported from python
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Sep 15 2015, 14:29:08)
[GCC 4.2.1 (Apple Inc. build 5577)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
Anaconda is brought to you by Continuum Analytics.
Please check out: http://continuum.io/thanks and https://anaconda.org
>>> import MDSplus as mds
whereas it does not load on ipython, claiming not to finding a library.
Python 2.7.10 |Anaconda 2.3.0 (x86_64)| (default, Sep 15 2015, 14:29:08)
Type "copyright", "credits" or "license" for more information.
IPython 4.0.0 -- An enhanced Interactive Python.
? -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help -> Python's own help system.
object? -> Details about 'object', use 'object??' for extra details.
Using matplotlib backend: MacOSX
In [1]: import MDSplus as mds
Error importing MDSplus package: Error finding library: MdsShr
The library is correctly installed, it is listed in /etc/profile, it is located in /usr/local (so there should not be a problem with the new security system of OSX el Captain) and the python interpreted which is called via python or ipython is the same (Anaconda 2.3.0).
How can I diagnose which is the problem?
EDIT:
the solution proposed in https://groups.google.com/a/continuum.io/forum/#!topic/anaconda/PEuOKEM5fdc does not worked for me. It does not seem to be an environment problem and the command
echo $DYLD_LIBRARY_PATH
points to the correct path
EDIT:
which -a ipython
ipython is /Users/vianello/anaconda/bin/ipython
which -a python
python is /Users/vianello/anaconda/bin/python
python is /usr/bin/python
>>> import sys
>>> for x in sys.path: print x
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/pyhht-0.0.1-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/MitDevices-0.3-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.157-py2.7.egg
>>>//anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.147-py2.7.egg/MDSplus
>>>/Users/vianello/anaconda/lib/python27.zip
>>>/Users/vianello/anaconda/lib/python2.7
>>>/Users/vianello/anaconda/lib/python2.7/plat-darwin
>>>/Users/vianello/anaconda/lib/python2.7/plat-mac
>>>/Users/vianello/anaconda/lib/python2.7/plat-mac/lib-scriptpackages
>>>/Users/vianello/anaconda/lib/python2.7/lib-tk
>>>/Users/vianello/anaconda/lib/python2.7/lib-old
>>>/Users/vianello/anaconda/lib/python2.7/lib-dynload
>>>/Users/vianello/anaconda/lib/python2.7/site-packages
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/PIL
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/aeosa
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/lmfit-0.8.3-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/setuptools-18.3.2-py2.7.egg
>>>/Users/vianello/anaconda/lib/python2.7/site-packages/suds-0.4-py2.7.egg
in iPython the same give the results
/Users/vianello/anaconda/bin
/Users/vianello/anaconda/lib/python2.7/site-packages/pyhht-0.0.1-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/MitDevices-0.3-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/MDSplus-alpha_7.0.157-py2.7.egg
/Users/vianello/anaconda/lib/python27.zip
/Users/vianello/anaconda/lib/python2.7
/Users/vianello/anaconda/lib/python2.7/plat-darwin
/Users/vianello/anaconda/lib/python2.7/plat-mac
/Users/vianello/anaconda/lib/python2.7/plat-mac/lib-scriptpackages
/Users/vianello/anaconda/lib/python2.7/lib-tk
/Users/vianello/anaconda/lib/python2.7/lib-old
/Users/vianello/anaconda/lib/python2.7/lib-dynload
/Users/vianello/anaconda/lib/python2.7/site-packages/Sphinx-1.3.1-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/lmfit-0.8.3-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/setuptools-18.3.2-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages/suds-0.4-py2.7.egg
/Users/vianello/anaconda/lib/python2.7/site-packages
/Users/vianello/anaconda/lib/python2.7/site-packages/PIL
/Users/vianello/anaconda/lib/python2.7/site-packages/aeosa
/Users/vianello/anaconda/lib/python2.7/site-packages/IPython/extensions
/Users/vianello/.ipython
Thus the MDSplus-alpha_7.0.157-py2.7.egg can be seen both from python and ipython
My first thought is that you are running afoul of the new Apple System Integrity Protection. iPython is launched using a shell script that loads via /bin/bash. That will result in your DYLD_LIBRARY_PATH environment variable being stripped when python launches. You can check this by importing os and then attempting to print the contents of os.environ["DYLD_LIBRARY_PATH"]. If that print throws an exception then you know what the problem is.
A simple fix that works for me is to edit the first line of the ipython script so that the #! calls your python binary directly rather than going via bash. This is not a long term solution as it will have to be redone each time ipython is updated until upstream changes (to be fair I'm not sure why bash is involved).
I have written a report on python and library paths at http://dmtn-001.lsst.io

TypeError when downloading rapidsms using pip in django-tables2

I'm a total python newbie. I installed python 3.3.1 on a 32 bit windows 7 professional. I'm trying to install RapidSMS, and it should be as easy as "pip install rapidsms" and it does start the process, but it doesn't complete and I'm left with the below error message.
I've been trying to google it, but I haven't been able to find this specific problem, for the error I find fixes for people who have written the code themselves, and I haven't seen anyone mention this problem about rapidsms themselves. Since it stops in Django-tables, I wonder if I messed up that installation somehow or if there's a problem with the python version compatibility. I've used pip when installing some other software, so I don't think that's the problem.
So if anyone's encountered this error when installing pyhton packages or really have any idea what the cause might be I'd really appreciate it! (I also plan to post this in the RapidSMS mailing list when I get approved, but wanted to see if this was a more general problem that might have a fix.)
Downloading/unpacking django-tables2==0.13.0 (from rapidsms)
Running setup.py egg_info for package django-tables2
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "c:\users\mhealth1\appdata\local\temp\pip-build-mhealth1\django-tables2\setup.py", line 7, in <module>
version = re.search('__version__ = "(.+?)"', f.read()).group(1)
File "C:\Python33\lib\re.py", line 161, in search
return _compile(pattern, flags).search(string)
TypeError: can't use a string pattern on a bytes-like object
Complete output from command python setup.py egg_info:
Traceback (most recent call last):
File "<string>", line 16, in <module>
File "c:\users\mhealth1\appdata\local\temp\pip-build-mhealth1\django-tables2\setup.py", line 7, in <module>
version = re.search('__version__ = "(.+?)"', f.read()).group(1)
File "C:\Python33\lib\re.py", line 161, in search
return _compile(pattern, flags).search(string)
TypeError: can't use a string pattern on a bytes-like object
----------------------------------------
Command python setup.py egg_info failed with error code 1 in c:\users\mhealth1\a
ppdata\local\temp\pip-build-mhealth1\django-tables2
Storing complete log in C:\Users\mhealth1\pip\pip.log
Django-tables2 fails to install in Python3 because of the following code from line 7 of setup.py in the traceback:
version = re.search('__version__ = "(.+?)"', f.read()).group(1)
This should work if the search pattern was a bytes object. A byte literal can be created by simply pre-pending the string literal with a b like the following:
version = re.search(b'__version__ = "(.+?)"', f.read()).group(1)
This is why Python is throwing a TypeError with the message "Can't use a string pattern on a bytes-like object". The file contents are being read in as bytes.
I don't think RapidSMS currently supports Python3 yet based on its list of environments that are tested. This can be seen in the tox environments list of the project seen here, in which only Python 2.6 and 2.7 are listed: https://github.com/rapidsms/rapidsms/blob/develop/tox.ini#L2
To solve the immediate problem, you'll want to install RapidSMS in a virtual environment with Python 2.6 or 2.7. The documentation for RapidSMS describes virtual environment setup briefly here: http://www.rapidsms.org/en/develop/topics/virtualenv.html
You'll want to install Python 2.6 or 2.7 on your system and specify which Python for the virtualenv to use, using the -p or --python argument. The following is taken from the RapidSMS docs linked above, but ammended to specify using Python 2.7:
mkvirtualenv --distribute --no-site-packages rapidsms --python=python2.7

Numpy on osx 10.8.2 and python 3.2.3

What EXACTLY do I have to do to get numpy to work? I've read that it's supported in 3.2.3 and that it should work by using setup.py. I'm getting errors talking about os_path. Is there something I'm missing?
Here is the full traceback:
Python 3.2.3 (v3.2.3:3d0686d90f55, Apr 10 2012, 11:25:50)
[GCC 4.2.1 (Apple Inc. build 5666) (dot 3)] on darwin
Type "copyright", "credits" or "license()" for more information.
>>> ================================ RESTART ================================
>>>
Converting to Python3 via 2to3...
Traceback (most recent call last):
File "/Users/cameron/Downloads/numpy-master/setup.py", line 214, in <module>
setup_package()
File "/Users/cameron/Downloads/numpy-master/setup.py", line 175, in setup_package
__file__ = os.path.join(os.curdir, os.path.basename(__file__))
NameError: global name '__file__' is not defined
>>>
You are probably using the Python version pre-installed with Mac OS X. I strongly suggest you use a distribution where all the components (python, scipy, numpy, etc) are built to work together properly, like macports. Macports is a package manager for Open Source software, and it automatically resolves and installs dependencies.
After the installation of Macports, the only thing you need to do is type "sudo port install py27-numpy" or "sudo port install py33-numpy" at the command prompt, depending on whether you prefer to use Python 2 or 3 (some packages might not run on Python3 yet). This will automatically download and install the correct version of python and all other software packages that might be required for numpy to run properly.
Try to compile a Python from scratch first (by downloading the source) and not using the one shipped by Apple. See the recommendation on the related SciPy/Numpy page.

Resources