After successfully building TensorFlow with GPU support, I'm trying to build the pip package and I'm getting an error saying it can't read the simple_console_for_windows.zip file.
I've confirmed that the file is in C:\tensorflow\bazel-bin\tensorflow\tools\pip_package folder, but it is 0 bytes.
This is my pip build command:
bazel-bin\tensorflow\tools\pip_package\build_pip_package C:/tmp/tensorflow_pkg
This is the full error:
Unzipping simple_console_for_windows.zip to create runfiles tree...
[./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip]
End-of-central-directory signature not found. Either this file is not
a zipfile, or it constitutes one disk of a multi-part archive. In the
latter case the central directory and zipfile comment will be found on
the last disk(s) of this archive.
unzip: cannot find zipfile directory in one of ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip or
./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.zip, and cannot find ./bazel-bin/tensorflow/tools/pip_package/simple_console_for_windows.zip.ZIP, period.
I'm on Windows 10, using Bazel 0.16.1, Tensorflow 1.11, CUDA 9.2 and CUDNN 7.2.1.
My build command was:
bazel build --config=opt --config=cuda //tensorflow/tools/pip_package:build_pip_package
Anyone have an idea what I could try or check to get the pip package to build?
Here is an open issue with workaround. https://github.com/tensorflow/tensorflow/issues/20332
Edit the following file to remove all the lines with ".zip".
bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip-0.params
Run the following command line to generate the required zip file, which is about 107 MB in my case. The verbose option will list all the files.
external\bazel_tools\tools\zip\zipper\zipper.exe vcC bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip #bazel-out/x64_windows-opt/bin/tensorflow/tools/pip_package/simple_console_for_windows.zip-0.params
Now the original pip build command will build a whl.
This is a known issue, and unfortunately its root cause is https://github.com/tensorflow/tensorflow/issues/22390
TF is a big codebase, and when the size of the zip file exceeds 2 GBs, bazel's zipping tools break because they only use zip32.
The ultimate solutions are to contribute a fix to bazel to use zip64, and TensorFlow to lose some weight, the package has grown substantially lately.
However, bazel team was able to give us a workaround.
Only when building the pip package, you should add --define=no_tensorflow_py_deps=true to your bazel command. You will still not be able to build a debug binary, but at least you will be able to build a release binary.
Related
I am having a similar issue to this problem.
I want to download Meson for Windows and used the following command:
pip3 install meson
This installs in my site-packages folder, specifically c:\users\user\appdata\local\packages\pythonsoftwarefoundation.python.3.8_qbz5n2kfra8p0\localcache\local-packages\python38\site-packages\mesonbuild
However, running meson or python3 meson.py results in an error:
'meson' is not recognized as an internal or external command, operable
program or batch file.
When looking at the mesonbuild directory within site-packages, I seem to be missing the meson or meson.py file. Has anybody ever come across this issue before?
After opening up Visual Studio, and looking at the installed Python packages in my environment, I noticed this interesting information window above the list of my Python packages:
Due to new security restrictions, installing from the internet may not
work on this version of Python.
After seeing this, I decided to install Meson through the website's MSI installer. Indeed, after trying to download the installer, Windows threw up all kinds of security warnings and "are you sure you want to do this" notifications before I convinced Windows that I really did want to install Meson.
I just wanted to share this with anybody that might have the same issues. The MSI installer worked for my needs.
Try the following :
python3 -m mesonbuild.mesonmain build
Meson pip package contains meson and mesonbuild modules. The meson module serves as Python entry point, which, during an initial execution of setup.py, associates mesonbuild.mesonmain:main with command line name 'meson'. (Explain Python entry points?). To invoke meson via python3 use python3 -m mesonbuild.mesonmain build, which writes build config into 'build' directory (provided that there is meson.build file in the current directory.) There is no such file 'meson.py' in mesonbuild module and meson module does not contain any Python code.
Hi I downloaded the last release of bazel 0.24.1 for windows and I did the same instructions But, when I run
./compile.sh
It shows the error as mentioned and I tried bazel build //src:bazel but it didn't work to it shows me
-bash: bazel: command not found
after searching I tried : bazel --output_user_root=c:\tmp build //src:bazel.exe
but the same error
-bash: bazel: command not found
Ps: I run it from msys64 any suggestions please thanks a lot for reacting
./compile.sh
🍃 Building Bazel from scratch ERROR: Must specify PROTOC if not
bootstrapping from the distribution artifact
-------------------------------------------------------------------------------- NOTE: This failure is likely occuring if you are trying to bootstrap
bazel from a developer checkout. Those checkouts do not include the
generated output of the protoc compiler (as we prefer not to version
generated files)
.
To build a developer version of bazel, do
bazel build //src:bazel
To bootstrap your first bazel binary, please download a dist archive from our release page at
https://github.com/bazelbuild/bazel/releases and run compile.sh on
the unpacked archive.
The full install instructions to install a release version of bazel
can be found at https://docs.bazel.build/install-compile-source.html
For a rationale, why the bootstrap process is organized in this way,
see https://bazel.build/designs/2016/10/11/distribution-artifact.html
Right now I am trying to compile Courguette in order to generate a diff patch. The source and some documentation can be found in the following link:
Courguette
I downloaded it but I don't find the way of compiling it.
I have Googled and found some references even to this forum, such as:
StackOverflow link
And other references:
Chromium Linux build instructions
After reading things about ninja and gpy I decided to install them, but there are not references to a .ninja file. I find a courguette.gyp file but using gpy I neither get it compiled...
I am following the guide they provide:
Installed the depot_tools and added to the path.
gclient runhooks
gclient sync
And when I execute gn gen out/Default it fails with the following message:
gn.py: Could not find gn executable at: ~/binary_diff_tools/courgette-master/courgette/buildtools/linux64/gn
Then I used git to pull from the repository the buildtools, but I get the same result.
Any help will be appreciated.
Best regards,
Iván
Ok... It seems that is needed to install all Chromium dependencies before being able to compile courgette so... it must be done as follows:
Install depot_tools git clone and you will have them installed.
After including the depot_tools in your patch (.profile or .bashsrc for example as indicate the depot_tools installation tutorial) execute fetch chromium ouside the depot_tools folder ( it always failed me when I executed it inside the same folder).
I want to build it in Debian so I have to do "gclient runhooks" instead of the .sh script for Ubuntu.
ninja -C out/Default courgette
You will find the executable in the folder out/Default, so ./courgette and it will indicate you the right parameters in order to generate your patch properly.
Best regards,
Iván.
I'm trying to install Armadillo (and thus also BLAS and LAPACK) on a linux server for which I do not have root permission. I have jumped through a few of the first hurdles, but I am getting an error:
/usr/bin/ld: cannot find -llapack
This question addresses the same problem, with the solution being to install
liblapack-dev
liblapack3
libopenblas-base
libopenblas-dev
The problem is the only way I've found to do this online is by doing something like sudo-aptget install or yum install. Both are not allowed on the server I use. I can download binaries and install them locally - that is it.
My question is: how can I install these packages without the above permissions and get on my way to using Armadillo?
I'm using Centos linux, if it helps.
Since you do not have root permissions, the best way is to download and build LAPACK and BLAS. You can download the source code from netlib.
Description of installation instructions can be found here and here.
The basic steps are:
Unzip and tar the file.
Copy and edit the file LAPACK/make.inc.example to LAPACK/make.inc.
Edit the file LAPACK/Makefile
type make.
Similarly you can download and build BLAS.
I was trying to build Boost library from source for MinGW. The Boost website says no guarantee but there seem to be people done it successfully. However I couldn't find much instructions on the web.
I updated the title to better reflect my problem right now.
======================== Original post ==============================
I downloaded Boost 1.53.0, unzipped it and cd to the folder in MinGW shell. It failed at the very first step I tried:
$ ./bootstrap.sh mingw
Building Boost.Build engine with toolset gcc...
Failed to build Boost.Build build engine
Consult 'bootstrap.log' for more details
Inside the bootstrap.log the errors are:
builtins.c:33:23: fatal error: sys/wait.h: No such file or directory
compilation terminated.
execunix.c:17:26: fatal error: sys/resource.h: No such file or directory
compilation terminated.
fileunix.c:98:17: fatal error: ar.h: No such file or directory
compilation terminated.
Please help! Thanks!
================== End of original and beginning of update =====================
Update: I found this detailed instruction on line:
http://vijay.axham.com/blog/478/building-boost-binaries-on-mingw
I followed it along and now got stuck in the final build step that is supposed to take a long time but I got an error instead:
$ b2 --build-dir=$BOOST_BUILD_DIR --prefix=$BOOST_INSTALL_DIR toolset=gcc variant=release link=static threading=multi runtime-link=static install 2>&1 | tee $BOOST_BUILD_DIR/build.log
error: Unable to find file or target named
error: 'boost/tr1/tr1/bcc32'
error: referred from project at
error: '.'
but the directory is there (it should be since it's just extracted from the zip file)
$ ls boost/tr1/tr1/bcc32/
array.h random.h regex.h tuple.h type_tra.h unordere.h
Getting closer but still need help! Thanks!
OK I got it working. The trick was to download the tar.bz2 file, not the zip file from sourceforge (specifically http://sourceforge.net/projects/boost/files/boost/1.53.0/). Even though both the zip and the tar.bz2 files are listed under the same file folder for the same version of Boost, the contents are different. There are some missing folders in the zip file, and the line ending conventions of the compressed files are different. At any rate after I downloaded and extracted the tar.bz2 file. I followed the instructions given here:
http://vijay.tech/articles/wiki/Programming/Cpp/Boost/BuildingBoostOnMinGw
and successfully built the Boost library from source using MinGW shell (mintty to be exact). There were some failures but probably not important: has_icu_test, has_iconv, has_icu_obj, has_icu64_obj, .masm. At the end it says
...failed updating 2 targets...
...skipped 3 targets...
...updated 10623 targets...
Hope this will help others in the future.
Do not use bash. Build it using cmd.exe.
bootstrap.bat gcc
Compiler executable should be on PATH.
Do not use bash. Build it using cmd.exe as described below:
Install MinGw on your system. I recommend using the same bit system as your processor is. Then set the path in System Environment to the bin folder which contains g++, ... (Compiler executable should be on PATH.) files. Now you are ready to go.
For obvious reasons, start a fresh terminal (cmd.exe), don't use the already open terminals that doesn't know your new setting.
Download boost, the latest stable release, unzip it and in the command window follow the path too the main directory of the extracted boost.
Run this command: bootstrap mingw
Run this command afterwards, which will install in the folder you select as your destination.
b2 install --prefix=c:\boost\custominstallationfolder\gcc toolset=gcc
Like said above, but more specifically for me compiling boost-1.54 with gcc-mingw-4.8.1
Using a Windows shell (cmd.exe) navigate to root of boost directory directory then
bootstrap.bat gcc
b2.exe toolset=gcc