cfx xpi command deleting compressed files in addon /lib directory? - firefox

I have a weird problem when trying to package a Firefox add-on built using version 1.9 of the SDK. The extensions directory structure is something like this:
├── data
│   ├── file1.js
│   ├── file2.js
│   ├── jquery.min.js
│   └── uri.js
├── lib
│   ├── file3.js
│   ├── main.js
│   ├── services
│   │   ├── file4.js
│   │   ├── file5.js
│   │   └── file6.js
│   └── uri.js
├── package.json
└── package.json.backup
As part of the build process, I am running the data and 'lib` directories through uglify.js. This appears to work fine. Basically I copy the codebase to a different location, run it through uglify and I get the same directory structure except the JS files are compressed.
Next, I run cfx xpi --pkgdir=path/to/ugly/codebase to package the code into an xpi.
If I then move the produced .xpi to a new directory, unzip it with unzip and inspect the contents, most of my lib directory has been deleted. Files in the data directory are fine.
tree resources/addon_name
resources/addon_name
├── data
│   ├── file1.js
│   ├── file2.js
│   ├── jquery.min.js
│   └── uri.js
└── lib
└── main.js
If I don't uglify the JS files then everything seems to work fine and when I unzip the xpi I will have a full lib directory as I would expect.
Note that this is not a problem with the uglifying process (that was the first thing I checked). When I copy the codebase and uglify it, I can stop the process at that point and list the lib directory. It will contain all the uglified JS files I would expect. It's only after packaging and subsequent unzipping that they are gone.
I have tried reproducing this issue with a brand new extension but I get a slightly different problem. Basically, files in the lib directory are deleted on packaging regardless of whether they are compressed or not. Basically my steps are:
mkdir test_extension
cd test_extension && cfx init
touch lib/uri.js // this is
cd .. && cfx xpi pkgdir=test_extension // Have to run this part twice to get ID
mkdir unpack && mv test_textension.xpi unpack
cd unpack
unzip test_extension.xpi
ls resources/test_extension/lib
=> main.js // the uri.js file is missing

If lib/uri.js is not required from any js file of your add-on, it will
be removed from the final XPI. So if you have require('./uri.js') in
your main.js, the file should be there after the packaging.
My guessing is that uglifying the libraries makes
impossible for the current cfx tool generates the proper manifest with
all dependencies. See Manifest Generation.
Note original post on mozilla-labs-jetpack mailing list, copied the answer here to be useful to someone else that doesn't know the ML.

Related

Where do I put common development methods for rspec and rake in my gem file tree?

Here is current gem structure:
├── Gemfile
├── LICENSE.txt
├── README.md
├── Rakefile
├── mygem.gemspec
├── images
│   ├── 1b1d4bde376084011d027bba1c047a4b.jpg
│   ├── 1d468d064d2e26b5b5de9a0241ef2d4b.jpg
│   ├── 309666c7b45ecbf8f13e85a0bd6b0a4c.jpg
│   ├── 3f9f3db06db20d1d9f8188cd753f6ef4.jpg
│   ├── 679634ff89a31279a39f03e278bc9a01.jpg
│   ├── 6d97739b4a08f965dc9239dd24382e96.jpg
│   ├── 71662d4d4029a3b41d47d5baf681ab9a.jpg
│   ├── 92d90b8977f813af803c78107e7f698e.jpg
│   ├── ad8a37f872956666c3077a3e9e737984.jpg
│   └── df0a3b93e9412536ee8a11255f974141.jpg
├── lib
│   └── mygem.rb
└── spec
   └── _spec.rb
The ./images folder does not really exist until you run rspec the first time -- tests do download these binary files from a remote storage, use them and cache them here for later.
There are also rake tasks that use these files and instead of leaving the developer a memo "run this rake task only after you ran rspec" I would like to put the download_and_keep() procedure in some common place callable by both rspec and rake. The question is what is the recommended place to put such method?
There is a folder ./spec/support by rspec design but I'm thinking about switching to minitest so I need something more universal.
P.S.: can't think up more tags for this question.
There isn't really a standard for such a directory. The important part is to pick a name that, at a glance, doesn't sound like it's supposed to be included in the shipped gem.
For some inspiration, rails/rails has a tools, and rubygems/rubygems has util. A top-level support seems like it'd make a fair choice too.

Maven throwing could not find class error after setting up home path

I am on mac and Installed maven using the brew install maven command. The maven was installed in /usr/local/Cellar/maven/3.3.9 path. Then i edited my ~/.bash_profile file and put the following entries
export M2_HOME="/usr/local/Cellar/maven/3.3.9"
export PATH="$PATH:$M2_HOME/bin"
now when i try mvn -version maven throws the error that
Error: Could not find or load main class org.codehaus.plexus.classworlds.launcher.Launcher
Did i configure it wrong ?
EDIT: Maven folder
$ tree -L 2 $M2_HOME
/usr/local/Cellar/maven/3.3.9
├── INSTALL_RECEIPT.json
├── LICENSE
├── NOTICE
├── README.txt
├── bin
│   ├── mvn
│   ├── mvn.cmd
│   ├── mvnDebug
│   ├── mvnDebug.cmd
│   └── mvnyjp
└── libexec
├── bin
├── boot
├── conf
└── lib
I have also seen as an option:
Create a file ~/.MacOSX/environment.plist with:
{
"M2_HOME" = "/usr/local/Cellar/maven/3.3.9/libexec";
"M2" = "/usr/local/Cellar/maven/3.3.9/libexec/bin";
}
and restart / log back in. This would make tools like IntelliJ also pick it up.
However in my case, I found some other setup script had installed an export for M2_HOME in my .profile which broke the maven install.

Cannot require ruby files from a directory other then the one with main files (LoadError)

My problem is the following:
I can't require a ruby file from any directory other than the one with all the main files, i.e. requires like this: ./file work fine, but requires like this: dir_name/file fail.
Here's my project structure:
├── bin
│   ├── console
│   └── setup
├── CODE_OF_CONDUCT.md
├── Gemfile
├── lib
│   ├── riverbattle
│   │   ├── base.rb <------- main file that starts everything
│   │   ├── colorful.rb
│   │   ├── computer.rb
│   │   ├── constants.rb
│   │   ├── exit_error.rb
│   │   ├── field.rb
│   │   ├── game.rb
│   │   ├── human.rb
│   │   ├── invalid_move_error.rb
│   │   ├── move.rb
│   │   ├── player.rb
│   │   ├── version.rb
│   │   └── victory_error.rb
│   └── riverbattle.rb <------ the file from where I get the error
├── LICENSE
├── LICENSE.txt
├── Rakefile
├── README.md
├── spec
│   ├── spec_helper.rb
│   └── riverbattle_spec.rb
├── riverbattle-0.1.2.gem
└── riverbattle.gemspec
So, when I launch the app like this: ruby base.rb when I am in the lib/riverbattle/ directory, everything launches fine, but when I try to do ruby riverbattle.rb from the directory lib/, I get the following error:
➜ lib git:(master) ✗ ruby riverbattle.rb
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in require': cannot load such file -- ./field (LoadError) from
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:inrequire' from
/home/denis/.rvm/gems/ruby-2.2.1/gems/riverbattle-0.1.2/lib/riverbattle/game.rb:1:in
<top (required)>' from
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:inrequire' from
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in require' from
/home/denis/.rvm/gems/ruby-2.2.1/gems/riverbattle-0.1.2/lib/riverbattle/base.rb:1:in
' from
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:in require' from
/home/denis/.rvm/rubies/ruby-2.2.1/lib/ruby/site_ruby/2.2.0/rubygems/core_ext/kernel_require.rb:54:inrequire' from riverbattle.rb:2:in `'
I just don't get it why it cannot require one single file.
By the way, here's the content of the file (riverbattle.rb) from which I get the error:
require "riverbattle/version"
module Riverbattle
require "riverbattle/base"
end
and nothing else.
I was told that require actually requires an absolute path to the file and I need to do something with my $LOAD_PATH or something, but this information doesn't help at all to solve the issue. Why is it okay to require files from the same directory, but not okay, if I require them from another one?
1) Could anybody help me with solving that problem?
2) Is it the problem with only my computer and I can safely publish it as a gem (i.e. it may work properly on another computer), or it is the wrong code that doesn't work?
require searches in your $LOAD_PATH. require should generally be used if you want to use a gem that is installed on your system. For your purposes you want to use require_relative:
require_relative 'riverbattle/version'
Note that at first glance
require './riverbattle/version'
might also seem to work. However, . here points to the current directory from where you run the process, not the directory where the file resides, obviously this is not what you want.
If lib/base.rb is your main file, then perhaps it should be the "executable" file? If that is the case, should it be in bin?
$LOAD_PATH can be added to by using -I option with the ruby executable.
ruby -Ipath_to_project_root/lib lib/main.rb
would then let any file that requires files in your project look in the lib folder.
It can also be used like the answer in your other question with the $: or $LOAD_PATH.
Look at how other gems are set up, and you will find that your base.rb file can live in the bin folder, just like your setup and console files are. If it is the main script that starts it all, it could be named riverbattle as is common with gems to have an executable of the same or similar name.
require is not used to load gems, it is used to load files and without any explicit path given, will look in the load path. It doesn't matter if it is a gem or not.
require_relative is used to load files that are always local relative to the file that is using the require_relative method.
Of course, the documentation is your friend.

How to execute code from a Ruby GEM I'm building?

I'm new in ruby and I'm trying to create a GEM. The "problem" is that I found really hard to execute the code I'm building, without having to repeat a lot of steps.
This is basically my gem structure
├── Gemfile
├── Gemfile.lock
├── Guardfile
├── LICENSE.txt
├── README.md
├── Rakefile
├── bin
│   └── hermes
├── hermes.gemspec
├── lib
│   ├── hermes
│   │   ├── code.rb
│   │   ├── issue.rb
│   │   └── version.rb
│   └── hermes.rb
├── requirements.md
└── spec
├── hermes_spec.rb
└── spec_helper.rb
So I'm running bundle exec irb, then require 'hermes'and then Hermes.init, but if I change any of the files code, I have to exit irb and repeat the steps. Is there a better way of doing it? Which is the "correct" way of doing it (without using rspec) ?
Thanks!
You can use pry and then load "path/to/file.rb" every time you change your code. You can also give it a simple web interface with sinatra and use shotgun to have your code reloaded on the fly.
But, the correct way is using rspec. Nothing better than to start fresh and effective employing test driven development. Trust me, you'll be ahead of 90% of the market by just starting with tdd from scratch.

This application failed to start because it could not find or load the Qt platform plugin "cocoa"

I think I did everything I could in the last 20 hours, but nothing seems to work. My app is running and working -- just as it should -- the only problem I have is that I cannot create a .app bundle from it. I tried both Py2App and cx_Freeze but non of them is working. Because of the multi-platform support I would stick with the latter -- if possible.
The setup.py looks like this:
import sys
from cx_Freeze import setup, Executable
base = None
if sys.platform == 'win32':
base = 'Win32GUI'
OPTIONS = {'build_exe': {'includes': ['sip',
'PyQt5',
'PyQt5.QtCore',
'PyQt5.QtGui',
'PyQt5.QtWidgets',
'PyQt5.QtMultimediaWidgets',
'PyQt5.QtMultimedia',
'PyQt5.QtNetwork']}}
EXECUTABLES = [Executable('main.py', base=base)]
NAME = 'coublet'
VERSION = '0.5.70'
setup(name = NAME,
version = VERSION,
options = OPTIONS,
executables = EXECUTABLES)
The error message I have is this:
objc[28404]: Class NotificationReceiver is implemented in both
/Users/.../build/coublet-0.5.70.app/Contents/MacOS/QtWidgets and
/usr/local/Cellar/qt5/5.3.1/lib/QtWidgets.framework/Versions/5/QtWidgets. One of
the two will be used. Which one is undefined.
QObject::moveToThread: Current thread (0x7fc4b96e98b0) is not the object's thread
(0x7fc4b95dbc80).
Cannot move to target thread (0x7fc4b96e98b0)
On Mac OS X, you might be loading two sets of Qt binaries into the same process.
Check that all plugins are compiled against the right Qt binaries. Export
DYLD_PRINT_LIBRARIES=1 and check that only one set of binaries are being loaded.
This application failed to start because it could not find or load the Qt
platform plugin "cocoa".
Available platform plugins are: cocoa, minimal, offscreen.
Reinstalling the application may fix this problem.
Abort trap: 6
My system Info:
Mac OS X : 10.9.4
Python : 3.4.1
cx_Freeze : 0.9
PyQt5: : 5.3.1
- - -
Packages installed via: Homebrew and PIP
.app structure:
build/coublet-0.5.70.app
└── Contents
├── Frameworks
├── Info.plist
├── MacOS
│   ├── PyQt5.QtCore.so
│   ├── PyQt5.QtGui.so
│   ├── PyQt5.QtMultimedia.so
│   ├── PyQt5.QtMultimediaWidgets.so
│   ├── PyQt5.QtNetwork.so
│   ├── PyQt5.QtWidgets.so
│   ├── Python
│   ├── QtCore
│   ├── QtCore.so
│   ├── QtGui
│   ├── QtGui.so
│   ├── QtMultimedia
│   ├── QtMultimedia.so
│   ├── QtMultimediaWidgets
│   ├── QtMultimediaWidgets.so
│   ├── QtNetwork
│   ├── QtNetwork.so
│   ├── QtOpenGL
│   ├── QtWidgets
│   ├── QtWidgets.so
│   ├── _bisect.so
│   ├── _bz2.so
│   ├── _codecs_cn.so
│   ├── _codecs_hk.so
│   ├── _codecs_iso2022.so
│   ├── _codecs_jp.so
│   ├── _codecs_kr.so
│   ├── _codecs_tw.so
│   ├── _datetime.so
│   ├── _hashlib.so
│   ├── _heapq.so
│   ├── _json.so
│   ├── _lzma.so
│   ├── _md5.so
│   ├── _multibytecodec.so
│   ├── _opcode.so
│   ├── _pickle.so
│   ├── _posixsubprocess.so
│   ├── _random.so
│   ├── _scproxy.so
│   ├── _sha1.so
│   ├── _sha256.so
│   ├── _sha512.so
│   ├── _socket.so
│   ├── _ssl.so
│   ├── _struct.so
│   ├── array.so
│   ├── binascii.so
│   ├── grp.so
│   ├── imageformats
│   │   ├── libqdds.dylib
│   │   ├── libqgif.dylib
│   │   ├── libqicns.dylib
│   │   ├── libqico.dylib
│   │   ├── libqjp2.dylib
│   │   ├── libqjpeg.dylib
│   │   ├── libqmng.dylib
│   │   ├── libqsvg.dylib
│   │   ├── libqtga.dylib
│   │   ├── libqtiff.dylib
│   │   ├── libqwbmp.dylib
│   │   └── libqwebp.dylib
│   ├── libcrypto.1.0.0.dylib
│   ├── liblzma.5.dylib
│   ├── library.zip
│   ├── libreadline.6.dylib
│   ├── libssl.1.0.0.dylib
│   ├── main
│   ├── math.so
│   ├── platforms
│   │   ├── libqcocoa.dylib
│   │   ├── libqminimal.dylib
│   │   └── libqoffscreen.dylib
│   ├── pyexpat.so
│   ├── readline.so
│   ├── select.so
│   ├── sip.so
│   ├── termios.so
│   ├── time.so
│   ├── unicodedata.so
│   └── zlib.so
└── Resources
The question is I think pretty obvious: what am I doing wrong? (or what am I not doing?)
When building an app with cx_Freeze on MacOSX all dependent libraries (.so files on MacOSX) are packaged into the application bundle. It is this that makes the application portable to other systems, without requiring a second install of Qt.
When launching the Application, the libraries should therefore be loaded from within the bundle. However, in your case the system libraries are still being loaded:
/Users/.../build/coublet-0.5.70.app/Contents/MacOS/QtWidgets
/usr/local/Cellar/qt5/5.3.1/lib/QtWidgets.framework/Versions/5/QtWidgets
The result One of the two will be used. Which one is undefined. means that either of these could be loaded. If it picks the correct one, great! If it doesn't you've got two separate sets of libraries loading simultaneously, and that fails shortly afterwards. As an aside, you may find that if you try your Application on another system it will work fine! Sometimes.
For an overview of the problem I suggest taking a look at the following bug #33.
Before you begin
Make sure you have an up-to-date version of cx_Freeze installed. I would suggest trying to clone the repository and installing from there.
Secondly, ensure that the Qt plugins are being correctly built into your application. Cx_Freeze previously looked for the qt-menu.nib file to determine if it was building a Qt application. This is no longer available in Qt5, but you can pass it on the command-line when building your app. Set it to whatever you want, it really doesn't matter:
python setup.py bdist_mac --qt-menu-nib=/usr/local/Cellar/qt5/5.3.1/plugins/platforms/
This may be enough to fix your problem. But if not, you have two options:
Option 1
Each library file contains the paths to it's dependencies. If you're receiving this error, it means some of those paths are either a) still pointing to the original file, or b) not specific enough (and being found on your PATH or DYLD_LIBRARY_PATH). However, you can re-write paths using install_name_tool from the command line (as described here:
install_name_tool -change /usr/local/Cellar/qt5/5.3.1/lib/QtWidgets.framework/Versions/5/QtWidgets #executable_path/QtWidgets build/MyApp.app/Contents/MacOS/qt_plugins/platforms/libqcocoa.dylib
install_name_tool -change /usr/local/Cellar/qt5/5.3.1/lib/QtCore.framework/Versions/5/QtCore #executable_path/QtCore build/MyApp.app/Contents/MacOS/qt_plugins/platforms/libqcocoa.dylib
install_name_tool -change /usr/local/Cellar/qt5/5.3.1/lib/QtPrintSupport.framework/Versions/5/QtPrintSupport #executable_path/QtPrintSupport build/MyApp.app/Contents/MacOS/qt_plugins/platforms/libqcocoa.dylib
install_name_tool -change /usr/local/Cellar/qt5/5.3.1/lib/QtGui.framework/Versions/5/QtGui #executable_path/QtGui build/MyApp.app/Contents/MacOS/qt_plugins/platforms/libqcocoa.dylib
This rewrites the paths in the libraries to point into your application folder, using #executable_path as the base. You will need to do this for all the paths that you find loading incorrectly. I'd suggest wrapping it up into an script, to run automatically after the build.
If you want to look at which libraries a file is referencing you can use otool. For example, in a successfully built application of mine:
otool -L libqcocoa.dylib
libqcocoa.dylib:
#executable_path/../Resources/qt_plugins/platforms/libqcocoa.dylib (compatibility version 0.0.0, current version 0.0.0)
/System/Library/Frameworks/Cocoa.framework/Versions/A/Cocoa (compatibility version 1.0.0, current version 20.0.0)
...
There was an updated workaround in the issue tracker that suggests that just importing the correct module in your app will get it to function, however it seems unlikely that you've made an application without QtWidgets.
Option 2
If the above doesn't work, there is another approach outlined here. This is a bit of a sledgehammer approach in that it simply prevents loading of plugins at all.
Add a qt.conf file next to the executable (in the .app bundle that contains:
[Paths]
Plugins = '.'
Either set the environment variable QT_PLUGIN_PATH="" (you can do this within your application before importing PyQt. Or call QtGui.QApplication.setLibraryPaths([]) before creating your application object.
The result is no plugins, so your application will not have access to the MacOSX Cocoa style and UI (e.g. file, colour dialogs).
Use this command :
pip install opencv-python-headless
It resolved the same issue in my case.
This is in addition to #mfitzp's answer.
This QT Plugins Document came in handy.
To look for default locations that your QT app is trying to search you can use following command:
$sudo dtruss MacOS/ncher
getattrlist("/ncher.app\0", 0x7FFF954B51A4, 0x7FFF5C8FDD20) = 0 0
getattrlist("/ncher.app/Contents\0", 0x7FFF954B51A4, 0x7FFF5C8FDD20) = 0 0
getattrlist("/ncher.app/Contents/MacOS\0", 0x7FFF954B51A4, 0x7FFF5C8FDD20) = 0 0
stat64("/ncher.app/Contents/MacOS\0", 0x7FFF5C8FDED8, 0x7FFF5C8FDD20) = 0 0
stat64("/ncher.app/Contents/MacOS/platforms/.\0", 0x7FFF5C8FDF58, 0x7FFF5C8FDD20) = -1 Err#2
open("/dev/tty\0", 0x1000000, 0x1FF) = 5 0
fcntl(0x5, 0x2, 0x1) = 0 0
close(0x5) = 0 0
write_nocancel(0x2, "This application failed to start because it could not find or load the Qt platform plugin \"cocoa\".\n\nReinstalling the application may fix this problem.\n\0", 0x97) = 151 0
sigprocmask(0x3, 0x7FFF5C8FE6B4, 0x0) = 0x0 0
__pthread_sigmask(0x3, 0x7FFF5C8FE6C0, 0x0) = 0 0
__pthread_kill(0x603, 0x6, 0x0) = 0 0
kevent64(0x4, 0x0, 0x0) = -1 Err#4
You can see this QT app trying to look into MacOS/platforms, once i copied my libqcocoa.dylib plugin (its path were modified by install_name_tool command as per #mfitzp's answer) there, my application worked like charm.
BTW it is advisable by MAC codesigning guide to have this directory structure correctly setup, DONT put Frameworks, plugins in ncher.app/Contents/MacOS directory
I had this problem in the context of just starting a Qt Widgets app from Qt Creator.
What helped me was to set the variable QT_PLUGIN_PATH to the value <Qt-dir>/plugins (where <Qt-dir> is the absolute path to the folder in the Qt directory containing, among others, bin, doc, include, lib and of course plugins).
This probably did not fix the root cause of the problem but it was a quick fix that's working fine so far.

Resources