Unable to deploy HDP via Ambari 2.2.1.1 - hadoop

I can't deploy HDP stack(hdfs, yarn, spark .etc) via Ambari 2.2.1.1. I got this error:
Traceback (most recent call last):
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 35, in
BeforeAnyHook().execute()
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/script.py", line 219, in execute
method(env)
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py", line 29, in hook
setup_users()
File "/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/shared_initialization.py", line 43, in setup_users
fetch_nonlocal_groups = params.fetch_nonlocal_groups
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 149, in init
self.arguments[key] = arg.validate(value)
File "/usr/lib/python2.6/site-packages/resource_management/core/base.py", line 56, in validate
if not value in (True, False):
File "/usr/lib/python2.6/site-packages/resource_management/libraries/script/config_dictionary.py", line 81, in getattr
raise Fail("Configuration parameter '" + self.name + "' was not found in configurations dictionary!")
resource_management.core.exceptions.Fail: Configuration parameter 'fetch_nonlocal_groups' was not found in configurations dictionary!
Error: Error: Unable to run the custom hook script ['/usr/bin/python', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY/scripts/hook.py', 'ANY', '/var/lib/ambari-agent/data/command-387.json', '/var/lib/ambari-agent/cache/stacks/HDP/2.0.6/hooks/before-ANY', '/var/lib/ambari-agent/data/structured-out-387.json', 'INFO', '/var/lib/ambari-agent/tmp']
Did someone see this error before? Thanks in advance!

Related

unable to create executable using Py2exe, throwing error as mentioned below

Hi I'm trying to make an executable using py2exe, I checked my script but cannot find the actual issue why below error is keep getting pop-up during building of exe
Building 'dist\py.exe'.
Traceback (most recent call last):
File "C:\Users\User\PycharmProjects\Test\pyexe\setup.py", line 15, in <module>
freeze(
File "C:\Python39\lib\site-packages\py2exe\__init__.py", line 176, in freeze
builder.build()
File "C:\Python39\lib\site-packages\py2exe\runtime.py", line 254, in build
self.build_archive(exe_path)
File "C:\Python39\lib\site-packages\py2exe\runtime.py", line 479, in build_archive
base = dist_path.rsplit('\\', 1)[0]
AttributeError: 'NoneType' object has no attribute 'rsplit'
Below is my setup.py script
from py2exe import freeze
from py_version import VERSION, DESCRIPTION # This is my own script
freeze(
console=[{"script": "py.py"}],
options={
"py2exe": {
"dist_dir": "dist",
"bundle_files": 1, # bundle as much as possible
"compressed": True,
}},
version_info={"version":VERSION,
"description":DESCRIPTION,
"company_name":"Priyanshu Kumar",
"copyright":"Opensource, credit goes to David Manthey",
"product_name":"Stand-Alone Python3 Interpreter",
"product_version":VERSION},
zipfile=None)
UPDATE:
I was searching on google for help and found that it should be <<<zipfile="None">>> instead of <<<zipfile=None>>>.
I changed it in my code but now got new error
Traceback (most recent call last):
File "C:\Users\User\PycharmProjects\Test\pyexe\setup.py", line 29, in <module>
options={
File "C:\Python39\lib\site-packages\py2exe\__init__.py", line 176, in freeze
builder.build()
File "C:\Python39\lib\site-packages\py2exe\runtime.py", line 273, in build
self.build_archive(libpath, delete_existing_resources=True)
File "C:\Python39\lib\site-packages\py2exe\runtime.py", line 397, in build_archive
resource.add(type=os.path.basename(pydll), name=1, value=pydll_bytes)
File "C:\Python39\lib\site-packages\py2exe\resources.py", line 49, in add
raise WindowsError(details) from None
OSError: [WinError 87] The parameter is incorrect.
Press any key to continue . . .

Error while restoring Couchbase database using cbrestore command in mac

I am getting the below error while restoring the couchbase database to my local mac from a server.
Traceback (most recent call last):
File "/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/python/cbrestore", line
12, in <module>
pump_transfer.exit_handler(pump_transfer.Restore().main(sys.argv))
File "/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/python/pump_transfer.py", line 80, in main
rv = pumpStation.run()
File "/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/python/pump.py", line 136, in run
rv = self.transfer_bucket_msgs(source_bucket, source_map, sink_map)
File "/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/python/pump.py", line 233, in transfer_bucket_msgs
source_map)
File "/Applications/Couchbase Server.app/Contents/Resources/couchbase-core/lib/python/pump_bfd.py", line 546, in total_msgs
rv, db, ver = connect_db(x, opts, CBB_VERSION)
ValueError: need more than 2 values to unpack
Any help is appreciated.
Thanks,
Emraan

Celery not starting in OS X - dbm.error: db type is dbm.gnu, but the module is not available

I'm trying to run celery worker in OS X (Mavericks). I activated virtual environment (python 3.4) and tried to start Celery with this argument:
celery worker --app=scheduling -linfo
Where scheduling is my celery app.
But I ended up with this error: dbm.error: db type is dbm.gnu, but the module is not available
Complete stacktrace:
Traceback (most recent call last):
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/kombu/utils/__init__.py", line 320, in __get__
return obj.__dict__[self.__name__]
KeyError: 'db'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/Users/other/PhoenixEnv/bin/celery", line 9, in <module>
load_entry_point('celery==3.1.9', 'console_scripts', 'celery')()
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/__main__.py", line 30, in main
main()
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/celery.py", line 80, in main
cmd.execute_from_commandline(argv)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/celery.py", line 768, in execute_from_commandline
super(CeleryCommand, self).execute_from_commandline(argv)))
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/base.py", line 308, in execute_from_commandline
return self.handle_argv(self.prog_name, argv[1:])
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/celery.py", line 760, in handle_argv
return self.execute(command, argv)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/celery.py", line 692, in execute
).run_from_argv(self.prog_name, argv[1:], command=argv[0])
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/worker.py", line 175, in run_from_argv
return self(*args, **options)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/base.py", line 271, in __call__
ret = self.run(*args, **kwargs)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bin/worker.py", line 209, in run
).start()
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/__init__.py", line 100, in __init__
self.setup_instance(**self.prepare_args(**kwargs))
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/__init__.py", line 141, in setup_instance
self.blueprint.apply(self, **kwargs)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bootsteps.py", line 221, in apply
step.include(parent)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bootsteps.py", line 347, in include
return self._should_include(parent)[0]
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/bootsteps.py", line 343, in _should_include
return True, self.create(parent)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/components.py", line 220, in create
w._persistence = w.state.Persistent(w.state, w.state_db, w.app.clock)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/state.py", line 161, in __init__
self.merge()
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/state.py", line 169, in merge
self._merge_with(self.db)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/kombu/utils/__init__.py", line 322, in __get__
value = obj.__dict__[self.__name__] = self.__get(obj)
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/state.py", line 238, in db
return self.open()
File "/Users/other/PhoenixEnv/lib/python3.4/site-packages/celery/worker/state.py", line 165, in open
self.filename, protocol=self.protocol, writeback=True,
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 239, in open
return DbfilenameShelf(filename, flag, protocol, writeback)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/shelve.py", line 223, in __init__
Shelf.__init__(self, dbm.open(filename, flag), protocol, writeback)
File "/Library/Frameworks/Python.framework/Versions/3.4/lib/python3.4/dbm/__init__.py", line 91, in open
"available".format(result))
dbm.error: db type is dbm.gnu, but the module is not available
Please help.
I switched to python3.5 and got the same error. On Ubuntu I could fix it with
aptitude install python3.5-gdbm
I got the same error. On Macbook I could fix it with
brew install gdb

Google Apis Client Generator Python Not Running

I'm trying to execute Google APIs Client Generator to build new ADMIN-SDK for Java Language however I'm stuck in the generator, I've tried step by step using python2.6 and python2.7 both had similar outputs.
$ python $(/bin/pwd)/googleapis/codegen/generate_library.py --api_name=plus --api_version=v1 --language=java --output_dir=/tmp/gen --language=java
Traceback (most recent call last): File
"/Users/alejacquet/Development/git/google-apis-client-generator/src/googleapis/codegen/generate_library.py",
line 245, in
app.run() File "/Library/Python/2.7/site-packages/google_apputils-0.3.0-py2.7.egg/google/apputils/app.py",
line 216, in run
return _actual_start() File "/Library/Python/2.7/site-packages/google_apputils-0.3.0-py2.7.egg/google/apputils/app.py",
line 244, in _actual_start
really_start() File "/Library/Python/2.7/site-packages/google_apputils-0.3.0-py2.7.egg/google/apputils/app.py",
line 203, in really_start
sys.exit(main(argv)) File "/Users/alejacquet/Development/git/google-apis-client-generator/src/googleapis/codegen/generate_library.py",
line 152, in main
language_variant=FLAGS.language_variant) File "/Users/alejacquet/Development/git/google-apis-client-generator/src/googleapis/codegen/generate_library.py",
line 179, in Generate
language_variations = Targets().VariationsForLanguage(language) File
"/Users/alejacquet/Development/git/google-apis-client-generator/src/googleapis/codegen/targets.py",
line 68, in init
files.GetFileContents(self.targets_path)) File "/Users/alejacquet/Development/git/google-apis-client-generator/src/googleapis/codegen/json_with_comments.py",
line 54, in Loads
return json.loads(stripped, **kw) File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/init.py",
line 326, in loads
File "/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py",
line 366, in decode File
"/System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/json/decoder.py",
line 382, in raw_decode ValueError: Expecting property name: line 9
column 9 (char 198)
Any ideas?
You may have to specify --language_variant=XXX, where XXX is one of the directory names under .../templates/java

File permission issue with apache+mod_wsgi with pyramid&pyramid_beaker

I am trying to set up a pyramid app. I am using wsgi and apache2.
I keep getting Internal server error and the contents of the apache log is
mod_wsgi (pid=11200): Exception occurred processing WSGI script '/home/ubuntu/modwsgi/env/pyramid.wsgi'.
Traceback (most recent call last):
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/pyramid-1.3-py2.6.egg/pyramid/router.py", line 191, in __call__
request._process_response_callbacks(response)
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/pyramid-1.3-py2.6.egg/pyramid/request.py", line 243, in _process_response_callbacks
callback(self, response)
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/pyramid_beaker-0.6.1-py2.6.egg/pyramid_beaker/__init__.py", line 26, in session_callback
self.persist()
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/Beaker-1.6.3-py2.6.egg/beaker/session.py", line 706, in persist
self._session().save()
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/Beaker-1.6.3-py2.6.egg/beaker/session.py", line 400, in save
**self.namespace_args)
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/Beaker-1.6.3-py2.6.egg/beaker/container.py", line 622, in __init__
util.verify_directory(self.file_dir)
File "/home/ubuntu/modwsgi/env/lib/python2.6/site-packages/Beaker-1.6.3-py2.6.egg/beaker/util.py", line 85, in verify_directory
os.makedirs(dir)
File "/usr/lib/python2.6/os.py", line 150, in makedirs
makedirs(head, mode)
File "/usr/lib/python2.6/os.py", line 157, in makedirs
mkdir(name, mode)
OSError: [Errno 13] Permission denied: 'beaker_dir'
I can use simple pserve to serve the page and run wget http://localhost:6543/user/form on the terminal. It works well with form getting downloaded. But over the browser I get the 500 error.
I am using EC2 to host the app.
In the app:main stanza I am using the following code:
session.type = file
session.data_dir = beaker_dir
session.key = mvc
session.encrypt_key = mysecretencryptionkey
session.validate_key = mysecretvalidationkey
session.cookie_on_exception = true
Thanks a lot for reading
Supply an absolute path for:
session.data_dir = beaker_dir
The current working directory could be anything, usually '/', and so path will be wrong.

Resources