Setup Google cloud shell environment for Cloud ML - shell

I am trying to setup a google cloud shell environment for my google cloud project that uses Cloud ML.
I am following the instructions on their setup page itself:
Getting Setup | Google Cloud Machine Learning
I executed this command in my cloud shell:
curl
https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/setup_cloud_shell.sh
| bash
The last part of ouput trace for this was :
echo 'Success! Your environment has the required tools and dependencies.' Success! Your environment has the required tools and
dependencies.
This command:
export PATH=${HOME}/.local/bin:${PATH}
Produces no ouput.
And finally the command to check the environment:
curl
https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/check_environment.py
| python
Produces this output:
vineetkaushik053#cloudshell:~$ curl https://raw.githubusercontent.com/GoogleCloudPlatform/cloudml-samples/master/tools/check_environment.py | python
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 3814 100 3814 0 0 17274 0 --:--:-- --:--:-- --:--:-- 17257
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
You are using pip version 8.1.1, however version 9.0.1 is available.
You should consider upgrading via the 'pip install --upgrade pip' command.
Your active configuration is: [cloudshell-11180]
ERROR: Unable to list Cloud ML models: {
"error": {
"code": 400,
"message": "Field: parent Error: Name should be in the form of 'projects/your-project-id'",
"status": "INVALID_ARGUMENT",
"details": [
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"field": "parent",
"description": "Name should be in the form of 'projects/your-project-id'"
}
]
}
]
}
}
When it should be producing:
"Success! Your environment is configured correctly." when the script finishes successfully.
Please tell me how to resolve this so that I may move to the next part of the setup.

Resolved this.
Need to set the project for the environment. You can use this command:
gcloud config set project your-project-ID

Related

Ansible on SLES: zypper plugin not able to install PostgreSQL 14

I am trying my hand on Ansible after having a very nice training in it. Currently, my task is to create a playbook that sets up a PostgreSQL cluster (with Patroni and etcd).
However, while installing PostgreSQL should be a pretty easy task, doing it using the zypper plugin throws an error. First, the part of the playbook that should install PostgreSQL:
- name: Installation PostgreSQL 14 Latest ohne Recommendations
become: true
zypper:
disable_recommends: true
name:
postgresql14-server
postgresql14-contrib
postgresql14-devel
update_cache: true
when: ansible_host in pgservers
The error message given is this:
fatal: [goeccdb22l]: FAILED! => {"changed": false, "cmd": ["/usr/bin/zypper", "--quiet", "--non-interactive", "--xmlout", "install", "--type", "package", "--auto-agree-with-licenses", "--no-recommends", "--", "+postgresql14-server postgresql14-contrib postgresql14-devel"], "msg": "No provider of '+postgresql14-server postgresql14-contrib postgresql14-devel' found.", "rc": 104, "stderr": "", "stderr_lines": [], "stdout": "<?xml version='1.0'?>\n<stream>\n<message type=\"error\">No provider of &apos;+postgresql14-server postgresql14-contrib postgresql14-devel&apos; found.</message>\n</stream>\n", "stdout_lines": ["<?xml version='1.0'?>", "<stream>", "<message type=\"error\">No provider of &apos;+postgresql14-server postgresql14-contrib postgresql14-devel&apos; found.</message>", "</stream>"]}
Let's extract the error message:
"msg": "No provider of '+postgresql14-server postgresql14-contrib postgresql14-devel' found."
I tried to replicate the problem using the shell on the target server. However, running the command seems to be able to install the packages:
ansible#goeccdb22l:~> sudo /usr/bin/zypper install --type package --auto-agree-with-licenses --no-recommends -- +postgresql14-server postgresql14-contrib postgresql14-devel
Loading repository data...
Reading installed packages...
Resolving package dependencies...
The following 12 NEW packages are going to be installed:
libecpg6 libopenssl-1_1-devel libpq5 postgresql postgresql14 postgresql14-contrib postgresql14-devel postgresql14-server postgresql-contrib postgresql-devel postgresql-server zlib-devel
The following package needs additional customer contract to get support:
postgresql14
12 new packages to install.
Overall download size: 8.0 MiB. Already cached: 0 B. After the operation, additional 35.4 MiB will be used.
Continue? [y/n/v/...? shows all options] (y):
I've removed only the --quiet and --non-interactive options from the command, but kept all other given options.
The best idea I have is that the user/privilege escalation workings could be different from me logging in as the Ansible user to the target and just using sudo before the command.
Edit 1: I have developed an idea what the problem could be. As I mentioned above, when I tested the command, I removed two options: --quiet and --non-interactive. Testing the command with those two options gives the message:
The flag --quiet is not known.
However, using man zypper, I can clearly see that --quiet is a documented option:
-q, --quiet
Suppress normal output. Brief (esp. result notification) messages and error messages will still be printed, though. If used together with conflicting --verbose option, the --verbose option takes preference.
Now, my idea is that Ansible calls the command it documents in the return XML, but that because somehow --quiet is not understood it interprets that as nothing providing the requested package list. So that would leave two questions:
Why is --quiet not understood, yet documented? Is that a problem of SLES vs. OpenSuse?
How to work around that?
As the Ansible zypper module has no option to suppress the --quiet option I don't see any chance of working around it with parameters. The last option would be to split the zypper task into smaller shell tasks which I would like to avoid if possible.
So, with the help of a knowledgeable sysadmin I was able to diagnose the problem.
The list of packages documented above was not in fact a list. As I missed the dashes in front of the packages, Ansible accepted the packages with the newlines and everything as one package name and tried to install it.
The solution is to change the packages into a list of packages by prefixing them with dashes/minus signs.
The problem with --quiet was that it is a positional argument, and I used the wrong position when testing it.

Unable to get heroku started on cloud9 ide

I am running the command - curl https://cli-assets.heroku.com/install-ubuntu.sh | sh
Which throws the following error -
% Total % Received % Xferd Average Speed Time Time Time Current
Dload Upload Total Spent Left Speed
100 1232 100 1232 0 0 5133 0 --:--:-- --:--:-- --:--:-- 5112
This script requires superuser access to install apt packages.
You will be prompted for your password by sudo.
+ dpkg -s apt-transport-https
+ echo ''
sh: line 4: /etc/apt/sources.list.d/heroku.list: No such file or directory
I also ran sudo snap install --classic heroku which returned sudo: snap: command not found.
Then I ran sudo apt install snapd which returned the following -
apt: invalid flag: install
Usage: apt <apt and javac options> <source files>
where apt options include:
-classpath <path> Specify where to find user class files and annotation processor factories
-cp <path> Specify where to find user class files and annotation processor factories
-d <path> Specify where to place processor and javac generated class files
-s <path> Specify where to place processor generated source files
-source <release> Provide source compatibility with specified release
-version Version information
-help Print a synopsis of standard options; use javac -help for more options
-X Print a synopsis of nonstandard options
-J<flag> Pass <flag> directly to the runtime system
-A[key[=value]] Options to pass to annotation processors
-nocompile Do not compile source files to class files
-print Print out textual representation of specified types
-factorypath <path> Specify where to find annotation processor factories
-factory <class> Name of AnnotationProcessorFactory to use; bypasses default discovery process
See javac -help for information on javac options.
warning: The apt tool and its associated API are planned to be
removed in the next major JDK release. These features have been
superseded by javac and the standardized annotation processing API,
javax.annotation.processing and javax.lang.model. Users are
recommended to migrate to the annotation processing features of
javac; see the javac man page for more information.
Finally, I ran wget 0- wget https://toolbelt.heroku.com/install-ubuntu.sh | sh which returned
--2020-09-10 18:15:53-- http://0-/
Resolving 0- (0-)... failed: Name or service not known.
wget: unable to resolve host address ‘0-’
--2020-09-10 18:15:54-- http://wget/
Resolving wget (wget)... failed: Name or service not known.
wget: unable to resolve host address ‘wget’
--2020-09-10 18:15:54-- https://toolbelt.heroku.com/install-ubuntu.sh
Resolving toolbelt.heroku.com (toolbelt.heroku.com)... 54.164.74.108, 107.23.162.152, 34.194.108.77, ...
Connecting to toolbelt.heroku.com (toolbelt.heroku.com)|54.164.74.108|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 719 [text/plain]
Saving to: ‘install-ubuntu.sh’
install-ubuntu.sh 100%[=============================================================>] 719 --.-KB/s in 0s
2020-09-10 18:15:54 (105 MB/s) - ‘install-ubuntu.sh’ saved [719/719]
FINISHED --2020-09-10 18:15:54--
Total wall clock time: 0.4s
Downloaded: 1 files, 719 in 0s (105 MB/s)
So then, I ran bash install-ubuntu.sh which returned -
This script requires superuser access to install apt packages.
You will be prompted for your password by sudo.
sh: line 3: /etc/apt/sources.list.d/heroku.list: No such file or directory
sh: line 6: apt-key: command not found
--2020-09-10 18:16:51-- https://toolbelt.heroku.com/apt/release.key
Resolving toolbelt.heroku.com (toolbelt.heroku.com)... 54.145.36.98, 54.164.74.108, 107.23.162.152, ...
Connecting to toolbelt.heroku.com (toolbelt.heroku.com)|54.145.36.98|:443... connected.
HTTP request sent, awaiting response... 200 OK
Length: 1737 (1.7K) [application/octet-stream]
Saving to: ‘STDOUT’
- 0%[ ] 0 --.-KB/s in 0s
Cannot write to ‘-’ (Success).
sh: line 9: apt-get: command not found
sh: line 12: apt-get: command not found
I am taking an online course on Upskill and am on video #125. Please advise how to progress.
Thanks for your time and help.
I have a gist I use specifically for setting up rails on Cloud9 (though I haven't updated it for Rails 6 yet). Recommend you read through it twice before you try to do it. You may need to go out of order in some cases, as exact approach depends on if you are cloning an existing repo or building a new app.
https://gist.github.com/MyklClason/791d6b14606bc56e72eba2995aab8e76
You probably don't need snap.
Also useful bash aliases for Cloud9:
https://gist.github.com/MyklClason/d71a39ace28b9ec9f0ad
As for your actual issue. Heroku toolbelt is obsolete, use this instead:
wget -qO- https://cli-assets.heroku.com/install-ubuntu.sh | sh
Also it's often best to just look online and check how to install the heroku cli (or anything really) for your OS using the offical documentation. Though that may not work for older setup. However, heroku is something where you basically have to use the newest version otherwise you are going to run into problems.
If you didn't figure this out...
nvm i v8
Followed by...
npm install -g heroku

i'm unable to get http-get response in OpenDayLight karaf console

as i'm new to this OpenDayLight, the following is my requirement
after installation of southbound plugin OVSDB in karaf console, i'm unable to get the http response from network-topology
installation is completely done by providing the following command in karaf console
feature:install odl-ovsdb-southbound-impl-ui
but after this installation,if i run the follow command
GET http://<controller-ip>:8181/restconf/config/network-topology:network-topology/topology/ovsdb:1/
or
GET http://<controller-ip>:8181/restconf/operational/network-topology:network-topology/topology/ovsdb:1
the result body should be:
{
"topology": [
{
"topology-id": "ovsdb:1"
}
]
}
but the result of my console is showing only
INFO
please give me any idea to get resultant output.
what ODL version are you using?
what tool are you using to issue the REST call?
I just tried with a very recent distribution (but I've known this to work
for a long time on much older versions as well):
opendaylight-user#root>feature:install odl-ovsdb-southbound-impl-ui
opendaylight-user#root>
then:
$ curl -u "admin:admin" http://127.0.0.1:8181/restconf/config/network-topology:network-topology/topology/ovsdb:1
which gave me:
{"topology":[{"topology-id":"ovsdb:1"}]}

Hyperledge Composer publish definitions failed

This is not a duplicates, at least I have the right version of Composer, I have also read the 3 stackflow composer questions.
I tried to publish definition, the Basic-Sample-Network package came from the git respository, the Basic-sample-network.bna archive file, here's the command :
composer network deploy -p hlfv1.json -a basic-sample-network.bna -i admin -s adminpw
The error (seems to be a generic error message):
Identifier: basic-sample-network#0.1.3
Description: The Hello World of Hyperledger Composer samples
events.js:160
throw er; // Unhandled 'error' event
^
Error: event message must be properly signed by an identity from the same organization as the peer: [failed deserializing event creator: [Expected MSP ID Org1MSP, received ]]
at ClientDuplexStream._emitStatusIfDone (/usr/local/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:189:19)
at ClientDuplexStream._receiveStatus (/usr/local/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:169:8)
at /usr/local/lib/node_modules/composer-cli/node_modules/grpc/src/node/src/client.js:634:14
Here're my environments :
composer -v :
composer-cli v0.10.0
composer-admin v0.10.0
composer-client v0.10.0
composer-common v0.10.0
composer-runtime-hlf v0.10.0
composer-runtime-hlfv1 v0.10.0
npm c 3.10.10
hlfv1.json profile :
{
"type": "hlfv1",
"orderers": [
{ "url" : "grpc://localhost:7050" }
],
"ca": { "url": "http://localhost:7054",
"name": "ca.org1.example.com"
},
"peers": [
{
"requestURL": "grpc://localhost:7051",
"eventURL": "grpc://localhost:7053"
}
],
"keyValStore": "${HOME}/.composer-credentials",
"channel": "composerchannel",
"mspID": "Org1MSP",
"timeout": "300"
}
Note I used the fabric-tool script the start the fabric and to generate the profile (hlfv1.json).
docker ps:
570ae25a586e hyperledger/fabric-peer:x86_64-1.0.0 "peer node start -..." 30 minutes ago Up 30 minutes 0.0.0.0:7051->7051/tcp, 0.0.0.0:7053->7053/tcp peer0.org1.example.com
513431e5d217 hyperledger/fabric-ca:x86_64-1.0.0 "sh -c 'fabric-ca-..." 31 minutes ago Up 31 minutes 0.0.0.0:7054->7054/tcp ca.org1.example.com
2e7bf444481d hyperledger/fabric-couchdb:x86_64-1.0.0 "tini -- /docker-e..." 31 minutes ago Up 31 minutes 4369/tcp, 9100/tcp, 0.0.0.0:5984->5984/tcp couchdb
5d5ba67cc602 hyperledger/fabric-orderer:x86_64-1.0.0 "orderer" 31 minutes ago Up 30 minutes 0.0.0.0:7050->7050/tcp orderer.example.com
If you follow the developer tutorial you will see that the id PeerAdmin must be used to deploy the network.
https://hyperledger.github.io/composer/tutorials/developer-guide.html
Problem fixed.
The confusion was providing the connectionProfileName, the -p option, based on reading the document, I thought I have to provide a file with connection information (even thought the provided profile information contain all necessary information), I didn't know it was actually picking up the profile file from ~/.composer-connection-profiles folder based on the name provided in the -p option.

Logstash install error: can't get unique system GID (no more available GIDs)

I am trying to install logstash with yum on a red hat vm, I already have the logstash.repo file setup according to the guide and i ran
yum install logstash
but I get the following error after it downloads everything
...
logstash-2.3.2-1.noarch.rpm | 72 MB 00:52
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
groupadd: Can't get unique system GID (no more available GIDs)
useradd: group 'logstash' does not exist
error: %pre(logstash-1:2.3.2-1.noarch) scriptlet failed, exit status 6
Error in PREIN scriptlet in rpm package 1:logstash-2.3.2-1.noarch
error: install: %pre scriptlet failed (2), skipping logstash-1:2.3.2-1
Verifying : 1:logstash-2.3.2-1.noarch 1/1
Failed:
logstash.noarch 1:2.3.2-1
Complete!
I can't find much information about this. Any suggestions?
groupadd determines gids for the creation of regular groups from the /etc/login.defs file.
In my centos 6 box. /etc/login.defs contains following two lines:
#
# Min/max values for automatic gid selection in groupadd
#
GID_MIN 500
GID_MAX 60000
For system accounts add these two lines to your /etc/login.defs
# System accounts
SYS_GID_MIN 100
SYS_GID_MAX 499
I updated the SYS_GID_MAX Value and it worked for me.

Resources