Terraform GCP Instance Metadata Startup Script Issue - bash

I've been working with Terraform, v0.15.4, for a few weeks now, and have gotten to grips with most of the lingo. I'm currently trying to create a cluster of RHEL 7 instances dynamically on GCP, and have, for the most part, got it to run okay.
I'm at the point of deploying an instance with certain metadata passed along to it for use in scripts built into the machine image for configuration thereafter. This metadata is typically just passed via an echo into a text file, which the scripts then pickup as required.
It's... very simple. Echo "STUFF" > file... Alas, I am hitting the same issue OVER AND OVER and it's driving me INSANE. I've Google'd around for ages, but all I can find is examples of the exact thing that I'm doing, the only difference is that theirs works, mine doesn't... So hopefully I can get some help here.
My 'makes it half-way' code is as follows:
resource "google_compute_instance" "GP_Master_Node" {
...
metadata_startup_script = <<-EOF
echo "hello
you" > /test.txt
echo "help
me" > /test2.txt
EOF
Now the instance with this does create successfully, although when I look onto the instance, I get one file called ' /test.txt? ' (or if I 'ls' the file, it shows as ' /test.txt^M ') and no second file.. I can run any command instead of echo, and whilst the first finishes, the second+ does not. Why?? What on earth is causing that??
The following code I found also, but it doesn't work for me at all, with the error, 'Blocks of type "metadata" are not expected here.'
resource "google_compute_instance" "GP_Master_Node" {
...
metadata {
startup-script = "echo test > /test.txt"
}

Okaaaaay! Simple answer for a, in hindsight, silly question (sort of). The file was somehow formmated in DOS, meaning the script required a line continuation character to run correctly (specifically \ at the end of each individual command). Code as follows:
resource "google_compute_instance" "GP_Master_Node" {
...
metadata_startup_script = <<-EOF
echo "hello
you" > /test.txt \
echo "help
me" > /test2.txt \
echo "example1" > /test3.txt \
echo "and so on..." > /final.txt
EOF
However, what also fixed my issue was just 'refreshing' the file (probably a word for this, I don't know). I created a brand new file using touch, 'more'd the original file contents to screen, and then copy pasted them into the new one. On save, it is no longer DOS, as expected, and then when I run terraform the code runs as expected without requiring the line continuation characters at the end of commands.
Thank you to commentors for the help :)

Related

How to deal with shell commands that never stops

Here is the case;
There is this app called "termux" on android which allows me to use a terminal on android, and one of the addons are androids API's like sensors, tts engines, etc.
I wanted to make a script in ruby using this app, specifically this api, but there is a catch:
The script:
require('json')
JSON.parse(%x'termux-sensor -s "BMI160 Gyro" -n 1')
-s = Name or partially the name of the sensor
-n = Count of times the command will run
returns me:
{
"BMI160 Gyroscope" => {
"values" => [
-0.03...,
0.00...,
1.54...
]
}
}
I didn't copied and pasted the values, but that's not the point, the point is that this command takes almost a full second the load, but there is a way to "make it faster"
If I use the argument "-d" and not use "-n", I can specify the time in milliseconds to delay between data being sent in STDOUT, it also takes a full second to load, but when it loads, the delay works like charm
And since I didn't specify a 'n' number of times, it never stops, and there is the problem
How can I retrieve the data continuously in ruby??
I thought about using another thread so it won't stop my program, but how can I tell ruby to return the last X lines of the STDOUT from a command that hasn't and will not ever stop since "%x'command'" in ruby waits for a return?
If I understood you need to connect to stdout from a long running process.
see if this works for your scenario using IO.popen:
# by running this program
# and open another terminal
# and start writing some data into data.txt
# you will see it appearing in this program output
# $ date >> data.txt
io_obj = IO.popen('tail -f ./data.txt')
while !io_obj.eof?
puts io_obj.readline
end
I found out a built in module that saved me called PTY and the spawn#method plus thread management helped me to keep a variable updated with the command values each time the command outputted new bytes

Bash: What does "file < FILE > FILE.extension &" means?

I was running AutoPhrase and after running it's auto_phrase.sh shell script, I am getting the following error:
After backtracing it's source, I found this line (line 32) which is causing the above mentioned error.
All I know is it is doing something like:
file < FILE > FILE.extension
However, I failed to understand what is happening in this line.
Can someone could explain it to me ? And how may I resolve this issue ?
The problem here seems to be that ./bin/tree-tagger does not exist. Note that ./bin is not the same as /bin.
That being said, the answer to your question of what file < FILE > FILE.extension does is unrelated, but these are redirects in bash. FILE is piped < into the $STDIN of file and the output is directed > to the file FILE.extension.
Locate ./bin/tree-tagger by installing it or redirecting its path appropriately to resolve the error.
As for the actual question in the title, the command means
./cmd/tree-tagger-english run this command ...
< ... with standard input from ...
$f ... an (incorrectly unquoted) variable which should contain a file name ...
> ... with standard output redirected to ...
$f.tagged ... a file named by the (still incorrectly unquoted) value of $f with the string .tagged appended at the end ...
& ... as a background job.
Apparently ./cmd/tree-tagger-english in turn attempts to execute a command which doesn't exist, many times.
Probably the instructions tell you to run the thing in a different directory than where you are actually running this; but this is obviously merely a speculation. Another fairly common scenario is that something failed during installation, but you failed to notice (and so some files which should exist in these locations do not actually exist ... disk full? Wrong permissions?)
The fact that the code contains quoting bugs suggests that it's also quite possible that the code is simply buggy.

Rake task selectivly ignoring new code

In a rake task I'm writing some puts statements show changes while others don't. For instance changing
puts model+" | "+id
into
puts model+" * "+id
doesn't change in the output of the script. However in some places changing
puts "Connecting to "+site
into
puts "Connecting to ----"+site
shows the changes that where made.
In the places where any changes to the line doesn't change the output, adding a new puts statement before or after don't show up when the task is run. Commenting out lines of code around the unchanging puts statements that do the actual work cause the script to not execute those lines, just as it should, but changing or adding puts statements there do not change the output of the script.
Removing all other tasks and emacs backup files from the lib/tasks folder doesn't help. I've been bitten before by having a backup copy of a task with the same namespace and task name running instead of the one I was working on.
This is being run with Ruby 2.4.3 on OpenBSD 6.3-stable on a fx-8350. I would post the whole script but the company I'm working for won't allow it.
How about
puts "#{model} +/*/whatever #{site}"
It shouldn't matter to what sounds like a filesystem update issue (reboot), but it's probably better form to put the variables in the string like that instead of + "" them.

bash: wrong behavior in for... loop together with a test statement

I am trying to test if certain files, called up in a list of textfiles, are in a certain directory. Every once in a while (and I am quite certain I use the same statements every time) I get an error, complaining that the echo command cannot be found.
The textfiles I have in my directory /audio/playlists/ are named according to their date on which they are supposed to be used: 20130715.txt for example for today:
me#computer:/some/dir# ls /audio/playlists/
20130715.txt 20130802.txt 20130820.txt 20130907.txt 20130925.txt
20130716.txt 20130803.txt 20130821.txt 20130908.txt 20130926.txt
(...)
me#computer:/some/dir# cat /audio/playlists/20130715.txt
#A Comment line goes here
00:00:00 141-751.mp3
00:03:35 141-704.mp3
00:06:42 140-417.mp3
00:10:46 139-808.mp3
00:15:13 136-126.mp3
00:20:26 071-007.mp3
(...)
23:42:22 136-088.mp3
23:46:15 128-466.mp3
23:50:15 129-592.mp3
23:54:29 129-397.mp3
So much for the facts. The following statement, which lets me test if all files called upon in all of the textfiles in the given directory are actually a file in the directory /audio/mp3/, produces an error:
me#computer:/some/dir# for i in $(cat /audio/playlists/*.txt|cut -c 10-16|sort|uniq); do [ -f "/audio/mp3s/$i.mp3" ] || echo $i; done
 echo: command not found
me#computer:/some/dir#
I would guess bash wants to complain about the "A Comment"-line (actually " line ") not being a file, but why would that cause echo not to be found? Again, mostly this works, but every so often I get this error. Any help is greatly appreciated.
That space before echo isn't U+0020, it's U+00A0. And indeed, the command " echo" doesn't exist.

startup script in freebsd is not running

I have been trying to run a shell script at boot time of freebsd. I have read all simmilar questions in stackoverflow and tried. But nothing is worked. This is the sample code that i tried is dummy.
#!/bin/sh
. /etc/rc.subr
name="dummy"
start_cmd="${name}_start"
stop_cmd=":"
dummy_start()
{
echo "Nothing started."
}
load_rc_config $name
run_rc_command "$1"
Saved with name of dummy.
Permissions are -r-xr-xr-x.
in rc.conf file made dummy_enable="YES".
The problem is, when i rebooted my system to test, dummy file is not there. So script is not executing. what else need to do run my dummy script.
SRC:http://www.freebsd.org/doc/en/articles/rc-scripting/article.html#rc-flags
You need to add rcvar="dummy_enable" to your script. At least for FreeBSD 9.1.
Call your script with parameter rcvar to get the enabled status:
# /etc/rc.d/dummy rcvar
# dummy
#
dummy_enable="YES"
# (default: "")
And finally start it with parameter start - this won't start the service/script unless dummy_enable is set in /etc/rc.conf (or /etc/rc.conf.local, or /etc/defaults/rc.conf)
# /etc/rc.d/dummy start
Nothing started.
One possible explanation is that rcorder(8) says:
Within each file, a block containing a series of "REQUIRE", "PROVIDE",
"BEFORE" and "KEYWORD" lines must appear.
Though elsewhere I recall that if a file doesn't have "REQUIRE", "PROVIDE" or "BEFORE", then it will be arbitrarily placed in the dependency ordering. And, it could be that the arbitrary placement differs between the first run up to $early_late_divider and in the second run of those after $early_late_divider.
OTOH, is this a stock FreeBSD, or some variant? I recall reading that FreeNAS saves its configuration somewhere else and recreates its system files on every boot. And, quite possibly that /etc is actually on a ramdisk.
Also, /usr/local/etc/rc.d doesn't come into existence until the first port installing an rc file is installed.

Resources