I am trying to download Google Group data onto my Windows 10 computer with bash 10.0. When I try retrieving the file from my bin, it is a 0-byte file called t.0, and has no text when I access it in Atom 1.27.2.
Here is the script crawler.sh I used as a guide, it requires installation of bash-4, sort, wget (1.17.1), sed (4.2.2-7), and awk(1.3.3). I have the newest versions:
https://github.com/icy/google-group-crawler
This is the forum I am trying to download:
https://groups.google.com/forum/#!forum/django-developers
I made the script executable with chmod 755, and they are in my local bin.
I entered the following code from the README into the main method of crawler.sh (~line 254):
export _GROUP="django-developers"
export _RSS_NUM=50
export _HOOK_FILE=/path/to.sh
After that, I ran the testing command from the README in the shell:
./crawler.sh -sh # first run for testing
./crawler.sh -sh > wget.sh # save your script
bash wget.sh # downloading mbox files
I am new to using bash, so I am unsure of whether this is a simple error on my part, or an actual issue with the download.
Related
I'm working on a script to package up an installer into a self extracting archive. Initially I was working on batch script using
copy /b .\7zSD.sfx + .\config.txt + .\installer.7z .\installer.exe
and the self extracting archive functioned as expected.
As the developers are working in a cygwin environment I tried converting to a bash script for this particular part using
cat 7zSD.sfx config.txt installer.7z > installer.exe
The file is properly created and can be extracted using for example winrar, yet executing results in the attached windows error:
Are there differences that I should be aware of or does somebody know of the right way to approach this?
While I did unsuccessfully try yesterday to change the permissions via the windows file properties, I did not try changing the permissions with chmod.
Checking the permissions using ls -l installer.exe revealed them as only having read/write permissions for users and groups.
My solution was to do a
chmod 770 installer.exe
to get this working.
How can I load my output generated from my bash script into a gcs location
My bash command is like:
echo " hello world"
I want this output(hello world) to be shown in a location in gcs.
How to write a location command in Bash?
First, you should follow the install Cloud SDK instructions in order to use the cp command form gsutil tool on the machine you're running the script.
Cloud SDK requires Python; supported versions are Python 3 (preferred, 3.5 to 3.8) and Python 2 (2.7.9 or higher)
Run one of the following:
Linux 64-bit archive file from your command-line, run:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-362.0.0-linux-x86_64.tar.gz
For the 32-bit archive file, run:
curl -O https://dl.google.com/dl/cloudsdk/channels/rapid/downloads/google-cloud-sdk-362.0.0-linux-x86.tar.gz
Depending on your setup, you can choose other installation methods
Extract the contents of the file to any location on your file system (preferably your Home directory). If you would like to replace an existing installation, remove the existing google-cloud-sdk directory and extract the archive to the same location.
Run gcloud init to initialize the SDK:
./google-cloud-sdk/bin/gcloud init
After you have installed the Cloud SDK, you should create a bucket to uploadthe files that will contain the output generated by your script.
Use the gsutil mb command and a unique name to create a bucket:
gsutil mb -b on -l us-east1 gs://my-awesome-bucket/
This uses a bucket named "my-awesome-bucket". You must choose your own, globally-unique, bucket name.
Then you can redirect your output to a local file and upload to Google Cloud Storage like this:
#!/bin/bash
TIMESTAMP=$(date +'%s')
BUCKET="my-awesome-bucket"
echo "Hello world!" > "logfile.$TIMESTAMP.log"
gsutil cp logfile.$TIMESTAMP.log gs://$BUCKET/logfile.$TIMESTAMP.log
I am running a machine translation code in google colab for that I cloned the code into my drive and stored the drive location in a variable:
!PATH='/content/mydrive/My Drive/facebook/unsup'
The cloning was succesfully done in the desired location.
The problem arises when I try to run a shell file:
!."$PATH/NMT/get_data_enfr.sh"
it returns
/bin/bash: ./content/mydrive/My Drive/facebook/unsup/NMT/get_data_enfr.sh: No such file or directory
But if I run
!head "$PATH/NMT/get_data_enfr.sh"
It shows the file content.
# Copyright (c) 2018-present, Facebook, Inc.
# All rights reserved.
#
# This source code is licensed under the license found in the
# LICENSE file in the root directory of this source tree.
#
set -e
#
Can you try running the script as :
sh "$PATH/NMT/get_data_enfr.sh"
Before running it like above, please run the below command :
chmod +x "$PATH/NMT/get_data_enfr.sh"
It seems #dash-o is correct, When you are running the script using . in front of script, it actually looking for script from the current directory.
It's working for head command because there the complete path is being passed to head and it is finding the file at that path but not true while running it.
./path/to/script actually trying to run the script from the current directory and ignoring the absolute path. But when using sh and then $PATH, it actually running from the absolute path.
You have to escape the space with a single backslash.
Example: PATH=/omg/a\ space
Also its a good idea to put the output in " or ' with echo \"${PATH}\"
I have a device installed remotely that has Internet access. As i cannot SSH directly to it, the device downloads updates from a .txt file located in a server. This .txt file is interpreted by the device as a sequence of bash instructions.
Now, i'm planning an update that requires re-compiling a C program in the device after downloading and overwritting some files. The content of the .txt file for this update looks like:
#!/bin/bash
curl -o /path-to-file/newfile.c http://myserver/newfile.c
sleep 10 #wait so it can download
cd /path-to-file
make
sleep 10 #wait to make
sudo /path-to-file/my-program #test if it worked
I previously tested this method and it worked as expected, but never tested make. A couple of questions:
Should it work?
Is the sleep after make necessary?
Here is an example of how to retrieve a source code file into another directory, change to that directory, compile the source code with make and then execute the resulting binary:
mkdir -p path-to-file/
curl -o path-to-file/newfile.c http://www.csit.parkland.edu/~cgraff1/src/hello_world.c
cd path-to-file/
make newfile
./newfile
The cd is really not an integral part of the process, but it seems as if the question specifically pertains to performing the work in a directory other than the present working directory.
I've finished a little useful script written in Bash, hosted on github. It's tested and documented. Now, I struggle with how to make it installable, i.e. where should I put it and how.
It seems other such projects use make and configure but I couldn't really find any information on how to do this for bash scripts.
Also I'm unsure into which directory to put my script.
I know how to make it usable by myself but if a user downloads it, I want to provide the means for him to install it easily.
There is no standard for this because most of the time, a project isn't a single script file. Also single file scripts don't need a build step (the script is already in an executable form) and configuration usually comes from an external config file (so no need for a configure script, either).
But I suggest to add a comment near the top of the file which explains what it does and how to install it (i.e. chmod +x + copy to folder).
Alternatively, you could create an installer script which contains your original script plus a header which asks the user where she wants to install the real script and which does everything (mkdir, set permissions with sudo, etc) but it really feels like overkill in your case.
If you want to make it installable so the package manager can easily install and remove (!) it, you need to look at the documentation for rpm or Debian packaging. These are the two most used package managers but they can't install a script per-user (so it would probably end up in /usr/bin)
instruct them to create a file named after the script in their home directory, chmod ug+x the file so it has executable permissions than put the script inside the file, don't forget the #!/bin/bash up top of the vim. This example is a script to copy a file, archive the copied file than remove the copied file leaving only the original file and the archived file.
#!/bin/bash
#### The following will copy the desired file
cp -r /home/wes/Documents/Hum430 /home/wes/docs
#### Next archives the copied file
tar -zcvf Hum430.tar.gz /home/wes/docs
#### and lastly removes the un-archived copy leaving only the original and the archived file.
rm -r /home/wes/docs
### run the file with ./filename (whatever the file is named)