Remove git files/directories older than x days via GitHub Action - bash

We use a gh-pages branch in our repository to host a static website and frequently commit new information to this branch. Those files often get stale, as we push to a subdirectory per feature branch in the same repository.
The directory structure in my gh-pages branch is similar to the following:
.
|-- README.md
|-- JIRA-1234-feature
| `-- graph
|-- JIRA-4567-bugfix
| `-- graph
|-- JIRA-7890-branch-name
| `-- testing
I want to remove directories via a GitHub actions for which the last update was more than 5 days ago.
I naively tried to remove them via find /path/to/files* -mtime +5 -exec rm {} ;, but the operating system obviously uses the clone date as the last modified time.
I also found
git ls-tree -r --name-only HEAD | while read filename; do
echo "$(git log -1 --format="%ad" --date="short" -- $filename) $filename"
done
which prints the last git update and the file name like this:
2023-01-12 JIRA-1234-test/index.html
2023-01-12 JIRA-1234-test/static/test.css
I don't know how to trigger file removal commands from this list, though.
How would I have to modify the following action to remove the old files?
name: Prune GH Pages branch
on:
workflow_dispatch:
jobs:
upload:
runs-on: ubuntu-latest
timeout-minutes: 15
steps:
- name: Checkout
uses: actions/checkout#v3
with:
ref: gh-pages
- name: Cleanup
run: |
# TODO: REMOVE FILES HERE
git ls-tree -r --name-only HEAD | while read filename; do
echo "$(git log -1 --format="%ad" --date="short" -- $filename) $filename"
done
- name: Commit & Push
run: |
if [ $(git status --porcelain | wc -l) -eq "0" ]; then
echo "git repo is clean."
else
git add -A
git commit -m "branch cleanup"
git push
fi

Unfortunately, I didn't find a way to make a nice one-liner for the requirement. We need the following bash script. I have commented all the important steps.
#!/bin/bash
# Validate if $1 is a positive number days
[[ $1 != +([[:digit:]]) ]] &&
echo "$1: The script has to be run with positive number argument" && exit 1
# Get the X days ago timestamp
X_DAYS_AGO_TIMESTAMP=$(date -d "$1 days ago" +%s)
# Iterate over all files in the repository
for file in $(git ls-files); do
echo -n "."
# Get the timestamp of the last commit that modified this file
LAST_MODIFIED_TIMESTAMP=$(git log -1 --format="%at" -- "$file")
# If the file hasn't been modified within the last $1 days
if [ "$LAST_MODIFIED_TIMESTAMP" -lt "$X_DAYS_AGO_TIMESTAMP" ]; then
# Remove the file from the repository
echo -e "\nRemoving $file last modified at $(date -d "#$LAST_MODIFIED_TIMESTAMP")"
git rm --quiet "$file"
fi
done
# Commit the changes (if any)
if ! git diff --exit-code --quiet --staged; then
git commit -m "Remove files not modified within the last $1 days"
else
echo "No files removed"
fi
I can elaborate if something is not clear enough.

Related

Bash script to check git commits of last 2 days

I am trying to write a bash script which checks if there are any commits of a repository and returns a message. Here is the script so far.
MY_PATH="C:/test"
cd "$MY_PATH"
git clone https://github.com/test-repo/docker-react.git
cd "docker-react"
git checkout master
if [[ -n "`git log --pretty=format: --name-only --since="200 days ago" | sort | uniq`" ]]
then
echo -e No Commits last 2 days
else
echo -e Commits available
fi
When I execute this code it always show commits available even when I change the days.
Where am I going wrong?
That looks like a bash script, which would be interpreted by the Git for Windows bash shell.
It means it should be checked with ShellCheck, which would then recommend:
#!/bin/bash
MY_PATH="C:/test"
cd "$MY_PATH" || exit
git clone https://github.com/test-repo/docker-react.git
cd "docker-react" || exit
git checkout master
if [[ -n "$(git log --pretty=format: --name-only --since="200 days ago" | sort | uniq)" ]]
then
echo -e No Commits last 2 days
else
echo -e Commits available
fi
Not yet ideal: ideally, it would check if the repository is not already cloned. But it should work better.

Bash script to close multiple remote branches from text-file

Background
I need to close multiple Git branches (hundreds) that have been left open on a remote repository.
I wanted to sort them by last-commit and put them in a text-file so others could confirm they were no longer needed.
I found a method that allowed me to dump them in a way that I could distribute (and open in Excel to sort by date)
git for-each-ref --format='%(committerdate:short) %09 %(authorname) %09 %(refname)' | sort -k5n -k2M -k3n -k4n >> branches.txt
And then I need to read the updated text-file back in and delete the remote branches:
#!/bin/bash
#prefix of the branches if they are all remote
prefix="refs/remotes/origin/"
prefix1="refs/remotes/"
# path to branches, compiled with:
# git for-each-ref --format='%(committerdate:short) %09 %(authorname) %09 %(refname)' | sort -k5n -k2M -k3n -k4n >> branches.txt
# read through entire input
input="branches.txt"
while IFS= read -r line
do
# echo "$line"]
IFS=' ' read -r -a array <<< "$line"
# search the array for the prefix of the branch
for index in "${!array[#]}"
do
if [[ ${array[$index]} == *"refs/remotes/origin"* ]]; then
#echo -e " \e[34m ${array[$index]} \e[39m Last commit: \e[34m" ${array[0]} ${array[1]} ${array[2]} ${array[3]} ${array[4]}
# echo ${array[$index]#"$prefix"}
#remove the prefix if they are not pulled locally
branch=${array[$index]#"$prefix"}
echo $(git push origin --delete $branch)
fi
done
done < "$input"
Issue
However, keep getting an error that the refspec does not exist:
fatal: invalid refspec ':PracticeBranch?'
I don't have the ' or ? in the name of the branch variable - so this is probably my ignorance of using echo with `bash but I don't know where it is coming from.
A command like:
git push origin --delete PracticeBranch
When calling directly from the shell?

Git-Bash File Lookup Depending On File Type

I am trying to navigate through all existing all branches and lookup if files with a certain extension such as (.zip or .exe exist)
I tried to write a bash script to achieve this task.
for branch in $(git branch);
do
echo "I am in: $branch"
git ls-files *.exe
done
I would like to see the file path when it is detected.
You are not changing to the branch so you are always checking the last branch you checked out. Try this:
# In the repo's working directory
for branch in $(git branch -a|grep -v remotes|sed 's/\*//g'); do
echo "I am in branch: ${branch}"
git checkout ${branch}
find . -type f -name '*.md'
done
Following is how I solved my problem:
read -p "Extension to lookup [example: .zip]: " extensionType
for branch in $(git branch);
do
if [[ $branch == *"Release"* ]]; then
echo "----------------------------------"
echo ">>Navigating to: $branch"
echo ">>$branch..."
git checkout $branch
git ls-files "*$extensionType"
echo "----------------------------------"
fi
done
I hope this helps.

Can't Add '/path/to/file' to a parent directory scheduled for deletion

I've been attempting to use the following script to commit a wordpress plugin to the WP svn repo from my public git repository and keep getting error:
svn: E155013: Can't add '/path/to/file' to a parent directory scheduled for deletion
Whenever there is a new file, not previously known to svn, when adding files, resulting in either an aborted commit, or worse, a broken commit where it commits with missing files.
#! /bin/bash
# A modification of Dean Clatworthy's deploy script as found here: https://github.com/deanc/wordpress-plugin-git-svn
# The difference is that this script lives in the plugin's git repo & doesn't require an existing SVN repo.
# main config
PLUGINSLUG="my-plugin-slug"
CURRENTDIR=`pwd`
MAINFILE="my-plugin.php" # this should be the name of your main php file in the wordpress plugin
# git config
GITPATH="$CURRENTDIR/" # this file should be in the base of your git repository
# svn config
SVNPATH="/tmp/$PLUGINSLUG" # path to a temp SVN repo. No trailing slash required and don't add trunk.
SVNURL="http://plugins.svn.wordpress.org/my-plugin" # Remote SVN repo on wordpress.org, with no trailing slash
SVNUSER="wp_username" # your svn username
# Let's begin...
echo ".........................................."
echo
echo "Preparing to deploy wordpress plugin"
echo
echo ".........................................."
echo
# Check if subversion is installed before getting all worked up
#if [ $(dpkg-query -W -f='${Status}' subversion 2>/dev/null | grep -c "ok installed") != "1" ]
#then
# echo "You'll need to install subversion before proceeding. Exiting....";
# exit 1;
#fi
# Check version in readme.txt is the same as plugin file after translating both to unix line breaks to work around grep's failure to identify mac line breaks
NEWVERSION1=`grep "^Stable tag:" $GITPATH/readme.txt | awk -F' ' '{print $NF}'`
echo "readme.txt version: $NEWVERSION1"
NEWVERSION2=`grep "^ \* Version:" $GITPATH/$MAINFILE | awk -F' ' '{print $NF}'`
echo "$MAINFILE version: $NEWVERSION2"
if [ "$NEWVERSION1" != "$NEWVERSION2" ]; then echo "Version in readme.txt & $MAINFILE don't match. Exiting...."; exit 1; fi
echo "Versions match in readme.txt and $MAINFILE. Let's proceed..."
if git show-ref --tags --quiet --verify -- "refs/tags/$NEWVERSION1"
then
echo "Version $NEWVERSION1 already exists as git tag. Exiting....";
exit 1;
else
echo "Git version does not exist. Let's proceed..."
fi
cd $GITPATH
echo -e "Enter a commit message for this new version: \c"
read COMMITMSG
git commit -am "$COMMITMSG"
echo "Tagging new version in git"
git tag -a "$NEWVERSION1" -m "Tagging version $NEWVERSION1"
echo "Pushing latest commit to origin, with tags"
git push origin master
git push origin master --tags
echo
echo "Creating local copy of SVN repo ..."
svn co $SVNURL $SVNPATH
echo "Clearing svn repo so we can overwrite it"
svn rm $SVNPATH/trunk/*
echo "Exporting the HEAD of master from git to the trunk of SVN"
git checkout-index -a -f --prefix=$SVNPATH/trunk/
echo "Ignoring github specific files and deployment script"
svn propset svn:ignore "deploy.sh
README.md
bower_components
node_modules
.DS_Store
.gitmodules
advanced
assets
gulpfile.js
bower.json
package.json
.git
.gitignore" "$SVNPATH/trunk/"
echo "Changing directory to SVN and committing to trunk"
cd $SVNPATH/trunk/
# Add all new files that are not set to be ignored
echo "Doing the file adding"
# THIS IS WHERE ERROR OCCURS
svn status | grep -v "^.[ \t]*\..*" | grep "^?" | awk '{print $2}' | xargs svn add
echo "Committing"
svn commit --username=$SVNUSER -m "$COMMITMSG"
echo "Creating new SVN tag & committing it"
cd $SVNPATH
svn copy trunk/ tags/$NEWVERSION1/
cd $SVNPATH/tags/$NEWVERSION1
svn commit --username=$SVNUSER -m "Tagging version $NEWVERSION1"
echo "Removing temporary directory $SVNPATH"
rm -fr $SVNPATH/
echo "*** FIN ***"
Could anyone elaborate on what is causing this error and possibly recommend either a different or adapted approach?
Dean's version doesn't contain brainless svn rm $SVNPATH/trunk/*
rm $SVNPATH/trunk/* or checkout with --depth+commit with --set-depth

svn checkout to deploy via shell

I have the following problem. I need to organize automatic upload to deploy server from svn repository, but with some feautures.
There is how I wrote it:
# $1 - project; $2 - version (optional)
# rm -rf $projectDir
if [ "$2" == '' ]; then
svn export $trunk $projectDir --force >> $log
version=`svn info $trunk | grep Revision | awk '{print$2}'`
svn copy $trunk $tags/$version -m "created while uploading last version of $1"
echo "New stable version #$version of $1 is created
Uploading to last version is completed successfully"
else
version=$2
svn export $tags/$version/ $projectDir --force >> $log
echo "Revert to version #$version is completed successfully"
fi
echo $version > $projectDir/version
chown -R $1:$1 $projectDir
But svn export doesn't delete deleted via svn files, so I need to clean directory before export every time. It's not good.
Before this, I work with checkout for deploy like this:
svn co $trunk >> $log
cp -ruf trunk/* $projectDir
svn info $trunk | grep Revision > $projectDir/version
chown -R $project:$project $projectDir
echo "uploading finished"
This work very well and very very faster (it changes only changed files) than the export, but:
without automatic tag creating;
without opportunity for nice reverting.
In my last script co doesn't work, because it trying to checkout in one directory from different repository directories (trunk/some tag), which isn't real.
So, question:
Can I relocate project before checkout?
Can I find the diff with co version and existing version before export?
What can I do with diff result? (remove unneeded files after export?)
Thanks in advance.
Have you evaluated Capistrano? It can do a lot of what you're trying to achieve.
For the basis for the solution was taken following code:
It's simpler and fully solves the problem as for me.
if [ "$2" == '' ]; then
version=`svn info ${trunk} | grep Revision | awk '{print$2}'`
if [ `cat ${projectWWW}/version` == "${version}" ]; then
resultMessage="Project is up to date"
else
svn co ${trunk} ${projectRoot}/co >> ${log}
cp -ruf ${projectRoot}/co/ ${projectRoot}/releases/${version}
chown -R $1:$1 ${projectRoot}/releases/${version}
resultMessage="New stable version #$version of $1 is created
Uploading to last version is completed successfully"
fi
else
version=$2
resultMessage="Revert to version #$version is completed successfully"
fi
ln -s ${projectRoot}/releases/${version} ${projectWWW}
echo ${version} > ${projectWWW}/version
echo ${resultMessage} >> ${log}

Resources