Using bash to automate dotfiles - bash

I want to create my own automated dotfiles folder. (I'll be using git to use version control on my dotfiles, but that's irrelevant for the question)
What i simply want is to symbolically link all the files and folders in ~/dotfiles to my home folder. Being not good at all with bash I can't do this. Please help me with this.
I would also appreciate the following features if possible.
Folders are only shallowly linked
My files could be in the dotfiles folder without the actual dot in the file-name (like ~/dotfiles/vimrc rather than ~/dotfiles/.vimrc)
It should be able to ignore some files, like my .git file which are stored in the same folder
Of course if you already know a service providing this, that is at least as good as providing some do-myself commands. Note how I specifically want it to be bash or something that most likely exists on all unix machines (so i guess commands using g++ are fine).

Give this a try:
ln -s ~/dotfiles/* ~
There shouldn't be any need for a loop. Of course, you can use find if you need something recursive.
Edit:
To make the destination files hidden:
for f in ~/dotfiles/*
do
ln -s "$f" "$HOME/.${f##*/}"
done

I am not sure if I'm getting the question right, but if you looking for symlinks of dir-content, try
for f in `ls -1 .dotfiles`
do
ln -s .dotfiles/$f ~/$f
done
maybe that already does the trick

For the sake of managing dotfiles, I really like Zach Holman's approach. I've made the same thing with my dotfiles which you can find it here :)
https://github.com/rhacker/dotFiles

Maybe you are looking for a dotfiles manager, I will recommend you to check DFM (dotfiles manager). DFM addresses the problem you have in a very clean way:
Here is my dotfiles selection using dfm: vicente's dotfiles
Github official repository DFM site

Atlassian has a tutorial on using a git work tree instead of symlinks. The approach uses:
a bare git repository in a side folder (such as $HOME/.cfg or $HOME/dotfiles), and
a config bash alias to execute git commands that manage the configuration files.
For instance, you can run config status to check which files have been modified, config checkout to get the files in the repository and config commit to update the repository. It requires only git and the bash.

I was also looking for some way to set up a new machine in a minimal number of steps, after spending some time I found that almost all the developers use git to store and share these files and symlinks to sync them.
Well, symlinks works, but it isn’t the best way to sync your local files to the git repository. There is a much better solution to this, written by people at Atlassian – https://www.atlassian.com/git/tutorials/dotfiles.
So, to git bare repository is the best and most elegant way to sync your files with your remote copy create a bash script to automate installation and set up.
Managing dotfiles
The trick to managing these dotfiles is by creating a bare git repository. If you are starting from scratch and have not tracked your dotfiles before, make a bare repository in the $HOME directory.
git init --bare $HOME/.dotfiles
Just to make it easier to use we’ll alias this to dotfiles which we will use instead of regular git to interact with our dotfiles repository.
alias dotfiles="/usr/bin/git --git-dir=$HOME/.dotfiles --work-tree=$HOME"
Now we are good to track our dotfiles using the dotfiles command, some of the examples are:
# to check the version history
dotfiles log
# to check the status of the tracked and untracked files
dotfiles status
# to add a file for tracking
dotfiles commit .vimrc -m ".vimrc added"
# push new files or changes to the github
dotfiles push origin main
Use Cases and Advantages of dotfiles
This method of managing and sharing has various advantages some of them are listed below.
Easy setup
Set up of a new machine can be a time consuming task, but with this method we can to use our personalized configs in under a minute. We just need to clone the repository and source the .bashrc or .zshrc file.
git clone --bare https://github.com/<username>/dotfiles.git $HOME/.dotfiles && source ~/.zshrc
Versioned dotfiles
Best for experimenting with new configurations and keep the change history (Basically all the pros of using git)
# to check the version history
dotfiles log
Share on Multiple devices
Share the same configs of multiple devices with minimal changes using branch, create a branch for your new machine, example:-
# Create configurations specific to your aws machines
dotfiles checkout -b aws
Profiles for dotfiles
Create configs based on your environment using branch, create a branch and configure according to you work env.
# Manage multiple profiles - check out to the work profile
dotfiles checkout work
I also use this way to sync and store my dotfiles, see my dotfiles repository and can read at Storing dotfiles with Git where I wrote about managing for multiple devices.

You could also use dotfile_manager. It allows you to set up a custom path for every symlink you want to create. So with this, you can easily ignore some files and add the . as a prefix to other files.
In your case you would setup the config file dotfile_manager.yaml to contain
symlinks:
- vimrc: ~/.vimrc
Full disclosure: I am the author of this package and use it every day.

Related

error: Sparse checkout leaves no entry on working directory

First, let me tell you that I have already checked all the similar threads and searched google to find what the problem may be, but no success. My problem is that I'm trying to use sparse checkout in git, but I get this error:
error: Sparse checkout leaves no entry on working directory
I have this 60GB repository, which I need to clone. I need only a part of it, so to save a disk space I wanted to use sparse checkout. This is what I do:
mkdir repoDir
git init repoDir
cd repoDir
git remote add origin <repo url>
git config core.sparsecheckout true
echo "some/dir/" >> .git/info/sparse-checkout
git pull --depth=1 origin master
Note I add the remote without -f flag, so nothing is fetched.
The result:
error: Sparse checkout leaves no entry on working directory
I tried several things:
Instead of initialising new repo and adding the remote manually, I did git clone --no-checkout and then set up the sparse checkout. This didn't help as my git status showed as if I deleted all the files in my repo. The git pull origin master command results in the same error.
Tried all possible combination of paths in step 6, with preceding slashes, slashes after the path, stars, spaces between path and > or >>. Btw I'm confused what is the correct format here, from the comments on SO I see mutually exclusive ways of formatting this.
Tried to make sure my .git/info/sparse-checkout is ASCII, and has proper line endings as found here. This is probably only problematic on Windows, but I just checked this anyway.
My git version: git version 1.9.3 (Apple Git-50)
OSX Yosemite 10.10.2 (14C109)
It turns out that "some/dir/" was wrong, since I didn't have the repository I didn't know it's real structure. I was able to browse it through web interface but I just discovered the folders don't correspond exactly to the real repository folder structure.
Lesson for the future: make sure you know the folder structure before creating sparse-checkout file.
The sequence of operations I wanted to follow is similar, namely the one suggested in this other post https://stackoverflow.com/a/13738951/5459638. I get the error message
error: Sparse checkout leaves no entry on working directory
when launching git pull <remote> <branch> with branch being master.
As #lawicko said, in the project webpage I can click my way to the subdirectory to be cloned and copy the URL of that page; ctrl+L ctrl+C makes doing this nice and quick. This URL has the form
https://gitlab.com/<username>/<project>/tree/master/<subdir>/<subdir>
The part that my sparse-checkout file accepts is the children of master
<subdir>/<subdir>/
with the trailing slash.
As an alternative to the same aim, there is another path in the form <project>/<subdir>/<subdir> to the right of the drop-down menu for switching branches at the top of the webpage.
In this case, I would have copied and pasted the children of <project> plus the trailing slash.
And this path is not as easy to copy and paste as the URL is.
Note that if you are using sparse-checkout on windows, you may need to add core.protectNTFS false per https://github.com/git-for-windows/git/issues/2777

Why does git sparse checkout leave behind directories?

Some background: My company's service model began as an appliance-based server model. We would send our client a server with Windows Server 2003/2008 on it, pre-loaded with a webserver and our software. We're moving all of the client-specific configuration to a Git repository, and using sparse-checkout to make each server only contain what's necessary for the client's software to function properly.
While setting up sparse-checkout, we've run into a huge inconsistency. We'll do
git clone git#github.com:ourclientconfigrepo.git .
git config core.sparsecheckout true
echo www.thisclient.com/ > .git/info/sparse-checkout
git read-tree -m -u HEAD
The expected result would be
ls
www.thisclient.com/
but we get
ls
www.thisclient.com/
www.randomclient1.com/
www.randomclient2.com/
www.randomclient3.com/
I've tried multiple times, in a new directory each time, and the issue has happened each time. My ju-git-su fails me here. We're using git version 1.8.1.msysgit.1.
Thanks for your help, let me know if I need to supply more information.
---EDIT 1---
Clarification: The repository is nothing but our client's configuration directories. Each client has a different directory in the repository, and we're trying to sparse-checkout on each client's individual server, so we're trying to exclude everything except for the client in question.
---EDIT 2---
Just an update, turns out it was something wonky with Windows folder permissions, and Git was trying to delete non-empty directories. Fixed it by deleting the empty folders. Thanks for any and all attempts at helping!
Your clone operation is not the correct way to create a sparse clone. Instead of:
git clone ...
You should use this sequence:
mkdir repo ; cd repo ; git init
git remote add -f origin <url>
git config core.sparsecheckout true
echo <dir1>/ >> .git/info/sparse-checkout
echo <dir2>/ >> .git/info/sparse-checkout
echo <dir3>/ >> .git/info/sparse-checkout
git pull origin master
A partial solution: According to git-read-tree(1), you can exclude specific files by putting a ! in front of the file name. This may be a miswrite, however (meaning that that's for directories as well, not files).
I'll continue investigating.
According to https://stackoverflow.com/a/11222033/1210278, you can manually mark directories with git update-index --skip-worktree www.randomclient1.com/.
https://stackoverflow.com/a/9575837/1210278 implies that exclusion is only for directories, and says that there are some bugs with it.

Git (windows 7) won't add directories, but adds files files typed explicitly

trying to find an answer to this, have seen it hinted at, but no solution.
Not entirely new to git, but haven't used it in a few months... this is my first time using it on windows. using git bash.
I have a cake PHP site that I want to add exluding any cache files my .gitignore file contains only:
app/tmp/*
Here is the most basic thing I have tried:
git init
git add .
git commit
I get a message on commit saying there are untracked files - all files and dirs at the root directory.
git status
gives me the same message.
However, if I type
git add .htaccess
it stages the .htaccess file.
Also, if I do:
git add app/.htaccess
It will add that file too.
I have also tried
git add app/*
no luck.
Thanks in advance.
The files aren't ignored, because otherwise you would be forced to do a git add --force afile.
However, you might want to check GIT_DIR and GIT_WORK_TREE environment variable, and see if they reference another repository, which could explain the discrepancy between the index and what you think is your working tree.

Tower (Git client) not cloning all directories to the local machine from the remote repository

I have an issue with a new remote repository that when cloned using Tower, doesn't clone all the directories.
When I created the remote repository, I did the following:
Created a .gitignore file containing the following line
files/cache/*
Then ran:
git init
git add .
git commit
I then cloned the remote repository to my local machine using a Mac OSX git client called Tower but noticed that many of the directories did not clone.
When I go back and look at my terminal session on the webserver, I can see the directories that weren't cloned listed after the initial commit - I see a whole bunch of lines that look like this:
create mode 10644 directory-name/path/to/file.php
create mode 10644 directory-name/path/file.php
create mode 10644 directory-name/path/to/file.php
create mode 10644 directory-name/path/file.php
So I'm guessing they were added but I'm also wondering if my .gitignore file is not setup right and is conflicting with Tower somehow?
I tried the .gitignore file a couple of ways, firstly like this:
files/cache/*
then like this:
files/cache/
After changing it to the second one, I ran git add -A (which didn't seem to add anything new) on the server and pulled the repo down using Tower again - but no luck.
Not sure if this is Git or Tower or both - not sure what I'm doing wrong, sorry.
Any help would be much appreciated.
Cheers
Ben
Don't forget that git won't add (and clone) empty directories.
(or directories with ignored content, making them empty for Git)
See:
"Does git ignore empty folders?"
"How do I add an empty directory to a git repository?"

How to solve a Mercurial case-folding collision?

I use Mercurial as source control with the main repository managed on KILN.
At one point in time I changed my iOS project name from WeatherTimeMachine to weathertimemachine.
This resulted in a case change of several files and folders:
WeatherTimeMachine.xcode
WeatherTimeMachine_Prefix.pch
WeatherTimeMachine-Info.plist
In the meantime I've added a tag to a revision in KILN... So I now have:
a head in KILN
a head on my local repo with case changes
When trying to merge I get the following error message: "Mercurial case-folding collision"
How can I fix this?
If you're on Mac OS X, you don't need to export your repository to Linux or another foreign case-sensitive file system as suggested by the Mercurial documentation. Just use Disk Utility to create a case-sensitive, journaled disk image slightly bigger than your repository, copy your repo there, then remove the conflicting files and commit.
I have found some information here: FixingCaseCollisions, but somehow this did not work for me. Here is how I managed to solve this issue:
Make a copy of your existing repository folder (for safety). For example:
cp -r WeatherTimeMachine WeatherTimeMachineCopy
Fool mercurial into thinking the problematic revision is the current tip:
hg debugsetparents <bad revision>
hg debugrebuildstate
Remove the files which are causing the problem (-f is required to force the removal). Here is an example:
hg rm -A -f WeatherTimeMachine-Info.plist
Once all problematic files have been removed, commit the changes
hg ci -m "fixed collision-folding issue" -u michael
Then restore mercurial to the proper revision
hg debugsetparents tip
hg debugrebuildstate
After this the merge is possible and the problem is gone.
And now I can happily resume working with MacHg to manage my Mercurial repository and push my change sets to KILN.
This is a no-programming no-hg answer, but it solved my mercurial case-folding problems once and for all! For now anyway..
I gave up trying to avoid and "fix" the case-collision problems. That just looks ugly and you can never really "solve" the problem, only can do a workaround.
The only way (that I can think of) to really solve the problem is to have a case-sensitive filesystem. No need to reformat your entire disk, a single partition will do the job nicely.
I used the Disk Utility app that comes with the os, pretty straightforward, just remember to select Mac OS Extended (Case-sensitive, Journaled) when creating new partition. Also, note that the Disk Utility can resize partitions only by moving the end (not the beginning) of partition.
You can probably create a symlink to where your old source code lived, so no need to change the IDE settings and stuff (but I haven't tried this, just happy with the new partition).
We resolved this without resorting to a case-sensitive filesystem by issuing HG rename commands. Say you are having trouble because "Foo.txt" needs to be called "foo.txt":
> hg rename Foo.txt Foo.txt.renamed
> hg rename Foo.txt.renamed foo.txt
We encountered this problem when a file was deleted and then later re-created in the main repository with the same name, but different case. A branch repository that was created before these changes could not then be merged in, despite the changesets from the main repository having been pulled.
For Mac OS X, what worked for me is to simply copy the folder (duplicate works - cmd-D) and continue working on it, from the new path.
OSX same I wanted to clone a repo and I had case-folding error preventing me top clone.
If the repository has multiple commit simply clone an earlier revision with this command:
hg clone -r 7
Then add anything conflicting to the .hgignore file and update.

Resources