Spring and GitHub: hide sensitive data - spring

I have a repository on GitHub that I would like to make public so recruiters can view it.
This repository though holds my SMTP and a MongoDB URI that shouldn't be shared with others. This information is in my application.properties file.
What's the simplest way to hide this sensitive data and also make sure no one can go look at old commits and see how it was before hiding it?
I have seen some ways on the web but they all look quite complicated...
Thank you for your experience and time

Use environment variables to hide your sensitive data. Like
spring.data.mongodb.host=${MONGO_DB_HOST}
spring.mail.host=${MAIL_HOST}
Set the values at your dev environment.
I don't have any idea about how to hide your old commits.

Make a .gitignore file at the root of your project and inside list whatever files you don't want git to have access to it when you push into GitHUb, for example:
/public/packs
/node_modules/
.pnp.js
/ (forward slash) is used for folders and
. (dot) is used for files
Here follows a picture of the location of the .gitignore file.

If the goal is just for recruitment, would it be acceptable to have a second copy for recruitment, while leaving the original copy alone?
While there's certainly more idiomatic ways of achieving this through git, a simple solution with minimal git knowledge or advanced techniques would be:
Create a new empty git project on GitHub
Clone the new project locally
Copy the (non-.git) files from the existing project into the new project (using either the console or your OS's windowed UI)
Delete or redact the offending entries from the new project
Commit the changes as a single commit
Push the new project back to GitHub

I have not used it myself, but the open source BFG Repo-Cleaner looks like it might satisfy your requirements of simplicity while retaining the activity chart for reviewers to view. This can be done on a publicly-facing copy of the repo if you wish to keep your private working copy, while still keeping the activity history viewable.
Following the tool's usage instructions, you should be able do the following (assuming you want these changes in a fresh copy of the repo):
The first step is to duplicate the repository on GitHub, following the instructions in the GitHub docs.
To do this, first create a new repository.
Next, mirror the repository, following the GitHub instructions:
Open Terminal.
Create a bare clone of the repository.
$ git clone --bare https://github.com/exampleuser/old-repository.git
Mirror-push to the new repository.
$ cd old-repository.git
$ git push --mirror https://github.com/exampleuser/new-repository.git
Remove the temporary local repository you created earlier.
$ cd ..
$ rm -rf old-repository.git
Now that you have the duplicate repository, you can run the BFG Repo-Cleaner to replace all instances of text you want hidden with ***REMOVED***.
$ java -jar bfg.jar --replace-text replacements.txt my-repo.git
The replacements.txt file would contain the SMTP, MongoDB URI, and any other text you want hidden.
mongodb://my-username:my-password#host1.example.com:27017,host2.example.com:27017/my-database
marco-f#example.com
Note that this does not update the latest commit on the master/HEAD branch, so this will need to be manually changed, and then committed. This can either achieved using a final commit using the --amend option, or by making a new commit prior to running the BFG Repo-Cleaner with the files manually changed.
$ git commit --amend
Now that the changes have been made, they can be pushed to GitHub.
$ git push

Related

git automatic add and remove?

so to add a file I need to run git add and to remove a file git remove
but this seems to be a very time consuming job if the project has a lot of files that change on my local copy and then the remote repo needs to be updated.
is there some automatic way to sync the local repo with the remote one, like in the GUI version of git? the gui vesion automatically adds new files and removes deleted files
You can use git add -A. It works on your entire working copy and stages (adds to the "Changes to be committed" section) all new (not ignored), modified and deleted files.
There are GUIs available that may help you add files in bulk. I have used Atlassian SourceTree with some success. However there are ways to add multiple files easily from the command line.
You can use wild cards to add multiple files (i.e. git add CurrentDir/*.c to add all the .c files in the current directory. There are more examples of using wildcards in the git documentation.

How to delete the content of github repository?

I wrongly pushed my code using Xcode toolbar into my repository on github. now, I want to delete all files in my master branch, but not the repository. Then I want to pull my code into repository this time instead of pushing. Does any one know how to delete all contents of master branch?
In git, you can't delete the content of a branch. All you can do is to push a commit that removes all your files.
If you want to start over from a clean repository, you have to delete the current one a create a new one with the same name for example.
Just to support quentin's answer, adding the git commands:
Keep a backup of the local repository folder and remove all its contents.
Add these changes to commit
$ git add *
Add a Commit message
$ git commit "Remove everything"
Push changes
$ git push -f
I made a tool that helps with that .. and it doesn’t require you to enter your auth token since netlify handles that for you :)
check out
https://repo-cleaner.netlify.com
Delete the git repository and create new one with the same name

Extra directories and files added to git repository

I had always used Git locally on Mac OSX and decided to try out Github and Bitbucket. I had some issues getting it working and tried a few fixes that others suggested who had similar problems. I was able to push to Github and Bitbucket, but I also managed to somehow add extra directories and files to my repository in the process.
I'm using Xampp to develop locally, and I have a folder in there that I created my repository with, so the file structure is something like this
Applications
XAMPP
xamppfiles
htdocs
myproject -this is was originally the repository that I had been using
randomdirectory1
randomdirectory2
randomdirectory3
randomfile1
randomfile2
One thing to note. Not every single directory and file in htdocs was added to my repository. There were only 3 added, and about 7 other folders.
Prior to trying out github my repository just consisted of myproject located in /Applications/XAMPP/xamppfiles/htdocs/myproject. Now, there are a bunch of other directories and files added to it. I know this, because in conjunction with the command line I also used GitX for a visual reference. Now, all of these extra files and directories are showing up.
Maybe I don't fully understand how git repositories works. Prior to this, I had only used add, commit and checkout commands. All I know is that my repository appears to be different in GitX and matches on Bitbucket/Github.
What folder did you initialize the Git repo from? It sounds like you meant to init from the myproject folder, but instead you did it from the Applications folder.
Does GitHub show all those extra directories?
Where is the hidden .git folder?
Is this a private repo just for you, or just for you and a few known others? Are you willing to rewrite history?

Working with multiple Git

I have following dir stucture
root
root/framework (Yii)
root/protected/messages
All of this folder must be separate git repos
What I want to do is
root and root/framework must be separate repos. But
root/framework must be pull only because I have no push access to this repository. I mean I want to pull yii when I pull parent repo, but don't want to push when I push parent repo.
Another problem is, remote dir structure of Yii (root/framework) looks like http://screencast.com/t/mU1TgXuZDv
I need only framework folder's contents. How can I pull only this folder's contents into root/framework ?
To make root/protected/messages separate git repo so that, when I push & pull root git repo, to do it for this one too. In other words, to push & pull with parent one to 2 separate remotes.
To solve second problem, I initialized new repo inside root/protected/messages but now they push & pull separatelly. I mean, I want them to push & pull changes to/from 2 remotes at once. Can't figure out how to do it.
Also I have no idea about first problem.
Any suggestions?
In order to create a separate and independent git repos within a parent git repo, you want to look into Git Submodules (http://git-scm.com/book/en/Git-Tools-Submodules). These basically allow you to create a completely independent git repos inside a directory which by itself is a git repository.
To create the submodule the command is git submodule add git://path/to/gitname.git folder-containing-the-inner-git. Of course you will need to cd into the parent folder before firing this command, which in your case will be root. The git://path/to/gitname.git will be the git url for Yii and folder-containing-the-inner-git will be root/framework.
In order to pull a specific folder of Yii of the entire git repo you might want to try out git checkout as suggested by this question on stackoverflow How to pull specific directory with git. I have never tried this myself.
Also, as of Git 1.7 you can also do a sparse checkout (https://www.kernel.org/pub/software/scm/git/docs/v1.7.0/git-read-tree.html#_sparse_checkout). Although you will still have to fetch the entire repo.
Once you create a separate git repo using git submodules inside root, you will have to push and pull the git inside root/protected/messages seperately. You can however automate this process by creating a git hook (http://git-scm.com/book/en/Customizing-Git-Git-Hooks) for the repo inside root. A hook is a script that can be executed upon specific git events/operations like committing, merging, etc. For a full list of these events you can refer to this page ... http://www.manpagez.com/man/5/githooks/
It seems that there is no event for a git push or pull. However there is an event for git merge ... post-merge :
This hook is invoked by git merge, which happens when a git pull is
done on a local repository. The hook takes a single parameter, a status
flag specifying whether or not the merge being done was a squash merge.
This hook cannot affect the outcome of git merge and is not executed,
if the merge failed due to conflicts.
This hook can be used in conjunction with a corresponding pre-commit
hook to save and restore any form of metadata associated with the
working tree (eg: permissions/ownership, ACLS, etc). See
contrib/hooks/setgitperms.perl for an example of how to do this.
So you can write a simple bash script like :
cd root/protected/messages
git pull origin master
So everytime you pull from the outer repo in root this script will get fired and you will be able to pull the contents of your inner repo as well. However, this will happen on every merge, not just the merges that happen on a pull so you might want to be careful.
Hope this helps.
You may try more straightforward way:
Init your git repo in root;
Add your root/framework to .gitignore in it;
Go to root/framework and init new git repository there;
You will have matroshka styled repos. But, to be frankly, they will be harder to support than git-submodules solution, since root repo does not aware about other repos at all, and all pushesh, pulls need to be done separately inn each repo.

uploading code to github

I have created a repository named appengine-testers on github. It is a public repository. Though I easily managed to create a repository but I do not know how to store code there. Do I need to upload the code/folder ?
There are multiple options to do that, i'll just briefly tell the simple one.
git clone ssh-path-to-project
It creates .git folder in the project which is used for references.
cd project
copy the entire project code from any location and paste it in this folder.
Now
Add all the untracked files.
git add .
git commit -am <"commit message">
or
git commit -a
Which automatically takes the changes.Lastly
git push
It pushed the entire code to the repository
One more simple option is do
git init
in the project folder and then change the remote url in the .git folder created inside the project folder
Once you create the repo on github, it'll give you a URL (that looks kinda like git#github.com:yourusername/appengine-testers.git) with push access. From there, you just push from your local repo to that URL.
It's probably easier if you add that URL as a remote. Typically you'd call it 'origin'.

Resources