I have created a cloud build with Trigger event push to branch. I have added 2 files in included file filter. So when ever I updated those files my cloud build is triggering fine. But I want to know which file is updated, based on that I need to implement some other logic.
I can use $(body.commit) it printing all the commit information. but i need only files list which are modified in this commit.
Here a code sample to know which Cloud Run directory has changed in my project
- name: 'gcr.io/cloud-builders/git'
id: "Init devops and list changes"
entrypoint: bash
args:
- -c
- |
git fetch --depth=2
# List the Cloud Run changes
git diff --dirstat=files,0 HEAD^ HEAD -- cloud-run/ | sed 's/^[ 0-9.]\+% //g' | cut -d'/' -f2 | uniq > /workspace/change-in-cloudrun
I got the previous commit and compare what changed. I saved the result in a workspace file to be able to reuse the file later on.
Related
So I have this little bash script to output a csv file that shows all my commit history from a specific month.
function make-git-report() {
if [ "$1" != "" ]
then
local month=$1
else
local month=$(date +%m)
fi
local year=$(date +%Y)
local nextYear=$year
local nextMonth=$((month+1))
if [ "$nextMonth" = "13" ]
then
local nextMonth="01"
local nextYear=$((year+1))
fi
local start="$year-$month-01"
local end="$nextYear-$nextMonth-01"
rm -f log.csv
git --no-pager log \
--author="Evert" \
--since="$start" \
--before="$end" \
--branches --remotes --tags --no-decorate --no-merges \
--pretty=format:'§"%ch";"%an";"%s";' --stat | \
grep -v \| | tr -s "\n\n" | tr "\n" '"' | tr "§" "\n" > templog.csv
echo "\nDate;Author;Message;Changes" >> templog.csv
tac templog.csv > log.csv
rm -f templog.csv
}
But I just realized that if a branch is deleted during that month, and it was only merged using a squash merge, then a lot of commits will not show up in my csv file.
I've understood that git reflog will somehow still contain that missing data, but I'm not sure how to merge that information into the output from git log while graciously avoiding things like duplicate entries and maybe more unwanted results that I now can't think of.
Can anybody give me a little hint, a push in the right direction, on how to solve this?
You can't use the reflog to reliably get information about deleted branches :
the log gets updated only if you see the commit at all
e.g : if you take 2 days off, your local repo will have no trace of what happened during those two days ...
two clones of the same repo will not have the same reflog
the reflog will also contain information about rebased/cherry-picked commit (and possibly add duplicate information on the log you are trying to build ?)
the reflog for HEAD will stay around, but reflog for specific branches/remote branches will get deleted if said branch is deleted too
etc ...
I guess you describe a workflow based around a known git central service (github ? gitlab ? azure devops ? ...) where your branches get merged via pull requests.
Each of these services keep a link to the branch of its pull requests :
a generic way to see that is to run git ls-remote, and see what refs are listed (you should see a list of refs/pull/xxx/... or refs/merge-requests/xxx/... or ...)
and read the documentation for said service
You can fetch these pull requests :
# with this refspec : all pull requests will be stored under origin/pull/*,
# as if they were remote branches
git fetch origin +refs/pull/*:refs/remotes/origin/pull/*
# or you may choose to store them in a different set of refs :
git fetch origin +refs/pull/*:refs/pull/*
You may also use the API of said service to check the state of a pull request.
Combining the information above, you may choose, pull request by pull request, whether you should add pull/xyz/head to the list of refs in your git log command or not.
note that, if your branch are squash merged, the "merge commits" will actually not be merge commits, so you would have to choose another way than --no-merges to exclude them from your log (for example : print the commits sha and skip the ones that are known to be a merge request result)
I would like to get current branch or tag by reading only the content in .git folder.
I have read many solutions and they all depend on executing git status, git branch, git describe, or something similar and then parse the output. But what if we can't be sure that there is a git binary to call? We can't rely on that.
For a branch, it looks almost very straight forward: cat .git/HEAD, but for tags, it get's a little more complicated. I use git-flow to create my feature-branches and my tags. When I switch to a tag I get:
$ git checkout tags/v0.11.2
Note: checking out 'tags/v0.11.2'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 86ce70a... Merge branch 'hotfix/0.11.2'
And now the content at .git/HEAD is just a hash: 86ce70a29fdb2c0bdb0b683d00ab61607d8531de.
If I want to see the content of the object related to that hash I do:
$ zlib-flate -uncompress < .git/objects/86/ce70a29fdb2c0bdb0b683d00ab61607d8531de
commit 309tree 9d01f72a058e705a7bc6f9ffc5489096edd2e85a
parent 8c767a0d7538f735c5a537ed14f7f96eb8ae05f8
parent 67d98e0149c72856ddb07ff42197071a4c35fa87
author ####################################### 1520980212 -0600
committer #################################### 1520980212 -0600
Merge branch 'hotfix/0.11.2'
The last line is the message I put in the commit, but it doesn't mean I can get the tag version from there as the message is different on every commit.
I also tried to find any file containing that hash within the .git folder running:
$ grep -ilr `cat .git/HEAD` .git/
.git/gitk.cache
.git/HEAD
.git/FETCH_HEAD
.git/logs/HEAD
.git/logs/refs/heads/master
.git/logs/refs/remotes/origin/master
But none of the files had anything that pointed me to the tag name.
I'm running out of ideas. Any light would be really appreciated.
We can actually address your use case (knowing which tag is currently checked-out in prod, without using the git binary nor by re-implementing it) if we assume that
you have local access to the git repository (and to git commands) before deploying to prod, and that the prod environment (which does not contains git) contains the same repo with the same .git folder.
Indeed, it suffices to run git pack-refs beforehand, keep the repo and the .git folder in the prod environment and run something like
grep -B1 -e "\^$(cat .git/HEAD)" .git/packed-refs | head -n1 | cut -d ' ' -f 2
to get the ambient tag.
FYI Here is a demo performed on a GitHub repo:
$ git clone https://github.com/coq/coq.git
Cloning into 'coq'...
remote: Counting objects: 230444, done.
remote: Compressing objects: 100% (8/8), done.
remote: Total 230444 (delta 1), reused 2 (delta 0), pack-reused 230436
Receiving objects: 100% (230444/230444), 104.35 MiB | 1.23 MiB/s, done.
Resolving deltas: 100% (189775/189775), done.
$ cd coq
$ git pack-refs
$ cat .git/packed-refs
# pack-refs with: peeled fully-peeled
72d6d5e87759b62dcd9974c87bf59496d27e10b0 refs/remotes/origin/master
6aecb9a1fe3f9b027dfd702931298bc61d40b6d3 refs/remotes/origin/v8.0
f7cdf553d983a79fe0fbb08403f6a55230016074 refs/remotes/origin/v8.1
be16dcb34d1d09aa9c850997b3ef3b0cc0e7a864 refs/remotes/origin/v8.2
04a6362feb6cbfaa00be4d001dee2b390d0ff21c refs/remotes/origin/v8.3
7f2240ff25f232e8a27100a619881d0742ab7976 refs/remotes/origin/v8.4
df8c706c2fbdc30b4e2c514b97282e621cd5c9a3 refs/remotes/origin/v8.5
0106ca8db489cd9a202a0c8c3715504d5d1dc86c refs/remotes/origin/v8.6
88c5a874d4767c2e61c885cd8f51d4600e90086a refs/remotes/origin/v8.7
f8b9d25ea4c1b229c56c97ef92cb24ee5f70f53b refs/remotes/origin/v8.8
c6857bddd1bdda169dcb5c93e9f680b5379165a7 refs/tags/V7-0
^e3de2b7791fe5c0798c69e390b3fe58ef6574d01
ca2de40f47aed2204dbf75ddf1b7683654682938 refs/tags/V7-0beta
^36183770e5b6cd724ae18a423c8c2e10bcb5f574
9cb91decf1bb9b8b5c7b7e08a347b75f42d46189 refs/tags/V7-0beta2
^82be21875deb1df309f036137f8b1e29411f50fb
7b34b15de5541ff72d191a18cfc7270eff25b4f5 refs/tags/V7-0beta3-ocaml3-01
^31bf213dea6062c7f27140d76528db59ab287538
6c40cca52d505589f1cb87e8008074cce74467eb refs/tags/V7-0beta4
^45d447c4394c838f3ead2d44a3c7e9965365bc3d
03b24eb11846ac23678782dae41bbbba14c40f2a refs/tags/V7-1
^8e6d0c79f6d5b53441edefc2c1b179d37bee483d
cf335d602ab4d58a36d4d5cca3f03951c6da442a refs/tags/V7-2
^43b06dafec4ceec4a4fd21bda3345f90e9eb76c6
41382705c233b06397c7652a4191fd61fcdcd748 refs/tags/V7-3
^d81ffa972da947a5e99fc4a694768d80382a1d26
1f29769c278aced5153eb2df8d3e7d10bdb95917 refs/tags/V7-3-1
^7c408eb83a36ec055d2a69c783e6c23565841a0d
e0028558217335ea68edd859e939320f5f7a8602 refs/tags/V7-4
^6f6e35a49761ef5b825a08c735999a1ad431994b
f6375d6a93140797817b6ad0868534ca9f98a200 refs/tags/V8-0
^6bdd52469e6bfa16aa9fb118cf7ecbf70825172b
57f2c33e94dad2baf0efe0115949ca21ba4aab7c refs/tags/V8-0beta
^1c4da62a877abbe570dc0618a4212cbf1e6d0166
5a5ab6279dbcbed573c0bb0dae6fc08f8ac6827b refs/tags/V8-0cdrom
^74b16389d20003978bd216f410e1d4ffac60002a
a97e6b42f251b91df24f7cb89598c5e267186fd9 refs/tags/V8-0pl1
^6724ff26374f34c954a95d515c749b68bb39ea6c
71d0080ff394fa7d7898109f9c1dfb43ca596c8d refs/tags/V8-0pl2
^2cb5ddf3462b346dbef715a7d5cd69913accf5a5
b04a60c01c8d1221677b7882dd218922a2bc2fea refs/tags/V8-0pl3
^3e4ce7f60451a648b430b354127c232861f50331
8d3a6b922a37dc006afa91e2fe2ed20c9d448456 refs/tags/V8.0-APP
^d204fb7a7cbdbc17559b8e8e437092a0e91bc926
5d12a600774b236d841be6c9f9e1625d0c638ade refs/tags/V8.0pl4
^9cec31ed26778fd1b786ae7962ad10a8810f02dc
bbba7006e341c0254d44778790e874c301ff368d refs/tags/V8.1
^919e894f7be4bed5be27afc30dcb09ae5eb0f429
e14d28196f6da95c0cdd235e11b780e209319e5f refs/tags/V8.1-APP
^a92a08d28f4ac3d9e14c7a18cecf01079c214774
c730c5de35040942b1c06ba93b100ebd1ca725a1 refs/tags/V8.1beta
^4160197d55eb664a5d906c4d85e2c0341e725327
62af7ca2b960da3a0824c85f7d4348608562e068 refs/tags/V8.1gamma
^46a97e6b74efc6bc814140d196c3c14e37b86497
206a042d14fd5348f86800b783dda9e8c684b2fb refs/tags/V8.1pl1
^6b29e2e184041c257cc419c3b80e71bb6806b517
5b16d061d93e7ade027990f99be47a0938954c23 refs/tags/V8.1pl2
^b54738571f96d4d168090820248eb9b213ba7ee4
d730b7ec5436f5849559621e79145b28f0f98e43 refs/tags/V8.1pl3
^259b6943bb9356d78adb97eb57835429bcb0b136
7976ddcc38a2d8f4c47f8952a2d93745932b0ce9 refs/tags/V8.1pl4
^657da139cb9a3e298384dcf6687457869410b308
a181741597e9ef3a5ce7c87cee46b708ccfab7ea refs/tags/V8.1pl5
^4bb87a72c7f906f8ca1a1e9850321a7fc7e86ec5
fd36c77f1eb70597a19137be89819aad2830cb29 refs/tags/V8.1pl6
^6b4b897e4bea40c1aa01a6b7b5d774667656c951
e3111a5f34d17e817dffc772ea911492aff0876e refs/tags/V8.1pre-beta
^39e218aea065e8502e017d5cd055717586287b49
5c868756d2f74c0a9ccb64d24c6d349b49738f05 refs/tags/V8.2
^23173e8c40c63eb5d0975b96a83cf8dae6d76759
1a355cdf6e34fe62b9cea8afe85f022b7503c2b2 refs/tags/V8.2-1
^3f34ae7aacbb5010382b82387a95a055d6bf9756
19bb470a8d3abce24fb2917f9996c14e9cad5e6d refs/tags/V8.2alpha
^286b99ebc0735eb68b3793036a84e6c7d42a9b3c
0849bb25451bbf6e9fce5b5a1400eac4b76b6502 refs/tags/V8.2beta
^fe36cf9ba43eb15221e3e74bd3dac5c3b79a5bf0
f12a923b29d6df611e2fdecbc31fbc3d1c2da06e refs/tags/V8.2beta2
^35b3e53106007bd9459f196d3b6ad05983557a7d
f985acb0bc8b048d832b9790f0120584efdfda69 refs/tags/V8.2beta3
^f059735ff7f27a548ce5f505b7b20b8cfcc1d3e3
9936999f647bf018dffd3a8c7d8d60cb583fd805 refs/tags/V8.2beta4
^bbf8a42aae2dfbd89b3a19a22970e2a734a68ca0
fd708216478666979f80dac3c1e213e1cab30c71 refs/tags/V8.2pl1
^2ae63e7171cb052034fb10b08c2b9ca124408e7b
c685466562e116a9fd09f0e2ea20d2d5612d90de refs/tags/V8.2pl2
^c0639c58819b6ce7869521ede5c2510ea72627e7
a5e77b347f7d1b17f912a73a2bd4de00166e19ed refs/tags/V8.2pl3
^0f56e3eb52bf294143387dfeff5bb0b2b00d353e
92502cc84d6fe38e202f72deb43761657d1da70b refs/tags/V8.2rc1
^8d70d3bded72ce12bd64d991f5003e3f839e8d45
6c7ddfff6aa08a64b4c22af6e24a1130894f0e30 refs/tags/V8.2rc2
^d9bc69a778977aecd4407849f323f5692165f3c8
c1a4081a73c80b73e87f30e07f81a46355e173a4 refs/tags/V8.3
^828c278f69b3aef75cfcf0493641546c94aa4133
f3836c951894fb4654aaca3f635593a28e7c2712 refs/tags/V8.3-beta0
^f7e80f56a6ebdd87a1a8ac5d75a0c4ba0943ec56
bb7677e9e1570b5981f256e770b9e1c0a7f2cdde refs/tags/V8.3-rc1
^3471d0b44ae2e7f932153adba1ce830016610b6d
39d4b181cb9834df3037c1759486341450c673a0 refs/tags/V8.3pl1
^dc3e9e8802f968db2d6b80760a6f955d8fe9b824
37bc52ae4e4eea6e4488dbb2e2b1ad98acc9b8f5 refs/tags/V8.3pl2
^9341752f9bb20db7e36a5470c97b59b40bb9ae53
586ac0022cdc4f353f510fb2921387ef00c51ef2 refs/tags/V8.3pl3
^d6e1259142f3bb9fcf652cd255418552dbd7b9f2
d67128226a73234a89cad78711424429ad5f7455 refs/tags/V8.3pl4
^b0421db33d6632828b2088b82bdaec832c5aaebc
51526fe09288caed67a5d7c7d705945d2bcc89a6 refs/tags/V8.3pl5
^3bf75b1ff7c00467b8269e2f789eb1968e54d63c
08af22b7de836a5fef0f9947a5f0894d371742de refs/tags/V8.4
^3366f276c63b17a3d78865e12f6d94595f87bb18
808ecb68d2885de84d0e0e2f298a4d42c49a08fa refs/tags/V8.4beta
^a9a068d694699bb2c0c7ce229acfb95ff168e38a
bc905f82d46d94f0e2c78cf4a4ce02f64cf2c534 refs/tags/V8.4beta2
^53191e6e1cd4f8b614219ac99fa7f9bae5c3850b
275706e5fc6319545ee9e7020bc9f7ecb7681848 refs/tags/V8.4pl1
^46b4aac455472f03dd63d01d40d4891c44db0e8c
16b09a075e8c0e0479331986141060c4829e0613 refs/tags/V8.4pl2
^9680d833f1cebf2a3e3082e76494bd36f7b9ef1a
985f884b4a59a75522d5421138ab0b88f128a7ae refs/tags/V8.4pl3
72b423c9497033b3b4ca4e023899f204ceaac9e9 refs/tags/V8.4pl4
0ec5646cbcc725dbe1121e24258fc060223e4d51 refs/tags/V8.4pl5
b705cf029b9db7003c8324366c049c49c21dd5c6 refs/tags/V8.4pl6
2cc42290a2f9af4f6e4c9daaa8415feda784a7c4 refs/tags/V8.4rc1
^e06fd439e3a6193b6efc0234e96c222e66211096
5e23fb90b39dfa014ae5c4fb46eb713cca09dbff refs/tags/V8.5
eaa3d0b15adf4eb11ffb00ab087746a5b15c4d5d refs/tags/V8.5beta1
94afd8996251c30d2188a75934487009538e1303 refs/tags/V8.5beta2
0fd6ad21121c7c179375b9a50c3135abab1781b2 refs/tags/V8.5beta3
d5cbd7b881dcc8b3599b3330e342f0aa55ef467f refs/tags/V8.5pl1
e1661dc9a43b34526437e9bc3029e6320e09f899 refs/tags/V8.5pl2
2290dbb9c95b63e693ced647731623e64297f5c8 refs/tags/V8.5pl3
04394d4f17bff1739930ddca5d31cb9bb031078b refs/tags/V8.5rc1
0d1438851ba3a0b9f76847abc42f3bf8ad26c4cb refs/tags/V8.6
b095c4a1d754d4a003d1324cb15b58666b313221 refs/tags/V8.6.1
bdcf5b040b975a179fe9b2889fea0d38ae4689df refs/tags/V8.6beta1
d24aaa4d0e45dc3ec31c5f576516b01ded403dd8 refs/tags/V8.6rc1
15edfc8f92477457bcefe525ce1cea160e4c6560 refs/tags/V8.7+alpha
169afbbc9c560ea3d2fa63a421a639cf59e4cfb5 refs/tags/V8.7+beta1
^bf128a420614ced228c4eb0fcfd901994c2efb65
4b98c97ceecd547a4191b854b58a3c553341bcf3 refs/tags/V8.7+beta2
^9704cd12804dd036637460da803773f67d6031d1
56f98b99f34eb657c3288c6a8839cfc6133c5e9f refs/tags/V8.7.0
^78e3385221c5c6d024b33107517f5674b3d341c2
8c6816def18031126edd99c89bd0257244299276 refs/tags/V8.7.1
^391bb5e196901a3a9426295125b8d1c700ab6992
714c1769144139ca2187cb4f5362f9056218d188 refs/tags/V8.7.2
^2881a184ef4e8a3275ddf34c07d740db42e0c5d3
71f5c4efd6d62c5283f76c263b6c2d6a6b7e64ae refs/tags/V8.8+alpha
^307f08d2ad2aca5d48441394342af4615810d0c7
fc22a4181b178532eabd1f33b7120374d17cbcd6 refs/tags/V8.8+beta1
^8dee3cd515600d50ae95188d44aad8dcb161b0ea
0ec67923c45fb09acc5be96cb19b3e1b603e5b25 refs/tags/V8.8.0
^6a929e8b94fc95f81699668cea95bc4b91ec67ca
1f48326c7edf7f6e7062633494d25b254a6db82c refs/tags/last-coqide-for-8.4pl3
$ git checkout V8.8.0
Note: checking out 'V8.8.0'.
You are in 'detached HEAD' state. You can look around, make experimental
changes and commit them, and you can discard any commits you make in this
state without impacting any branches by performing another checkout.
If you want to create a new branch to retain commits you create, you may
do so (now or later) by using -b with the checkout command again. Example:
git checkout -b <new-branch-name>
HEAD is now at 6a929e8b9... Backport PR #7277: Mention sphinxcontrib-bibtex in INSTALL.doc
$ cat .git/HEAD
6a929e8b94fc95f81699668cea95bc4b91ec67ca
$ grep -B1 -e 6a929e8b94fc95f81699668cea95bc4b91ec67ca .git/packed-refs
0ec67923c45fb09acc5be96cb19b3e1b603e5b25 refs/tags/V8.8.0
^6a929e8b94fc95f81699668cea95bc4b91ec67ca
$ grep -B1 -e "\^$(cat .git/HEAD)" .git/packed-refs | head -n1 | cut -d ' ' -f 2
refs/tags/V8.8.0
To elaborate a bit on this solution, it relies on the git pack-refs command which extracts the SHA1 corresponding to all the refs (tags and remote branches) from the objects of the repo, and write a file .git/packed-refs that recapitulates this information.
It can be noted that the problem of identifying the two SHA1 that can associated to an annotated tag (namely, the SA1 of the tag itself and the SHA1 of the underlying commit) is easily solved by this packed-refs file, without needing to use zlib-flate or so.
(But of course, the prototype shell command proposed in this answer could be adapted because it will only work if the checked-out commit is an annotated tag, cf. the ^ character involved here.)
You might as well assume (or require) that Git is installed, becaues the closest you will come to being able to produce the output of git describe is by copying the algorithm from git describe. You'll also want to have access to at least git rev-parse and git reflog if you try the idea I suggest here.
Note that even git describe might not produce the tag name that you used to run git checkout. If you have two tags that point to the same commit, it's not possible, after-the-fact, to go from revision hash ID to tag name, since checking out either one gets you the same state—except for reflog traces:
You can look at the HEAD reflog (if it exists), because git checkout writes a message here along with the hash IDs. For a git checkout command that checks out a tag, the message is checkout: moving from <name-or-ID> to <tag-name>. So HEAD is detached and the reflog entry for HEAD#{0} matches moving from .* to (.*)$ and the captured regular expression at the end is a valid tag name and running git rev-parse $tag^{commit} produces the correct hash ID, the last checkout was no doubt given that tag name. (Adjust the regexp syntax as necessary depending on which RE grammar it uses.)
You could open and read .git/logs/HEAD directly to avoid using the git binary, but I think the reflog format has changed at least once, and there's no guarantee that it will be stable in the future. It's probably better to use tools that do make some guarantee.
We use liquibase with oracle. So we have some packages with procedures inside.
In our case changelog structure looks like:
master.xml
|
| release1
| |
| release-maser.xml
| release2
| |
| release-maser.xml
| softObjects
| |
| package-master.xml
| packages
| |
| somePackage.pkb
| somePackage.pks
change sets inside "releases" with runOnChange=false and "softObjects"(that can be create used CREATE OR REPLACE) with runOnChange=true like in best practices:
Try to maintain separate changelog for Stored Procedures and use
runOnChange=”true”. This flag forces LiquiBase to check if the
changeset was modified. If so, liquibase executes the change again.
So every update intall some "delta" change sets from "release" and AFTER reinstal all "softObjects" that was changed and in usual case all is OK.
But if I need to setup new DB, I will get a problem:
in change set from second release I use somePackage(v1), but in next release i need to change logic/API somePackage so I will get somePackage(v2) that can not be used in previously created change sets. So for now I have change set that on update will try to use wrong package version.
For avoid this I can add soft objects directly inside release folder without runOnChange=true when i create it. When I need to change it, i should just copy previous version of file to my new release and make changes inside a COPY.
This approach has some disadvantages:
you make a lot of copy files that can consist from thousands lines of
code(yes I know that it isn`t good)
version control system recognize it as a new file (good challenge for
reviewers)
What i undestand wrong? How should I work with "softObjects", if this objects might be changed?
I am trying a code in shell script. while I am trying to convert the code from batch script to shell script I am getting an error.
BATCH FILE CODE
:: Create a file with all latest snapshots
FOR /F "tokens=5" %%a in (' ec2-describe-snapshots ^|find "SNAPSHOT" ^|sort /+64') do set "var=%%a"
set "latestdate=%var:~0,10%"
call ec2-describe-snapshots |find "SNAPSHOT"|sort /+64 |find "%latestdate%">"%EC2_HOME%\Working\SnapshotsLatest_%date-today%.txt"
CODE IN SHELL SCRIPT
#Create a file with all latest snapshots
FOR snapshot_date in $(' ec2-describe-snapshots | grep -i "SNAPSHOT" |sort /+64') do set "var=$snapshot_date"
set "latestdate=$var:~0,10"
ec2-describe-snapshots |grep -i "SNAPSHOT" |sort /+64 | grep "$latestdate">"$EC2_HOME%/SnapshotsLatest_$today_date"
I want to sort the snapshots according to dates and to save the snapshots that are created in latest date in a file.
SAMPLE OUTPUT OF ece-describe-snapshots:
SNAPSHOT snap-5e20 vol-f660 completed 2013-12-10T08:00:30+0000 100% 109030037527 10 2013-12-10: Daily Backup for i-2111 (VolID:vol-f9a0 InstID:i-2601)
It will contain records like this
I got this code :
latestdate=$(ec2-describe-snapshots | grep ^SNAPSHOT | sort -k 5 | awk '{print $5}')
ec2-describe-snapshots | grep SNAPSHOT.*$latestdate | > "$EC2_HOME/SnapshotsLatest_$today_date"
but getting this error :
grep: 2013-12-10T09:55:34+0000: No such file or directory
grep: 2013-12-11T04:16:49+0000: No such file or directory
grep: 2013-12-11T04:17:57+0000: No such file or directory
i have some snapshots made on amazon, i want to find the latest snapshots made on a date and then want to store them in a file. like date 2013-12-10 snapshots made on this date should be stored in file. Contents of snapshotslatest file should be
SNAPSHOT snap-c17f3 vol-f69a0 completed 2013-12-04T09:24:50+0000 100% 109030037527 10 2013-12-04: Daily Backup for Sanjay_Test_Machine (VolID:vol-f66409a0 InstID:i-26048111)
SNAPSHOT snap-c7d617f9 vol-3d335f6b completed 2013-12-04T09:24:54+0000 100% 109030037527 10 2013-12-04: Daily Backup for sacht_VPC (VolID:vol-3db InstID:i-ed6)
please not that if there are snapshots created on 2013-12-10, 2013-12-11, 2013-12-12. It means that the latest_date should be 2013-12-12 and all the snaphshot created on 2013-12-12 should be saved in file.
Any suggestion or lead is appreciated.
Neither the batch script nor the shell script you posted are a good starting point so let's start from scratch. Sorry, this is too big for a comment.
You want to find the latest snapshots made on a date and then want to store them in a file.
What does that mean?
Do the snapshot files have a timestamp in their name or in their content?
If not - UNIX does not store file creation timestamps so is a last-modified timestamp adequate?
Do you literally want to concatenate all of your snapshot files into one singe file or do you want to create a file that has a list of the snapshot file names?
Post some sample input (e.g. some snapshot file names and contents if that's where the timestamp is stored) and the expected output given that input.
Update your question to address all of the above, do not try to reply in a comment.
Minor issue, you don't need a pipe when re-directing output, so your line to save should be
ec2-describe-snapshots | grep SNAPSHOT.*$latestdate > "$EC2_HOME/SnapshotsLatest_$today_date"
Now the main issue here, is that the grep is messed up. I haven't worked with amazon snapshots, but judging by your example descriptions, you should be doing something like
latestdate=$(ec2-describe-snapshots | grep -oP "\d+-\d+-\d+" | sort -r | head -1)
This will get all the dates containing the form dddd-dd-dd from the file (I'm assuming the two dates in each snapshot line always match up), sort them in reverse order (latest first) and take the head which is the latest date, storing it in $latestdate.
Then to store all snapshots with the given date do something like
ec2-describe-snapshots | grep -oP "SNAPSHOT(.*?)$lastdateT(.*?)\)" > "$EC2_HOME/SnapshotsLatest_$today_date"
This will get all text starting with SNAPSHOT, containing the given date, and ending in a closing ")" and save it. Note, you may have to mess around with it a bit, if ")" can be present elsewhere.
I've created a snapshot using ec2-api-tools of a volume within my AWS EC2 account. Currently I have:
>> ec2addsnap vol-xxxxxxxx -d 'My-first-Snapshot'
SNAPSHOT snap-12345678 vol-xxxxxxxx pending 2013-01-30T17:09:35+0000 086018780037 8 My-first-Snapshot
What I want to do is add a --tag Name='Name Tag' to this newly created snapshot from the snap-12345678 id in the response.
This works>
>> ec2addtag snap-12345678 --tag Name='Name Tag'
How can I automate this? I've started writing a simple shell script - but I'm not sure how I would query the response from the initial ec2addsnap to grab the newly created snapshot id in order to apply ec2addtag? Cheers (Thought I was posting this in Serverfault - my apologies)
I managed to solve this via the use of awk. My Bash Script =
today=$(date +"%d-%m-%Y")
tagname=$2
ec2addsnap vol-$1 -d $2'-'$today;
ec2dsnap | grep $2'-'$today | awk -v tagname=$tagname '{print "Adding Tag too: " $2}; system("ec2addtag "$2" --tag Name=\""tagname"\"")';