Git blame delivers different results in Gitlab CI than on local machine - bash

I've created a small shell script that uses the output of git blame to generate some reports. This works great on my local machine. But then I wanted to integrate the shell script in our gitlab ci/cd pipeline and for some reason when I run git blame on gitlab, all lines are attributed to me and the latest commit.
This is what I put in my pipeline file to debug this issue:
run-git-blame:
stage: mystage
image: alpine
before_script:
- apk add bash git gawk
- git status
- git log
script:
- git blame HEAD -- somefile.txt
When I run git blame HEAD -- somefile.txt locally I get:
4588eb70a0b (DXXXXXXX 2019-05-07 15:35:22 +0200 1) abc=123
4588eb70a0b (DXXXXXXX 2019-05-07 15:35:22 +0200 2) def=456
4588eb70a0b (DXXXXXXX 2019-05-07 15:35:22 +0200 3) ghi=789
4588eb70a0b (DXXXXXXX 2019-05-07 15:35:22 +0200 4) jkl=abc
4588eb70a0b (DXXXXXXX 2019-05-07 15:35:22 +0200 5) mno=def
[...]
However the gitlab ci output looks like this:
^5e95b8e5 (Sebastian Gellweiler 2020-09-24 09:32:54 +0200 1) abc=123
^5e95b8e5 (Sebastian Gellweiler 2020-09-24 09:32:54 +0200 2) def=456
^5e95b8e5 (Sebastian Gellweiler 2020-09-24 09:32:54 +0200 3) ghi=789
^5e95b8e5 (Sebastian Gellweiler 2020-09-24 09:32:54 +0200 4) jkl=abc
^5e95b8e5 (Sebastian Gellweiler 2020-09-24 09:32:54 +0200 5) mno=def
[...]
Bu I have definitevly not touched the file being inspected.
The git status command outputs the following on the server:
HEAD detached at 5e95b8e5
nothing to commit, working tree clean
And this looks fine, as 5e95b8e5 is the hash of the latest commit.
What also puzzles me is the output of git log on gitlab because it only shows one commit and no history:
commit 5e95b8e5efcfd155d4248f4849b848e9f1580a20
Author: Sebastian Gellweiler <sebastian.gellweiler#dm.de>
Date: Thu Sep 24 09:32:54 2020 +0200
...
This is what the history on my local machine looks like:
commit 5e95b8e5efcfd155d4248f4849b848e9f1580a20 (HEAD -> feature/XXX)
Author: Sebastian Gellweiler <sebastian.gellweiler#example.org>
Date: Thu Sep 24 09:32:54 2020 +0200
...
commit bc689804f87d906e1b2435249fc1aaaf32e49b20
Author: Sebastian Gellweiler <sebastian.gellweiler#dm.de>
Date: Wed Sep 23 18:40:18 2020 +0200
XXX
commit 9d6fa2336aee0938aef0bd78563b6f12dee5934f (master)
Merge: 90ef282515 3eabec88c2
Author: XXXX <XXX#example.org>
Date: Thu Sep 17 08:31:20 2020 +0000
XYZ
[...]
As you can see the top commit hash is the same on my local machine and on the remote.
I'm stuck here. Anybody has an explanation for this discrepancy?

I think you have GitLab configured to make a shallow clone that has depth 1. Since it has only one commit, every file in the repository is the way it is due to the (single, one) commit that is in the repository, so that's the hash ID it produces.
In particular, this notation:
^5e95b8e5
indicates that Git knows there is something before 5e95b8e5, but not what it is, which in this case is the mark of a shallow repository. (Technically it's a boundary mark and you'd see it on some other commit if you had used a range expression, such as abc1def..HEAD. You can use the -b option, or some configuration items, to alter how these are shown.)

Git commits only connect to their parents, Git can only go through history backwards. If you have a commit checked out that's older than 5e95b8e5 you will not see it in git blame nor git log.
Alternatively, your local repo is out of datee and you need to git pull.

Related

files don't exist yet "this exceeds GitHub's file size limit of 100.00 MB" [duplicate]

This question already has answers here:
How to remove file from Git history?
(8 answers)
Closed 1 year ago.
I am trying to git add, commit, push an update made to some Python code, where I changed the naming convention of the files.
NB: I want my local branch to replace the remote version.
I have also deleted these files from data/ folder. However, git push and git push --force yield the same error:
remote: error: File workers/compositekey_worker/compositekey/data/20210617-031807_dataset_.csv is 203.87 MB; this exceeds GitHub's file size limit of 100.00 MB
remote: error: File workers/compositekey_worker/compositekey/data/20210617-032600_dataset_.csv is 180.20 MB; this exceeds GitHub's file size limit of 100.00 MB
But data/ only contains example datasets from online:
$ ls
MFG10YearTerminationData.csv OPIC-scraped-portfolio-public.csv
Is the problem to do with caching? I have limited understanding of this.
git status:
On branch simulate-data-tests
Your branch is ahead of 'origin/simulate-data-tests' by 6 commits.
(use "git push" to publish your local commits)
nothing to commit, working tree clean
git rm --cached 20210617-031807_dataset_.csv:
fatal: pathspec '20210617-031807_dataset_.csv' did not match any files
git log -- <filename> in data/:
$ git log -- 20210617-031807_dataset_.csv
commit 309e1c192387abc43d8e23f378fbb7ade45d9d3d
Author: ***
Date: Thu Jun 17 03:28:26 2021 +0100
Exception Handling of Faker methods that do not append to Dataframes. Less code, unqiueness enforced by 'faker.unique.<method>()'
commit 959aa02cdc5ea562e7d9af0c52db1ee81a5912a2
Author: ***
Date: Thu Jun 17 03:21:23 2021 +0100
Exception Handling of Faker methods that do not append to Dataframes. Less code, unqiueness enforced by 'faker.unique.<method>()'
A bit of a round about way but works for this situation effectively.
If you are sure that you want your local branch files to be in your remote branch; and have been experiencing these issues of once committed but since deleted files.
On GitHub Online, go to your folder, select you're branch.
Then "Add file" > "Upload files" file manually that you initially wanted pushed.
Then on your machine:
git checkout master
git branch -d local_branch_name
git fetch --all
I was successfully able to make a git push thereafter.

When did owner disappear from the net_device struct? 'struct net_device' has no member named 'owner'

Isn't owner supposed to be part of the net_device structure? I saw threads indicating the macro SET_MODULE_OWNER (which assigned the member) was/is/potentially being deprecated, but I thought I understood that the member would remain in the structure. Did it disappear at some version of the kernel?
Modern kernel git repository doesn't have this change. You can use full kernel history repository to find out the change:
$ git log -S'struct module *owner;' -- include/linux/netdevice.h
shows this commit:
Author: David S. Miller <davem#nuts.ninka.net>
Date: Mon May 19 04:30:48 2003 -0700
[NET]: Fix netdevice unregister races.
Wow, it was 13 years ago, pretty old stuff. Here is the commit itself: link. And the change you are looking for:
- /* open/release and usage marking */
- struct module *owner;
So it looks like (from the patch) that you can just forget about that owner stuff. To figure out the kernel version when this commit was done:
$ git tag --contains 558d700c48 | head -1
which shows v2.6.0. And previous tag is v2.4.0. So it was done between 2.4 and 2.6.
Actually, by googling the patch name, we can find out that it was added exactly in 2.5.70 version.
The same can be done to figure out where SET_MODULE_OWNER has gone. This patch removes it:
Author: Ralf Baechle <ralf#linux-mips.org>
Date: Mon Sep 17 13:11:17 2007 -0700
[NET]: Nuke SET_MODULE_OWNER macro.
And the whole patch can be seen here. Patch was merged in v2.6.24.

Teamcity Configuration Settings

I need to know the teamcity settings which prevents the re-trigger/trigger of outdated builds/jobs if the new builds are successful.
I am facing a issue where teamcity jobs can be re triggered even if the next builds are successful.And If the trigger event is fired before, then it must stop teamcity to run that job if the latest build is successful.
So I have to 2 jobs in TC for 1 branch -- Build-Precheck and the other is Build-compile
So I could see that Build-compile is just picking the latest available successful build from Build-Precheck and then queing up the next which may be the outdated build.
Build-Precheck is just taking 2 min to finish the builds , it quickly triggers the latest builds , I guess following the principal First In First Out
Build-Precheck
06 Oct 14 14:33 - 14:35 (2m:01s) –7.1.4345
06 Oct 14 14:41 - 14:43 (2m:16s)- 7.1.4346
06 Oct 14 14:45 - 14:47 (2m:10s)- 7.1.4347
Build-compile
06 Oct 14 14:35 - 15:00 -7.1.0.4345
06 Oct 14 14:52 - 15:20 (28m:02s)- 7.1.4347
06 Oct 14 16:08 - 16:33 (24m:52s)- 7.1.4346
Is there any fix for this that TC runs incremental builds rather than outdated ones
Sounds like you are looking for Configuring Build Trigger.
AFAIK, there isn't a way to cancel queued builds if a given build passes. However, you can adjust the Build Triggers that queue those builds. Most likely, you'll need to set the Quiet Period on your VCS Build Trigger to longer than it takes for your build.
For example, if your full build takes 5 minutes, you should set the Quiet Period to 7. This way additional builds wont queue while a build is running.

Xcode Continuous Integration bot fails after Git force push

We have an Xcode CI bot setup to poll our git repository for new commits and build accordingly. Generally it runs fine. However after forced pushes (I know) the bot will fail and never build correctly again. The solution has been to delete the bot and start over (and admonish ourselves for force-pushing).
Within Xcode build logs there is no error, hitting the console we can only confirm what the problem is (see the last log)...
Jan 28 08:15:51 macmini.local xcsbuildd[80853]: [CSBotSCMAction gitCloneRepositoryAtURL:branch:destinationPath:createDirectoryNamed:completionBlock:] : https://github.com/XXXXXXXX/iOS.git
Jan 28 08:15:51 macmini.local xcsbuildd[80853]: newRepoURL: https://github.com/XXXXXXXX/iOS.git
"https:\/\/githubuserformacmini#github.com\/XXXXXXXX\/iOS.git",
"https_github_com_XXXXXXXX_iOS_git"
"launchCommand" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git clone https:\/\/githubuserformacmini#github.com\/XXXXXXXX\/iOS.git --recursive --verbose --progress https_github_com_XXXXXXXX_iOS_git",
"launchPath" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git",
"GIT_ASKPASS" : "\/Applications\/Server.app\/Contents\/ServerRoot\/usr\/libexec\/xcs_ssh_auth_agent",
Jan 28 08:18:29 macmini.local xcsbuildd[80853]: Obtaining the HEAD hash at: /Library/Server/Xcode/Data/BotRuns/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle/tmp/https_github_com_XXXXXXXX_iOS_git
Jan 28 08:18:29 macmini.local xcsbuildd[80853]: [CSBotSCMAction gitHeadHashesRepositoryAtPath:branch:completionBlock:] : /Library/Server/Xcode/Data/BotRuns/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle/tmp/https_github_com_XXXXXXXX_iOS_git
Jan 28 08:18:29 macmini.local xcsbuildd[80853]: newRepoURL: file:///Library/Server/Xcode/Data/BotRuns/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle/tmp/https_github_com_XXXXXXXX_iOS_git/
"launchCommand" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git show-ref --heads",
"launchPath" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git",
"currentDirectoryPath" : "\/Library\/Server\/Xcode\/Data\/BotRuns\/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle\/tmp\/https_github_com_XXXXXXXX_iOS_git",
"launchCommand" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git checkout release",
"launchPath" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git",
"GIT_ASKPASS" : "\/Applications\/Server.app\/Contents\/ServerRoot\/usr\/libexec\/xcs_ssh_auth_agent",
"currentDirectoryPath" : "\/Library\/Server\/Xcode\/Data\/BotRuns\/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle\/tmp\/https_github_com_XXXXXXXX_iOS_git",
Jan 28 08:18:29 macmini.local xcsbuildd[80853]: [CSBotSCMAction gitHeadHashesRepositoryAtPath:branch:completionBlock:] : https://github.com/XXXXXXXX/iOS.git
Jan 28 08:18:29 macmini.local xcsbuildd[80853]: newRepoURL: https://github.com/XXXXXXXX/iOS.git
"https:\/\/githubuserformacmini#github.com\/XXXXXXXX\/iOS.git"
"launchCommand" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git ls-remote --heads https:\/\/githubuserformacmini#github.com\/XXXXXXXX\/iOS.git",
"launchPath" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git",
"GIT_ASKPASS" : "\/Applications\/Server.app\/Contents\/ServerRoot\/usr\/libexec\/xcs_ssh_auth_agent",
Jan 28 08:18:30 macmini.local xcsbuildd[80853]: [CSBotSCMAction gitCommitSummaryForRepositoryURL:betweenHashIdentifier:andHashIdentifier:completionBlock:] : /Library/Server/Xcode/Data/BotRuns/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle/tmp/https_github_com_XXXXXXXX_iOS_git
"launchCommand" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git log --no-color --name-status --format=fuller --date=iso e478616e4b3915846f7938fec24e8dc12cdae52a..f2c1b24a6b801ed9f7e60dce60add1851618da64",
"launchPath" : "\/Applications\/Xcode.app\/Contents\/Developer\/usr\/bin\/git",
"GIT_ASKPASS" : "\/Applications\/Server.app\/Contents\/ServerRoot\/usr\/libexec\/xcs_ssh_auth_agent",
"currentDirectoryPath" : "\/Library\/Server\/Xcode\/Data\/BotRuns\/BotRun-187dbc1a-dae2-4ddc-a75b-75831de7ff09.bundle\/tmp\/https_github_com_XXXXXXXX_iOS_git",
Jan 28 08:18:30 macmini.local xcsbuildd[80853]: [XCSCheckoutOperation.m:1033 7d02a310 +168ms] Error getting Git commit log in range e478616e4b3915846f7938fec24e8dc12cdae52a:f2c1b24a6b801ed9f7e60dce60add1851618da64 <stderr>= fatal: Invalid revision range e478616e4b3915846f7938fec24e8dc12cdae52a..f2c1b24a6b801ed9f7e60dce60add1851618da64
Jan 28 08:18:30 macmini.local xcsbuildd[80853]: [XCSCheckoutOperation.m:610 7d02a310 +0ms] Failed to get Git commit history for repo with error Error Domain=CSBotSCMAction Code=-1000 "fatal: Invalid revision range e478616e4b3915846f7938fec24e8dc12cdae52a..f2c1b24a6b801ed9f7e60dce60add1851618da64
I can see...
git log --no-color --name-status --format=fuller --date=iso e478616e4b3915846f7938fec24e8dc12cdae52a..f2c1b24a6b801ed9f7e60dce60add1851618da64
...coming from the integration. e478616e4b3915846f7938fec24e8dc12cdae52a is the commit that was removed (I assume) when I forced in f2c1b24a6b801ed9f7e60dce60add1851618da64. I don't know where the bot is keeping around that information. I tried deleting everything out of /Library/Server/Xcode/Data/BotRuns and that didn't work. I figured maybe it was pulling the last hash from /Library/Server/Xcode/Data/BotRuns/Latest, nope. I also dug around a ton of the other directories in /Library/Server/Xcode without seeing anything.
It would be nice if Xcode's CI approach gave us any control over the git workflow. There's a ludicrously few amount of configuration options. Maybe delete is the only way to go.
It seems like someone removed a commit that was already pushed to a remote. Most CI servers detect changes in the repository with this basic workflow:
Fetch from remote
Do a git log or diff where the starting commit is the current HEAD of a cloned repository, and the end commit is the current HEAD of the remote branch the cloned repo is tracking
If the HEAD of the current checked out branch is removed in the remote repository, Git will probably fail.
You might be able to go to the directory where the CI build has the Git repository checked out and do:
git reset --hard origin/branch_name
This will bring your local and remote branches back in to parity. Then you'll probably need to kick off a manual build.
The real answer is to not alter Git's commit history when those commits have already been pushed to a remote.

Getting specific part of text from file based on in-line date

I am not savvy at all when it comes to scripting. The script I have basically dumps the results of git log command into a file.
However, I would only like to show the lines for the day the script was run. So if I run myscript.sh on Thu Jun 20, I want to see all lines from a file down until Wed Jun 19.
Here is what the file looks like:
commit 8da0dd9bsd23899d11b4ee7348af0640b98ed4b17
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Thu Jun 20 12:08:59 2013 -0400
Testing Git push 13 6
Multiple lines
commit aca564549f91329fcfa9a9f908f7fdeffa83f139b
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Thu Jun 20 12:01:48 2013 -0400
Testing Git push 13 5
commit b80c51b32f48364c2108588aff4c9e12fbb78370b
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Thu Jun 20 11:59:57 2013 -0400
Testing Git push 13 4
commit c4f8f8d4196f7c0f2deaf8g0ecc61797e7b8afdd9
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Wed Jun 19 11:48:37 2013 -0400
Testing Git push 13 3
commit 9a296b2273528868e3e4dc19310fa802daf76b1f3
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Wed Jun 19 11:45:49 2013 -0400
Testing Git push 13 2
commit 55cb8f2399242f051f577a042713a402137df4456
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Sat Jun 15 11:40:48 2013 -0400
Testing Git push 13 1
commit a48e59ec1de227cc2878dce3330ge7776336eb289
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Thu Jun 13 11:28:56 2013 -0400
Switched datasource to SuprPakJ
Created WWButton and WWLabel (extends JButton and JLabel)
Designed Sales Order screen
commit 57ce2da4673a35f50a5146d43a1f1a969c590c8c9
Author: Denis <Denis#WWOscar.Waudware.local>
Date: Tue Jun 11 08:20:58 2013 -0400
I tried searching, but best I got was the sed -e command to print out everything until the first blank line, which isn't exactly what I need.
Any help is appreciated!
How does this work for you:
git log --since=yesterday

Resources