Tracking down dependencies in darcs - darcs

I am using darcs at work for over a year now, but I keep asking myself the same question:
What is the best way to track down the code line/file/code change that is causing a dependency between two patches? For now my way of doing this is as follows:
I check which files the two patches affect using darcs changes -i and typing x for the appropriate patches.
I check which files are affected by both patches.
I guess which file is causing the dependency and by using darcs amend --unrecord I remove changes to this file from both patches.
I check whether the dependency is resolved and add the changes back to the patches, while constantly checking whether the dependency reoccured or not and thereby tracking down the part of the code that is causing the dependency.
This trial and error technique of finding the code line/file that causes the dependency takes a lot of time. There must be an easier way to do this, am I missing something? Thanks in advance!

Darcs show dependencies:
starting with darcs 2.12 (release notes)
you can generate a dependency graph with the command
darcs show dependencies
release notes
official documentation.
Note on the darcs Version
I recommend installing darcs with stack as described in the release notes.
Displaying the output
The command yields the dependency graph in the graphviz dot format.
You have to take care of displaying the output yourself (no surprise there).
A simple way would be to pipe the output directly into dot, have it create some output with (for instance a png using Tpng) and then pipe that into Display # ImageMagick:
darcs show dependencies --last 20 | head -n -1 | dot -Tpng | display
Or you can write the stuff into a File and open it with evince or whatever viewer you prefer:
darcs show dependencies --last 20 | head -n -1 | dot -Tpdf > darcs-dependencies.pdf && xdg-open darcs-dependencies.pdf
Graphical Interface:
I've actually written a gui that automates this process and does some color coding.
you can find it here: iHateDarcs on github
(Disclaimer/Warning: I'm currently the only user of that gui. It's highly customized to my workflow, includes lots of other stuff you might not be interested in, such as an integration with the redmine issue tracker and requires some setup work to get running, but has not been tested by anyone who isn't me as of now)

Related

Building custom Go Plugin

I'm in the process of creating a custom transformer for kustomize. However, I'm running into issues creating even the most basic Go Plugin. I'm trying to follow these steps here https://github.com/kubernetes-sigs/kustomize/blob/master/docs/plugins/goPluginGuidedExample.md
I'm using one of the plugins in mainline kustomize, ie. secretsfromdatabase [1]
According to the documentation, the instructions I'm following are:
tmpGoPath=$(mktemp -d)
GOPATH=$tmpGoPath go install sigs.k8s.io/kustomize/kustomize
GOPATH=$tmpGoPath go build -buildmode plugin -o SecretsFromDatabase.so SecretsFromDatabase.go
cp SecretsFromDatabase.so ~/.config/kustomize/plugin/mygenerators/sopsencodedsecrets/SopsEncodedSecrets
Now when I run kustomize, I get the following error:
Error: accumulating resources: recursed accumulation [...] fails to load: plugin.Open("$HOME/.config/kustomize/plugin/mygenerators/sopsencodedsecrets/SopsEncodedSecrets"): plugin was built with a different version of package internal/cpu
What is strange is I'm using the same tag in git as the version that is installed on my system.
kustomize version tags/kustomize/v3.5.4^0
{Version:3.5.4 GitCommit:3af514fa9f85430f0c1557c4a0291e62112ab026 BuildDate:2020-01-17T14:23:25+00:00 GoOs:darwin GoArch:amd64}
[1] https://github.com/kubernetes-sigs/kustomize/tree/master/plugin/someteam.example.com/v1/secretsfromdatabase
As for now plugins are very difficult to write and support because the environment should be identical and in practice only original build system can reliably build the plugins. In result a lot of people like you finding little differences in their build environments. I think it is bad idea from design and strongly recommend to get acquainted with Reddit discussion here

Make maven output show progressed sub-modules only

I am working with an automatic build script in maven 3.x. The parent project contains of more than 60 modules. The compilation is done in a shell script simplified this way:
for each module:
cd module
mvn clean install > compile.$module.log
echo "Compiled $module"
I like to see a list of compiled modules in order to see the progress or the build. I like to have a big maven command and avoid the manual for loop. I hope to speed up the build this way, since splitting the parent project into more independent modules is not a short time option, yet.
The --quiet flag might be enough already. Alternatively a user defined logging implementation would be fine as well, as described in the manual (https://maven.apache.org/maven-logging.html)
The questions are:
What is the prefered way to modify maven log output?
Does anyone already know a ready-to-use plugin for my purpose?
Thanks

How to pin revision number of Jenkins build?

I have a Jenkins multiphase job that
gets an update from version control (Subversion)
does a Maven build. Couldn't be more cut and dry.
The 1. above, svn update, does return a latest revision number. I would like to fetch that into 2. so that the build and its associated artifacts do have that number pinned to them, if any way possible in the artifact name itself but, if not, in the build history. Is there a way to do it and how (e.g. using the subversion or another plugin)?
I am using the buildnumber plugin to fetch the build number and generate a small text file that is contained inside my WAR artifacts, which makes it subsequently available via HTTP. But to see it, one must either first deploy the artifact or at least extract its contents. I would like it more readily visible in Jenkins.
You should take a look at How to get SVN revision number in Jenkins Workflow Plugin? - He solved the problem with:
def revision = 'svn info'.execute().in.text.split('\n').find { it.startsWith('Revision') }.split(':')[1].trim()
He's obviously using the latest Jenkins version, but if you're using one of the old Jenkins versions you can simply run the following command and parse the result to get the revision number you want:
svn info -r HEAD
BTW you can use a Jenkins constant too. In the browser you can open http://your-jenkins-host/env-vars.html/ and you will find a constant named SVN_REVISION. Each job build keep the SVN revision into that variable.

Generate an .xcscheme file from the command line

I am working on my company's continuous integration server, and the build process is failing because the server does not have access to schemes in an xcode project.
Basically, they are using Cmake to generate xcode projects on the fly to be used for a single build, and then discarded until the next check in.
My research indicates that this problem will be fixed if there is an .xcscheme file with the .xcodeproj file, but for various reasons that can't be generated and checked in ahead of time.
Is there a way to generate this file using xcodebuild or some other command line tool so that we can work it into existing build shell scripts?
The xcodebuild documentation, google, and S.O. are surprisingly lacking on this topic.
I generate the XCode project using the -G Xcode too; I'm using the scan-build plugin ( http://blog.manbolo.com/2014/04/15/automated-static-code-analysis-with-xcode-5.1-and-jenkins ) in jenkins.
It requires the workspace files.
My script to launch & watch xcode looks like that:
($WORKSPACE is set by jenkins)
#!/bin/bash
/Applications/Xcode.app/Contents/MacOS/Xcode "${WORKSPACE}/Build/arangodb.xcodeproj" &
XCODE_PID=$!
# now we wait for xcode to build the workspace:
WAIT_FOR_XCODE=0
while test ${WAIT_FOR_XCODE} -lt 6; do
WAIT_FOR_XCODE=`find "${WORKSPACE}/Build/arangodb.xcodeproj" |wc -l`
sleep 2
COUNT=`ps -p ${XCODE_PID} |wc -l`
if test ${COUNT} -lt 2; then
echo "XCode went away unexpectedly!"
exit -1
fi
done
#ok, we believe its done. kill it, and wait until its realy dead.
kill ${XCODE_PID}
COUNT=2;
while test ${COUNT} -gt 1; do
sleep 1
COUNT=`ps -p ${XCODE_PID} |wc -l`
done
Unfortunately, as of Xcode 5.1.1 there does not exist a mechanism for auto-generating .xcodeproj or .xcworkspace build schemes from the command line as the Xcode UI does. The good news though is that an Xcode project's pbxproj markup is an order of magnitude more complex than the XML markup that describes a build scheme. If you've managed to get CMake to spin up a well-formed Xcode project on-commit, then using a very similar procedure you can build out the 100 or so lines of XML that describe the build-run-test-profile-archive actions of that Xcode project.
If you've not taken a peek at the underlying XML structure of a scheme, create a sample iOS project from the new project wizards and then go poking through the contents of the .xcodeproj or .xcworkspace file for .xcscheme files. The structure is fairly self-documenting and you might even be able to get away without actually specifying the XML markup for those actions you know that will not be run on CI.
Failing that, a less robust approach would be to looking into opening up the Xcode project/workspace file upon the completion of your CMake build process. After a handful of seconds, Xcode's indexer will have had time to identify the projects and auto-generate the schemes for the projects within the master project file itself. Obviously, as you are relying on a UI-layer operation in this approach, you are subject to Xcode's whims, and the indexer may take more than a few seconds to build its index (ex. larger projects will take longer to auto-generate schemes!) ...and there is no trigger advising command-line processes that the indexing and generation has succeeded or failed. You'd wind up having to poll for the existence of a file with an appropriate timeout which can get a bit dicey in an automated build and test environment.
I was actually able to do this by using cmake to generate the project, then using the xcode gui to make the scheme files I need. I used the terminal to extract the xcscheme files from the project and put them in another directory being tracked by source control. As part of the generation process, I just added a bit of shell script to copy the copies I made earlier into the newly generated project, then continue the build process as normal.
The latest version of cmake has added this functionality:
https://blog.kitware.com/cmake-3-9-0-rc3-is-now-ready-for-testing/
The "Xcode" generator learned to create Xcode schema files. This
is an experimental feature and can be activated by setting the
"CMAKE_XCODE_GENERATE_SCHEME" variable to a "TRUE" value.
For CMake based project use XCODE_GENERATE_SCHEME setting for your target.
Setting:
set_target_properties(<your_target> PROPERTIES
XCODE_GENERATE_SCHEME YES
)
will produce following file
your_project.xcodeproj/xcshareddata/xcschemes/your_target.xcscheme

hadoop2 build hdfs without yarn and mapreduce

I want to make some changes to hadoop hdfs according to a published paper. After that I just need to build HDFS and get it running. How can I do that?
Refer the following Hadoop documentation
http://wiki.apache.org/hadoop/HowToContribute
This assumes you build on Linux. If you use a different OS, you may need to do some extra steps; for details see this - I've never done this on non-Linux myself.
Install Git, Java (JDK), Maven and ProtocolBuffer (2.5+ version required)
Clone https://github.com/apache/hadoop-common.git by typing something like this in your command line:
git clone https://github.com/apache/hadoop-common.git
Note: you may want to use a particular branch corresponding to the version of HDFS you're looking to build. To list all branches, type git branch -a. Then to switch to branch 2.3, for example, type:
git checkout --track origin/branch-2.3
If you did everything correctly, you should see a message about tracking the remote branch you've selected.
Make whatever changes you need to make in HDFS; the code lives under hadoop-hdfs-project.
Compile the project by running the following from the root of your tree:
mvn install -DskipTests
This will take some time the first time you do it, but will be a lot quicker during re-runs.
Your final jars will be placed into directories like hadoop-hdfs-project/hadoop-hdfs/target (this is accurate for at least 2.3, but it might have been different in older version, or it may change in the future).

Resources