I have some .tar.gz-Archives and would like to only unpack the Tarball to create a sha256-checksum hash of the .tar-File. (The reason for this is that the archive will be un- and repacked later on, as we are generating patch-files.)
Now this seems like an easy task but I'm stuck. There are either Gradle examples for:
getting the unpacked tarTree (with a Gradle Copy-Task and tarTree(resources.gzip('model.tar.gz')) (from documentation: working with files)
unzipping Files (with zipTree), which does not work with gzipped files
Both approaches do not work, since I need to create a checksum of the .tar-File itself. Unfortunately I can't use commandLine or gunzip as the tasks should run on both Windows and Linux.
The only solution I can imagine of right now is unpacking the tar.gz to a fileTree and repacking it to a tar-file, but I'm not even sure the checksum would be the same.
Is there really no way to do this directly?
Finally I got it to work with help from a colleague.
Using resources.gzip(), which returns a ReadableResource, we can copy the resulting InputStream into a .tar-file with IOUtils.copy:
file("test.tar").withOutputStream { outputFile ->
IOUtils.copy(resources.gzip(file("test.tgz")).read(), outputFile)
}
We also needed to add Apache commons ("commons-io:commons-io:2.6") as a dependency.
Related
We are using TeamCity as CI and we are struggling with the final build step: We pull a dependency from another build step (a zip) and want to add a few extra files. Do we really need to extract the zip (quite large file), copy the files over there and zip it again? Is there support or a plugin to add files to existing zip files?
TeamCity itself does not support this and I haven't seen any related plugins, however, if you really need to wait until the final step to add the extra files (maybe you are doing some kind of file generation at this point), then I would recommend using something like 7Zip.
you can update existing zip files (assuming they are not "solid" archives") with a very simple command:
7za.exe u targetZip.zip file.ext
this will add "file.ext" to the zip file "targetZip.zip" without decompressing and re-compressing the archive.
you can find 7Zip here: http://www.7-zip.org/
It would be much better to include those files at the previous step, which lists which files should be included to the final artifact. Rather than trying to modify the already generated artifact. So basically all you need is to add an additional build step that will simply copy those other files to the output folder from which you are producing your final artifact.
I have created a new Java project in
eclipse-jee-kepler-SR2-win32-x86_64.
I have included the Jars in
flink-0.8.1\lib.
I have created the standard WordCount and it works.
I have modified my WordCount to take input from text files and csv files and it works.
all the imports work perfectly.
then i tried import org.apache.flink.api.java.io.jdbc.JDBCInputFormat.
Eclipse doesn't find it?
Why does Eclipse not find the import?
Because inside the jar flink-java-0.8.1.jar there is no directory io/jdbc.
I tried the same thing with flink-0.9.0-bin-hadoop27 and in the jar flink-dist-0.9.0.jar there is no org/apache/flink/api/java/io/jdbc directory. I uncompressed the jar and searched for the string "jdbcinputformat" with 0 results. I searched the string "jdbc" and it is only mentioned in org/apache/log4j, org/eclipse/jetty, and in other places that are not org.apache.flink.api.java.io
So my question is: Where do I find the class JDBCInputFormat?
What can I do to access SqlServer2012 in Flink (apart from accessing it outside Flink, create csv files, and then reading them in Flink (It sounds horrible to me since there should be a class specific for that))?
The corresponding module is not included. In order to use it, you need to build Flink from scratch. Run the following commands:
git clone https://github.com/apache/flink.git
cd flink
mvn -DskipTests clean install
This builds the latest snapshot for flink-0.10-SNAPSHOT. If you want to use stable version 0.9 run different git clone command:
git clone -b release-0.9 https://github.com/apache/flink.git
In your current project, you need to change the used Flink version in your pom file accordingly, eg, 0.10-SNAPSHOT or 0.9-SNAPSHOT.
In Unix I can do the following to delete a file from my jar
zip -d myfile.jar META-INF/SIGN.RSA
Is there a simple way to do it in Gradle/Groovy?
I don't want to unzip and re-zip the whole file as I have some very large jars.
As you can probably tell I am un-signing some jars before re-signing them with my own. If you have a solution for that (and doesn't require unpacking and repacking) I'll accept your answer too.
Per my knowledge there's no built-in task for this specific case.
However, you can easily do it by using exec, e.g.:
exec {
executable "jar.exe"
args "-d", "myfile.jar", "META-INF/SIGN.RSA"
}
I'm using Leiningen to automate the creation of a single 'uberjar' from several dependencies. (For a Web Start Application).
Some of the jar files are signed, and the uberjar process brings across their signature files (which don't match the new jar). I want to add my own self-signed cert and remove the old ones.
Is there an easy way in leiningen to remove these old signing files (or stop them being copied across)? I'm doing it as a manual step at present.
See lein ticket #31. I submitted a patch, but if it isn't in the version you're using, you can put this in your project.clj for a fairly comprehensive ban on .sf files:
:uberjar-exclusions [#"(?i)^META-INF/[^/]*\.SF$"]
This is what I was looking for:
:uberjar-exclusions [#"foo.sf"]
From here: Leiningen has problems building a working uberjar
I have a zip archive artifact. I'm interested in downloading a single file from that artifact. I can't upload that file outside of the archive right now. The documentation says you download an archive like this
/repository/download/BUILD_TYPE_ID/BUILD_ID:id/ARTIFACT_PATH
So, my URL for that looks like this (and I need anonymous access, so you see the guest flag). And it works!
/repository/download/bt23/2253:id/mypackage.zip?guest=1
However, I want one single file from that artifact. And the docs are confusing on how to do that. They do not specify what replaces <zip or jar archive>. And I am not constructing the whole buildNumber or buildTypeId values properly.
/repository/archive/<zip or jar archive>/buildTypeId/BUILD_TYPE_ID/buildNumber/BUILD_NUMBER/index.html
I tried using zip and `.zip'. I tried filling in the build values directly, but it's much different from the working archive download URL.
/repository/archive/zip/buildTypeId/bt23/buildNumber/2253/myfile.txt?guest=1
Does anyone have working concrete example of the URL for a single file in an archive?
You need to replace <zip or jar archive> with relative path to your artifact. For example, if you want to retrieve file.txt from package.zip which is uploaded under dist directory in your build artifacts you need to use this url:
http://server/repository/archive/dist/package.zip/buildTypeId/bt23/buildNumber/2253/file.txt?guest=1
BTW, a new, more straightforward URL syntax will be available in TeamCity 7.0:
http://server/repository/download/bt23/2253/dist/package.zip!file.txt?guest=1
Old URLs will be supported too.