nexus configure initial repositories non-interactively - continuous-integration

I would like to create a docker for our nexus instance with the correct repositories, proxies etc already created.
Inspired by this question I started using the script API to configure my repositories. The repositories configured through this API don't work like the ones configured manually though (how sad; especially if you imagine the trouble I went through to get the configuration done with the non-documented script API...). I have already filed a bug therefore if you really want to know the details: https://issues.sonatype.org/browse/NEXUS-19891
Now my question: is there another way to configure the repositories non-interactively?
For jenkins it is possible to put some default configuration in /usr/share/jenkins/ref which will then be used only at the first startup; to give you an initial configuration. I was wondering if something similar exists for nexus? Or some other way that I don't know about?

I use python to do something similar to this:
curl -X POST -u admin:admin123 --header 'Content-Type: application/json' http://localhost:8081/service/rest/v1/script -d '{"name":"test","type":"groovy","content":"repository.createYumProxy('\''test'\'', '\''http://repository:8080/'\'')"}'
curl -X POST -u admin:admin123 --header "Content-Type: text/plain" 'http://127.0.0.1:8081/service/rest/v1/script/test/run'
the exact script that I post (more readable here than with all those escaped quotes):
repository.createYumProxy('{name}', '{url}');
configuration = repository.repositoryManager.get('{name}').configuration.copy();
configuration.attributes['proxy'] = [
remoteUrl : "{url}",
contentMaxAge : 0,
metadataMaxAge : 0
]
configuration.attributes['negativeCache'] = [
timeToLive : 1.0
]
repository.repositoryManager.update(configuration)
The part that was missing in my case was the repositoryManager.update(). As quoted on the ticket:
I think the important item(s) missing from your script is that you are not updating the repositoryManager with the new (copied) configuration (which causes the repository to stop/start and therefore reload config)

Related

How can I download a maven package from GitLab with curl or wget?

I have a maven package (dummy) in my Gitlab Package Registry that I want download with a curl or wget command.
Following this I tried:
curl --user "username:DEPLOY_TOKEN" \
"https://gitlab.com/api/v4/projects/666/packages/maven/dummy/0.0.1-SNAPSHOT/dummy-0.0.1-SNAPSHOT.jar"
but I have:
{"message":"404 Project Not Found"}
The project id is correct.
How can I download the maven package?
To download any package, including a maven one, you will need to use the Packages API.
Following those docs, you need to use:
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.example.com/api/v4/projects/:id/packages/:package_id"
Assuming the 666 in the description is the project ID, then it'd be:
curl --header "PRIVATE-TOKEN: <your_access_token>" "https://gitlab.com/api/v4/projects/666/packages/:package_id"
but you would still need to figure out the package id.
If you don't know the package id, you can use the packages API to list the available packages in the project first.
The endpoint you're using looks like it's from the Maven API documentation page, which specifically states it's not meant for manual consumption, so it's not the recommended method.
If you need to use that endpoint anyway, (as per the note at the top of the page) you need to follow the package registry authentication documentation.
This means that if you want to use a deploy token, you need to make sure your deploy token has read_package_registry, write_package_registry, or both.
Your curl command would then look like this:
curl --header "Deploy-Token: <token>" "https://gitlab.com/api/v4/projects/666/packages/maven/dummy/0.0.1-SNAPSHOT/dummy-0.0.1-SNAPSHOT.jar"
Your download script using curl should be like this.
GLB_PRIVATE_TOKEN=<private-token>;
GLB_GROUP_PJT_ID="<numeric project id>";
MAVEN_GROUP_ID="<maven-group-id replace . with />";
MAVEN_ARTIFACT_ID="<maven-artifact-id>";
MAVEN_ARTIFACT_VERSION="<maven-artifact-version>"
GLB_ARTIFACT_FILE_NAME="<maven-artifact-version w/o SNAPSHOT>-<file-specific-number as found in gitlab>";
FILE_TYPE=".jar";
echo "Running curl for $MAVEN_ARTIFACT_ID-$GLB_ARTIFACT_FILE_NAME"
curl --header "Private-Token: $GLB_PRIVATE_TOKEN" "https://gitlab.com/api/v4/projects/$GLB_GROUP_PJT_ID/packages/maven/$MAVEN_GROUP_ID/$MAVEN_ARTIFACT_ID/$MAVEN_ARTIFACT_VERSION/$MAVEN_ARTIFACT_ID-$GLB_ARTIFACT_FILE_NAME$FILE_TYPE" >> <filename>.<filetype>
In your case your GLB_ARTIFACT_FILE_NAME would be 0.0.1- < some hyphenated number > .jar

Spark REST API, submit application NullPointerException on Windows

I used my PC as the Spark Server and at the same time as the Spark Worker, using Spark 2.3.1.
At first, I used my Ubuntu 16.04 LTS.
Everything works fine, I tried to run the SparkPi example (using spark-submit and spark-shell)and it is able to run without problem.
I also try to run it using REST API from Spark, with this POST string:
curl -X POST http://192.168.1.107:6066/v1/submissions/create --header "Content-Type:application/json" --data '{
"action": "CreateSubmissionRequest",
"appResource": "file:/home/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar",
"clientSparkVersion": "2.3.1",
"appArgs": [ "10" ],
"environmentVariables" : {
"SPARK_ENV_LOADED" : "1"
},
"mainClass": "org.apache.spark.examples.SparkPi",
"sparkProperties": {
"spark.jars": "file:/home/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar",
"spark.driver.supervise":"false",
"spark.executor.memory": "512m",
"spark.driver.memory": "512m",
"spark.submit.deployMode":"cluster",
"spark.app.name": "SparkPi",
"spark.master": "spark://192.168.1.107:7077"
}
}'
After testing this and that, I have to move to Windows, since it is will be done on Windows anyway.
I able to run the server and worker (manually), add the winutils.exe, and run the SparkPi example also using spark-shell and spark-submit, everything able to run too.
The problem is when I used the REST API, using this POST string:
curl -X POST http://192.168.1.107:6066/v1/submissions/create --header "Content-Type:application/json" --data '{
"action": "CreateSubmissionRequest",
"appResource": "file:D:/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar",
"clientSparkVersion": "2.3.1",
"appArgs": [ "10" ],
"environmentVariables" : {
"SPARK_ENV_LOADED" : "1"
},
"mainClass": "org.apache.spark.examples.SparkPi",
"sparkProperties": {
"spark.jars": "file:D:/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar",
"spark.driver.supervise":"false",
"spark.executor.memory": "512m",
"spark.driver.memory": "512m",
"spark.submit.deployMode":"cluster",
"spark.app.name": "SparkPi",
"spark.master": "spark://192.168.1.107:7077"
}
}'
Only the path is a little different, but my worker always failed.
The logs said:
"Exception from the cluster: java.lang.NullPointerException
org.apache.spark.deploy.worker.DriverRunner.downloadUserJar(DriverRunner.scala:151)
org.apache.spark.deploy.worker.DriverRunner.prepareAndRunDriver(DriverRunner.scal173)
org.apache.spark.deploy.worker.DriverRunner$$anon$1.run(DriverRunner.scala:92)"
I searched but no solutions has come yet..
So, finally I found the cause.
I read the source from:
https://github.com/apache/spark/blob/master/core/src/main/scala/org/apache/spark/deploy/worker/DriverRunner.scala
From inspecting it, I conclude that the problem is not from Spark, but the parameter is not being read correctly. Which means somehow, I put wrong parameter format.
So, after trying out several things, this one is the right one :
appResource": "file:D:/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar"
changed to:
appResource": "file:///D:/Workspace/Spark/spark-2.3.1-bin-hadoop2.7/examples/jars/spark-examples_2.11-2.3.1.jar"
And I did the same with spark.jars param.
That little differences had cost me almost 24 hours work... ~~~~

Xcode Server Bots API: Updating a bot with PATCH request

I would like to edit my Xcode bot through the Xcode Server API by sending the blueprint through PATCH.
However, when I send my PATCH request, Xcode Server replies back an unchanged json of my old blueprint.
My request is curl -X PATCH -H "Content-Type: application/json" -d "{\"my\": \"json\"}" https://<username>:<password>#<my_domain>:20343/api/bots/<bot_id>
What am I missing?
There are two missing parameters that will cause the following problems:
Missing xcsclientversion: server will return 400 Bad Request.
Missing overwriteBlueprint=true: server will not change the blueprint.
Your final request should look like the following:
curl -X PATCH -H "Content-Type: application/json" -H "x-xcsclientversion: 18" -d "{\"json goes\": \"here\"}" https://<username>:<password>#<domain>:20343/api/bots/<_id>?overwriteBlueprint=true
Source: radar and Developer Relations (thanks!)

create project for sonarqube with the rest-api / web-api

we try to automate the creation of projects (including user/group Management) in sonarqube and I already found the Web-API-documentation in our sonarqube 5.6-Installation. But if I try to create a project with the following settings
JSON-File create-project.json:
{"key": "test1", "name": "Testprojekt1"}
curl-request
curl --noproxy '*' -D -X POST -k -u admin:admin -H 'content-type: application/json' -d create_project.json http://localhost:9000/api/projects/create
I get the Error:
{"err_code":400,"err_msg":"Missing parameter: key"}
It's a bit strange because if I try e.g. the URL:
http://localhost:9000/api/projects/index
I get the list of the projects I created manuelly and if I try a request like
curl -u admin:admin -X POST 'http://localhost:9000/api/projects/create?key=myKey&name=myProject'
it works too, but I would like to use the new api because it looks like it support much more function that the 4.X API of sonarqube.
Maybe someone here can help me with this problem, if would very thanksful for every useful hint.
best regards
Dan
I found this question because I got the same "parameter missing" error message.
So what we both did not understand: The SQ API expects the parameters as plain URL parameters and not as json formatted parameters as most REST APIs do today.
PS: Would be nice if this could be added to the SQ documentation.

Jersey 2.2 Client POST request to Parse.com utf-8 issue

I am a begginer in Dropwizard / Jersey so please bear with me. I am creating request from my Dropwizard application resource:
Client client = new JerseyClientBuilder(environment)
.using(new JerseyClientConfiguration())
.build("my-app-name");
String response = client
.target("https://api.parse.com/1/functions/myFunction")
.request(MediaType.APPLICATION_JSON)
.accept(MediaType.APPLICATION_JSON)
.header("X-Parse-Client-Key", "mysecredclientcode")
.header("X-Parse-Session-Token", "mysecretsessiontoken")
.header("X-Parse-Application-Id", "mysecretapplicationid")
.post(Entity.text(
"{\"postId\":\"xP0Jc2lrqS\"}"
));
I think I tried all possible variations of Entity.* followed by MediaType.* with no luck.
This should work according to documentation. And I can call other endpoints of my app locally this way with no problems. But requests to parse.com return following entity:
{"code":107,"error":"invalid utf-8 string was provided"}
When I curl from bash I get expected response. I feel like I tried everything, can you please point me in the right direction, what am I doing wrong here?
curl request that works:
#!/usr/bin/env bash
curl -X POST \
-H "X-Parse-Application-Id: mysecretapplicationid" \
-H "X-Parse-Client-Key: mysecretclientkey" \
-H "X-Parse-Session-Token: mysecretsessiontoken" \
-d '{"postId":"xP0Jc2lrqS"}' \
https://api.parse.com/1/functions/myFunction
Just in case you still facing this issue, I solved this adding in my maven project these properties:
<properties>
<project.build.sourceEncoding>UTF-8</project.build.sourceEncoding>
<project.reporting.outputEncoding>UTF-8</project.reporting.outputEncoding>
</properties>
Regards,

Resources