My question:
I am using SonarQube version 7.1 and trying to extract the metrics and quality gate related to individual projects.
What we have tried
We were using Python SonarQube API to extract these data before our company upgraded to version 7.1. "api/resources" web service Deprecated since sonarqube5.4, so we cannot use it anymore.
I have also tried using getting data using CURL command via Web API using
curl -i -H "Content-Type: application/json" -H "x-api-key:token" -X GET 'http://MY_HOST/api/measures/component?metricKeys=key&component=project_key'
We are able to get a json payload for individual metrics, but involves tedious task of creating the URL every single time.
But I wanted to know if there is a better/smarter way to access these "measures", be it any language or implementation.
You could do this:
Call the API api/metrics/search first to get a (json) list of all the metrics and then iterate over that list and create a comma separated string of all the metric keys.
For example something like this: ncloc,complexity,violations .. as mentioned in the parameters example value in the API documentation here.
Then you could just add this comma separated list to the url as a parameter something like: http://MY_HOST/api/measures/component?metricKeys=ncloc,complexity,violations&component=project_key
and call it once to get the response for all metrics.
Also, I haven't tried this, but as per the latest documentation, the parameter component is optional. So if you omit that, ideally you should get a response with metrics of all the projects.
Related
I have deployed an AWS Lambda function, written in Python, and AWS API Gateway structure to cause POST requests to an API endpoint to be redirected to my function. I want to upload a PDF document to my function and have it store the document in a S3 bucket. The problem I have is that the payload of any POST request to my API is being UTF-8 encoded. I don't want that but can't figure out the magic mojo to disable encoding of the request payload.
I am testing using curl, with the following command line:
curl -XPOST https://xxxxxxxxxx.execute-api.us-west-1.amazonaws.com/test -H 'content-type: application/pdf' --data-binary #document.pdf
UPDATE: I just found the following article describing how API Gateway and Lambda support uploading binary data:
https://aws.amazon.com/blogs/compute/handling-binary-data-using-amazon-api-gateway-http-apis/
This article suggests that all of the complexities that I discussed in the initial formation of my question (still provided below) should not be necessary. All I should need to do to upload binary content to my Lambda function is insure that my request includes an appropriate Content-Type header. I was already doing that, but I massaged my Curl command a bit (modified above) to define my request in exactly the way that is done in this article. I still get UTF-8 encoded data and NOT base-64 encoded data. I tried uploading a jpeg file rather than a PDF so I was doing exactly what was done in the article. Still no love. I don't get it. This article demonstrates exactly what I'm doing. But I don't get the result it suggests I should. Ggggrrrr.
ORIGINAL POST:
I am using Terraform to define my deployment. I want to cause the PDF to not be encoded/mangled at all. This is my first time using API Gateway, and I'm obviously missing some bit of config. The one thing I'm doing specifically right now to say that I want incoming payloads to be treated as binary is via the binary_media_types argument to my API definition in Terraform:
resource aws_api_gateway_rest_api proxy {
...
binary_media_types = [
"application/pdf",
"application/octet-stream",
"*/*"
]
This sets the Binary Media Types configuration associated with the API I've defined. I've confirmed via the AWS Console that this setting is having the desired effect...I can see these types in the console. I should need just the first item in the list, but I've added the others while I try to figure out the problem here. By adding that wildcard item, I believe that it shouldn't matter what the incoming Content-Type is...all payloads should be being treated as binary.
The other bit of config that I know about that might be important is the "integration contentHandling property". Here is the key bit of AWS docs that seems to explain all this:
I think the case that applies to me here is the one I've highlighted, per what I say above. This says to me that I shouldn't need to do anything else, per the "unspecified" value in the table for "contentHandling. I've tried setting the "contentHandling" argument on the integration record of my Terraform config, like this:
resource aws_api_gateway_integration proxy {
...
passthrough_behavior = "WHEN_NO_MATCH"
content_handling = "CONVERT_TO_BINARY"
}
I first tried only specifying the content_handling value. I've also tried setting that value to "CONVERT_TO_TEXT", hoping to then get base64-encoded data. Neither of these has any effect. I've tried adding the passthrough_behavior value as shown. I've also tried replacing "WHEN_NO_MATCH" with "WHEN_NO_TEMPLATES". Nothing I do changes the behavior. I haven't been able to figure out where these settings would show up in the AWS console. If I knew they were necessary, I'd explore this further. But I don't think I need to set these.
What am I missing? How can I POST a PDF document to my AWS Lambda function through API Gateway and have the payload of the request not be converted in any way? TIA!
NOTE: I am aware of this Q/A: PDF Uploaded via AWS API Gateway getting corrupted. The answer there doesn't apply to me, as I need to avoid having to form-encode the upload. The client code that will eventually be doing the upload is set in stone and sends a POST request with a payload that is just the bytes of the PDF.
Is there a way to get an issues report by querying the SonarQube web API?
With previous versions of SonarQube, I was able to generate an HTML report after each build but this feature looks like it's been deprecated in newer installments.
At the moment, I'm trying this bit of code
curl -u foo:bar https://sonar.example.com/api/issues/search?pageSize=100&componentKeys=my-app&metricKeys=violations,ncloc,line
But it errors with {"errors":[{"msg":"The 'component' parameter is missing"}]
Ideally, what I'm after is to just get the lines of code, the number of bugs, vulnerabilities, and Code smells in each run/scan.
Something like this
But through querying the API after each analysis.
Is this possible, please?
Had the same issue. The problem is that there is something wrong with the CURL command and you need to specify the full url as string using quotes.
Your case would be:
curl -u foo:bar "https://sonar.example.com/api/measures/search?pageSize=100&componentKeys=my-app&metricKeys=ncloc,violations,complexity"
This is an example with measures. Be sure to check the required parameters.
I am using NiFi API to build a custom application. I am unable to find an API that captures the processor history in a format as shown below. Does anyone know if this API exists?
I have tried many of the API's here, but it's not obvious any of these can do the trick.
NiFi REST Api documentation
You can use the GET/processors/{id} to get back this information in the form of a ProcessorEntity. This contains all fields as shown in your screenshot: name, type, status, counters of last 5 minutes...
Inside the ProcessorEntity object, you can find the ProcessorStatus object which contains the name, type, status, 5min snapshot information per node and aggregated etc...
I included some links as examples to these objects from the NiFi python client, but you can also find some examples on the NiFi REST API docs you linked yourself.
I am trying to use Sonar API for getting multiple metrices. When I am trying to get single Metric it is working fine but when giving a comma separated list it is returning blank response. E.g.
https://localhost/sonar/api/resources?format=json&metrics=tests [Works fine and returns response]
https://localhost/sonar/api/resources?format=json&metrics=ncloc [Works fine and returns response]
https://tools.publicis.sapient.com/sonar/api/resources?format=json&metrics=ncloc,tests [Returns blank reponse i.e [] json ]
Am I doing it wrong. I have used comma separated list previously and it used to work fine. But now I am not able to figure out what is the issue
My Sonar version is 5.6
api/resources is deprecated. Its metrics parameter does accept comma-separated values (test it in a browser first, and careful with any reverse-proxy that may rewrite URLs), but you're better off using the most modern and up-to-date API: api/measures (documentation embedded in your SonarQube server), which also supports multiple values for metricKeys parameter.
for me api/resources works fine with multiples metrics, i call like that
https://localhost/sonar/api/resources?resource=[keyproject]&metrics=coverage,blocker_violations,critical_violations,sqale_debt_ratio
I have multiple nifi servers that I would like to be able to POST templates to via the REST interface from a script
The "/controller/templates" endpoint appears to be the proper REST endpoint to support POSTing an arbitrary template to my Nifi installation.
The "snippetId" field is what is confusing me, how do I determine "The id of the snippet whose contents will comprise the template"? Does anyone have an example of how I can upload a template "test.xml" to my server without having to use the UI?
The API has moved in 1.0 to:
POST /process-groups/{id}/templates/upload
Example, using Python's requests library:
res = requests.post( "{hostname}/nifi-api/process-groups/{destination_process_group}/templates/upload".format( **args ),
files={"template": open( file_path, 'rb')} )
The provided documentation is somewhat confusing, and the solution I worked out was derived from the nifi api deploy groovy script at https://github.com/aperepel/nifi-api-deploy
Ultimately, to POST a template directly, you can use the following in Python requests
requests.post("%s/nifi-api/controller/templates"%(url,), files={"template":open(filename, 'rb')})
Where filename is the filename of your template and url is the path to your nifi instance. I haven't figured it out in curl directly but this should hopefully get folks with a similar question started!
Edit:
Note that you also can't upload a template with the same name as an existing template. Make sure to delete your existing template before attempting to re-upload. Using the untangle library to parse the XML of the template, the following script works just fine:
import untangle, sys, requests
def deploy_template(filename, url):
p = untangle.parse(filename)
new_template_name=p.template.name.cdata
r=requests.get("%s/nifi-api/controller/templates"%(url,), headers={"Accept":"application/json"})
for each in r.json()["templates"]:
if each["name"]==new_template_name:
requests.delete(each["uri"])
requests.post("%s/nifi-api/controller/templates"%(url,), files={"template":open(filename, 'rb')})
if __name__=="__main__":
deploy_template(sys.argv[1], sys.argv[2])
If you want to POST a template to NiFi via cURL you can use the following command:
curl -iv -F template=#my_nifi_template.xml -X POST http://nifi-host:nifi-port/nifi-api/controller/templates
This will add the template to the NiFi instance with the same name that the template was given when it was generated.
And the -iv is optional - It's just there for debugging purposes.
You can use Nifi Api to upload a template, to do that follow these two steps:
1. Get a token from Nifi Api:
token=$(curl -k -X POST --negotiate -u : https://nifi_hostname:port/nifi-api/access/kerberos)
2. Upload the template file using the token:
curl -k -F template=#template_file.xml -X POST https://nifi_hostname:port/nifi-api/process-groups/Process_group_id/templates/upload -H "Authorization: Bearer $token"
The documentation can be confusing because that endpoint is overloaded and the documentation tool only generates doc for one of them (see NIFI-1113). There is an email thread that addresses the import of a template using curl, so between the above answer and the email thread, hopefully you can find the approach that works for you.
I've implemented a full Python client for doing this in NiPyApi
Key functions for templates are:
[
"list_all_templates", "get_template_by_name", "deploy_template",
"upload_template", "create_pg_snippet", "create_template",
"delete_template", "export_template", 'get_template'
]
The client supports NiFi-1.1.2 - 1.7.1 currently, and NiFi-Registry (which is a lot better than templates for flow deployment)