I'm running into 2 separate issues using the Grafeas golang v1beta1 API.
What I'm trying to do
Call ListOccurrencesRequest() with a Filter to get a list of occurrences for deletion
Call DeleteOccurrence() on each occurrence from above list to delete it
Issue #1
I'm trying to set the Filter field using this GCP reference grafeas golang code as a guide.
filterStr := fmt.Sprintf(`kind=%q`, grafeas_common_proto.NoteKind_BUILD.String())
listReq := &grafeas_proto.ListOccurrencesRequest{
Parent: BuildProject,
Filter: filterStr,
PageSize: 100,
}
listOccResp, err := r.grafeasCommon.ListOccurrences(ctx, listReq)
for {
if err != nil {
log.Error("failed to iterate over occurrences", zap.NamedError("error", err))
return nil, err
}
...
But it looks like my filterStr is invalid, here's the error:
filterStr {"filterStr": "kind=\"BUILD\""}
failed to iterate over occurrences {"error": "rpc error: code = Internal desc = error while parsing filter expression: 4 errors occurred:\n\t* error parsing filter\n\t* Syntax error: token recognition error at: '=\"' (1:4)\n\t* Syntax error: token recognition error at: '\"' (1:11)\n\t* Syntax error: extraneous input 'BUILD' expecting <EOF> (1:6)\n\n"}
It looks like the \ escape character is causing trouble but I've tried it without it and get another flavor of same type of error.
Issue #2
When I call DeleteOccurrence(), I can see that the occurrence is in fact deleted from Grafeas by checking:
curl http://localhost:8080/v1beta1/projects/broker_builds/occurrences
But DeleteOccurrence() always sets the err
Code:
for _, o := range occToDelete {
log.Info("occToDelete", zap.String("occurrence", o))
_, err := r.grafeasCommon.DeleteOccurrence(ctx, &grafeas_proto.DeleteOccurrenceRequest{
Name: o,
})
if err != nil {
log.Error("failed to delete occurrence", zap.String("occurrence", o), zap.NamedError("error", err))
}
}
Error:
failed to delete occurrence {"occurrence": "projects/broker_builds/occurrences/f61a4c57-a3d3-44a9-86ee-5d58cb6c6052", "error": "rpc error: code = Internal desc = grpc: error while marshaling: proto: Marshal called with nil"}
I don't understand what the error is referring to.
This question was cross-posted on Grafeas message board.
Appreciate any help. Thanks.
Can you shed some details around the storage engine used, and the filtering implementations details?
Issue 1. filtering is not implemented in any of the storage engines in gitHub.com/grafeas/grafeas.
Issue 2. it depends what store you use, memstore/embededstore do not seem to be producing any errors similar to what you mentioned... if using postgresql store, are you trying to delete an occurrence twice?
Solution for Issue #1
I'm using grafeas-elasticsearch as the storage backend. It uses a different filter string format than the examples I had looked at in my original post.
For example, instead of = -> ==, AND -> &&, etc.
More examples can be seen here:
https://github.com/rode/grafeas-elasticsearch/blob/main/test/v1beta1/occurrence_test.go#L226
Solution for Issue #2
Known issue with grafeas
https://github.com/grafeas/grafeas/pull/456
https://github.com/grafeas/grafeas/pull/468
Unfortunately the latest tagged release of grafeas v0.1.6 does not include these fixes yet. So will need to pick them up on the next release.
Thanks to #Ovidiu Ghinet, that was a good tip
Related
My department uses the olivere elastic v7.0.26 as ElasticSearch Client,sometimes our front-end system returns some errormsg such as "Error 400 (Bad Request): all shards failed [type=search_phase_execution_exception]" but doesn't have any other useful msg to help troubleshooting,so I've searched GitHub about parsing errors to *elastic.Error and codes are just like this:
if err != nil {
log.WithFields(ctx, log.Fields{}).WithError(err).Warn("list query es error")
if ex, ok := err.(*elastic.Error); ok {
log.WithFields(ctx, log.Fields{"query": query, "status": ex.Status, "detail": ex.Details}).WithError(ex).Warnf("list query es err")
}
return res, err
}
But the strange thing happened is "list query es error" was printed on our log system but the next log "list query es err" didn't,i've used deep-paging (large from+size) to check the log,it works and print the elastic error such as root cause so i can get the max_result_window tips..But it returns a unparsing error which is returned as the search_phase_execution_exception error,the company code is not allowed to paste to the open-sourcing website,and I just wanna know what error can make elasticsearch returns the search_phase_execution_exception and the errcode 400?Really appreciate your help!
same to above description,but the point I need to supple is that the problem is not always happened,so I can exclude the index mapping field type error such as "text/keyword",just wanna get the whole error/exceptions elasticsearch can return,but I cannot find the relative documentions on elastic guide,that's ok if u can provide the guide/doc about this
uhh it seems like a joke XD,I've never learned about some troubleshooting technique,but after I asked this question,I realized that I can get the ElasticSearch server log,so I asked our company's SRE and got the log,it's a error about terms query exceed maximum,and the reason caused is our RPC interface return a exceptional result about nil data redis set key,it will be a big-key problem,but we found this and fix it.
I want to patch an existing secret resource within Kubernetes. The object is called centos-secretstorage within the default namespace. I want to add a simple label of test: empty. However, this fails when the secret object centos-secretstorage exists, but it doesn't have any existing labels. If I manually label the secret with something else beforehand via kubectl label centos-secretstorage hello=world, and rerun my golang code. It is able to add the test: empty label successfully.
However, I want to have this be able to add a label regardless if existing labels exist or not.
type secret struct {
namespace string
name string
}
func main() {
k8sClient := k8CientInit()
vaultSecret := secret{
namespace: "default",
name: "centos-secretstorage",
}
vaultSecret.patchSecret(k8sClient)
}
type patchStringValue struct {
Op string `json:"op"`
Path string `json:"path"`
Value string `json:"value"`
}
func (p *secret) patchSecret(k8sClient *kubernetes.Clientset) {
emptyPayload := []patchStringValue{{
Op: "add",
Path: "/metadata/labels/test",
Value: "empty",
}}
emptyPayloadBytes, _ := json.Marshal(emptyPayload)
fmt.Println(string(emptyPayloadBytes))
emptyres, emptyerr := k8sClient.CoreV1().Secrets(p.namespace).Patch(p.name, types.JSONPatchType, emptyPayloadBytes)
if emptyerr != nil {
log.Fatal(emptyerr.Error())
}
fmt.Println(emptyres.Labels)
}
Error: the server rejected our request due to an error in our request
The problem is that the add operation in the JSON patch strategy requires the path to point to an existing map, while the object you are patching does not have this map at all. This is why when any label exists, the patch succeeds. We can work around this by using a different patch strategy. I think the merge strategy should work well.
I was able to reproduce this (on a namespace, but the object doesn't matter) using kubectl (which is generally useful when debugging the Kubernetes API):
kubectl patch ns a --type='json' -p='[{"op": "merge", "path": "/metadata/labels/test", "value":"empty"}]' -> fails
kubectl patch ns a --type='merge' -p='{"metadata": {"labels": {"test": "empty"}}}' -> succeeds
Using Golang client-go it would look something like this (didn't actually compile / run this):
payload := `{"metadata": {"labels": {"test": "empty"}}}`
emptyres, emptyerr := k8sClient.CoreV1().Secrets(p.namespace).Patch(p.name, types.MergePatchType, []byte(payload))
You can make the creation of the payload JSON nicer using structs, as you did with patchStringValue.
More info on patch strategies can be found here:
https://kubernetes.io/docs/tasks/manage-kubernetes-objects/update-api-object-kubectl-patch/
https://erosb.github.io/post/json-patch-vs-merge-patch/
I am not sure of the behaviour of error messages in Substrate runtimes in relation to Substrate UI, and if they inherently cause a transaction failure or not.
For example in the democracy SRML I see the following line:
ensure!(!<Cancellations<T>>::exists(h), "cannot cancel the same proposal twice");
Which presumably is a macro that ensures that the transaction fails or stops processing if the h (the proposal hash) already exists. There is clearly a message associated with this error.
Am I right to assume that the transaction fails (without the rest of the SRML code being executed) when this test fails?
If so, how do I detect the failure in Substrate UI, and possibly see the message itself?
If not, then presumably some further code is necessary in the runtime module which explicitly creates an error. I have seen Err() - but not in conjunction with ensure!()
As https://github.com/paritytech/substrate/pull/3433 is merged, the ExtrinsicFailed event now includes a DispatchError, which will provide additional error code.
There isn't much documentations available so I will just use system module as example.
First you need to decl_error, note the error variants can only be simple C like enum
https://github.com/paritytech/substrate/blob/5420de3face1349a97eb954ae71c5b0b940c31de/srml/system/src/lib.rs#L334
decl_error! {
/// Error for the System module
pub enum Error {
BadSignature,
BlockFull,
RequireSignedOrigin,
RequireRootOrigin,
RequireNoOrigin,
}
}
Then you need to associate the declared Error type
https://github.com/paritytech/substrate/blob/5420de3face1349a97eb954ae71c5b0b940c31de/srml/system/src/lib.rs#L253
decl_module! {
pub struct Module<T: Trait> for enum Call where origin: T::Origin {
type Error = Error;
Then you can just return your Error in dispatch calls when things failed
https://github.com/paritytech/substrate/blob/5420de3face1349a97eb954ae71c5b0b940c31de/srml/system/src/lib.rs#L543
pub fn ensure_root<OuterOrigin, AccountId>(o: OuterOrigin) -> Result<(), Error>
where OuterOrigin: Into<Result<RawOrigin<AccountId>, OuterOrigin>>
{
match o.into() {
Ok(RawOrigin::Root) => Ok(()),
_ => Err(Error::RequireRootOrigin),
}
}
Right now you will only able to see a two numbers, module index and error code from JS side. Later there could be support to include the error details in metadata so that frontend will be able to provide a better response.
Related issue:
https://github.com/paritytech/substrate/issues/2954
The ensure! macro is expaneded as:
#[macro_export]
macro_rules! fail {
( $y:expr ) => {{
return Err($y);
}}
}
#[macro_export]
macro_rules! ensure {
( $x:expr, $y:expr ) => {{
if !$x {
$crate::fail!($y);
}
}}
}
So basically, it's just a quicker way to return Err. At 1.0, the error msg will only be printed to stdout(at least what I've tested so far), doesn't know if it'll be included in blockchain in the future(so can be viewed in substrate ui)..
How to solve this error in XQuery. I want the data to be distinct with out duplication in XML result. I tried to add distinct-values in front of the doc in for statement, but this error was depicted.
Engine name: Saxon-PE XQuery 9.5.1.3
Severity: fatal
Description: XPTY0019: Required item type of first operand of '/' is node(); supplied value has item type xs:anyAtomicType
Start location: 23:0
URL: http://www.w3.org/TR/xpath20/#ERRXPTY0019
This is code :
for $sv1 in distinct-values(doc('tc.xml')//term/year)
let $sv2 := doc('tc.xml')//term[year= $sv1]
let $sv3 := doc('tc.xml')//student[idStudent= $sv1/idStudent](:HERE IS THE ERROR LINE:)
let $sv4 := doc('tc.xml')//program[idStudent= $sv3/idStudent]
return
<Statistics>
{$sv1 }
<Count_Student>{count($sv2)}</Count_Student>
<a50_60>{count(doc('tc.xml')/mydb//program[doc('tc.xml')/mydb//term/year =$sv1][avg>= 50 and avg < 60])}</a50_60>
</Statistics>
thank you in advance.
distinct-values() will atomize your input, this means that $sv1/idStudent won't work because $sv1 is not an element. Instead of using $sv1 on the line that give an error I think you should be using $sv2.
I am trying to call cudppSort to sort a set of keys/values. I'm using the following code to set up the sort algorithm:
CUDPPConfiguration config;
config.op = CUDPP_ADD;
config.datatype = CUDPP_UINT;
config.algorithm = CUDPP_SORT_RADIX;
config.options = CUDPP_OPTION_KEY_VALUE_PAIRS | CUDPP_OPTION_FORWARD | CUDPP_OPTION_EXCLUSIVE;
CUDPPHandle planHandle;
CUDPPResult result = cudppPlan(&planHandle, config, number_points, 1, 0);
if (CUDPP_SUCCESS != result) {
printf("ERROR creating CUDPPPlan\n");
exit(-1);
}
The program exits, however on the line:
CUDPPResult result = cudppPlan(&planHandle, config, number_points, 1, 0);
and prints to stdout:
Cuda error: allocScanStorage in file 'c:/the/path/to/release1.1/cudpp/src/app/scan_app.cu' in line 279 : invalid configuration argument.
I looked at the line in scan_app.cu. It is,
CUT_CHECK_ERROR("allocScanStorage");
So apparently my configuration has an error that is causing the allocScanStorage to bomb out. There are only two calls to CUDA_SAFE_CALL in the function and I don't see a reason why either has anything to do with the configuration.
What is wrong with my configuration?
So that this doesn't sit around as an unanswered question (I'm not sure if this is the right SO etiquette but it seems like an answered question shouldn't sit around unanswered...), I'm copying the comment I made above here as an answer since it was the solution:
I figured this out (I'm still learning CUDA at the moment.) Because the error checking is asynchronous errors can show up in strange places if you don't check for them from time to time. My code had caused an error before I called cudppPlan but because I didn't check for errors the cudppPlan reported the error as if it was in cudppPlan.