I am attempting to deploy a workflow using the https://registry.terraform.io/providers/hashicorp/google/latest/docs/resources/workflows_workflow terraform resource and its failing with error:
Error: Error creating Workflow: googleapi: Error 400: request contains errors
Details:
[
{
"#type": "type.googleapis.com/google.rpc.BadRequest",
"fieldViolations": [
{
"description": "The referenced service account is not user-managed, please verify the correctness of the service account name",
"field": "workflow.service_account"
}
]
}
]
I can see from running terraform plan that this is the definition of my workflow:
+ resource "google_workflows_workflow" "my_first_workflow" {
+ create_time = (known after apply)
+ description = "Magic"
+ id = (known after apply)
+ name = "myworkflow"
+ name_prefix = (known after apply)
+ project = "myproject"
+ region = "europe-west4"
+ revision_id = (known after apply)
+ service_account = "projects/myproject/serviceAccounts/service-account"
+ source_contents = <<-EOT
- postCallBigqueryStoredProcedure:
call: http.post
args:
url: https://bigquery.googleapis.com/bigquery/v2/projects/myproject/jobs
body: {
"configuration": {
"query": {
"query": "call mydataset.mystoredprocedure()"
}
}
}
EOT
+ state = (known after apply)
+ update_time = (known after apply)
}
The error messages is complaining about the service account however I'm certain that the service account named here: projects/myproject/serviceAccounts/service-account is valid and exists so I'm clueless as to why I'm getting this error. Googling the error message hasn't turned up anything useful.
Does anyone know what might be the problem?
You mentioned that the service account is valid and it exists. When you are referencing it, are you including the full account name including the details after the '#' ie. 7**********-compute#developer.gserviceaccount.com?
I was able to replicate this behaviour by using either an incorrect name or a service account name without the full email address.
You must use the complete email address of your service account. Here's a sample of a correct format. I'm currently using Terraform v0.14.7:
service_account = "projects/project_id/serviceAccounts/7**********-compute#developer.gserviceaccount.com"
I'm looking for a way how to integrate a notification for Ansible Tower / AWX to Rocket.Chat? I can't find a suitable script for Rocket.Chat integration.
First go in Rocket.Chat in Administration > Integration and then create a new incoming webhook. Configure it as wanted (name, bot, channel, etc.) enable scripting and add the following script:
class Script {
process_incoming_request({ request }) {
// UNCOMMENT THE BELOW LINE TO DEBUG IF NEEDED.
// console.log(request.content);
let body = request.content.body;
if (!body) {
let id = request.content.id;
let name = request.content.name;
let url = request.content.url;
let status = request.content.status;
let type = request.content.friendly_name;
let project = request.content.project;
let playbook = request.content.playbook;
let hosts = request.content.hosts;
let created_by = request.content.created_by;
let started = request.content.started;
let finished = request.content.finished;
let traceback = request.content.traceback;
let inventory = request.content.inventory;
let credential = request.content.credential;
let limit = request.content.limit;
let extra_vars = request.content.extra_vars;
let message = "";
message += "AWX "+type+" "+name+" ("+id+") ";
message += "on project _"+project+"_ ";
message += "running playbook _"+playbook+"_ ";
message += "has status *"+status+"*.";
message += "\n";
message += type+" was created by _"+created_by+"_ for inventory _"+inventory+"_ ";
if (limit !== "") {
message += "with limit _"+limit+"_ ";
}
message += " and using the _"+credential+"_ credentials.\n";
if (Object.keys(hosts).length != 0) {
message += "Hosts: "+Object.keys(hosts).length+" (ok/changed/skipped/failures)\n";
for (let [name, host] of Object.entries(hosts)) {
message += "- "+name+" ("+host.ok+"/"+host.changed+"/"+host.skipped+"/"+host.failures+")";
if (host.failed === false) {
message += " is *ok*\n";
} else {
message += " has *failed*\n";
}
}
}
return {
content: {
"text": "AWX notification *"+status+"* on "+type+" "+name+" ("+id+")",
"attachments": [
{
"title": type+": "+name+"",
"title_link": url,
"text": message,
"color": "#764FA5"
}
]
}
};
} else {
return {
content: {
text: "AWX notification: " + request.content.body
}
};
}
}
}
Save and activate the webhook. Now you get a Webhook URL from Rocket.Chat. Copy that URL.
Go to your AWX instance and create a new Notification of type Webhook and paste the Webhook URL from Rocket.Chat. You can test the notifcation within AWX.
The script does not print extra vars, because they could contain passwords etc. But you'll see failed hosts and some more information about the job.
AWX/Tower has the ability to send notifications to rocket.chat without any custom scripts.
In Tower go to Notifications and add a new one with type 'Rocket.Chat' then set the Target URL to be the URL of a blank incoming webhook in Rocket.Chat (Make sure it's enabled at the top).
(Note: Be careful of the URL Rocket.Chat gives you for the integration, mine didn't give me a URL with the correct port of 3000 within the URL so it failed at first)
Heres what the notifcations read as:
Bot -
3:13 PM
Tower Notification Test 1 https://ruupansi01
Bot -
3:15 PM
Project Update #2 'Test Project' succeeded: https://tower/#/jobs/project/1
I am using the adwords api to generate reports.
Please bear with me as I am not too familiar with the same.
I am using version v201409 of the api.
I get the report columns for KEYWORD_PERFORMANCE_REPORT using getReportFields.
I then try to download the report using a subset of those columns.
For KEYWORD_PERFORMANCE_REPORT I get the error:
Cannot select a combination of Device and
AssistClicks,AssistClicksOverLastClicks,AssistImpressions,AssistImpressionsOverLastClicks,AveragePageviews,AverageTimeOnSite,BounceRate,Bounces,ClickAssistedConversionValue,ClickAssistedConversionValueLong,ClickAssistedConversionValueNonMoney,ClickAssistedConversions,ClickAssistedConversionsOverLastClickConversions,ImpressionAssistedConversionValue,ImpressionAssistedConversionValueLong,ImpressionAssistedConversionValueNonMoney,ImpressionAssistedConversions,ImpressionAssistedConversionsOverLastClickConversions,LastClickConversions,LastClicks,NewVisitors,Pageviews,PercentNewVisitors,VisitDuration,Visits,
Type: ReportDefinitionError.INVALID_FIELD_NAME_FOR_REPORT.
The question is: How do I find out a valid set of combinations of columns without going through a trial and error process.. Is there any documentation which will help me with the same.
I looked at the columns for KEYWORD_PERFORMANCE_REPORT in http://developers.guge.io/adwords/api/docs/appendix/reports and exclude the colums which the api said were "not compatible". Got a similar error.
Thanks
N.B> If I try this code with the columns provided in the online example it works and downloads the report as expected.
The code is:
`
String[] columnNames = {
"ConversionRateManyPerClickSignificance",
"ConversionRateSignificance",
"ViewThroughConversionsSignificance",
"AccountCurrencyCode",
"AccountDescriptiveName",
"AccountTimeZoneId",
"AdGroupId",
"AdGroupName",
"AdGroupStatus",
"AssistImpressions",
"AssistImpressionsOverLastClicks",
"AverageCpc",
"AverageCpm",
"AveragePageviews",
"AveragePosition",
"AverageTimeOnSite",
"BiddingStrategyId",
"BiddingStrategyName",
"BiddingStrategyType",
"CampaignId",
"CampaignName",
"CampaignStatus",
"ClickAssistedConversionsOverLastClickConversions",
"ClickAssistedConversionValue",
"Clicks",
"ClickSignificance",
"ClickType",
"ConversionManyPerClickSignificance",
"ConversionRate",
"ConversionRateManyPerClick",
"Conversions",
"ConversionSignificance",
"ConversionsManyPerClick",
"ConversionTypeName",
"ConversionValue",
"Cost",
"CostPerConversion",
"CostPerConversionManyPerClick",
"CostPerConversionManyPerClickSignificance",
"CostPerConversionSignificance",
"CostSignificance",
"CpcBid",
"CpcBidSource",
"CpmBid",
"CpmSignificance",
"CriteriaDestinationUrl",
"Ctr",
"CtrSignificance",
"CustomerDescriptiveName",
"CvrSignificance",
"Date",
"DayOfWeek",
"Device",
"ExternalCustomerId",
"FinalAppUrls",
"FinalMobileUrls",
"FinalUrls",
"FirstPageCpc",
"Id",
"ImpressionAssistedConversions",
"ImpressionAssistedConversionsOverLastClickConversions",
"ImpressionAssistedConversionValue",
"Impressions",
"ImpressionSignificance",
"IsNegative",
"KeywordMatchType",
"LabelIds",
"Labels",
"Month",
"MonthOfYear",
"PlacementUrl",
"PositionSignificance",
"PrimaryCompanyName",
"QualityScore",
"Quarter",
"SearchExactMatchImpressionShare",
"SearchImpressionShare",
"SearchRankLostImpressionShare",
"Slot",
"TrackingUrlTemplate",
"UrlCustomParameters",
"ValuePerConversion",
"ValuePerConversionManyPerClick",
"ViewThroughConversions",
"Week",
"Year"
};
public static void downloadConsolidatedReportFile(String[] columnNames, final ReportDefinitionDateRangeType forDateRange, final ReportDefinitionReportType reportDefinitionReportType, final String to) throws Exception {
com.google.api.ads.adwords.lib.jaxb.v201409.Selector selector = new com.google.api.ads.adwords.lib.jaxb.v201409.Selector();
selector.getFields().addAll(Lists.newArrayList(columnNames));
ReportDefinition reportDefinition = new ReportDefinition();
reportDefinition.setReportName("Report " + reportDefinitionReportType.value() + " for dateRange " + forDateRange.value());
reportDefinition.setDateRangeType(forDateRange);
reportDefinition.setReportType(reportDefinitionReportType);
reportDefinition.setDownloadFormat(DownloadFormat.CSV);
ReportingConfiguration reportingConfiguration = new ReportingConfiguration.Builder()
.skipReportHeader(true)
.skipReportSummary(true)
.build();
session.setReportingConfiguration(reportingConfiguration);
reportDefinition.setSelector(selector);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(to)));
String mccId = session.getClientCustomerId(); //The id from ads.properties file
Collection<Client> clientIds = getClientAccountIds(mccId);
try {
for (Client cl : clientIds) {
BufferedReader reader = null;
String customerId = cl.id;
String name = cl.name;
session.setClientCustomerId(cl.id);
try {
ReportDownloadResponse response =
new ReportDownloader(session).downloadReport(reportDefinition);
if (response == null || response.getHttpStatus() != 200) {
handleError(response);
}
BufferedInputStream bs = new BufferedInputStream(response.getInputStream());
reader = new BufferedReader(new InputStreamReader(bs));
String line = null;
log.info("getting " + reportDefinition.getReportType().value() + " for " + customerId+" "+name);
reader.readLine(); //Skip the first line of column names
while ((line = reader.readLine()) != null) {
bw.write(line + "\n");
}
} catch (DetailedReportDownloadResponseException e) {
log.error("An error was thrown downloading report for Customer id: " + customerId+" "+name, e);
//We have to do this as we have to filter out the mcc id. An exception is thrown by MCC id
if (e.getType().equals("ReportDefinitionError." + ReportDefinitionErrorReason.CUSTOMER_SERVING_TYPE_REPORT_MISMATCH.getValue())) {
continue;
} else {
throw e;
}
} catch (Exception e) {
log.error("An error was thrown downloading report for Customer id: " + customerId+" "+name, e);
throw e;
} finally {
if (reader != null) {
reader.close();
}
}
}
} finally {
if (bw != null) {
bw.flush();
bw.close();
}
}
}
`
None of the columns you mentioned in the comment below are used.
Check the following documentation.
https://developers.google.com/adwords/api/docs/appendix/reports/keywords-performance-report#activeviewcpm
For some fields "Not compatible with the following fields" option is provided. Click on that option to check the combinations that are not compatible
The following columns are not selectable in the Keywords Performance Report in v201409:
AssistClicksOverLastClicks
Bounces
ClickAssistedConversionValueLong
ClickAssistedConversionValueNonMoney
ImpressionAssistedConversionValueLong
ImpressionAssistedConversionValueNonMoney
LastClickConversions
LastClicks
NewVisitors
Pageviews
VisitDuration
Visits
Suggest you try removing them and trying again. Failing that, post some code so we can see how you are making the call.
For future reference the AdWords Ad Hoc Reporting docs provide a downloadable CSV file for each of the report types listing the allowed metrics, attributes and segments.
I got an same issue with you. And from my understandings, there's no API to identify which column combinations are valid in adwords so far.
So I check whether the combination is valid or not before download the file by requesting real AWQL.
If the dummy AWQL returns error such as Adwords: Reporting Error: HTTP code: 400, error type: 'ReportDefinitionError.INVALID_FIELD_NAME_FOR_REPORT', trigger: 'Cannot select a combination of ActiveViewCpm and ConversionCategoryName,ConversionTrackerId,ConversionTypeName', field path: 'ActiveViewCpm', then I modify the combination of columns by using the error details through trial and error.
I don't know if this is the best way...
I am attempting to retrieve the HttpStatusCode from every UploadAsync method call. I need the status code as to properly perform an exponential back-off algorithm to retry a failed upload, display an error message to the user when not retrying the upload and to report success of the upload. I do not care how it is received, so long as it is clean and not being parsed from the Exception.Message (string) property like Tor Jonsson suggested in the link provided below.
To force the "Bad Request Error [400]" I simply provided an invalid userkey (email) in the constructor for MailResource.InsertMediaUpload.
e.g. MailResource.InsertMediaUpload(mailItem, "invalidEmail#domain.com", stream, "message/rfc822")
Problem
1) GoogleApiException.HttpStatusCode is always 0 (unavailable). Even when Exception.Message appears to contain a status code in brackets. e.g. [400]
2) Cannot find GoogleApiRequestException.
Questions
1) What is the best way to perform the exponential back-off algorithm???
2) Is this the expected behaviour for this property in this case?
3) Does GoogleApiRequestException still exist, if so where?
Side Note:
I also noticed that the GoogleApiRequestException class is no longer in the same file as GoogleApiException class. Has it been moved to another namespace or deleted? Because I would like to attempt to catch a GoogleApiRequestException object and grab its RequestError object.
I added links to the two diffs for what I mean:
Before: http://code.google.com/p/google-api-dotnet-client/source/browse/Src/GoogleApis/GoogleApiException.cs?r=a8e27790f8769c1d6aaae030bb46c79daa7cdbad
After: http://code.google.com/p/google-api-dotnet-client/source/browse/Src/GoogleApis/GoogleApiException.cs?r=d6f06e92d90b635c179013e2c287b42b82909c09
Sources
I'm using the latest binaries from NuGet (1.6.0.8-beta)
The only question I found related to my problem: (Can only post two links... heres the raw)
stackoverflow.com/questions/18985306/httpstatuscode-not-set-in-exceptions-when-using-google-net-apis
Code: (Using a custom logger to write to debugview)
public int Index; // Used to Id the process
private void TryUpload(MailResource.InsertMediaUpload upload, out IUploadProgress uploadProgress, out bool retryUpload)
{
uploadProgress = null;
retryUpload = false;
CancellationToken token;
try
{
uploadProgress = upload.UploadAsync(token).Result;
if (uploadProgress.Exception != null)
{
_logger.WriteTrace("EXCEPTION!!! Type: {0}", uploadProgress.Exception.GetType().ToString()); // Remove:
// *) Handle all of the various exceptions
if (uploadProgress.Exception is JsonReaderException)
{
JsonReaderException jreEx = uploadProgress.Exception as JsonReaderException;
_logger.WriteTrace("JsonReaderException-> Message: {0}", jreEx.Message);
}
if (uploadProgress.Exception is TokenResponseException)
{
TokenErrorResponse trEx = uploadProgress as TokenErrorResponse;
_logger.WriteTrace("TokenErrorResponse-> Message: {0}", trEx.Error);
}
if (uploadProgress.Exception is HttpRequestValidationException)
{
HttpRequestValidationException hrvEx = uploadProgress.Exception as HttpRequestValidationException;
_logger.WriteTrace("HttpRequestValidationException-> Message: {0}", hrvEx.Message);
_logger.WriteTrace("HttpRequestValidationException-> Status Code: {0}", hrvEx.GetHttpCode());
}
if (uploadProgress.Exception is GoogleApiException)
{
GoogleApiException gApiEx = uploadProgress.Exception as GoogleApiException;
_logger.WriteTrace("GoogleApiException-> Message: {0}", gApiEx.Message);
_logger.WriteTrace("GoogleApiException-> Status Code: {0}", gApiEx.HttpStatusCode);
}
}
}
catch (Exception ex)
{
_logger.WriteTrace(ex, "An exception occured while uploading...");
}
finally
{
if (uploadProgress != null)
_logger.WriteTrace("Upload Completed... Status: {0} Exception?: {1}",
uploadProgress.Status,
(uploadProgress.Exception == null) ? "None" : uploadProgress.Exception.ToString());
else
_logger.WriteTrace("Upload Aborted... Exited without returning a status!");
}
}
Output Snippet
[5224] (T101) VSLLC: EXCEPTION!!! Type: Google.GoogleApiException
[5224] (T101) VSLLC: GoogleApiException-> Message: Google.Apis.Requests.RequestError
[5224] Bad Request [400]
[5224] Errors [
[5224] Message[Bad Request] Location[ - ] Reason[badRequest] Domain[global]
[5224] ]
[5224] (T101) VSLLC: GoogleApiException-> Status Code: 0
[5224] (T101) VSLLC: Upload Completed... Status: Failed Exception?: The service admin has thrown an exception: Google.GoogleApiException: Google.Apis.Requests.RequestError
[5224] Bad Request [400]
[5224] Errors [
[5224] Message[Bad Request] Location[ - ] Reason[badRequest] Domain[global]
[5224] ]
[5224]
[5224] at Microsoft.Runtime.CompilerServices.TaskAwaiter.ThrowForNonSuccess(Task task)
[5224] at Microsoft.Runtime.CompilerServices.TaskAwaiter.HandleNonSuccess(Task task)
[5224] at Microsoft.Runtime.CompilerServices.TaskAwaiter.ValidateEnd(Task task)
[5224] at Google.Apis.Upload.ResumableUpload`1.d__0.MoveNext() in c:\code\google.com\google-api-dotnet-client\default\Tools\Google.Apis.Release\bin\Debug\output\default\Src\GoogleApis\Apis[Media]\Upload\ResumableUpload.cs:line 373
Sorry for the extensive post! Thanks for your time!
The library already supports exponential back-off for 503 responses. In case of 400 (bad request) you should not retry, because you will get the same response over and over again.
Take a look in the service initializer parameter DefaultExponentialBackOffPolicy
You can also take a look in our ExponentialBackOff implementation. BackOffHandler wraps the logic and implements unsuccessful response handler and exception handler.
GoogleApiRequest doesn't exists anymore.
It looks like we are not setting the status code properly, as you can find here. I open a new issue in our issue tracker, available here - https://code.google.com/p/google-api-dotnet-client/issues/detail?id=425. Feel free to add more content to it.
I'm working on an XML message process program using dom4j, and encounter a problem with XML node attribute parsing. The business requirement is: parse and validate input XML, if a field or an attribute is invalid, return an XML which structure denotes the error. And I'v snipped a test program below:
import org.dom4j.Document;
import org.dom4j.DocumentHelper;
import org.dom4j.Element;
import org.dom4j.Node;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class XPathStudy {
private final static Logger log = LoggerFactory.getLogger(XPathStudy.class);
private static String xmlInput;
private static void initXmlInput() {
xmlInput = "<RootTag> <BizRec FLAG=\"5\">";
xmlInput += " <FieldOne>11111</FieldOne>";
xmlInput += " <FieldTwo></FieldTwo>";
xmlInput += " <FieldThree>33333</FieldThree>";
xmlInput += " </BizRec> </RootTag>";
}
private static Document makeErrRspsDoc(String xpath, String errCode, String errDesc) {
Document errDoc = DocumentHelper.createDocument();
Element errElem = DocumentHelper.makeElement(errDoc, xpath);
errElem.addElement("ErrCode").addText(errCode);
errElem.addElement("ErrDesc").addText(errDesc);
return errDoc;
}
public static void main(String[] args) throws Exception {
initXmlInput();
log.info("xmlInput = " + xmlInput);
Document doc = DocumentHelper.parseText(xmlInput);
log.info("xmlInput parsed done");
String xpath = "*//FieldTwo";
Node node = doc.getRootElement().selectSingleNode(xpath);
if (node == null) {
log.warn("node [" + xpath + "] not found");
System.exit(1);
}
log.info("node [" + node.getPath() + "] located");
if (node.getText().trim().isEmpty()) {
Document errDoc = makeErrRspsDoc(node.getPath(), "1201", "FieldTwo can not be empty");
log.warn("errDoc: " + errDoc.asXML());
} else {
log.info("FieldTwo validation ok");
}
xpath = "*//#FLAG";
node = doc.getRootElement().selectSingleNode(xpath);
if (node == null) {
log.warn("node [" + xpath + "] not found");
System.exit(1);
}
log.info("node [" + node.getPath() + "] located");
int flagVal = Integer.parseInt(node.getText().trim());
if (flagVal == 1 || flagVal == 2) {
log.info("FLAG " + flagVal + " is ok");
} else {
Document errDoc = makeErrRspsDoc(node.getPath(), "1001", "FLAG attr should be 1 or 2");
log.warn("errDoc: " + errDoc.asXML());
}
}
}
And run it, the log info below:
15:01:08.143 [main] INFO XPathStudy - xmlInput = <RootTag> <BizRec FLAG="5"> <FieldOne>11111</FieldOne> <FieldTwo></FieldTwo> <FieldThree>33333</FieldThree> </BizRec> </RootTag>
15:01:08.203 [main] INFO XPathStudy - xmlInput parsed done
15:01:08.255 [main] INFO XPathStudy - node [/RootTag/BizRec/FieldTwo] located
15:01:08.259 [main] WARN XPathStudy - errDoc: <?xml version="1.0" encoding="UTF-8"?>
<RootTag><BizRec><FieldTwo><ErrCode>1201</ErrCode><ErrDesc>FieldTwo can not be empty</ErrDesc></FieldTwo></BizRec></RootTag>
15:01:08.260 [main] INFO XPathStudy - node [/RootTag/BizRec/#FLAG] located
15:01:08.260 [main] WARN XPathStudy - errDoc: <?xml version="1.0" encoding="UTF-8"?>
<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attr should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>
Everything seems ok, and the errDoc will be logged to an Oracle10g table (a VARCHAR2(1000) field), the following SQL is ok:
select extractvalue(xmltype('<RootTag><BizRec><FieldTwo><ErrCode>1201</ErrCode><ErrDesc>FieldTwo can not be empty</ErrDesc></FieldTwo></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual;
1201
But this SQL will fail:
select extractvalue(xmltype('<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attribute should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual;
Error starting at line 1 in command:
select extractvalue(xmltype('<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attribute should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual
Error report:
SQL Error: ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00231: invalid character 64 ('#') found in a Name or Nmtoken
Error at line 1
ORA-06512: at "SYS.XMLTYPE", line 301
ORA-06512: at line 1
31011. 00000 - "XML parsing failed"
*Cause: XML parser returned an error while trying to parse the document.
*Action: Check if the document to be parsed is valid.
And Oracle's doc explains that XML's node can not contain special chars. So, my question is: howto change my Java code to solve the problem of the 'FLAG' attribute's error response?
<#FLAG> is not a legal XML element name, since the # character cannot be the start of a name. DOM4J is not known for being best in class when it comes to enforcing well formed and valid XML documents.
To communicate the failing node as an XPath expression, consider storing it in an attribute or as a text node.