Related
version info:
Server version: Apache Tomcat/9.0.17
Server built: Mar 13 2019 15:55:27 UTC
Server number: 9.0.17.0
OS Name: Windows 8.1
OS Version: 6.3
Architecture: amd64
JVM Version: 1.8.0_40-b25
spring-boot-2.1.4
spring-core-5.1.6
I have a html page like this: localhost/example/37%.html
(sorry I can not start url with http// because the edit page will auto-change it to link and will add 25 after %)
when I access this url,I got an error:HTTP Status 400.
It should be, because the % is a escape char.
so I changed the url to: localhost/example/37%25.html.
this time,I got a new error:
Whitelabel Error Page
This application has no explicit mapping for /error, so you are seeing this as a fallback.
Thu Sep 03 09:25:13 CST 2020
There was an unexpected error (type=Internal Server Error, status=500).
URLDecoder: Illegal hex characters in escape (%) pattern - For input string: ".h"
I checked tomcat console,it outputs error message:
2020-09-03 09:25:12 [http-nio-8080-exec-3] ERROR
o.s.b.w.s.support.ErrorPageFilter - Forwarding to error page from request [/37%.html] due to exception [URLDecoder: Illegal hex characters in escape (%) pattern - For input string: ".h"]
java.lang.IllegalArgumentException: URLDecoder: Illegal hex characters in escape(%) pattern - For input string: ".h"
at java.net.URLDecoder.decode(URLDecoder.java:194)
at org.springframework.web.servlet.resource.PathResourceResolver.isInvalidEncodedPath(PathResourceResolver.java:285)
at org.springframework.web.servlet.resource.PathResourceResolver.isResourceUnderLocation(PathResourceResolver.java:254)
It seems like tomcat(or browser) has been decode %25 to %,but springboot still to decode % again (PathResourceResolver.java)
I don't know whether this is an bug issue or this is a right way that springboot do not allow % in url
I found something interesting.
now I have two webpages, one named 37%.html,other named 37%25.html
Follow the logic of springboot that it will decode % twice, I accessed this url:
localhost/example/37%2525.html
and it will show me the 37%.html, BUT, it show me the 37%25.html.
so,how many times or how to the springboot decode % ???
then I found two class file in springboot:
ResourceHttpRequestHandler.java
PathResourceResolver.java
they all have a function named: isInvalidEncodedPath()
code is here:
ResourceHttpRequestHandler.isInvalidEncodedPath(path)
private boolean isInvalidEncodedPath(String path) {
if(path.contains("%")) {
try {
String decodedPath = URLDecoder.decode(path, "UTF-8");
if(this.isInvalidPath(decodedPath)) {
return true;
}
decodedPath = this.processPath(decodedPath);
if(this.isInvalidPath(decodedPath)) {
return true;
}
} catch (UnsupportedEncodingException | IllegalArgumentException var3) {
;
}
}
return false;
}
PathResourceResolver.isInvalidEncodedPath(resourcePath)
private boolean isInvalidEncodedPath(String resourcePath) {
if(resourcePath.contains("%")) {
try {
String decodedPath = URLDecoder.decode(resourcePath, "UTF-8");
if(decodedPath.contains("../") || decodedPath.contains("..\\")) {
this.logger.warn("Resolved resource path contains encoded \"../\" or \"..\\\": " + resourcePath);
return true;
}
} catch (UnsupportedEncodingException var3) {
;
}
}
return false;
}
see something different?? they both do URLDecoder.decode(resourcePath, "UTF-8"),but catch different exceptions.
when you access url,springboot will call the funcion at this order:
1 ResourceHttpRequestHandler.isInvalidEncodedPath(path)
2 PathResourceResolver.isInvalidEncodedPath(resourcePath)
so when hit the /37%25.html, in ResourceHttpRequestHandler.isInvalidEncodedPath(path),it gets /37%.html,because tomcat(or browser) decode %25 to %. then URLDecoder.decode("/37%.html", "UTF-8"), trigger IllegalArgumentException, be catched, but do nothing, return false. In PathResourceResolver.isInvalidEncodedPath(resourcePath),URLDecoder.decode("/37%.html", "UTF-8"),trigger IllegalArgumentException, no catced, throw the exception.
when hit the /37%2525.html, in ResourceHttpRequestHandler.isInvalidEncodedPath(path),it gets /37%25.html, URLDecoder.decode("/37%25.html", "UTF-8"),no problem. In PathResourceResolver.isInvalidEncodedPath(resourcePath),URLDecoder.decode("/37%25.html", "UTF-8"),also no problem.then display the 37%25.html
If you just type /37%.html,u will get 400 error, Invalid URI: isHexDigit.
so,unfortunately, the url which has % char, can not be accessed correctly.
I'm using scrapy framework and I can not load the other data from my analyzed webpage, it has a tag to see more. Could you tell me what you could do about it, thank you.
import scrapy
from scrapy.spiders import CrawlSpider,Rule
from scrapy.linkextractors import LinkExtractor
from prueba1.items import Prueba1Item
from scrapy.exceptions import CloseSpider
class PruebaSpider(CrawlSpider):
name = 'prueba1'
item_count = 0
allowed_domain = ['http://www.abc.com.py/']
start_urls = ['http://www.abc.com.py/buscar/?buscar=Santiago+Pe%C3%B1a'
'http://www.abc.com.py/buscar/?buscar=Santi+Pe%C3%B1a',
'http://www.abc.com.py/buscar/?buscar=santiago+pe%C3%B1a',
'http://www.abc.com.py/buscar/?buscar=santi+pe%C3%B1a']
rules = {
Rule(LinkExtractor(allow =(),canonicalize = True, unique =
True,restrict_xpaths=('//html/body/div/a[#id="load-more"]'))),
Rule(LinkExtractor(allow =(),canonicalize = True, unique =
True,restrict_xpaths=('//div[#class="article"]')),
callback = 'parse_item', follow=True)
}
def parse_item(self, response):
ml_item=Prueba1Item()
ml_item['article'] = response.xpath('normalize-space(//h1)').extract()
ml_item['fecha'] = response.xpath('normalize-
space(//small)').extract()
ml_item['contenido'] = response.xpath('normalize-
space(//p[#class="summary"])').extract()
ml_item['contenido2'] = response.xpath('normalize-
space(//div[#class="text"])').extract()
ml_item['url'] = response.xpath('normalize-
space(//link/#href)').extract()
ml_item['comentarioFacebook'] = response.xpath('normalize-
space(//div[#class="_30o4"]/span/span[#class="_5mdd"])').extract()
self .item_count += 1
if self.item_count > 50:
raise CloseSpider('item_exceeded')
yield ml_item
According to the search results I have more than 4000 results but I can not bring more than 50 with this code.
The contents are loaded dynamically using json in this format:
{
"titulo": "Tuma se suma a Marito",
"copete": "El diputado \u00d3scar Tuma oficializ\u00f3 su respaldo a la candidatura de Mario Abdo Ben\u00edtez a la presidencia, ya que sus reportes indicaron que el candidato de Colorado A\u00f1etet\u00e9 tiene mayor intenci\u00f3n de votos. El 100% de su dirigencia se lo pidi\u00f3, dice. ",
"publicacion": "09-11-2017 08:00",
"imagen": "2017\/10\/02\/el-diputado-scar-tuma-inscribio-ayer-las-precandidaturas-de-su-movimiento-tu-asuncion-en-la-junta-de-gobierno--200750000000-1634915.jpg",
"url": "nacionales\/tuma-se-suma-a-marito-1648202.html",
"autor": "",
"hits": "3163",
"comentarios": "1",
"corte_url": "https:\/\/s3-sa-east-1.amazonaws.com\/assets.abc.com.py\/2017\/10\/02\/_146_162_1542245.jpg",
"corte_width": 146,
"corte_height": 162,
"autor_nombre": null,
"autor_url": "http:\/\/www.abc.com.py\/autor\/-.html",
"total": "4861"
}
Therefore you can get the data in json format directly from the url below without using Xpath or css selector(try open this url in your browser):
http://www.abc.com.py/ajax.php?seccion=busqueda-avanzada&tipo=4&tipoplant=0&buscar=Santiago+Pe%C3%B1a&desde=&hasta=&seccion-noticia=&temas=&begin=0&limit=7&aditional=
I believe it's not difficult to modify the url to get the data you want and put them into your scripts, for example, get 10 items from the second item, just change begin to 2 and limit to 10:
http://www.abc.com.py/ajax.php?seccion=busqueda-avanzada&tipo=4&tipoplant=0&buscar=Santiago+Pe%C3%B1a&desde=&hasta=&seccion-noticia=&temas=&begin=2&limit=10&aditional=
I'm having trouble connecting to an Oracle database from Powershell using the Oracle.ManagedDataAccess.dll.
I followed this tutorial on Technet and ended up with this code:
add-type -path "C:\oracle\product\12.1.0\client_1\ODP.NET\managed\common\Oracle.ManagedDataAccess.dll"
$username = "XXXX"
$password = "XXXX"
$data_source = "(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=XXXX)(PORT=XXXX))(CONNECT_DATA = (SERVER=dedicated)(SERVICE_NAME=XXXX)))"
$connection_string = "User Id=$username;Password=$password;Data Source=$data_source"
try{
$con = New-Object Oracle.ManagedDataAccess.Client.OracleConnection($connection_string)
$con.Open()
} catch {
Write-Error (“Can’t open connection: {0}`n{1}” -f `
$con.ConnectionString, $_.Exception.ToString())
} finally{
if ($con.State -eq ‘Open’) { $con.close() }
}
Unfortunately I'm getting this error:
C:\Users\XXXX\Desktop\oracle_test.ps1 : Can’t open connection: User Id=XXXX;Password=XXXX;Data Source=(DESCRIPTION=(ADDRESS=(PROTOCOL=TCP)(HOST=XXXX)(PORT=XXXX))(CONNECT_DATA =
(SERVER=dedicated)(SERVICE_NAME=XXXX)))
System.Management.Automation.MethodInvocationException: Exception calling "Open" with "0" argument(s): "The type initializer for 'Oracle.ManagedDataAccess.Types.TimeStamp' threw an exception." --->
System.TypeInitializationException: The type initializer for 'Oracle.ManagedDataAccess.Types.TimeStamp' threw an exception. ---> System.Runtime.Serialization.SerializationException: Unable to find assembly
'Oracle.ManagedDataAccess, Version=4.121.2.0, Culture=neutral, PublicKeyToken=XXXX'.
at OracleInternal.Common.OracleTimeZone.GetInstance()
at Oracle.ManagedDataAccess.Types.TimeStamp..cctor()
--- End of inner exception stack trace ---
at OracleInternal.ConnectionPool.PoolManager`3.CreateNewPR(Int32 reqCount, Boolean bForPoolPopulation, ConnectionString csWithDiffOrNewPwd, String instanceName)
at OracleInternal.ConnectionPool.PoolManager`3.Get(ConnectionString csWithDiffOrNewPwd, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)
at OracleInternal.ConnectionPool.OraclePoolManager.Get(ConnectionString csWithNewPassword, Boolean bGetForApp, String affinityInstanceName, Boolean bForceMatch)
at OracleInternal.ConnectionPool.OracleConnectionDispenser`3.Get(ConnectionString cs, PM conPM, ConnectionString pmCS, SecureString securedPassword, SecureString securedProxyPassword)
at Oracle.ManagedDataAccess.Client.OracleConnection.Open()
at CallSite.Target(Closure , CallSite , Object )
--- End of inner exception stack trace ---
at System.Management.Automation.ExceptionHandlingOps.CheckActionPreference(FunctionContext funcContext, Exception exception)
at System.Management.Automation.Interpreter.ActionCallInstruction`2.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
at System.Management.Automation.Interpreter.EnterTryCatchFinallyInstruction.Run(InterpretedFrame frame)
+ CategoryInfo : NotSpecified: (:) [Write-Error], WriteErrorException
+ FullyQualifiedErrorId : Microsoft.PowerShell.Commands.WriteErrorException,oracle_test.ps1
I have checked, that the connection string is correct (it works for tnsping)
The Username and password are correct as well
I replaced anything that shouldn't be disclosed with XXXX
The database version is: Oracle Database 12c Release 12.1.0.1.0
The error is the same for the 32-Bit and 64-Bit Powershell instances
Before the first run the Oracle.ManagedDataAccess.dll isn't loaded, after the run it stays loaded until I close that shell
Unfortunately I'm neither an Oracle nor a Powershell Expert (I prefer MySQL and Python), therefore I would really appreciate any ideas and/or insights you might have.
Im not sure if this is technically a solution - I'd classify it more as a workaround, but it worked for me.
After doing some more research I found a suitable alternative to the Oracle.ManagedDataAccess.dll. I found the System.Data.OracleClient class of the .Net Framework. (It requires an installed Oracle Client, which the machine fortunately has)
Here's an outline of the solution that worked for me:
add-type -AssemblyName System.Data.OracleClient
$username = "XXXX"
$password = "XXXX"
$data_source = "XXXX"
$connection_string = "User Id=$username;Password=$password;Data Source=$data_source"
$statement = "select level, level + 1 as Test from dual CONNECT BY LEVEL <= 10"
try{
$con = New-Object System.Data.OracleClient.OracleConnection($connection_string)
$con.Open()
$cmd = $con.CreateCommand()
$cmd.CommandText = $statement
$result = $cmd.ExecuteReader()
# Do something with the results...
} catch {
Write-Error (“Database Exception: {0}`n{1}” -f `
$con.ConnectionString, $_.Exception.ToString())
} finally{
if ($con.State -eq ‘Open’) { $con.close() }
}
Use the server:port/service syntax.
$dataSource="server.network:1522/service1.x.y.z"
Here is my (Powershell, but you can adapt to C#) code to do this with ODP.NET managed data access.... which I load directly using the Powershell Add-Type but in C# would be a using/reference...
create a connection, using the connection builder class to ensure the resulting connection string is exactly as desired. Of course you can simplify but this is self-documenting:
function New - OracleConnection { <
#
.SYNOPSIS# generate a well - formed connection string with individual properties
Create and open a new Oracle connection using optional connectionstring argument
.DESCRIPTION
Create and open a new Oracle connection using optional connectionstring argument
.EXAMPLE
New - OracleConnection
New - OracleConnect - connectionString "My well-formed Oracle connections string"
.NOTES
Connection is opened here by
default# > #Add - Type - Path ".\Oracle.ManagedDataAccess.dll" [OutputType([Oracle.ManagedDataAccess.Client.OracleConnection])]
Param(
[Parameter(Mandatory = $false)]
[string] $theConnectionString, [Parameter(Mandatory = $false)]
[string] $openOnCreate = "1"#
means true - open the connection...
)[Oracle.ManagedDataAccess.Client.OracleConnection] $con = New - Object - TypeName Oracle.ManagedDataAccess.Client.OracleConnection;
if ([string]::IsNullOrEmpty($theConnectionString)) {#
$dataSource = "*********:1521/******;"
$conStringBuilder = New - Object Oracle.ManagedDataAccess.Client.OracleConnectionStringBuilder;
$conStringBuilder["Data Source"] = $dataSource;
$conStringBuilder["User ID"] = "*******";
$conStringBuilder["Password"] = "*******";
$conStringBuilder["Persist Security Info"] = $True;
$conStringBuilder["Connection Lifetime"] = 180;
$conStringBuilder["Connection Timeout"] = 10;
$conStringBuilder["Pooling"] = $true;
$conStringBuilder["Min Pool Size"] = 10;
$conStringBuilder["Max Pool Size"] = 20;
$conStringBuilder["Incr Pool Size"] = 5;
$conStringBuilder["Decr Pool Size"] = 2;
$conStringBuilder["Statement Cache Size"] = 200;
$conStringBuilder["Statement Cache Purge"] = $false;#
default
$con.ConnectionString = $conStringBuilder.ConnectionString;
} else {
$con.ConnectionString = $theConnectionString;
}
if (Get - IsTrue - yesNoArg $openOnCreate) {
if (-not(Get - ConnectionStateIsOpen($con))) {#
attempt open, ignore error
if already open.State is normally open after successful create
try {
$con.Open();
} catch {
$null;
}
}
}
Write - Output - NoEnumerate $con;
}
get a connection:
$con = New - OracleConnection;
open / close as necessary(it defaults to open...)
...
...
if ($con.State - ne[System.Data.ConnectionState]::Open) {
$con.Open();#
no arguments that I know of ...
}
use it and then close/dispose...
$sql = "SELECT * FROM YOUR_TABLE t WHERE t.SOMETHING = :the_parm";
[Oracle.ManagedDataAccess.Client.OracleCommand]$cmd = New-Object -TypeName Oracle.ManagedDataAccess.Client.OracleCommand;
$cmd.Connection = Get-OracleConnection;
$cmd.CommandText = $sql;
$cmd.BindByName = $true;
$dbType = ([Oracle.ManagedDataAccess.Client.OracleDbType]::VarChar2);
$cmd.Parameters.Add(( New-Param -name "the_parm" -type $dbType -size 15 )).Value = $the_criteria_value ;
Write-Output -NoEnumerate ($cmd.ExecuteScalar());
$cmd.Connection.Close();
$cmd.Dispose();
It could very well be an issue with oracle 12.2.x
I had to add the following lines to the sqlnet.ora file on the database server to allow connections from older oracle clients:
SQLNET.ALLOWED_LOGON_VERSION_CLIENT=8
SQLNET.ALLOWED_LOGON_VERSION_SERVER=8
Once added I could login with oracle 10g and 11g clients
I had the exact same issue and I switched from ODAC version 12.2.0.1.0 to version 11.2.0.3 and it worked like a charm to Open connection and read data. Hope this helps.
Thanks,
SK
I am using the adwords api to generate reports.
Please bear with me as I am not too familiar with the same.
I am using version v201409 of the api.
I get the report columns for KEYWORD_PERFORMANCE_REPORT using getReportFields.
I then try to download the report using a subset of those columns.
For KEYWORD_PERFORMANCE_REPORT I get the error:
Cannot select a combination of Device and
AssistClicks,AssistClicksOverLastClicks,AssistImpressions,AssistImpressionsOverLastClicks,AveragePageviews,AverageTimeOnSite,BounceRate,Bounces,ClickAssistedConversionValue,ClickAssistedConversionValueLong,ClickAssistedConversionValueNonMoney,ClickAssistedConversions,ClickAssistedConversionsOverLastClickConversions,ImpressionAssistedConversionValue,ImpressionAssistedConversionValueLong,ImpressionAssistedConversionValueNonMoney,ImpressionAssistedConversions,ImpressionAssistedConversionsOverLastClickConversions,LastClickConversions,LastClicks,NewVisitors,Pageviews,PercentNewVisitors,VisitDuration,Visits,
Type: ReportDefinitionError.INVALID_FIELD_NAME_FOR_REPORT.
The question is: How do I find out a valid set of combinations of columns without going through a trial and error process.. Is there any documentation which will help me with the same.
I looked at the columns for KEYWORD_PERFORMANCE_REPORT in http://developers.guge.io/adwords/api/docs/appendix/reports and exclude the colums which the api said were "not compatible". Got a similar error.
Thanks
N.B> If I try this code with the columns provided in the online example it works and downloads the report as expected.
The code is:
`
String[] columnNames = {
"ConversionRateManyPerClickSignificance",
"ConversionRateSignificance",
"ViewThroughConversionsSignificance",
"AccountCurrencyCode",
"AccountDescriptiveName",
"AccountTimeZoneId",
"AdGroupId",
"AdGroupName",
"AdGroupStatus",
"AssistImpressions",
"AssistImpressionsOverLastClicks",
"AverageCpc",
"AverageCpm",
"AveragePageviews",
"AveragePosition",
"AverageTimeOnSite",
"BiddingStrategyId",
"BiddingStrategyName",
"BiddingStrategyType",
"CampaignId",
"CampaignName",
"CampaignStatus",
"ClickAssistedConversionsOverLastClickConversions",
"ClickAssistedConversionValue",
"Clicks",
"ClickSignificance",
"ClickType",
"ConversionManyPerClickSignificance",
"ConversionRate",
"ConversionRateManyPerClick",
"Conversions",
"ConversionSignificance",
"ConversionsManyPerClick",
"ConversionTypeName",
"ConversionValue",
"Cost",
"CostPerConversion",
"CostPerConversionManyPerClick",
"CostPerConversionManyPerClickSignificance",
"CostPerConversionSignificance",
"CostSignificance",
"CpcBid",
"CpcBidSource",
"CpmBid",
"CpmSignificance",
"CriteriaDestinationUrl",
"Ctr",
"CtrSignificance",
"CustomerDescriptiveName",
"CvrSignificance",
"Date",
"DayOfWeek",
"Device",
"ExternalCustomerId",
"FinalAppUrls",
"FinalMobileUrls",
"FinalUrls",
"FirstPageCpc",
"Id",
"ImpressionAssistedConversions",
"ImpressionAssistedConversionsOverLastClickConversions",
"ImpressionAssistedConversionValue",
"Impressions",
"ImpressionSignificance",
"IsNegative",
"KeywordMatchType",
"LabelIds",
"Labels",
"Month",
"MonthOfYear",
"PlacementUrl",
"PositionSignificance",
"PrimaryCompanyName",
"QualityScore",
"Quarter",
"SearchExactMatchImpressionShare",
"SearchImpressionShare",
"SearchRankLostImpressionShare",
"Slot",
"TrackingUrlTemplate",
"UrlCustomParameters",
"ValuePerConversion",
"ValuePerConversionManyPerClick",
"ViewThroughConversions",
"Week",
"Year"
};
public static void downloadConsolidatedReportFile(String[] columnNames, final ReportDefinitionDateRangeType forDateRange, final ReportDefinitionReportType reportDefinitionReportType, final String to) throws Exception {
com.google.api.ads.adwords.lib.jaxb.v201409.Selector selector = new com.google.api.ads.adwords.lib.jaxb.v201409.Selector();
selector.getFields().addAll(Lists.newArrayList(columnNames));
ReportDefinition reportDefinition = new ReportDefinition();
reportDefinition.setReportName("Report " + reportDefinitionReportType.value() + " for dateRange " + forDateRange.value());
reportDefinition.setDateRangeType(forDateRange);
reportDefinition.setReportType(reportDefinitionReportType);
reportDefinition.setDownloadFormat(DownloadFormat.CSV);
ReportingConfiguration reportingConfiguration = new ReportingConfiguration.Builder()
.skipReportHeader(true)
.skipReportSummary(true)
.build();
session.setReportingConfiguration(reportingConfiguration);
reportDefinition.setSelector(selector);
BufferedWriter bw = new BufferedWriter(new OutputStreamWriter(new FileOutputStream(to)));
String mccId = session.getClientCustomerId(); //The id from ads.properties file
Collection<Client> clientIds = getClientAccountIds(mccId);
try {
for (Client cl : clientIds) {
BufferedReader reader = null;
String customerId = cl.id;
String name = cl.name;
session.setClientCustomerId(cl.id);
try {
ReportDownloadResponse response =
new ReportDownloader(session).downloadReport(reportDefinition);
if (response == null || response.getHttpStatus() != 200) {
handleError(response);
}
BufferedInputStream bs = new BufferedInputStream(response.getInputStream());
reader = new BufferedReader(new InputStreamReader(bs));
String line = null;
log.info("getting " + reportDefinition.getReportType().value() + " for " + customerId+" "+name);
reader.readLine(); //Skip the first line of column names
while ((line = reader.readLine()) != null) {
bw.write(line + "\n");
}
} catch (DetailedReportDownloadResponseException e) {
log.error("An error was thrown downloading report for Customer id: " + customerId+" "+name, e);
//We have to do this as we have to filter out the mcc id. An exception is thrown by MCC id
if (e.getType().equals("ReportDefinitionError." + ReportDefinitionErrorReason.CUSTOMER_SERVING_TYPE_REPORT_MISMATCH.getValue())) {
continue;
} else {
throw e;
}
} catch (Exception e) {
log.error("An error was thrown downloading report for Customer id: " + customerId+" "+name, e);
throw e;
} finally {
if (reader != null) {
reader.close();
}
}
}
} finally {
if (bw != null) {
bw.flush();
bw.close();
}
}
}
`
None of the columns you mentioned in the comment below are used.
Check the following documentation.
https://developers.google.com/adwords/api/docs/appendix/reports/keywords-performance-report#activeviewcpm
For some fields "Not compatible with the following fields" option is provided. Click on that option to check the combinations that are not compatible
The following columns are not selectable in the Keywords Performance Report in v201409:
AssistClicksOverLastClicks
Bounces
ClickAssistedConversionValueLong
ClickAssistedConversionValueNonMoney
ImpressionAssistedConversionValueLong
ImpressionAssistedConversionValueNonMoney
LastClickConversions
LastClicks
NewVisitors
Pageviews
VisitDuration
Visits
Suggest you try removing them and trying again. Failing that, post some code so we can see how you are making the call.
For future reference the AdWords Ad Hoc Reporting docs provide a downloadable CSV file for each of the report types listing the allowed metrics, attributes and segments.
I got an same issue with you. And from my understandings, there's no API to identify which column combinations are valid in adwords so far.
So I check whether the combination is valid or not before download the file by requesting real AWQL.
If the dummy AWQL returns error such as Adwords: Reporting Error: HTTP code: 400, error type: 'ReportDefinitionError.INVALID_FIELD_NAME_FOR_REPORT', trigger: 'Cannot select a combination of ActiveViewCpm and ConversionCategoryName,ConversionTrackerId,ConversionTypeName', field path: 'ActiveViewCpm', then I modify the combination of columns by using the error details through trial and error.
I don't know if this is the best way...
I'm working on an XML message process program using dom4j, and encounter a problem with XML node attribute parsing. The business requirement is: parse and validate input XML, if a field or an attribute is invalid, return an XML which structure denotes the error. And I'v snipped a test program below:
import org.dom4j.Document;
import org.dom4j.DocumentHelper;
import org.dom4j.Element;
import org.dom4j.Node;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
public class XPathStudy {
private final static Logger log = LoggerFactory.getLogger(XPathStudy.class);
private static String xmlInput;
private static void initXmlInput() {
xmlInput = "<RootTag> <BizRec FLAG=\"5\">";
xmlInput += " <FieldOne>11111</FieldOne>";
xmlInput += " <FieldTwo></FieldTwo>";
xmlInput += " <FieldThree>33333</FieldThree>";
xmlInput += " </BizRec> </RootTag>";
}
private static Document makeErrRspsDoc(String xpath, String errCode, String errDesc) {
Document errDoc = DocumentHelper.createDocument();
Element errElem = DocumentHelper.makeElement(errDoc, xpath);
errElem.addElement("ErrCode").addText(errCode);
errElem.addElement("ErrDesc").addText(errDesc);
return errDoc;
}
public static void main(String[] args) throws Exception {
initXmlInput();
log.info("xmlInput = " + xmlInput);
Document doc = DocumentHelper.parseText(xmlInput);
log.info("xmlInput parsed done");
String xpath = "*//FieldTwo";
Node node = doc.getRootElement().selectSingleNode(xpath);
if (node == null) {
log.warn("node [" + xpath + "] not found");
System.exit(1);
}
log.info("node [" + node.getPath() + "] located");
if (node.getText().trim().isEmpty()) {
Document errDoc = makeErrRspsDoc(node.getPath(), "1201", "FieldTwo can not be empty");
log.warn("errDoc: " + errDoc.asXML());
} else {
log.info("FieldTwo validation ok");
}
xpath = "*//#FLAG";
node = doc.getRootElement().selectSingleNode(xpath);
if (node == null) {
log.warn("node [" + xpath + "] not found");
System.exit(1);
}
log.info("node [" + node.getPath() + "] located");
int flagVal = Integer.parseInt(node.getText().trim());
if (flagVal == 1 || flagVal == 2) {
log.info("FLAG " + flagVal + " is ok");
} else {
Document errDoc = makeErrRspsDoc(node.getPath(), "1001", "FLAG attr should be 1 or 2");
log.warn("errDoc: " + errDoc.asXML());
}
}
}
And run it, the log info below:
15:01:08.143 [main] INFO XPathStudy - xmlInput = <RootTag> <BizRec FLAG="5"> <FieldOne>11111</FieldOne> <FieldTwo></FieldTwo> <FieldThree>33333</FieldThree> </BizRec> </RootTag>
15:01:08.203 [main] INFO XPathStudy - xmlInput parsed done
15:01:08.255 [main] INFO XPathStudy - node [/RootTag/BizRec/FieldTwo] located
15:01:08.259 [main] WARN XPathStudy - errDoc: <?xml version="1.0" encoding="UTF-8"?>
<RootTag><BizRec><FieldTwo><ErrCode>1201</ErrCode><ErrDesc>FieldTwo can not be empty</ErrDesc></FieldTwo></BizRec></RootTag>
15:01:08.260 [main] INFO XPathStudy - node [/RootTag/BizRec/#FLAG] located
15:01:08.260 [main] WARN XPathStudy - errDoc: <?xml version="1.0" encoding="UTF-8"?>
<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attr should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>
Everything seems ok, and the errDoc will be logged to an Oracle10g table (a VARCHAR2(1000) field), the following SQL is ok:
select extractvalue(xmltype('<RootTag><BizRec><FieldTwo><ErrCode>1201</ErrCode><ErrDesc>FieldTwo can not be empty</ErrDesc></FieldTwo></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual;
1201
But this SQL will fail:
select extractvalue(xmltype('<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attribute should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual;
Error starting at line 1 in command:
select extractvalue(xmltype('<RootTag><BizRec><#FLAG><ErrCode>1001</ErrCode><ErrDesc>FLAG attribute should be 1 or 2</ErrDesc></#FLAG></BizRec></RootTag>'),
'*//ErrCode') as err_code from dual
Error report:
SQL Error: ORA-31011: XML parsing failed
ORA-19202: Error occurred in XML processing
LPX-00231: invalid character 64 ('#') found in a Name or Nmtoken
Error at line 1
ORA-06512: at "SYS.XMLTYPE", line 301
ORA-06512: at line 1
31011. 00000 - "XML parsing failed"
*Cause: XML parser returned an error while trying to parse the document.
*Action: Check if the document to be parsed is valid.
And Oracle's doc explains that XML's node can not contain special chars. So, my question is: howto change my Java code to solve the problem of the 'FLAG' attribute's error response?
<#FLAG> is not a legal XML element name, since the # character cannot be the start of a name. DOM4J is not known for being best in class when it comes to enforcing well formed and valid XML documents.
To communicate the failing node as an XPath expression, consider storing it in an attribute or as a text node.