Although I saw similar posts, I can't seem to solve this problem (so I am parsing the xml by hand and not using an xpath expression)... This is the XML I have...
<soapenv:Envelope xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/"
xmlns:tnew="http://PR_Library/TNeWebService">
<soapenv:Header/>
<soapenv:Body>
<tnew:ApercuenRequest>
<tnew:OpcionCarga>0</tnew:OpcionCarga>
<tnew:Sucursal>1</tnew:Sucursal>
<tnew:Vendedor>8000</tnew:Vendedor>
<tnew:CanalConsulta>1</tnew:CanalConsulta>
<tnew:UsuarioCarga>8000</tnew:UsuarioCarga>
<tnew:Comercio>0</tnew:Comercio>
<tnew:FechaEvaluacion>20140109</tnew:FechaEvaluacion>
<tnew:ModeloDeEvaluacion></tnew:ModeloDeEvaluacion>
<tnew:IdEvaluacion>3694852</tnew:IdEvaluacion>
<tnew:Dudosa>0</tnew:Dudosa>
<tnew:Exceptuado>1</tnew:Exceptuado>
<tnew:TipoDocumento>1</tnew:TipoDocumento>
<tnew:NumeroDocumento>5881040</tnew:NumeroDocumento>
<tnew:Sexo>F</tnew:Sexo>
<tnew:Rol>5</tnew:Rol>
<tnew:RelacionLaboral>5</tnew:RelacionLaboral>
<tnew:TipoEmpleado>0</tnew:TipoEmpleado>
<tnew:FechaIngresoEmpleo></tnew:FechaIngresoEmpleo>
<tnew:AntiguedadLaboral>0</tnew:AntiguedadLaboral>
<tnew:IngresoMensualAnalista>0</tnew:IngresoMensualAnalista>
<tnew:IngresoMensual>6000</tnew:IngresoMensual>
<tnew:EstadoCivil>4</tnew:EstadoCivil>
<tnew:FechaDeNacimiento>19540423</tnew:FechaDeNacimiento>
<tnew:SituacionVivienda>3</tnew:SituacionVivienda>
<tnew:DomicilioProvincia></tnew:DomicilioProvincia>
<tnew:DomicilioLocalidad></tnew:DomicilioLocalidad>
<tnew:NivelEstudios></tnew:NivelEstudios>
<tnew:RelacionConTN>1</tnew:RelacionConTN>
<tnew:FechaAltaAbogado></tnew:FechaAltaAbogado>
<tnew:MarcaAbogado>N</tnew:MarcaAbogado>
<tnew:Cdi></tnew:Cdi>
<tnew:Campana></tnew:Campana>
</tnew:ApercuenRequest>
</soapenv:Body>
</soapenv:Envelope>
And the following Java code...
private XPathFactory xpathFactory = null;
private XPath xpath = null;
private DocumentBuilderFactory factory = null;
private DocumentBuilder builder= null;
private XPathExpression expr;
private Node nodo = null;
private Node nodoAux = null;
private NodeList result = null;
private NodeList nodeList = null;
...
try {
xpathFactory = XPathFactory.newInstance();
xpath = xpathFactory.newXPath();
factory = DocumentBuilderFactory.newInstance();
builder= factory.newDocumentBuilder();
NamespaceContext context = new NameSpaceNevada();
xpath.setNamespaceContext(context);
}
catch (Exception e) {
}
...
evaluarExpresion(documentoXML,"/soapenv:Envelope/soapenv:Body/tnew:ApercuenRequest/tnew:OpcionCarga")
private String evaluarExpresion(Document documentoXML, String expresion) throws Exception{
try {
expr = xpath.compile(expresion);
result = (NodeList)expr.evaluate(documentoXML, XPathConstants.NODESET);
System.out.println((result.item(0)!=null?result.item(0).getNodeName():"") + " ->>>> " + (result.item(0).getFirstChild()!=null?result.item(0).getFirstChild().getNodeValue():""));
return result.item(0).getFirstChild()!=null?result.item(0).getFirstChild().getNodeValue():null;
}
catch (Exception e) {
e.printStackTrace();
return null;
}
}
And I can't seem to get the "0" value as you see it in the XML... I have tried a lot of things... I'll be watching... The thing is, I can parse it manualy but the joke is doing it through xpath expressions so I can save code...
Hope I made me understood...
The getNamespaceURI method from the NameSpaceNevada object is as follows:
public String getNamespaceURI(String prefix) {
if (prefix == null) {
throw new IllegalArgumentException("No prefix provided!");
} else if (prefix.equals(XMLConstants.DEFAULT_NS_PREFIX)) {
return "http://univNaSpResolver/book";
} else if (prefix.equals("soapenv")) {
return "http://schemas.xmlsoap.org/soap/envelope/";
} else if (prefix.equals("tnew")) {
return "http://PR_Library/TNeWebService";
} else {
return XMLConstants.NULL_NS_URI;
}
}
Rather than evaluating the expression as a node set and then doing complex DOM manipulations to extract the text value, you could just evaluate the expression as a string in the first place.
expr.evaluate(documentoXML);
would evaluate the expression and use XPath rules to convert the result to a String, which in this case would give you the text content of the first (in fact the only) tnew:OpcionCarga element.
But the real reason why it is not working is that Java's DocumentBuilderFactory is non-namespace-aware by default (a very bad design decision in many people's eyes, but one that now cannot be corrected without breaking backwards compatibility) - you must do
factory.setNamespaceAware(true);
before you call .newDocumentBuilder() in order to create a namespace-aware parser.
Related
i don't know really how to formulate this but what i want is to achieve is the following, read a JSON file from a storage (direct url) but if that JSON file does not exist it should return an empty array.
this is what i got so far (and works)
public object GetFromFile(int Code)
{
string uriPath = "http://mystorage.com/folder/" + Code + ".json";
var allText = (new WebClient()).DownloadString(uriPath);
object jsonObject = JsonConvert.DeserializeObject(allText);
return jsonObject;
}
it returns to me the requested list of codes as a array, now if the code does not exists on the storage, possible, then the webservice should just return and empty array []
desired result:
from file (works):
[{001},{002},{003}]
if file does not exist
[]
//The call to WebClient.DownloadString(string) will throw an exception if the
//uri does not exist, in your case, the json file does not exist
var allText = null;
object jsonObject = null;
try
{
allText = (new WebClient()).DownloadString(uriPath);
jsonObject = JsonConvert.DeserializeObject(allText);
}
catch(WebException ex)
{
jsonObject = new object[0];
}
It seems that you expect that the JSON-file contains a sequence of similar items, from the look of it a sequence of integer numbers.
If you already know that the JSON-file contains this type of objects, it is better to use the overload of DeserializeObject<int>.
IEnumerable<int> ReadNumbersFromWebClient(int Code)
{
string uriPath = "http://mystorage.com/folder/" + Code + ".json";
var downloadedText = DownloadText(uriPath);
return JsonConvert.DeserializeObject<int[]>(allText);
}
string DownloadText(string uriPath)
{
using (var webClient = new WebClient())
{
return webClient.DownloadString(uriPath);
}
}
You said that you wanted to return an empty sequence if the "file does not exist". I assume that you meant: "if the web client says there is no string to download"
I looked at WebClient.DownloadString, and I couldn't find what happens if you use an uriPath that does not exist. Do you get empty string, or an exception?
ICollection<int> ReadNumbersFromWebClient(int Code)
{
// in case of exception: return empty string
try
{
string uriPath = "http://mystorage.com/folder/" + Code + ".json";
var downloadedText = DownloadText(uriPath);
return JsonConvert.DeserializeObject<int[]>(allText);
}
catch (Exception exc) // or use specific exception
{
return new int[];
}
}
It would be neater if you let DownloadText catch the expression. In case of exception return empty string. Experiment what happens if you try to Deserialize an empty string. It will probably be an empty int[]
We have a situation where the CrmServiceClient class cannot be instantiated, with an 'Object reference not set to an object' error coming from deep within the bowels of the constructor. I've also received a Collection was modified; enumeration operation may not execute error in a few situations.
This does not happen all the time, but we seem to be able to reproduce it when we trigger multiple requests very quickly.
We create the object as follows:
var ctx = new CrmServiceClient(ConfigurationManager.ConnectionStrings["Xrm"].ConnectionString);
The connection string is valid and we have set RequireNewInstance to true
We were originally using the ctx in a using block but we were advised that we shouldn't be disposing of the CrmServiceClient, so we've removed the using block, but this has not resolved the problem.
The stack trace is below - i've only pasted the relevant part. The stack leading up to this point can be any piece of code that attempts to connect to the CRM to retrieve data
at System.ThrowHelper.ThrowInvalidOperationException(ExceptionResource resource)
at System.Collections.Generic.List`1.Enumerator.MoveNextRare()
at System.Collections.Generic.List`1.Enumerator.MoveNext()
at Microsoft.Xrm.Tooling.Connector.Utilities.GetOrgnameAndOnlineRegionFromServiceUri(Uri serviceUri, String& onlineRegion, String& organizationName, Boolean& isOnPrem)
at Microsoft.Xrm.Tooling.Connector.CrmConnection.SetOrgnameAndOnlineRegion(Uri serviceUri)
at Microsoft.Xrm.Tooling.Connector.CrmConnection..ctor(String serviceUri, String userName, String password, String domain, String homeRealmUri, String authType, String requireNewInstance, String clientId, String redirectUri, String tokenCacheStorePath, String loginPrompt, String certStoreName, String certThumbprint, String skipDiscovery)
at Microsoft.Xrm.Tooling.Connector.CrmConnection..ctor(IDictionary`2 connection)
at Microsoft.Xrm.Tooling.Connector.CrmConnection.Parse(String connectionString)
at Microsoft.Xrm.Tooling.Connector.CrmServiceClient.ConnectToCrmWebService(String crmConnectionString)
at Microsoft.Xrm.Tooling.Connector.CrmServiceClient..ctor(String crmConnectionString)
I believe I've tracked down the issue. I used DotNetPeek to look at the underlying code that was failing. The static method GetOrgnameAndOnlineRegionFromServiceUriwas where the error was occurring.
I tracked it down to a static list (discoSvcs) that was being set to null before the method returns. Other threads that call this method are also trying to do things with this list. It ends up that there is a race condition where one thread could check to see if it isn't null.
The only way I can get around it now is making sure only one CrmServiceClient is being instantiated at any time, by using a lock. This isn't ideal but I am running out of time
Static list definition
namespace Microsoft.Xrm.Tooling.Connector
{
public class Utilities
{
private static CrmOnlineDiscoveryServers discoSvcs;
private static List<string> _autoRetryRetrieveEntityList;
private Utilities()
{
}
Problem Function
The static list variable is checked at the beginning of this function and if it is null, it is populated with some values. It is then used later in the method before being set to null in the finally block.
public static void GetOrgnameAndOnlineRegionFromServiceUri(
Uri serviceUri,
out string onlineRegion,
out string organizationName,
out bool isOnPrem)
{
isOnPrem = false;
onlineRegion = string.Empty;
organizationName = string.Empty;
if (serviceUri.Host.ToUpperInvariant().Contains("DYNAMICS.COM") || serviceUri.Host.ToUpperInvariant().Contains("MICROSOFTDYNAMICS.DE") || (serviceUri.Host.ToUpperInvariant().Contains("MICROSOFTDYNAMICS.US") || serviceUri.Host.ToUpperInvariant().Contains("DYNAMICS-INT.COM")))
{
if (Utilities.discoSvcs == null)
Utilities.discoSvcs = new CrmOnlineDiscoveryServers();
try
{
List<string> stringList = new List<string>((IEnumerable<string>) serviceUri.Host.Split(new string[1]
{
"."
}, StringSplitOptions.RemoveEmptyEntries));
organizationName = stringList[0];
stringList.RemoveAt(0);
StringBuilder stringBuilder = new StringBuilder();
foreach (string str in stringList)
{
if (!str.Equals("api"))
stringBuilder.AppendFormat("{0}.", (object) str);
}
string crmKey = stringBuilder.ToString().TrimEnd('.').TrimEnd('/');
stringBuilder.Clear();
if (!string.IsNullOrEmpty(crmKey))
{
CrmOnlineDiscoveryServer onlineDiscoveryServer = Utilities.discoSvcs.OSDPServers.Where<CrmOnlineDiscoveryServer>((Func<CrmOnlineDiscoveryServer, bool>) (w =>
{
if (w.DiscoveryServer != (Uri) null)
return w.DiscoveryServer.Host.Contains(crmKey);
return false;
})).FirstOrDefault<CrmOnlineDiscoveryServer>();
if (onlineDiscoveryServer != null && !string.IsNullOrEmpty(onlineDiscoveryServer.ShortName))
onlineRegion = onlineDiscoveryServer.ShortName;
}
isOnPrem = false;
}
finally
{
Utilities.discoSvcs.Dispose();
Utilities.discoSvcs = (CrmOnlineDiscoveryServers) null;
}
}
else
{
isOnPrem = true;
if (((IEnumerable<string>) serviceUri.Segments).Count<string>() < 2)
return;
organizationName = serviceUri.Segments[1].TrimEnd('/');
}
}
private List<Activity> sortByDate(List<Activity> activityList){
System.out.println(activityList.size());
SimpleDateFormat sdf = new SimpleDateFormat("dd-MM-yyyy HH:mm:ss");
Collections.sort(activityList, new Comparator<Activity>() {
public int compare(Activity a1, Activity a2) {
System.out.println("aaaaaaaaaaaaa");
int result = 0;
try {
result = sdf.parse(a2.getDate()).compareTo(sdf.parse(a1.etDate()));
} catch (ParseException e) {
//The date should be validated to (dd-MM-yyyy HH:mm:ss) this format in front end before add to the db
}
return result;
}
});
return activityList;
}
This is the method that I have created. The first SOP printed 2 as output so activityList is not empty, but the second SOP line wasn't executed. The code that I use to call the sortByDate method is below. I can't figure out why Collections.sort is not executed.
public List<RetrieveActivity> getPage(int pageNumber) throws CustomException{
PageRequest request = new PageRequest(pageNumber - 1, PAGESIZE, Sort.Direction.ASC, "id");
List<Activity> tempList = repository.findAll(request).getContent();
if (tempList.size() > 0) {
tempList = sortByDate(tempList);
}
But Activity model class has the date attribute as a String. But the thing is, the line 'System.out.println("aaaaaaaaaaaaa");' was never executed.
The pattern that you're using ("dd-MM-yyyy HH:mm:ss") doesn't maintain order when you're moving to the string representation of the object.
Instead of using string comparison:
result = sdf.parse(a2.getDate()).compareTo(sdf.parse(a1.etDate()));
use date comparison:
result = a2.getDate().compareTo(a1.getDate());
I am trying to extract data from below mention json format by pig using jsonLoader():
{"Partition":"10","Key":"618897","Properties2":[{"K":"A","T":"String","V":"M "}, {"K":"B","T":"String","V":"N"}, {"K":"D","T":"String","V":"O"}]}
{"Partition":"11","Key":"618900","Properties2":[{"K":"A","T":"String","V":"W”"},{"K":"B","T":"String","V":"X"}, {"K":"C","T":"String","V":"Y"},{"K":"D","T":"String","V":"Z"}]}
Right now I am able to extract data from “partition” ,“key” and “V” for every array objects with the following code:
A= LOAD '/home/hduser/abc.jon' Using JsonLoader('Partition:chararray,Key:chararray,Properties2:{(K:chararray,T:chararray,V:chararray)},Timestamp:chararray');
B= foreach A generate $0,$1,BagToString(Properties2.V,'\t') as vl:chararray;
store B into './Result/outPut2';
From above code I am getting "Properties2" array value on the sequence basis not column basis, it is creating problem whenever sequence changed or new object comes in existence.
Please help me to extract data on the basis of column( K values.)
My Output
Expected Output
Thanks In Advance
You have two options here
1.Use elephant-bird which will give you a map of key and value.
A = LOAD '/apps/pig/json_sample' USING com.twitter.elephantbird.pig.load.JsonLoader('-nestedLoad') as (json:map[]);
B = FOREACH A GENERATE json#'Partition',json#'Key',json#'Properties2';
dump B;
will give you an output of :
(10,618897,{([T#String,K#A,V#M ]),([T#String,K#B,V#N]),([T#String,K#D,V#O])})
(11,618900,{([T#String,K#A,V#W”]),([T#String,K#B,V#X]),([T#String,K#C,V#Y]),([T#String,K#D,V#Z])})
Or you have to write a custom loader which has to do this
a).It should know what is the correct order of values that will be coming
for the key K
b).Go through each of these values and see if the json is missing any of this key and return an empty/null char for that location.
Am posting the getNext() method of the CustomJsonLoader which will do the same:
#Override
public Tuple getNext() throws IOException {
// TODO Auto-generated method stub
try {
boolean notDone = in.nextKeyValue();
if (!notDone) {
return null;
}
Text value = (Text) in.getCurrentValue();
List<String> valueList = new ArrayList<String>();
if (value != null) {
String jsonString = value.toString();
System.out.println(jsonString);
JSONParser parser = new JSONParser();
JSONObject obj = null;
try {
obj = (JSONObject) parser.parse(jsonString);
} catch (ParseException e) {
// TODO Auto-generated catch block
e.printStackTrace();
}
System.out.println("obj is "+obj);
if (obj != null) {
String partition = (String) obj.get("Partition");
String key = (String) obj.get("Key");
valueList.add(partition);
valueList.add(key);
JSONArray innArr = (JSONArray) obj.get("Properties2");
char[] innKeys = new char[] { 'A', 'B', 'C', 'D' };
Map<String,String> keyMap = new HashMap<String,String>();
for (Object innObj : innArr) {
JSONObject jsonObj = (JSONObject) innObj;
keyMap.put(jsonObj.get("K")+"",jsonObj.get("V")+"");
}
for (int i = 0; i < innKeys.length; i++) {
char ch = innKeys[i];
if (keyMap.containsKey(ch+"")) {
valueList.add(keyMap.get(ch+""));
}else{
valueList.add("");
}
}
Tuple t = tupleFactory.newTuple(valueList);
return t;
}
}
return null;
} catch (InterruptedException e) {
}
}
and register it and run :
REGISTER udf/CustomJsonLoader.jar
A = LOAD '/apps/pig/json_sample' USING CustomJsonLoader();
DUMP A;
(10,618897,M,N,,O)
(11,618900,W,X,Y,Z)
Hope this helps!
I have the following code.
XElement opCoOptOff = doc.Descendants(ns + "OpCoOptOff").FirstOrDefault();
String opCo = opCoOptOff.Element(ns + "strOpCo").Value;
Now if the element I return is null, I am getting a NullReferenceException since the XElement is null. So I changed it to the following.
String opCo = opCoOptOff.Element(ns + "strOpCo").Value;
if(opCoOptOff != null)
{
String opCo = opCoOptOff.Element(ns + "strOpCo").Value;
I am hoping there has to be a more elegant way to do this since this scenario comes up often and I would like to avoid doing this type of check every time there is an issue. Any assistance would be greatly appreciated
You can write an extension method and use it anywhere:
public static class XDocumentExtension
{
public static string GetSubElementValue(this XElement element, string item)
{
if(element != null && element.Value != null)
{
if (element.Element(item) != null)
{
return element.Element(item).Value;
}
}
return null;
}
public static XElement GetElement(this XElement element, string item)
{
if (element != null)
return element.Element(item);
return null;
}
public static XElement GetElement(this XDocument document, string item)
{
if (document != null)
return document.Descendants("item").FirstOrDefault();
return null;
}
}
Use it as :
String opCo = opCoOptOff.Element(ns + "strOpCo").GetSubElementValue(ns + "strOpCo");
Also you can add other extensions for your purpose.
Edit: I'd updated answer but if you read it carefully before I wrote you can add other extensions for your purpose. I wrote this because I guess you may be want to call on null objects Element, I don't know what's exact situation of yours but I add some code for more clarification, depend on your situation complete the XDocumentExtension class, and one note extension methods can work on null objects.
You can actually cast the XElement directly to a string:
http://msdn.microsoft.com/en-us/library/bb155263.aspx
so
String opCo = opCoOptOff.Element(ns + "strOpCo").Value;
could be
string opCo = (string) opCoOptOff.Element(ns + "strOpCo");