Extracting a list of XElements from an XDocument with where clause - xelement

I have an XDocument object from which I am trying to extract all the List which specify a specific criteria.
Following is my XML
<MyQueue xmlns:ns0 = "http://myprogram">
<ns0:QueueReport>
<ns0:number>001</ns0:number>
<ns0:id>A</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>1</ns0:hours>
</ns0:QueueReport>
<ns0:QueueReport>
<ns0:number>001</ns0:number>
<ns0:id>B</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>2</ns0:hours>
</ns0:QueueReport>
<ns0:QueueReport>
<ns0:number>001</ns0:number>
<ns0:id>B</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>10</ns0:hours>
</ns0:QueueReport>
<ns0:QueueReport>
<ns0:number>002</ns0:number>
<ns0:id>A</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>12</ns0:hours>
</ns0:QueueReport>
<ns0:QueueReport>
<ns0:number>003</ns0:number>
<ns0:id>A</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>20</ns0:hours>
</ns0:QueueReport>
</ns0:MyQueue>
The above XML is in an XDocument object say xdoc. I write the following to exract the nodes and its children.
XNamespace b = #"http://myprogram";
var elements = from e xdoc.Elements(b + "MyQueue")
where e.Element(b + "QueueReport").Element(b + "number").Value == "001"
&& e.Element(b + "QueueReport").Element(b + "id").Value == "B"
select e.Elements(b + "QueueReport").ToList();
(Please excuse the typos if any in the code, it was not written in the editor)
The elements variable does not have any results, ideally i would have expected the below two elements in the List.
<ns0:QueueReport>
<ns0:number>001</ns0:number>
<ns0:id>B</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>2</ns0:hours>
</ns0:QueueReport>
<ns0:QueueReport>
<ns0:number>001</ns0:number>
<ns0:id>B</ns0:id>
<ns0:name>ABC</ns0:name>
<ns0:hours>10</ns0:hours>
</ns0:QueueReport>
Please help, I am very new to Linq and have little clue about how else can I achieve this.


Here is the answer to my question, I could get this working with some mind bending..
XNamespace b = #"http://myprogram";
var elements = from e xdoc.Elements(b + "MyQueue").(b + "QueueReport")
where e.Element(b + "number").Value == "001"
&& e.Element(b + "id").Value == "B"
select e;

Related

lua table sorting data

I have a table that is generated by lua code and returned as a callback to client side, but the data is not ordered and I want to return a list sorted by id
local players = {}
for k, player in ipairs(QBCore.Functions.GetPlayers()) do
local charinfo = QBCore.Functions.GetPlayer(player).PlayerData.charinfo
local csn = QBCore.Functions.GetPlayer(player).PlayerData.citizenid
local playerjob = QBCore.Functions.GetPlayer(player).PlayerData.job.label
local ped = GetPlayerPed(player)
local playerCoords = GetEntityCoords(ped)
players[k] = {
["id"] = player,
["name"] = tostring(GetPlayerName(player)),
["charName"] = ("%s %s"):format(charinfo.firstname, charinfo.lastname),
["csn"] = csn,
["playerjob"] = playerjob,
["serverid"] = player,
["ped"] = GetPlayerPed(player),
["coords"] = playerCoords,
}
end
cb(players)

you can use table.sort to order your players table. this will sort the table in-place using the function you provide to determine the order.
table.sort is also explained in Programming in Lua: 19.3 – Sort
Here is an example based on your question:
players =
{
{id = 3217},
{id = 6716},
{id = 3432},
{id = 5575},
{id = 6124},
{id = 1156},
{id = 1789},
}
table.sort(players, function(p1,p2) return p1.id < p2.id end)
for k, v in ipairs(players) do
print(k, v.id)
end
Output:
1 1156
2 1789
3 3217
4 3432
5 5575
6 6124
7 6716

update_cells not working as expected

I have written this function:
def duplicate_sheet1(wb, title=None):
if title is None:
title = wb.sheet1.title + ' DUPLICATE'
wb._sheet_list = [wb.sheet1]
wb.add_worksheet(title, wb.sheet1.row_count, wb.sheet1.col_count)
wb._sheet_list = wb._sheet_list[::-1]
wb._sheet_list[0].update_cells(wb._sheet_list[1]._fetch_cells())
...everything works as expected upon inspection with a debugger except update_cells, when I _fetch_cells for worksheet 0 after running the code, the sheet is empty.

Apparently the list returned by _fetch_cells is not the same as what is expected by update_cells. This may be because _fetch_cells does not include empty cells in the returned list, update_cells may only work with a 1 or 2-D grid--I am unsure.
Here is the work-around I found, apologies as the code could could probably be improved:
def duplicate_sheet1(wb, title=None):
if title is None:
title = wb.sheet1.title + ' DUPLICATE'
wb._sheet_list = [wb.sheet1]
wb.add_worksheet(title, wb.sheet1.row_count, wb.sheet1.col_count)
wb._sheet_list = wb._sheet_list[::-1]
cell_list = build_cell_list(wb._sheet_list[0], wb._sheet_list[1])
wb._sheet_list[0].update_cells(cell_list)
def build_cell_list(new_worksheet, old_worksheet):
fetched = old_worksheet._fetch_cells()
max_row = fetched[-1].row
max_col = max([cell.col for cell in fetched])
cell_list = new_worksheet.range('A1:' + chr(max_col + 64) + str(max_row))
for cell in cell_list:
cell.value = next(
(
f.value for f in fetched
if f.col == cell.col and f.row == cell.row
),
'',
)
return cell_list

how to find value with path x Groovy

please advise how to find and output cust_JiraTaskId. I need the value of cust_JiraTaskId based on the max number of inside node . In this example it'll be 111111.
I managed to find the max externalCode and now i need cust_JiraTaskId value.
<SFOData.cust_JiraReplication>
<cust_HRISId>J000009</cust_HRISId>
<externalCode>7</externalCode>
<cust_JiraTask>
<externalCode>3</externalCode>
<cust_JiraTaskId>12345</cust_JiraTaskId>
</cust_JiraTask>
<cust_JiraTask>
<externalCode>5</externalCode>
<cust_JiraTaskId>111111</cust_JiraTaskId>
</cust_JiraTask>
</SFOData.cust_JiraReplication>
My script is below
// Create an XPath statement to search for the
element or elements you care about:
XPath x;
x = XPath.newInstance("//cust_JiraTask/externalCode");
myElements = x.selectNodes(doc);
String maxvalue = "";
for (Element myElement : myElements) {
if (myElement.getValue() > maxvalue)
{
maxvalue = myElement.getValue();
}
}
props.setProperty("document.dynamic.userdefined.externalCode", maxvalue);
thanks for help.

This works for me with Groovy 2.4.5:
def xml = """
<SFOData.cust_JiraReplication>
<cust_HRISId>J000009</cust_HRISId>
<externalCode>7</externalCode>
<cust_JiraTask>
<externalCode>3</externalCode>
<cust_JiraTaskId>12345</cust_JiraTaskId>
</cust_JiraTask>
<cust_JiraTask>
<externalCode>5</externalCode>
<cust_JiraTaskId>111111</cust_JiraTaskId>
</cust_JiraTask>
</SFOData.cust_JiraReplication>
"""
def xs = new XmlSlurper().parseText(xml)
def nodes = xs.cust_JiraTask.cust_JiraTaskId
def maxNode = nodes.max { it.text() as int }
assert 111111 == maxNode.text() as int

How to scrape all the image url and alt tag within it using scrapy

My target is to crawl image url and image alt tag using scrapy . I tried many combinations but still didn't achieve it.
Here is what i tried
def parse_item(self, response):
sel = Selector(response)
item = imageItem()
item['crawl_time'] = time.asctime( time.localtime(time.time()))
item['crawl_date'] = time.asctime( time.localtime(time.strftime("%Y%m%d")))
item['url'] = response.url
for img in hxs.select('//img'):
item['title'] = node.xpath("#alt").extract()
item['iurl'] = node.xpath("#src").extract()
if response.meta['depth'] == 1:
exit
return item

Some issues there:
You already have sel selector. But you use hxs in the loop
in the loop, you are using node instead of img
does it make more sense that each loop should yield one image item
This is my tested and working code:
def parse_item(self, response):
sel = Selector(response)
images = sel.xpath('//img')
for img in images:
item = imageItem()
item['url'] = response.url
title = img.xpath('./#alt').extract() or ''
item_title = title[0] if title else ''
item['title'] = item_title
iurl = img.xpath('./#src').extract() or ''
item_iurl = iurl[0] if iurl else ''
item['iurl'] = item_iurl
yield item

Here is the below code using which I achieved the result , but depth is still 1
class MySpider(CrawlSpider):
name = 'imageaggr'
start_urls = ['http://www.dmoz.org/','http://timesofindia.indiatimes.com/','http://www.nytimes.com','http://www.washingtonpost.com/','http://www.jpost.com','http://www.rediff.com/']
rules = (
# Extract links matching 'category.php' (but not matching 'subsection.php')
# and follow links from them (since no callback means follow=True by default).
Rule(SgmlLinkExtractor(allow=('', ), deny=('defghi\.txt')), callback='parse_item'),
# Extract links matching 'item.php' and parse them with the spider's method parse_item
# Rule(SgmlLinkExtractor(allow=('\.cms','\.html' )), deny=('parse_item\.html'))),
#Rule(SgmlLinkExtractor(allow=('news', )), callback='parse_item'),
)
def parse_item(self, response):
sel = Selector(response)
images = sel.xpath('//img')
image_count = len(images)
count = 0
while(count < image_count):
item = imageItem()
item['url'] = response.url
title = sel.xpath('//img/#alt').extract()[count] or ''
if title == '':
break
item['title'] = title
iurl = sel.xpath('//img/#src').extract()[count] or ''
item['iurl'] = iurl
item['crawl_time'] = time.asctime( time.localtime(time.time()))
crawl_date = time.strftime("%Y%m%d")
item['crawl_date'] = crawl_date
count = count + 1
return item

Get a value using Ling to XML and where clause

I want to use Linq to XML and I am trying to figure out to way to get the value for currency CAD, BuyRateForeign, basically I need to get value 5,602895.
<ExchRates>
<ExchRate>
<Bank>Bank</Bank>
<CurrencyBase>USD</CurrencyBase>
<Date>24.05.2013.</Date>
<Currency Code="036">
<Name>AUD</Name>
<Unit>1</Unit>
<BuyRateCache>5,569450</BuyRateCache>
<BuyRateForeign>5,625707</BuyRateForeign>
<MeanRate>5,711378</MeanRate>
<SellRateForeign>5,797049</SellRateForeign>
<SellRateCache>5,855019</SellRateCache>
</Currency>
<Currency Code="124">
<Name>CAD</Name>
<Unit>1</Unit>
<BuyRateCache>5,546866</BuyRateCache>
<BuyRateForeign>5,602895</BuyRateForeign>
<MeanRate>5,688218</MeanRate>
<SellRateForeign>5,773541</SellRateForeign>
<SellRateCache>5,831276</SellRateCache>
</Currency>
</ExchRate>
</ExchRates>

var xDoc = XDocument.Load("path/to/your.xml");
var cadForeignBuyRate =
xDoc.Root.Element("ExchRate").First(e => e.Element("Bank").Value == "Bank")
.Elements("Currency").First(e => e.Element("Name").Value == "CAD")
.Element("BuyRateForeign").Value;

var xDoc = XDocument.Load("path/to/your.xml");
var BuyRateForeign = from nm in xDoc.Descendants("Currency")
where (string)nm.Element("Name") == "CAD"
select (string)nm.Element("BuyRateForeign");
Using Lambda expression
var stringRate = xDoc.Descendants("Currency")
.Where(p=> (string)p.Element("Name") == "CAD")
.Select(p => (string)p.Element("BuyRateForeign"))
.SingleOrDefault();

Resources