I have an XML document which is missing some namespace declaration. I know I can define it when I use doc.xpath() method, like the following:
doc.xpath('//dc:title', 'dc' => 'http://purl.org/dc/elements/1.1/')
However I would like to add it once since I have a lot of xpath calls.
I found out that my Nokogiri::XML::Document is inherited from Nokogiri::XML::Node. And the Node class contains an add_namespace() method. However I can't call it, because it says it is undefined.
Is this because Ruby does not allow calling parent class's functions? Is there a way to go around this?
EDIT
I add the following console example:
> c = Nokogiri.XML(doc_text)
> c.class
=> Nokogiri::XML::Document
> c.add_namespace('a','b')
NoMethodError: undefined method `add_namespace' for #<Nokogiri::XML::Document:0x007fea4ee22c60>
And here is the API document about Nokogiri::XML class
EDIT again:
The original doc was valid xml like this:
<root xmlns:ra="...">
<item>
<title/>
<ra:price/>
</item>
<item>...
</root>
However there are too many items, and I have to create one object for each of these, serialize and save in the database. So for each object I took the item node and turn it into string and saved in the object.
Now after I revive the object from DB and I want to parse the item node again I came to this namespace issue.
While Nokogiri::XML::Document does inherit from Nokogiri::XML::Node, some methods are explicitly removed at the document level, including add_namespace
https://github.com/tenderlove/nokogiri/blob/master/lib/nokogiri/xml/document.rb#L203
As #pguardiario notes, you want to add namespaces to the root element, not the document.
However, doing this after parsing the document is too late. Nokogiri has already created the nodes, discarding the namespaces:
require 'nokogiri'
xml = "<r><a:b/></r>"
doc = Nokogiri.XML(xml)
p doc.at('b').namespace
#=> nil
doc.root.add_namespace 'a', 'foo'
puts doc
#=> <?xml version="1.0"?>
#=> <r xmlns:a="foo">
#=> <b/>
#=> </r>
You'll need to fix your source XML as a string before parsing with Nokogiri. (Unless there's some way with the SAX parser to add the namespace when you hit the first node, before moving on.)
Related
I have a simple task of accessing the values of some attributes. This is a simple script that uses Nokogiri::XML::Builder to create a simple XML doc.
require 'nokogiri'
builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml|
xml.Placement(:messageId => "392847-039820-938777", :system => "MOD", :version => "2.0") {
xml.objects {
xml.object(:myattribute => "99", :anotherattrib => "333")
xml.nextobject_ '9387toot'
xml.Entertainment "Last Man Standing"
}
}
end
puts builder.to_xml
puts builder.root.attributes["messageId"]
The results are:
<?xml version="1.0" encoding="UTF-8"?>
<Placement messageId="392847-039820-938777" version="2.0" system="MOD">
<objects>
<object anotherattrib="333" myattribute="99"/>
<nextobject>9387toot</nextobject>
<Entertainment>Last Man Standing</Entertainment>
</objects>
</Placement>
C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/document.rb:178:in `add_child': Document already has a root node (RuntimeError)
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/node.rb:455:in `parent='
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/builder.rb:358:in `insert'
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/builder.rb:350:in `method_missing'
from C:/Documents and Settings/etrojan/workspace/Lads/tryXPATH2.rb:15
The XML that is generated looks fine. However, my attempts to access attributes cause an error to be generated:
Document already has a root node
I don't understand why puts would cause this error.
Using Nokogiri::XML::Reader works for your example, but probably isn't the full answer you are looking for (Note that there is no attributes method for Builder).
reader = Nokogiri::XML::Reader(builder.to_xml)
reader.read #Moves to next node in document
reader.attribute("messageId")
Note that if you issued reader.read again and then tried reader.attribute("messageId") the result will be nil since the current node will not have this attribute.
What you probably want to do is use Nokogiri::XML::Document if you want to search an XML document by attribute.
doc = Nokogiri::XML(builder.to_xml)
elems = doc.xpath("//*[#messageId]") #get all elements with an attribute of 'messageId'
elems[0].attr('messageId') #gets value of attribute of first elem
Here is a slightly more succinct way to access attributes using Nokogiri (assuming you already have your xml stored in a variable called xml, as covered by #atomicules' answer):
xml.xpath("//Placement").attr("messageId")
I have an XML file, and before I process it I need to make sure that a certain element exists and is not blank.
Here is the code I have:
CSV.open("#{csv_dir}/products.csv","w",{:force_quotes => true}) do |out|
out << headers
Dir.glob("#{xml_dir}/*.xml").each do |xml_file|
gdsn_doc = GDSNDoc.new(xml_file)
logger.info("Processing xml file #{xml_file}")
:x
#desc_exists = #gdsn_doc.xpath("//productData/description")
if !#desc_exists.empty?
row = []
headers.each do |col|
row << product[col]
end
out << row
end
end
end
The following code is not working to find the "description" element and to check whether it is blank or not:
#desc_exists = #gdsn_doc.xpath("//productData/description")
if !#desc_exists.empty?
Here is a sample of the XML file:
<productData>
<description>Chocolate biscuits </description>
<productData>
This is how I have defined the class and Nokogiri:
class GDSNDoc
def initialize(xml_file)
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
#doc.remove_namespaces!
The code had to be moved up to an earlier stage, where Nokogiri was initialised. It doesn't get runtime errors, but it does let XML files with blank descriptions get through and it shouldn't.
class GDSNDoc
def initialize(xml_file)
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
#doc.remove_namespaces!
desc_exists = #doc.xpath("//productData/descriptions")
if !desc_exists.empty?
You are creating your instance like this:
gdsn_doc = GDSNDoc.new(xml_file)
then use it like this:
#desc_exists = #gdsn_doc.xpath("//productData/description")
#gdsn_doc and gdsn_doc are two different things in Ruby - try just using the version without the #:
#desc_exists = gdsn_doc.xpath("//productData/description")
The basic test is to use:
require 'nokogiri'
doc = Nokogiri::XML(<<EOT)
<productData>
<description>Chocolate biscuits </description>
<productData>
EOT
# using XPath selectors...
doc.xpath('//productData/description').to_html # => "<description>Chocolate biscuits </description>"
doc.xpath('//description').to_html # => "<description>Chocolate biscuits </description>"
xpath works fine when the document is parsed correctly.
I get an error "undefined method 'xpath' for nil:NilClass (NoMethodError)
Usually this means you didn't parse the document correctly. In your case it's because you're not using the right variable:
gdsn_doc = GDSNDoc.new(xml_file)
...
#desc_exists = #gdsn_doc.xpath("//productData/description")
Note that gdsn_doc is not the same as #gdsn_doc. The later doesn't appear to have been initialized.
#doc = File.open(xml_file) {|f| Nokogiri::XML(f)}
While that should work, it's idiomatic to write it as:
#doc = Nokogiri::XML(File.read(xml_file))
File.open(...) do ... end is preferred if you're processing inside the block and want Ruby to automatically close the file. That isn't necessary when you're simply reading then passing the content to something else for processing, hence the use of File.read(...) which slurps the file. (Slurping isn't necessary a good practice because it can have scalability problems, but for reasonable sized XML/HTML it's OK because it's easier to use DOM-based parsing than SAX.)
If Nokogiri doesn't raise an exception it was able to parse the content, however that still doesn't mean the content was valid. It's a good idea to check
#doc.errors
to see whether Nokogiri/libXML had to do some fix-ups on the content just to be able to parse it. Fixing the markup can change the DOM from what you expect, making it impossible to find a tag based on your assumptions for the selector. You could use xmllint or one of the XML validators to check, but Nokogiri will still have to be happy.
Nokogiri includes a command-line version nokogiri that accepts a URL to the document you want to parse:
nokogiri http://example.com
It'll open IRB with the content loaded and ready for you to poke at it. It's very convenient when debugging and testing. It's also a decent way to make sure the content actually exists if you're dealing with HTML containing DHTML that loads parts of the page dynamically.
When I try to create an XML document with Nokogiri::XML::Builder:
builder = Nokogiri::XML::Builder.new do |xml|
xml.my_tag({key: :value})
end
I get the following XML tag:
<my_tag key="value"/>
It is self-closed, but I need the full form:
<my_tag key="value"></my_tag>
When I pass a value inside the node (or even a space):
xml.my_tag("content", key: :value)
xml.my_tag(" ", key: :value)
It generates the full tag:
<my_tag key="value">content</my_tag>
<my_tag key="value"> </my_tag>
But if I pass either an empty string or nil, or even an empty block:
xml.my_tag("", key: :value)
It generates a self-closed tag:
<my_tag key="value"/>
I believe there should be some attribute or something else that helps me but simple Googling didn't find the answer.
I found a possible solution in "Building blank XML tags with Nokogiri?" but it saves all tags as non-self-closed.
You can use Nokogiri's NO_EMPTY_TAGS save option. (XML calls self-closing tags empty-element tags.)
builder = Nokogiri::XML::Builder.new do |xml|
xml.my_tag({key: :value})
end
puts builder.to_xml(save_with: Nokogiri::XML::Node::SaveOptions::NO_EMPTY_TAGS)
<?xml version="1.0"?>
<my_tag key="value"></my_tag>
Each of the options is represented in a bit, so you can mix and match the ones you want. For example, setting NO_EMPTY_TAGS by itself will leave your XML on one line without spacing or indentation. If you still want it formatted for humans, you can bitwise or (|) it with the FORMAT option.
builder = Nokogiri::XML::Builder.new do |xml|
xml.my_tag({key: :value}) do |my_tag|
my_tag.nested({another: :value})
end
end
puts builder.to_xml(
save_with: Nokogiri::XML::Node::SaveOptions::NO_EMPTY_TAGS
)
puts
puts builder.to_xml(
save_with: Nokogiri::XML::Node::SaveOptions::NO_EMPTY_TAGS |
Nokogiri::XML::Node::SaveOptions::FORMAT
)
<?xml version="1.0"?>
<my_tag key="value"><nested another="value"></nested></my_tag>
<?xml version="1.0"?>
<my_tag key="value">
<nested another="value"></nested>
</my_tag>
There are also a handful of DEFAULT_* options at the end of the list that already combine options into common uses.
Your update mentions "it saves all tags as non-self-closed", as if perhaps you only want this single tag instance to be non-self-closed, and the rest to self close. Nokogiri won't produce an inconsistent document like that, but if you must, you can concatenate some XML strings together that you built with different options.
The XML file I am trying to parse has all the data contained in attributes. I found how to build the string to insert into the text file.
I have this XML file:
<ig:prescribed_item class_ref="0161-1#01-765557#1">
<ig:prescribed_property property_ref="0161-1#02-016058#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
<ig:prescribed_property property_ref="0161-1#02-016059#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
</ig:prescribed_item>
</ig:identification_guide>
And I want to parse it into a text file like this with the class ref duplicated for each property:
class_ref|property_ref|is_required|UOM_ref
0161-1#01-765557#1|0161-1#02-016058#1|false|0161-1#05-003260#1
0161-1#01-765557#1|0161-1#02-016059#1|false|0161-1#05-003260#1
This is the code I have so far:
require 'nokogiri'
doc = Nokogiri::XML(File.open("file.xml"), 'UTF-8') do |config|
config.strict
end
content = doc.xpath("//ig:prescribed_item/#class_ref").map {|i|
i.search("//ig:prescribed_item/ig:prescribed_property/#property_ref").map { |d| d.text }
}
puts content.inspect
content.each do |c|
puts c.join('|')
end
I'd simplify it a bit using CSS accessors:
xml = <<EOT
<ig:prescribed_item class_ref="0161-1#01-765557#1">
<ig:prescribed_property property_ref="0161-1#02-016058#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
<ig:prescribed_property property_ref="0161-1#02-016059#1" is_required="false" combination_allowed="false" one_of_allowed="false">
<dt:measure_number_type representation_ref="0161-1#04-000005#1">
<dt:real_type>
<dt:real_format pattern="\d(1,)\.\d(1,)"/>
</dt:real_type>
<dt:prescribed_unit_of_measure UOM_ref="0161-1#05-003260#1"/>
</dt:measure_number_type>
</ig:prescribed_property>
</ig:prescribed_item>
</ig:identification_guide>
EOT
require 'nokogiri'
doc = Nokogiri::XML(xml)
data = [ %w[ class_ref property_ref is_required UOM_ref] ]
doc.css('|prescribed_item').each do |pi|
pi.css('|prescribed_property').each do |pp|
data << [
pi['class_ref'],
pp['property_ref'],
pp['is_required'],
pp.at_css('|prescribed_unit_of_measure')['UOM_ref']
]
end
end
puts data.map{ |row| row.join('|') }
Which outputs:
class_ref|property_ref|is_required|UOM_ref
0161-1#01-765557#1|0161-1#02-016058#1|false|0161-1#05-003260#1
0161-1#01-765557#1|0161-1#02-016059#1|false|0161-1#05-003260#1
Could you explain this line in greater detail "pp.at_css('|prescribed_unit_of_measure')['UOM_ref']"
In Nokogiri, there are two types of "find a node" methods: The "search" methods return all nodes that match a particular accessor as a NodeSet, and the "at" methods return the first Node of the NodeSet which will be the first encountered Node that matched the accessor.
The "search" methods are things like search, css, xpath and /. The "at" methods are things like at, at_css, at_xpath and %. Both search and at accept either XPath or CSS accessors.
Back to pp.at_css('|prescribed_unit_of_measure')['UOM_ref']: At that point in the code pp is a local variable containing a "prescribed_property" Node. So, I'm telling the code to find the first node under pp that matches the CSS |prescribed_unit_of_measure accessor, in other words the first <dt:prescribed_unit_of_measure> tag contained by the pp node. When Nokogiri finds that node, it returns the value of the UOM_ref attribute of the node.
As a FYI, the / and % operators are aliased to search and at respectively in Nokogiri. They're part of its "Hpricot" compatability; We used to use them a lot when Hpricot was the XML/HTML parser of choice, but they're not idiomatic for most Nokogiri developers. I suspect it's to avoid confusion with the regular use of the operators, at least it is in my case.
Also, Nokogiri's CSS accessors have some extra-special juiciness; They support namespaces, like the XPath accessors do, only they use |. Nokogiri will let us ignore the namespaces, which is what I did. You'll want to nose around in the Nokogiri docs for CSS and namespaces for more information.
There are definitely ways of parsing based on attributes.
The Engine yard article "Getting started with Nokogiri" has a full description.
But quickly, the examples they give are:
To match “h3″ tags that have a class
attribute, we write:
h3[#class]
To match “h3″ tags whose class
attribute is equal to the string “r”,
we write:
h3[#class = "r"]
Using the attribute matching
construct, we can modify our previous
query to:
//h3[#class = "r"]/a[#class = "l"]
I have a simple task of accessing the values of some attributes. This is a simple script that uses Nokogiri::XML::Builder to create a simple XML doc.
require 'nokogiri'
builder = Nokogiri::XML::Builder.new(:encoding => 'UTF-8') do |xml|
xml.Placement(:messageId => "392847-039820-938777", :system => "MOD", :version => "2.0") {
xml.objects {
xml.object(:myattribute => "99", :anotherattrib => "333")
xml.nextobject_ '9387toot'
xml.Entertainment "Last Man Standing"
}
}
end
puts builder.to_xml
puts builder.root.attributes["messageId"]
The results are:
<?xml version="1.0" encoding="UTF-8"?>
<Placement messageId="392847-039820-938777" version="2.0" system="MOD">
<objects>
<object anotherattrib="333" myattribute="99"/>
<nextobject>9387toot</nextobject>
<Entertainment>Last Man Standing</Entertainment>
</objects>
</Placement>
C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/document.rb:178:in `add_child': Document already has a root node (RuntimeError)
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/node.rb:455:in `parent='
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/builder.rb:358:in `insert'
from C:/Ruby/lib/ruby/gems/1.8/gems/nokogiri-1.4.2-x86-mingw32/lib/nokogiri/xml/builder.rb:350:in `method_missing'
from C:/Documents and Settings/etrojan/workspace/Lads/tryXPATH2.rb:15
The XML that is generated looks fine. However, my attempts to access attributes cause an error to be generated:
Document already has a root node
I don't understand why puts would cause this error.
Using Nokogiri::XML::Reader works for your example, but probably isn't the full answer you are looking for (Note that there is no attributes method for Builder).
reader = Nokogiri::XML::Reader(builder.to_xml)
reader.read #Moves to next node in document
reader.attribute("messageId")
Note that if you issued reader.read again and then tried reader.attribute("messageId") the result will be nil since the current node will not have this attribute.
What you probably want to do is use Nokogiri::XML::Document if you want to search an XML document by attribute.
doc = Nokogiri::XML(builder.to_xml)
elems = doc.xpath("//*[#messageId]") #get all elements with an attribute of 'messageId'
elems[0].attr('messageId') #gets value of attribute of first elem
Here is a slightly more succinct way to access attributes using Nokogiri (assuming you already have your xml stored in a variable called xml, as covered by #atomicules' answer):
xml.xpath("//Placement").attr("messageId")