I have the following Ruby code processing a Markdown file:
html = Markdown.new(markdownContents).to_html
The result of this includes a comment block at the top of the file, which I would like it not to do, but I can't seem to find an option in the documentation to not render this comment block.
<!-- === begin markdown block =====================================================
generated by markdown 1.0.0 on Ruby 1.9.3 (2012-04-20) [i386-mingw32]
on 2012-08-15 15:34:51 -0400 with Markdown engine kramdown (0.13.7)
using options { !to be done! }
-->
Is there an option for this, or would I have to submit a pull request? ;)
The code to add this header seems to have been added in this commit and only applies to the kramdown engine. It seems the solutions are:
to submit a pull request with code to make this optional
configure Markdown to use a different engine. Details here in the Markdown Engine Loading Order section
Related
We have an internal python API that we have documented using Sphinx. part of the system uses YAML for configuration files that contain reference to py file that use the API. I've been asked to see if there is a way, using sphinx, to link the YAML config files to the the appropriate API docs. I've been doing research on google, here and sphinx site and it looks like I may be able to use intersphinx but I'm unclear how to make the connection between the two.
so for example: here is the yaml config file:
HALT_LEVEL: Any
SUITE: "Checkin Tests"
DESCRIPTION: "checkin test suite"
TESTLIST:
- TESTCASE: install stuff
DESC: "Installs RPM"
TESTGROUP: sprint_0
TESTFILE: install_stuff.py # I would like to turn this into a link to our sphinx docs
# for this. This file is already part of sphinx docs"
So then when someone is looking at the html/sphinx version of the the above file they could click on install_stuff.py and it take them to the docs that exist
Is this possible?
thanks in advance,
Greg.
Just for those that may be interested. I was able to do this but not using Sphinx. I used pyyaml to read the file and pygments to generate the html then hacked the generated html.
Since pygments doesn't appear to allow live href links what to add them what I is
# create a yaml string using pyyaml and then modify the string.
href = '[ahref="%s"]%s[/a]' % (hrefData, hrefString)
yamldata['KEYWORD'][idx]['HOST'] = href
This changes the referenced yaml to a string that looks something like '[ahref="http://example.com"]Example.com[/a]'. The brackets make it easier to change after the html is generated. Further down in the code I use pygments to generate the html
htmlpage = highlight(yamldata,lexer, HtmlFormatter(full=True, title=yamldata['TITLE']))
Now I convert the href that I created above to a real href with:
webpage = htmlpage.replace(''', '').replace('ahref', 'a href').replace('[','<').replace(']','>').replace('"','"')
Following instructions on this page by Google's Oauth for API access guide. It requires that I run a terminal command to generate a yaml to be used for Oauth. However, the command no longer runs. The gem has removed the functionality, and I tried using the previous version of the gem, but there are conflicting versions in the gem dependencies which keep it from running. Any ideas on how I could generate this yaml file another way? I know how to download the json file with my client_id, can I simply conver the json to a yaml?
I might stick to the newer version of the Gem (for stability reasons), and just do the conversion yourself. I've had success with the JSON-YAML conversion with this, using both the JSON and YAML modules:
require 'json'
require 'yaml'
json = '{ "boo": "bop" }'
data = JSON.parse(json)
yml = YAML::dump(data)
My rails app fetches a bunch of xml feeds once a day, loads them into the db and then displays them in aggregate. I'm thinking that I can save on server memory if I just output the pages as static files and let them be served directly by the front-end server (nginx in my case). I asked in an IRC room and was told to not use rails and create the files using rake tasks. However, I'm wondering what the easiest way would be to go about doing this. Layout, asset files and content are in different places in rails obviously, so I guess I would need to combine the layout and content and then insert the css/javascript.
Any thoughts/ideas are welcome.
[Solved]
I ended up using the examples from render_to_string from a rake task and made some tweaks to get the following code inside my rake task:
views_path = Rails.root.to_s + "/app/views"
av = ActionView::Base.new(views_path)
av.class_eval do
include ApplicationHelper
end
products = Product.all
a = av.render(:template => "products/show", :layout => "layouts/application", :locals => { :#products => products } )
This then renders both the template and the layout, and allows the use of the #products instance variable inside the template just as you would if you were using a controller.
Then I just need to write the output of the render to a file.
For a task like this you can use Rails' built in caching mechanisms.
There is another stack overflow post which shows some example code of how to build code to write that cache manually from something like a rake task.
Perhaps middleman or jekyll could be used?
I've only used middleman, but you could use a rake task and support script to get the latest xml feeds and stick that into middlemans data dir (i.e. data/feeds.yml), then use your existing layouts to render that yaml file. Middleman and rails share a lot of similar tech for rendering etc.
You'd have to modify your layouts a little bit.
You could probably find gems to replace yaml with something else if you wanted.
What are the screen scraping options in Rails 3 - gem/library? I have used Nokogiri in the past but just wanted to know if there are better options in Rails 3.
If this is a one-off task or if your target data set is relatively small (under a hundred of pages), use Mechanize (browse & scrape) or Anemone (does whatever Mechanize does + some additional crawling-specific options).
If you need to automate this collection or if you are dealing with large data sets, consider using a web service. Bobik is a good choice in this bucket.
Rails doesn't do screen scraping. You are free to use Ruby code that would add that functionality, but by itself it does the generation of the pages.
Mechanize, which uses Nokogiri internally, is a good choice, otherwise I always roll my own using Nokogiri and OpenURI.
In the fantastic RubyTools website you can find several Ruby libraries to parsing HTML. Still Nokogiri is the most popular.
You can also use the Scrapifier gem to get metadata from URIs found in a string. It's very simple to use:
'Wow! What an awesome site: http://adtangerine.com!'.scrapify
#=> {
# title: "AdTangerine | Advertising Platform for Social Media",
# description: "AdTangerine is an advertising platform that uses the tangerine as a virtual currency for advertisers and publishers in order to share content on social networks.",
# images: ["http://adtangerine.com/assets/logo_adt_og.png", "http://adtangerine.com/assets/logo_adt_og.png", "http://s3-us-west-2.amazonaws.com/adtangerine-prod/users/avatars/000/000/834/thumb/275747_1118382211_1929809351_n.jpg", "http://adtangerine.com/assets/foobar.gif"],
# uri: "http://adtangerine.com"
# }
Intro: I'm working on a gem that, by default, will pull information out of some XML data and do some sort of processing on the document. I'm using nokogiri to parse the XML. However, I wish to allow the user to parse the XML themselves and pass in the necessary information for my data processing methods to run, in case they don't want to install nokogiri or have already parsed the XML.
Question: Is there any way to allow the user to specify, during gem installation, that they don't wish to install the nokogiri dependency? For example (very hand-wavey here),
gem install crazy_gem --no-nokogiri
and in the gemspec perhaps something like
Gem::Specification.new do |s|
...
s.add_dependency 'nokogiri' unless Proc.new { install_flags('no-nokogiri') }
...
end
[edit] I don't want to focus too much on the gemspec code above, as I know it doesn't work--it's just an example of the kind of thing I want to do. [/edit]
gem install crazy_gem --ignore-dependencies works great until there are additional dependencies.
I don't think you can do exactly what you're after, but there's apossible solution if you reframe what your gem does. Rather than a gem that by default parses some XML and processes the data, but optionally allows you to pass in the pre-parsed data, what about a gem that is mainly concerned with the processing, but optionally will parse the XML for you (if you have Nokogiri).
To do this just leave Nokogiri out of your gemspec dependencies (you could add it as a development dependency or a requirement).
Inside your code, make sure you only call require 'nokogiri' in a begin..end block with a rescue LoadError and deal with it as appropriate.
Gemspecs are turned into static files at build time, so that wouldn't work. You could try using -f which bypasses dependency checks.