I'm developing a plugin in Jekyll that inserts a new Liquid Tag (a block tag) named latex. It's purpose is to insert a block of LaTeX source code inside a post source file this way:
... post file contents ...
{% latex density=300 usepackages=pstricks-all, %}
\pspicture(5,5)
\psframe(0,0)(5,5) \psline(0,0)(5,5) \psline(0,5)(5,0)
\endpspicture
{% endlatex %}
... post file contents ...
The output post will contain a <img> tag instead of the Liquid Tag block once compiled and the LaTeX source will be compiled by a chained execution of latex, dvips, convert. So it will depend on external programs (TexLive and ImageMagick).
I've not found any issue with the explained so far. I can take the block of LaTeX, put it in a temporary file, and finally compile it to PNG. The rendered files must be put in the same folder of the post.
But just there I'm stuck: I want to put the generated PNG image to this folder inside the destination output folder. No problem defining a site-config variable to define the folder (in fact, I do that), nor put a file there. The problem is simple: I can write to the source folder, but the generated files will not be copied to the destination folder.
I know the reason: Jekyll generates first and renders afterwards. The copy process happens before the render part, so the generated files will not be copied.
I've found this SO entry: "How to generate files from Liquid blocks in Jekyll?", but the answer doesn't feel right: You must make a two-pass build for the files to be copied.
Also found "Generate file inside _site with Jekyll plugin", but this is not applicable because I don't want to render documents by templates.
The solution I've been planning is this:
Keep a folder with all the generated files and some index strategy for their final placement.
Implement a Sitewide method for the final placement procedure, something like:
class Site
def generate_latex_adds
// Code that copies the generated files to destination folders
end
end
Add self.generate_latex_adds calling that method inside the site_process.rb script just before the self.cleanup and self.write calls.
This will probably solve the problem, but I feel wrong to modify the site_process.rb. I'm considering pull this Liquid Tag to the community as a GitHub project, and as such, I must document this manual edit of site_process.rb to the users of this plugin. I know it's probably a common situation and a common solution, but I wonder if there is a better way to pospone the copy of the generated files without modifying core files. I'd like to keep the plugin simple: just copy the plugin file to the _plugins directory.
Any ideas?
EDIT: I'm more interested in the reasoning that on the actual code. I want to learn, not asking for a code solution.
As anyone has answered my question up to now, and I had kept investigating the issue. Finally I've got an elegant solution: use the Jekyll::StaticFile class. I have missed that class (sidenote: Read more carefully the docs).
When you add one object of this class to the site.static_files array, you are marking this file as pending for copy after the render process is completed. In fact, the copy of such files is done in the site.write process. Take a look at the site_process.rb file in your Jekyll installation.
The usage of this class is easy. When you need to mark a file for future copy, you simply execute a code like this:
site.static_files << Jekyll::StaticFile.new(site, site.source, path, filename)
Where path and filename depends on the location of your file in the src folder.
And that's all: You generate the files in the src folder, mark them as pending for copy and let Jekyll do the rest. No modification of site_process.rb at all!
You can take a look at the resulting LaTeX -> PNG liquid tag code at GitHub: https://github.com/fgalindo/jekyll-liquid-latex-plugin
I've also implemented a cleanup method there to eliminate orphaned generated files from posts that have been modified and rebuilt.
Related
Using a simple template block, is there a way to copy all template files in a templates/default/SubDir/ (templates/default/SubDir/*) to a single location (/tmp/)?
Here is an example block:
template '/tmp/...' do
source 'default/SubDir/...'
owner node['attribute']['os_user']
group node['attribute']['os_group']
mode '0755'
end
EDIT: these are all xml files and I assume that I would not be adding the .erb extension to the template files, as that would require me to mention each template file name so that i can revert them back to .xml (as apposed to recursively copying the templates to the server).
NOTE: If this is not supported, then is there any recommended approach requiring strait ruby code?
EDIT: I am going to try making an aray of template files and see if that would work if I were to copy the entire array to the node.
I solved this problem by creating an array with the objects(templates) I needed to copy to the server.. it looks something like this:
auth_schemas = ['file1.xml','file2.xml','file3.xml','file4.xml','file5.xml','file6.xml','file7.xml','file8.xml','file9.xml','file10.xml']
auth_schemas.each do |schemaFile|
template "/tmp/#{schemaFile}" do
source "path/to/SubDir/#{schemaFile}"
owner node['attribute']['os_user']
group node['attribute']['os_group']
mode '0755'
end
end
Worked Like a charm!
No, that is not supported. Some additional characters because SO forces me to.
In a Maven project with subprojects, each subproject gets an index.html with some content that comes from its POM's description element.
In one of these subprojects, I need that content to contain additional information, including links. There is a section of the doc that suggests I should not do it by trying to put HTML markup in CDATA in the description element (in fact, that doesn't work anyway; the HTML markup just comes out literal). Instead, it suggests there is some better way to get my own content included in the file.
While this element can be specified as CDATA to enable the use of HTML tags
within the description, it is discouraged to allow plain text representation.
If you need to modify the index page of the generated web site, you are able
to specify your own instead of adjusting this text.
Can anyone describe how to do that? I have tried several methods unsuccessfully (I can supply Markdown files with other names and they generate HTML, but a subproject's index.md has no effect on the generated index.html). I have also read about the custom element in site.xml but it seems to require writing a custom Velocity template for the site; I hope the passage "you are able to specify your own" must mean there is some method more straightforward than that.
Of course I would also appreciate a pointer into the docs if there is already an answer I have simply failed to find. (Just pointing me to docs I've already read isn't in itself helpful, though pointing out the answer I missed would be helpful, if it's there.)
In response to inquiries
Directory structure under src/site:
src/site
src/site/resources
src/site/resources/images
src/site/markdown
src/site/markdown/use
src/site/markdown/install
src/site/markdown/examples
src/site/markdown/build
maven-site-plugin version: 3.4
What I mean by 'adding a link':
The part of the index.html that comes from the POM description element
is the central content of the page (not the navigation bar, not the sidebar menus, but the actual content).
I would like that actual-content portion of the page to be able to have a paragraph or two explaining that this is a generated page for developers, and providing links (HTML <a href=...>) for people who arrived at the page from a web search but are really looking for the user-oriented pages.
I can't put that in the description element (even using CDATA), because HTML elements just come out literal. A comment below gives a link to a page on writing a whole custom Velocity template for the site, but is there honestly no simpler way to accomplish this?
I have the same issue. The only thing the generated index.html gives you of value is the list of modules. You can add your own index.md page to src/site/markdown, putting in whatever content you want. To reproduce the list of modules, include something like this:
###Project Modules
This project has declared the following modules:
| Name | Description |
|-|-|
|[Module1 name](module1/index.html)| Module 1 description|
|-|-|
|[Module2 name](module2/index.html)| Module 2 description|
Of course the text is not lifted from the POM. You also have to manually change this file if you have a new module. Not a perfect solution, but the best I could come up with.
Where I wrote:
I can supply Markdown files with other names and they generate HTML,
but a subproject's index.md has no effect on the generated index.html
it turns out the truth is more complex. In a project with subprojects,
there are two places such an index.md might go: in src/site/markdown/subproject-name of the parent project (where all of the other human-written docs for the whole project happen to be), or in a new src/site/markdown directory created within the subproject. A file with any other non-special name can be added in either place, and end up where you expect it in the target. But not for index.md, in that case only the second location can work, and even then, only after a clean.
I had tried both places without success, but trying the second again with a full clean install site site:stage makes it work. Out of the four combinations (parent/clean, parent/noclean, sub/clean, sub/noclean), that was the one I missed trying before posting the question, so of course that's the one that works. :)
If there had been an answer or comment like "hmm, are you sure an index.md in the subproject doesn't work, it works for me?" it probably would have put me quickly back on track. Sometimes after trying several avenues all without success, all that's needed is to know which of them is the one that's supposed to work (if indeed one of them is) and therefore worth spending more time on.
I have a project which have a lot of useful docs outside of the src directory which I'd like to render as usual DocPad documents.
Examples:
Files at the root of the project: README.md, LICENSE, Contributing.md and similar, which are already there and can be used in things like GitHub. I would like to reuse the content from those files to create the corresponding readme, license and contributing pages, or to include the contents from those files somewhere in layout or a document.
I have a project that have some docs inside, and I'd like to render the .md files as DocPad documents from it by including it in package.json, so those files would be in node_modules at root.
In both those cases there are files outside of the src/documents that I'd like to use as partials or documents, and it seems that the partial plugin can't help me (or I couldn't find a way to make it do what I need), and the #getCollection can only get things from the src/documents.
So, the question is: Is there a way I can tell DocPad to treat some of the files/folders from the outside of the src folder? Do I miss something?
If not, then what would be the best way to do it as a plugin, which direction should I dig?
DocPad natively supports storing documents outside of the default src folder. The way you do this is via the documentsPaths config option in the DocPad Configuration File (e.g. docpad.coffee). Something like this:
path = require('path')
docpadConfig = {
documentsPaths: [
'documents'
path.resolve('..','data','documents')
]
....
Of course, where this will fall down is if you want to just include arbitrary, individual files somewhere on the file system. In such cases symlinks would be the way to go.
Template helpers can also be used for this, as they can do whatever you want.
For instance, the Bevry Learning Centre website uses template helpers to render arbitrary files by relative paths as code examples:
Template Helper: https://github.com/bevry/learn/blob/6e202638f2321eec2633d1dbeaf1078bdb953562/docpad.coffee#L244-L257
Tempalte using the Template Helper: https://github.com/bevry/documentation/blob/f24901251d19ec1cfa56fcee14c2c6836c0a995c/node/handsonnode/03-server.html.md.eco
If you would also like to render them, you could combine such a solution with the Text Plugin.
The combination would be like so:
Template Helper in DocPad Configuration File:
docpadConfig =
templateData:
readProjectPath: (relativePath) ->
fullPath = require('path').join(__dirname, relativePath)
return #readFullPath(fullPath)
readRelativePath: (relativePath) ->
fullPath = #getPath(relativePath)
return #readFullPath(fullPath)
readFullPath: (fullPath) ->
result = require('fs').readFileSync(fullPath)
if result instanceof Error
throw result
else
return result.toString()
Usage of Template Helper with Text Plugin using Eco as templating engine:
<t render="markdown"><%- #readProjectPath('README.md') %></t>
The answer would be a rather simple one: relative symbolic links. Docpad handles them perfectly.
This way, to have a symlink of README.md inside your documents, you should do this (with pwd of src/documents):
ln -s ../../README.md readme.html.md
Or, in case of a docs from inside one of the project's modules:
ln -s ../../node_modules/foobar/docs/ docs
Both those variants work perfectly.
Note: Symlinks can be tricky. Refer to these for some common gotchas:
https://github.com/docpad/docpad/issues/878#issuecomment-53197720
https://github.com/docpad/docpad/issues/878#issuecomment-53209674
Just as a comparison between the different answers:
Use the paths solution when you want to add entire extra directories to be included inside the DocPad database to be treated as normal by DocPad.
Use the sym/hard link solution when you want to include specific documents or files that you would like to be treated as a DocPad document or file, with all the intelligent file parsing and document rendering, including layout, database, and caching features.
Use the template helper solution when you want to include specific files that you do not want included in the DocPad database for whatever reason.
Made this answer a community wiki one, so it can be updated accordingly for new answers and better details.
I am rebuilding a site with docpad and it's very liberating to form a folders structure that makes sense with my workflow of content-creation, but I'm running into a problem with docpad's hard-division of content-to-be-rendered vs 'static'-content.
Docpad recommends that you put things like images in /files instead of /documents, and the documentation makes it sound as if otherwise there will be some processing overhead incurred.
First, I'd like an explanation if anyone has it of why a file with a
single extension (therefore no rendering) and no YAML front-matter,
such as a .jpg, would impact site-regeneration time when placed
within /documents.
Second, the real issue: is there a way, if it does indeed create a
performance hit, to mitigate it? For example, to specify an 'ignore'
list with regex, etc...
My use case
I would like to do this for posts and their associated images to make authoring a post more natural. I can easily see the images I have to work with and all the related files are in one place.
I also am doing this for an artwork I am displaying. In this case it's an even stronger use case, as the only data in my html.eco file is yaml front matter of various meta data, my layout automatically generates the gallery from all the attached images located in a folder of the same-name as the post. I can match the relative output path folder in my /files directory but it's error prone, because you're in one folder (src/files/artworks/) when creating the folder of images and another (src/documents/artworks/) when creating the html file -- typos are far more likely (as you can't ever see the folder and the html file side by side)...
Even without justifying a use case I can't see why docpad should be putting forth such a hard division. A performance consideration should not be passed on to the end user like that if it can be avoided in any way; since with docpad I am likely to be managing my blog through the file system I ought to have full control over that structure and certainly don't want my content divided up based on some framework limitation or performance concern instead of based on logical content divisions.
I think the key is the line about "metadata".Even though a file does NOT have a double extension, it can still have metadata at the top of the file which needs to be scanned and read. The double extension really just tells docpad to convert the file from one format and output it as another. If I create a straight html file in the document folder I can still include the metadata header in the form:
---
tags: ['tag1','tag2','tag3']
title: 'Some title'
---
When the file is copied to the out directory, this metadata will be removed. If I do the same thing to a html file in the files directory, the file will be copied to the out directory with the metadata header intact. So, the answer to your question is that even though your file has a single extension and is not "rendered" as such, it still needs to be opened and processed.
The point you make, however, is a good one. Keeping images and documents together. I can see a good argument for excluding certain file extensions (like image files) from being processed. Or perhaps, only including certain file extensions.
I'm building a site with Nanoc and have one file (with it's own layout file) that reads all other files metadata, does some magic and saves the file. Unfortunately, this process takes a long time and developing other views with autocompile is taking ages. Is it possible to add some files to ignore list that won't be enqueued to compilation (and during compilation to production remove them from ignores)? Or are there other methods to achieve this?
To the best of my knowledge, nanoc will always read all data from your contents directory. You can remove files from further processing by implementing a preprocess method which removes some generated items entries... e.g. in Rules:
preprocess do
skip_unimportant_items
end
In a .rb file in lib/:
def skip_unimportant_items
#items.delete_if { |i| !i[:important] }
end
This will remove all items which have no important element (or the element set to false) in their respective metadata.
Did you tried the Nanoc Doc recommendation:
ignore '/assets/layouts/your_files/*/'
http://nanoc.ws/doc/rules/