Programmatically get markdown files from external repositories and convert them to pages - docpad

Hey I know I can use the docpad-plugin-downloader plugin to grab markdown files from external repositories.
However, I am unsure how I can conver these grabbed md files so that they can render as a page in docpad.
What I am trying to do is setup a documentation site using docpad and it is working great. However I would love it if docpad could grab my readme files from my external repositories and convert them to actual pages, so I would only have to update my readme files to update my documentation site.
Was wondering if you had any ideas? :)

Related

Repack extracted .img file that I just extracted with binwalk

I used "binwalk" to extract an image file and I got what I needed as a directory hierarchy. So I made my changes and now I want to repack it and get the .img file back out of it.
Is there a solution?
I found a solution to accomplish this, I used imgRePacker_203 to get my image extracted.
If you could not find imgRePacker_203 online, then I have created a Git repo on my Github account and I included that tool with many other tools related to work with .img files. Also there is a very useful video on which you can get all links here:
https://github.com/AhmadVakil/Image-file-tools
https://www.youtube.com/watch?v=3zm91DfdkIo
Note: This solution is worked with ALLWINNER, and Whatsminer image that upgrade or downgrade the firmware of ASIC miner's control board. Hope you find it useful for other problems and extracting other images.

Using Vimwiki to embed images to a wiki that needs to be transferable between systems

I am building an internal project wiki for a group software development project. The project wiki is currently powered by VimWiki and I send the HTML files to both the project supervisor and each of the development team on a weekly basis. This keeps our Intellectual property secure and internal, but also organized and up to date. I would like to put diagram images into the wiki itself so that all diagrams and documentation can be accessed together with ease. I am however having trouble making the images transferable between systems. Does vimwiki give a way for image files to be embedded such that they can be transferred between systems? Ideally the solution would make it possible to transfer the output directory of the Vimwiki as a singular entity containing the HTML files and the image files.
I have tried reading the documentation on images in the vimwiki reference document. I have not had luck using local: or file: variants. The wiki reference states that local should convert the image links to a localized location based on the output directory of the HTML files, but it breaks my image when I use it.
I have currently in my file
{{file:/images/picture.png}}
I expect the system to be able to transfer the file between computers but it registers to an absolute link and also does not include the image directory in the output directory of the vimwikiAll2HTML command.
I know this is an old question, but try to use {{local:/images/picture.png}} instead. If you open :help vimwiki in Vim, you can find a part that says:
In Vim, "file:" and "local:" behave the same, i.e. you can use them with both
relative and absolute links. When converted to HTML, however, "file:" links
will become absolute links, while "local:" links become relative to the HTML
output directory. The latter can be useful if you copy your HTML files to
another computer.

Download a working local copy of a webpage as a single html file

I followed the instructions provided in this previous post. I am able to download a working local copy of the webpage (e.g. wget -p -k https://shapeshed.com/unix-wget/) but I would like to integrate all the files (js, css and images e.g. using base64 encoding) into a single html file (or another convenient format). Would this be possible?
Try using HTTrack
It is very efficient and easy to use website copier. All you have to do is paste the link of the website you want to make a local copy of
Follow these steps as you want everything in single page
Minify all the stylesheets and put them in <style> in your main
HTML page use CSS minifier
Minify all the scripts and put them inside <script> in the same file. Use JavaScript Minifier
To deal with images use spites
It certainly can be done. But you’ll have to do couple of simple things manually, since there are no available tools to automate some of the steps.
Download the web page using Wget with all dependencies.
Copy the contents of linked stylesheets and scripts to main HTML file.
Convert images to Base64 data URIs contained in HTML and CSS, then insert them to main HTML file.
Minify the edited HTML file.
Convert HTML file to Base64 data URI.
Here is an example of a single-page application encoded to Base64 data URI created to demonstrate the concept (copy and paste below code to web browser address bar):
data:text/html;charset=utf-8;base64,PCFkb2N0eXBlIGh0bWw+DQo8aHRtbCBsYW5nPSJlbiI+DQoJPG1ldGEgY2hhcnNldD0idXRmLTgiPg0KCTx0aXRsZT5TaW5nbGUtcGFnZSBBcHBsaWNhdGlvbiBFeGFtcGxlPC90aXRsZT4NCgk8c3R5bGU+DQoJCS8qIENvZGUgZnJvbSBDU1MgZmlsZXMgZ29lcyBoZXJlLiAqLw0KCQlib2R5IHsNCgkJCWZvbnQtZmFtaWx5OiBzYW5zLXNlcmlmOw0KCQl9DQoJCWJ1dHRvbiB7DQoJCQlkaXNwbGF5OiBibG9jaw0KCQl9DQoJPC9zdHlsZT4NCgk8c2NyaXB0Pg0KCQkvLyBDb2RlIGZyb20gLmpzIGZpbGVzIGdvZXMgaGVyZS4gDQoJCWZ1bmN0aW9uIGNoYW5nZVBhcmFncmFwaCgpIHsNCgkJICAgIGRvY3VtZW50LmdldEVsZW1lbnRzQnlUYWdOYW1lKCJwIilbMF0uaW5uZXJIVE1MID0gIkNvbnRlbnQgb2YgcGFyYWdyYXBoIGNoYW5nZWQuIjsNCgkJfQ0KCTwvc2NyaXB0Pg0KCTxib2R5Pg0KCQk8aW1nIHNyYz0iZGF0YTppbWFnZS9wbmc7YmFzZTY0LGlWQk9SdzBLR2dvQUFBQU5TVWhFVWdBQUFVQUFBQUR3QkFNQUFBQ0RBNkJZQUFBQU1GQk1WRVZVVmx1T2o1TC8vLzlrWm1xbXA2bUJnb1dhbTUyeHNyTnpkSGk4dkw3dDdlNzI5dmJHeDhqazVPVFEwZExhMnR2SHNtSDhBQUFDSjBsRVFWUjRBZXpCZ1FBQUFBQ0FvUDJwRjZrQ0FBQUFBQUFBQUFBQUFBQUFBQUFBWUExdElLU2twRERxUUdMQXFBTkhIY2dzSWd3a3d4SUJ6SllCaEJSaEdJYmZiWGZiMWUzcU5vRUU5NVN1bTJuM1Z1SndNSHNRa0FGUVpBVUF4bDA2UU9zRXVNaENDTWNRQVRFWEJhaURBOGdFSUpJQXNKYUFNdmsrVGdrQTVuL2cvN3p2NE9HYitZMmN4djdqVkVaMzRLZG5kNStrTlFudXd1b2NNbDJCOTVZZUZoRHZTVHFmRTAwdldhV3RBcUtrTnNHcndFWUw0S1BrSjNFcW5WanNndTBTWURTdVM5Qk1lQUN3WnFGenJBN0dyZ2x1NHl6cUVuUnlnSkdVdzlzU050ekt5YlNFelNXczF5VzR1WjhEcDY4QXRlR1dXaEJaTVp6TWdhd0J3M0d6SkI3WEpQaFoyN0N1aGd0VzFVSXFRVXY0WXFwa1BiZ21IVUJTazJDaUh0ejA3Y294T1JVdzlTbTdBQXVwRHkvcXVtYlVzY20xcEdkSHZ3RUVTRlpuNTNCZ0VZTGdJUTVOd0o4aHV4MlNZTHZBUVlFS1hvVG81YVQ4ZjhXZkJrWWFnT0FCTEh4U0RvbFVRcllDMytUVUwrZ3JWYk1BZlljM1Z2ZzFjeXoxcWlvTFEvQ0RuZ042QlBGcGVYWlJ6NXB6U0FJUVhBRytBcWlQVVVCbXhYQUprUUlRN0dEa1o5OXp2UFBQejhKYUNJSTZBYTc3ZEI5NDdlOWt0d1NJVjRNUWJPV01VcDkwci9veGRrRjFjb2oyRkFiZHdWaC9zUlZiZUhreVUyQThyYXBVV3NKVVliSUQ3MllQSVZhZzlNRzVvVUJwbGppSlFtVUw0NmZDNWM1UjlldFBlM0FnQUFBQWdBQm83UEZYR0tCcUFBQUFBQUFBQUFBQUFBQUFBQUFBQUxnTmtYVy9TUloxSldBQUFBQUFTVVZPUks1Q1lJST0iIGFsdD0iIj4NCgkJPGgxPlNpbmdsZS1wYWdlIEFwcGxpY2F0aW9uIEV4YW1wbGU8L2gxPg0KCQk8cD5UaGlzIGlzIGFuIGV4YW1wbGUgb2YgYSB3ZWIgYXBwIHRoYXQgaW50ZWdyYXRlcyBIVE1MLCBDU1MsIEphdmFTY3JpcHQsIGFuZCBhbiBpbWFnZSBpbnRvIG9uZSAuaHRtbCBmaWxlIHRoYXQgaXMgZW5jb2RlZCB0byBCYXNlNjQuPC9wPg0KCQk8YnV0dG9uIHR5cGU9ImJ1dHRvbiIgb25jbGljaz0iY2hhbmdlUGFyYWdyYXBoKCkiPkNoYW5nZSBQYXJhZ3JhcGg8L2J1dHRvbj4NCgk8L2JvZHk+DQo8L2h0bWw+
Another solution would be to use a web proxy with a custom extension in order to store the sources, cf. https://github.com/SommerEngineering/WebProxy
This GitHub project is a simple web proxy by me, written in Go. Inside the Main.go line 71 and beyond will copy any data from the original site to your browser.
In your case, you would add a query if the data is already stored or not. If so, load from disk and send it to your browser. If not, load it from the source and store it to the disk.
Your condition of using a singe-file storage would not be an issue: Go can read and write e.g. ZIP files, cf. https://golang.org/pkg/archive/zip/. If you need these web site dumps immediately, a bit of code is needed to follow all links in order to store anything now.
Therefore, this answer is not a ready-to-go solution to your question. Rather, it would need to code a little bit. Go code could be compiled to any platform (x86, ARM, PPC) and operating system (Linux, macOS, Windows).
Hope, this answer gives an option for you.
There is a Chrome extension SingleFile that does exactly this

Middleman - Import source files from an external git

I just started using Middleman and I love it! unfortunately my needs are a little different from the stock configuration and I couldn't find any Middleman extensions to help with this so I thought maybe there's a simpler solution before I start writing my own extension.
Here's the scenario:
Most of my source files (like templates, js and css) are stored within the source directory but the real textual content is in another git repository.
Basically I want Middleman to fetch these files before build and integrate them as regular source files for the static site.
I figured I can make this work by running something like git clone on a before-build hook and then copy the needed files to the source directory.
Can anyone maybe suggest an extension that's already doing it? or a configuration option I can set in middleman to get the same behavior?
Thank's

Including template files in Go executable

I want to parse a folder of template HTML files and cache (or buffer) them on build for later use in a web project. Basically I am trying to make it so when I build my project I dont need to take the HTML files with me when I upload the executable file.
I think maybe you are talking about embedding the html files in your binary?
If so the best way to do this that I've seen so far has been camlistores fileembed: https://code.google.com/p/camlistore/source/browse/pkg/fileembed/fileembed.go
See here for directions on use: https://groups.google.com/d/msg/golang-nuts/LQMv7Zsmsi0/8Aax1XSekjAJ
That thread has addtional info on other packages for this as well.

Resources