I am trying to made a code block on my Jekyll website but I cannot preserve whitespace properly.
I am using a YAML file in my _data file to display certain data on my website.
Here is what I have in my YAML file:
Sample Title:
type: Script
description: "Description of script"
code: "for(double i = Start; i < End; i += 1){
display();
}"
And in one of my Jekyll layouts (titled: card.html) I have:
<div class="column">
<div class="content">
{{page.card.description | markdownify}}
<h2>Script</h2>
<pre><code>{{page.card.code}}</code></pre>
</div>
</div>
I have kramdown installed and on my _config.yml
When I do this, whitespace is not preserved on my website. I get this, in a codeblock:
for(double i = Start; i < End; i += 1){
display();
}
However, whitespace in-between text and leading whitespace on the first line is preserved. If I do this with my YAML file, the whitespaces in-between text and the leading whitespace on the first line is preserved.
Sample Title:
type: Script
description: "Description of script"
code: " for(double i = Start; i < End; i += 1){
display();
}"
My website will look like this, note the preserved whitespace but not on the second line:
for(double i = Start; i < End; i += 1){
display();
}
I am wondering what I am doing wrong and how do I preserve leading whitespaces on every line?
You can use YAML's literal style, which preserves whitespace. A minimal example of a complete page could look like
---
layout: home
code: |
for(double i = Start; i < End; i += 1){
display();
}
---
<h2>Script</h2>
<pre><code>{{page.code}}</code></pre>
resulting in a code block
for(double i = Start; i < End; i += 1){
display();
}
Related
I have several text files (utf-8) that I want to process in shell script. They aren't excactly the same format, but if I could only break them up into edible chunks I can handle that.
This could be programmed in C or python, but I prefer not.
EDIT: I wrote a solution in C; see my own answer. I think this may be the simplest approach after all. If you think I'm wrong please test your solution against the more complicated example input from my answer below.
-- jcxz100
For clarity (and to be able to debug more easily) I want the chunks to be saved as separate text files in a sub-folder.
All types of input files consist of:
junk lines
lines with junk text followed by start brackets or parentheses - i.e. '[' '{' '<' or '(' - and possibly followed by payload
payload lines
lines with brackets or parentheses nested within the top-level pairs; treated as payload too
payload lines with end brackets or parantheses - i.e. ']' '}' '>' or ')' - possibly followed by something (junk text and/or start of a new payload)
I want to break up the input according to only the matching pairs of top-level brackets/parantheses.
Payload inside these pairs must not be altered (including newlines and whitespace).
Everything outside the toplevel pairs should be discarded as junk.
Any junk or payload inside double-quotes must be considered atomic (handled as raw text, thus any brackets or parentheses inside should also be treated as text).
Here is an example (using only {} pairs):
junk text
"atomic junk"
some junk text followed by a start bracket { here is the actual payload
more payload
"atomic payload"
nested start bracket { - all of this line is untouchable payload too
here is more payload
"yet more atomic payload; this one's got a smiley ;-)"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
} trailing junk
intermittent junk
{
payload that goes in second output file }
end junk
...sorry: Some of the input files really are as messy as that.
The first output file should be:
{ here is the actual payload
more payload
"atomic payload"
nested start bracket { - all of this line is untouchable payload too
here is more payload
"yet more atomic payload; this one's got a smiley ;-)"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
}
... and the second output file:
{
payload that goes in second output file }
Note:
I haven't quite decided whether it's necesary to keep the pair of start/end characters in the output or if they themselves should be discarded as junk.
I think a solution that keeps them in is more general use.
There can be a mix of types of top-level bracket/paranthesis pairs in the same input file.
Beware: There are * and $ characters in the input files, so please avoid confusing bash ;-)
I prefer readability over brevity; but not at an exponential cost of speed.
Nice-to-haves:
There are backslash-escaped double-quotes inside the text; preferably they should be handled
(I have a hack, but it's not pretty).
The script oughtn't break over mismatched pairs of brackets/parentheses in junk and/or payload (note: inside the atomics they must be allowed!)
More-far-out-nice-to-haves:
I haven't seen it yet, but one could speculate that some input might have single-quotes rather than double-quotes to denote atomic content... or even a mix of both.
It would be nice if the script could be easily modified to parse input of similar structure but with different start/end characters or strings.
I can see this is quite a mouthful, but I think it wouldn't give a robust solution if I broke it down into simpler questions.
The main problem is splitting up the input correctly - everything else can be ignored or "solved" with hacks, so
feel free to ignore the nice-to-haves and the more-far-out-nice-to-haves.
Given:
$ cat file
junk text
"atomic junk"
some junk text followed by a start bracket { here is the actual payload
more payload
"atomic payload"
nested start bracket { - all of this line is untouchable payload too
here is more payload
"yet more atomic payload; this one's got a smiley ;-)"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
} trailing junk
intermittent junk
{
payload that goes in second output file }
end junk
This perl file will extract the blocks you describe into files block_1, block_2, etc:
#!/usr/bin/perl
use v5.10;
use warnings;
use strict;
use Text::Balanced qw(extract_multiple extract_bracketed);
my $txt;
while (<>){$txt.=$_;} # slurp the file
my #blocks = extract_multiple(
$txt,
[
# Extract {...}
sub { extract_bracketed($_[0], '{}') },
],
# Return all the fields
undef,
# Throw out anything which does not match
1
);
chdir "/tmp";
my $base="block_";
my $cnt=1;
for my $block (#blocks){ my $fn="$base$cnt";
say "writing $fn";
open (my $fh, '>', $fn) or die "Could not open file '$fn' $!";
print $fh "$block\n";
close $fh;
$cnt++;}
Now the files:
$ cat block_1
{ here is the actual payload
more payload
"atomic payload"
nested start bracket { - all of this line is untouchable payload too
here is more payload
"yet more atomic payload; this one's got a smiley ;-)"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
}
$ cat block_2
{
payload that goes in second output file }
Using Text::Balanced is robust and likely the best solution.
You can do the blocks with a single Perl regex:
$ perl -0777 -nlE 'while (/(\{(?:(?1)|[^{}]*+)++\})|[^{}\s]++/g) {if ($1) {$cnt++; say "block $cnt:== start:\n$1\n== end";}}' file
block 1:== start:
{ here is the actual payload
more payload
"atomic payload"
nested start bracket { - all of this line is untouchable payload too
here is more payload
"yet more atomic payload; this one's got a smiley ;-)"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
}
== end
block 2:== start:
{
payload that goes in second output file }
== end
But that is a little more fragile than using a proper parser like Text::Balanced...
I have a solution in C. It would seem there's too much complexity for this to be easily achieved in shell script.
The program isn't overly complicated but nevertheless has more than 200 lines of code, which include error checking, some speed optimization, and other niceties.
Source file split-brackets-to-chunks.c:
#include <stdio.h>
/* Example code by jcxz100 - your problem if you use it! */
#define BUFF_IN_MAX 255
#define BUFF_IN_SIZE (BUFF_IN_MAX+1)
#define OUT_NAME_MAX 31
#define OUT_NAME_SIZE (OUT_NAME_MAX+1)
#define NO_CHAR '\0'
int main()
{
char pcBuff[BUFF_IN_SIZE];
size_t iReadActual;
FILE *pFileIn, *pFileOut;
int iNumberOfOutputFiles;
char pszOutName[OUT_NAME_SIZE];
char cLiteralChar, cAtomicChar, cChunkStartChar, cChunkEndChar;
int iChunkNesting;
char *pcOutputStart;
size_t iOutputLen;
pcBuff[BUFF_IN_MAX] = '\0'; /* ... just to be sure. */
iReadActual = 0;
pFileIn = pFileOut = NULL;
iNumberOfOutputFiles = 0;
pszOutName[OUT_NAME_MAX] = '\0'; /* ... just to be sure. */
cLiteralChar = cAtomicChar = cChunkStartChar = cChunkEndChar = NO_CHAR;
iChunkNesting = 0;
pcOutputStart = (char*)pcBuff;
iOutputLen = 0;
if ((pFileIn = fopen("input-utf-8.txt", "r")) == NULL)
{
printf("What? Where?\n");
return 1;
}
while ((iReadActual = fread(pcBuff, sizeof(char), BUFF_IN_MAX, pFileIn)) > 0)
{
char *pcPivot, *pcStop;
pcBuff[iReadActual] = '\0'; /* ... just to be sure. */
pcPivot = (char*)pcBuff;
pcStop = (char*)pcBuff + iReadActual;
while (pcPivot < pcStop)
{
if (cLiteralChar != NO_CHAR) /* Ignore this char? */
{
/* Yes, ignore this char. */
if (cChunkStartChar != NO_CHAR)
{
/* ... just write it out: */
fprintf(pFileOut, "%c", *pcPivot);
}
pcPivot++;
cLiteralChar = NO_CHAR;
/* End of "Yes, ignore this char." */
}
else if (cAtomicChar != NO_CHAR) /* Are we inside an atomic string? */
{
/* Yup; we are inside an atomic string. */
int bBreakInnerWhile;
bBreakInnerWhile = 0;
pcOutputStart = pcPivot;
while (bBreakInnerWhile == 0)
{
if (*pcPivot == '\\') /* Treat next char as literal? */
{
cLiteralChar = '\\'; /* Yes. */
bBreakInnerWhile = 1;
}
else if (*pcPivot == cAtomicChar) /* End of atomic? */
{
cAtomicChar = NO_CHAR; /* Yes. */
bBreakInnerWhile = 1;
}
if (++pcPivot == pcStop) bBreakInnerWhile = 1;
}
if (cChunkStartChar != NO_CHAR)
{
/* The atomic string is part of a chunk. */
iOutputLen = (size_t)(pcPivot-pcOutputStart);
fprintf(pFileOut, "%.*s", iOutputLen, pcOutputStart);
}
/* End of "Yup; we are inside an atomic string." */
}
else if (cChunkStartChar == NO_CHAR) /* Are we inside a chunk? */
{
/* No, we are outside a chunk. */
int bBreakInnerWhile;
bBreakInnerWhile = 0;
while (bBreakInnerWhile == 0)
{
/* Detect start of anything interesting: */
switch (*pcPivot)
{
/* Start of atomic? */
case '"':
case '\'':
cAtomicChar = *pcPivot;
bBreakInnerWhile = 1;
break;
/* Start of chunk? */
case '{':
cChunkStartChar = *pcPivot;
cChunkEndChar = '}';
break;
case '[':
cChunkStartChar = *pcPivot;
cChunkEndChar = ']';
break;
case '(':
cChunkStartChar = *pcPivot;
cChunkEndChar = ')';
break;
case '<':
cChunkStartChar = *pcPivot;
cChunkEndChar = '>';
break;
}
if (cChunkStartChar != NO_CHAR)
{
iNumberOfOutputFiles++;
printf("Start '%c' '%c' chunk (file %04d.txt)\n", *pcPivot, cChunkEndChar, iNumberOfOutputFiles);
sprintf((char*)pszOutName, "output/%04d.txt", iNumberOfOutputFiles);
if ((pFileOut = fopen(pszOutName, "w")) == NULL)
{
printf("What? How?\n");
fclose(pFileIn);
return 2;
}
bBreakInnerWhile = 1;
}
else if (++pcPivot == pcStop)
{
bBreakInnerWhile = 1;
}
}
/* End of "No, we are outside a chunk." */
}
else
{
/* Yes, we are inside a chunk. */
int bBreakInnerWhile;
bBreakInnerWhile = 0;
pcOutputStart = pcPivot;
while (bBreakInnerWhile == 0)
{
if (*pcPivot == cChunkStartChar)
{
/* Increase level of brackets/parantheses: */
iChunkNesting++;
}
else if (*pcPivot == cChunkEndChar)
{
/* Decrease level of brackets/parantheses: */
iChunkNesting--;
if (iChunkNesting == 0)
{
/* We are now outside chunk. */
bBreakInnerWhile = 1;
}
}
else
{
/* Detect atomic start: */
switch (*pcPivot)
{
case '"':
case '\'':
cAtomicChar = *pcPivot;
bBreakInnerWhile = 1;
break;
}
}
if (++pcPivot == pcStop) bBreakInnerWhile = 1;
}
iOutputLen = (size_t)(pcPivot-pcOutputStart);
fprintf(pFileOut, "%.*s", iOutputLen, pcOutputStart);
if (iChunkNesting == 0)
{
printf("File done.\n");
cChunkStartChar = cChunkEndChar = NO_CHAR;
fclose(pFileOut);
pFileOut = NULL;
}
/* End of "Yes, we are inside a chunk." */
}
}
}
if (cChunkStartChar != NO_CHAR)
{
printf("Chunk exceeds end-of-file. Exiting gracefully.\n");
fclose(pFileOut);
pFileOut = NULL;
}
if (iNumberOfOutputFiles == 0) printf("Nothing to do...\n");
else printf("All done.\n");
fclose(pFileIn);
return 0;
}
I've solved the nice-to-haves and one of the more-far-out-nice-to-haves.
To show this the input is a little more complex than the example in the question:
junk text
"atomic junk"
some junk text followed by a start bracket { here is the actual payload
more payload
'atomic payload { with start bracket that should be ignored'
nested start bracket { - all of this line is untouchable payload too
here is more payload
"this atomic has a literal double-quote \" inside"
"yet more atomic payload; this one's got a smiley ;-) and a heart <3"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
"here's a totally unprovoked $ sign and an * asterisk"
} trailing junk
intermittent junk
<
payload that goes in second output file } mismatched end bracket should be ignored >
end junk
Resulting file output/0001.txt:
{ here is the actual payload
more payload
'atomic payload { with start bracket that should be ignored'
nested start bracket { - all of this line is untouchable payload too
here is more payload
"this atomic has a literal double-quote \" inside"
"yet more atomic payload; this one's got a smiley ;-) and a heart <3"
end of nested bracket pair } - all of this line is untouchable payload too
this is payload too
"here's a totally unprovoked $ sign and an * asterisk"
}
... and resulting file output/0002.txt:
<
payload that goes in second output file } mismatched end bracket should be ignored >
Thanks #dawg for your help :)
I have written a code that have a if condition. After checking if then I want to show the results in a TexBox. The if statement is satisfied for more than one case and then I need to append them.
For example:
for (i=1;i<10;i++){
if (i > 8){
String^ Num = Convert::ToString(i);
textbox1->Text = Num;
}
}
The answer is 10. But I want to have 8,9,10.
How Could I have such a answer?
String^ Num = "";
for (i=1;i<10;i++){
if (i > 8){
Num = Convert::ToString(i);
if(Num == "")//first iteration so don't add ", "
{
textbox1-> += Num;
}
else
{
textbox1->Text += ", " + Num;
}
}
The default behavior of the string::operator+ is to concatenate so += will just concatenate whatever is already in the string with what the new value is. So assuming the Text field is a string this should work. Apologies for the lack of explanation.
I have a file that is in the format:
0000000540|Q1.1|margi|Q1.1|margi|Q1.1|margi
0099940598|Q1.2|8888|Q1.3|5454|Q1.2|8888
0000234223|Q2.10|saigon|Q3.9|tango|Q1.1|money
I am trying to remove the duplicates that appear on the same line.
So, if a line has
0000000540|Q1.1|margi|Q1.1|margi|Q1.1|margi
I'll like it to be
0000000540|Q1.1|margi
If the line has
0099940598|Q1.2|8888|Q1.3|5454|Q1.2|8888
I'll like it to be like
0099940598|Q1.2|8888|Q1.3|5454
I would like to do this on a shell script that takes an input file and outputs the file without the duplicates.
Thanks in advance to anyone who can help
This should do it but may not be efficient for large files.
awk '
{
delete p;
n = split($0, a, "|");
printf("%s", a[1]);
for (i = 2; i <= n ; i++)
{
if (!(a[i] in p))
{
printf("|%s", a[i]);
p[a[i]] = "";
}
}
printf "\n";
}
' YourFileName
I have about 2200 different files in a few different folders, and I need to rename about about 1/3 of them which are in their own subfolder. Those 700 are also in various folders as well.
For example, there might be
The top-most folder is Employees, which has a few files in it, then the folder 2002 has a few, 2003 has more files, 2004 etc.
I just need to attach the word "Agreement" before the existing name of each file. So instead of it just being "Joe Schmoe.doc" It would be "Agreement Joe Schmoe.doc" instead.
I've tried googling such scripts, and I can find stuff similar to what I want but it all looks completely foreign to me so I can't understand how I'd modify it to suit my needs.
Oh, and this is for windows server '03.
I need about 2 minutes to write such script for *NIX systems (may be less), but for Windows it is a long song ... ))
I've write simple VBS script for WSH, try it (save to {script-name}.vbs, change Path value (on the first line of the script) and execute). I recommend to test script on small amount of data for the first time just to be sure if it works correctly.
Path = "C:\Users\rootDirectory"
Set FSO = CreateObject("Scripting.FileSystemObject")
Sub visitFolder(folderVar)
For Each fileToRename In folderVar.Files
fileToRename.Name = "Agreement " & fileToRename.Name
Next
For Each folderToVisit In folderVar.SubFolders
visitFolder(folderToVisit)
Next
End Sub
If FSO.FolderExists(Path) Then
visitFolder(FSO.getFolder(Path))
End If
I used to do bulk renaming with batch scripts under Windows. I know it's a snap on *nix (find . -maxdepth N -type f -name "$pattern" | sed -e 'p' -e "s/$str1/$str2/g" | xargs -n2 mv). Buf after some struggle in vain, I found out, to achieve that effect using batch scripts is almost impossible. So I turned to javascript.
With this script, you can add prefix to file names by 'rename.js "s/^/Agreement /" -r *.doc'. A caret(^) means to match the beginning. The '-r' options means 'recursively', i.e. including sub-folders. You can specify a max depth with the '-d N' option. If neither '-r' or '-d N' is given, the script does not recurse.
If you know the *nix 'find' utility, you would notice that 'find' will match the full path (not just the file name part) to specified regular expression. This behavior can be achieved by supplying the '-f' option. By default, this script will match the file name part with the given regular expression.
If you are familiar with regular expressions, complicated renaming is possible. For example, 'rename.js "s/(\d+)/[$1]/" *' which uses grouping to add brackets to number sequences in filenames.
// rename.js --- bulk file renaming utility (like *nix rename.pl)
// (c) Copyright 2012, Ji Han (hanji <at> outlook <dot> com)
// you are free to distribute it under the BSD license.
// oops... jscript doesn't have array.map
Array.prototype.map = function(f, t){
var o = Object(this);
var a = new Array(o.length >>> 0);
for (var i = 0; i < a.length; ++i){ if (i in o) a[i] = f.call(t, o[i], i, o) }
return a;
};
/// main
(function(){
if (WScript.Arguments.Length == 0){
WScript.Echo('rename "<operator>/<pattern>/<string>/[<modifiers>]" [-f] [-r] [-d <maxdepth>] [<files>]');
WScript.Quit(1);
}
var fso = new ActiveXObject('Scripting.FileSystemObject');
// folder is a Folder object [e.g. from fso.GetFolder()]
// fn is a function which operates on File/Folder object
var recurseFolder = function(folder, fn, depth, maxdepth){
if (folder.Files){
for (var e = new Enumerator(folder.Files); !e.atEnd(); e.moveNext()){
fn(e.item())
}
}
if (folder.Subfolders){
for (var e = new Enumerator(folder.SubFolders); !e.atEnd(); e.moveNext()){
fn(e.item());
if (depth < maxdepth){ arguments.callee(e.item(), fn, depth + 1, maxdepth) }
}
}
}
// expand wildcards (asterisk [*] and question mark [?]) recursively
// given path may be relative, and may contain environment variables.
// but wildcards only work for the filename part of a path.
// return an array of full paths of matched files.
// {{{
var expandWildcardsRecursively = function(n, md){
var pattern = fso.GetFileName(n);
// escape regex metacharacters (except \, /, * and ?)
// \ and / wouldn't appear in filename
// * and ? are treated as wildcards
pattern = pattern.replace(/([\[\](){}^$.+|-])/g, '\\$1');
pattern = pattern.replace(/\*/g, '.*'); // * matches zero or more characters
pattern = pattern.replace(/\?/g, '.'); // ? matches one character
pattern = pattern.replace(/^(.*)$/, '\^$1\$'); // matches the whole filename
var re = new RegExp(pattern, 'i'); // case insensitive
var folder = fso.GetFolder(fso.GetParentFolderName(fso.GetAbsolutePathName(n)));
var l = [];
recurseFolder(folder, function(i){ if (i.Name.match(re)) l.push(i.Path) }, 0, md);
return l;
}
// }}}
// parse "<operator>/<pattern>/<string>/[<modifiers>]"
// return an array splitted at unescaped forward slashes
// {{{
var parseExpr = function(s){
// javascript regex doesn't have lookbehind...
// reverse the string and lookahead to parse unescaped forward slashes.
var z = s.split('').reverse().join('');
// match unescaped forward slashes and get their positions.
var re = /\/(\\\\)*(?!\\)/g;
var l = [];
while (m = re.exec(z)){ l.push(m.index) }
// split s at unescaped forward slashes.
var b = [0].concat(l.map(function(x){ return s.length - x }).reverse());
var e = (l.map(function(x){ return s.length - x - 1 }).reverse()).concat([s.length]);
return b.map(function(_, i){ return s.substring(b[i], e[i]) });
}
// }}}
var expr = WScript.Arguments(0);
var args = [];
var options = {};
for (var i = 1; i < WScript.Arguments.Length; ++i){
if (WScript.Arguments(i).substring(0, 1) != '-'){
args.push(WScript.Arguments(i));
} else if (WScript.Arguments(i) == '-f'){
options['fullpath'] = true;
} else if (WScript.Arguments(i) == '-r'){
options['recursive'] = true;
} else if (WScript.Arguments(i) == '-d'){
options['maxdepth'] = WScript.Arguments(++i);
} else if (WScript.Arguments(i) == '--'){
continue;
} else {
WScript.Echo('invalid option \'' + WScript.Arguments(i) +'\'');
WScript.Quit(1);
}
}
if (options['maxdepth']){
var md = options['maxdepth'];
} else if (options['recursive']){
var md = 1<<31>>>0;
} else {
var md = 0;
}
var tokens = parseExpr(expr);
if (tokens.length != 4){
WScript.Echo('error parsing expression \'' + expr + '\'.');
WScript.Quit(1);
}
if (tokens[0] != 's'){
WScript.Echo('<operator> must be s.');
WScript.Quit(1);
}
var pattern = tokens[1];
var substr = tokens[2];
var modifiers = tokens[3];
var re = new RegExp(pattern, modifiers);
for (var i = 0; i < args.length; ++i){
var l = expandWildcardsRecursively(args[i], md);
for (var j = 0; j < l.length; ++j){
var original = l[j];
if (options['fullpath']){
var nouveau = original.replace(re, substr);
} else {
var nouveau = fso.GetParentFolderName(original) + '\\' + fso.GetFileName(original).replace(re, substr);
}
if (nouveau != original){
(fso.FileExists(original) && fso.GetFile(original) || fso.GetFolder(original)).Move(nouveau)
}
}
}
})();
I would like to replace every blank spaces in a string by a fixnum (which is the number of blank spaces).
Let me give an example:
s = "hello, how are you ?"
omg(s) # => "hello,3how10are2you1?"
Do you see a way (sexy if possible) to update a string like this?
Thank you Rubists :)
gsub can be fed a block for the "replace with" param, the result of the block is inserted into place where the match was found. The argument to the block is the matched string. So to implement this we capture as much whitespace as we can ( /\s+/ ) and feed that into the block each time a section is found, returning that string's length, which gets put back where the whitespace was originally found.
Code:
s = "hello, how are you ?"
res = s.gsub(/\s+/) { |m| m.length }
puts res
# => hello,3how10are2you1?
it is possible to do this via an array split : Javascript example
var s = "hello, how are you ?";
function omg( str ) {
var strArr = str.split('');
var count = 0;
var finalStr = '';
for( var i = 0; i < strArr.length; i++ ) {
if( strArr[i] == ' ' ) {
count++;
}
else
{
if( count > 0 ) {
finalStr += '' + count;
count = 0;
}
finalStr += strArr[i];
}
}
return finalStr
}
alert( omg( s ) ); //"hello,3how10are2you1?"
Lol, this seems the best it can be for javascript