using sed to convert file contents to scheme list - bash

Question

I would like to convert data.txt into scheme list using sed into the following format:
-Each every line with same starting number will be parsed and combined like so:
data.txt
1,{},344.233
1,{2},344.197
2,{16},290.281
2,{18},289.093
3,{1},220.896
foo.scm
(define v1 '(1 (() 344.233) ((2) 344.197))) ;; this is for first two lines starting with 1
(define v2 '(2 ((16) 290.281) ((18) 289.093))) ;; ... 2
(define v3 '(3 (() 237.558))) ;; ... 3

Answer 1

I know nothing about scheme, so I'd probably do this in awk rather than sed.
[ghoti#pc ~]$ cat data.txt
1,{},344.233
1,{2},344.197
2,{16},290.281
2,{18},289.093
3,{1},220.896
[ghoti#pc ~]$ cat doit.awk
#!/usr/bin/awk -f
BEGIN {
FS=",";
last1=1;
}
$1 != last1 {
printf("(define v%.0f '(%.0f %s))\n", last1, last1, substr(sect,2));
last1=$1; sect="";
}
{
gsub(/[^0-9]/,"",$2);
sect=sprintf("%s ((%s) %s)", sect, $2, $3);
}
END {
printf("(define v%.0f '(%.0f %s))\n", last1, last1, substr(sect,2));
}
[ghoti#pc ~]$ ./doit.awk data.txt
(define v1 '(1 (() 344.233) ((2) 344.197)))
(define v2 '(2 ((16) 290.281) ((18) 289.093)))
(define v3 '(3 ((1) 220.896)))
[ghoti#pc ~]$
It could certainly be written more tightly, but this gets the job done.
UPDATE: (per comments)
[ghoti#pc ~]$ tail -1 data.txt
3,{1,3,4},220.896
[ghoti#pc ~]$ diff -u doit.awk doitnew.awk
--- doit.awk 2012-05-30 00:38:34.549680376 -0400
+++ doitnew.awk 2012-05-30 00:38:52.893810815 -0400
## -10,8 +10,15 ##
last1=$1; sect="";
}
+$2 !~ /}$/ {
+ while ($2 !~ /}$/) {
+ pos=match($0, /,[0-9,]+}/);
+ $0=substr($0, 0, pos-1) " " substr($0, pos+1);
+ }
+}
+
{
- gsub(/[^0-9]/,"",$2);
+ gsub(/[^0-9 ]/,"",$2);
sect=sprintf("%s ((%s) %s)", sect, $2, $3);
}
[ghoti#pc ~]$ ./doitnew.awk data.txt
(define v1 '(1 (() 344.233) ((2) 344.197)))
(define v2 '(2 ((16) 290.281) ((18) 289.093)))
(define v3 '(3 ((1 3 4) 220.896)))
[ghoti#pc ~]$
What's going on here?
In the new block we're adding, test to see whether the second field ends in a }. If it doesn't, we'll loop until it does. For each run of the loop, we'll remove a comma before the }, replacing it with a space.
Sometimes, brute-force works. :-P

Answer 2

In racket (a.k.a. scheme):
#lang racket
;; parse a line (we will join them later)
(define (line-parse l)
(match (regexp-match #px"([0-9]+),\\{([0-9,]*)\\},([0-9.]+)" l)
[(list dc first-num bracket-nums rest)
(list (string->number first-num)
(match bracket-nums
["" empty]
[else (map string->number
(regexp-split #px"," bracket-nums))])
(string->number rest))]
[else
(error "unexpected line format in line: ~s\n" l)]))
;; join together lines that start with the same number
(define (join-lines lines)
(cond [(empty? lines) empty]
[else (join-lines-of-n (first (first lines))
lines
empty)]))
;; gather together lines starting with 'n':
(define (join-lines-of-n n lines accum)
(cond [(empty? lines)
(list (cons n (reverse accum)))]
[(equal? (first (first lines)) n)
(join-lines-of-n n (rest lines) (cons (rest (first lines))
accum))]
[else
(cons (cons n (reverse accum))
(join-lines lines))]))
(define (dress-up line)
(format "~a\n" `(define ,(format "v~s" (first line))
',line)))
(display
(apply
string-append
(map dress-up
(join-lines
(map line-parse
(sequence->list (in-port read-line)))))))
Save this as rewrite.rkt, run it like this:
oiseau:/tmp clements> racket ./rewrite.rkt < foo.txt
(define v1 (quote (1 (() 344.233) ((2) 344.197))))
(define v2 (quote (2 ((16) 290.281) ((18) 289.093))))
(define v3 (quote (3 ((1) 220.896) ((4 5) 2387.278))))
... note that I added a {4,5} line to the input example to test your extension.
also, note that the output uses (quote ...) rather than '(...). This "should work fine"; that is, Scheme readers produce the same output for these two forms, and the resulting file should work fine as scheme input.
If this were my code, I think I wouldn't do the (define v1 ...) dance, and just write the thing out as a big piece of data that a scheme/racket program can slurp in with a single "read", but that's your choice, not mine. Also, there's some ambiguity in your specification re: the uniqueness of the initial indexes; that is, you might "go back" to an earlier line number. For instance, what should be the output when given this input file:
3,{1},1.0
4,{1},1.0
3,{1},1.0
?
Also, note that I chopped out all of the test cases in order to make it look shorter/prettier :).
EDIT: OH! Gather the lines this way, instead. It'll actually be a bit slower, but it reads much more nicely:
#lang racket
;; parse a line (we will join them later)
(define (line-parse l)
(match (regexp-match #px"([0-9]+),\\{([0-9,]*)\\},([0-9.]+)" l)
[(list dc first-num bracket-nums rest)
(list (string->number first-num)
(match bracket-nums
["" empty]
[else (map string->number
(regexp-split #px"," bracket-nums))])
(string->number rest))]
[else
(error "unexpected line format in line: ~s\n" l)]))
;; does the line start with the number k?
(define ((starts-with k) l) (equal? (first l) k))
;; join together lines starting with the same thing:
(define (join-lines lines)
(for/list ([k (remove-duplicates (map first lines))])
(cons k (map rest (filter (starts-with k) lines)))))
(define (dress-up line)
(format "~a\n" `(define ,(format "v~s" (first line))
',line)))
(display
(apply
string-append
(map dress-up
(join-lines
(map line-parse
(sequence->list (in-port read-line)))))))

Answer 3

This might work for you (GNU sed):
sed ':a;$!N;s/^\(\([^,])*\).*\)\n\2/\1/;ta;h;x;s/\n.*//;s/,{\([^}]*\)},\([^,]\+\)/ ((\1) \2)/g;s/,/ /g;s/^\([^ ]*\).*/(define v\1 '\''(&)) ;;...\1/p;x;D' file
Explanation:
Reduce like values to a single line :a;$!N;s/^\(\([^,])*\).*\)\n\2/\1/;ta
Copy pattern space (PS) to hold space (HS). h
Swap to HS x
Chop off previous line. s/\n.*//
Formulate lists. s/,{\([^}]*\)},\([^,]\+\)/ ((\1) \2)/g
Replace any remaining ,'s with spaces. s/,/ /g
Surround lists with function definition and comments and print. s/^\([^ ]*\).*/(define v\1 '\''(&)) ;;...\1/p
Swap back to PS. x
Delete upto previous line and repeat. D