My csv is dynamically generated and doesn't have any headers because the number of columns and rows are varying with each run. An example below
A, 30, 40, 35, 25
B, 25, 35, 45, 35
Which if there were headers would look like as below
Age1, Age2, Age1, Age2
A, 30, 40, 35, 25
B, 25, 35, 45, 35
For each row the data is in pairs, i.e. col1 & col[2] need to be stacked and col [3] & col [4] need to be stacked. Goal is to have a clustered stacked bar chart with A and B in X axis and two stacked bars for each pair.
I was trying to follow the example at https://bl.ocks.org/SpaceActuary/6233700e7f443b719855a227f4749ee5 but I am not able to get, how to use the stack function in the absence of headers/keys.
You can use d3.text to load the CSV data, and then iterate over the text to create an array of objects with named values, which could then be stacked or whatever you would normally do in D3 with your data
d3.text("data.csv", function(text) {
console.log(text);
var data = []
d3.csvParseRows(text).forEach(function(row) {
let obj = {}
row.forEach(function(value, i) {
let pairIndex = Math.floor((i - 1) / 2)
//assume first value is the index or name for the row, eg A, B, etc
if (i == 0) {
obj.index = value
}
else if (i % 2 == 0) {
let v = "age2-" + pairIndex
obj[v] = value
} else {
let v = "age1-" + pairIndex
obj[v] = value
}
});
data.push(obj)
});
console.log(data);
// continue with your code
I am using Kotlintest and data tables to test an application that uses Kotlin, SpringBoot and Gradle because the syntax is way more concise than ParameterizedJunitTests when you have complex data in your tables.
Is there a way to use the parameter names in the method-titles like there is for parameterized tests in JUnit? Moreover, all my test-executions are listed as one test, but I would like to have a row in my test results per data table row. I found neither of the two topics in the Documentation.
To make things clearer an example with Kotlintest:
class AdditionSpec : FunSpec() {
init {
test("x + y is sum") {
table(
headers("x", "y", "sum"),
row(1, 1, 2),
row(50, 50, 100),
row(3, 1, 2)
).forAll { x, y, sum ->
assertThat(x + y).isEqualTo(sum)
}
}
}
}
and a corresponding example with JUnit:
#RunWith(Parameterized::class)
class AdditionTest {
#ParameterizedTest(name = "adding {0} and {1} should result in {2}")
#CsvSource("1,1,2", "50, 50, 100", "3, 1, 5")
fun testAdd(x: Int, y: Int, sum: Int) {
assertThat(x + y).isEqualTo(sum);
}
}
With 1/3 failures:
Kotlintest:
Junit:
Is there something similar to #ParameterizedTest(name = "adding {0} and {1} should result in {2}") in kotlintest when using data tables?
You can reverse nesting. Instead of having table within test, simply nest test within table.
class AdditionSpec : FunSpec() {
init {
context("x + y is sum") {
table(
headers("x", "y", "sum"),
row(1, 1, 2),
row(50, 50, 100),
row(3, 1, 2)
).forAll { x, y, sum ->
test("$x + $y should be $sum") {
x + y shouldBe sum
}
}
}
}
}
This is possible using the FreeSpec and the minus operator like this:
class AdditionSpec : FreeSpec({
"x + y is sum" - {
listOf(
row(1, 1, 2),
row(50, 50, 100),
row(3, 1, 2)
).map { (x: Int, y: Int, sum: Int) ->
"$x + $y should result in $sum" {
(x + y) shouldBe sum
}
}
}
})
This gives this output in Intellij:
See the documentation on this here
Closed. This question needs details or clarity. It is not currently accepting answers.
Want to improve this question? Add details and clarify the problem by editing this post.
Closed 9 years ago.
Improve this question
No idea if this is ok to ask here since it's not programming but I have no idea where else to go:
I want to organise the following data in a consistent way. At the moment it's a mess, with only the first two columns (comma separated) consistent. The remaining columns can number anywhere from 1-9 and are usually different.
In other words, I want to sort it so the text matches (all of the value columns in a row, all of the recoil columns in a row, etc). Then I can remove the text and add a header, and it will still make sense.
bm_wp_upg_o_t1micro, sight, value = 3, zoom = 3, recoil = 1, spread_moving = -1
bm_wp_upg_o_marksmansight_rear, sight, value = 3, zoom = 1, recoil = 1, spread = 1
bm_wp_upg_o_marksmansight_front, extra, value = 1
bm_wp_m4_upper_reciever_edge, upper_reciever, value = 3, recoil = 1
bm_wp_m4_upper_reciever_round, upper_reciever, value = 1
bm_wp_m4_uupg_b_long, barrel, value = 4, damage = 1, spread = 1, spread_moving = -2, concealment = -2
Any suggestions (even on just where the right place is to actually ask this) would be great.
Context is just raw data ripped from a game file that I'm trying to organise.
I'm afraid regex isn't going to help you much here because of the irregular nature of your input (it would be possible to match it, but it would be a bear to get it all arranged one way or another). This could be done pretty easily with any programming language, but for stuff like this, I always go to awk.
Assuming your input is in a file called input.txt, put the following in a program called parse.awk:
BEGIN {
FS=" *, *";
formatStr = "%32s,%8s,%8s,%8s,%10s,%16s,%8s,%18s,%10s,%10s,%16s,%16s\n";
printf( formatStr, "id", "sight", "value", "zoom", "recoil", "spread_moving", "extra", "upper_receiver", "barrel", "damage", "spread_moving", "concealment" );
}
{
split("",a);
for( i=2; i<=NF; i++ ) {
if( split( $(i), kvp, " *= *" ) == 1 ) {
a[kvp[1]] = "x";
} else {
a[kvp[1]] = gensub( /^\s*|\s*$/, "", "g", kvp[2] );
}
}
printf( formatStr, $1, a["sight"], a["value"], a["zoom"], a["recoil"],
a["spread_moving"], a["extra"], a["upper_receiver"],
a["barrel"], a["damage"], a["spread_moving"], a["concealment"] );
}
Run awk against it:
awk -f parse.awk input.txt
And get your output:
id, sight, value, zoom, recoil, spread_moving, extra, upper_receiver, barrel, damage, spread_moving, concealment
bm_wp_upg_o_t1micro, x, 3, 3, 1, -1, , , , , -1,
bm_wp_upg_o_marksmansight_rear, x, 3, 1, 1, , , , , , ,
bm_wp_upg_o_marksmansight_front, , 1, , , , x, , , , ,
bm_wp_m4_upper_reciever_edge, , 3, , 1, , , , , , ,
bm_wp_m4_upper_reciever_round, , 1, , , , , , , , ,
bm_wp_m4_uupg_b_long, , 4, , , -2, , , x, 1, -2, -2
Note that I chose to just use an 'x' for sight, which seems to a present/absent thing. You can use whatever you want there.
If you're using Linux or a Macintosh, you should have awk available. If you're on Windows, you'll have to install it.
I did make another awk version. I think this should a little easier to read.
All value/column are read from the file to make it as dynamic as possible.
awk -F, '
{
ID[$1]=$2 # use column 1 as index
for (i=3;i<=NF;i++ ) # loop through all fields from #3 to end
{
gsub(/ +/,"",$i) # remove space from field
split($i,a,"=") # split field in name and value a[1] and a[2]
COLUMN[a[1]]++ # store field name as column name
DATA[$1" "a[1]]=a[2] # store data value in DATA using field #1 and column name as index
}
}
END {
printf "%49s ","info" # print info
for (i in COLUMN)
{printf "%15s",i} # print column name
print ""
for (i in ID) # loop through all ID
{
printf "%32s %16s ",i, ID[i] # print ID and info
for (j in COLUMN)
{
printf "%14s ",DATA[i" "j]+0 # print value
}
print ""
}
}' file
Output
info spread recoil zoom concealment spread_moving damage value
bm_wp_m4_upper_reciever_round upper_reciever 0 0 0 0 0 0 1
bm_wp_m4_uupg_b_long barrel 1 0 0 -2 -2 1 4
bm_wp_upg_o_marksmansight_rear sight 1 1 1 0 0 0 3
bm_wp_upg_o_marksmansight_front extra 0 0 0 0 0 0 1
bm_wp_m4_upper_reciever_edge upper_reciever 0 1 0 0 0 0 3
bm_wp_upg_o_t1micro sight 0 1 3 0 -1 0 3
Stick with Ethan's answer — this is just me enjoying myself. (And yes, that makes me pretty weird!)
awk script
awk 'BEGIN {
# f_idx[field] holds the column number c for a field=value item
# f_name[c] holds the names
# f_width[c] holds the width of the widest value (or the field name)
# f_fmt[c] holds the appropriate format
FS = " *, *"; n = 2;
f_name[0] = "id"; f_width[0] = length(f_name[0])
f_name[1] = "type"; f_width[1] = length(f_name[1])
}
{
#-#print NR ":" $0
line[NR,0] = $1
len = length($1)
if (len > f_width[0])
f_width[0] = len
line[NR,1] = $2
len = length($2)
if (len > f_width[1])
f_width[1] = len
for (i = 3; i <= NF; i++)
{
split($i, fv, " = ")
#-#print "1:" fv[1] ", 2:" fv[2]
if (!(fv[1] in f_idx))
{
f_idx[fv[1]] = n
f_width[n++] = length(fv[1])
}
c = f_idx[fv[1]]
f_name[c] = fv[1]
gsub(/ /, "", fv[2])
len = length(fv[2])
if (len > f_width[c])
f_width[c] = len
line[NR,c] = fv[2]
#-#print c ":" f_name[c] ":" f_width[c] ":" line[NR,c]
}
}
END {
for (i = 0; i < n; i++)
f_fmt[i] = "%s%" f_width[i] "s"
#-#for (i = 0; i < n; i++)
#-# printf "%d: (%d) %s %s\n", i, f_width[i], f_name[i], f_fmt[i]
#-# pad = ""
for (j = 0; j < n; j++)
{
printf f_fmt[j], pad, f_name[j]
pad = ","
}
printf "\n"
for (i = 1; i <= NR; i++)
{
pad = ""
for (j = 0; j < n; j++)
{
printf f_fmt[j], pad, line[i,j]
pad = ","
}
printf "\n"
}
}' data
This script adapts to the data it finds in the file. It assigns the column heading 'id' to column 1 of the input, and 'type' to column 2. For each of the sets of values in columns 3..N, it splits up the data into key (in fv[1]) and value (in fv[2]). If the key has not been seen before, it is assigned a new column number, and the key is stored as the column name, and the width of key as the initial column width. Then the value is stored in the appropriate column within the line.
When all the data's read, the script knows what the column headings are going to be. It can then create a set of format strings. Then it prints the headings and all the rows of data. If you don't want fixed width output, then you can simplify the script considerably. There are some (mostly minor) simplifications that could be made to this script.
Data file
bm_wp_upg_o_t1micro, sight, value = 3, zoom = 3, recoil = 1, spread_moving = -1
bm_wp_upg_o_marksmansight_rear, sight, value = 3, zoom = 1, recoil = 1, spread = 1
bm_wp_upg_o_marksmansight_front, extra, value = 1
bm_wp_m4_upper_receiver_edge, upper_receiver, value = 3, recoil = 1
bm_wp_m4_upper_receiver_round, upper_receiver, value = 1
bm_wp_m4_uupg_b_long, barrel, value = 4, damage = 1, spread = 1, spread_moving = -2, concealment = -2
Output
id, type,value,zoom,recoil,spread_moving,spread,damage,concealment
bm_wp_upg_o_t1micro, sight, 3, 3, 1, -1, , ,
bm_wp_upg_o_marksmansight_rear, sight, 3, 1, 1, , 1, ,
bm_wp_upg_o_marksmansight_front, extra, 1, , , , , ,
bm_wp_m4_upper_receiver_edge,upper_receiver, 3, , 1, , , ,
bm_wp_m4_upper_receiver_round,upper_receiver, 1, , , , , ,
bm_wp_m4_uupg_b_long, barrel, 4, , , -2, 1, 1, -2