Goroutines overhead and performance analysis when subsetting DataFrames (Gota) - performance
I've been working since the beginning of 2016 on implementing a Pandas/R DataFrame implementation for Go: https://github.com/kniren/gota.
Recently, I've been focusing on improving the performance of the library to try to match that of Pandas/Dplyr. You can follow the progress so far here: https://github.com/kniren/gota/issues/16
Since one of the more frequently used operations is the DataFrame subsetting, I thought it could be a good idea to introduce concurrency to try to increase the performance of the system.
Before:
columns := make([]series.Series, df.ncols)
for i, column := range df.columns {
s := column.Subset(indexes)
columns[i] = s
}
After:
columns := make([]series.Series, df.ncols)
var wg sync.WaitGroup
wg.Add(df.ncols)
for i := range df.columns {
go func(i int) {
columns[i] = df.columns[i].Subset(indexes)
wg.Done()
}(i)
}
wg.Wait()
As far as I understand, creating a goroutine for each of the columns of a DataFrame should not introduce much overhead, so I was expecting to achieve at least a x2 speedup with respect to the serial version (At least for large datasets). However, when benchmarking this change with datasets and indexes of different sizes the results are very disappointing (NROWSxNCOLS_INDEXSIZE-CPUCORES):
benchmark old ns/op new ns/op delta
BenchmarkDataFrame_Subset/1000000x20_100 55230 109349 +97.99%
BenchmarkDataFrame_Subset/1000000x20_100-2 51457 67714 +31.59%
BenchmarkDataFrame_Subset/1000000x20_100-4 49845 70141 +40.72%
BenchmarkDataFrame_Subset/1000000x20_1000 518506 518085 -0.08%
BenchmarkDataFrame_Subset/1000000x20_1000-2 476661 311379 -34.67%
BenchmarkDataFrame_Subset/1000000x20_1000-4 505023 316583 -37.31%
BenchmarkDataFrame_Subset/1000000x20_10000 6621116 6314112 -4.64%
BenchmarkDataFrame_Subset/1000000x20_10000-2 7316062 4509601 -38.36%
BenchmarkDataFrame_Subset/1000000x20_10000-4 6483812 8394113 +29.46%
BenchmarkDataFrame_Subset/1000000x20_100000 105341711 106427967 +1.03%
BenchmarkDataFrame_Subset/1000000x20_100000-2 94567729 56778647 -39.96%
BenchmarkDataFrame_Subset/1000000x20_100000-4 91896690 60971444 -33.65%
BenchmarkDataFrame_Subset/1000000x20_1000000 1538680081 1632044752 +6.07%
BenchmarkDataFrame_Subset/1000000x20_1000000-2 1292113119 1100075806 -14.86%
BenchmarkDataFrame_Subset/1000000x20_1000000-4 1282367864 949615298 -25.95%
BenchmarkDataFrame_Subset/100000x20_100 50286 106850 +112.48%
BenchmarkDataFrame_Subset/100000x20_100-2 54537 70492 +29.26%
BenchmarkDataFrame_Subset/100000x20_100-4 58024 76617 +32.04%
BenchmarkDataFrame_Subset/100000x20_1000 541600 625967 +15.58%
BenchmarkDataFrame_Subset/100000x20_1000-2 493894 362894 -26.52%
BenchmarkDataFrame_Subset/100000x20_1000-4 535373 349211 -34.77%
BenchmarkDataFrame_Subset/100000x20_10000 6298063 7678499 +21.92%
BenchmarkDataFrame_Subset/100000x20_10000-2 5827185 4832560 -17.07%
BenchmarkDataFrame_Subset/100000x20_10000-4 8195048 3660077 -55.34%
BenchmarkDataFrame_Subset/100000x20_100000 105108807 82976477 -21.06%
BenchmarkDataFrame_Subset/100000x20_100000-2 92112736 58317114 -36.69%
BenchmarkDataFrame_Subset/100000x20_100000-4 92044966 63469935 -31.04%
BenchmarkDataFrame_Subset/1000x20_10 9741 53365 +447.84%
BenchmarkDataFrame_Subset/1000x20_10-2 9366 36457 +289.25%
BenchmarkDataFrame_Subset/1000x20_10-4 9463 46682 +393.31%
BenchmarkDataFrame_Subset/1000x20_100 50841 103523 +103.62%
BenchmarkDataFrame_Subset/1000x20_100-2 49972 62344 +24.76%
BenchmarkDataFrame_Subset/1000x20_100-4 72014 81808 +13.60%
BenchmarkDataFrame_Subset/1000x20_1000 457799 571292 +24.79%
BenchmarkDataFrame_Subset/1000x20_1000-2 460551 405116 -12.04%
BenchmarkDataFrame_Subset/1000x20_1000-4 462928 416522 -10.02%
BenchmarkDataFrame_Subset/1000x200_10 90125 688443 +663.88%
BenchmarkDataFrame_Subset/1000x200_10-2 85259 392705 +360.60%
BenchmarkDataFrame_Subset/1000x200_10-4 87412 387509 +343.31%
BenchmarkDataFrame_Subset/1000x200_100 486600 1082901 +122.54%
BenchmarkDataFrame_Subset/1000x200_100-2 471154 732304 +55.43%
BenchmarkDataFrame_Subset/1000x200_100-4 542846 659571 +21.50%
BenchmarkDataFrame_Subset/1000x200_1000 5926086 6686480 +12.83%
BenchmarkDataFrame_Subset/1000x200_1000-2 5364091 3986970 -25.67%
BenchmarkDataFrame_Subset/1000x200_1000-4 5904977 4504084 -23.72%
BenchmarkDataFrame_Subset/1000x2000_10 1187297 7800052 +556.96%
BenchmarkDataFrame_Subset/1000x2000_10-2 1217022 3930742 +222.98%
BenchmarkDataFrame_Subset/1000x2000_10-4 1301666 3617871 +177.94%
BenchmarkDataFrame_Subset/1000x2000_100 6942015 10790196 +55.43%
BenchmarkDataFrame_Subset/1000x2000_100-2 6588351 7592847 +15.25%
BenchmarkDataFrame_Subset/1000x2000_100-4 7067226 14391327 +103.63%
BenchmarkDataFrame_Subset/1000x2000_1000 62392457 69560711 +11.49%
BenchmarkDataFrame_Subset/1000x2000_1000-2 57793006 37416703 -35.26%
BenchmarkDataFrame_Subset/1000x2000_1000-4 59572261 58398203 -1.97%
benchmark old allocs new allocs delta
BenchmarkDataFrame_Subset/1000000x20_100 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_100-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_100-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_1000 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_1000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_1000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_10000 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_10000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_10000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_100000 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_100000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_100000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_1000000 41 42 +2.44%
BenchmarkDataFrame_Subset/1000000x20_1000000-2 41 43 +4.88%
BenchmarkDataFrame_Subset/1000000x20_1000000-4 41 46 +12.20%
BenchmarkDataFrame_Subset/100000x20_100 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_100-2 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_100-4 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_1000 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_1000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_1000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_10000 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_10000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_10000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_100000 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_100000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/100000x20_100000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_10 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_10-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_10-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_100 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_100-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_100-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_1000 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_1000-2 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x20_1000-4 41 42 +2.44%
BenchmarkDataFrame_Subset/1000x200_10 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_10-2 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_10-4 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_100 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_100-2 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_100-4 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_1000 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_1000-2 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x200_1000-4 401 402 +0.25%
BenchmarkDataFrame_Subset/1000x2000_10 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_10-2 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_10-4 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_100 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_100-2 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_100-4 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_1000 4001 4002 +0.02%
BenchmarkDataFrame_Subset/1000x2000_1000-2 4001 4010 +0.22%
BenchmarkDataFrame_Subset/1000x2000_1000-4 4001 4003 +0.05%
benchmark old bytes new bytes delta
BenchmarkDataFrame_Subset/1000000x20_100 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000000x20_100-2 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000000x20_100-4 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000000x20_1000 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000000x20_1000-2 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000000x20_1000-4 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000000x20_10000 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/1000000x20_10000-2 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/1000000x20_10000-4 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/1000000x20_100000 29083520 29083536 +0.00%
BenchmarkDataFrame_Subset/1000000x20_100000-2 29083520 29083547 +0.00%
BenchmarkDataFrame_Subset/1000000x20_100000-4 29083542 29083563 +0.00%
BenchmarkDataFrame_Subset/1000000x20_1000000 290121600 290121616 +0.00%
BenchmarkDataFrame_Subset/1000000x20_1000000-2 290121600 290121696 +0.00%
BenchmarkDataFrame_Subset/1000000x20_1000000-4 290121600 290121840 +0.00%
BenchmarkDataFrame_Subset/100000x20_100 32400 32416 +0.05%
BenchmarkDataFrame_Subset/100000x20_100-2 32400 32416 +0.05%
BenchmarkDataFrame_Subset/100000x20_100-4 32400 32416 +0.05%
BenchmarkDataFrame_Subset/100000x20_1000 298880 298896 +0.01%
BenchmarkDataFrame_Subset/100000x20_1000-2 298880 298896 +0.01%
BenchmarkDataFrame_Subset/100000x20_1000-4 298880 298896 +0.01%
BenchmarkDataFrame_Subset/100000x20_10000 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/100000x20_10000-2 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/100000x20_10000-4 2971520 2971536 +0.00%
BenchmarkDataFrame_Subset/100000x20_100000 29083520 29083536 +0.00%
BenchmarkDataFrame_Subset/100000x20_100000-2 29083520 29083536 +0.00%
BenchmarkDataFrame_Subset/100000x20_100000-4 29083542 29083553 +0.00%
BenchmarkDataFrame_Subset/1000x20_10 4880 4896 +0.33%
BenchmarkDataFrame_Subset/1000x20_10-2 4880 4896 +0.33%
BenchmarkDataFrame_Subset/1000x20_10-4 4880 4896 +0.33%
BenchmarkDataFrame_Subset/1000x20_100 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000x20_100-2 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000x20_100-4 32400 32416 +0.05%
BenchmarkDataFrame_Subset/1000x20_1000 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000x20_1000-2 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000x20_1000-4 298880 298896 +0.01%
BenchmarkDataFrame_Subset/1000x200_10 49568 49584 +0.03%
BenchmarkDataFrame_Subset/1000x200_10-2 49568 49584 +0.03%
BenchmarkDataFrame_Subset/1000x200_10-4 49568 49585 +0.03%
BenchmarkDataFrame_Subset/1000x200_100 324768 324784 +0.00%
BenchmarkDataFrame_Subset/1000x200_100-2 324768 324784 +0.00%
BenchmarkDataFrame_Subset/1000x200_100-4 324768 324784 +0.00%
BenchmarkDataFrame_Subset/1000x200_1000 2989568 2989584 +0.00%
BenchmarkDataFrame_Subset/1000x200_1000-2 2989568 2989584 +0.00%
BenchmarkDataFrame_Subset/1000x200_1000-4 2989569 2989588 +0.00%
BenchmarkDataFrame_Subset/1000x2000_10 491072 491088 +0.00%
BenchmarkDataFrame_Subset/1000x2000_10-2 491072 491133 +0.01%
BenchmarkDataFrame_Subset/1000x2000_10-4 491072 491088 +0.00%
BenchmarkDataFrame_Subset/1000x2000_100 3243072 3243088 +0.00%
BenchmarkDataFrame_Subset/1000x2000_100-2 3243074 3243102 +0.00%
BenchmarkDataFrame_Subset/1000x2000_100-4 3243076 3243100 +0.00%
BenchmarkDataFrame_Subset/1000x2000_1000 29891072 29891088 +0.00%
BenchmarkDataFrame_Subset/1000x2000_1000-2 29891086 29891797 +0.00%
BenchmarkDataFrame_Subset/1000x2000_1000-4 29891115 29891167 +0.00%
Running the profiler (cpu/mem) over this benchmark didn't seem to reveal nothing significant. The concurrent version seem to spend some time on rumtime.match_semaphore_signal but I guess this is to be expected when waiting for the goroutines to finish.
I've tried limiting the number of goroutines launched to the maximum number of cores as reported by runtime.GOMAXPROCS(0) but the results are somewhat even worse. Am I doing something horribly wrong here or is the goroutines overhead so big that it has such a significant effect on the performance?
Goroutines are cheap, but not free.
I didn't read your code, but if you are spawning NCOLS_INDEXSIZE goroutines for each row you process, then it's a very bad practice.
This can be seen in your benchmark where you have 2k columns and only 1k rows - you get very big improvement. But in all other cases, when number of columns << number of rows, goroutine spawning becomes a bottleneck.
Instead you should spawn a pool of goroutines (close to your CPU count) and distribute work between them through channels - it's the canonical way. You may want to read https://blog.golang.org/pipelines
Related
Why are all the values same in ARIMA model predictions?
The data set had 1511 observations. I used the first 1400 values to fit ARIMA model of order (1,1,9), keeping the rest for predictions. But when I look at the predictions, apart from the first 16 values all the remaining values are the same. Here's what I tried: model2=ARIMA(tstrain,order=(1,1,9)) fitted_model2=model2.fit() And for prediction: start=len(tstrain) end=len(tstrain)+len(tstest)-1 predictions=fitted_model2.predict(start,end,typ='levels') Here tstrain and tstest are the train and test sets. predictions.head(30) 1400 214.097742 1401 214.689674 1402 214.820804 1403 215.621131 1404 215.244980 1405 215.349230 1406 215.392444 1407 215.022312 1408 215.020736 1409 215.021384 1410 215.021118 1411 215.021227 1412 215.021182 1413 215.021201 1414 215.021193 1415 215.021196 1416 215.021195 1417 215.021195 1418 215.021195 1419 215.021195 1420 215.021195 1421 215.021195 1422 215.021195 1423 215.021195 1424 215.021195 1425 215.021195 1426 215.021195 1427 215.021195 1428 215.021195 1429 215.021195 Please help me out here. What am I missing?
Use null.Time values in a golang template
I'm using gopkg.in/guregu/null.v4 to get some data from a Postgres DB and the results are coming back fine, and I can put them into json format and the world is happy... however, I'm trying to email the results using a template and have hit a problem. The structure is (partially) type DataQuery struct { Date null.Time `json:"DateTime"` .... The template is {{define "plainBody"}} Hi, Here are the results for the check run for today. The number of rows returned is {{.Rows}} The data is {{ range .Data}} {{.Date}} {{end}} {{end}} And the results of running that template are Hi, Here are the results for the check run for today. The number of rows returned is 57 The data is {{2021-09-13 00:00:00 +0000 +0000 true}} {{2021-08-16 00:00:00 +0000 +0000 true}} {{2021-09-19 00:00:00 +0000 +0000 true}} {{2021-09-18 00:00:00 +0000 +0000 true}} I tried using {{.Date.EncodeText}} and ended up with [50 48 50 49 45 48 57 45 49 51 84 48 48 58 48 48 58 48 48 90] [50 48 50 49 45 48 56 45 49 54 84 48 48 58 48 48 58 48 48 90] [50 48 50 49 45 48 57 45 49 57 84 48 48 58 48 48 58 48 48 90] For the datetime fields (which might be a []byte of the strings but I'm not sure. If I use {{Date.Value}} I get 2021-09-13 00:00:00 +0000 +0000 The other field types (string, int, float) all work fine with {{Variable.ValueOrZero}} I think I'm close.. but can't quite crack it for the date time fields
First, you are using html/template which provides context-sensitive escaping, that's why you're seeing those + sequences. If you want text output, use text/template instead. For details, see Template unnecessarily escaping `<` to `<` but not `>` Next, null.Time is not just a simple time.Time value, it wraps other fields too (whether the time is valid). When simply outputting it, that valid field will also be rendered (the true texts in your output). You may render only its Time field: {{.Date.Time}}. With these changes output will be for example: Hi, Here are the results for the check run for today. The number of rows returned is 2 The data is 2021-09-20 12:10:00 +0000 UTC 2021-10-11 13:50:00 +0000 UTC Try it on the Go Playground.
What solution should I use to generate a list of all possible alphabetic combinaisons?
I want to generate a list of all the possible combinations of the following characters with a minimum length of 3 characters and a maximum length of 12 characters. abcdefghijklmnopqrstuvwxyz1234567890_ I though of using PHP to do so this but this operation requires too much memory. What would be the best tool to achieve this?
It would be better if you set a limit on each run; For example all possibilities with 5 characters in one run, and all with 7 in another. And write a code to send the output after each run to a text file so you have all the possibilities and That would take less memory. example with numbers in python: # 1 2 3 4 5 6 7 8 9 0 listx=[1,2,3,4,5,6,7,8,9,0] #one letter for i in listx: print(i) #two letters for i in listx: for j in listx: print(f"{i}{j}") and it goes on and on... output=> 1 2 3 4 5 6 7 8 9 0 11 12 13 14 15 16 17 18 19 10 21 22 23 24 25 26 27 28 29 20 31 32 33 34 35 36 37 38 39 30 41 42 43 44 45 46 47 48 49 40 51 52 53 54 55 56 57 58 59 50 61 62 63 64 65 66 67 68 69 60 71 72 73 74 75 76 77 78 79 70 81 82 83 84 85 86 87 88 89 80 91 92 93 94 95 96 97 98 99 90 01 02 03 04 05 06 07 08 09 00
In python, there is a function itertools.product which returns the combinations you want for a fixed number of characters. You can call it repeatedly to get each number of characters between 3 and 12. def get_combinations(charset, begin, end): result = [] for i in range(begin, end+1): result.extend(''.join(p) for p in itertools.product(charset, repeat=i)) return result print(get_combinations('abcdefghijklmnopqrstuvwxyz0123456789_', 3, 5)) # ['aaa', 'aab', 'aac', 'aad', 'aae', 'aaf', 'aag', 'aah', 'aai', 'aaj', 'aak', 'aal', 'aam', 'aan', 'aao', 'aap', 'aaq', 'aar', 'aas', 'aat', 'aau', 'aav', 'aaw', 'aax', 'aay', 'aaz', 'aa0', 'aa1', 'aa2', 'aa3', 'aa4', 'aa5', 'aa6', 'aa7', 'aa8', 'aa9', 'aa_', 'aba', 'abb', 'abc', 'abd', 'abe', 'abf', 'abg', 'abh', 'abi', 'abj', 'abk', 'abl', 'abm', 'abn', 'abo', 'abp', 'abq', 'abr', 'abs', 'abt', 'abu', 'abv', 'abw', 'abx', 'aby', 'abz', 'ab0', 'ab1', 'ab2', 'ab3', 'ab4', 'ab5', 'ab6', 'ab7', 'ab8', 'ab9', 'ab_', 'aca', 'acb', 'acc', 'acd', 'ace', 'acf', 'acg', 'ach', 'aci', 'acj', 'ack', 'acl', 'acm', 'acn', 'aco', 'acp', 'acq', 'acr', 'acs', 'act', 'acu', 'acv', 'acw', 'acx', 'acy', 'acz', 'ac0', 'ac1', 'ac2', 'ac3', 'ac4', 'ac5', 'ac6', 'ac7', 'ac8', 'ac9', 'ac_', 'ada', 'adb', 'adc', 'add', 'ade', 'adf', 'adg', 'adh', 'adi', 'adj', 'adk', 'adl', 'adm', 'adn', 'ado', 'adp', 'adq', 'adr', 'ads', 'adt', 'adu', 'adv', 'adw', 'adx', 'ady', 'adz', 'ad0', 'ad1', 'ad2', 'ad3', 'ad4', 'ad5', 'ad6', 'ad7', 'ad8', 'ad9', 'ad_', 'aea', 'aeb', 'aec', 'aed', 'aee', 'aef', 'aeg', ..., '__o0', '__o1', '__o2', '__o3', '__o4', '__o5', '__o6', '__o7', '__o8', '__o9', '__o_', '__pa', '__pb', '__pc', '__pd', '__pe', '__pf', '__pg', '__ph', '__pi', '__pj', '__pk', '__pl', '__pm', '__pn', '__po', '__pp', '__pq', '__pr', '__ps', '__pt', '__pu', '__pv', '__pw', '__px', '__py', '__pz', '__p0', '__p1', '__p2', '__p3', '__p4', '__p5', '__p6', '__p7', '__p8', '__p9', '__p_', '__qa', '__qb', '__qc', '__qd', '__qe', '__qf', '__qg', '__qh', '__qi', '__qj', '__qk', '__ql', '__qm', '__qn', '__qo', '__qp', '__qq', '__qr', '__qs', '__qt', '__qu', '__qv', '__qw', '__qx', '__qy', '__qz', '__q0', '__q1', '__q2', '__q3', '__q4', '__q5', '__q6', '__q7', '__q8', '__q9', '__q_', '__ra', '__rb', '__rc', '__rd', '__re', '__rf', '__rg', '__rh', '__ri', '__rj', '__rk', '__rl', '__rm', '__rn', '__ro', '__rp', '__rq', '__rr', '__rs', '__rt', '__ru', '__rv', '__rw', '__rx', '__ry', '__rz', '__r0', '__r1', '__r2', '__r3', '__r4', '__r5', '__r6', '__r7', '__r8', '__r9', '__r_', '__sa', '__sb', '__sc', '__sd', '__se', '__sf', '__sg', '__sh', '__si', '__sj', '__sk', '__sl', '__sm', '__sn', '__so', '__sp', '__sq', '__sr', '__ss', '__st', '__su', '__sv', '__sw', '__sx', '__sy', '__sz', '__s0', '__s1', '__s2', '__s3', '__s4', '__s5', '__s6', '__s7', '__s8', '__s9', '__s_', '__ta', '__tb', '__tc', '__td', '__te', '__tf', '__tg', '__th', '__ti', '__tj', '__tk', '__tl', '__tm', '__tn', '__to', '__tp', '__tq', '__tr', '__ts', '__tt', '__tu', '__tv', '__tw', '__tx', '__ty', '__tz', '__t0', '__t1', '__t2', '__t3', '__t4', '__t5', '__t6', '__t7', '__t8', '__t9', '__t_', '__ua', '__ub', '__uc', '__ud', '__ue', '__uf', '__ug', '__uh', '__ui', '__uj', '__uk', '__ul', '__um', '__un', '__uo', '__up', '__uq', '__ur', '__us', '__ut', '__uu', '__uv', '__uw', '__ux', '__uy', '__uz', '__u0', '__u1', '__u2', '__u3', '__u4', '__u5', '__u6', '__u7', '__u8', '__u9', '__u_', '__va', '__vb', '__vc', '__vd', '__ve', '__vf', '__vg', '__vh', '__vi', '__vj', '__vk', '__vl', '__vm', '__vn', '__vo', '__vp', '__vq', '__vr', '__vs', '__vt', '__vu', '__vv', '__vw', '__vx', '__vy', '__vz', '__v0', '__v1', '__v2', '__v3', '__v4', '__v5', '__v6', '__v7', '__v8', '__v9', '__v_', '__wa', '__wb', '__wc', '__wd', '__we', '__wf', '__wg', '__wh', '__wi', '__wj', '__wk', '__wl', '__wm', '__wn', '__wo', '__wp', '__wq', '__wr', '__ws', '__wt', '__wu', '__wv', '__ww', '__wx', '__wy', '__wz', '__w0', '__w1', '__w2', '__w3', '__w4', '__w5', '__w6', '__w7', '__w8', '__w9', '__w_', '__xa', '__xb', '__xc', '__xd', '__xe', '__xf', '__xg', '__xh', '__xi', '__xj', '__xk', '__xl', '__xm', '__xn', '__xo', '__xp', '__xq', '__xr', '__xs', '__xt', '__xu', '__xv', '__xw', '__xx', '__xy', '__xz', '__x0', '__x1', '__x2', '__x3', '__x4', '__x5', '__x6', '__x7', '__x8', '__x9', '__x_', '__ya', '__yb', '__yc', '__yd', '__ye', '__yf', '__yg', '__yh', '__yi', '__yj', '__yk', '__yl', '__ym', '__yn', '__yo', '__yp', '__yq', '__yr', '__ys', '__yt', '__yu', '__yv', '__yw', '__yx', '__yy', '__yz', '__y0', '__y1', '__y2', '__y3', '__y4', '__y5', '__y6', '__y7', '__y8', '__y9', '__y_', '__za', '__zb', '__zc', '__zd', '__ze', '__zf', '__zg', '__zh', '__zi', '__zj', '__zk', '__zl', '__zm', '__zn', '__zo', '__zp', '__zq', '__zr', '__zs', '__zt', '__zu', '__zv', '__zw', '__zx', '__zy', '__zz', '__z0', '__z1', '__z2', '__z3', '__z4', '__z5', '__z6', '__z7', '__z8', '__z9', '__z_', '__0a', '__0b', '__0c', '__0d', '__0e', '__0f', '__0g', '__0h', '__0i', '__0j', '__0k', '__0l', '__0m', '__0n', '__0o', '__0p', '__0q', '__0r', '__0s', '__0t', '__0u', '__0v', '__0w', '__0x', '__0y', '__0z', '__00', '__01', '__02', '__03', '__04', '__05', '__06', '__07', '__08', '__09', '__0_', '__1a', '__1b', '__1c', '__1d', '__1e', '__1f', '__1g', '__1h', '__1i', '__1j', '__1k', '__1l', '__1m', '__1n', '__1o', '__1p', '__1q', '__1r', '__1s', '__1t', '__1u', '__1v', '__1w', '__1x', '__1y', '__1z', '__10', '__11', '__12', '__13', '__14', '__15', '__16', '__17', '__18', '__19', '__1_', '__2a', '__2b', '__2c', '__2d', '__2e', '__2f', '__2g', '__2h', '__2i', '__2j', '__2k', '__2l', '__2m', '__2n', '__2o', '__2p', '__2q', '__2r', '__2s', '__2t', '__2u', '__2v', '__2w', '__2x', '__2y', '__2z', '__20', '__21', '__22', '__23', '__24', '__25', '__26', '__27', '__28', '__29', '__2_', '__3a', '__3b', '__3c', '__3d', '__3e', '__3f', '__3g', '__3h', '__3i', '__3j', '__3k', '__3l', '__3m', '__3n', '__3o', '__3p', '__3q', '__3r', '__3s', '__3t', '__3u', '__3v', '__3w', '__3x', '__3y', '__3z', '__30', '__31', '__32', '__33', '__34', '__35', '__36', '__37', '__38', '__39', '__3_', '__4a', '__4b', '__4c', '__4d', '__4e', '__4f', '__4g', '__4h', '__4i', '__4j', '__4k', '__4l', '__4m', '__4n', '__4o', '__4p', '__4q', '__4r', '__4s', '__4t', '__4u', '__4v', '__4w', '__4x', '__4y', '__4z', '__40', '__41', '__42', '__43', '__44', '__45', '__46', '__47', '__48', '__49', '__4_', '__5a', '__5b', '__5c', '__5d', '__5e', '__5f', '__5g', '__5h', '__5i', '__5j', '__5k', '__5l', '__5m', '__5n', '__5o', '__5p', '__5q', '__5r', '__5s', '__5t', '__5u', '__5v', '__5w', '__5x', '__5y', '__5z', '__50', '__51', '__52', '__53', '__54', '__55', '__56', '__57', '__58', '__59', '__5_', '__6a', '__6b', '__6c', '__6d', '__6e', '__6f', '__6g', '__6h', '__6i', '__6j', '__6k', '__6l', '__6m', '__6n', '__6o', '__6p', '__6q', '__6r', '__6s', '__6t', '__6u', '__6v', '__6w', '__6x', '__6y', '__6z', '__60', '__61', '__62', '__63', '__64', '__65', '__66', '__67', '__68', '__69', '__6_', '__7a', '__7b', '__7c', '__7d', '__7e', '__7f', '__7g', '__7h', '__7i', '__7j', '__7k', '__7l', '__7m', '__7n', '__7o', '__7p', '__7q', '__7r', '__7s', '__7t', '__7u', '__7v', '__7w', '__7x', '__7y', '__7z', '__70', '__71', '__72', '__73', '__74', '__75', '__76', '__77', '__78', '__79', '__7_', '__8a', '__8b', '__8c', '__8d', '__8e', '__8f', '__8g', '__8h', '__8i', '__8j', '__8k', '__8l', '__8m', '__8n', '__8o', '__8p', '__8q', '__8r', '__8s', '__8t', '__8u', '__8v', '__8w', '__8x', '__8y', '__8z', '__80', '__81', '__82', '__83', '__84', '__85', '__86', '__87', '__88', '__89', '__8_', '__9a', '__9b', '__9c', '__9d', '__9e', '__9f', '__9g', '__9h', '__9i', '__9j', '__9k', '__9l', '__9m', '__9n', '__9o', '__9p', '__9q', '__9r', '__9s', '__9t', '__9u', '__9v', '__9w', '__9x', '__9y', '__9z', '__90', '__91', '__92', '__93', '__94', '__95', '__96', '__97', '__98', '__99', '__9_', '___a', '___b', '___c', '___d', '___e', '___f', '___g', '___h', '___i', '___j', '___k', '___l', '___m', '___n', '___o', '___p', '___q', '___r', '___s', '___t', '___u', '___v', '___w', '___x', '___y', '___z', '___0', '___1', '___2', '___3', '___4', '___5', '___6', '___7', '___8', '___9', '____'] Note how I called the function with parameters 3 and 5 instead of 3 and 12. With parameters 3 and 5, the number of combinations is already 71268771. Over 71 millions. With parameters 3 and 12, the number of combinations would be 6765811783780034854. That's 6.8 * 10**18. This is nearly one thousand million times the number of humans on Earth.
ncurses: init_color() has no effect
Trying to define color pairs, I was getting strange results. All 256 colors are already defined, and attempt to change any color with init_color() has no affect. I'm using Putty with 256-colors enabled and TERM=xterm-256color (also putty-256color), ncurses 6.0 compiled with --enable-widec and --enable-ext-colors. This shows all colors are defined and the init_color() doesn't change anything even though it succeeds: init_scr(); start_color(); if (has_colors() && COLORS == 256 && can_change_color()) { NCURSES_COLOR_T f; for (f = 1; f < 256; f++) { if (init_pair(f, f, COLOR_BLACK) == ERR) break; attron(COLOR_PAIR(f)); printw("(%d)", f); attroff(COLOR_PAIR(f)); refresh(); } getch(); clear(); for (f = 1; f < 256; f++) { if (init_color(f, 0, 0, f*3) == ERR) break; if (init_pair(f, f, COLOR_BLACK) == ERR) break; attron(COLOR_PAIR(f)); printw("(%d)", f); attroff(COLOR_PAIR(f)); refresh(); } getch(); clear(); } I've read that the default colors can't be changed, but only refers to COLOR_BLACK, etc (0-7). Where are these 256 default colors defined and why can't I change them? If they can't be changed, I could make use of the colors defined, but only if I can rely on them being the same on any 256-color capable terminal.
short: PuTTY doesn't do that, ncurses can't tell if PuTTY can... long: In ncurses, the init_color function checks its parameters (in the example given, those appear okay if your $TERM is "xterm-256color"), as well as checking if the terminal description has the initc (initialize_color) capability. If that is missing or cancelled, ncurses returns an error. However, that's only the terminal description. ncurses cannot tell if you have chosen an incorrect or inappropriate terminal description. In a quick check, PuTTY doesn't respond to the control sequence which is used in initc. This is a known limitation, as indicated in the (more appropriate) terminal description putty-256color provided by ncurses: putty-256color|PuTTY 0.58 with xterm 256-colors, use=xterm+256setaf, use=putty, That xterm+256setaf is used for terminals whose palette is hard-coded. PuTTY is not the only terminal which both sets TERM=xterm` and lacks the ability to change its palette. If you happen to be using an old version of the terminal database, you may be misled, since that error was fixed in 2014: # 2014-03-30 # * cancel ccc in putty-256color and konsole-256color for consistency # with the cancelled initc capability (patch by Sven Zuhlsdorf). # * add xterm+256setaf building block for various terminals which only # get the 256-color feature half-implemented -TD # * updated "st" entry (leaving the 0.1.1 version as "simpleterm") to # 0.4.1 -TD # Like the other terminals whose developers set TERM=xterm (or TERM=xterm-256color), there are differences between those and xterm. Further reading: Why not just use TERM set to "xterm"?
Couple of things I discovered. First, yes I was apparently referencing an old putty-256color terminfo that had "ccc", allowing can_change_color() to succeed, but then init_color() would fail. But the same Putty window using "xterm-256color" would init_color() OK and color_content() even shows the new values, but nothing changed on the screen. What was really confusing is sometimes the colors I set would appear and other times seemingly random colors appeared instead. Here's what I found: putty-256color xterm-256color gnome-256color xterm-256color (putty) (putty) (gnome-terminal) (MobaXterm) -------------- -------------- ---------------- -------------- change_color OK/ERR OK OK OK init_color ERR ERR OK OK color_content ERR OK/NOCH OK OK color changed? NO NO YES YES So there's basically no way to determine whether colors can be changed or not. But I did find that every terminal had already defined the standard 256 xterm colors, whether they could be changed or not. So, now, I just define the colors I want to use using the same color numbers as in the xterm palette. That way, the colors I expect will appear whether I needed to define them or not. So, to use "PaleGreen3", I just use: init_color(77, 372, 843, 372) If it works, it works, and if not, its probably already defined. For reference, I converting all the Xwindow/xterm colors from GUI hex notation to the ncurses (0-1000) values: # Name Tk Ncurses --- ---------------- ------- ------------- 16 Grey0 #000000 0,0,0 17 NavyBlue #00005f 0,0,372 18 DarkBlue #000087 0,0,529 19 Blue3 #0000af 0,0,686 20 Blue3 #0000d7 0,0,843 21 Blue1 #0000ff 0,0,1000 22 DarkGreen #005f00 0,372,0 23 DeepSkyBlue4 #005f5f 0,372,372 24 DeepSkyBlue4 #005f87 0,372,529 25 DeepSkyBlue4 #005faf 0,372,686 26 DodgerBlue3 #005fd7 0,372,843 27 DodgerBlue2 #005fff 0,372,1000 28 Green4 #008700 0,529,0 29 SpringGreen4 #00875f 0,529,372 30 Turquoise4 #008787 0,529,529 31 DeepSkyBlue3 #0087af 0,529,686 32 DeepSkyBlue3 #0087d7 0,529,843 33 DodgerBlue1 #0087ff 0,529,1000 34 Green3 #00af00 0,686,0 35 SpringGreen3 #00af5f 0,686,372 36 DarkCyan #00af87 0,686,529 37 LightSeaGreen #00afaf 0,686,686 38 DeepSkyBlue2 #00afd7 0,686,843 39 DeepSkyBlue1 #00afff 0,686,1000 40 Green3 #00d700 0,843,0 41 SpringGreen3 #00d75f 0,843,372 42 SpringGreen2 #00d787 0,843,529 43 Cyan3 #00d7af 0,843,686 44 DarkTurquoise #00d7d7 0,843,843 45 Turquoise2 #00d7ff 0,843,1000 46 Green1 #00ff00 0,1000,0 47 SpringGreen2 #00ff5f 0,1000,372 48 SpringGreen1 #00ff87 0,1000,529 49 MediumSpringGreen #00ffaf 0,1000,686 50 Cyan2 #00ffd7 0,1000,843 51 Cyan1 #00ffff 0,1000,1000 52 DarkRed #5f0000 372,0,0 53 DeepPink4 #5f005f 372,0,372 54 Purple4 #5f0087 372,0,529 55 Purple4 #5f00af 372,0,686 56 Purple3 #5f00d7 372,0,843 57 BlueViolet #5f00ff 372,0,1000 58 Orange4 #5f5f00 372,372,0 59 Grey37 #5f5f5f 372,372,372 60 MediumPurple4 #5f5f87 372,372,529 61 SlateBlue3 #5f5faf 372,372,686 62 SlateBlue3 #5f5fd7 372,372,843 63 RoyalBlue1 #5f5fff 372,372,1000 64 Chartreuse4 #5f8700 372,529,0 65 DarkSeaGreen4 #5f875f 372,529,372 66 PaleTurquoise4 #5f8787 372,529,529 67 SteelBlue #5f87af 372,529,686 68 SteelBlue3 #5f87d7 372,529,843 69 CornflowerBlue #5f87ff 372,529,1000 70 Chartreuse3 #5faf00 372,686,0 71 DarkSeaGreen4 #5faf5f 372,686,372 72 CadetBlue #5faf87 372,686,529 73 CadetBlue #5fafaf 372,686,686 74 SkyBlue3 #5fafd7 372,686,843 75 SteelBlue1 #5fafff 372,686,1000 76 Chartreuse3 #5fd700 372,843,0 77 PaleGreen3 #5fd75f 372,843,372 78 SeaGreen3 #5fd787 372,843,529 79 Aquamarine3 #5fd7af 372,843,686 80 MediumTurquoise #5fd7d7 372,843,843 81 SteelBlue1 #5fd7ff 372,843,1000 82 Chartreuse2 #5fff00 372,1000,0 83 SeaGreen2 #5fff5f 372,1000,372 84 SeaGreen1 #5fff87 372,1000,529 85 SeaGreen1 #5fffaf 372,1000,686 86 Aquamarine1 #5fffd7 372,1000,843 87 DarkSlateGray2 #5fffff 372,1000,1000 88 DarkRed #870000 529,0,0 89 DeepPink4 #87005f 529,0,372 90 DarkMagenta #870087 529,0,529 91 DarkMagenta #8700af 529,0,686 92 DarkViolet #8700d7 529,0,843 93 Purple #8700ff 529,0,1000 94 Orange4 #875f00 529,372,0 95 LightPink4 #875f5f 529,372,372 96 Plum4 #875f87 529,372,529 97 MediumPurple3 #875faf 529,372,686 98 MediumPurple3 #875fd7 529,372,843 99 SlateBlue1 #875fff 529,372,1000 100 Yellow4 #878700 529,529,0 101 Wheat4 #87875f 529,529,372 102 Grey53 #878787 529,529,529 103 LightSlateGrey #8787af 529,529,686 104 MediumPurple #8787d7 529,529,843 105 LightSlateBlue #8787ff 529,529,1000 106 Yellow4 #87af00 529,686,0 107 DarkOliveGreen3 #87af5f 529,686,372 108 DarkSeaGreen #87af87 529,686,529 109 LightSkyBlue3 #87afaf 529,686,686 110 LightSkyBlue3 #87afd7 529,686,843 111 SkyBlue2 #87afff 529,686,1000 112 Chartreuse2 #87d700 529,843,0 113 DarkOliveGreen3 #87d75f 529,843,372 114 PaleGreen3 #87d787 529,843,529 115 DarkSeaGreen3 #87d7af 529,843,686 116 DarkSlateGray3 #87d7d7 529,843,843 117 SkyBlue1 #87d7ff 529,843,1000 118 Chartreuse1 #87ff00 529,1000,0 119 LightGreen #87ff5f 529,1000,372 120 LightGreen #87ff87 529,1000,529 121 PaleGreen1 #87ffaf 529,1000,686 122 Aquamarine1 #87ffd7 529,1000,843 123 DarkSlateGray1 #87ffff 529,1000,1000 124 Red3 #af0000 686,0,0 125 DeepPink4 #af005f 686,0,372 126 MediumVioletRed #af0087 686,0,529 127 Magenta3 #af00af 686,0,686 128 DarkViolet #af00d7 686,0,843 129 Purple #af00ff 686,0,1000 130 DarkOrange3 #af5f00 686,372,0 131 IndianRed #af5f5f 686,372,372 132 HotPink3 #af5f87 686,372,529 133 MediumOrchid3 #af5faf 686,372,686 134 MediumOrchid #af5fd7 686,372,843 135 MediumPurple2 #af5fff 686,372,1000 136 DarkGoldenrod #af8700 686,529,0 137 LightSalmon3 #af875f 686,529,372 138 RosyBrown #af8787 686,529,529 139 Grey63 #af87af 686,529,686 140 MediumPurple2 #af87d7 686,529,843 141 MediumPurple1 #af87ff 686,529,1000 142 Gold3 #afaf00 686,686,0 143 DarkKhaki #afaf5f 686,686,372 144 NavajoWhite3 #afaf87 686,686,529 145 Grey69 #afafaf 686,686,686 146 LightSteelBlue3 #afafd7 686,686,843 147 LightSteelBlue #afafff 686,686,1000 148 Yellow3 #afd700 686,843,0 149 DarkOliveGreen3 #afd75f 686,843,372 150 DarkSeaGreen3 #afd787 686,843,529 151 DarkSeaGreen2 #afd7af 686,843,686 152 LightCyan3 #afd7d7 686,843,843 153 LightSkyBlue1 #afd7ff 686,843,1000 154 GreenYellow #afff00 686,1000,0 155 DarkOliveGreen2 #afff5f 686,1000,372 156 PaleGreen1 #afff87 686,1000,529 157 DarkSeaGreen2 #afffaf 686,1000,686 158 DarkSeaGreen1 #afffd7 686,1000,843 159 PaleTurquoise1 #afffff 686,1000,1000 160 Red3 #d70000 843,0,0 161 DeepPink3 #d7005f 843,0,372 162 DeepPink3 #d70087 843,0,529 163 Magenta3 #d700af 843,0,686 164 Magenta3 #d700d7 843,0,843 165 Magenta2 #d700ff 843,0,1000 166 DarkOrange3 #d75f00 843,372,0 167 IndianRed #d75f5f 843,372,372 168 HotPink3 #d75f87 843,372,529 169 HotPink2 #d75faf 843,372,686 170 Orchid #d75fd7 843,372,843 171 MediumOrchid1 #d75fff 843,372,1000 172 Orange3 #d78700 843,529,0 173 LightSalmon3 #d7875f 843,529,372 174 LightPink3 #d78787 843,529,529 175 Pink3 #d787af 843,529,686 176 Plum3 #d787d7 843,529,843 177 Violet #d787ff 843,529,1000 178 Gold3 #d7af00 843,686,0 179 LightGoldenrod3 #d7af5f 843,686,372 180 Tan #d7af87 843,686,529 181 MistyRose3 #d7afaf 843,686,686 182 Thistle3 #d7afd7 843,686,843 183 Plum2 #d7afff 843,686,1000 184 Yellow3 #d7d700 843,843,0 185 Khaki3 #d7d75f 843,843,372 186 LightGoldenrod2 #d7d787 843,843,529 187 LightYellow3 #d7d7af 843,843,686 188 Grey84 #d7d7d7 843,843,843 189 LightSteelBlue1 #d7d7ff 843,843,1000 190 Yellow2 #d7ff00 843,1000,0 191 DarkOliveGreen1 #d7ff5f 843,1000,372 192 DarkOliveGreen1 #d7ff87 843,1000,529 193 DarkSeaGreen1 #d7ffaf 843,1000,686 194 Honeydew2 #d7ffd7 843,1000,843 195 LightCyan1 #d7ffff 843,1000,1000 196 Red1 #ff0000 1000,0,0 197 DeepPink2 #ff005f 1000,0,372 198 DeepPink1 #ff0087 1000,0,529 199 DeepPink1 #ff00af 1000,0,686 200 Magenta2 #ff00d7 1000,0,843 201 Magenta1 #ff00ff 1000,0,1000 202 OrangeRed1 #ff5f00 1000,372,0 203 IndianRed1 #ff5f5f 1000,372,372 204 IndianRed1 #ff5f87 1000,372,529 205 HotPink #ff5faf 1000,372,686 206 HotPink #ff5fd7 1000,372,843 207 MediumOrchid1 #ff5fff 1000,372,1000 208 DarkOrange #ff8700 1000,529,0 209 Salmon1 #ff875f 1000,529,372 210 LightCoral #ff8787 1000,529,529 211 PaleVioletRed1 #ff87af 1000,529,686 212 Orchid2 #ff87d7 1000,529,843 213 Orchid1 #ff87ff 1000,529,1000 214 Orange1 #ffaf00 1000,686,0 215 SandyBrown #ffaf5f 1000,686,372 216 LightSalmon1 #ffaf87 1000,686,529 217 LightPink1 #ffafaf 1000,686,686 218 Pink1 #ffafd7 1000,686,843 219 Plum1 #ffafff 1000,686,1000 220 Gold1 #ffd700 1000,843,0 221 LightGoldenrod2 #ffd75f 1000,843,372 222 LightGoldenrod2 #ffd787 1000,843,529 223 NavajoWhite1 #ffd7af 1000,843,686 224 MistyRose1 #ffd7d7 1000,843,843 225 Thistle1 #ffd7ff 1000,843,1000 226 Yellow1 #ffff00 1000,1000,0 227 LightGoldenrod1 #ffff5f 1000,1000,372 228 Khaki1 #ffff87 1000,1000,529 229 Wheat1 #ffffaf 1000,1000,686 230 Cornsilk1 #ffffd7 1000,1000,843 231 Grey100 #ffffff 1000,1000,1000 232 Grey3 #080808 31,31,31 233 Grey7 #121212 70,70,70 234 Grey11 #1c1c1c 109,109,109 235 Grey15 #262626 149,149,149 236 Grey19 #303030 188,188,188 237 Grey23 #3a3a3a 227,227,227 238 Grey27 #444444 266,266,266 239 Grey30 #4e4e4e 305,305,305 240 Grey35 #585858 345,345,345 241 Grey39 #626262 384,384,384 242 Grey42 #6c6c6c 423,423,423 243 Grey46 #767676 462,462,462 244 Grey50 #808080 501,501,501 245 Grey54 #8a8a8a 541,541,541 246 Grey58 #949494 580,580,580 247 Grey62 #9e9e9e 619,619,619 248 Grey66 #a8a8a8 658,658,658 249 Grey70 #b2b2b2 698,698,698 250 Grey74 #bcbcbc 737,737,737 251 Grey78 #c6c6c6 776,776,776 252 Grey82 #d0d0d0 815,815,815 253 Grey85 #dadada 854,854,854 254 Grey89 #e4e4e4 894,894,894 255 Grey93 #eeeeee 933,933,933
Merge and matching tables in Oracle
Does anyone know how to merge two tables with a common column name and data into a single table? The shared column is a date column. This is part of a project at work, no one here quite knows how it works. Any help would be appreciated. table A Sub Temp Weight Silicon Cast_Date 108 2675 2731 0.7002 18-jun-11 18:45 101 2691 3268 0.6194 18-jun-11 20:30 107 2701 6749 0.6976 18-jun-11 20:30 113 2713 2112 0.6616 18-jun-11 20:30 116 2733 3142 0.7382 19-jun-11 05:46 121 2745 2611 0.6949 19-jun-11 00:19 125 2726 1995 0.644 19-jun-11 00:19 table B Si Temperature Sched_Cast_Date Treadwell 0.6622 2542 01-APR-11 02:57 114 0.6622 2542 01-APR-11 03:07 116 0.7516 2526 19-jun-11 05:46 116 0.7516 2526 01-APR-11 03:40 107 0.6741 2372 01-APR-11 04:03 107 0.6206 2369 01-APR-11 09:43 114 0.6741 2372 19-jun-11 00:19 125 the results would look like: Subcar Temp Weight Silicon Cast_Date SI Temperature Sched_Cast_Date Treadwell 116 2733 3142 0.7382 19-jun-11 05:46 0.7516 2526 19-jun-11 05:46 116 125 2726 1995 0.644 19-jun-11 00:19 0.6741 2372 19-jun-11 00:19 125 I would like to run a query that returns a results data only where Sched_Cast_Date and Cast_Date are the same. A table with the same qualities would work just as well. I hope that this makes more sense.
Are you asking how to join two tables on a common column? i.e. select a.Sub, a.Temp, a.Weight a.Silicon a.Cast_Date, b.SI, b.Temperature, b.Sched_Cast_Date, b.Treadwell from a join b on b.sched_cast_date = a.cast_date