I added an attribute to a control curve that changes the start frame of multiple groups. This is the expression that I come up with that I put in the expression editor.
$basename = "R_M_TM_Step_01_ctrl";
string $selected[] = `ls -sl -sn` ;
string $result = substituteAllString($selected[0], $basename , "");
string $addName = ($basename + $result);
if ($selected[0] == $addName )
{
if ($selected[0] == $addName )
{
int $tyme=`getAttr $selected.StartFrame`;
SelectHierarchy;
copyKey -time ":" ;
cutKey -time ":" ;
pasteKey -time $tyme;
select $selected;
}
if ($selected[0] == $addName )
{
int $tyme=`getAttr $selected.StartFrame`;
SelectHierarchy;
copyKey -time ":" ;
cutKey -time ":" ;
pasteKey -time $tyme;
select $selected;
}
}
The problem is that I need to import this control and groups multiple times. I need some way to query the select control curve (R_M_TM_Step_01_ctrl) even when it imported (new name R_M_TM_Step_01_ctrl1, R_M_TM_Step_01_ctrl2, and so on). For the most part this script works but is there a better to move all those keys. Or is there an expression evaluates only under that particular control curve.
Also I need a command change evaluation. If I have a evaluation set to "always" then it runs the expression every frame. I thought that "demand" would work but it doesn't. I think I need some kind of "if statement". Can you guys help?
Related
I'm currently transforming my survey results for analysis. The survey program I used (limesurvey) automatically generates value labels, but I need to change the value the labels corresponds with. SPSS sadly doesn't do that automatically with the recode command, which only changes the values but doesn't change the value labels that correspond to it (They keep pointing at the old value which doesn't have any data entries anymore).
Basically I want to change the value labels from this:
Value labels before change
to this: Value labels after (Changing the labeled Values from A1,A2,A3,A4,A5 into 1,2,3,4,5)
And I don't just want to do it for a single variable, but dozens of variables. While this can actually be accomplished by the search/replace function via the User Interface of SPSS (that's actually how I did it in the example), I would like to do this via Syntax, so I don't have to handcraft every dataset and don't risk screwing up my data with typing mistakes.
Does anyone know how it can be done with Syntax? (Or alternatively some plugin)
So I've found a solution to my problem.
I did the recodes and the relabeling with Python.
BEGIN PROGRAM PYTHON3.
import spss, spssaux, re
sDict = spssaux.VariableDict() #Get a copy of the Variable Dictionary
infotext = "Variables processed: " # begin infotext
#Iterate through all variables (in the dictionary - all variables in this case)
for var in sDict:
#Only Adress variables which are of type String and have value labels.
if spss.GetVariableType(var.index) > 0 and var.ValueLabels != {} :
infotext += spss.GetVariableName(var.index) + ', ' #Add processed variable to infotext
#Begin Value Labels and Recode commands for active variable
commandlab = 'Value Labels ' + spss.GetVariableName(var.index) + ' '
commandrecode = 'Recode ' + spss.GetVariableName(var.index) + ' '
#commandalter = 'Alter Type ' + spss.GetVariableName(var.index) + '(f2).'
#Open entry for each value of valuelabels
for key,val in var.ValueLabels.items():
newkey = re.findall(r'\d+', key)[0] #Get first number in key (value of label)
newkey = re.sub(r'0+(.+)', r'\1', newkey) #Cut leading zeros from number
#Add entry to recode and Value labels command:
commandlab = commandlab + newkey + ' ' + '"' + val + '"' + ' '
commandrecode = commandrecode + '("' + key + '"="' + newkey + '") '
#Complete commands for SPSS:
commandlab = commandlab + '.'
commandrecode = commandrecode + '.\n Execute.'
#Execute recode and relabel.
spss.Submit(commandlab)
spss.Submit(commandrecode)
#spss.Submit(commandalter)
#Print Infos
print(infotext)
print("Warning: Only Values with value labels have been recoded. Please check your dataset to make sure all entries have been affected.")
end program.
Basically this program is finding all string variables with value labels, gets the numbers from the value/key and recodes and relabels the variables. It can also automatically turn all variables numeric, but I wouldn't recommend doing that without checking all values got converted first. In my case I merged several datasets and one variable had a value without label as a result.
Does anyone know how to get the values (or frequencies) of a variable? If possible, I'd like to catch that problem by doing the recode independent of the value labels.
Firstly thank you for the forum members
I need to find time difference b'w two rows of timestamp using awk/shell.
Here is the logfile:
cat business_file
start:skdjh:22:06:2010:10:30:22
sdfnskjoeirg
wregn'wergnoeirnfqoeitgherg
end:siifneworigo:22:06:2010:10:45:34
start:srsdneriieroi:24:06:2010:11:00:45
dfglkndfogn
sdgsdfgdfhdfg
end:erfqefegoieg:24:06:2010:11:46:34
oeirgoeirg\
start:sdjfhsldf:25:07:2010:12:55:43
wrgnweoigwerg
ewgjw]egpojwepr
etwasdf
gwdsdfsdf
fgpwj]pgojwth
wtr
wtwrt
end:dgnoeingwoit:25:07:2010:01:42:12
===========
The above logfile is kind of api file, and there are some rows start with "start" and "end",and the corresponding row's column 3rd to end of the row is timestamp (take delimiter as ":" )
we have to find the time difference between start and end time consecutive rows
Hope I am clear with the question,please let me know if you need more explanation.
Thx
Srinivas
Since the timestamp is separated by the same field separator as the rest of the line, this does not even require manual splitting. Simply
awk -F : 'function timestamp() { return mktime($5 " " $4 " " $3 " " $6 " " $7 " " $8) } $1 == "start" { t_start = timestamp() } $1 == "end" { print(timestamp() - t_start) }' filename
works and prints the time difference in seconds. The code is
# return the timestamp for the current row, using the pre-split fields
function timestamp() {
return mktime($5 " " $4 " " $3 " " $6 " " $7 " " $8)
}
# in start lines, remember the timestamp
$1 == "start" {
t_start = timestamp()
}
# in end lines, print the difference.
$1 == "end" {
print(timestamp() - t_start)
}
If you want to format the time difference in another manner, see this handy reference of relevant functions. By the way, the last block in your example has several hours negative length. You may want to look into that.
Addendum: In case that's because of the am/pm thing some countries have, this opens up a can of worms in that all timestamps have two possible meanings (because the log file does not seem to include the information whether it's an am or pm timestamp), so you have an unsolvable problem with durations of more than half a day. If you know that durations are never longer than half a day and the end time is always after the start time, you might be able to hack your way around it with something like
$1 == "end" {
t_end = timestamp();
if(t_end < t_start) {
t_end += 43200 # add 12 hours
}
print(t_end - t_start)
}
...but in that case the log file format is broken and should be fixed. This sort of hackery is not something you want to rely on in the long term.
Hi I'm doing my first programming assignment (it's with Jython, i.e. Python using Java) and I've run into a syntax error on line 14 (bolded below). I've tried changing the variable to something less useful like "L" or "I" but it still gives the error. It's annoying because it makes no sense. I have tried indenting again and adding comments around it.
This is a program that outputs a picture of a soccer ball factory. It's as much as an artistic project as a comp sci project. So the printing looks complicated but is only like long checklist for building.
def prettyPic():
#building materials and parts
spacer = " "
ceiling_part = "-"
ball = "o"
wheel = ""
door_joint = "#"
left_half_arch = "/"
right_half_arch = "\\"
ladder = "\\"
wall = "|"
glass = (
#biox
**left_box = "u"
right_box = "u"**
#begin printing
print (spacer*30 + ceiling_part*30)
print (spacer*32 + wall*1) + (spacer*47 + wall*1)
#three balls, leaving space for drop
print (spacer*32 + wall*1) + (ball*27) + (wall*1)
#arches, not touching ceiling
etc, etc
The problem is in this line:
glass = (
This means that to glass variable, you assign tuple just like:
glass = (1, 2, 'some string')
Python interpreter searches for termination of just opened tuple but it finds only Python code that is not correct in this context.
Remove or comment out line with glass, or assign to glass variable some value.
I'm doing a large text mining project. I have 100,000 text files. I've extracted two- and three-word phrases from sets of 1,000 documents at a time and have created 100 files. Each file has roughly 8 million lines in this format:
total_references num_docs_referencing_phrase phrase
I want to create an aggregate list of total references and number of docs referencing each phrase by processing the 100 intermediate files. To that end I wrote this program.
#!/usr/bin/perl -w
$| = 1 ; # Don't buffer output
use File::Find ;
$dir = "/home/sl/phrase-counts" ;
find(\&processFile, $dir) ;
for $key ( keys %TOTALREFS ) {
print "$TOTALREFS{$key} $NUMDOCS{$key} ${key}\n" ;
}
sub processFile {
my $file = $_ ;
my $fullName = $File::Find::name ;
if ( $fullName =~ /\.txt$/ ) {
$date = `date` ;
chomp $date ;
print "($date) file: $fullName\n" ;
open INFILE, "$fullName" or die "Cannot read ${fullName}";
while ( <INFILE> ) {
my $line = $_ ;
chomp $line ;
( $totalRefs, $numDocs, $phrase ) = split (/\s+/, $line, 3) ;
$TOTALREFS{$phrase} += $totalRefs ;
$NUMDOCS{$phrase} += $numDocs ;
}
close ( INFILE ) ;
}
}
The code produces strange errors after 8 or so files are processed and then it hangs, i.e. it stops listing files it should be processing.
Use of uninitialized value $date in scalar chomp at ./getCounts line 21.
Use of uninitialized value $date in concatenation (.) or string at ./getCounts line 22.
I don't believe the problem is really my date command, especially since it runs fine for a number of early files processed and because the problem does not occur at the same point in the run every time I run it. I assume the problem is that my program is consuming too much system resource and corrupting the state of the running environment. Running top and watching memory use go up to 97% of the machine concerns me although I notice that the errors and hang occur before top shows little memory left. And, there is some swap on the machine.
My question is, how can I rewrite this program to actually complete its execution? With 8 million lines of data for each of 100 files there could be 800 million lines of output although I would guess that the total is more likely in the range of 50-100 million lines. I have done some cleanup of the data and could consider more aggressive sanitizing of phrases to cut down on the numbers but I'd like to understand how I can design this code better.
I've seen articles that tell programmers to put their data into a database. My concern is the time it might take to update a database 100 million times.
Suggestions?
It looks like you're running on a *nix system, so make sort do all the work for you. It knows how to use memory efficiently.
sort -k 3 all_your_input_files*.txt > sorted.txt
Why do this? Because now all lines corresponding to the same phrase appear in a single block within the file, so you can compute totals easily: just write a short Perl script that adds the current line's numbers to the current totals, and writes them out whenever the phrase changes from the previous line (and at the end):
my ($oldPhrase, $totTotalRefs, $totNumDocs) = (undef, 0, 0);
while ( <INFILE> ) {
my $line = $_ ;
chomp $line ;
( $totalRefs, $numDocs, $phrase ) = split (/\s+/, $line, 3) ;
if (defined($oldPhrase) && $phrase ne $oldPhrase) {
print "$totTotalRefs $totNumDocs $oldPhrase\n" ;
$totTotalRefs = $totNumDocs = 0;
}
$totTotalRefs += $totalRefs ;
$totNumDocs += $numDocs ;
$oldPhrase = $phrase;
}
close ( INFILE ) ;
print "$totTotalRefs $totNumDocs $oldPhrase\n" ;
The above code is untested, but should work with appropriate boilerplate added I think.
[EDIT: Fix bug in which $oldPhrase never gets set, as suggested by Sol.]
You are storing all of the different phrases as keys for both %TOTALREFS and %NUMDOCS, so things are at least twice as bad as they need to be.
I suggest you try the following
Add use strict and use warnings (instead of -w) and declare all of your variables properly
Don't use capitals in your variable names. Capital letters are reserved for global identifiers
Don't start 100 subprocesses just to get the time of day. Just use localtime like this
printf "(%s) file: %s\n", scalar localtime, $full_name;
Use find just to generate an array of the files to be processed, so it would look like this
my #files;
find(sub {
push #files, $File::Find::name if -f and /\.txt$/i;
}, $dir) ;
Then you can process each file with a simple for loop
for my $file (#files) {
...
}
Take two passes through the files, the first time generating a hash that relates each phrase to an integer starting at zero, and the second that uses those integers to index arrays #total_refs and #num_docs and increment their elements
You may still run out of memory, but those measures will certainly give you a better chance.
Update
Just to be clear, this is how I imagine it would work. I've done this as a single pass, but it may be better to write it as two passes as I described so that you can check your intermediate data.
Note that this isn't tested apart from making sure that it compiles.
#!/usr/bin/perl
use strict;
use warnings;
use 5.010;
use autodie;
STDOUT->autoflush;
use File::Find;
my $dir = '/home/sl/phrase-counts';
my #files;
find(sub {
push #files, $File::Find::name if -f and /\.txt$/i;
}, $dir);
my (%phrases, #total_refs, #num_docs);
my $num_phrases = 0;
for my $file (#files) {
printf "(%s) file: %s\n", scalar localtime, $file;
open my $in_fh, '<', $file;
while (<$in_fh>) {
chomp;
my ($total_refs, $num_docs, $phrase) = split ' ', $_, 3;
my $phrase_num = $phrases{$phrase} //= $num_phrases++;
$total_refs[$phrase_num] += $total_refs;
$num_docs[$phrase_num] += $num_docs;
}
}
for my $phrase (keys %phrases) {
my $phrase_num = $phrases{$phrase};
printf "%s %s %s\n",
$total_refs[$phrase_num],
$num_docs[$phrase_num],
$phrase_num;
}
Trying to use more resources than available causes exceptions for being unable to allocate memory or results in system calls returning error messages. It doesn't corruption memory.
In this case, the result of backticks is undef, which means the command could not be executed. That could very well be because you have insufficient memory left. Where did you get the idea that being unable to execute a program is the result of corrupted memory?! Furthermore, you have an error you don't understand, yet you didn't check what error was returned? Backticks sets $? (and $! when $? is negative) as per system. Assuming it's a bug in Perl is a very bad assumption to make, especially when the system tells you what error occurred.
Use less memory, either through the use of a more appropriate and/or efficient data structure, or by keeping a portion of the data out of memory (e.g. on disk or in a database).
The built-in VIM :sort command sorts lines of text. I want to sort words in a single line, e.g. transform the line
b a d c e f
to
a b c d e f
Currently I accomplish this by selecting the line and then using :!tr ' ' '\n' | sort | tr '\n' ' ', but I'm sure there's a better, simpler, quicker way. Is there?
Note that I use bash so if there's a shorter and more elegant bash command for doing this it's also fine.
EDIT: My use-case is that I have a line that says SOME_VARIABLE="one two three four etc" and I want the words in that variable to be sorted, i.e. I want to have SOME_VARIABLE="etc four one three two".
The end result should preferably be mappable to a shortcut key as this is something I find myself needing quite often.
In pure vim, you could do this:
call setline('.', join(sort(split(getline('.'), ' ')), " "))
Edit
To do this so that it works over a range that is less than one line is a little more complicated (this allows either sorting multiple lines individually or sorting part of one line, depending on the visual selection):
command! -nargs=0 -range SortWords call SortWords()
" Add a mapping, go to your string, then press vi",s
" vi" selects everything inside the quotation
" ,s calls the sorting algorithm
vmap ,s :SortWords<CR>
" Normal mode one: ,s to select the string and sort it
nmap ,s vi",s
function! SortWords()
" Get the visual mark points
let StartPosition = getpos("'<")
let EndPosition = getpos("'>")
if StartPosition[0] != EndPosition[0]
echoerr "Range spans multiple buffers"
elseif StartPosition[1] != EndPosition[1]
" This is a multiple line range, probably easiest to work line wise
" This could be made a lot more complicated and sort the whole
" lot, but that would require thoughts on how many
" words/characters on each line, so that can be an exercise for
" the reader!
for LineNum in range(StartPosition[1], EndPosition[1])
call setline(LineNum, join(sort(split(getline('.'), ' ')), " "))
endfor
else
" Single line range, sort words
let CurrentLine = getline(StartPosition[1])
" Split the line into the prefix, the selected bit and the suffix
" The start bit
if StartPosition[2] > 1
let StartOfLine = CurrentLine[:StartPosition[2]-2]
else
let StartOfLine = ""
endif
" The end bit
if EndPosition[2] < len(CurrentLine)
let EndOfLine = CurrentLine[EndPosition[2]:]
else
let EndOfLine = ""
endif
" The middle bit
let BitToSort = CurrentLine[StartPosition[2]-1:EndPosition[2]-1]
" Move spaces at the start of the section to variable StartOfLine
while BitToSort[0] == ' '
let BitToSort = BitToSort[1:]
let StartOfLine .= ' '
endwhile
" Move spaces at the end of the section to variable EndOfLine
while BitToSort[len(BitToSort)-1] == ' '
let BitToSort = BitToSort[:len(BitToSort)-2]
let EndOfLine = ' ' . EndOfLine
endwhile
" Sort the middle bit
let Sorted = join(sort(split(BitToSort, ' ')), ' ')
" Reform the line
let NewLine = StartOfLine . Sorted . EndOfLine
" Write it out
call setline(StartPosition[1], NewLine)
endif
endfunction
Using great ideas from your answers, especially Al's answer, I eventually came up with the following:
:vnoremap <F2> d:execute 'normal i' . join(sort(split(getreg('"'))), ' ')<CR>
This maps the F2 button in visual mode to delete the selected text, split, sort and join it and then re-insert it. When the selection spans multiple lines this will sort the words in all of them and output one sorted line, which I can quickly fix using gqq.
I'll be glad to hear suggestions on how this can be further improved.
Many thanks, I've learned a lot :)
EDIT: Changed '<C-R>"' to getreg('"') to handle text with the char ' in it.
Here's the equivalent in pure vimscript:
:call setline('.',join(sort(split(getline('.'),' ')),' '))
It's no shorter or simpler, but if this is something you do often, you can run it across a range of lines:
:%call setline('.',join(sort(split(getline('.'),' ')),' '))
Or make a command
:command -nargs=0 -range SortLine <line1>,<line2>call setline('.',join(sort(split(getline('.'),' ')),' '))
Which you can use with
:SortLine
:'<,'>SortLine
:%SortLine
etc etc
:!perl -ne '$,=" ";print sort split /\s+/'
Not sure if it requires explanation, but if yes:
perl -ne ''
runs whatever is within '' for every line in input - putting the line in default variable $_.
$,=" ";
Sets list output separator to space. For example:
=> perl -e 'print 1,2,3'
123
=> perl -e '$,=" ";print 1,2,3'
1 2 3
=> perl -e '$,=", ";print 1,2,3'
1, 2, 3
Pretty simple.
print sort split /\s+/
Is shortened version of:
print( sort( split( /\s+/, $_ ) ) )
($_ at the end is default variable).
split - splits $_ to array using given regexp, sort sorts given list, print - prints it.
Maybe you prefer Python:
!python -c "import sys; print(' '.join(sorted(sys.stdin.read().split())))"
Visual select text, and execute this line.
My AdvancedSorters plugin now has a :SortWORDs command that does this (among other sorting-related commands).