First of all: sorry for using c shell, blame my company not me. I hate the damn thing as much as most of you do now (at first I was like, hey this ain't so bad).
I am trying to subtract large numbers obtained from time stamps. Here is what I am trying:
set curTime = `date +%s%N`
#... some stuff
#curTime = `date +%s%N` - $curTime #get the diff
echo "time taken: $curTime"
However I guess the numbers are too big - before I tried with just seconds and it worked fine. Here's what I see in the log:
#curMilli = 1349996279792995000 - 1349996279170458000
#curMilli: Command not found.
As I said I do the exact same thing with date +%s and it's fine, so I'm assuming it's something about the largeness of the numbers.
How can I do this? Thanks a lot.
The article http://en.wikipedia.org/wiki/Bc_programming_language has a short section "Using bc in shell scripts". A test:
set curTime = `/bin/date +%s%N`
/bin/sleep 2
set prevTime = $curTime
set curTime = `/bin/date +%s%N`
set diff = `echo "$curTime - $prevTime;" | /usr/bin/bc`
echo $diff
will give (with the digits after the initial 20 variable):
2016204108
P.s: I wish I could vote you up twice for "I hate the damn thing as much as most of you do now (at first I was like, hey this ain't so bad)."
Related
I am trying to create a loop to iterate through 2 dates.
I have this so far, however it is going back in time. I want it to start with the start_date and end with the end_date.
I've tried to swap them around but it doesn't like that and adds +1day infinitely. Any help would be appreciated.
#!/usr/bin/env bash
end_date="21-12-21"
start_date="21-11-20"
# Set a counter variable
counter=0
#Increase the counter to get back in time
while [ "$end_date" != "$start_date" ]; do
end_date=$(date -v -${counter}d '+%y-%m-%d')
echo $end_date
counter=$((counter + 1))
done
This is what it produces
21-12-23
21-12-22
21-12-21
21-12-20
21-12-19
21-12-18
21-12-17
21-12-16
21-12-15
21-12-14
21-12-13
21-12-12
21-12-11
21-12-10
21-12-09
21-12-08
21-12-07
21-12-06
21-12-05
21-12-04
21-12-03
21-12-02
21-12-01
21-11-30
21-11-29
21-11-28
21-11-27
21-11-26
21-11-25
21-11-24
21-11-23
21-11-22
21-11-21
21-11-20
I'm also running this on a mac which seems to be quite fussy with the date formatting. This is the only way I have found it to work so far.
I have some large dataset plain text files (wikipedia articles) and I have to remove latin-1 characters like here:
kemer } şehir kır toplam }}
use specific terminology . for example , it is often more appropriate for people or things from ethiopia ( a country in africa ) to be described as ethiopian , not carelessly ( with the risk of stereotyping ) as african .
bat avg .
label ਕਾਲਜ
ਅਡੋਲਫ ਹਿਟਲਰ ਨੇ ਦੇਸ਼ ਵਿਚ ਕਮਿਊਨਿਸਟ ਪਾਰਟੀ ਬਣਾਉਣ ਦੀ ਇਜਾਜ਼ਤ ਦੇਣ ਤੋਂ ਨਾਂਹ ਕਰ ਦਿਤੀ।
alt }
if not extra_units then
utc_offset +
ਕਬਜਾ ( )
demographics _title regional
I want to get only like
ਕਾਲਜ
ਅਡੋਲਫ ਹਿਟਲਰ ਨੇ ਦੇਸ਼ ਵਿਚ ਕਮਿਊਨਿਸਟ ਪਾਰਟੀ ਬਣਾਉਣ ਦੀ ਇਜਾਜ਼ਤ ਦੇਣ ਤੋਂ ਨਾਂਹ ਕਰ ਦਿਤੀ।
ਕਬਜਾ
and eventually trim white space lines that is trivial.
The approach I have used was the following
<?php
$in = fopen('php://stdin','rb');
while($line = stream_get_line($in, 64000)) {
foreach(str_split($line) as $char) {
$ordChar = ord($char);
if($ordChar > 127 || $ordChar <= 31) {
echo $char;
}
}
}
used like cat wiki.hi.txt | php -d memory_limit=1024M escape_latin.php > wiki.hi.esc.txt
This approach works ok, the only issue is that performances are getting worst as the file size grows as I can see with a watch du -h filename on the file I'm working on with. I'm surprised because I'm working on a local disk and I'm using stream_get_line to get the lines in streaming.
I have tried the same approach in python, but I get pretty the same performances with file size of ~1GB.
see here for more details.
[UPDATE]
I'm reporting here some results from alternative approaches proposed
Using the regex approach, that seems to produce pretty much the same output file:
A ~50MB file
$ time tr -d "[:alnum:][:punct:]" < wiki.as.txt > wiki.as.test.txt
real 0m2.990s
user 0m2.818s
sys 0m0.088s
A ~100MB file
$ time tr -d "[:alnum:][:punct:]" < wiki.gu.txt > wiki.gu.test.txt
real 0m7.322s
user 0m6.772s
sys 0m0.282s
A ~600MB file
$ time tr -d "[:alnum:][:punct:]" < wiki.ta.txt > wiki.ta.test.txt
real 0m35.973s
user 0m33.498s
sys 0m1.254s
A ~1000MB (1GB) file
$ time tr -d "[:alnum:][:punct:]" < wiki.ja.1.txt > wiki.ja.1.test.txt
real 1m5.409s
user 1m0.669s
sys 0m2.068s
try a regex.
If you're running it from a CLI, try something like
tr -d "[:alnum:][:punct:]" < wiki.hi.txt > wiki.hi.esc.txt
If you prefer to do the same in php -
<?php
$in = fopen('php://stdin','rb');
while($line = stream_get_line($in, 64000)) {
echo preg_replace('/[:alnum:][:punct:]/', '', $line);
}
But please check these to make sure they are doing what you want - esp. the php, since I'm working without a test setup here. It's likely to have syntax issues and/or worse. With luck someone will edit it or offer a better solution, or at least comment and point out whatever I may have done wrong.
Hope it helps.
Calculate duration in HH:MM:SS format using difference in start and end time.
# Time Arithmetic
TIME1="00:30:20"
TIME2="00:30:50"
# Convert the times to seconds from the Epoch
SEC1=`date -j -f '%T' $TIME1 "+%s"`
#echo $SEC1
SEC2=`date -j -f '%T' $TIME2 "+%s"`
#echo $SEC2
# Use expr to do the math, let's say TIME1 was the start and TIME2 was the finish
DIFFSEC=`expr ${SEC2} - ${SEC1}`
#echo $DIFFSEC
echo Start ${TIME1}
echo Finish ${TIME2}
echo Took ${DIFFSEC} seconds.
# And use date to convert the seconds back to something more meaningful
result=`date -r $DIFFSEC "+%T"`
echo Result ${result}
Expected : Result 00:30:30 00:00:30(corrected)
Actual : Result 05:30:30
EDIT Actual TIME1 and TIME2 will be coming as params, and intension here is not to calculate time elapsed. It is a sample code i have used to demonstrate the issue. Strangely when TIME1=05:30:20 and TIME2=05:30:50 then also Result is : 05:30:30
Corrected Subject.
I finally settled with below, thanks anyways.
result=$(printf '%02d:%02d:%02d' $(($DIFFSEC/3600)) $(($DIFFSEC%3600/60)) $(($DIFFSEC%60)))
I have a bash script that creates a file with a timestamp as the name. Once some time passes, it is supposed to pick up that file and do something with it. I want it to pick it up after two hours, but for some reason, it is picking it up after 57 minutes (and 6 seconds). Can anyone point me to an error in my logic or assumptions?
Here are the details:
I have a variable set to 2 hours (7200 seconds):
SERVICE_DURATION=${SERVICE_DURATION:-7200} # seconds
I am setting the filename equal to the Unix timestamp concatenated with nanoseconds:
active_name=`date +%s%N`
echo "${1}" >> ${ACTIVE_DIR}/${active_name}
I then loop forever until the time is right:
while true
do
for fa in ${ACTIVE_DIR}/*
do
if [ $(basename ${fa}) -le $(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000)) ]
then
exec 6< "${fa}"
read old_port <&6
read old_host <&6
read old_config <&6
exec 6<&-
logger -p daemon.info "Recycling port ${old_port}."
start_remote_service "${old_port}"
stop_remote_service "${old_port}" "${old_host}" "${old_config}" "${fa}"
sleep 2
fi
done
sleep 30
done
I can't see what is wrong with this. The filename ($(basename ${fa})) shouldn't be less than the current time minus the specified duration in nanoseconds ($(($(date +%s%N) - ${SERVICE_DURATION} * 1000000000))) until the duration has passed.
In order to keep the script from constantly checking, there is a sleep 30 at the end of the loop, so it could be that the time is somewhere between 56:36 and 57:06.
Any help would be appreciated. Thanks.
I need help using time in the c-shell
I want to know how much time it took to execute a script,so i want to do in the following way
1.set start_time=time
2 script part
3.set end_time=time
4. set diff=end_time-start_time
5.echo "It took $diff seconds"
but i couldn't get the time value using any command.
could any one suggest a command to read the time value in c-shell
I think you want the "date" command, with the format to give you raw seconds:
# start_time = `date +%s`
script part
# end_time = `date +%s`
# diff = $end_time - $start_time
echo "It took $diff seconds"