janusgraph.diskstorage.PermanentBackendException: Permanent exception while executing backend operation IndexQuery - janusgraph

I have trouble in query to janusgraph
I query 50 same request and the result is:
request count 50 (OK=11 KO=39 )
min response time 5391 (OK=5391 KO=5697 )
max response time 5902 (OK=5902 KO=5883 )
mean response time 5740 (OK=5642 KO=5768 )
std deviation 98 (OK=158 KO=42 )
response time 50th percentile 5760 (OK=5580 KO=5761 )
response time 75th percentile 5801 (OK=5773 KO=5801 )
response time 95th percentile 5835 (OK=5870 KO=5828 )
response time 99th percentile 5893 (OK=5896 KO=5864 )
mean requests/sec 5.556 (OK=1.222 KO=4.333 )
---- Response Time Distribution ------------------------------------------------
t < 800 ms 0 ( 0%)
800 ms < t < 1200 ms 0 ( 0%)
t > 1200 ms 11 ( 22%)
failed 39 ( 78%)
---- Errors --------------------------------------------------------------------
> status.find.is(200), but actually found 500 39 (100.0%)
I use gatling and dropwizard to test janusGrpah.
I see this error in dropwizard
ERROR [2018-09-06 07:16:18,159] io.dropwizard.jersey.errors.LoggingExceptionMapper: Error handling a request: f0cb664169b00267
! java.util.concurrent.TimeoutException: null
! at org.apache.http.nio.pool.AbstractNIOConnPool.processPendingRequest(AbstractNIOConnPool.java:364)
! at org.apache.http.nio.pool.AbstractNIOConnPool.processNextPendingRequest(AbstractNIOConnPool.java:344)
! at org.apache.http.nio.pool.AbstractNIOConnPool.release(AbstractNIOConnPool.java:318)
! at org.apache.http.impl.nio.conn.PoolingNHttpClientConnectionManager.releaseConnection(PoolingNHttpClientConnectionManager.java:303)
! at org.apache.http.impl.nio.client.AbstractClientExchangeHandler.releaseConnection(AbstractClientExchangeHandler.java:239)
! at org.apache.http.impl.nio.client.MainClientExec.responseCompleted(MainClientExec.java:387)
! at org.apache.http.impl.nio.client.DefaultClientExchangeHandlerImpl.responseCompleted(DefaultClientExchangeHandlerImpl.java:168)
! at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.processResponse(HttpAsyncRequestExecutor.java:436)
! at org.apache.http.nio.protocol.HttpAsyncRequestExecutor.inputReady(HttpAsyncRequestExecutor.java:326)
! at org.apache.http.impl.nio.DefaultNHttpClientConnection.consumeInput(DefaultNHttpClientConnection.java:265)
! at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:81)
! at org.apache.http.impl.nio.client.InternalIODispatch.onInputReady(InternalIODispatch.java:39)
! at org.apache.http.impl.nio.reactor.AbstractIODispatch.inputReady(AbstractIODispatch.java:114)
! at org.apache.http.impl.nio.reactor.BaseIOReactor.readable(BaseIOReactor.java:162)
! at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvent(AbstractIOReactor.java:337)
! at org.apache.http.impl.nio.reactor.AbstractIOReactor.processEvents(AbstractIOReactor.java:315)
! at org.apache.http.impl.nio.reactor.AbstractIOReactor.execute(AbstractIOReactor.java:276)
! at org.apache.http.impl.nio.reactor.BaseIOReactor.execute(BaseIOReactor.java:104)
! at org.apache.http.impl.nio.reactor.AbstractMultiworkerIOReactor$Worker.run(AbstractMultiworkerIOReactor.java:588)
! ... 1 common frames omitted
! Causing: java.lang.RuntimeException: error while performing request
! at org.elasticsearch.client.RestClient$SyncResponseListener.get(RestClient.java:682)
! at org.elasticsearch.client.RestClient.performRequest(RestClient.java:220)
! at org.elasticsearch.client.RestClient.performRequest(RestClient.java:192)
! at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.performRequest(RestElasticSearchClient.java:334)
! at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.search(RestElasticSearchClient.java:290)
! at org.janusgraph.diskstorage.es.rest.RestElasticSearchClient.search(RestElasticSearchClient.java:60)
! at org.janusgraph.diskstorage.es.ElasticSearchIndex.query(ElasticSearchIndex.java:1105)
! at org.janusgraph.diskstorage.indexing.IndexTransaction.queryStream(IndexTransaction.java:108)
! at org.janusgraph.diskstorage.BackendTransaction$6.call(BackendTransaction.java:417)
! at org.janusgraph.diskstorage.BackendTransaction$6.call(BackendTransaction.java:414)
! at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:68)
! ... 103 common frames omitted
! Causing: org.janusgraph.diskstorage.PermanentBackendException: Permanent exception while executing backend operation IndexQuery
! at org.janusgraph.diskstorage.util.BackendOperation.executeDirect(BackendOperation.java:81)
! at org.janusgraph.diskstorage.util.BackendOperation.execute(BackendOperation.java:54)
! at org.janusgraph.diskstorage.BackendTransaction.executeRead(BackendTransaction.java:470)
! at org.janusgraph.diskstorage.BackendTransaction.indexQuery(BackendTransaction.java:414)
! at org.janusgraph.graphdb.database.IndexSerializer.query(IndexSerializer.java:530)
! at org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:66)
! ... 98 common frames omitted
! Causing: org.janusgraph.core.JanusGraphException: Could not call index
! at org.janusgraph.graphdb.util.SubqueryIterator.<init>(SubqueryIterator.java:68)
! at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1295)
! at org.janusgraph.graphdb.transaction.StandardJanusGraphTx$3.execute(StandardJanusGraphTx.java:1190)
! at org.janusgraph.graphdb.query.QueryProcessor$LimitAdjustingIterator.getNewIterator(QueryProcessor.java:194)
! at org.janusgraph.graphdb.query.LimitAdjustingIterator.hasNext(LimitAdjustingIterator.java:68)
! at org.janusgraph.graphdb.query.ResultSetIterator.nextInternal(ResultSetIterator.java:54)
! at org.janusgraph.graphdb.query.ResultSetIterator.<init>(ResultSetIterator.java:44)
! at org.janusgraph.graphdb.query.QueryProcessor.iterator(QueryProcessor.java:66)
! at com.google.common.collect.Iterables$4.iterator(Iterables.java:578)
! at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.executeGraphCentryQuery(JanusGraphStep.java:156)
! at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.lambda$null$1(JanusGraphStep.java:95)
! at java.lang.Iterable.forEach(Iterable.java:75)
! at org.janusgraph.graphdb.tinkerpop.optimize.JanusGraphStep.lambda$new$2(JanusGraphStep.java:95)
! at org.apache.tinkerpop.gremlin.process.traversal.step.map.GraphStep.processNextStart(GraphStep.java:142)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
! at org.apache.tinkerpop.gremlin.process.traversal.step.filter.FilterStep.processNextStart(FilterStep.java:37)
! at org.apache.tinkerpop.gremlin.process.traversal.step.filter.DedupGlobalStep.processNextStart(DedupGlobalStep.java:105)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
! at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.hasNext(AbstractStep.java:143)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.ExpandableStepIterator.next(ExpandableStepIterator.java:50)
! at org.apache.tinkerpop.gremlin.process.traversal.step.map.MapStep.processNextStart(MapStep.java:36)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:128)
! at org.apache.tinkerpop.gremlin.process.traversal.step.util.AbstractStep.next(AbstractStep.java:38)
! at org.apache.tinkerpop.gremlin.process.traversal.Traversal.fill(Traversal.java:179)
! at org.apache.tinkerpop.gremlin.process.traversal.Traversal.toList(Traversal.java:117)
! at ir.mohaymen.utilities.analytics.libintegration._GremlinKt.mapTo(_Gremlin.kt:7)
! at ir.mohaymen.analytics.janusgraphbenchmark.dao.UserDao.findByFullPMixed(UserDao.kt:176)
! at ir.mohaymen.analytics.janusgraphbenchmark.resources.UserResource.findByPMixed(UserResource.java:187)
! at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
! at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
! at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
! at java.lang.reflect.Method.invoke(Method.java:498)
! at org.glassfish.jersey.server.model.internal.ResourceMethodInvocationHandlerFactory$1.invoke(ResourceMethodInvocationHandlerFactory.java:81)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher$1.run(AbstractJavaResourceMethodDispatcher.java:144)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.invoke(AbstractJavaResourceMethodDispatcher.java:161)
! at org.glassfish.jersey.server.model.internal.JavaResourceMethodDispatcherProvider$TypeOutInvoker.doDispatch(JavaResourceMethodDispatcherProvider.java:205)
! at org.glassfish.jersey.server.model.internal.AbstractJavaResourceMethodDispatcher.dispatch(AbstractJavaResourceMethodDispatcher.java:99)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.invoke(ResourceMethodInvoker.java:389)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:347)
! at org.glassfish.jersey.server.model.ResourceMethodInvoker.apply(ResourceMethodInvoker.java:102)
! at org.glassfish.jersey.server.ServerRuntime$2.run(ServerRuntime.java:326)
! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:271)
! at org.glassfish.jersey.internal.Errors$1.call(Errors.java:267)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:315)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:297)
! at org.glassfish.jersey.internal.Errors.process(Errors.java:267)
! at org.glassfish.jersey.process.internal.RequestScope.runInScope(RequestScope.java:317)
! at org.glassfish.jersey.server.ServerRuntime.process(ServerRuntime.java:305)
! at org.glassfish.jersey.server.ApplicationHandler.handle(ApplicationHandler.java:1154)
! at org.glassfish.jersey.servlet.WebComponent.serviceImpl(WebComponent.java:473)
! at org.glassfish.jersey.servlet.WebComponent.service(WebComponent.java:427)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:388)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:341)
! at org.glassfish.jersey.servlet.ServletContainer.service(ServletContainer.java:228)
! at io.dropwizard.jetty.NonblockingServletHolder.handle(NonblockingServletHolder.java:49)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1655)
! at io.dropwizard.servlets.ThreadNameFilter.doFilter(ThreadNameFilter.java:34)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at io.dropwizard.jersey.filter.AllowedMethodsFilter.handle(AllowedMethodsFilter.java:45)
! at io.dropwizard.jersey.filter.AllowedMethodsFilter.doFilter(AllowedMethodsFilter.java:39)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:311)
! at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:265)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:89)
! at com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
! at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
! at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
! at com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
! at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
! at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1642)
! at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:255)
! at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1317)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:203)
! at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
! at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:201)
! at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1219)
! at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:144)
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
! at com.codahale.metrics.jetty9.InstrumentedHandler.handle(InstrumentedHandler.java:241)
! at io.dropwizard.jetty.RoutingHandler.handle(RoutingHandler.java:52)
! at org.eclipse.jetty.server.handler.gzip.GzipHandler.handle(GzipHandler.java:674)
! at io.dropwizard.jetty.BiDiGzipHandler.handle(BiDiGzipHandler.java:69)
! at org.eclipse.jetty.server.handler.RequestLogHandler.handle(RequestLogHandler.java:56)
! at org.eclipse.jetty.server.handler.StatisticsHandler.handle(StatisticsHandler.java:169)
! at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
! at org.eclipse.jetty.server.Server.handle(Server.java:531)
! at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:352)
! at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:260)
! at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:281)
! at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
! at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:118)
! at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:762)
! at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:680)
! at java.lang.Thread.run(Thread.java:748)
I search this error in web but I can't find any helpful post.
can anyone help me?

Related

Date: option requires an argument

Can someone tell me what I'm doing wrong, please?
I've got this block of code
if [ -n "${MFA_Exp}" ]; then
exp_sec="$(expr '(' $(date -d "${MFA_Exp}" +%s) - $(date +%s) ')' )";
if [ "${exp_sec}" -gt 0 ]; then
output+=", MFA TTL: $(date -u -d #"${exp_sec}" +"%Hh %Mm %Ss")";
else
output+=", MFA DEAD!";
fi;
that should output the expiration time of my MFA token, but I get this error
date: option requires an argument -- d
usage: date [-jnRu] [-d dst] [-r seconds] [-t west] [-v[+|-]val[ymwdHMS]] ...
[-f fmt date | [[[mm]dd]HH]MM[[cc]yy][.ss]] [+format]
I'm on a Macbook and I suspect it's something to do with the date format.
I'm just not sure what it is.
The default date format for BSD date is [[[mm]dd]HH]MM[[cc]yy][.ss]]. If MFS_Exp is in that format, you can use
exp_sec=$(( $(date -j "$MFS_Exp" +%s) - $(date +%s) ))
If not, you need to specify the input format using the -f option. For example, if your string is like 2020-12-18 12:34:56, then use date -j -f '%Y-%m-%d %H:%M:%S' "$MFS_Exp" +%s.
For the second call, I wouldn't recommend using date at all, as you are working with a duration, not a timestamp.
hours=$(( exp_sec / 3600 ))
rem=$(( exp_sec % 3600 ))
minutes=$(( rem / 60 ))
sec=$(( rem % 60 ))
output+=", MFA TTL: ${hours}h ${minutes}m ${sec}s"

Shell Scripting to extract date

Need a logic in shell scripting where I give the start and end date,
say
startDate=20140101 &
endDate=20160130
I should be able to extract startDate_new=20140122 and endDate_new=20140221 in first loop.
In next loop will need startDate_new=20140222 and endDate_new=20140321 so on, till startDate_new=20151221 & endDate_new=20160122 and exit out. Can you provide any shell scripting logic that can be used to achieve this.
test.sh has,
startdate=2013-03-01
enddate=2013-04-30
curr="$startdate"
while true; do
echo "$curr"
[ "$curr" \< "$enddate" ] || break
curr=$( date +%Y-%m-%d --date "$curr +1 day" )
done
the above code prints,
2013-03-01
2013-03-02
2013-03-03
2013-03-04
2013-03-05
2013-03-06
2013-03-07
2013-03-08
.
.
.
.
2013-04-30
Can you help me how to extract the range I mentioned above ?
Came up with a new logic,
startdate=2013-03-21
enddate=2014-05-30
curr="$startdate"
while true; do
[ "$curr" \< "$enddate" ] || { echo "$curr"; break; }
echo "$curr"
curr=$( date +%Y-%m-%d --date "$curr +1 month" )
end=$( date +%Y-%m-%d --date "$curr +1 month +1day" )
done
the above logic gives
2013-03-21
2013-04-21
2013-05-21
2013-06-21
2013-07-21
2013-08-21
2013-09-21
2013-10-21
2013-11-21
2013-12-21
2014-01-21
2014-02-21
2014-03-21
2014-04-21
2014-05-21
2014-06-21
Can you help with how to retrieve end date as 22nd of each month ?
Make a function:
# Usage: drange startdate endate [ time_increment ]
drange ()
{
curr="$( date +%Y-%m-%d --date $1)"
enddate="$( date +%Y-%m-%d --date $2)"
shift 2
inc="${*:-+1 day}"
until [ "$curr" \> "$enddate" ]; do
echo $curr
curr=$( date +%Y-%m-%d --date "$curr $inc" )
done
}
Run it like so:
drange 20140222 20140321
drange 20151221 20160122 +1 month
s=20151221 e=20160122 i="+1 month"
drange $((s + 1)) $e $i

bash if invalid arithmetic operator

value1=ns1.abc.nl.
value2=ns2.abc.nl.
value3=ns3.abc.nl.
for domains in $(cat /home/carlito/Desktop/output.txt)
do
for nameserver in $(dig +short NS $domains)
do
if [[ ( $nameserver -eq $value1 ) || ( $nameserver -eq $value2 ) || ( $nameserve$
then
echo "$domains Namserver is local" >>/home/carlito/Desktop/resultaat.txt
break
else
echo "$domains namserver IS NOT local" >>/home/carlito/Desktop/resultaat.txt
fi
done
done
So i'm having trouble with the IF line of the script the error message=" nszero1.axc.nl.: syntax error: invalid arithmetic operator (error token is ".axc.nl."".
The purpose of this script= 'get output.txt (list of domains)' 'then dig +short NS $domains' and then check if its on my preferred name server $Value1,2,3 if true=good if false=bad
i already tried something like:
value="ns1.abc.nl./ns2.abc.nl/ns3.abc.nl."
for domains in $(cat /home/carlito/Desktop/output.txt)
do
if dig +short NS $domains == $value
then
echo "$domains is LOCAL"
else
echo"$domains Is NOT LOCAL"
fi
done
But what this does is if domain.nl=ns1.abc.nl./ns2.abc.nl it still echo's:Is NOT LOCAL. But i want the script to Echo=true if it has at least 1 of the values.
can some one point me in the right direction which function i should use and what am i doing wrong with the if line ?
thanks in advance
You are comparing a string with a numeric value. The syntax in the below if condition is to compare two numeric value and not for string comparison. Also your if condition is not properly ended with ]]. I thing you are comparing a string with a numeric value. Please see below correct if conditions:-
First a correct if condition if both the right and left hand values are numeric :-
if [[ ( $nameserver -eq $value1 ) || ( $nameserver -eq $value2 ) ]]
For correct string comparison:-
if [[ ( "$nameserver" == "$value1" ) || ( "$nameserver" == "$value2" ) ]]
Hope this will help you.

Error in while loop for checking the file existence only for a fixed time

I need to write a while loop to check for a file existence.
My requirement is: check for the file only for 5 minutes. If file come in that path within 5 minutes exit the loop and continue rest of the script otherwise exit from the script after 5 minutes with an error 'file not found'. I wrote the code like this :
SOURCEFILE=/path/*file.csv
StartTime=$(date +'%s')
TimeSpan=300
EndTime=$((StartTime + TimeSpan))
while [[ ! -f ${SOURCEFILE} && $(date +'%s') < ${EndTime} ]]
do
echo "inside loop"
sleep 25
done
echo "outside loop"
But with this while loop, even if the file is present in the mentioned path, it is going inside loop and will exit only after 300 seconds. I am beginner in shell scripting and I am not able to understand the issue. I am using ksh.
I could also tell you that it works find with while [ ! -f {SOURCEFILE} ] only. But whenever I add any && condition to while loop , then the -f is not working properly.
The SOURCEFILE=/path/*file.csv is wrong in your case. It can't be evaluated right with the -f flag.
An easy solution would be to use find or ls and count the result:
find /path/ -name "*file.csv" -type f
# then count the result...
Now I think there is a logic issue with the operators precedence. To force evaluation of the ! for the -f only, use parenthesis. Here is what works for me, and you must adapt it a little to match the * before file.csv:
while [[ ( ! -f file.csv ) && $(date +'%s') < ${EndTime} ]]
do
echo "inside loop"
sleep 25
...
There are some more explanation on this answer. The "and" operator precedes the "not", that's why you had the issue.
Your primary issue is getting the asterisk (*) to expand at the 'right time'.
It doesn't help that the [ ] and [[ ]] constructs behave differently, especially when it comes to if/when to expand that asterisk. [You can peruse the google search for 'ksh single bracket vs double bracket' for more details.]
Try running the following to see the differences between single/double brackets and unquoted/single-quoted/double-quoted variable:
SOURCEFILE=/path/*file.csv
set -x
[ ! -f ${SOURCEFILE} ] && echo 'missing'
[ ! -f '${SOURCEFILE}' ] && echo 'missing'
[ ! -f "${SOURCEFILE}" ] && echo 'missing'
[[ ! -f ${SOURCEFILE} ]] && echo 'missing'
[[ ! -f '${SOURCEFILE}' ]] && echo 'missing'
[[ ! -f "${SOURCEFILE}" ]] && echo 'missing'
NOTE: Notice which tests expand the asterisk and which are looking for a (literal) asterisk in the name.
NOTE: Try adding a space to your file name (eg, *file XX.csv) and run the above tests ... tricky, tricky, tricky ...
For this particular case ... asterisk/wildcard in file name, no spaces, ksh ... you'll likely be ok with something like:
while [[ ! -f ${SOURCEFILE} ]] && [[ $(date +'%s') < ${EndTime} ]]

Round up integer in shell script

I'm ALL NEW in shell script , hence have a question about how to round up integer.
Here is a line of the code from a script by keke(smstools3 developer)
balance=$(substr "$result" "$balance_prefix" "$balance_suffix")
And my balance is 111.12 , and I wish to round it up.
I tried
balance1=$(substr "$result" "$balance_prefix" "$balance_suffix")
balance=$("%0.f\n" "$balance1")
or
balance1=$(substr "$result" "$balance_prefix" "$balance_suffix")
balance=$(ceil($balance1))
Both refer from some answers after google it , I not even know if the syntax is correct.And of course both example return blank.
Any hints or advice?Thank you.
Edit:
# Check that required words exists:
if [[ "$result" == *${balance_prefix}* ]] && \
[[ "$result" == *${balance_suffix}* ]]
then
# Get the balance and check it:
balance=$(substr "$result" "$balance_prefix" "$balance_suffix")
balance_low=0
if [ $(expr "$balance" + 1 2> /dev/null) ]; then
[ $balance -le $alert_balance ] && balance_low=1
else
echo "Error while parsing an integer: $balance"
fi
else
echo "Error while parsing the answer (balance): $result"
fi
source : http://smstools3.kekekasvi.com/topic.php?id=320
balance=`python -c "from math import ceil; print(ceil($balance1))"`
or
balance=`perl -MPOSIX -e "print ceil($balance1)"`
Using only a minimum of standard tools (perl and python are quite common though):
balance=$(echo "x=${balance1}; scale=0; xx=x/1; if(x>xx) xx+=1; print xx"|bc -l)

Resources