I'm working on a system security project with the topic: buffe roverflow attack.
I do have a program and the sourcecode I should gain root permissions with.
The problem I have is caused by a random canary, at the beginning ot the program the random number generator is initialized:
srand(time(NULL) ^ (getpid() << 16));
later on the canary gets set by
canary = rand();
My question: Is it possible to regenerate the canary? I would like to regenerate the salt (time(NULL) returns the time since 1970 in seconds and pid is constant as the program starts) and then get the canary by calling rand().
I'm not familiar with any script language and do not have a lot linux experience, so I hope not to waste time with a solution that would never work.
Thank you in advance! :)
Yes, it's possible. All you need to know is the program's start time and process ID. You can get this information from the command line by entering ps -eo pid,cmd,lstart | grep «program-name». For example, suppose the program you're interested in is called test:
$ ps -eo pid,cmd,lstart | grep test
26798 ./test Thu Oct 26 22:03:52 2017
The number at the start of the output line is the process ID, and the time at which this process started is at the end of the line. You can easily convert this to a Unix timestamp:
$ date --date='Thu Oct 26 22:03:52 2017' '+%s'
1509055432
Last time I checked, PHP was still using the same random number generator. So you can use PHP to calculate the canary:
$ php -r 'srand(1509055432 ^ (26798 << 16)); echo rand()."\n";'
1670313680
(If you don't have PHP, you could easily roll your own C application to do this calculation for you.)
It should be quite obvious that srand(time(NULL)) is of no use whatsoever if you want to generate values that are hard to guess. A better approach would be to fetch bytes from \dev\urandom instead.
Let's say we had a loop that we want to have run as quickly as possible. Let's say something was being done to a list of hosts inside that loop; just for the sake of argument, let's say it was a redis query. Let's say that the list of hosts may change occasionally due to hosts being added/removed from a pool (not load balanced); however, the list is predictable (e.g., they all start with “foo” and end with 2 digits. So we want to run this occasionally; say, once every 15 minutes:
listOfHosts=$(dig +noall +ans foo{00..99}.domain | while read -r n rest; do printf '%s\n' ${n%.}; done)
to get the list of hosts. Let's say our loop looked something like this:
while :; do
for i in $listOfHosts; do
redis-cli -h $i llen something
done
(( ( $(date +%s) % 60 * 15) == 0 )) && callFunctionThatSetslistOfHosts
done
(now obviously there's some things missing, like testing to see if we've already run callFunctionThatSetslistOfHosts in the current minute and only running it once, and doing something with the redis output, and maybe the list of hosts should be an array, but basically this is it.)
How can we run callFunctionThatSetslistOfHosts asynchronously so that it doesn't slow down the loop. I.e., have it running in the background setting listOfHosts occasionally (e.g. once every 15 minutes), so that the next time the inner loop is run it gets a potentially different set of hosts to run the redis query on?
My major problem seems to be that in order to set listOfHosts in a loop, that loop has to be a subshell, and listOfHosts is local to that subshell, and setting it doesn't affect the global listOfHosts.
I may resort to pipes, but will have to poll the reader before generating a new list — not that that's terribly bad if I poll slowly, but I thought I'd present this as a problem.
Thanks.
How can I run the simulation with different configurations? I am using omnet++ version 4.6.
My omnetpp.ini file looks as below :
[General]
[Config Dcn2]
network = Dcn2
# leaf switch
#**.down_port = 2
**.up_port = 16 #12 # 4
# spine switch
**.port = 28 # 20 #2048
# crossconnect
**.cross_down_port = 28 # 20 #2048
**.cross_up_port = 28 # 20 #2048
# to set destination of packet
**.number_leaf_switch = 28 # 20 #2048
# link speed
#**.switch_switch_link_speed = 40 Mbps
**.interArrivalTime = ${exponential(.0001),exponential(0.0002),exponential(0.0003)}
**.batch_length = 10
**.buffer_length = 10
sim-time-limit = 1000s
I want to run the code with different values of interArrivalTime. But I can neither run with different configs (one after another), nor can I run individual runs in parallel on separate cores.
I have tried with cmdev option in run configurations but the different runs doesn't show up apart from the 1st one. When I try mentioning the number of processes to be more than one then also only the first run gets simulated. I really cannot find out the reason.
Config Examinataion
In your case you can perform config examination. OMNeT++ offers different options for that. They are explained under the Parameter Studies section of the OMNeT++ manual.
So you can try one of the following options to examine your configs and thus config file:
./run –a - will show all the configurations in the omnet.ini
./run -x <config_name> - will give more info about a specific config
./run -x <config_name> -g - see all the combinations of configs
First you will have to navigate to your example folder, and there execute one of the aforementioned commands.
I executed: ./run -x Dcn2 -g and got the following resuls
OMNeT++ Discrete Event Simulation (C) 1992-2014 Andras Varga, OpenSim Ltd.
Version: 4.6, build: 141202-f785492, edition: Academic Public License -- NOT FOR COMMERCIAL USE
See the license for distribution terms and warranty disclaimer
Setting up Tkenv...
Config: Dcn2
Number of runs: 3
Run 0: $0=exponential(.0001), $repetition=0
Run 1: $0=exponential(0.0002), $repetition=0
Run 2: $0=exponential(0.0003), $repetition=0
End.
This confirms indeed that you have 3 different runs for the simulation parameter you are trying to modify. However, variable name you are using for the interArrivalTime parameter is assigned to $0 by default because you have not specified it.
If you change the following line in your config:
**.interArrivalTime = ${exponential(.0001),exponential(0.0002),exponential(0.0003)}
to
**.interArrivalTime = ${interArrivalTime = exponential(0.0001),exponential(0.0002),exponential(0.0003)}
you will get a more descriptive output for ./run -x Dcn2 -g
Running different runs of a config:
Next step for you would be to run the different runs for your config. You can do that by navigating to your example directory and execute:
./run -c <config-name> -r <run-number> -u Cmdenv
Note that the <config-name> would be Dcn2 for you, and the -r specifies which of the runs given above you would like to execute.
In other words you can open three terminal windows and navigate to your example directory and do:
./run -c Dcn2 -r 0 -u Cmdenv - for interArrivalTime = exponential(0.0001)
./run -c Dcn2 -r 1 -u Cmdenv - for interArrivalTime = exponential(0.0002)
./run -c Dcn2 -r 2 -u Cmdenv - for interArrivalTime = exponential(0.0003)
Distinguishing Different run results
To be able to distinguish between the output result files of the different runs for your given config you can modify the default name of the output file.
The "how-to" is given in the 12.2.3 Result File Names section of the OMNeT++ manual.
output-vector-file = "${resultdir}/${configname}-${runnumber}.vec"
output-scalar-file = "${resultdir}/${configname}-${runnumber}.sca"
As you can see by default your output files will be distinguished by the ${runnumber} variable. You can further improve it by adding the interArrivalTime to the output file name.
Example:
output-scalar-file = "${resultdir}/${configname}-${runnumber}-IAtime=${interArrivalTime}.sca/vec"
I have not tested the final approach. So you might get some error along the path.
I am trying to classify binary data. In the data file, class [0,1] is converted to [-1,1]. Data has 21 features. All features are categorical. I am using neural network for training. The training command is:
vw -d train.vw --cache_file data --passes 5 -q sd -q ad -q do -q fd --binary -f model --nn 22
I create raw prediction file as:
vw -d test.vw -t -i neuralmodel -r raw.txt
And normal prediction file as:
vw -d test.vw -t -i neuralmodel -p out.txt
First five lines of raw file are:
0:-0.861075,-0.696812 1:-0.841357,-0.686527 2:0.796014,0.661809 3:1.06953,0.789289 4:-1.23823,-0.844951 5:0.886767,0.709793 6:2.02206,0.965555 7:-2.40753,-0.983917 8:-1.09056,-0.797075 9:1.22141,0.84007 10:2.69466,0.990912 11:2.64134,0.989894 12:-2.33309,-0.981359 13:-1.61462,-0.923839 14:1.54888,0.913601 15:3.26275,0.995055 16:2.17991,0.974762 17:0.750114,0.635229 18:2.91698,0.994164 19:1.15909,0.820746 20:-0.485593,-0.450708 21:2.00432,0.964333 -0.496912
0:-1.36519,-0.877588 1:-2.83699,-0.993155 2:-0.257558,-0.251996 3:-2.12969,-0.97213 4:-2.29878,-0.980048 5:2.70791,0.991148 6:1.31337,0.865131 7:-2.00127,-0.964116 8:-2.14167,-0.972782 9:2.50633,0.986782 10:-1.09253,-0.797788 11:2.29477,0.97989 12:-1.67385,-0.932057 13:-0.740598,-0.629493 14:0.829695,0.680313 15:3.31954,0.995055 16:3.44069,0.995055 17:2.48612,0.986241 18:1.32241,0.867388 19:1.97189,0.961987 20:1.19584,0.832381 21:1.65151,0.929067 -0.588528
0:0.908454,0.72039 1:-2.48134,-0.986108 2:-0.557337,-0.505996 3:-2.15072,-0.973263 4:-1.77706,-0.944375 5:0.202272,0.199557 6:2.37479,0.982839 7:-1.97478,-0.962201 8:-1.78124,-0.944825 9:1.94016,0.959547 10:-1.67845,-0.932657 11:2.54895,0.987855 12:-1.60502,-0.92242 13:-2.32369,-0.981008 14:1.59895,0.921511 15:2.02658,0.96586 16:2.55443,0.987987 17:3.47049,0.995055 18:1.92482,0.958313 19:1.47773,0.901044 20:-3.60913,-0.995055 21:3.56413,0.995055 -0.809399
0:-2.11677,-0.971411 1:-1.32759,-0.868656 2:2.59003,0.988807 3:-0.198721,-0.196146 4:-2.51631,-0.987041 5:0.258549,0.252956 6:1.60134,0.921871 7:-2.28731,-0.97959 8:-2.89953,-0.993958 9:-0.0972349,-0.0969177 10:3.1409,0.995055 11:1.62083,0.924746 12:-2.30097,-0.980134 13:-2.05674,-0.967824 14:1.6744,0.932135 15:1.85612,0.952319 16:2.7231,0.991412 17:1.97199,0.961995 18:3.47125,0.995055 19:0.603527,0.539567 20:1.25539,0.84979 21:2.15267,0.973368 -0.494474
0:-2.21583,-0.97649 1:-2.16823,-0.974171 2:2.00711,0.964528 3:-1.84079,-0.95087 4:-1.27159,-0.854227 5:-0.0841799,-0.0839635 6:2.24566,0.977836 7:-2.19458,-0.975482 8:-2.42779,-0.98455 9:0.39883,0.378965 10:1.32133,0.86712 11:1.87572,0.95411 12:-2.22585,-0.976951 13:-2.04512,-0.96708 14:1.52652,0.909827 15:1.98228,0.962755 16:2.37265,0.982766 17:1.73726,0.939908 18:2.315,0.980679 19:-0.08135,-0.081154 20:1.39248,0.883717 21:1.5889,0.919981 -0.389856
First five lines of (normal) prediction file are:
-0.496912
-0.588528
-0.809399
-0.494474
-0.389856
I have tallied this (normal) output with raw output. I notice that the (last or) ending float value in each of the five raw lines is the same as above.
I would please like to understand the raw output as also the normal output. That each line holds 22 pairs of values is something to do with 22 neurons? How to interpret the output as [-1,1] and why a sigmoid function is needed to convert either of the above to probabilities. Will be grateful for help.
For binary classification, you should use a suitable loss function (--loss_function=logistic or --loss_function=hinge). The --binary switch just makes sure that the reported loss is the 0/1 loss (but you cannot optimize for 0/1 loss directly, the default loss function is --loss_function=squared).
I recommend trying the --nn as one of the last steps when tuning the VW parameters. Usually, it improves the results only a little bit and the optimal number of units in the hidden layer is quite small (--nn 1, --nn 2 or --nn 3). You can also try adding a direct connections between the input and output layer with --inpass.
Note that --nn uses always tanh as the sigmoid function for the hidden layer and only one hidden layer is possible (it is hardcoded in nn.cc).
If you want to get probabilities (real number from [0,1]), use vw -d test.vw -t -i neuralmodel --link=logistic -p probabilities.txt. If you want the output to a be real number from [-1,1], use --link=glf1.
Without --link and --binary, the --pred output are the internal predictions (in range [-50, 50] when logistic or hinge loss function is used).
As for the --nn --raw question, your guess is correct:
The 22 pairs of numbers correspond to the 22 neurons and the last number is the final (internal) prediction. My guess is that each pair corresponds to the bias and output of each unit on the hidden layer.
I need to examine the output of a certain script 1000s of times on a unix platform and check if any of it has changed from before.
I've been doing this:
(script_stuff) | md5sum
and storing this value. I actually don't really need "md5", JUST a simple hash function which I can compare against a stored value to see if its changed. Its okay if there are an occassional false positive.
Is there anything better than md5sum that works faster and generates a fairly usable hash value? The script itself generates a few lines of text - maybe 10-20 on average to max 100 or so.
I had a look at fast md5sum on millions of strings in bash/ubuntu - that's wonderful, but I can't compile a new program. Need a system utility... :(
Additional "background" details:
I've been asked to monitor the DNS record of a set of 1000 or so domains and immediately call certain other scripts if there has been any change. I intend to do a dig xyz +short statement and hash its output and store that, and then check it against a previously stored value. Any change will trigger the other script, otherwise it just goes on. Right now, we're planning on using cron for a set of these 1000, but can think completely diffeerently for "seriously heavy" usage - ~20,000 or so.
I have no idea what the use of such a system would be, I'm just doing this as a job for someone else...
The cksum utility calculates a non-cryptographic CRC checksum.
How big is the output you're checking? A hundred lines max. I'd just save the entire original file then use cmp to see if it's changed. Given that a hash calculation will have to read every byte anyway, the only way you'll get an advantage from a checksum type calculation is if the cost of doing it is less than reading two files of that size.
And cmp won't give you any false positives or negatives :-)
pax> echo hello >qq1.txt
pax> echo goodbye >qq2.txt
pax> cp qq1.txt qq3.txt
pax> cmp qq1.txt qq2.txt >/dev/null
pax> echo $?
1
pax> cmp qq1.txt qq3.txt >/dev/null
pax> echo $?
0
Based on your question update:
I've been asked to monitor the DNS record of a set of 1000 or so domains and immediately call certain other scripts if there has been any change. I intend to do a dig xyz +short statement and hash its output and store that, and then check it against a previously stored value. Any change will trigger the other script, otherwise it just goes on. Right now, we're planning on using cron for a set of these 1000, but can think completely diffeerently for "seriously heavy" usage - ~20,000 or so.
I'm not sure you need to worry too much about the file I/O. The following script executed dig microsoft.com +short 5000 times first with file I/O then with output to /dev/null (by changing the comments).
#!/bin/bash
rm -rf qqtemp
mkdir qqtemp
((i = 0))
while [[ $i -ne 5000 ]] ; do
#dig microsoft.com +short >qqtemp/microsoft.com.$i
dig microsoft.com +short >/dev/null
((i = i + 1))
done
The elapsed times at 5 runs each are:
File I/O | /dev/null
----------+-----------
3:09 | 1:52
2:54 | 2:33
2:43 | 3:04
2:49 | 2:38
2:33 | 3:08
After removing the outliers and averaging, the results are 2:49 for the file I/O and 2:45 for the /dev/null. The time difference is four seconds for 5000 iterations, only 1/1250th of a second per item.
However, since an iteration over the 5000 takes up to three minutes, that's how long it will take maximum to detect a problem (a minute and a half on average). If that's not acceptable, you need to move away from bash to another tool.
Given that a single dig only takes about 0.012 seconds, you should theoretically do 5000 in sixty seconds assuming your checking tool takes no time at all. You may be better off doing something like this in Perl and using an associative array to store the output from dig.
Perl's semi-compiled nature means that it will probably run substantially faster than a bash script and Perl's fancy stuff will make the job a lot easier. However, you're unlikely to get that 60-second time much lower just because that's how long it takes to run the dig commands.