How Can I Create Multiple Files of Random Data? - macos

Question

I've been using dd if=/dev/random of=<file_name> bs=XX count=XX to create single files of random data. But I wondered how I could create multiple files at once. Say 5, 10, or 80,000. I need to do some bandwidth testing and need unique data. The above code works great for 1 or 2 files. My programming (Python) and terminal skill are still very minimal.
Any help would be awesome.

Answer 1

man split.
Split splits a file or stdin to multiple files based on file size or number of lines. It doesn't happen "at once" or in parallel though.
dd if=/dev/random bs=1 count=10 | split -b 2 produces 5 files xaa,xab..xae each consisting of 2 bytes.

Answer 2

I would use GNU Parallel for this. The command line syntax is very intuitive and concise and it does all your work in parallel, i.e. FAST, and uses all those lovely CPU cores in your Mac.
So, to create 8 files in parallel, each of 100MB, you would type this in the Terminal:
$ parallel dd if=/dev/random of=random-{} bs=1000000 count=100 ::: {0..7}
and you will end up with these 8 files just 60 seconds later:
$ ls -l random-*
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-0
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-1
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-2
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-3
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-4
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-5
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-6
-rw-r--r--# 1 mark staff 100000000 19 Dec 11:52 random-7
Or, if you wanted one file of 1kB, two files of 64kB, one file of 32kB and one of 128kB, you would do this:
$ parallel dd if=/dev/random of=random-{%} bs=1024 count={1} ::: 1 64 64 32 128
which will give you this:
-rw-r--r-- 1 mark staff 131072 19 Dec 12:10 random-5
-rw-r--r-- 1 mark staff 32768 19 Dec 12:10 random-4
-rw-r--r-- 1 mark staff 65536 19 Dec 12:10 random-3
-rw-r--r-- 1 mark staff 65536 19 Dec 12:10 random-2
-rw-r--r-- 1 mark staff 1024 19 Dec 12:10 random-1
The easiest way, to my mind, of installing parallel on OSX is to get homebrew from the homebrew website, then all you do is:
brew install parallel