Trying to Create a new Policy for Multi Disk Operations - clickhouse

Question

I am using clickhouse with just one disk which is specified at config.xml file under <path>
Now I want to extend this disk, so I updated the clickhouse version for enabling multi disk support.
What I want to do now is using the two disks together. I want to read from both of them but write data to second one only.
I have many tables, I thought changing the storage policy of the tables would do the trick but i can't change it.
For example i have a table called default_event which has default policy, after this query:
alter table default_event modify setting storage_policy='newStorage_only';
I got this error : Exception: New storage policy default shall contain volumes of old one
My storage xml is like this:
<?xml version="1.0" encoding="UTF-8"?>
<yandex>
<storage_configuration>
<disks>
<!--
default disk is special, it always
exists even if not explicitly
configured here, but you can't change
it's path here (you should use <path>
on top level config instead)
-->
<default>
<!--
You can reserve some amount of free space
on any disk (including default) by adding
keep_free_space_bytes tag
-->
<keep_free_space_bytes>1024</keep_free_space_bytes>
</default>
<test_disk>
<!--
disk path must end with a slash,
folder should be writable for clickhouse user
-->
<path>/DATA/newStorage/</path>
</test_disk>
<test_disk_2>
<!--
disk path must end with a slash,
folder should be writable for clickhouse user
-->
<path>/DATA/secondStorage/</path>
</test_disk_2>
<test_disk_3>
<!--
disk path must end with a slash,
folder should be writable for clickhouse user
-->
<path>/DATA/thirdStorage/</path>
</test_disk_3>
</disks>
<policies>
<newStorage_only>
<!-- name for new storage policy -->
<volumes>
<newStorage_volume>
<!-- name of volume -->
<!--
we have only one disk in that volume
and we reference here the name of disk
as configured above in <disks> section
-->
<disk>test_disk</disk>
</newStorage_volume>
</volumes>
</newStorage_only>
</policies>
</storage_configuration>
</yandex>
I tried adding default volume to the new policy but i can't start clickhouse with that config.

Answer 1

So, your main problem is that before that you did not explicitly specify the storage policy, but the default disk is written there by default. New policy should include all old disks and volumes with same names.
I gave a configuration based on yours, removing everything unnecessary. And that, I mean that in addition to those listed, you have a drive specified in path with the name default. All disks are listed in the volumes section of the new policy. Writing to new disks will happen thanks to move_factor. The value 0.5 tells us that when 50% of the disk space is reached, we need to write to the next one, and so on.
As soon as the rest of the disks fill evenly, you can lower this value.
PS: you can not use old disks in the new policy, for this you need to execute ALTER TABLE ... MOVE PARTITIONS/PARTS ... to transfer partitions/parts to new disks. Then the table will not be tied to the old disk and it will not be tedious to specify it in the new storage policy. Disks, of course, must be pre-configured in the settings.
<yandex>
<storage_configuration>
<disks>
<test_disk>
<path>/DATA/newStorage/</path>
</test_disk>
<test_disk_2>
<path>/DATA/secondStorage/</path>
</test_disk_2>
<test_disk_3>
<path>/DATA/thirdStorage/</path>
</test_disk_3>
</disks>
<policies>
<!--... old policy ... -->
<new_storage_only> <!-- policy name -->
<volumes>
<default>
<disk>default</disk>
</default>
<new_volume>
<disk>test_disk</disk>
<disk>test_disk_2</disk>
<disk>test_disk_3</disk>
</new_volume>
</volumes>
<move_factor>0.5</move_factor>
</new_storage_only>
</policies>
</storage_configuration>
</yandex>