Seperate Database storage from data storage

This would allow to use a small ssd for the databases and a slow disk for the data itself.

 

IO load would drop significantly and allow more throughput with rotary drives.

  • Guest
  • Dec 26 2019
  • Attach files
  • chris gilligan commented
    25 Apr 09:35pm

    it would also allow the database to be stores on drives in raid with snap shot etc and easy backup. So sick of corrupted databases every now and then. You say don't store the data on raid and that is fine but those database mean everything even if all the files are fine and the db is corrupt you can loose everything

  • Ryan Curtis commented
    16 Apr 11:01pm

    I'd like to add this will also benefit Storj, in that:

    1) The requirement to reserve 10% overhead on the storage drive can be removed.

    a) On an 10TB total node, you're asking for 1TB of storage for the database overhead. This seems a little bit over the top, and would return this storage to be available to the network.

    100x 10TB nodes = 100TB returned to the network.

    Alternative: Ask for a recommended realistic reservation. 50GB?

    2) as mentioned previously, performance for both the Node and the StorJ network would improve by removing IOPS demand to the storage drive. Also resulting in less repair traffic

  • Tomasz Melcer commented
    1 Apr 12:22pm

    My workaround is to modify the source code by hardcoding the paths to databases, and it works.

  • Robin Slot commented
    15 Jan 12:06am

    For now symlinks may work

  • Guest commented
    26 Dec, 2019 10:10pm

    to holld all db files on system disk in configurable location. As storagenode all time writing them, and Data hdd all time has to jump from one point to other and do lot of smal operations, than read and write files. It will defenetly rise performance and responce time. At same time will less kill HDD