Multi-HD setup for a single node

  • Guest
  • Feb 2 2019
  • Will not implement
  • Attach files
  • Chris Stevens commented
    February 18, 2019 20:33
    Not sure why this would be an enhancement/feature of Storj. I understand we're dealing with Storage, but Storj itself is a software layer that makes use of available hardware. This hardware *should* be controlled by the OS or even better, a hardware raid. Look into RAIDs. Software Raid on Windows, you can use Storage Spaces/Storage Pools (Windows 10, Server 2012+), or StableBit DrivePool.
  • adam lentz commented
    February 24, 2019 04:30
    Better management of multiple drives available on a single node. That's the idea to improve storage of unused drives. If storj accessed multiple drives on a single node, would make better management. If a single drive fails the data could be recovered and all other drive data would still be valid. Storj is already a form of raid with the nodes. The point is node operators may have multiple drives of different sizes and want to share this unused space, thus the definition of storj and decentralization. If a single drive in a multi drive node fails, the drive could be replaced and recovered with minimal impact on reputation instead of losing an entire node to a single disk failure. A Node on RAID0 will fail completely if one drive fails. On other RAID formats less space is available for redundancy if a drive fails. If this is too much then it's back to one node per drive. Node operators want to share as much free unused space as possible.
  • H3 commented
    March 08, 2019 09:13
    Workaround: mhddfs (file system for unifying several mount points into one).
  • Nicolas Hugo commented
    March 16, 2019 19:50
    A Multi-HD setup for a single node could "transfer" the complexity to setup the multiple hardisks (Raid0, Raid5, linear, logical volumes, ...) to the storage node software. It could also avoid storage node disqualification if one drive fails (in raid0 by exemple).
  • Aleksei Leonov commented
    April 03, 2019 18:58
    If you lose data on any of this "RAID" it could lead the disqualification of the node and losing reputation. If you will use a normal RAID with parity, the one failed drive will not leads of losing data, reputation and escrow. So, I think it's better to have the RAID with parity outside of the node's software
  • Chris VdG commented
    April 05, 2019 13:28
    Using erasure coding (Storj) on top of RAID (node) with parity is like throwing salt in the Sea... IMHO if someone is really concerned about keeping their reputation high (e.g.: a more professional farmer) then sure, go for it. But using even more disk space for RAID parity on top of erasure coding parity does not seem like efficient use of disk space to me. I'm active in cloud tech and we do not use erasure coding on top of RAID, only erasure coding for our storage back-end. This would also allow as suggested here for less reputation loss to have one disk fail on a node instead losing a whole node.
  • Chris VdG commented
    April 05, 2019 14:39
    Maybe if the NAT traversal implementation allows it, we could have several storage node containers on a host each serving a different disk?
  • Xiao Lu commented
    April 10, 2019 19:57
    It is a good idea,hope it could be achieve soon.
  • Admin
    Brandon Iglesias commented
    September 09, 2019 17:54

    Hey everyone Brandon the Product Manager @storj here. Thanks for submitting this idea, Chris VdG has a point about the redundancy between raid and erasure coding. Our recommendation is to run one node per HD. This will actually spread your overall risk of a HD failing and losing the withholdings for that specific node ID. If you have 4 2 TB HDs then the best thing to do is run 1 node per HD and if one of those fails then at most you would only lose 25 percent of your overall held amount amongst all 4 nodes. Instead of using raid to backup one of your nodes its may be in your best interest to just allocate that extra raid space to the network and profit off of it instead.