Ahighly reliable storage system achieved by low spatial, computation & bandwidth overhead based on a new method to construct LDPC codes for adding redundancy. It offers several innovations andsignificantadvantages over existing solutions as follow : Increased reliability can be achieved with the same computational complexity and consumption of system resources. This is in comparison with Reed Solomon codes where the complexity is increased linearly with the reliability Efficient block placement is enabled to reduce the consumption of the network resources Efficient appending is enabled. For certain configurations, the encoding efficiencycan be improved with increased reliabilty of chunks required to he re-encoded. or even more depending on different setting of the parameters asa small portion of old encoded chunks have to be re-encoded These benefits have been proven theoretically and tested in the Hadoop platform
Redundancy has to be added into any cloud storage to ensure certain level of reliability. The invention discloses a novel way to construct erasure codes to add redundancy, to distribute the data to the distributed storage or storage arrays, and to add redundancy efficiently for data with increasing size. Its advantages and potential benefits are as follow : High reliability is achieved with high efficiency and low consumption of system resources Increased reliability can be achieved with the same computational complexity and consumption of system resources. This is superior to traditional technology where the complexity increases linearly with the reliability Efficient appending is supported. For certain configurations, the encoding efficiency can be improved with an increase in reliability by 67% in terms of the percentages of chunks required to be re-encoded. The increment can be greater, depending on the specific parameters setting These benefits are proved on the Hadoop platform using our in-house infrastructure and Amazon Web Services.
For any companies who are concerned with the reliability and security of the distributed storage, our patented codes provides highly efficient way to add redundancy and data confidentiality unlike the traditional replication-based or RAID-based methods. Potential applications are as follow : Middleware for storage service providers or data centre operators Storage gateway for enterprise cloud storage users Secure and reliable cloud storage services leverage on public cloud Efficient Archiving