Algorithm for data integrity assurance in distributed storages with compensation for the results of destabilizing impacts and verification of recovery results

Network and telecommunication security
Authors:
Abstract:

An algorithm for ensuring data integrity in distributed storage systems is considered. The algorithm is aimed at monitoring the integrity of stored data and compensating for the effects of random and deliberate destabilizing factors, with verification of the reliability of recovery results. It is intended to support write and read operations for data arrays under conditions of storage node failures, sub-block corruption, network degradation, and attempts at coordinated substitution of integrity reference values for stored data arrays. The mathematical basis of the algorithm consists of modular arithmetic methods, the Chinese Remainder Theorem, and cryptographic hashing. A data array is represented as a vector of residues with separation of the system’s informational and control modules. Verification of the reliability of the data integrity assurance result is achieved by means of an independent global reference value of the original array and local reference values of its sub-blocks, placed with threefold replication in an isolated cluster of storage nodes. In the absence of signs of integrity violation, the algorithm performs reading using the minimum required number of sub-blocks and a single global cryptographic check. When signs of integrity violation are detected, verification against local reference values is performed, a minimally sufficient set of correct sub-blocks is formed, and the original array is reconstructed according to the Chinese Remainder Theorem. The result is then verified by cryptographic integrity control, after which the corrupted sub-blocks are rewritten and the metadata are updated. The proposed algorithm ensures the integrity of processed data, localization of sub-blocks exhibiting signs of integrity violation, and early completion of integrity control and recovery procedures in distributed storage systems.