Decentralized data: The challenges of data retrieval from the blockchain

As blockchain technology becomes increasingly integral to financial systems, understanding the flow of data from blockchains is essential for accurate financial reporting. Despite the common perception of blockchains as immutable and transparent, a lack of understanding of the quality of blockchain data within open-source and third-party interfaces remains an issue for financial reporting. 

Each source and type of on-chain data can present unique challenges. By implementing stringent vendor management and data verification controls, organizations can mitigate risks associated with blockchain data and enhance the reliability of their financial reporting.

Understanding the flow of data from a blockchain

Public blockchains are well known for their transparency, but structuring data for analytical purposes can be complicated. Raw blockchain data is initially stored within a blockchain node or copy of a blockchain’s ledger. Raw blockchain data is then extracted into a database and goes through a computationally complex process called indexing. Indexing takes the vast amount of unstructured blockchain data and transforms it into a searchable format. The final step involves querying the indexed data for analysis and documentation. 

Options for accessing blockchain data

There are generally three avenues a company or individual can take to access the data flow: a block explorer, running a node and self-indexing, and using a third-party node provider and indexing service.

Here are some advantages and limitations of each option as it relates to financial reporting:

1. Block explorers: Public block explorers are tools that enable individuals to view transaction data without running a node, contracting a third party or developing an indexing solution. However, there may be heightened concern with their use for financial reporting due to their often organizationally opaque nature. The data flow for block explorers is not always public information. Furthermore, depending on the chain, the explorer may not display all internal transactions and smart contract interactions or provide sufficient historical data. As a result, block explorers may lack the security and data integrity assurances required for enterprise use.
   
   
2. Running a node and self-indexing: Running a node is akin to operating your own copy of a blockchain’s transaction ledger, including validating all transactions that are added to a blockchain on an ongoing basis and having access to the full archival data of a blockchain’s activity. Operating a full node can be beneficial because the data comes straight from the chain itself and enables full control over assessing the accuracy, completeness and timeliness of the blockchain data used in financial reporting. Still, there are significant limitations to this route. Running a node can vary in complexity greatly by chain, but it almost always includes high-performance hardware and substantial storage capacity. The cost associated with hardware acquisition, maintenance and energy consumption can be prohibitive.
   
   Likely even more prohibitive is the complexity of running a node, and different types of cryptocurrencies require different indexing approaches. Developing and maintaining indexing infrastructure also involves dealing with blockchain reorganizations, fork handling and real-time data updates. The complexity increases with blockchains with high throughput or support smart contracts, like Ethereum.
   
   
3. Third-party node data providers: Using a third-party node provider allows organizations to access blockchain data without the complexities and costs of running their own nodes, as these providers offer access to full nodes via options such as application programming interfaces. This option significantly reduces operational overhead, such as hardware maintenance, storage and synchronization efforts. However, using a third-party node provider and indexer introduces new risks, such as failing to recognize when there are data inaccuracies and a lack of control over the third-party node data provider’s validation process and data maintenance protocol. Most notably, many node providers lack a Service Organization Control (SOC) 1 report—which is essential for assessing internal controls related to financial reporting—and many have indexing limitations or sell data obtained from another third party for more nascent chains they do not run nodes for.

Recommendations

While no option for blockchain data retrieval is without its limitations in financial reporting, there are ways organizations can mitigate these challenges. First, organizations must implement a data validation framework, which begins with establishing a reconciliation process that cross-verifies data obtained from third-party vendors or block explorers against internal transaction logs and ledger entries. By completing regular reconciliations, companies can identify and fix discrepancies, ensuring financial statements are based on accurate and complete data.

Second, organizations must have a stringent third-party vendor management system and conduct due diligence on their vendors’ data acquisition methods, indexing processes and error-handling procedures. Understanding the vendor's entire data flow—from node operation to data delivery—enables companies to identify potential points of failure or inaccuracy and align the vendor’s controls with the organization's standards for data integrity.

Third, companies must establish internal controls over their data processing systems. More specifically, companies should have validation checks that flag anomalies or deviations in data. These checks need to include a detailed audit trail with logs of data retrieval and processing activities to ensure accountability. This audit trail needs to be reconciled from multiple sources, not only from a custodian of the digital assets. For similar reasons why many third-party data vendors lack SOC 1 reports, numerous digital asset custody platforms also do not have SOC certifications. As a result, companies must independently verify that the assets represented as their holdings on a custodian’s user interface are accurately reflected on-chain. Internal controls also need to enforce policies that limit access to data systems and have a role-based permissions architecture to reduce the risk of unauthorized data manipulation.

The takeaway

Accurate blockchain financial reporting requires a deep understanding of on-chain data flows and the challenges associated with data retrieval. Organizations must carefully evaluate their options for accessing blockchain data, considering the costs, technical complexities and risks involved. By implementing robust processes and controls, including thorough vendor management and reconciliation practices, organizations can ensure the completeness and accuracy of their financial data derived from on-chain activities. As the blockchain landscape continues to evolve, staying informed and adapting to new technologies and methods will be essential for maintaining data integrity and compliance in financial reporting.