I am working on an on-chain data model in a corporate data warehouse setting. All data must live on an internal system where it can be easily joined to other datasources, and support a variety of high performance analytics use cases.
The model will serve common stats you would see in a block explorer like Mempool Space, but it should also support arbitrary queries like filtering by date or blockheight, or aggregations over some subset of addresses or address types. The secret to fast aggregate query performance here is a columnar database (i.e. Snowflake, Google BigQuery, or Apache HBase).
While exploring options to ingest on-chain data, I found that all of them require some form of Electrum Server, and it seems that Electrs is preferred among modern projects. What I have not been able to find is a detailed description of what exactly Electrum Server does on a technical level. Does anyone know where I can find:
- A full technical description (not implementation) of everything Electrum Server does to make data more available to block explorers and wallet software? This is a great start but only covers the high level idea: https://bitcoin.stackexchange.com/a/116582/142060
- Whether it's possible to recreate this additional data indexing, etc. from data extracted directly from Bitcoin Core? If not, how does one trust the results that an Electrum Server provides?
Thanks in advance
