Managing Blockchain Data Efficiently

19 replies 74 views
AtomicSeedFull Member
Posts: 30 · Reputation: 621
#1Nov 28, 2021, 02:32 PM
Got an idea for managing blockchain data better. As we know, Bitcoin's blockchain is getting bigger, and it's putting pressure on node storage. I suggest we could summarize transactions and expire older blocks. This could help keep things manageable.
3 Reply Quote Share
gang2015Member
Posts: 682 · Reputation: 62
#2Nov 28, 2021, 07:09 PM
But have you considered that Bitcoin has pruned mode on some full node software? Like, you can just store recent blocks and the UTXO set, right? Seems like your idea complicates things without really giving an advantage.
0 Reply Quote Share
AtomicSeedFull Member
Posts: 30 · Reputation: 621
#3Nov 28, 2021, 11:50 PM
I get the pruned mode thing. But my idea focuses on summarizing old data and using backup nodes for access. It’s about making sure we can scale without losing access to the history. What do you think of that?
5 Reply Quote Share
Posts: 32 · Reputation: 237
#4Nov 30, 2021, 06:25 PM
You're missing a key point here: how much storage are we actually saving? If it’s just a small percentage, all that complexity isn't worth it. Seriously, how much less storage are you proposing?
3 Reply Quote Share
gang2015Member
Posts: 682 · Reputation: 62
#5Dec 1, 2021, 12:01 AM
And hey, have you heard of the Mimblewimble protocol? It basically lets you forget almost all data of spent outputs while still keeping things verifiable.
4 Reply Quote Share
AtomicSeedFull Member
Posts: 30 · Reputation: 621
#6Dec 1, 2021, 05:54 AM
1. Pruned mode covers all the benefits you mentioned without adding unnecessary complexity. 2. We can rely on existing full nodes to store the entire blockchain, so no need for backup nodes.
2 Reply Quote Share
lonesatHero Member
Posts: 2 · Reputation: 3534
#7Dec 3, 2021, 02:14 AM
I calculated that after 200 years, a full node would need around 105 TB. It’s crazy how Satoshi’s design holds up.
1 Reply Quote Share
wizard_2016Full Member
Posts: 127 · Reputation: 575
#8Dec 3, 2021, 03:41 AM
Absolutely! Reducing storage is key, but we have to think about tax and compliance issues too. Summarizing transactions might complicate audits and legal stuff. We need to find a balance.
2 Reply Quote Share
Posts: 185 · Reputation: 37
#9Dec 3, 2021, 09:13 AM
I think the bare minimum to keep is the last block, the UTXO set, and timestamps from the last difficulty adjustment. That could be secured with hashes in blocks.
0 Reply Quote Share
AtomicSeedFull Member
Posts: 30 · Reputation: 621
#10Dec 3, 2021, 03:24 PM
I doubt your prediction about blockchain size. My full node shows different space usage. We should analyze the growth since the spammers started clogging mempools.
5 Reply Quote Share
gang2015Member
Posts: 682 · Reputation: 62
#11Dec 4, 2021, 06:54 AM
Are you saying the blockchain will grow more than your estimate in 20 years? I started this conversation to see how we can approach the future together.
0 Reply Quote Share
gr3g.4ltFull Member
Posts: 58 · Reputation: 419
#12Dec 5, 2021, 09:00 AM
You’re referring to UTXO commitment, right? That stuff was never implemented in Bitcoin Core due to the data size concerns, but there's a similar feature called UTXO snapshot.
0 Reply Quote Share
0xLynxMember
Posts: 113 · Reputation: 205
#13Dec 5, 2021, 02:08 PM
Can blockchain size go over 1 MB? Not sure about that.
4 Reply Quote Share
AtomicSeedFull Member
Posts: 30 · Reputation: 621
#14Dec 7, 2021, 12:45 PM
Before SegWit, blocks were capped at 1MB. But after SegWit, they can go up to 4MB.
4 Reply Quote Share
gr3g.4ltFull Member
Posts: 58 · Reputation: 419
#15Dec 8, 2021, 05:44 AM
So SegWit lifted the 1MB limit, huh? The average block size is like 1.7MB now. So growth is way beyond your 51.36GB estimate.
2 Reply Quote Share
Posts: 185 · Reputation: 37
#16Dec 8, 2021, 10:21 AM
Your calculations seem off. With SegWit, I think we’re looking at a growth of about 87.3GB, which fits better with my findings from charts.
3 Reply Quote Share
Posts: 32 · Reputation: 237
#17Dec 8, 2021, 05:21 PM
What do you mean by "from scratch"? Did you set assumeValid to 0?
1 Reply Quote Share
0xNodeMember
Posts: 671 · Reputation: 80
#18Dec 8, 2021, 08:25 PM
There’s already a proposal for compact block rollups, but it needs a way to bundle up unspent UTXOs from a certain date. All nodes would need to serve this in an alternative version.
3 Reply Quote Share
orbit1337Senior Member
Posts: 434 · Reputation: 941
#19Dec 8, 2021, 10:25 PM
You’re basing your proposal on a flawed prediction of blockchain growth. When I got into Bitcoin, I had a 512 GB disk. Now I have 12 times that, and I just prune when I need. Your idea might work for personal nodes, but it's not good for new users who want to verify from genesis.
2 Reply Quote Share
Posts: 185 · Reputation: 37
#20Dec 9, 2021, 01:14 AM
When I say "from scratch," I mean I didn’t store any blocks yet. Folders were empty, and I didn't touch assumeValid in my config.
4 Reply Quote Share

Related topics