Indexer Office Hours Summaries

Indexer Office Hours - YouTube

104

Introducing The Graph Substreams for High-Performance Indexing by Alexandre Bourget - YouTube

INDEX OFFICE HOURS

Welcome back to another edition of Index Office Hours. This is the town square for Graph ecosystem participants to discuss the most important topics. Please contribute to the conversation via text in the text box or by unmuting your mic. This session is going to be recorded and available on public channels like YouTube.

ARAGON 2.43

Aragon 2.43 adds first phase of Kaplan, an embeddable consensus layer implementation. Release also removes support for BSE, adds more block snapshots for Bor polygon. If you're still using a Lighthouse CL still and not trying the new CL at this time or anytime soon.

NETHERMIND 1.17.3 NEW RELEASE

Next release is Nethermind. release number 1718.4. This is a low priority release intended only for nodes that are syncing from scratch. Let us know if you've tried this version.

CELLO, PRISM, AND AVALANCHE GO

Next release is Cello and this is version 1.7.4. This includes a fix for the format of headers sent via WebSocket connection. For Prism we have had a new release. This is version 4.0.3. This release has a number of important fixes.

MAINET 4.0.2 RELEASE

This is a high priority release for Mainet users. It fixes an issue which caused excessive CPU usage shortly after the main net capital upgrade. Users on version 4.0.1 or earlier should upgrade as soon as possible. The minimum supported Rust version has been set to 1.68.2.

GRAPH STACK RELEASE 0.16.2

New release on Index Error Service and Agent. G-I-P 51, which is Exponential Query Fee Rebate for indexers. New clarification from Justin regarding curation tax and protocol fees. And with that, this is all I had for this week.

ARBITRARIUM: MIGRATION HELPERS GIP 46 & 52

Next up we have GIP 46, which is all to migration helpers. There's an interesting discussion regarding the implications of the migration on existing subgraphs. Next up is a new JP regarding timeline and requirements to increase rewards in all two.

NEXT UP, FORUM RESEARCH. PROTOCOL WATCH.

Next up, forum research. We don't have any new updates since last Ire. Contracts Repository we have a few chores as well as some testing and new implementations of the exponential rebate. Protocol watch. GF 52 is going to be covered more extensively in a few minutes.

GIP 52 LAUNCH

We're going to be speaking about GIP 52. I'm going to share my screen moment. I can show you folks GIP 52 and we can quickly go over it. I recommend that you folks give it a read as well, just in case we miss anything today.

MIGRATION FROM L1 TO L2

At some point, we think it makes sense that we should migrate fully to L Two. Because indexing rewards are such a big incentive for indexers. We want to do smooth migration and a smooth rewards shift from a one to two.

MIGRATION TO L2: THE 25% MILESTONE

Pablo: I think Anton makes some really great points around the 25% milestone. Once we're sort of comfortable at that 25% point, then I feel like the game is more about delegators and curators. How do we maximize the awareness of this program for our indexers?

ON THE NEED FOR DELEGATORS

The council will need to send an onchain transaction. The very important thing missing everywhere is the query fees. What if our customers will still use Ethereum subgraphs instead of arbitrary subs. That's something that we need to maybe think about what would be a right amount.

MIGRATION TO L2

The only thing that can drive delegators to move to L two are incentives. Maybe a metric could be some sort of multiplier on the delegation to stake that we can use. I think extravale explorer is a great tool for most of the things.

ON THE DECENTRALIZATION PLAN

There is going to be several millions of Jersey wasted, not stuck on layer One, but doing nothing layer one. As long as the network health is good and everything is working, it's up to the user to choose. There are some network effects in having people using the DL. Two.

MIGRATION: SHOULD WE SPLIT THE GIP?

Pablo: I'd like to try to distill this conversation into things we can change on the GIP. The first decision I think we need to make as a group is do we have a delay between the first that the helpers going live and then the first bump in rewards?

ARE THERE ANY BRIDGES ON GRAPH?

There will be a bridge launched by the Graph itself for those who want to have some GRT unstaked on Ethereum and want to bridge it to Arbitrarium. Have we thought about engaging into some bridges just to get better exposure?

LET'S TALK HELPERS GO LIVE

All right. Maybe we keep the discussion going either on Discord or on the forums. My preference is generally the forums because it's more sticky and people can keep reading afterwards. But happy to chat on Discord as well. Let me know what makes sense as a cadence.

#103

#IndexerOfficeHours 103 - YouTube
AssemblyAI | Playground

INDEX OFFICE HOURS

This is the town square where ecosystem participants come together and discuss the most important and relevant topics for Indexers. The call is recorded and will be published onto public channels like YouTube. At certain parts of today's conversation, please unmute your mic or use the chat box.

GRAPHICAL ECOSYSTEM CONFERENCE

This is the GRT IQ podcast. Chidaboom is a Graph advocate and a member of the Graph Advocates Dow. He recently launched Graph Central, which is an all in one resource for the community. Anna will take us into our regularly scheduled repo watch.

AVALANCHE 1.10 U3 UPDATE ON MAINNET

All Mainet nodes should be upgraded before 11:00 A.m. Eastern time, April 25. This upgrade contains linearizes, the X chain, which introduces delegation batching to the P chain and increases the maximum block size on the C chain. Matthew is telling us that he upgraded yesterday.

PRISMA 4.2.1 RELEASE

On the consensus layer client, we have a new Prism release. This is version 4.0.2. There were a lot of fixes, particularly in the Prism API. You need to upgrade to this release if you run Prism with Relayer and Mavboost. I changed from internal consensus layer to Lighthouse on Aragon.

LIGHTHOUSE RELEASE 40.2

All Mainet users should update this release to prevent high CPU usage from interfering with normal node operation. Anton confirms that Syncing the new version of Prism and is going well. I will include internal CL in Repo watch for next week.

INDEXER AGENT AND SERVICE RELEASE 20.13

New release for indexer agent and service. Indexers can now connect to any protocol network and claim rewards for indexing subgraphs. Action Queue was fully added with the new subcommon. New action query commands available to inspect and manage the action queue using the indexer CLI.

REPO WATCH: NEW RELEASE AND INTERNAL CL

Please let me know if anyone has tried this release. Vincent from Data Nexus is saying after every hard fork you can't restart the internal CL for 1215 minutes due to light client limitation. Unless there are any other comments or questions, I'm going to give it 1 minute.

PROTOCOL WATCH

There was a clarification on the removal of channel dispute epochs. Derek explained some potential use cases for SQL aggregations on patterns that aren't particularly performant in subgraphs. Next up, we got G-I-P 42, a world of data services.

G-I-P 46

Next up, we have a new update in G-I-P 46. This is the l Two Migration. There were a few changes to the current helpers. Forum Research has no new updates since last IOH. Then we're moving to Contracts repository.

PROTOCOL WATCH 0.19

Next up, we have the network subgraphs. There was a small issue the past week regarding arbitrary. This was due to ENS not being present in Arbitrarium. That's pretty much it for all of the changes for Protocol Watch. So I'm going to hand it over to Abel now.

WORKSHOP ON INDEXING

David Luther will give us a presentation that will help hopefully many of you level up your indexing game. This presentation came out of an internal presentation I gave a few weeks ago. There's a lot of good stuff for indexers in there too.

WHAT A BIG GRAPH NODE LOOKS LIKE WITH SHARDING

A really big graph node installation looks like in terms of shards. We have shards for just the block cache. Then we have shards that just have subgraph data. Finally we have a shard for really high volume subgraphs. Not something I recommend anybody doing right away.

GRAPH MAN: THE PRIMARY, SUBGRAPHS, AND MORE

The primary is a few tables in there that are sort of the authoritative source for information. For newer setups, the block cache for each chain goes in its own database schema. If you're curious about more of what the metadata for Graph Node. looks like.

CACHING IN C#

One of the problems we encounter quite a bit in the hosted service is that some RPC provider gives feeds us back bad data. In those cases, you might have to use this call cache, Remove Command, and then rewind subgraphs that are affected.

POSTGRES: HOW TO FIGURE OUT HOW BIG A SUBGRAPH IS

In the database, there's materialized views in the info namespace. Total bytes is like how much storage that subgraph takes up on disk altogether. Toast is one of the drivers of why subgraphs become really humongous. How would the user know who to complain to on the decentralized network?

POSTGRES 10.2: SLOW QUERIES

We have a dashboard that shows the PG Stat Activity table from postgres. Shows how long the query took, how many entities came back and if you need to investigate. Can also get a trace of what makes a GraphQL query slow. Find that query that's slow and go nuts with plans to improve it.

POSTGRES 2.8 GRAPH AND STATS, ANALYZE & OPTIM

The Graph and Stats command can show you a bunch of statistics about an individual deployment. The next command you can use to analyze a table will tell Postgres to go through that table and gather statistics. It can occasionally happen that running an analyze actually fixes a query performance problem.

GRAPHENE 2.8: GRAPH NODE & INDEX CREATION

Graphene also has some commands for managing indexes. You can use graph men to create an index that's better than doing it in psql. The last one is just a way to drop an index.

GRAPHMAN: GRAFTING AND COPYING

There's ways to move subgraphs between shards. The way you do that is by copying a sub graph behind the scenes. Grafting and copying are almost the same thing. If you want to keep an eye on how much grafting or copying is happening in your graph node, you might want to have a dashboard.

ETHEREUM'S INDEXING METRICS

Next? Yeah, indexing metrics. There's some metrics that we collect that can help shed some light on what a subgraph is doing while it's indexing. If you see a lot of time in transact block, that means that the trigger processing stage is waiting for the database writer.

GRAPHQL 2.8: LARGE ARRAYS, AND MORE

subgraphs should use bytes as the type for the ID field as much as possible. Large arrays are really bad for performance, for storage and all that. Using these two things together showed a 30% improvement in indexing time and in storage size.

HOW TO GET STARTED IN DATA SYSTEMS

If you really want to get into databases, the CMU, Carnegie Mellon has the database group there, has an absolutely fantastic YouTube channel. They have two of their database classes online. And then the advanced database systems class.

POSTGRES: GETTING STARTED WITH INDEXES

The Postgres docs are if you haven't looked at them, they're fantastic. The Index loop is a really good resource on how indexes work. Bruce Momjin gives presentations at pretty much any postgres event. It's always worth checking them.

FOOLED BY SCIENCE: DAVID LUCK

David Luck: That was a great in depth presentation. I'm going to link it there so people have access to it. It's not public, right? Not yet. I'll make it public. Fantastic.

POSTGRES 8.2

Have you tried any of the postgres extensions like Ciders, Timescale DB for indexes? No, we haven't tried it. If there are any ones that are still outstanding, maybe you could answer them.

REPLICATION WITH SHARDING IN NODE 2.8

When implementing sharding, would our Replica also need to have the same sharded structure? From the graph node side, copying is transparent. No serving data won't be compromised during copy. If anyone else has any additional questions please either type them in the chat.

DB CHAT: MORE QUESTIONS THAN ANSWERS

How large of a performance impact is there to the DB when copying? What's the best way to ask more questions? Maybe a DB channel? Maybe just the index of channel.

WONDERS OF IOH WORKSHOP

Luther: Well, I think we can wrap today's conversation. Thank you so much, Luther, for the workshop. We will hopefully do another one of these at some point in the future again. Take care, everyone, and bye bye.

#102

INDEX OFFICE HOURS

Today is the 11 April and this is 102 IOH. This is town square for Graph Ecosystem participants to come together. We'll be breaking out into various different topics. If you'd like to mention anything, please do via the text box in the corner or by trying to unmute your mic.

GTR IQ: SUPERFLUID AND THE BLOCKCHAIN

Vincent is a web three full stack engineer at Superfluid. The company is planning to revolutionize the way that value is transmitted on Blockchains. Vincent and Nick also discuss the importance of the graph. And then Anna will take us into Repo Watch.

FIREHOUSE 1.8.3 REPO WATCH

Today's repo watch is ridiculously light. We only have two releases, so we're going to fly through it really fast. On the execution layer clients, we have had a new release for Firehose Eat. Does anyone have any comments or would anyone like to share anything about the repos?

AT THE FORUM GOVERNANCE 51: EXPONENTIAL QUERY FEE REBATE

Forum Governance 51, which we will be discussing today, is exponential query fee rebate for indexers. No MIPS updates today, but there's obviously been an email, so everyone should have the updates in their inboxes today.

QOS DATA

Craig and Ricky talk to us all about QoS data, something that we've been thinking a lot about here at IOH. We want the graph to be a viable solution for anyone wanting to use blockchain data. Give a little background on the architecture and some of the ways that you can use the sub graph.

ZEPPELIN 2.8: INDEXER ATTEMPTS AND QUERY RESULTS

We use Open Zeppelin auto defender for the on chain transactions. Instead of the requests that gateways make to indexers, you can think about this from the perspective of the user. We net out user attributed errors. You can use the data to create dashboards, et cetera.

MIPS DATA PORTAL

This is essentially just like a public facing dashboard which ingests data from the subgraph. It was just something we made to help support the MIPS program. We would love to see Indexers or whoever else in the community create dashboards analysis.

INDEXER VISUALIZATION IN CSP FOR MIPS

The code for this is open source. This is written in Python using streamlit, so it should be pretty straightforward to work. We definitely encourage people to kind of create their own solutions.

IPFS 2.8 IN STREAMLIT

Craig: By end of week Friday, everything should be in there. Basically, the data in the subgraph itself should be there. There's still like an alpha release here, so we don't guarantee the availability or the timeliness of the data. Use at your own risk.

VINCE MCMAHON ON SHANGHAI WATCH PARTY

If people are interested in a Shanghai Watch party, can you put a plus? Let me just do a plus plus. If there's sufficient demand, I think we could definitely entertain that.

QOS IN THE NETWORK

We think QoS is super important. We want to keep an eye on that as indexes in the ecosystem and try and make it as best as possible. We're also going to have some sort of like, network watch segment that we've been speaking about that for a while. Keep your eyes peeled for that.

GIP 51

Next section features Howard, Justin and Tomas talking about the GIP. If any of you want to kick off, feel free to unmute your mic and get straight into it. Adam, if you could do maybe a little primer on the jip while we're getting that sorted.

EXPLONENTIAL QUERY FEE REBATES FOR INDEXES

Drawing attention to GIP-51. So exponential query rebates for indexes. I'd really encourage people to check it out on the forum. We're pretty excited about it. We want to make sure that it gets broad coverage.

INDEXERS' REBATES UNDER THE KAP DOUGLAS MECHANISM

The new mechanism is called exponential rebates. It proposes to remove the current cop dogs based formula. It removes the dependence from other indexers and it also makes it very predictable to know how much you are getting out of the amount that you are generating. Burning less fees means more fees are distributed to indexers.

INDEXER AND THE STAKING CONTRACT

The proposal would remove the need for rebate pools. The main levers to control this formula are again what we call a stake ratio. There are some notes regarding this for the transition period. One of the challenges of this G IP is the transition from one model to the next one.

A SPECIAL THANK YOU! FOR SPEAKING ON NODE

Thank you, everyone, for hopping in. Special thank you to all of those who spoke today and engaged and shared with the community. Looking forward to doing this again next week. We have David, David Lutter from Edge Node who's going to be sharing some amazing stuff.

GIP-51 Exponential Query Fee Rebates

#100

YouTube

INDEX OFFICE HOURS

Today is the 28 March, and this is IOH 100. At various parts of today's episode, we would love to hear your voice either by text or unmuting your mic. As a reminder, this is going to be recorded and put onto public channels like YouTube.

METIC INTERVIEWS GRTIQ AND MORE

In this episode, Nick interviews Fauci, the co founder of Mimic. Mimic is a DFI solution that makes it super easy and safe to be able to invest in DFI via smart vaults. On Aragon, we have had a new release.

NEW NETHERMIND RELEASE 1.17.3

New Nethermind release. This updates the change spec for the Shanghai Mainet upgrade. No configuration changes are required on Nethermind side to work properly on the Chapela Hard fork. Main at ChappellA hard Fork will be happening again on April 2023.

ARAGON 1.9.15, LIGHTHOUSE 1.8

On Avalanche, I've added Avalanche this week so only covering the new releases from this week. Matthew has upgraded Aragon and Lighthouse and both are working fine. If there are any comments, please share in the chat or unmute yourself.

CAPELLA HARD FORK ANNOUNCEMENT FOR PRISM 4.3.

Prism upgrade from version three to version four. On Teku, we have had a new release. This is again required update containing the Capella fork configuration. All users should upgrade before April 12. You should upgrade your execution layer client first before upgrading Consensus Layer client.

ARAGON 2.5.1

There is a known incompatibility between Lighthouse version four onwards and Arrogant version 2.40.1. If you're running Aragon with Lighthouse, you should remain on versions three point X. Any comments, questions?

PROTOCOL WATCH

For Protocol Watch, we have a few updates going into firm governance. Next we have GP 43, which is activate Indexing Rewards in the graph network. For research getting into the contracts repository, the work related to the gaps that we were discussing. That's pretty much it for Protocol Watch.

INDEX OFFICE HOURS

Today is the 100th episode of Index Office Hours. How do we feel about bringing in a drop in style clinic? More office hoursy type call? So potentially recorded, non recorded? We can figure that out.

FIREHOSE 100

Matthews asked the group if you want more presentations on Firehose substreams, Evmx, and specifically what topics in addition to those ones you'd like to hear. This is the perfect opportunity as we kind of sit back and reflect on which 100 and think about what the next hundred will be like.

THE NETWORK AS A THINK TANK

Derek: I definitely enjoyed the ability of indexers to kind of coordinate and get a feel for what's happening in the protocol. And that's really useful because we can definitely explore bringing that back as well.

INDEX OFFICE HOURS

I think for me, what I see as one of the functions of index or office hours is to help keep track of everything that is happening and everything that we should be aware of. That would be very helpful, at least in my opinion. That's fantastic feedback. We definitely want to serve that function.

HTTP://ASYNC.IO/NEWSLETTER

What about a newsletter? Is that something that people will be. We're debating setting one up as a way of like, you know, serving as an async function. How do we feel about that? I'm going to do another poll.

THE NEXT STEP: A NEWSLETTER

Anton: Some people are saying they want a newsletter, some people say it's too much. I would like to know beyond what's already in the chat, what people would want to see in the newsletter. I think we'll definitely continue doing the call.

MIPS: HELP NEW INDEXERS GET STARTED

Anyone in here like a new indexer who's joined the MIPS program? A lot of you new indexers are saying that IOH is great but a bit overwhelming. We're also considering setting up anew index of call, maybe biweekly or monthly.

WANT TO HEAR MORE ON THE CALL? HERE!

We're at the 100 mark. We're constantly trying to evolve this call to make it as useful as possible for the indexers in the call. There's a little survey thingy that you can pop your feedback in. You can either share topics that you want to hear in the future.

ANY TOPICS YOU WANT TO DISCUSS ON IOH?

Well, I will pause here because I want to create the space for people to chat about any interesting things, any interesting topics. And so, yeah, if anyone wants to kind of shoot the sh*t on any topics, now is the opportunity.

DEREK ON DISASTER RECOVERY & HOSTED NETWORK

Have we had any update on the sunsetting of the hosted network? Disaster recovery or disaster preparedness potentially could be a very big topic. The team have been super busy trying to get back to everyone and rest assured you'll hear back from them.

CHAT IS ON FIRE

Wow, the chat is really on fire. I think we can wrap today's episode, but keep the call going. If anyone wants to stick around and continue the chat in the chat chat box, you're more than welcome to.

#99

#IndexerOfficeHours 99 - YouTube
INDEXER OFFICE HOURS

Another episode of Indexer Office Hours. This is IOH 99, very close to that 100 mark. You can contribute to the conversation either via text in the text box in the corner, or you can unmute your mic. This session is recorded and will go onto public channels like YouTube.

BLAIR DARBY ON THE GRAPH ADVOCATES

Darby is a very active member in the graph advocates and the dow. This was a very interesting podcast episode because it was a bit of a personal story of her. And with that, I will hand it over to Anna, who's going to take us into our regularly scheduled Repo Watch.

FIREHOUSE ETHEREUM: NEW RELEASES FOR POLYGON AND SHANGHAI

On Repo Watch from an Execution Layer Client, we have had a couple of new releases related to new polygon versions on Firehouse Protocol version two two. And then on streaming fast. We have had quite a few release. Matthew, if you have anything to add to what I just mentioned, either in Chat or unmute yourself.

ETHEREUM 17.2 AND LIGHTHOUSE 3.5.1

We have had two new Nethermind releases. 17.2 is a hot fix addressing an age case issue with occasional bad blocks production. On Consensus Layer Client we have had the new Prism release. We have also had a new release for Lighthouse 3.5.1. That's all we've had.

A TOAST TO JUAN

With that, I'm going to hand over to Juan. Can everyone hear me? Yeah, loud and clear. Great. Thanks Anna.

FORUM GOVERNANCE & PROTOCOL WATCH

This is going to be a super short protocol watch because it's not that much to update. Forum Governance we don't have any updates since last IOH. Next up, we have the optimizing GRT liquidity on uniswap B Three by range protocol. And now I'm going to hand it over to Pedro.

QOS AND THE DASHBOARD UPDATE

The Oracle and the subgraph have been working fine since March 11. We did export the QoS data once more. The dashboard is indeed working fine, so I find that super helpful. Not sure if you've been using it? Let me know in the comments.

GIRLY 2.8

Next, we have the polygon missions. We've shared a bunch of updates there and mostly some bootstrap help from Pinocchs. For those that are running the Girly firehose as a test, before you go on to the full polygon or whatever other chain you want to work with.

L2 MISSION UPDATE!

We're basically collecting all of your contributions here on this mission. You'll have until March 27, meaning next month, to share all of these. Evmx workshop that was scheduled for next week's, IOH, will be postponed for a week.

L2 ANNOUNCEMENT

The main action item, I believe, right now is the L Two forum, where we have a deadline, March 27. The best thing for you to reach out is to do so by by email. Give us some time and we'll respond.

GRAPHCAST LAUNCHES TODAY

Graphcast is launching officially today. A blog post will come very shortly after today. Let me just go ahead and share my screen so that you guys can see the presentation that I have put together. Very rough presentation, but it is one nonetheless.

GRAPHCAST LAUNCHES THE GRAPHQL NETWORK

Today is the day that we launch Graphcast, the Graphcast network as well as the Poi Radio and the SDK. The purpose of the network is to enhance productivity, efficiency and cost effectiveness in the ecosystem. There are many types of radios that could be built on top of thenetwork.

POI RADIO

On top of the Graph cast network, we can build radios. A radio is basically for exchanging information between ecosystem participants. We have added MultiChain support to see the full list of supported chains. Anyone who wants a session with someone from our team to kind of ask questions, give feedback, or just in general helping out with setting up Poi Radio.

DEMO TIME!

Thanks to index Adao. They've been a massive help in terms of testing this thing. Looks like PECO is back up, so I will mute my mic. You need to switch streams from Ablestream to Petco stream in order to see his share.

POI RADIO

Poi radio really soon is going to have very cool data rich prometheus metrics. Right now we only have these boring graphs of number of valid messages. Let's see what happens when we start getting divergent messages.

POLL: TELEGRAM, SLACK OR NO MORE NOTIFICATIONS?

Chris has put a poll down to ask if Telegram, Discord, or Slack is the place that you guys would like to receive notifications. No trolling, please, because we'll actually take this as genuine feedback that we'll potentially implement into this.

BUILDING A GRAPHQL RADIO WITH THE SDK

We've built an SDK, and it's effectively a library for making graphcassing radios. What we've done is we've put together a small little ping pong app to show an example as to how to build a radio using the SDK. And then next, just to kind of give you guys a visual representation of how all of these things interlink.

POI RADIO AND SDK: ANY FEEDBACK?

For the Poi Radio, we would love for you guys to use it. And then the SDK, we'd love to see you guys build radios. The more feedback, the better. Now we can open up for Q and A.

A SWAG REQUEST FOR CHRIS

Any UI or grafana dashboards. Feel free to unmute your mic and ask questions like that. I see people typing. I'll just give people a second to finish typing. Vince, are you asking us to send you a journal for radio?

GRAPHCAST: COLLABORATION WITH NON-INDEXERS

How would someone who's not necessarily indexing, not necessarily be running the software necessary for the interfacing with various radios? How would they ultimately be able to communicate via radio. If you are interested in setting up a radio, I would love to support any additional questions.

SELF-HOSTED INDEXERS IN GRAPH NODE

Has anyone tried sharing chain data across isolated indexers? Has this been done before? Is it possible or is Graph Node really ever intended to work this way?

POI RADIO

At present we have the Poi radio running. Anyone can go ahead and spin up the radio and start playing around with it. We would love to hear feedback and kind of walk through this with you guys. All of what gets implemented will flow through GitHub first.

COMMENTS ON THE RADIO SDK

Final round of questions, thoughts. Please do comment in the comment box or feel free to unmute your mic. I want to say a big thank you to everyone who has joined us for today's conversation. We look forward to seeing all the magical things you guys build.

Substreams

Introducing The Graph Substreams for High-Performance Indexing by Alexandre Bourget - YouTube
AssemblyAI | Playground

THE GRAPH: A STORY OF DATA SCIENCE

Streaming Fast is one of the core developers of the Graph. Alexander: One day in 2013 I read the bitcoin white paper and that changed the trajectory of my life. We shifted our focus to make the Graph the greatest data platform on Earth.

WHAT IS SUBSTREAMS

substreams is a powerful new paralyzed engine to process blockchain data. Instead of the traditional subgraph handlers and assembly script, you will write modules in rust. They can be executed in real time as well as in parallel with unprecedented performance.

WHAT IS SUBSTREAMS

substreams is a powerful clustered engine to process blockchain data. It's a streaming first engine and it's powered by the firehosery and its data models of the chain. All results are streamed back to you as fast as possible.

SUBSCRIPTIONS AND THE BIG DATA

substreams Engine can allow you to do some things like that ad Hoc. You could write any program that supports gRPC and protobuff through graph Node. Now you have a really fast engine, allows you to process the whole history in. parallel. Stay tuned for that.

RUST 2.8: UNISWAP V3 DATA MODEL

Rust provides a simplified data model for units swap V Three. Here we show some code making use of it. To preserve parallelizability, the stores are write only. This means that once we've processed this layer once, it can be cached very efficiently.

BITCOIN 2.8

You can inspect the chain with your code at any place in time for a mapper. This really changes dynamics for debugging. This can also work for stores because you can ask for a store and it's going to process it in parallel.

SQLITE: GRAPH OUT IN PARALLEL

Graph Out graph out has refined the data up to entities that we're talking about, database tables and fields. This looks very much like Change. Change data capture CDCs that can power a lot of large scale systems. Prepare your question, please.

A VISION FOR THE GRAPH

I'm imagining the graph becoming that sort of huge worldwide cluster of processing and storage capabilities. I see also a new era of composability, which means more collaboration and in a tighter community. All of this is open source. Let's dig and build something like together the biggest blockchain platform on Earth.

BLOCKCHAIN AND ITS HYPER-MODULARITY

The transformation layer is not the SQL layer. It's an ADOC transform with stateful storage. To enable parallelization, you need to distinguish the data dependencies. It brings us to the next stage an evolution of blockchain indexing.