[Proposal][Status: Idea] Dynamic Fee Mechanism for Moonbeam and Moonriver

Dynamic Fee Mechanism for Moonbeam

Hi, I’m Stephen (aka notlesh) from the Moonbeam core dev team. I focus a lot on the security of Moonbeam, which includes the various fee mechanisms we support.

Summary

This is a proposal to overhaul Moonbeam’s fee mechanism to adopt an algorithmic, congestion-based fee multiplier in the same spirit as EIP-1559. Instead of adopting the exact EIP-1559 algorithm, we use pallet-transaction-payment’s algorithm and translate it to an equivalent EIP-1559 base-fee for Ethereum transactions while carefully tuning the algorithm’s parameters to ensure that it remains feasible for all meaningful transactions.

Abstract

Moonbeam strives to bring full Ethereum compatibility to the Polkadot ecosystem. This means developing with a preference for strict compatibility but thoughtfully deviating when it makes sense. Our fee structure, especially as it relates to EIP-1559, is such a case where deviation is preferred.

Our fee mechanism is inherently a duality. We have the obvious goal of supporting Ethereum transactions and must live with its “upstream” design choices, but we are also a Substrate-based chain and inherit functionality from it which forces us to support a different transaction type. We call these two transaction types Substrate-based txns and Ethereum-based txns.

I have been working on a dynamic fee mechanism which supports both of these in a unified way for some time now and will describe its design and trade-offs here.

Where are we now?

Currently, Moonbeam supports Ethereum-based txns, including EIP-1559, EIP-2930, and legacy types. However, as I will explain later, we do not use the EIP-1559 dynamic base_fee. Instead, we ignore the specified gas price and use a fixed price of 1 gwei (Moonbase, Moonriver) and 100 gwei (Moonbeam). In addition, we consume any applicable priority fee (tip) but it is not paid to block authors.

We also support a rather “vanilla” pallet-base-fee implementation for Substrate-based txns, which includes a dynamic fee (although it doesn’t in any way impact Ethereum-based txns – something this proposal attempts to fix).

Why Not Just Use EIP-1559?

First, a quick refresher. EIP-1559 has a lot to it, but we’ll only focus on a few parts and leave most of it out of scope. Specifically, we care about:

  • base_fee: A strictly enforced fee charged for every transaction included in a block. An optional tip is supported, but otherwise this is exactly the fee that every txn in a block must pay. This is the “dynamic fee” in an EIP-1559 implementation.
  • gas_limit: The max gas that can be used in a block. In the EIP, block authors may adjust this up or down by a small amount (at their discretion) in each block.
  • gas_target: Defined as 50% of gas_limit. This is the “congestion” target that base_fee responds to. When a block uses more gas than gas_target the base_fee increases and vice-versa.

Early on, supporting EIP-1559 was a goal. However, we knew immediately that we couldn’t implement 100% of the EIP. Specifically, we can’t fully implement the adjustable block gas_limit because Polkadot’s validator protocol would reject blocks if they are too big. This mechanism is directly related to the block gas_target in the EIP, so this also becomes questionable.

So what about the specific block base_fee adjustment? Well, this is closely related to the block gas_target, but perhaps we could use both a fixed gas_target and gas_limit and otherwise use the base_fee adjustment as prescribed.

In fact, this is actually what we started to do. The pallet-base-fee in frontier is exactly this. During our initial EIP-1559 support work, my colleague Telmo wrote a base-fee implementation which used fixed values for gas_target and gas_limit. But we were never comfortable enough with it to enable it, so it has remained disabled.

EIP-1559’s Feedback Loop

I like to think of the EIP’s design as a negative feedback loop whereby high txn congestion results in a dampening effect through increased pricing and vice versa:

This brings me to the rationale I have for not using EIP-1559:

  • This feedback loop relies on a constant supply of transactions (or think of it as demand for block space). It is my opinion that EIP-1559 is inappropriate for any chain which cannot meet this requirement as the algorithm will not operate in equilibrium. Moonbeam’s networks have experienced periods of brief, intense congestion and periods of prolonged low congestion, which would not produce good results with this algorithm.
  • The feedback loop is nuanced by adjustable gas_target and gas_limit, which we cannot implement in the same way. So we do not benefit from these mechanisms.
  • There are no guardrails built into this, so it becomes critical to tune things correctly otherwise base_fee would drift up or down over time towards either 0 or infinity.

Substrate’s pallet-transaction-payment

An alternative that was discussed early on was to use the same dynamic fee mechanism that our Substrate-based transactions have been using this whole time. It turns out that it works similarly, at least at a high level. Congestion up -> price up, congestion down -> price down. It also has some compelling benefits:

  • It was built for Substrate-based chains, and makes the general assumption that block space is fixed.
  • It includes a lower and upper bound on its fee multiplier, so we don’t have to worry about txns becoming either free or impossibly expensive.
  • The algorithm for adjusting fee from block to block is highly tunable, so we can tweak it based on real-world behavior.

This is the solution I am proposing: essentially we use pallet-transaction-payment’s algorithm and adjust its congestion-based multiplier to represent an equivalent EIP-1559 base-fee. From here we do the “normal” base-fee things with the caveat that the base-fee algorithm does not follow strict Ethereum consensus rules.

Brief Comparison of Transaction and Fee Structures

However, translating this fee mechanism for use with Ethereum-based transactions raises some concerns. Fundamentally, Substrate-based txns and Ethereum-based txns have vastly different fee mechanisms. pallet-transaction-payment’s fee multiplier can’t simply be converted to a base-fee without making some trade-offs. The fee structure is a bit out of scope, so I’ll describe the differences only briefly.

Ethereum’s fee mechanism includes concepts of base fee (21000 gas) and a length fee (input data is charged per-byte) and computation (the various costs per opcode). However, they are all lumped into the same category (“gas”) and then scaled by the base-fee multiplier to arrive at a final fee.

Substrate includes these same concepts, but each scales differently. Note that I am referring to Moonbeam’s current configuration; this will be different for other blockchains which use this pallet. Roughly, these are how things are scaled:

  • base-fee is a constant
  • length-fee is exponential (cheap for most txns, expensive for extremely large ones)
  • computation is scaled by the multiplier

As you can imagine, converting this multiplier to a base fee is problematic because it impacts the two transaction types in different ways. In general, it will have a bigger impact on Ethereum fees than it will on Substrate ones.

Finding a Suitable Range

In order to make this work, we needed to understand what range, if any, we could use such that the multiplier would work reasonably well for both types of transactions. This was no easy feat, as the different parts of the fee created a sort of multi-dimensional space to consider. It was easy to tune for a specific txn “workload” (think: balance transfer) but this wouldn’t work well for different scenarios (such as an Ethereum create txn or a very large Substrate runtime-upgrade extrinsic).

My colleague Nish did some excellent analysis of this problem by simulating the algorithm’s change over different ranges with different tuning parameters and then applying it to some specific transactions to compare its effects. This resulted in an ability to graph the multiplier across its range against theoretical transactions, allowing us to tune it so that the extremes were tenable.

In the example below, you can see a graph of comparable transactions (green/red: Substrate, blue/pink: Ethereum) graphed over a series of blocks where congestion is simulated by spamming junk txns (yellow: block congestion). In other words, we graph the cost of an Ethereum-based txn and a Substrate-based txn (which we think should be similar in fees) vs. the effect of the dynamic fee multiplier. As the congestion affects the multiplier, you see that it increases the cost of fees and causes both transaction types to converge in cost. This work helped us identify a usable range for setting upper/lower bounds in pallet-transaction-payment. You can also see the dynamic fees PR for some more discussion on this analysis.

Next steps

A snapshot voting session has been opened for 7 days here :
https://snapshot.org/#/moonbeam-foundation.eth/proposal/0x2ea736f08bdfc8a66935910b43afdf02486276de0e610f650711f9c51d2a9cb3

If the snapshot vote passes, this idea will be implemented and proposed to be included in one of the following Runtime Upgrade.

If you have any questions or comments, feel free to reply in the comment section below.

3 Likes

Hey, Stephen, first of all, many thanks to the devs for your work that you are doing. devs often stay behind the scenes, and I am sure that our entire community expresses its gratitude to you for your work!

i enjoyed reading your proposal, the only thing I think is that it will be a little difficult for an ordinary user to understand, and could you make a small conclusion in simple words, explaining what you propose to change?

as far as I understood, the EIP-1559 fee model is not particularly suitable for EVM-compatible blockchains built on a substrate, because in such blockchains there are two types of transactions based on the substrate and ethereum, and when we use the EIP-1559 model, transactions on the substrate side are calculated differently, and also, with high congestion, this does not work quite correctly on the Ethereum side. (correct me if I misunderstood)

Hi Turrizt,

I would summarize at a high level like this:

EIP-1559 requires a constant demand for block space in order to work in equilibrium. Without this, the algorithm will not produce good results; with long periods of low activity and short periods of high activity, it will still drift towards 0 over time, and this is the reason we have so far steered away from it. In addition, there are other related parts of the algorithm we can’t implement which also reduce its effectiveness.

The challenge of translating one multiplier between our fee mechanisms is a mostly unrelated problem, but it’s important to address it in whatever solution we come up with. Otherwise we would have two very independent and disconnected fee mechanisms. This won’t be perfect without much larger changes, but it can be a lot better than it currently is.

We have explored alternatives (such as a highly modified EIP-1559, pallet-transaction-payment as mentioned here, and even entirely home-grown solutions). I’m proposing this solution because I think it’s the best solution for now: it gives us something that is effective without a lot of risk and without a tremendous amount of effort. I also think it will let us learn a bit more about the problem, and I think we can make improvements down the road.

Thanks for taking the time to consider this. I know it’s complicated and there are a lot of details, so feel free to follow up with more questions :slight_smile:

2 Likes

Thank you so much for your comprehensive answer, Stephen!
It turns out that with this change, you expect that the gas fee mechanism will work more stably at high or low activity levels. In fact, I fully support your proposal! :pray:

btw, here is the link to Snapshot & Vote for how much value you feel this would add to the ecosystem: Snapshot

1 Like

Hey @notlesh thanks u for do this

I understand that you want to unify the current fee mechanism, which is divided into two, which produces a larger block space demand, to maintain both mechanisms and which go in different directions

But in simple terms, will the benefit of changing the model be less use of block space? more stable fees for both types of tx, ethereum based and substrate based? or ??

Hi @jrafaelangarita,

I could have been more clear about the benefits of this.

In plain terms, the benefits include:

  • Sensible fees (will not drift too high or low)
  • Consistent fees between Substrate-based and Ethereum-based txns
  • Responsive fees (should respond well to block congestion)
3 Likes

Thanks for the answer, this will help users have a better overview of the proposal when voting

This is a case where it is quite difficult to figure out on your own. And probably competent for the discussion here are mostly the developers of Moonbeam.

But the general point is clear. Take the best and tweak it to suit you. I guess it should sound something like this.
I don’t see anything here that could threaten the network. And we can always get the data after the update to take further steps, right?

So, with you my bow… oh, I mean, my vote )

2 Likes

Hey everyone,

Tomorrow (December 13) at 2pm UTC Yann Isola, Purestake Product Lead, will be explaining the details of this proposal. He will go over the benefits it has to offer and answer questions from the community.

You are welcome to join:

5 Likes
2 Likes

Can you explain the sentence “Responsive fees” exactly?

Can you explain the sentence “Responsive fees” exactly?

Good question.

At a high level, this refers to the goal of making fees responsive to congestion. When there is too much demand for block space, fees should increase and when there is very little demand fees should decrease.

To be more concrete: fees are currently not responsive for Ethereum-based txns because we left the fixed-gas-price placeholder in place during our EIP-1559 development process. On the other hand, our Substrate-based fees are not responsive because the pallet-transaction-payment algorithm is tuned to adapt slowly and “smooth out” fluctuations over a long period of time.

This proposal fixes these two issues by (1) combining the two mechanisms so that they both share the same logic and (2) tuning it to be more sensitive to short-term changes.

1 Like