Summary: Abnormal behavior is causing significant performance degradation to the network. I propose expediting the deployment of RT 2302 to Moonriver by using the OpenGov whitelist track to ensure transactions on the network are properly priced based on how much gas is being used. This is a measure to slow the transactions that are clogging the network and increasing the size of the chain state.
(See Original MR Referendum 8 post for RT 2302 here.)
Recently, community members have observed some abnormal behavior on the Moonbeam Network including block saturation, rapid increase in storage size, degraded collator performance and delays in transactions being accepted into blocks.
Upon further analysis, increased usage originating from the Xen crypto project appears to be a significant contributor. The project uses a “proof of participation” concept for minting its XEN token, but because Moonbeam currently has a flat base fee (100 Gwei per transaction, regardless of network utilization); these transactions are not getting properly priced. On March 23rd, the volume of transactions originating from XEN significantly increased with the launch of XENFT ERC-721s; which allow the automation of creating XEN ERC-20 tokens in bulk.
The “proof of participation” mechanism is generating a lot of storage and saturating the network with tiny smart contract creations whose sole use is proving the participation of the user. What’s more, users will create new addresses in order to participate repeatedly without having to wait for the “waiting period” to pass. As a result, it is significantly increasing the size of the chain state. Prior to XEN stepping up its efforts, the chain state was growing by about 1Mb per day. Currently it’s growing at roughly 500Mb every single day. The Moonbeam chain state is currently sitting at 16GB in size.
This will rapidly start to create significant performance degradation - we’re already seeing block saturation and if this continues, synchronizing the chain will take weeks, producing blocks will be slower, fees will increase and it will take a long time for transactions to make it into a block, which is obviously bad for the entire ecosystem.
I want to be clear that I’m not offering an opinion on the merits of the philosophy behind the XEN project, XENFTs or their utility. But as it currently stands, without properly pricing these transactions, it presents a serious stability issue for the chain.
As it happens, before becoming aware of this issue, the Moonbeam dev team had already been working on Runtime 2302 (in fact, the deployment to Moonriver is currently in democracy), which would enable dynamic fees - this would ensure that these transactions would be properly priced based on how much gas is being used; and the chain storage can’t be so easily grown at an ultra low cost.
As a concerned community member and given the urgency of the situation, I am proposing that we accelerate the deployment of RT 2302 to Moonriver by using the OpenGov whitelist track to have it deployed as soon as possible. Then, it is proposed to fast track the deployment to Moonbeam, targeting a deployment after 1 day of being live in Moonbeam.
In terms of the storage problem itself, the dev team will need to do some additional work post RT 2302 to prevent abuse.
Moonbeam core developers continue to investigate other solutions to solve the underlying storage issue (which will only be partially mitigated by the dynamic fees). Any solutions would likely need to be made in a future runtime upgrade.
Here are some charts to put this into perspective: