Less magic in FRAME, good or bad?

Tech Talk
1

I am opening this as a point of discussion to both inform, and talk about some of the upcoming Pull Requests that are in the process of being reviewed in the FRAME department.

The general sentiment of where we are moving toward is: Less magic, more explicit. We have come to this conclusion mainly though the Polkadot Blockchain Academy, realizing that it is extremely difficult to teach parts of FRAME that are being “magically generated” under the hood to new developers. I am personally worried that such difficulties are deterring a lot of developers away (despite reports such as Electric Capital indicating good numbers)

Obviously, education is just one side. Assuming you pass the initial hurdle, having some of this macro-magic will save you some keystrokes.

I personally prefer more explicit code and don’t mind writing a few extra characters here and there. I think the net benefit that having a more explicit FRAME will bring to the ecosystem will easily trump the slight verbosity that comes with it.

Some lines of work related to this:

Importantly, we are making sure that only within the scope of the newly introduced #[pallet::dev_mode], certain details are hidden away, for example this.

What do you think about this, and what do you prefer?

15 Likes
3

I am definitely in favor of more flexibility, sometimes the very nature of building an L1 conflicts with some of the assumptions made by FRAME, being able to customize everything (where it does make sense) is a must if we are to make full use of our chains and make them behave efficiently for their intended use cases.

4

As substrate ecosystem grows we actually need even ways to customize the end resulting structs and enums inside construct macros that will eventually be used in executive module.

The shifting paradigm in blockchain needs to be reflected on the tools used for building. The main issue is for newer devs.

But if we can have toggling experience to allow full control or macro help it will be useful and even for understanding whats going and not having too many black holes

1 Like
5

I’m personally not a runtime developer, but I have at this point 15-20 years of experience in software engineering.
If I were to write my own chain, because of its magic-ness I would personally not use FRAME at all but instead directly what is in sp_io. To me the magic-ness completely counter-balances any benefit that FRAME could provide.

The biggest challenge in software engineering nowadays (and probably for the foreseeable future) is “API complexity”. No matter what you do, your number one priority is to keep your system “mathematically simple”. As few intersecting concepts as possible, and as few corner cases as possible. Performance, elegance, code reuse, etc. should all be second. Simplicity should be in your mind 100% of the time when writing code. Don’t even think about anything else.

Magic-ness is generally the consequence of monkey-patching complexity. When something is too complex, and you decide to hide this complexity under the rug, you get magic-ness. It’s a non-solution that lasts only until people realize what you did.
It’s especially problematic in the blockchain world, where the entire point of blockchains is to be transparent and to make it possible for users of a chain to audit its source code. If your code is too obscure or too complex to be evaluable, why use a blockchain at all?

12 Likes
6

FRAME is complex, partly because of its heavy use of “behind the scenes” code and some of the most complex rust paradigms. I’d personally prefer to understand what my code is doing under the hood (and afford myself more flexibility) at the expenses of having to spend some more time writing code

7

I am not at all against the use of macros to abstract away boilerplate code, I don’t think it has to be either 0 or 1, there can be an in-between.
My idea of a perfect compromise is having complete granularity over what to auto-generate and what to leave behind for me to write manually. The abstractions are not all bad, they help reduce development time by a lot, I would like to continue using them, however I still need to modify some specific things that are currently being auto-generated.

2 Likes
8

This sentence hits an interesting point that I want to emphasize.

There is two types of complexity, native to language (what you called “complex rust paradigms”), and hiding it behind a lot of code generated in a macro (what you called “behind the scenes code”).

Two nice examples of the former are:

  1. a trait that has been mostly auto-implemented, and you finish it by providing the rest of the the items.
  2. By implementing a single trait on your type, you get access to lot more functionality by a lot of blanket implementations provided by FRAME.

These are good abstractions. If you find these hard, or want less of them, I would (opinionatedly) say that FRAME and Rust is not the right tool for you.

But the latter type, the bad abstractions, are those that generate too much code that you cannot even see behind the curtain. The old decl_xxx macros were the best example of this anti-pattern.

Even if there is going to be code auto-generated, the macro syntax should resemble real Rust abstraction sufficiently to make the code understandable.

Derive macros are an example of this. They do generate some code under the rug (as @tomaka phrased it), but are they bad? I don’t think so.

The best example here, in my opinion, is the existing construct_runtime. See this suggestion of how it can be simplified, and made easier to reason about, compared to the existing one. It is still a macro, it still generates code, but, under a more transparent rug, so to speak.

1 Like
9

I don’t want to intervene too much, because I’m not a runtime developer, but we don’t seem to be talking about the same complexity.

To me, the biggest issue for example is how the execution model is completely opaque.
In imperative programming languages, you have a “main” function containing a list of instructions, and these instructions are executed one after the other. It’s simple to understand: first instruction A is executed, then instruction B, then instruction C, etc.

If you look at runtime source code, however, you have functions here and there, and it’s completely unclear without being deeply familiar with Substrate when or how often they are called. When looking at runtime code, it seems that you’re injecting small chunks code into a hugely complex machinery, and this machinery calls your chunks here and there.

For example, the fact that the source code says type WeightInfo = WeightInfo; is very readable. It looks like Rust syntax. It’s a simple associated type that most non-beginner Rust developers are familiar with. And no code is automatically generated under the hood.
But understanding what this type WeightInfo = “actually” means is extremely complicated and requires deep Substrate knowledge.

That’s what the I have in mind issue is. type WeightInfo is “magic” because the machinery around your runtime uses these weightinfo “somehow”, and it is completely invisible how if you only read the runtime’s source code.

2 Likes
10

Your comment on execution flow is another separate facade in my opinion, and the discussion we are having here does apply to that as well. I think this goes back to our education approach, namely in Substrate docs and venues like PBA.

The runtime is at the end of the day is more of a library than binary. It has a bunch of functions exported as WASM, all within impl_runtime_apis! {}. There’s no main, and if you want to find the execution flow of anything, you must start there. Most of the execution flows in there start with frame-executive, which is one of the least educated/documented parts of the codebase.

The previous sentence is an important assertion that is not educated enough ^^

I am personally a big fan of talking about executive, and how it glues almost everything together, to the extent that I vouched for teaching FRAME entirely bottom up in the first PBA, which admittedly was a bit too much. But we still cover it in PBA towards the end, and we will add this piece of knowledge to the Substrate docs as well. I hope this addresses this concern to a high extent.

This is an accurate description, but also to a high extent the definition of what a “framework” (as opposed to a “library”) is.

End of the day, FRAME is a framework. Ergo, FRAME will call your code, not the other way around.

Final side note, while I generally agree with “…seems that you’re injecting small chunks code into a hugely complex machinery…”, I think WeightInfo is not a good example. Looking at where type WeightInfo is being fulfilled and used within eg. balances pallet, I think it is fairly easy to follow and entirely within the boundary of normal Rust traits and types, and my definition of FRAME being a framework above.

Perhaps the fact that the process to generate the weight files is complex and opaque is the main reason behind your comment?

1 Like
11

That is not really true. From the pov of FRAME execute_block of frame-executive is your main. It takes the block and executes it. But maybe a runtime should be more seen as some kind of library that you can talk to and for that it offers certain functions to call.

I think this is completely valid thinking. However, FRAME is also not just a collection of macros. FRAME is an highly opinionated DSL. If you don’t like it, you can write you are open to write your runtime in whatever way you like. I also think that at some point someone will come up with a more simple approach to write runtimes. This will maybe only be used for simple runtimes, but there will clearly be users. It always depends on what you want to achieve. Otherwise you could also start asking if you need SQL or why you don’t query the database using its c interface? The reason being that you want to abstract complexity from the user. But yeah, FRAME will probably also never be as simple as SQL because it supports too many use cases etc.

In general, I think the idea of @kianenigma to make FRAME more approachable and easier to understand is the right way. If you need to make it more look like plain Rust or if you just accept that FRAME is its own DSL, but this DSL should be consistent in itself.

FRAME will also never support all the use cases or pallets will be written in a way that do not work for every body. However, for that you can just fork a pallet and rewrite in the way you like :slight_smile:

1 Like
12

I don’t really understand this.
Do you believe that someone can read the Rust book about traits, then read the code you linked and understand it easily?

Do you think that someone who is only familiar with the traits in the standard library (PartialEq, Hash, etc.) can easily understand the kind of meta-programming that FRAME does?

To me it’s like learning what C++ templates are, then reading this kind of code.

To me there’s obviously a gap, and if you don’t realize it I think that you might be biased by the years that you’ve spent writing runtime code.

That seems like a pretty arbitrary self-imposed constraint.
When a framework calls your code instead of the other way around, it makes the code flow more difficult to understand, and makes things more complicated. That’s not really me inventing it, it was already true decades ago. I don’t think it would be a good argument to not look into simplifying this aspect just because “FRAME is a framework”.

2 Likes
14

I’m not in any way saying FRAME abstractions (MACROS, extensive use of traits etc) are bad.

I mentioned these because I think how we portray FRAME to devs do matter: are we portraying it more as a framework? or a DSL? I personally think how we objectively portray FRAME will influence how devs approach and work with it.

15

I want to chase you down further with this, not for the sake of the argument but more so because I am involved in a lot of education programs such as PBA and I want to improve my ability to detect what is hard to understand for others and what is not, or as you put it, prevent:

To me there’s obviously a gap, and if you don’t realize it I think that you might be biased by the years that you’ve spent writing runtime code.

I put together this gist to demonstrate the entire call path of a WeightInfo (also, so that we continue discussing it there to not derail this topic for one nitty gritty example), and I still think it is not a good example for “this is too complex”, because it can actually be easily followed by just knowing the Rust book section about traits. Comparing it to the C++ template example is a bit of an exaggeration.

I don’t disagree with all that much with FRAME is complex, and it should be simplified, my opinion about that is stated in the first post. But I do think we have different views on which parts are actually too complex.

What I understand from your comment is that you don’t like frameworks and prefer to use libraries, such as sp_x, which admittedly you can already do and write something pretty decent with them with not a whole lot of effrot. We have a frame-less runtime that we use in PBA, I will try and link it here if I find an up to date version of it somewhere public.

One analogous example that comes to my mind are web frameworks. I am not up to date on what’s hot in eg. Rust or Node.JS nowadays, but as far as I remember, they are all by definition frameworks, and wrap your code. Please show me a web framework (or library), that gives you the control flow, and you think it is easy to read? I guess that based on your expectations, no such thing must exist, and you would write an HTTP server from scratch every time :slight_smile:

16

I think as it stands, FRAME is a framework, not a DSL. Perhaps you can argue that the syntax of how to write a pallet is a DSL.

17

Well, this gist can be rewritten as this:

impl<T: Config> Pallet {
  fn foo() {
    let transfer_weight_info = 42;
  }
}

And I think we both agree that this is way more simple.

While I’m not familiar with benchmarking, I suppose that the reason why all this trait machinery exists is related to our “magic benchmarking system”.
But from a pure simplicity standpoint, all these indirections are just unnecessary obstacles that seem to exist just for the sake of existing.

1 Like
18

I have mentioned before that if FRAME had decided to keep the decl_* macro syntax, then I would never have joined Parity or worked on Substrate/Polkadot. This was certainly hyperbole, because I did not know about the existence of the pallet attribute macro syntax when I joined Parity, and that didn’t deter me from working on Substrate.

However, this does illustrate my unwillingness to use the decl_* macro syntax unless it was absolutely necessary. Aside from the fact that it’s magical, one big factor that really puts me off is the new syntax that I had to learn to make the magic happen. While it certainly didn’t drive me off from approaching Substrate, it did make me feel like I need to spend an enormous amount of effort to understand how everything is wired behind the scenes in order to know how I can effectively use it.

“Effectively” here means knowing which kinds of operations are possible, and which aren’t. Since we are relying heavily on macros to generate custom code, it is thus tough to know, at first glance, what things can be modified, and what things, once modified, would break the pallet. It is also difficult to know why things are done in a certain way, and all of this adds up to having the feeling that FRAME is just an entire subsystem that I have to learn very well in order to effectively contribute.

With regards to the ongoing discussion about benchmarking and its related macros, I looked into the macros and what code it is currently generating, and true enough, even the generated code themselves are mystical, in the sense that it is using some sort of abstraction or trait for no apparent reason. I heard too that the back-end code for benchmarking is even worse, so I shudder to think what exactly lies there. I do think however that they are generally not looked into simply because it “Just Works” and nobody so far has bothered to rethink how it should be done correctly. It is definitely time now that we spend some effort to repay all the technical debt that we’ve incurred as part of the development of FRAME and Substrate itself.

4 Likes
19

Jumping in from the outside, assuming that stuff was done complicated for the sake of making it complicated isn’t a constructive way to help. What you outlined there above is just plainly wrong. If it could have been done that easy it would have been done that easy. However, the point here is that you don’t know the weight for your call in all runtimes as it depends on all the configuration options. So, this needs to be generic to plug-in the weight from the outside as shown by @kianenigma. Everybody is running the benchmarking with their own runtime configuration and is possible getting different weights.

20

I think these two thoughts can be used to improve documentations and a readability of a runtime code.
I remember myself looking for something like main function, when first was looking into a runtime code.
Web frameworks is a good analogy, they usually have an index/main file, routes, and dependency injection (DI) container, today our runtimes’ lib.rs looks like one DI container.

1 Like
21

One thing worth noting is it is still quite possible to do code-gen with a non-magical dev UX in the form of either CLI tools or extremely detailed compile-time errors that show you some code you could add/change to resolve the error (rustc already does this in a lot of places in little ways).

So generally my thought is in the places where we find ourselves contemplating something magical to avoid a lot of boilerplate, we should instead think about making the boilerplate obvious (through auto-suggestions suggesting it) or easy to generate (through CLI tools that generate the boilerplate).

Just a thought

3 Likes