Consider JSX-like syntax inside dart code

[cbazza]

It would be great if aside from the current way of building widgets you could add JSX like capabilities. I mean add tiny syntactic sugar to enable XML like constructs inside dart code. It just makes code so much easier to read/develop/debug/maintain and also easier for powerful GUI builders to integrate with editable code.

Looking for something like DSX:
https://spark-heroku-dsx.herokuapp.com/index.html

Carlos.

---

The current issue with DSX is about proper integration with Flutter tools as to provide a great developer experience with debugger, auto-complete, etc. working on .dsx files.

Telling users that they can use DSX but can't use debugger or enjoy auto-complete is a non starter for me. If anybody wants to help, what I need is to figure out a way to add full preprocessing support (with source map) to Dart Tools and VS Code Dart plug in. Once the tools support that DSX or any other transpiling language (any language that is a superset of Dart but compiles everything down to Dart) would just work.

If you can and would like to help, let me know.

[sethladd]

cc @lukechurch

[Hixie]

@cbazza Can you elaborate on why you want this? Maybe show an example of what it would look like compared to today?

[cbazza]

Ok, so the "Basic widgets" example on 'https://flutter.io/widgets-intro/#basic-widgets' would look like the following:

import 'package:flutter/material.dart';

class MyAppBar extends StatelessWidget {
  MyAppBar({this.title});

  // Fields in a Widget subclass are always marked "final".

  final Widget title;

  @override
  Widget build(BuildContext context) {
    let style = {
        height: 56.0, // in logical pixels
        padding: const EdgeInsets.symmetric(horizontal: 8.0),
        decoration: <BoxDecoration color={Colors.blue[500]}/>,
    };
  
    return <Container style={style}>
      <Row>
        <IconButton
            icon={<Icon name={Icons.menu}/>}
            tooltip='Navigation menu'
            onPressed={null}
        />
        <Expanded>
           {title}
	</Expanded>  
        <IconButton
            icon={<Icon name={Icons.search}/>}
            tooltip='Search'
            onPressed={null}
        />
      </Row>
    </Container>;
  }
}

class MyScaffold extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    // Material is a conceptual piece of paper on which the UI appears.
    return <Material>
      <Column>
          <MyAppBar
             title={<Text 
               text='Example title'
               style={Theme.of(context).primaryTextTheme.title},
             />}
          />
          <Expanded>
            <Center>
              <Text text='Hello, world!'/>
            </Center>
          </Expanded>
      </Column>
    </Material>;
  }
}

void main() {
  runApp(<MaterialApp
    title='My app'
    home={<MyScaffold/>}
  />);
}

[Hixie]

How about this syntax?:

import 'package:flutter/material.dart';

class MyAppBar extends StatelessWidget {
  MyAppBar({this.title});

  // Fields in a Widget subclass are always marked "final".

  final Widget title;

  @override
  Widget build(BuildContext context) {
    return Container(
      height: 56.0, // in logical pixels
      padding: EdgeInsets.symmetric(horizontal: 8.0),
      decoration: BoxDecoration(color: Colors.blue[500]),
      child: Row(
        children: <Widget>[
          IconButton(
            icon: Icon(Icons.menu),
            tooltip: 'Navigation menu',
            onPressed: null,
          ),
          Expanded(
            child: title,
          ),
          IconButton(
            icon: Icon(Icons.search),
            tooltip: 'Search',
            onPressed: null,
          ),
        ],
      ),
    );
  }
}

class MyScaffold extends StatelessWidget {
  @override
  Widget build(BuildContext context) {
    // Material is a conceptual piece of paper on which the UI appears.
    return Material(
      child: Column(
        children: <Widget>[
          MyAppBar(
            title: Text(
              'Example title',
              style: Theme.of(context).primaryTextTheme.title,
            ),
          ),
          Expanded(
            child: Center(
              child: Text('Hello, world!'),
            ),
          ),
        ],
      ),
    );
  }
}

void main() {
  runApp(MaterialApp(
    title: 'My app',
    home: MyScaffold(),
  ));
}

[cbazza]

Huumm, a little improvement but not so good...
Here are the things that gets accomplished by using XML:
(1) No more 'child' & 'children' stuff
(2) easy for 3rd party tools to manipulate (parse, analyse and regenerate)
(3) notice that the switching between markup and programming is easily detected. I mean inside XML you have '{}' to delimit code and in code you have '<Capital' to delimit markup.
Also separate all the 'style' things from the main structure.
I know this is basically fully endorsing React's way but you are half way there anyways ;)

[sethladd]

cc @kasperl

[Hixie]

(1) No more 'child' & 'children' stuff

I don't really understand why that's desireable. "child" and "children" aren't special. Consider ListTile for example. How would you do that one? Why are "icon" in IconButton, or "home" in MaterialApp, something you want to give a name for, but not "child" in Expanded? All three are just arbitrary arguments that happen to take Widget objects. There's nothing magical about "child" vs "home".

(2) easy for 3rd party tools to manipulate (parse, analyse and regenerate)

You can parse, analyze, and regenerate Dart code. But I agree we should make that easier. Hopefully in the coming years the Dart team will provide better APIs for this.

(3) notice that the switching between markup and programming is easily detected.

Why is that desireable? I mean, why would any of this count as "programming"? It's all just expressions.

I mean inside XML you have '{}' to delimit code and in code you have '<Capital' to delimit markup.

I don't really understand the distinction.

Also separate all the 'style' things from the main structure.

You can do this today in Flutter if you really want to, just put the style in a variable like you did in the XML case.

[cbazza]

I don't really understand why that's desireable. "child" and "children" aren't special. Consider ListTile for example. How would you do that one? Why are "icon" in IconButton, or "home" in MaterialApp, something you want to give a name for, but not "child" in Expanded? All three are just arbitrary arguments that happen to take Widget objects. There's nothing magical about "child" vs "home".

Less boilerplate, you don't need to say it because it is inherited in the structure.

Why is that desireable? I mean, why would any of this count as "programming"? It's all just expressions.

It's related to (2) because it makes life of toolmakers, specially GUI builders, much easier since they don't need to fully parse Dart; but it also makes reading the code easier.

I don't really understand the distinction.

The format of XML is very simple so when you see '{}' you know it is calculating an expression in dart. Same for the opposite, when reading dart code and you see '<Capital' (a less-than followed by a word that is capitalized. example <Row>) you know that an object hierarchy is being created from XML markup.

[cbazza]

Also in the final XML processor I would avoid passing objects to attributes of parents and instead create child tags as below:

this...
          <MyAppBar>
             <Title style={Theme.of(context).primaryTextTheme.title}>  
                 Example title
             </Title>
          </MyAppBar>

instead of this...
          <MyAppBar
             title={<Text 
               text='Example title'
               style={Theme.of(context).primaryTextTheme.title},
             />}
          />

[Hixie]

Less boilerplate, you don't need to say it because it is inherited in the structure.

But why only for some of the properties? And how do you handle cases where there's two child slots, like ListItem? XML-ish syntax just doesn't seem to handle this very well.

Also I'm not really sure it's less boilerplate.

Compare:

   <Container style={style}>
      <Row>
        <IconButton
            icon={<Icon name={Icons.menu}/>}
            tooltip='Navigation menu'
            onPressed={null}
        />
        <Expanded> {title} </Expanded>  
      </Row>
    </Container>

   Container(style: style,
      child: Row(
        children: [
          IconButton(
            icon: Icon(Icons.menu),
            tooltip: 'Navigation menu',
            onPressed: null,
          ),
          Expanded(child: title),
        ],
      ),
    )

It's not at all clear to me that the XML-ish syntax is cleaner or less boilerplatey. There's lots more punctuation, and some duplication of content (e.g. in the close tags). And you had to add some names, so sure, you lose "child", but you gain "name" on Icon.

Also with XML how do you make it clear that Row can take zero, one, or more than one child, while Center has to have exactly one child? What would happen if someone did this?:

   <Center> <Test/> <Test/> </Center>

It's related to (2) because it makes life of toolmakers, specially GUI builders, much easier since they don't need to fully parse Dart;

They wouldn't need to fully parse Dart if we had a Dart parsing API either, though, right? I mean, you'd parse what you want to parse and leave the rest. Also I'm not sure it's actually easier to parse, since it's not actually XML; see below.

but it also makes reading the code easier.

I'm not at all convinced that the XMLy version here is easier to read. Once you've read a few build functions, you quickly get used to the nested constructor syntax.

The format of XML is very simple so when you see '{}' you know it is calculating an expression in dart.

It's not actually XML, though, right? It's some variant of XML. Are there well-defined parsing rules for it? For example, is this valid?

  <Test name={describe("}")}>

How does it know that the first "}" isn't the end of the attribute expression, without parsing Dart?

Same for the opposite, when reading dart code and you see '<Capital' (a less-than followed by a word that is capitalized. example <Row>) you know that an object hierarchy is being created from XML markup.

You know this today when you see the new keyword too, right? Or indeed in the new-less markup proposal above when you see any capitalised word. Is this really a benefit of XML, or are you more familiar with XML than Dart?

Also in the final XML processor I would avoid passing objects to attributes of parents and instead create child tags as below:

I really don't understand what you're proposing here. It's not well-formed XML at all as far as I can tell. Can you elaborate on exactly what the syntax you are proposing is and how it works? For example, the "Text" constructor seems to have disappeared; how does the processor know that <Title> creates a Text widget?

Sorry if I sound defensive or aggressive. :-) This is a topic that's come up several times but I've never had someone willing to actually argue the case before so I'm finding this conversation very useful in teaching me what the reasoning behind the request is. Please don't take my argumentative tone as dismissive, I'm very interested in your input here.

[cbazza]

Look, you are mixing everything I say and this conversation is going nowhere. In legal terms you are "Badgering the witness".

If you are really interested in learning why JSX is hot, just google for "react tutorial" and notice that for the past 2 years all articles on React use JSX. The original way of creating component hierarchies in React (which is equivalent to the current way in Flutter) is never mentioned again because JSX became the preferred method (best practice).

https://facebook.github.io/react/tutorial/tutorial.html
https://facebook.github.io/react-native/docs/flatlist.html

Another interesting thing is that Typescript has adopted JSX also:
https://www.typescriptlang.org/docs/handbook/jsx.html

You failed to grasp that XML parsing (with some extra code to skip '{}' properly) is orders of magnitude simpler than fully parsing a programming language like Dart. That is a fact. You are assuming that tool builders will use Dart on their development, not the case, Intellij most likely is using Kotlin and Java on their IDEs that supports Dart (they are a special case because they specialize in language parsing; everybody else will struggle to fully parse Dart from another language).

What I like about putting XML inside another language is that it provides a cognitive separation between the two because XML is very distinct from programming languages. Just scrolling through the source file you can easily see what is code and what is declarative markup. Can't accomplish that with dart code pretending to be markup.

Stop nit-picking things that are not fully specified. All your doubts have answered for it just learn more about how that was handled in JSX. I just don't have the time for this here.

[Hixie]

My apologies if I sound defensive or aggressive. This is a topic that's come up several times but I'd never had someone willing to actually argue the case before so I was finding this conversation very useful in teaching me what the reasoning behind the request was. Please don't take my argumentative tone as dismissive, I'm very interested in your input here.

Please don't feel you have to reply. I'm commenting here so that there is transparency regarding the issues that we would have to resolve before we're able to make a decision or design one way or the other. This is basically just a Socratic dialogue. Your participation is welcome but certainly you should not feel that it is your responsibility to defend your proposal.

---

I've no doubt that JSX is "hot". In React, the non-JSX syntax is a lot worse than the alternative syntax proposed in this issue (the one that looks like our current code but without the "new" and "const" keywords). What I'm trying to understand is whether the same reasons that JSX is "hot" in React apply to Flutter.

Regarding TypeScript doing JSX, In 2012 I was involved in efforts to specify E4H, and even before that there was E4X. Both efforts died. So it's important to me that we understand what exactly it is people like about JSX vs other syntaxes.

Parsing XML isn't easy, parsing sort-of-XML-but-with-curly-braces-somehow is not easy either. Parsing sort-of-XML-but-with-curly-braces-somehow-that-is-embedded-in-another-language is even less easy. However, how easy that is to implement probably isn't a big deal because it's going to be implemented once or twice and then the library that does it will be reused. (I've been heavily involved in writing the specs for parsing HTML and been involved in similar work for XML, and I've implemented a parser for Dart, so I have a pretty good idea of how difficult parsing markup languages vs programming languages actually is.)

What I like about putting XML inside another language is that it provides a cognitive separation between the two because XML is very distinct from programming languages. Just scrolling through the source file you can easily see what is code and what is declarative markup. Can't accomplish that with dart code pretending to be markup.

But why is it beneficial to be able to do that?

It's pretty obvious when scrolling through Flutter apps where the build functions are (they're the giant nested expressions). What is it about "declarative markup" that is important to separate from "code"?

Stop nit-picking things that are not fully specified. All your doubts have answered for it just learn more about how that was handled in JSX. I just don't have the time for this here.

As far as I can tell, JSX doesn't handle the things I was asking about. For example, React doesn't have the concept of child slots. The closest thing I could find is something they do by switching back to JS then back to JSX inside that, so you'd need to be able to parse both the programming language and the markup language.

[cbazza]

What I'm trying to understand is whether the same reasons that JSX is "hot" in React apply to Flutter.

Yes, the exact same thing applies here. The current way looks good to you because that's the only option you have. Give people a second option and the same will happen.

Whether E4X died or not is irrelevant because nothing lives forever. I have used ActionScript with E4X a lot and thought that Adobe did an excellent job there. In a way Flutter is just a newer version of Adobe Flash with Flex apps.

(I've been heavily involved in writing the specs for parsing HTML and been involved in similar work for XML, and I've implemented a parser for Dart, so I have a pretty good idea of how difficult parsing markup languages vs programming languages actually is.)

Great so you know that parsing a markup language is trivial compared to parsing an imperative programming language.

But why is it beneficial to be able to do that?

Code readability and simplicity which in turn drives a whole bunch of other benefits.

It's pretty obvious when scrolling through Flutter apps where the build functions are (they're the giant nested expressions). What is it about "declarative markup" that is important to separate from "code"?

On your giant nested expressions can you easily see structure? can this structure be easily manipulated by other tools like GUI builders interchangebly ? I mean like Adobe Flex Builder use to do, drag and drop widgets around, wire them on UI and then switch to code view and just edit anything you want and then switch back to gui mode and continue to manipulate the widgets. You can't do that easily when the programmer goes inside your "giant nested expressions" and writes any valid dart code that doesn't follow the structure that the GUI editor is expecting. With a fixed XML structure that is not a problem.

As far as I can tell, JSX doesn't handle the things I was asking about. For example, React doesn't have the concept of child slots

It handles it just fine, you just don't know how to do it. So going forward just put the example in question here and I will provide you with what I think the JSX version should be.

  new ListTile(
    title: new Text('CineArts at the Empire',
        style: new TextStyle(fontWeight: FontWeight.w500, fontSize: 20.0)),
    subtitle: new Text('85 W Portal Ave'),
    leading: new Icon(
      Icons.theaters,
      color: Colors.blue[500],
    ),
  ),

  <ListTile>
    <title> 
      <Text style={{fontWeight: FontWeight.w500, fontSize: 20.0}}>
         CineArts at the Empire
      </Text>
    </title>
    <subtitle>
      <Text>85 W Portal Ave</Text>
    </subtitle>
    <leading>
      <Icon data={Icons.theaters} color={Colors.blue[500]}/>
    </leading>
  </ListTile>,

It looks longer than the dart version but I could had placed everything in the same number of lines. Thing is an IDE/Editor can provide '+' & '-' to expand and collapse these XML nodes anyways.

Make Flutter look familiar to React developers and you have a chance of attracting a bunch of new users to Flutter.

[Hixie]

Whether E4X died or not is irrelevant because nothing lives forever.

Whether it died isn't the issue, it's why it died. Did it die because it didn't provide a solution that people wanted? Did it die because of implementation issues? Did it die because of patent issues? Was it too early? Too late? I think it's important to learn lessons from past experiences. Why did E4X and E4H fail where JSX has succeeded?

What I find interesting is that people who are new to Flutter often ask for two things: a markup language, and animated GIFs. Then three months in, they are still asking for animated GIFs, but not for a markup language. It could be that this is because the markup language isn't actually needed once you're used to writing build methods in Dart. It could be that they still want a markup language but have worked around the omission and so don't think to ask anymore. It's worth figuring out which because otherwise we risk spending effort on something that is the wrong choice (in either direction).

On your giant nested expressions can you easily see structure?

Yes, at least as easily as with XML. Personally, I find XML to be very verbose and it obfuscates the structure. But I think this is more about what you're used to.

(We're also experimenting with IDEs that put in virtual "closing tag" comments for you so you can see the structure without having to actually write it.)

Great so you know that parsing a markup language is trivial compared to parsing an imperative programming language.

My experience is that declarative vs imperative is not the distinction that matters when it comes to determining how easy a language is to parse. (For example, HTML is "declarative" but it may be among the most complicated languages to parse that I've ever dealt with.)

Code readability and simplicity which in turn drives a whole bunch of other benefits.

If this is the main benefit then this is something we can test. We could take a mixture of engineers who are used to writing HTML, React, iOS code, Android code, Flutter, command-line apps, and so on, and present them each with various syntaxes, and ask them to describe what they think the resulting UI would look like. We can then measure which syntax gets the best results. @InMatrix is this something we could look at after the animation work wraps up, maybe?

can this structure be easily manipulated by other tools like GUI builders interchangebly ?

Yes, in principle at least. It should be relatively straight-forward to mechanically convert Dart expressions to XML or JSON or whatever other structured language one might use. It's just a matter of converting the syntax, the actual information is the same. Personally I wouldn't convert it to a different syntax if I was making a GUI editor, I would just parse it into a data structure in memory straight from Dart.

You can't do that easily when the programmer goes inside your "giant nested expressions" and writes any valid dart code that doesn't follow the structure that the GUI editor is expecting. With a fixed XML structure that is not a problem.

The thing is, you can put exactly the same "any valid dart code" in the XML structure as in the Dart expression. They are literally mechanically interchangeable. So I don't really see how using XML helps with this particularly. For example how would you turn this into XML?:

new ListView.builder(
  padding: new EdgeInsets.all(8.0),
  itemExtent: 20.0,
  itemBuilder: (BuildContext context, int index) {
    return new Text('entry $index');
  },
)

It handles it just fine, you just don't know how to do it.

I meant specifically JSX. I agree that your proposed syntax would be a perfectly valid way to approach the problem.

[HansMuller]

I worked on Adobe's Flex SDK, which combined XML markup and ActionScript, for the last couple years that the product existed at Adobe. I understand the appeal of markup for defining UIs however I can also remember some drawbacks:

• Flex application visuals could be defined in terms of markup and code. As I remember it, code tended to dominate in large real-world apps. Readability isn't necessarily a benefit for apps defined as a complex hybrids of markup and code.
• The "Flex Builder" IDE had to support apps that were defined by markup and code. This made the IDE difficult to build and maintain. Developers tended to view it as an ActionScript tool.
• Evolving and maintaining the XML "compiler" was a significant burden that kept a team of engineers busy full-time. Keeping the compiler and toolkit in lockstep slowed down the evolution of the overall sdk.

It's been years and I can no longer recall all the details. However my overall impression is that defining UIs with a combination of markup and code is a mixed bag at best.