[META] Start using reactive declarative JS programming for some new core admin UIs

Special thanks to the following experts who provided review and input:
Miško Hevery (creator of Angular; Google)
Igor Minar (technical lead for Angular; Google)
Ed Faulkner (core maintainer for Ember)

Must choose between these two. It seems other projects main developers are rejected / do not willing / no way to contact them to share their thoughts ??

Lesson Learned myself, one of them retired a popular project so quick.
https://github.com/facebookarchive/three20

the matrix missing vuejs, one of top 5 most starred in github I think.
https://github.com/vuejs/vue

The matrix also misses PureScript, ClosureScript, Cycle, Riot and many more. For various reasons. I am sure that it is open for discussion though and not carved in stone. This is why this issue as created. To encourage further discussion, review and work on this topic.

Must choose between these two.

No. One of the questions in the issue summary is explicitly "Are there other frameworks that meet our minimum criteria?".

It seems other projects main developers are rejected / do not willing / no way to contact them to share their thoughts ??

This is now a public issue, so we'd love other projects' experts to comment here. Those were just the ones we were able to reach and who had availability to work with us on the matrix in late December, during the holidays. The current state of that matrix is just a start though. This issue is about continuing it, in public.

Another requirement: No matter what Javascript-based fancying up we do, the page must still be usable without JS. That's been an expectation of Drupal for a long long time, even for admin pages, and it is one we should keep. Build an HTML/CSS-only admin page first, then add JS incrementally to make it whizz-bang. I am totally down for adding JS to make the admin more whiz-bang, but for many reasons the underlying "series of pages linked together" architecture that is the Web must be maintained, even for administrative pages.

Since this is an open discussion can we have more info about (from Dries' post):

… I believe that Drupal needs to move quickly to embrace a framework…

And here:

We need to accelerate the pace of these improvements to match users’ growing expectations.

What's the timeline Dries has in mind? Because this does not sounds like a 'it's ready when it's ready' deal and more of a 'it's ready when we make it ready'. If we want to timebox this whole discussion that's fine with me but it needs to be very explicit from the start.

( edit ) +1 to what Crell said. Though I see the "server rendering" justification coming :)

My other concern is that this is a very volatile part of the industry. We expect Drupal 8 core to live for several years; I expect we're 3-4 years away from seeing a 9.0 release, which means 8.x will be our "latest" for that whole time. However, Javascript frameworks at this point have a lifespan measured in months; a year is an old one. :-) Just as Backbone is now passe in the eyes of many, I expect every library listed above, including whichever one "wins", to be passe within 24 months at most. And we'll be right back here again. The client side moves almost dangerously fast right now. And that's not even taking into account the often ridiculous build toolchains needed.

I'm not much of a JS dev at this point so I have no particular favorite, but I worry about tying ourselves too closely to any of these tools.

Side note: Twig.js is a thing, although I don't know if it's still viable for us given how much we've done to extent server-side Twig (render arrays, etc.)

(responding to #8) I'm not as sure. When we chose jQuery, there were many other frameworks out there including Prototype, Mootools, Dojo, etc. We evaluated against some specific criteria—wanting something modular that would work well with Drupal, wanting something with a large, active community around it and a responsive maintainer—and that choice seems to have served us pretty well. I don't hear about any of those other frameworks anymore.

And even though the VCS landscape was far less volatile, at the time we selected Git there was at least one other very strong contender, which was Bazaar, development on which has since stalled. So obviously we made the right choice there, as well.

So to me, it's about picking the right criteria, talking a lot to people in the know, and gauging the community response. We've done pretty well with that, historically.

#9: To some extent, yes. However, to some extent Drupal's selection of jQuery helped it win (not entirely, sure, but it was a factor), and we were then always stuck on old versions of jQuery. :-) As for Git, there were other contenders but it seemed fairly clear that the "market" was trending toward Git in a hard way already. It wasn't anywhere near as volatile as the JS framework market is today.

As counterpoints, we picked Bootstrap (which we're now talking about replacing with something else) when it was a stable and mature leading tool. I don't think I'd say we "got that wrong", just that it went out of vogue long before jQuery did.

Also, we originally went with create.js for QuickEdit. We spent a lot of time trying to make that work, then had to pull out and roll our own atop CKEditor, which wasn't in a usable state when we initially started. There was a lot of flapping and churn in that process as we changed our minds about what tool we should be using. (I'm not saying we "got it wrong", more that the definition of "right" was a lot more fuzzy and shifting.)

I'm less concerned about us "picking wrong" than I am "none of these options will have the longevity we want". Even the popular darling Angular 1 is now dying already, replaced by, potentially, Angular 2 (different enough to share little but the name) and React. That doesn't mean Angular 1 would have been a "wrong" choice a year ago; it's just that the market is moving that quickly.

I'd really like to see the following clarified:

- I think we're talking about using a JS framework to render admin pages on the server and then handle post-render interactions?
- I'm assuming we're not talking having every user-facing theme use this JS framework to render content by default?
- What about UI components that appear for authenticated users on user-facing pages, like Toolbar, Quick Edit and Contextual Links. Would those use this JS framework?

We've had a lot of challenges with how to properly handle these front-end admin components. It's complicated, because generally you probably want those components to match the backend theme, but frontend themes might want to override some things with those components at times. I had an idea for how to improve that experience in #2632584: Add a "public admin" theme subtype so that admin themes can have subthemes for front-end UI components. Frankly if we end up using a JS framework to render the admin pages, and a more normal Drupal process for user-facing themes, I think this intersection of admin UI and user facing themes becomes even more complicated.

I'm trying to reserve judgment until the picture is clearer, but I'm pretty concerned about this.

We just finished a contributor-driven initiative to overhaul how people interact with Drupal's markup through Twig. We've worked very deliberately to make it easy for people to edit markup without having to know PHP. Now we're talking about potentially throwing out that work and mixing Drupal markup with JS, which is pretty much the same situation we just got out of. I know there are loads of people who love working with JS that think mixing the two has a ton of advantages. There are also loads that were excited about D8's front-end improvements. How is this going to work? Some sort of JS templates for the admin side, and a completely different set of Twig templates for user-facing pages? People now need to two completely different way to work with Drupal markup?

I'm also really bothered that this is a completely top-down approach. We are being told this is what Drupal is going to do moving forward. Nobody brought this up with the folks who did the bulk of the work on Drupal 8's front end in the last year or two (particularly with the Twig initiative), at least that I know. We're potentially radically changing Drupal's theme system, and I guess we get to make a few comments, then Dries decides what we are all supposed to do. Yes, he's the benevolent dictator, but this doesn't feel at all like a community driven process.

I think it's also worthwhile to keep in mind that changing Drupal's front-end templating from PHPTemplate to Twig took several years! It feels like this is being discussed like we are going to make a decision now based on the current state of the JS landscape, and then bam, Drupal takes advantage of that. In reality, depending on how much of Drupal's theme system is going to use these JS frameworks, this could be a hugely complicated and time-consuming process. It's certainly not going to take weeks, and I doubt even a few months with loads of people working hard on this would do the trick. We're talking about a year or two or three, in all likelihood, in which time the landscape may shift entirely again.

No matter what Javascript-based fancying up we do, the page must still be usable without JS.

If by "JS" in that quote you mean client-side JS, then +1. I agree with the fundamental principle that an open web includes being open to people who wish to disable browser JS for whatever reason they have to do so. Semi-related: http://thewebahead.net/110 is worth a listen.

I see the "server rendering" justification coming

Yep, good foresight: I propose #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS for Drupal 9. In terms of how it impacts this 8.x issue, personally, I'd be ok with any "next-generation admin UIs" built in 8.x to require Node.js, so long as sites without Node.js can continue using the non-next-generation UIs. That's just my opinion though, I'm sure it'll get debated. Or if a JS framework/architecture is chosen that works with PHP-generated HTML as an effective fallback for when browser JS is disabled, that's fine too.

The purpose of this issue is to select a client-side framework (e.g. Angular, Ember, Knockout), library (e.g. React), or compile-to-JavaScript project (e.g. Elm, ClojureScript) that would help us build next-generation admin UIs.

I think considering admin UIs is certainly necessary. However, I think it's also important to consider site visitor facing UIs too. And as a lead up to responding to @mdrummond's points in #11, I'd like to offer a motivation for why looking into these JS frameworks is so important. Consider the process of commenting on a piece of site content. Currently in Drupal, there's latency between clicking Reply and getting a reply form, and latency between typing a comment, clicking Preview, and seeing a Preview, and latency between clicking Save and seeing the comment appear on the original page. Even if we replace the full page refreshes with a Drupal Ajax UI that only replaces the DOM element of the commenting area, we'd still have those latencies while waiting for the server to render each of those steps. And those latencies is the difference between a user feeling like they are doing something and them feeling like they are ordering a computer to do something and waiting for the computer to do it. Which changes the entire feel of the process. And I would argue that now that a billion people have experience with commenting in latency-free Facebook, that Drupal's with-latency commenting process feels weird and makes you wonder if it's even reliable, much like what happens with the aging of any technology (e.g., visual effects in movies). And the reason I bring this up is that if open-source software like Drupal doesn't provide what will soon be considered a minimum acceptable commenting experience, then people will be tempted to move the conversation to services like Facebook or Disqus. But in my opinion, that's a really bad result, especially for conversations about sensitive topics (say, a support group for people recovering from abuse) whose content shouldn't be in the hands of those companies. Having said that, I'm ok if this issue focuses more on admin UIs as an interim step. But per #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS, I think eventually, all Drupal UIs should be capable of latency-free client-side re-rendering in response to client-side interaction.

I'm also really bothered that this is a completely top-down approach. We are being told this is what Drupal is going to do moving forward. Nobody brought this up with the folks who did the bulk of the work on Drupal 8's front end in the last year or two (particularly with the Twig initiative), at least that I know. We're potentially radically changing Drupal's theme system, and I guess we get to make a few comments, then Dries decides what we are all supposed to do. Yes, he's the benevolent dictator, but this doesn't feel at all like a community driven process.

I don't think that's what's going on at all. As Drupal's project lead, Dries has been bringing up in multiple blog posts and keynotes the topic of decoupling, with last month's blog post being perhaps his strongest statement about this being a really important topic for Drupal's future. But he's outlining a broad vision:

• That latency-free UIs are increasingly important for Drupal to embrace and support.
• That using a JS framework/library that's more capable than Backbone (which doesn't do any client-side rendering at all on its own: in Drupal core's case, we hand that off to Underscore, but Underscore itself and our usage of it is very bare-bones) is likely to be necessary.
• That picking one to include in core is necessary in order to use it for core UIs and a helpful default option for contrib to standardize on.
• And that he'd rather we not postpone all this on a clear market winner emerging, because that could take years and he wants us improving Drupal UIs before then.

As to whether we pick Twig.js for client-side rendering and stick with Backbone for state management, or whether we pick Ember and move templating to Handlebars (which is very similar to Twig), or whether we pick Ember and try to make it work with Twig templates, or whether we pick React and move templating to JSX, or ..., he's leaving that to a community process. Hence this issue. The current matrix is just a start of the conversation, and it includes a row for "Template engine friendliness for Drupal themers", and as part of this issue, we can improve that row or add more rows.

We've worked very deliberately to make it easy for people to edit markup without having to know PHP. Now we're talking about potentially throwing out that work and mixing Drupal markup with JS, which is pretty much the same situation we just got out of.

Well, if the solution that best meets all the needs that we need to consider (not solely theming needs) ends up being to use Twig.js or to change the templates to Handlebars, then we won't have that problem. But if the best solution is to use something like React+JSX or Elm+ElmHtml, then you're right about the concern. However, I don't think it's the same situation. With markup in tpl.php files, we had the problems of ugly PHP syntax for simple operations, no opportunity to auto-sanitize printed strings, leaking string vs. array vs. object implementation details into the templates, and no way to prevent templates from invoking dangerous server/database operations. JSX and ElmHtml might not be subject to any of those problems (I don't know for sure, we'd need to evaluate further). As to whether there are other problems with JSX / ElmHtml, I think we'd need to do some more exploration to properly evaluate.

People now need to two completely different way to work with Drupal markup?

Yeah, eventually I think we'll need to bring it back to one way, which I propose in #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS to be whatever way is best for JS (which may or may not be JSX / ElmHtml). But how to manage two template sets in 8.x (if it comes to that) is a very good question. If nothing else, that's a huge point in favor of finding a JS solution that can work with Twig templates. But if we can't find such a solution (e.g., if React+JSX offers too many compelling benefits to pass up), then let's explore options for how to make the transition state of 8.x as sane as possible.

+1 for the NodeJS route: https://www.drupal.org/project/nodejs

Here is a video for Private Message integration https://www.youtube.com/watch?v=f8XAT1oB1eA&ab_channel=ASYNCDrupal. NodeJS already has Views and other integrations too...

I personally think we can have different rules for the front-facing site experience (for site visitors or end users) and for the administration experience (for content editors and site builders). I agree that we can't require site visitors to require JavaScript, but I do believe it would be reasonable to require content editors and site builders to have JavaScript enabled.

it would be reasonable to require content editors and site builders to have JavaScript enabled

^^^ This would mean JavaScript is required to contribute on drupal.org (or other community-based Drupal sites). I don't know if people think that's a bad thing but it's worth pointing out.

On the overall issue, wouldn't it benefit from first having a specific design/mockup/prototype for exactly what Drupal's user interface (or at least an example part of it) should look like in the future, and then pick a framework afterwards? The Calypso writeup is a good read in that regard, although it defers a discussion of exactly why they chose React to a future post.

It's at least worth considering adding this to all supported versions of Drupal core.

On his blog, Campbell Vertesi made the interesting point that contrib is a great place to handle the ever-changing, rapid landscape of front end frameworks. All Drupal core has to do is make it easy.

IMHO, Drupal core needs to standardize on a good font-end DI container, (polyfills for) standard language features (eg. ES6 module loading) and APIs (eg. Fetch API for HTTP request) and lightweight decoupled front-end components. So any front-end frameworks or libraries can easily re-use these lightweight decoupled components and standard APIs. all they need to do is being able to interface with the Drupal's front-end DI.

Once this is done, using a components based approach, Drupal core could then standardize on a view library for its own UI. Having a clean separation between the components (and the DI) and the view library would allow Drupal core to adopt and switch the later more easily. It would also allow the administration backend to move at a different speed than the visitor front-end. Finally, it will minimize overhead should two view libraries be used during a transition period. Better having React and Polymer loaded on the same page, than loading Ember and Angular.

If the framework approach is chosen, the ability to replace its components easily and being non-opinionated on the applications architecture and dataflow are determining factors. Sadly, most font-end framework are not designed this way. Backbone was, but it lost is momentum long ago.

TL;DR: Consider not using a framework but a combo of interchangeable front-end library/tool. Consider Polymer.

Note: In the choice of jQuery, front-end developers are still annoyed of by how difficult this is to change the jQuery version for the front-end. So I would not call jQuery integration in Drupal a total success.

One of the many advantages of a progressive-enhancement approach: If core provides a solid "traditional" approach with the right hooks, then contrib-based admin themes can experiment, evolve, and Darwin it out while still providing that more integrated, Facebook-esque experience. Distros like Lightning or Panolopy can easily pick up one or another. That makes it really easy in 8.4 or something to decide if one has clearly "won", or been done best, or whatever, and just toss that into core beside/instead of Seven. And (and this is super important) we keep it sufficiently separate that it's easy to swap back to Seven, or the theme itself degrades gracefully, if on a given site you don't want to go that far. We keep the traditional approach working, ever and always, which gives us a stronger foundation to build the glitz on top of anyway.

That is a much lower risk approach for us to take than saying "rewrite the whole admin in framework X, even though it's only 3 months old".

Given one of my personal bugbears at the moment is staring at a blank screen on my phone waiting for js-heavy pages to download from certain websites when I’m out in the sticks, the very last thing I would want to be stuck with is needing to do a content update on a site whilst out and about and not being able to.

I get the idea of a more comprehensive framework for handling some of the complex interfaces we have for, say, views, but such a large proportion of admin pages are and should be simple. We’re getting really quite fab now at sending ‘good’ html forms for admin pages that require the absolute minimum from our connections. I would hate to see us lose that.

Is there an “opt-in” scenario we could come up with, like Crell suggested at #18 with something initially in contrib that does “advanced admin” that might move into core later? (and still be optionally enabled)

First of all I strongly disagree with the Problem/Motivation part of the issue. I don't see the problem, nor anything there that cannot be done with current AJAX framework, maybe some improvements to that would be fine but that's it.

Frameworks are always nice, but including them into Drupal core means basically "enforcing them". Once enough contrib modules have used the framework, they just will do if it is in core, site builders won't have other option than using it... then using any other framework for the site itself will mean loading two frameworks... Then we are at the starting point again.

In other words, if we let a fancy js framework in Drupal core, that will become the one and only framework for Drupal sites. And this is a serious vendor lock-in, since all these frameworks are pretty new and constantly evolving.

So my option for the client-side framework in core would be: None.

Putting it some other way, front-end frameworks should be pluggable and disposable. And they need to be in Contrib for that.

I also agree with Crell #18, though I'd suggest a bit different way forward: Add more reusable widgets / components in Drupal core and use them. Then any contrib module can just rewrite those widgets using any nice framework, or none...

So we have to learn twig now just to drop it on next Drupal? And drop php for javascript while in this ?

Frameworks are always nice, but including them into Drupal core means basically "enforcing them". Once enough contrib modules have used the framework, they just will do if it is in core, site builders won't have other option than using it... then using any other framework for the site itself will mean loading two frameworks... Then we are at the starting point again.

In other words, if we let a fancy js framework in Drupal core, that will become the one and only framework for Drupal sites. And this is a serious vendor lock-in, since all these frameworks are pretty new and constantly evolving.

So my option for the client-side framework in core would be: None.

Putting it some other way, front-end frameworks should be pluggable and disposable. And they need to be in Contrib for that.

So what happens when Media requires Ember and Commerce requires React? Two mega frameworks, and one very heavy page.
The client-side framework selection will never happen in contrib, for fear of conflict, and for fear of losing convenience ("To install Media you must first install the Ember module").

On the other hand, I do want to point out that Drupal is a CMS, serving content. Outside of the admin panel, few pages have a need for fancy JS. Commerce has quite a few (cart, checkout, etc), but that doesn't qualify as an ordinary page (as represented by a node or a view).
This makes a component-based approach necessary, an all-or-nothing framework won't do it for us.

@bojanz,

So what happens when Media requires Ember and Commerce requires React? Two mega frameworks, and one very heavy page.

I'd call it a bug. Site building modules or API modules should not enforce end-user UI frameworks. That may be different for "Distributions" though, like 'Commerce Kickstart' that is more a site solution.

The client-side framework selection will never happen in contrib, for fear of conflict,

That's my hope, most modules will just won't require those frameworks, will manage without them, I will be able to use them for my site solution with whatever other framework I decide.

Outside of the admin panel, few pages have a need for fancy JS.

I think it will be quite the opposite. And worse once every contrib module starts providing nicer widgets because they just get the 'fancy js' for free.

But... do we at least agree that once we get a framework into core, that will be the one and only option for Drupal sites?

@Jose Reyero
Thanks for clarifying, didn't realize your solution was "do nothing, don't address the problem".

@bojanz,
Nope, "my solution" is: "Do nothing in core, don't make the problem worse".

Just because I believe that adding the framework in core is just an expensive workaround.

My 2 cents

I feel the way Problem/Motivation is worded, it sounds like the only solution is to add a framework to core. After reading all comments advocating to do so, I'm not convinced it is really needed.

The problems we're facing when building Drupal sites (most of what follows applies to D7) are mostly not related to the absence of such a framework. Depending on the site we use one of the following strategies to make it snappier in the front-end:

1. Use core/contrib javascript with or without an updated jQuery
2. Use core/contrib javascript with a lightweight alternative (like Zepto)
3. Ditch all core/contrib javascript and use vanilla javascript

For the projects were we customize the back-end entirely, we stick with jQuery, but we improve the PHP side especially for slow pages, like for instance the permissions page. Dries' posts and the discussion that started afterwards on Twitter made me release some code on how this could be done, feel free to have a look at Dream permissions project

I think we as a community can better focus on how we can make it easy for core/contrib to start using fancy UI elements, https://www.polymer-project.org/1.0/ comes to mind, if we can provide an easy way (like render elements) for core/contrib to use them, it will make everybody happy.

To conclude, yes we have to do something, but as pointed out by @Crell, @mdrummond, @rachel_norfolk, @bojanz, @Jose Reyero and others, the solution is not adding a framework, progressive enhancement/Cut the mustard is to way to go and not only for Drupal.

Regarding @effulgentsia comment #12: Comments is kind of a special beast, most of it can be solved by using plain javascript if you render the reply box on the same page as the node. It is also a bit misleading both facebook and disqus works slowly on mobile and both of them needs extra request to open/start the comment box, which leads to mobile phones having to activate the radio and start fetching resources. Tested on https://m.facebook.com: 15 requests, 120KB, 2+ seconds

Whilst it's easy for me to say this as a non-frontend-dev, for situations like the Comment form, what in the status quo is preventing us from making it more whiz-bang with just modern-JS and jQuery as needed? What in the existing Ajax framework would make a "press the button, new comment appears immediately, comment is posted async and we assume it will save" type module really hard to do?

Off hand (again, speaking as someone who knows only enough JS to be dangerous), the only part that sounds even slightly hard to me is the "render new comment immediately before Drupal responds" part. The rest should be fairly dead-simple now with the improvements that have been made to core. And that part is mainly a double-theming problem (server-side and client-side), which won't be solved by adding Angular, React, Ember, or whatever else to Drupal. We'd still need a template in both places.

So... someone help me out here. What's the pain point that we have currently that a standard large JS framework would solve, that smart use of the Ajax API, jQuery, and Javascript that relies on a browser written after the Bush presidency ended couldn't?

@attiksThe comment example is a good one to unpick things. For example there's text formats, image and other file uploads to deal with. All of those will be very difficult to handle client side in terms of instant rendering.

Another way to approach that would be draft + autosave support, then actually posting the comment is just switching the published status and it'll already be available otherwise for preview. Same goes for previews of other entities as well. All of which as you say can be done with Ajax.

#27/#28 The most difficult to implement would be a generic solution that will work for all possible comment forms, since these are fieldable and nestable, you don't know what you have on the client side. Since they are extendable as well using PHP (hooks) you have no idea what's going on on the server side, things like auto generate title, mark a user depending on his/her role.

So the only proper solution is to have a framework that supports all of the above, including a magic translation of all PHP logic to javascript. If we as a community want this I see 2 viable solutions, switch to javascript server side or add support for PHP client side, but this is being discussed in #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS. For the moment we handle this on a project per project basis, if you know how the form will look and what logic is added on the PHP side, you can build a snappy comment form (even in D7).

You cannot compare this to commenting on FB, they have one specific form to deal with.

Sounds too risky to bet on a big strongly opinionated JS framework at this point. I dislike Ember, just because it's the biggest one. We are killing the mobile UX just from the start.

It's a pity BackboneJS has lost momentum, it was a perfect progressive step-up for us: easy to learn, small, compatible with jQuery. But the world has moved on, Backbone and Coffescript have got stuck in the ES5 era.

It's not only about choosing a framework for its merits, it's the baggage the bring with them. React, Angular and Ember are very opinionated, and bring their own "tooling pack" that alienates communities.

You can't swap them so it makes it look scary to bet on a single framework.

At this point, If I would have to consider a framework, I would like VueJS or RiotJS to be at least considered. The documentation of VueJS about how it compares to React/Angular/Ember/Polymer is a must-read: http://vuejs.org/guide/comparison.html#React

Vue is smaller (~24kb min+gzip) supports IE9 and newer. No jQuery dependency. "Provide the benefits of reactive data binding and composable view components with an API that is as simple as possible". MIT licensed. It's not the most popular at this point, but during 2016 I'm sure it will "catch up" with rest.

Choosing Vuejs, ironically, sounds like a safer bet to me, at this point, than other bigger, more popular frameworks.

I don't mind that it's not isomorphic. It's overrated concept IMO, and I would bet on PHP to level-up soon (as Crell pointed) during the next years, instead of adding a nodejs dependency.

Also, see author words about VueJS in 2015. Highlights on being adopted by Laravel community,.. and VueJS being a progressive framework:
http://blog.evanyou.me/2015/12/20/vuejs-2015-in-review

For reference, two surveys to JS developers were published recently. We can't overlook that jQuery still is the number 1 library and the most familiar for every developer
http://ashleynolan.co.uk/blog/frontend-tooling-survey-2015-results
https://ponyfoo.com/articles/javascript-developer-survey-results

I feel like we have jumped straight to "which JS framework should be in core" skipping right past the discussion of whether or not it is a good idea for us to rely on a JS framework in core, and how much we want to rely on it on that framework outside of the administration interface.

I would be much more comfortable if this issue focused on "Backbone is obsolete, what did Backbone solve for us, and what do we replace it with?" and we stayed away from the grandiose vision of one particular framework solving all of our current and future front end problems.

Finding a replacement for Backbone, or rewriting that functionality without a framework should be the #1 JS in core problem to be solved in the current cycle. I don't think anyone on either side of this issue would argue that we should keep Backbone.

Part of this discussion should dig into history and figure out how we ended up picking the wrong horse when it came to Backbone and how we avoid that in our next decision especially considering the quick turnover and short lifespan of JS frameworks.

I was doing a little research about longevity and stumbled on http://blog.bitovi.com/longevity-or-lack-thereof-in-javascript-frameworks/ which is obviously biased but it is still an excellent read and needs consideration. Note this is not a vote for the framework they propose merely an endorsement of adjusting our evaluation criteria.

Longevity should be a factor but I have to question people arguing that Angular, React & Ember won't be around in a few years.

The enthusiastic (much more so than YUI etc) corporate backing and adoption (https://github.com/facebook/react/wiki/Sites-Using-React) has to count for something. It's hard to imagine Wordpress, Netflix or Facebook completely dropping React over the next 5 years when it's already being used in so many critical applications.

#1149866: Add Backbone.js and Underscore.js to core seems to be the best place to look at how and why Backbone was included in core and the discussion around that decision.

WIthin that issue there are references to both an improved authoring experience and onslaught of new front-end developers are both referenced as motivation for its inclusion.

We did get an improved authoring experience, but the onslaught of new front end devs did not occur.

It feels like we are making the same arguments for framework-next as we were for framework-last. I am wondering if the choice of framework is a red-herring for the continued lack of front end devs involved in core?

I just wanted to point out for @mdrummond and others with concerns about how this affects the theme layer that minor versions of Drupal (so Drupal 8.1.x, 8.2.x, etc.) can only add new APIs and functionality, and must retain backward compatibility. So breaking the render and theme APIs is not on the table for 8.1.x or any minor, and to me, that would include not breaking support for Twig. For example, the Stable theme is supposed to be, well, Stable. :) This was also my foremost concern originally upon hearing this discussion.

So for me the big picture is more about making Drupal friendlier for JS frameworks/frontends in general, and exploring how we could iterate that with API additions and optional new functionality. Picking a specific framework is a decision that would merely affect how much mileage we get out of these additions -- thence this issue, I believe.

Finally, developer experience, themer experience, testability, and security are all important criteria. Edit: For both the potential framework and the architecture we might use with it.

#31 is a good comment - i.e. we're using backbone for a number of core features, it needs to be replaced, and then we might want to update the Views, Field UI, Permissions and other large/complex interfaces to use whatever the replacement is during the 8.1.x cycle. This issue then becomes a lot more straightforward if you think like that.

and how much we want to rely on it on that framework outside of the administration interface.

So in general, we aim to serve anonymous page requests without any JavaScript at all by default, and I think we should keep that aim. Or if we don't there needs to be a very, very clear trade-off.

Where this gets tricky is entity forms - at least node, comment and user accounts are exposed to authenticated users and to an extent anonymous users very easily in the standard profile, and the comment form is on regular pages (such as Drupal.org) where performance is extremely important. Views exposed filters and similar are another case.

So I think it would be worth trying to restrict the discussion to choosing a new client side framework for site building and content administration interfaces. For now, assume that core at least won't use it for anything visitor-facing, which means not on content entity forms and some others.

That doesn't mean never updating the comment form or user account form etc., but doing so we'd start a new discussion for (and I don't think it would start with a discussion about a framework in those cases).

For now, assume that core at least won't use it for anything visitor-facing, which means not on content entity forms and some others.

+1. This also means all the great new front end things in Drupal 8 remain totally unaffected (from all things Twig to improved asset handling to …).

And in fact… this is exactly what that the IS already proposes. The first sentence reads: The fluid experience in well-written single-page applications today far exceeds what Drupal’s admin layer provides. And further down the issue summary, it keeps stating "admin UI". @Dries repeated this again in #14.

So, let's make that more explicit by updating the issue title.

(I suspect if this were the title in the first place, this would've helped avoid a lot of drama.)

Dries reasons for this are to improve UX and get new people. I tried to come up with deeper reasons that would make be back this up but it turns out it was just wishful thinking. Improving UX has nothing to do with a framework and everything to do with our issue workflow. It's never the code that really holds up UX issues. Now imagine anyone else other than Dries pushing this issue with such weak arguments, it would not have gotten on anyone's radar.

Proposing such disrupting changes without a strong case would hardly get past core committers and this is essentially what we're being asked to overlook now. There are a few claims in the IS that don't sit too well with me either:

• "For example, it is no longer acceptable to experience a full page refresh when deleting a node or applying filters to searches of content." Says which study?
• "It is also extremely important that when performing any action, users get immediate feedback (or even optimistic feedback, in which the user sees the result even before getting a response from the server from their action, e.g. liking a Facebook post triggers an instantaneous response)." I do agree with it but same thing, any study to back that up? surely not all interactions get the same benefit out of having instant responses, might even be harmful in some cases, who know?
• "Application-like UIs in Drupal (e.g. Views UI and block placement) would benefit from a standard framework to enable a fluid experience." this is on the verge of marketing talk. If it means consider views UI as a single page app, block placement as a single page app it would be more explicit.

Attracting contributors has nothing to do with the framework we'd eventually choose:

1. Drupal is a CMS, that limits the number of people interested in the problem space to begin with
2. We're not on github and there is no pull requests, pretty much all (if not all) of npm is on github that's what the "mass" of js devs are used to
3. Getting in the Drupal queue from the outside takes some serious will

What we have going for us is that we're using eslint and all our code validates, that's about it. Adding a framework will reduce the number of people from #1 but would slightly increase the motivation of #2. Given our strict (and fatally tedious) core process we'd be lucky to get even one person to stick to the community. Maybe there would be more js devs on Drupal projects, but with all the barriers to contributing I'm pretty sure they would say "Drupal sucks" and go around whatever issue they have. Ignoring the fact that a good JS dev is harder to find than a good Drupal dev.

To be honest in the issue summary "Adopting a framework with a better front-end developer experience would also attract new front-end contributors to Drupal." needs to be removed from the IS summary, that's not true. Still waiting for the swarm of new Backbone contributors.

Given all that we are left with three solutions

1. Do nothing, arguments are too weak and the solution doesn't even address exposed issues.
2. Select a framework get it in core for contrib to use and do nothing with it. Wait until panels or something else make use of it before starting to port core over.
3. Get some real arguments behind the proposal and get the process back on track.

There are a few claims in the IS that don't sit too well with me either:

"For example, it is no longer acceptable to experience a full page refresh when deleting a node or applying filters to searches of content." Says which study?
"It is also extremely important that when performing any action, users get immediate feedback (or even optimistic feedback, in which the user sees the result even before getting a response from the server from their action, e.g. liking a Facebook post triggers an instantaneous response)." I do agree with it but same thing, any study to back that up? surely not all interactions get the same benefit out of having instant responses, might even be harmful in some cases, who know?

More than that, even if we accept that these is a problem, if we're only going to use a framework for site building/content editing UIs, then there's no way it's going to affect either of these cases for site visitors anyway. So we can open new feature requests for these (AJAX-enabled entity forms in core, instant feedback for AJAX operations), but neither should be in the issue summary for this issue.

Re-title looks like a cross-post so switching back.

Maybe we should ask feedback from the Panels maintainers. They've chosen to use backbone for Panels IPE for the Drupal 8 version, and as far is I know the first contrib to use it (there might be others, don't know about that). What's their experience, pro's and cons why they made the decision etc ?

Acquia lift uses backbone too.

I have to hugely agree with nod_ at #38 that it really does feel we’re missing the point here. If this is about attracting frontend developers, adding a framework will not achieve that in any way I can think of.

Another query strikes me, too. Presumably, the thinking above was to replace Backbone with something? If so, isn’t that an API change of sorts? People will no longer be able to assume Backbone is available and current js api calls will fail. That means this needs to be a 9.x issue, not an 8.1.x issue?

Interesting point I've observed: most (if not all) of the people pushing for this so far are PHP devs; all the front-end people in this thread are pushing back. That tells a whole story :).
We definitely need more front end guys commenting here.

I can totally see the merits in selecting a client-side framework, so it's easy to go fully decoupled if you are doing a project where it makes sense.

However, I'm not sure "progressive decoupling" is the right goal - doesn't that come with a lot of added complexity? It means the front-end rendering process always ends up being partly server-side in PHP/twig and partly on the client side. So wouldn't we end up with two different, rather complex systems that have to be properly integrated to work properly together? so once you have troubles, you have to debug across two different systems...?

Instead, wouldn't it be easier to vastly improve the Drupal ajax system and keep all rendering server-side, e.g. to natively support history updates, improve ajax performance, e.g. add some optimistic feedback etc. Couldn't we achieve the goals this way while keeping all rendering server-side?
I see that optimistic feedback is probably the hardest part here. It should be pre-pare and send simple feedback together with the initial page load, but optimistic feedback that involves user input would be obviously harder. So we'd have to clarify first, what kind of optimistic feedback we'd like to do.

So I basically second the questions crell raised in #27, what are the points an improved ajax framework couldn't handle?

catch has a good point in #28:

Another way to approach that would be draft + autosave support, then actually posting the comment is just switching the published status and it'll already be available otherwise for preview. Same goes for previews of other entities as well. All of which as you say can be done with Ajax.

Thus, optimistic feedback involving user input wouldn't be possible with a client-side framework either as long as the whole input filtering pipeline isn't re-implemented in javascript. (What sounds a lot like moving towards node.js in general).

Using the AJAX framework doesn't get rid of latency: you always need a round trip. That's the biggest problem. Even in an imaginary world where the AJAX framework communicates via websockets (which would surely reduce the server latency i.e. by avoiding a Drupal bootstrap), it still cannot remove network latency.

That's the key thing: to eliminate network latency, you must run the logic on the client side, which requires JavaScript.

(Not saying whether that's desirable, just observing the facts.)

Agreed on optimistic feedback involving user feedback being impossible without duplicating/rewriting boatloads of things in JS. Which is exactly what I said in #2645666-3: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS.

Using the AJAX framework doesn't get rid of latency: you always need a round trip. That's the biggest problem. Even in an imaginary world where the AJAX framework communicates via websockets (which would surely reduce the server latency i.e. by avoiding a Drupal bootstrap), it still cannot remove network latency.

If you're doing entity previews, updating Views, inline editing etc., then in most cases we want to do a network round trip to ensure the current state of the object being worked on is in the database - whether that's an entity revision, temp store, key/value etc.

Even if we were to figure out client-side rendering of changes to Views forms (which seems extremely tricky - if you select a plugin, you need to get the options for that plugin, which is going to be a round-trip, or you have to pre-load them, which still has to come from the server beforehand), then I don't think we'd drop writing that change back to temp store. The main difference is it wouldn't be blocking and could happen in the background - but that backgrounding feels solveable in at least some cases while still relying on server-side rendering.

Good points! Especially ensure the current state of the object being worked on is in the database.

The main difference is it wouldn't be blocking and could happen in the background - but that backgrounding feels solveable in at least some cases while still relying on server-side rendering.

I've also thought about this. The difference is that doing the logic on the client-side makes it simple. No shortcuts necessary. Optimistic feedback without having the logic on the client means taking shortcuts while doing the preview, and possibly replacing that instantaneous-but-low-fidelity preview with a high-fidelity preview upon getting a response from the server.

It'll be interesting to see what we can come up with. (I'm particularly thinking of the filter system, for example when posting/previewing a comment.)

Do we really want to go that route ? Insightful (mostly) on-topic about JS frameworks : https://medium.com/@wob/the-sad-state-of-web-development-1603a861d29f#.x...

People now need to two completely different way to work with Drupal markup?

That is the situation we have right now. For example, this line for adding an "Add" button in Views UI and this whole file of QuickEdit components. In other words, we already have UIs where we're not using a Drupal AJAX request and server-side rendering, because the latency of that would be silly.

Maybe we should ask feedback from the Panels maintainers. They've chosen to use backbone for Panels IPE for the Drupal 8 version, and as far is I know the first contrib to use it (there might be others, don't know about that). What's their experience, pro's and cons why they made the decision etc ?

I'm not a Panels maintainer, so I can't answer that, but just a quick look at the code shows that they're continuing core's pattern of inlining client-side HTML into the .js files, with the addition of also using some Underscore.js template syntax.

Do nothing, arguments are too weak and the solution doesn't even address exposed issues.

Answering this issue with "do nothing" would probably result in:

• Some contrib modules (like Panels IPE) getting ported with the Backbone/Underscore pattern, which requires themes to use completely different patterns to customize that markup (it's not Twig, do themers even know what the API is to swap out Backbone views and Underscore templates, there are no preprocess functions, etc.).
• Other contrib module maintainers deciding that Backbone/Underscore is too limited and therefore picking Ember or Angular or React or Elm, each one with its own different approach to client-side markup generation. Then if a theme for a particular site wants to customize the markup across all these different contrib modules, it will need to learn the way to do so for all of these frameworks.
• If in Drupal 8.2 or 8.3, we want to add any of these modules to core (e.g., Panels or Media or ?), then if what these modules ended up picking is not the framework we want in core, then a rewrite to the framework we do want in core will be needed.

Maybe the above is an ok situation. Maybe we need the additional exploration time of each contrib module picking its own approach with no nudging of any kind from core, and then we look at the resulting mess in a year or two and figure out how to align everything then. But I'm not yet convinced that that's the best option. I think it ought to be possible for us to explore these frameworks a bit more, and based on that exploration, see if we can converge on a preferred direction. Contrib maintainers who feel strongly about one framework over another will be free to deviate from that direction if they want to, but my guess is that if we pick a good library/framework, then most contrib maintainers will be ok with writing their client-side UI code with it. The sooner we pick that direction, the sooner that those contrib mainainers who don't have a strong desire to deviate from it can benefit from knowing what it is, and thereby use it when porting/creating their modules.

Note that one option in all of this is to keep Backbone and add Twig.js to begin migrating the inlined HTML-in-JS that we currently have. But I think that option should be explored alongside the other ones, like Ember+Handlebars, Ember+Twig, React+JSX, etc.

So we know that Drupal 8 in its current form is compatible with some headless designs - why not have multiple teams work on example code for each framework we're interested in? Even if it doesn't lead to a decision, we would likely start to see patterns of limitation in Drupal core that could lead to important patches/issues for 8.x-1.x.

Each example project would need to meet certain requirements, some of which could be:

1. Render a page with content regions based on Drupal's block placement. Block HTML can be rendered by Drupal for now.
2. Render a Node with fields based on a template defined in Javascript. Field HTML can be rendered by Drupal, Node metadata should be rendered in the JS template.
3. Render a form with basic submit functionality, all HTML should come from the frontend app.
4. Demonstrate how a contributed module (another piece of JS) would extend or modify your code to customize templates, logic, etc.
5. Demonstrate how a piece of content (region, block, node, form, etc.) would be re-loaded from the server after the page is loaded.

The code written wouldn't have to be perfect, or even work completely, but each requirement we come up with should have a detailed response and piece of example code for each framework. Of course the real requirements would be set by the community.

Edit: I didn't make it super clear originally - but the main difference between the above and what's in the issue summary is that 1) We should have set requirements for the teams building proof-of-concepts. 2) We can make real-world proof of concepts with the current release of Drupal 8, otherwise we may as well be looking at existing examples for the frameworks in consideration.

@samuel.mortenson all those examples, apart from #3 and #4 in very limited cases, are for visitor-facing UIs. That's the sort of thing that should definitely be done in contrib, but it's not what the focus of this issue should be about (although the issue summary and original title do not help that by also including visitor-facing interfaces as examples).

@catch Admin themes have regions where blocks are placed (#1), modules like Views render Nodes/fields in the Admin UI (#2), and reloading content via AJAX is an extremely common admin feature in Drupal 8 (#5). Also, if we're only thinking about the admin UI, where do admin-tools that live in the "visitor-facing ui" fit in? Thinking of things like Quickedit, Toolbar, and CKEditor.

FWIW: here's my take on decoupled interfaces: www.colada.be/decoupled-drupal-a-couple-of-considerations.html

My blogpost highlights what Effugentsia states in #51 I considered a decoupled interface for a contrib module I maintained, two years ago, but didn't pull through because:

• I had to choose a framework for my module while other contribs might have chosen other frameworks.
• Without a common backend architecture, I had to create my own idiosyncratic API resulting in a lot of non-reusable boilerplate
• If the community settled on a consensus solution 6 to 12 months down the line, I risked rewriting the entire thing to conform.

I do believe decoupled interfaces are the wave of the future. But I think we should be smart about where and why we implement them. I don't think just slapping a framework in Drupal and creating a generic backend API on functional quicksand is good idea. I see them more as reusable web components that enhance the UX instead of fully decoupled interfaces. I agree with Dries point of view that decoupled components should be limited to the administrative backend at first. In my view: to specific places at first even. That way, introducing decoupling becomes a manageable problem.

Also, adding a decoupled interface inevitably means revisiting the backend API. Almost all user input in Drupal is processed through the Form API going from a simple contact form to the entity display configuration forms. Can we refit/amend/repurpose the Form API so that it can work in a decoupled context too? If we want to build a foundation that can process user input in a uniform way, the Form API is an important part to consider in the equation.

#55 is more about the other issue, especially #2645666-45: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS this comment from Wim.

While "the sad state of web development" blogpost are making the circles and was linked either here or the other issue, the answer to it has an incredible important and relevant insight:

Shiny new “products” rain from the Cloud almost everyday. [...] Its a good time to not be a fanboy.

I am not a JS person. Heaven forbid. If there's a worse language than PHP, JS it is. It was created to make the monkey dance when you moused over it and while it is slowly becoming sane(r) with ES6 (note the parallels of how PHP have crawled out of the pit and became a somewhat sane language over the years during 5.0-5.4) it is still a language I will only touch if forced. I'd rather go and line up at the soup kitchen than become a JS developer. Ahem.

With that said, Angular vs Angular v2 should make anyone pause. Say, Angular became popular about 2012 and you can toss your code with v2 coming this year if I am not mistaken. What you all will pick in this issue needs to be a "thing" for the next five or perhaps even more years. Yes, it's likely React will still be used in five years (or not. I posted about CanJS above which powered the Apple Store in late 2013 but if you try CAN.version today in the console, it doesn't seem to be any more and we are talking two years of a framework very much purported to be long term) but whether it's Perl-like has-been level of usage, who knows.

All I am saying: proceed with extreme caution. The change of pace in the JS world vs the change of pace in the Drupal world seems to be incompatible. I have no idea how we will resolve this.

I'd rather go and line up at the soup kitchen than become a JS developer.

Never say never, again :)

That is the situation we have right now. For example, this line for adding an "Add" button in Views UI and this whole file of QuickEdit components. In other words, we already have UIs where we're not using a Drupal AJAX request and server-side rendering, because the latency of that would be silly.

So we are sending pre-rendered HTML to the client - I see no reason though this HTML could not be generated with the usual render/templating system. Sending it pre-rendered to the client is no excuse to bypass the theme system imo. But yeah, the tooling to easily do that in a proper way is not necessary there where it should be and we could improve our existing tooling to avoid a further split.

Agreed on optimistic feedback involving user feedback being impossible without duplicating/rewriting boatloads of things in JS.

Right, so to me this discussion boils down to re-implementing many things in node.js (#10725328) or have just stick with server-side rendering and improve the tooling / AJAX api around.

#49, #52I created 2 modules (Drupal 7 for now) to see how easy/hard it will be and to see what people expect from it.

- https://www.drupal.org/project/dream_permissions
- https://www.drupal.org/project/dream_comments

#56

All I am saying: proceed with extreme caution. The change of pace in the JS world vs the change of pace in the Drupal world seems to be incompatible. I have no idea how we will resolve this.

Fully agree, it changes way to fast for whatever reason. I think the only way to solve this is baby-steps, and adding a full blown framework cannot be the first step. Maybe a first step is to create separate issues and define a single use case in each so we have a list of pain points, this will make it easier to define what needs to be solved and how it might be solved.

#58 Yes for prerendered HTML by the theme, it is the only sane option, both for performance and accessibility reasons.

Yes to make it easier for javascript to interact with forms, even yes to a 'framework' that makes this easier. Drupal AJAX system has it merits, but it is not the most performant one.

The last year or so we started ripping out all javascript for the front-end and replace it where needed with custom code, the main reason is that a lot of contrib (and probably my own modules as well) is adding way to much javascript. We now use small libraries to add the functionality we need, so maybe it is an idea as well to investigate if this might help for other sites. Have javascript libraries for different use cases and make it easy for developers to use them, although this sounds a lot like components proposed elsewhere.

Addition to #59, Here's a module I did for D7 with angular: https://github.com/netsensei/cats based on experiences with client projects. And this is a short screencast about above module: https://www.youtube.com/watch?v=p3zSQieBIe8 => garnered 7.800 views over 18 months. For a Drupal specific topic, I think that's a significant number.

For the Dutch speaking community: https://www.youtube.com/watch?v=sRva9hleFXY Presentation about a decoupled interface powered by a Drupal/SOLR backend. All Angular 1.x based.

Also, I fully agree with #56 and #59. Yes, I am excited about decoupled interfaces, but we should proceed with extreme care.

tl;dr: See https://twitter.com/pierrebuyle/status/687639939043975168

SAP and other complex UI are rightly called applications (here and mostly anywhere else). These are not just tin UI layer. Years of experience on applications development apply. We learned since long to create monolithic application tightly coupled to a specific framework and library. Even if Drupal was a rewritten as a Symfony or Laravel application, only few of us would expect things like Entity CRUD or Entity validation to be tied to the used framework. Instead, these would be properly encapsulated as decoupled components. Sure implementation classes may depends on Symfony's DI container and standard Symfony service. But it seems safe to assume that we would choose a framework that clearly favor decoupled components and architecture, in its design, documentation and practices. Yet this concern is completely absent in the initial reflexion.

There is a reason Drupal didn't switch to a PHP framework in all these year. And I don't think it was the amount of needed work for (not) rewriting it on framework X. I think we didn't do it because we didn't want to tie Drupal to any framework. So instead, Drupal was re-architectured around core concept and existing external components have been brought in, and some times enhanced.

Why should it be different for the front-end? Why should we pick a framework (some of the candidates are pretty opinionated and monolithic) and build (a better) Drupal (Admin UI) with it? Why is the discussion about picking a framework and not the needed font-end components that we need to build a better UI?

Let's assume we agree that the argument around this getting us more people is moot. If this is really about UX improvement we need to fix our UX process before, so that designers don't get burned out and our UX team plans not questioned in their entirety months after they've been agreed upon (that should be easier thanks to our regular releases).

Adding more very opinionated developers to UX issues is not going to help improve Drupal UX any faster. Any seemingly simple UX issue gets bikesheded to near-death, let alone complex ones. See issues for toolbar, module page, new content creation page, the search field on the module page (and this one is not even finished for some people) and let's not forget most issues from #2497361: [meta] Fix issues found during UMN Usability Testing 2015. From that list one, maybe two, would eventually be helped by a frontend framework but only if the framework integration has been done properly .

As it is now we already have jQuery, Backbone and Underscore loaded on every page for most power-users because of Toolbar. Adding another framework for admin UIs is just going to add a third big bundle of JS to admin pages. Killing what's left of usability of the Drupal admin of low powered phones (and that's what most users have). In the long term, getting standardized on a framework should mean we deprecated and remove jQuery, jQuery UI, Backbone, replace joyride, dialog, autocomplete, farbtastic and other utility libs, which in turn means rewriting quickedit and almost all other features. It's doable, sure, but there was the need of not wasting time expressed in this issue too. What Sally Young proposed in the lullabot podcast was the most sensible: If we really really want a framework, make a specific theme and tie the framework to it, not core modules.

After a lot of thoughts, reading back all of Dries' posts, listening to the latest lullabot podcast on this, reading posts from our folks I'm convinced that this is not the right solution to address the issues outlined which are: attract more devs, improve D8 UX, ride the wave of interest in JS (still need that issue summary update btw). Framworks are hot right now but don't address our problems and it shouldn't be our first step in the problem space of optimistic feedback and client-side templating.

As it was said before, most (all?) frontend people here think this is a bad idea — or at least a bad first step — I'd be nice if we didn't divitis our JS after all the work to streamline it.

It's pretty clear some steps have been forgotten in this process hence i'm postponing this issue and opened #2651660: Investigate where and how a frontend framework could be used where we can talk about the why and how. Once this is clear — and the UX process part is figured out — we can start talking about technical solutions. Making any kind of decision before is premature, after all we can't know which critera are relevant before we know our requirements.

On a side note, taking from a relevant example on the performance aspect: Typo 3 Neos 2.1. The whole admin is EmberJS, single page app, inline editing, very fancy stuff. You get all that after the 2MB of JS are loaded (a lot of it is Aloha to be fair, but there is also jQuery, Backbone, jQuery UI & a bunch of plugins). On my cheap device the admin is not usable (it's a small phone, buttons overlapping all over the place), and it takes 20 seconds for the admin components to load (with warm caches), changing a page (within the single page app) takes about 9 seconds. On a more powerful tablet loading takes 13 seconds and page change 7 seconds. On my laptop the fan kicks in when I browse the admin. On the desktop as long as you're browsing admin screens that have nothing to do with the content, things are pretty fast. The catch is that those admin pages go to a new page, they're not part of the SPA. It's so different I'm not sure it's relevant to compare but if there is anyone to ask for feedback it's those folks.

( edit ) and they already bikesheded that for Typo3/Neos https://discuss.neos.io/t/rfc-js-refactoring-process-of-backend/283

I agree with nod_, there is a gap in this selection process with where and how we are going about this, but I think there are some underlying discussions we should have instead of simply "which framework", and "what are the use cases". I want to break down a few points that I think will help the community approach this in a way that will benefit any front-end developer who is coming to work with Drupal as a decoupled framework.

Rather than focusing on a single solution, we should focus first on how we are going about deciding this, and with which core values or guiding principles we should use to judge proposed solutions.

In the context of this discussion, we should be focused on the problem of:

How can we dependably expose administrative forms in a way that allows for easy adoption of progressive decoupling?

Initial Thoughts

1. People come to Drupal for a CMS experience, so our APIs are always going to be heavily coupled to the purpose of working with content—"content management".
2. That means we need to be thinking of API contracts / conventions which are for the specific purpose of managing content, as opposed to driving an application.
3. As a community, we need to adopt defining standards for decoupling. Instead of focusing on the which framework question, we should be discussing: how can we expose content management in Drupal to ALL frameworks. This presents the question of progressive decoupling in the context where we can focus on making consuming Drupal APIs pleasurable for all JS/Front-End developers instead of simply just one subset. Defining standards on how we can do this will likely be challenging, but worthwhile.
4. Defining methodologies for decoupling in core should be advocated as a cultural practice. We can likely think up some kind of community slogan or initiative in order to encapsulate this ideology within Drupal. (e.g. "Proudly found elsewhere"), see list at the end for some ideas.

A Proposed "Drupal's Code of Decoupling" (to start)

1. Always assume there will be a new JS framework in the future.
2. Develop core content administrative APIs so that we can build Plugins on top of them for specific framework interfaces. Transformation into specific framework conventions can be adopted at the last phase before API rendering. This leaves the required room for framework/application specified interfaces.
3. Choose a decoupling methodology that focuses on frameworks that leverage view components. [note: With web components somewhere on the horizon, we can expect the web to move progressively towards view-based rendering.]

I'm sure others have great ideas to add to the above, so I won't try to be exhaustive here.

How can we expose Content Management in Drupal to ALL Frameworks

1. In order to expose Drupal as a strong content management tool for decoupled applications, we should first look at how we can create a common interface for administering content. Most of what we are talking about when we discuss content management is Forms and Views.
2. Deciding on a convention to expose the Form API and Views UI seems to me like an easy 80/20 selection for where we should start.
3. We should create a convention to expose components of the Form API either discreetly as a single object / payload.

Some bigger Decoupled Hurdles with Drupal's core Value Promises

What features/expectations do people have when choosing Drupal? How does delivering on these expectations become more complex when extending to a decoupled architecture?

I would argue that people choose Drupal for content management when they're looking for an extendable framework, but usually want these core features (or have expectations of these features being easily attainable):

1. WYSIWYG control of content
2. Fielded control of content
3. path management (path,redirect)
4. metadata
5. layout control via UI (Panels)
6. Robust content relationships (entityreference)
7. Views building
8. Cache invalidation management

Thoughts on Slogans for this initiative

1. Proudly designed anywhere.
2. Proudly rendered anywhere.
3. Proudly experienced elsewhere.

I'm sure some of these lists could be broken out into their own issues, too.

This is very true. Your main question does start with a "how", it belongs in the other issue.

Agree with @jordan8037310.
First of all, not everyone wants a decoupled architecture out of Drupal. Among those who need one, its uncertain at this point whether they would be happy to use the framework dictated by Drupal. There are so many factors influencing this decision like business requirements, licencing, familiarity with the "chosen" framework etc.

We should rather focus our energies on building an interface which can expose the content under the hood in a framework agnostic way. This wouldn't change a lot for contrib modules too. It could be something like adding views support or features support for a contrib module. A contrib module maintainer could opt in to support this interface or otherwise. It could also come in as a feature request.

That said, I think a component based system would play nice with this. This is because Drupal's rendering mechanism closely resembles this kind of an architecture and the cognitive overhead would be lower if we follow this path.

I agree with jordan8037310's Drupal's Code of Decoupling. Starts by building APIs (server and browser side) to be consumed by frameworks powered applications. And, for a time, let contrib deal with building these applications (powered by these APis in core).

Thanks for the update. I would argue that "It would also accelerate UX improvements by providing better code reusability across all of our UIs" still isn't true, frontend code is not the bottleneck on UX improvements, never was.

All in all the situation hasn't changed much so I'd say this should still be postponed. We still don't have compelling reasons to consider adding a framework (some things are talked about on twitter but still not in the issues so they don't count). If there is a plan for features or something that really really needs a framework just tell us. I'm all for getting better tools but not for no reasons.

Quickedit is doing it's job fine with backbone, toolbar shouldn't need it to begin with (let alone another framework), contextual links, meh, either way is fine. If what you want if a framework name without even thinking about what we'll do with it and how, then it really doesn't matter which one we choose. My vote would go to the most consistent and opinionated one, ember since I don't look forward to coming up with guidelines and recommendation for a framework we're not actually using. We can do solution #2 of comment #38 and be done with it.

On the, Decoupled "front-end components". Issues #1804488: [meta] Introduce a Theme Component Library amd #2289619: Add a new framework base theme to Drupal core are interesting, except the front-end the components I was talking about are not about theming at all and should not be viewed as a theme thing. If we want to move forward with Drupal UX (not just UI, and not just its look), we need to differentiated between theming and front-end code. I may be wrong, but IMHO, the most widely used meaning of "theme" for computer software is the "graphical appearance" of the software. The client side components I'm talking about have nothing to do with the graphical appearance.

The kind of components I was talking about are things like a Entity CRUD component (ie. an object responsible for client-side Entity CRUD operations, hidding the gruesome details of communication withs the server) or an Entity Validation component (ie. an object responsible for client-side validation of an entity object). I guess I should have used "services" instead of "components" when I wrote "lightweight decoupled front-end components". But I didn't use "services" on purpose, because these are not interface to web services, neither do them have to communicate with the server in order to do their job. And they don't have to have any behavior at all, they could be immutable collections of data (for instance, configuration variables used by other components like the list. Because yes, you can use DI to inject your configuration variables).

If this existence of this kind of components in Drupal core needs to be discussed in a separated issue, I doubt #1804488 or #2289619 are the ones to use.

frontend code is not the bottleneck on UX improvements, never was

Are you sure about this? While I agree that it's not the only bottleneck, my sense is that it is a significant barrier. Because to improve UX, ideas need to be tried out. But trying an idea out requires some implementation of it. And if creating that implementation is not enjoyable, who's gonna do it? Sure, maybe eventually someone turns up, but probably not as quickly as it would happen if it's a fun project. Especially since the implementation may well end up being thrown out (e.g., upon some real world testing of it, it turns out to not work very well from a UX standpoint), how satisfying it is to code it is an important factor.

Take for example, tabledrag.js. That's 1500 lines (to be fair, including docs) of some pretty scary looking code to me. Huge props to everyone who worked on this component over the last decade, with no library assistance other than from jQuery. But I wonder if, for example, something like React would make all the manual binding and triggering unnecessary, and reduce the code to something much friendlier. Maybe it wouldn't, but maybe it's an example of something worth exploring to find out.

And if just something like a tabledrag component requires this much of our own in-house code, what's the barrier to exploring even more sophisticated UIs?

As a (very imperfect) analogy, look at what happened when we moved markup from PHP files (tpl.php templates and theme functions) to Twig. By making working with markup more enjoyable, we got an order of magnitude more front-end developers improving Drupal core's markup during 8.x development than we had during 7.x development. And so, our markup became much better. We didn't need Twig to have better markup, but in practice, it helped by making the work of improving the markup more enjoyable.

Now, I'm not saying we should just blindly pick whichever framework is most popular solely to attract some JS developers to Drupal contribution, just like we didn't pick Twig for being the most popular template language (since it's not). Instead, we should look at the real merits and drawbacks of what's currently out there (frameworks, libraries, etc.) and evaluate whether there's something which has more benefits than drawbacks with respect to our needs. The answer may end up being that there isn't yet, but how will we know without exploring, and this issue can serve to capture people's knowledge that's relevant to answering that question. Or maybe #2651660: Investigate where and how a frontend framework could be used is the better issue for that, I don't know.

If you want to go down the Twig route then pick personas -- if they have real life equivalents even better -- and figure out what they need and then pick your framework that way. Expect a fight. Do NOT let yourself derailed.

Picking personas sounds like a great idea. This issue is frustrating because there are valid arguments for this but they need to get pulled out one by one. And indeed, that would be better to discuss in the other issue.

As for a quick reply on tabledrag that's the one component that might get simplified, it's true it's a pain. Then again we don't have a new tabledrag UX to investigate, no concrete plans to change it from UX team. Since Munich I wasted 3-4 days worth of sprints during events and more at home trying to make a replacement for it. The code was a pain but no UX direction meant I wasted time on useless aspects as well and it ended up not going anywhere. Even if we have a library/framework that is nice to work with, people will make their demo and wait for month for anything to happen because UX issues are still a mess because our process for them doesn't work. Then they won't be making a second demo.

For demos people can pick whatever they want to write their prototype, it's not like we prevent them from using any library in their experiments so I feel this is not that strong an argument. Maybe people don't dare making a demo with something else, but a lot of people don't dare touch core to begin with.

I really want to get behind this issue but it's hard when justification is done piece by piece. With all that Dries wrote about it I'd expect something more fleshed out.

I wasn't very familiar with Ember, the framework Dries indicated seemed the best fit after the initial analysis, so this week I completed a course on Code School to learn more (https://www.codeschool.com/courses/warming-up-with-ember-js). After I finished, I read through the Ember version update notes for the last couple years on the Ember blog (http://emberjs.com/blog/).

Many of the core techniques I learned about from the Ember course have changed in the couple years since that course was created. The Ember project appears to try to make changes gradually, with sensible deprecation warnings and semantic versioning. Their community appears to care about stability, but even so keeping an Ember app up to date would require a decent amount of work to handle all the deprecations between the 1.x and 2.x versions.

Anyhow, what struck me is that we're talking about these client-side frameworks like something we pop in and then we can make a few forms here and there snappier.

That is not the impression I got from the little that I've learned of Ember. It seemed like a nice enough framework, but it seems very much geared towards handling the routing and templating of an entire site, not bits here and there.

There's been a lot of talk about how it's important for our interfaces to render on the server first, then get quick updates based on user interactions. Ember does not seem compatible with initial rendering via Twig, and then just swapping in Ember templates when updates are needed. If we want server rendering, that means using Fastboot, Ember's isomorphic JS efforts. Fastboot isn't entirely stable yet, it seems, but it looks like that is right around the corner. Anyhow, if you want Fastboot, that means using Node.JS.

Ember also seems pretty tightly coupled with HTMLBars, which seems like an updated version of Handlebars from what I can tell. The syntax is pretty similar to Twig, to the point where I think it would be very difficult to use HTMLBars for one part of the page and Twig for another. Both use the {{ variable }} syntax, for example.

It also really seemed like Ember was geared around using its routing system from a single index page, with virtual paths like admin#/structure/content-types used to route in controllers/components.

Another thing to note is that Ember can now do the sort of virtual DOM diffing that React is known for through Ember's new Glimmer engine. This is what allows for the snappy updates that seems to be the goal. That means having JSON with the data needed for these updates, along with the virtual representation of the DOM, stored in client-side memory. So that means fast updates... if the device has the power to handle loading everything. We'd need testing, of course, but I'd have to think that on mobile browsers, in particular, that could be a challenge. We had an entire mobile initiative so D8 admin could work anywhere... I would be concerned how possible that would be with a heavy framework.

It is entirely possible that I am not understanding the full breadth of how Ember can be used. But to me, this does not seem like a trivial amount of work. I have to think the other monolithic JS frameworks that are being considered are going to have similar issues. It may be much more difficult than expected to use a monolithic framework to just replace bits and bobs of the admin UI here and there. I'm skeptical that we could use something like Ember for front-end admin components like Toolbar, Quick Edit and Contextual Links (if we are replacing Backbone) without a lot of challenges.

We're already using a fair amount of JS for our admin. Replacing that with something else doesn't seem unreasonable. But I do think this could end up being a ton of work for a somewhat marginal benefit. I do understand that some of the modern PHP techniques that could allow for faster interactions like ReactPHP and Icicle have server infrastructure implications. But from what I've seen so far, the same is true for something like Ember. If we really do care about content being rendered on the server, Node.JS will very likely be needed for that.

I'm trying to keep an open mind, and I am sure all have the best of intentions towards improving Drupal's admin interfaces. I can't help but feel like we are trying to find a justification for a solution before we truly examine the goals and user needs that would require that solution. I don't think a monolithic client-side framework is a cure all. I also really wish people would stop citing a JS framework as a recruiting tool for front-enders to help with Drupal.

Maybe including one particular JS framework gets people in that framework community excited, but people who prefer another JS framework? Not as much. I'm also not convinced that including a JS framework in a PHP project will necessarily get JS developers involved, particularly when our community isn't on GitHub, like most of the JS community is used to. And not all front-end developers love working with JS frameworks, for some they can be intimidating.

For me, this whole issue has sucked away a lot of the energy and excitement I have had towards Drupal 8. The uncertainty this is generating is draining my personal desire to work on Drupal 8 issues, which is unfortunate since we are in the window where 8.1 improvements can be made.

Emotion is hard to get across in posts like this, so I want to be explicit that I'm not upset or anything, I'm just feeling drained and disheartened about all of this. Anyhow, that's my two cents for what it's worth.

Just came across this article on RAIL, a way to talk about performance metrics: https://www.smashingmagazine.com/2015/10/rail-user-centric-model-perform...
- R: Response (respond to actions under 100ms)
- A: Animation (complete animation frames in 16ms)
- I: Idle (prep optimistic ui in chunks of 50 ms)
- L: Load (load page under 1 second)

This also links to a nice explanation of optimistic UI: http://info.meteor.com/blog/optimistic-ui-with-meteor-latency-compensation

These are exactly the sort of goals that I think are why this discussion is taking place.

For example: "Drupal would be easier to use if performance seemed seamless" with RAIL as metrics used to assess whether or not we are achieving that goal.

What bothers me is that we have jumped right to the notion that we need a client-side framework, which may or may not help achieve that goal. Heavy frameworks may increase mobile load time, for example. First figure out what we are trying to achieve, then look at the different ways we might achieve those goals, then once we settle on an approach, figure out the exact tactics we will use with that approach.

Most importantly, get community buy in each step of the way. Making large changes to a huge chunk of the way Drupal works will take a lot of people to make that happen. If the community is not on board, who will do that work? This is a volunteer effort. Yes, the project lead can make decisions for Drupal, but if the community is not on board with that vision, work will be difficult.

Finally, weigh the level of effort required to commit to this undertaking against the possible benefits, as well as what that means not doing instead. There is only so much we can do at any one time. Getting the admin side of Drupal to meet these sort of RAIL goals is maybe good, but how much effort will that take? What will we not have time to refine in the Drupal 8 cycle instead?

I'm sure there are some that will feel that the resistance to this push for a framework is mere bikeshedding, but I feel like we're being asked what color to paint the bike shed before we've decided whether it's even a good idea to have a bike shed in the first place.

I feel confident that the Drupal community when we work together can do really great things. I would really like to see us to do the hard work as a community to really examine the underlying issues that are driving this initiative before pushing forward. I'm certainly feeling some burnout from this, and I suspect others are as well. If we take a step back to think things through, I think we can stay more energized and positive.

Anyhow, Drupal hugs to you all.

this does not seem like a trivial amount of work

Absolutely right. Refactoring Drupal on top of Symfony components was not trivial. Refactoring theming around Twig was not trivial. Heck, even adding CKEditor to core was far from trivial.

this could end up being a ton of work for a somewhat marginal benefit

People raised the same concern for Symfony, Twig, Wysiwyg, and many other large changes. And they were right to do so, just as you're right for raising that question for this issue. There still needs to be more clarity and precision around what the benefits will be. The discussion in this issue and the related issues is about figuring that out. It took similar discussion to hash that out for Symfony/Twig/CKEditor/etc.

For me, this whole issue has sucked away a lot of the energy and excitement I have had towards Drupal 8. The uncertainty this is generating is draining my personal desire to work on Drupal 8 issues, which is unfortunate since we are in the window where 8.1 improvements can be made.

I'm really sorry to read that. I hope that ends up being a short blip for you, and that you'll rediscover your enjoyment of Drupal 8 soon. But I don't think we should shy away from discussions of important happenings in our industry and if/how/when Drupal should incorporate them. Projects that do shy away from that don't survive and thrive for 15 years, like Drupal has. I agree with you that there's a lot of uncertainty about this entire problem-space, but I think that's all the more reason to be discussing it widely, via a public issue. I really do hope that you'll find the right balance for yourself between working on more well-defined Drupal 8 improvements that you care about and offering your insights to this more chaotic discussion.

What bothers me is that we have jumped right to the notion that we need a client-side framework, which may or may not help achieve that goal...I feel like we're being asked what color to paint the bike shed before we've decided whether it's even a good idea to have a bike shed in the first place.

I think that's a fair criticism of this issue's original title, but I think the new issue title as of #68 and the corresponding comment is a start at incorporating that feedback. As with any issue, the resolution of this one may well end up being "Won't fix" if there turns out not to be a sufficiently compelling case to be made for it.

Drupal hugs to you all

And to you. Thanks for taking the time to write up your insights and concerns.

this could end up being a ton of work for a somewhat marginal benefit. People raised the same concern for Symfony

-- and in the case of Symfony CMF they were totally right. Since the removal of the same path serving different pages for different mime types (html, json etc) the new system offers at most marginal benefits while it's infinitely more complex and scales much worse (but even before that removal the costs far outweighted any benefits). Making routes the heart of everything led to a truly bad and byzantine DX which is hardly surprising because neither incoming nor outgoing links are routes so there's a lot of unnecessary conversion going on. My heart broke the other day when someone said on IRC "cache rebuild on a d8 site with 3000 menu links takes 3 minutes " -- the D6/D7 menu link system has been tested for hundreds of thousands links.

So: tread carefully. Change for the sake of change doesn't always work out.

@mdrummond thanks for the post and looking into ember properly, that helps to frame things a bit.

@effulgentsia:

People raised the same concern for Symfony, Twig, Wysiwyg, and many other large changes.

Well as chx says, Symfony has been a mixed bag. Refactoring on top of some components worked out fine, others created massive amounts of technical debt that has (is) taking months/years to resolve, some were in-between. In at least some cases, the most troublesome conversions happened due to insufficient requirements gathering. The issue was not just that there was a lot of work involved, it was that some of the work done was unnecessarily, or required multiple amounts of additional work later in the cycle to get it releasable.

For example see my comment from October 2012 #1793520-7: Add access control mechanism for new router system pointing out that access requirements for the new routing system had not been evaluated properly (i.e the quite basic point that menu link access depends on routing access in Drupal 7 and earlier and this was not a requirement that could be ditched). This was all before the main routing patch went in, and the fallout from those patches led us all the way up to #2339219: [meta] Finalize URL generation API (naming, docs, deprecation) and many release blocking issues in-between, because we went ahead anyway without stepping back and evaluating whether what was being proposed was actually fit for purpose and where it would eventually lead.

Then similar themes are repeated in #1447712: Evaluate Symfony form component as a replacement for Drupal FAPI which is being discussed this week, where there hasn't been a single requirements-based justification for using Symfony Forms at all.

WYSIWYG is a bit different. I don't remember people objecting to the amount of work involved in adding WYSIWYG to core (although similarly, not many volunteers to do that work). In previous years there's been a lot of push-back against using an editor at all, but that was a conceptual discussion (this issue and the node one reminded me of http://drupal.org/project/live, which was on groups.drupal.org for a time, and was supposed to be a WYSIWYG killer by actually rendering as you type).

The concerns during the Drupal 8 cycle were more about properly integrating any wysiwyg editor with the text format system, and this being a pre-requisite to adding the editor. Since this refactoring actually happened, in the end there was not a lot of controversy about adding the editor itself - it's optional, and core's architecture supports it as well as we could have anticipated. i.e. it was done properly, in a way that did not cause widespread objection. Concerns were less about the amount of work, but that it actually got done, as opposed to slapping a wysiwyg on top of an inadequate system (with the UX and security fallout that would have resulted from that).

Then with Twig, I think everyone knew it would be a massive amount of work, but the sheer amount of work and the relatively late start, meant at the end we were converting individual core admin tables to individual Twig templates just to get them done, which was the exact opposite direction of issues like #1804488: [meta] Introduce a Theme Component Library and #1876712: [meta] Convert all tables in core to new #type 'table'. Many things which should have been consolidated/refactored/reorganised, had to be done as one-to-one conversions to get 8.x releasable. Other ideas like mostly deprecating preprocess ended up getting more or less abandoned, and we still pass $teaser and $page variables to node templates in addition to view mode, actual theme-level infrastructure didn't really get modernised etc. - again because we had serious nuts and bolts release blocking issues to resolve like autoescape.

Back to @mdrummond:

I would really like to see us to do the hard work as a community to really examine the underlying issues that are driving this initiative before pushing forward. I'm certainly feeling some burnout from this, and I suspect others are as well. If we take a step back to think things through, I think we can stay more energized and positive.

To me this sounds like:

1. A higher-level discussion of what exactly Drupal 8's UX priorities should be (going back through usability test results. IA and terminology? Content administration? Field UI? Blocks? All of them?).

2. A bit of retrospective on the 8.x cycle up to now, and what exactly the current state of subsystems which would be affected by this is (existing js, AJAX framework, Twig, routing, rendering, Form API). Also examining relatively quick wins like removing the full server-side render + http redirect we do for all non-AJAX form submissions, which already has an issue open.

3. A commitment to tackling the underlying issues (as was done with WYSIWYG) as a pre-requisite to implementing new features. Whether those are technical (js testing?) or UX discovery, before jumping ahead with implementation of new and shiny things. Lewis Nyman's blog post covered this well.

mdrummond's feeling of burnout reminds me of the massive burnout I experienced when Drupal 8 was branched at DrupalCon Chicago. Fortunately Drupal 8's critical issues are in a much healthier state than 7.x was at this same point after release, and we've committed to not opening Drupal 9 for some time. But there's still a lot of things left over from beta/rc that we're only just in a position to circle back on now. The feeling of being pushed ahead while both contributors and the code base itself are still trying to recover (or 'being asked what colour to paint the bikeshed before we know if we need a bikeshed') is not a good one.

I found myself nodding along with every paragraph @catch wrote. Thanks for writing that, catch. It's a great description because it has the needed nuance yet remains sufficiently high-level.

Drupal hugs to you all.

I have only two constructive things to add, both only tangentially related:

1. @mdrummond: First: extra Drupal hugs for you! Second: I totally understand you feel drained. If/when you want to, let's continue our collaboration on #2061377: [drupalImage] Optionally apply image style to images uploaded in CKEditor 5 to get that moving forward. That'll result in tangible improvements in 8.1 :) I know it'll feel great to get that done for me personally, and I think it will also feel great for you.
2. @catch: RE: Also examining relatively quick wins like […]: I'd love to see what PJAX/Turbolinks would buy us in terms of making the UX faster: #2603794: Add PJAX/Turbolinks support. I think I'll be able to work on that somewhere in the next few weeks. It could of course end up being a marginal improvement. Let's find out.

1. A higher-level discussion of what exactly Drupal 8's UX priorities should be (going back through usability test results. IA and terminology? Content administration? Field UI? Blocks? All of them?).

We have a pretty complete (and prioritized) list over at #2497361: [meta] Fix issues found during UMN Usability Testing 2015, but honestly, if you've seen any of the usability testing results in the past 10 years or so, it's exactly the same boring, tired issues:

• Our terminology is completely arcane and causes Drupal to appear frightfully intimidating to normal web developers. (Not helped one way or the other by JS, will have to be targeted for D9; at best we might be able to play around with a contrib translation in D8.)
• Wayfinding is difficult to impossible. If you want to affect something on the page here, you have to somehow find your way to a completely other page in the admin interface there. — This could be helped a lot by JS, e.g. by expanding places where in-place editing is incorporated.
• Normal people have a huge reliance on previews, esp. when learning a new system, for reassurance they're doing the right thing, and Drupal shuns previews at every turn, except in very specific circumstances (node/comment forms, block UI (if you manage to find the special link, Colour module, Views (if you manage to scroll down enough to find it) - JS could help tremendously here, esp. if we start rebuilding UIs around "preview first," which would definitely be the preferred/modern approach.
• An overwhelming number of choices is presented to the user at every turn, with no clear hierarchy as to the importance of the options, thus the user is either paralyzed by choice or spends ages independently evaluating everything to see if it's important or not — Does touch JS, e.g. progressive disclosure
• The "russian dolls" / "order of operations is backwards" problem — in order to build a "page," you need to build some discrete bits of content, which is placed into some number of views, which is placed into some number of blocks, which is placed into some number of regions, which is placed into some number of themes. JS could help a lot with this as well, by shooting off to the relevant APIs but keeping a single central UI/dashboard for building things out (or not, but at least we could explore what UIs like this could look like).
• There's no example content in Standard profile which leaves new users baffled as to where to start — Not a JS problem.
• Most problematic UIs in D8: block UI (regressed from D7), field UI (regressed from D7), Views (always confusing :D). All of which could benefit from JS, esp. Block/Field which currently take like 45 clicks to get a simple thing done.

And frankly, I am sick of hearing about these issues. Usability testing Drupal has actually become super boring. We already know all the shit we need to fix, because it's the same shit we've needed to fix for the last decade. I'm sick and tired of looking at something like SquareSpace and feeling envious and sad. There's no reason Drupal can't have a UI as beautiful and elegant as the API under the hood. We need to remove blockers to getting it done.

Part of those are indeed process-related blockers, and that's definitely on my radar and I've been working with Bojhan and others on some proposals (which sadly got completely derailed due to the untimely end of my 17 year marriage over the past few months :\) to help with that. See the end of https://events.drupal.org/barcelona2015/sessions/making-drupal-better-ou... for a sneak preview.

But I think another huge part is that if you talk to your average JS developer (and Drupal core developers who are fluent in JS are not your average JS developers ;)) they wouldn't touch something like Drupal with a 50-foot barge pole. Our main JS APIs (e.g. states, AJAX, etc.) actually go significantly out of our way to not write any JS code if you can possibly help it, in favour of Drupalicious #properties. :P Because we're largely PHP developers, and why would we want to touch that smelly stuff? ;)

But that's not the way the web is trending. Normal web developers would look at the set of UX problems we have and the set of idealistic UIs we'd like to build, and reach for better tools. I think it's perfectly fine to explore what those better tools might/could look like while we're also attacking process problems and also coming up with next-gen UIs. We definitely don't want to waterfall this, or D9 UX testing is going to be just as boring as D6/D7/D8. :P

This comment sent me down memory lane.

if you talk to your average JS developer (and Drupal core developers who are fluent in JS are not your average JS developers ;)) they wouldn't touch something like Drupal with a 50-foot barge pole. Our main JS APIs (e.g. states, AJAX, etc.) actually go significantly out of our way to not write any JS code if you can possibly help it, in favour of Drupalicious #properties. :P Because we're largely PHP developers, and why would we want to touch that smelly stuff? ;)

Off the top of my head I can think of two full-time Drupal developers I have worked with who have left the Drupal world completely in order to write Javascript all day every day. Nevermind the mass exodus of Development Seed.

To me convincing this group to come back to Drupal feels like an impossible task, and would be a far easier task than convincing someone entrenched in the Javascript world who has never heard of Drupal to give Drupal a chance. No framework choice solves this problem, because it is not entirely a technical best-tool-for-the-job problem.

We need to face the hard reality that there are only two ways to get entrenched Javascript devs to use Drupal.
#1 Convince their boss that Drupal is a necessary part of the stack that needs to be forced on them.
#2 Focus on making Drupal a super-robust (and speedy) data-serving machine so that these devs don't even notice that it is there (this helps with #1).

#1 won't make us a lot of new friends but #2 definitely could.

Moving a big chunk of our JS code to a framework will not bring members of that framework community into ours to come solve our problems, especially if that code is still all sitting over here in our sandbox, but it MIGHT make some of our existing front end developers not rage-quit Drupal.

Keeping this small and dwindling audience happy is my personal number one priority in this decision. If we start to see a lot of excitement from the front-end Drupal community (like we did with Twig) I'm all for taking on any amount of work to make this happen. Right now I'm not seeing or hearing any level of excitement about this move.

See, I come at it from a different angle. If we acknowledge that the Development Seeds of the world, and the friends both you and I know who have left Drupal to pursue JS are not just weird "out there" flukey people but instead representative of the way the entire web is actually trending, then we have no choice but to make moves that make Drupal friendlier to non-Drupal JS developers. (Not in the hopes of attracting magic new contributors, which keeps getting bandied about for some reason, but just as a matter of survival for our project.) Just as we had no choice but to make Drupal easier and easier to use for content authors/site builders, as our competitors do the same. Otherwise, our project becomes a relic and one of the core strengths of Drupal has traditionally been not being afraid to stare down and tackle tough problems and keeping on the cutting edge of technology. I don't want to see us lose that.

And to be 100% clear, no one here is out to burn anyone else in the context of this discussion, so comments from Marc and other contributors who've given their all to D8 and feel this way do make me sad and worry. :( But at the same time, we cannot possibly make a fundamental technical decision for our project based on emotional arguments. We do need to acknowledge the concerns being brought up in this discussion (and elsewhere) as valid (and there are a lot of such valid concerns), and I feel like we are all trying to do that: to really listen to what one another has to say. And, to Dries's credit, the whole progressive decoupling discussion is an attempt to appease both audiences and gain the best of both worlds—client-side friendliness (including for fully-decoupled architectures) + not throwing our baby out with the bathwater. So I'm not understanding the resistance to at least exploring this.

I don't think there is resistance to exploration, but I am hearing a lot of fear about yet another front-end decision being foisted onto front-end devs by people that don't spend much if any of their time working in that layer.

I'm all for people passionate about the front-end of Drupal and well versed about these frameworks making an educated choice about how Drupal competes and stays relevant in this landscape. I just want to make sure that front-end devs are well represented in this discussion and driving the decision. As someone who has been out of the front-end game for a number of years I don't pretend to have the answers here, but can empathize with how they are feeling having been on the outside of these types of issues looking in in the past.

Letting front-end devs guide the decision is the only way we avoid making more Drupalicious decisions that drive front-end devs to start framework shopping on their own.

If you identify as a front-end dev, please speak up! I can only hope you are aware of this issue and are following along with the rest of us.

Reading the comments… I declare the issue summary off topic :þ

Read comment below instead of this one, much better than what I wrote.

I think we're all fine with exploring things, staying relevant technically and not bumming people out.

So I'm not understanding the resistance to at least exploring this.

The issue did not start as an exploration: "Select a new client-side framework for Drupal core" is some Rambo-grade explorating. Follow that by barely satisfactory arguments being given one by one and a little all over the place, it does not inspire confidence and make it look like a fearful reaction to… something — especially since apparently we "need" to get something in "fast" — so we definitely agree with you and: But at the same time, we cannot possibly make a fundamental technical decision for our project based on emotional arguments. Progressive decoupling is a nice idea but why select a framework before we even have a proof of concept somewhere so know what we should be careful of? There is something we absolutely have to waterfall and it's figuring out Why/How/Where before What, since it sucks to work in a vacuum.

I don't think anyone is dead set against progressive decoupling, but please at least let's agree on which kind of hammer we need before we discuss which brand we want to use.

Adding thoughts from my perspective as an infantry-level contributor, who works towards executing the strategic and tactical decisions determined by core maintainers, initiative leads, and sub-system maintainers:

Answering @webchick in #83:

So I'm not understanding the resistance to at least exploring this.

@mrf's response is #84 is a good start:

I don't think there is resistance to exploration, but I am hearing a lot of fear about yet another front-end decision being foisted onto front-end devs by people that don't spend much if any of their time working in that layer.

For me, the immediate reaction is resistance because the initial proposals are not phrased as questions ("What are some ways to solve X with Y?" or "Could X be solved with Y?"), but as foregone conclusions - "Add X which solves Y". Which leads to the response "Is X the best solution for Y?", which then appears to be resistance but is really the question we need to be discussing.

"Add X to do Y" is a good title for a task when the solution has buy-in and has been vetted by the whole range of stakeholders (from BDFL to the volunteers expected to be doing the work), but it's not so great when it appears (rightly or not) to come out of the blue. A better title might have been "Do Y" - which would initiate discussion of how to solve a particular problem and not immediately polarizing people into being for or against a particular action.

representative of the way the entire web is actually trending

There's more to the web than Github. Since 2008 I've been building primarily marketing/communications and academic websites using Drupal exclusively and don't have a single project hosted on Github. Where is HTML and CSS on that graph?

#2 Focus on making Drupal a super-robust (and speedy) data-serving machine so that these devs don't even notice that it is there (this helps with #1).

Isn't that already one of the benefits of D8 today? Sorry if I'm mistaken -- I haven't been following D8 development that closely.

To me, Drupal is a fantastic content management system, especially for websites that need richly structured content. I'd wager that the main reasons entrenched JavaScript developers don't want to use Drupal is because they like working on projects that don't need a CMS, and they don't care about richly structured content. Didn't DevSeed leave Drupal because they enjoyed working on non-CMS projects, and using a static website generator for content when needed? That's fine -- it's just a different sort of project than the typical CMS-based marketing site that Drupal is so good for. If I ever need to build a non-CMS web project I probably won't use Drupal. Isn't that OK?

I appreciate the kind comments and personal concern shown in this thread. However, I also shared technical concerns and thoughts that I think are worth restating:

• If we want client-side responsiveness (less than 100ms for example) in addition to fast page loads (particularly on mobile), that likely means we need isomorphic JS
• isomorphic JS is going to likely require that we run NodeJS either in addition to PHP but more likely instead of PHP
• Even if we are focused on admin tools, some of those admin tools run on user-facing pages: Toolbar, Quick Edit, Contextual Links, etc. Comments have also been mentioned.
• User-facing admin tools could maybe be made more responsive with a client-side framework, but again... for quick loads, that means isomorphic and thus possibly NodeJS.
• If we are running NodeJS on user-facing pages in order to speed up those admin tools, it may be difficult or impossible to do so alongside PHP rendering, which could force JS rendering of user-facing pages purely to improve admin tool responsiveness
• As others have pointed out (perhaps more on the NodeJS thread), some of the admin tools may need to be changed pretty far back in the toolchain to run in JS. It's not impossible that we find that more and more of Drupal needs to be converted to JS to make admin tools more responsive
• At some point it may seem silly to have half of Drupal built in JS and some in PHP. Thus, all of Drupal is converted to JS.

Maybe that's an extreme scenario, but I do not think it is implausible. There are huge technical implications and a huge amount of work that could be required simply because there is the notion that if we just use client-side frameworks for our admin, our UX will be better and front-end developers will like to work with Drupal more.

Drupal written entirely in JS I guess could still be called Drupal, but that's a pretty big change for a whole lot of people's lives that are predicated on their knowledge of PHP as a core part of their careers. I don't think it should be all that surprising, when such a big change is one possible result of this thread, that people want to make really, really, really sure that doing so is a good idea.

markabur: I think it's more accurate to say that being a "data serving machine" was a key goal of Drupal 8, and we made huge progress in that area but there's still plenty more to do. Also, there's no one-size-fits-all solution in that space.

And yes, using non-Drupal for a non-CMS project is totally legit. Making it easier to do so is one of the benefits of having a more conventional architecture now, so the relearning for backend devs is less. That doesn't mean having a more robust JS layer for the admin is a bad thing for a CMS; I agree that it can be a big help.

The problem does seem to be that the jump from "a more robust JS layer" to "let's pick one of the big frameworks and rewrite everything in it" was made, or implied, very prematurely.

I follow this discussion since the start, but I said is not a decision I can influence so I will just wait to see a conclusion, if any.

But now I will give my 2 cents and I hope you will forgive me my rambling and my English which tend to degrade rapidly when I'm rambling.

A few years ago when I was told to implement some javascript on front end I always said "sorry, I can't" so my experience was importing a library, make drupal output the markup the library desire and implement the code required to initialize it which was copy paste.

Through the years that change and now I've built full sites back-end and front-end in javascript (node.js and angular 1).

Crell's presentation in Barcelona I think was on point. A lot of people are drawn to node.js for the "live" feelings they have. A user register to my site and if I'm on the users page I can see a new row appear in my table. I can send notification through a web socket and they appear live.

To achieve that using Drupal I would need to set a javascript which regularly calls the back-end asking "is there anything new?". And I need to query the users table to see if anyone new was added since last request. If it would work in a node.js way you could on hook_user_insert() send that request to everyone listening. So not only node.js doesn't need to "start-up" every time, but it doesn't need to query the table for everyone listening.

Regarding the SPA's, that's perfectly achievable in Drupal. a SPA still queries the back-end for each page and receives a json. You can use ajax on every link and button in drupal if you really want.

I think the focus should be on making that easier. If I want to request the page throug ajax to first have an option of receiving a full json, no rendering done, or json with stuff rendered. IE

{
sidebar_first:
  {blocks: [{
    title: "<h2 class=\"title\">Some Title</h2>",
    content: "<div class=\"bla-bla\">stupid block content</div>"
  }]},
main_region: {...}
}

You get the idea I hope. I would also like something similar for forms (note that I didn't look into D8 that much, so if it already exists ignore me).

Last year we built a site using D8 beta 4 and angular 1. I thought sending a node to be saved was overly complicated.

save: function (node, fn) {
    var data = {
        "_links": {
            "type": {"href": window.location.origin + "/api/rest/type/node/nodetype"}
        },
        "title": [
            {
                "value": node.title.value
            }
        ],
        "body": [
            {
                "value": node.body ? node.body.value : ""
            }
        ],
        "field_email": [
            {
                "value": node.field_email ? node.field_email.value : ""
            }
        ],
        "field_website": [
            {
                "uri": (node.field_website && node.field_website.value) ? "http://" + node.field_website.value.replace('http://', '') : ""
            }
        ]

    };

    var headers = {
        'Accept': 'application/json',
        'Content-Type': 'application/hal+json',
        'Accept-Language': 'en-us'
    }
    if (node.token) {
        headers['X-CSRF-Token'] = node.token;
    }

    $http({
        method: 'POST',
        url: '/api/entity/node',
        headers: headers,
        data: data
    }).success(fn)
      .error(fn);
}

Probably there was a reason for it, but why so complicated? For someone working with JS I would have expected something much simpler for the data. Why do I have to figure out that link ("/api/rest/type/node/nodetype")? Why not just send the request to /api/entity/node/nodetype or just declare the node type in the array along with the other values? Why the need to repeat "value" over and over, or uri? The back-end should know that field saves the data in a URI, or value or target_id. I would have liked my data to be something like:

var data = {
        type: "nodetype",
        "title": [
            node.title.value
        ],
        "body": [
            node.body.value
        ],
        "field_email": [
            node.field_email.value
        ],
        "field_website": [
            node.field_website.value
        ]
    };

Like others have said we should focus on changing some of the UI we have to use more ajax and see what issues we find and solve them. Then we can talk what framework (if any) we should use. Me personally I can get behind any of them even if I didn't use them yet because really is not that hard. Even in using node.js the more complex stuff was on back-end and the fact that there is nothing like Drupal or WordPress in node.js world makes this even more difficult.

Regarding the usage of Javascript. Angular 2 is in beta and I see of lot of tweets about yet another boilerplate, this is using X, Y and Z, while the other uses U, X and Z. On top of that every little bit of tutorial will end up with "you can get the code on github at ..." even if it's some basic filtering, or a todo list application. This doesn't happen with any other language.

@webchick my point wasn't that we should do more usability testing (except when we have new things to test). The proposal does not at all mention those results and how a framework might or might not help fix those issues, but just mentioned facebook/twitter/google. As your list makes clear, some of our issues cannot be solved with js at all, some possibly could. As nod_ points out, @akalata's comment explains the concern better than either of our comments.

Also several people on this thread have already pointed out that the github stats are skewed towards fragmented projects (i.e. lots and lots of personal forks and duplication), and don't necessarily represent health at all. We really need to look at health/quality rather than pure numbers - both tumors and flowers 'grow'. If we moved Drupal.org project hosting to github and a 100% pull request workflow, that in itself might be enough to juke the stats in favour of PHP (i.e. assume one fork per patch contributor to each contrib project or core and you suddenly have tens or hundreds of thousands of forks after a year or two). It wouldn't represent a flowering of PHP at all though.

@mdrummond in case you've not seen it, a 'use node.js' discussion is happening at #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS. As you point out that's one potential conclusion - all rendering happening in js, both client and server-side, and PHP only providing a REST API for that to interact with.

I think it is worth taking a step back to discuss the actual goals we want to achieve and the various ways we might want to achieve them outside of an issue is geared towards one possible solution for those goals.

So I opened a new issue: #2655556: [META] Dramatically improve Drupal-provided user interactions through perceived performance boosts and optimistic feedback

I hope this is taken in the spirit with which it is intended: a recognition of the importance of addressing the core issues at stake here.

I do not in way mean to diminish all of the good and important work that has been done to evaluate a variety of client-side frameworks. I do care about the goals at stake here, even if I have concerns about the suggested approaches. I'm hoping that a fresh start to the discussion can help reset things and calm the frustrations that many—including me—have felt. I'd humbly suggest that the discussion about what approaches we should look at might better fit with this new thread, while evaluations of the various client-side frameworks (even if we end up not using them) might better fit in this thread. That's just my opinion, though, so continue on as you please. :)

A recent episode of Full Stack Radio (podcast) (http://www.fullstackradio.com/32) had DHH (of Ruby on Rails fame) on to talk about Basecamp 3, and included some discussion of the front-end vs. back-end rendering question. His take-away: Rendering happens on the backend, duh, but use a websocket to send it back to the browser as quickly as possible.

The directly relevant portion is from about the 18-31 minute mark, but the whole episode is worth a listen. (He even has positive things to say about PHP! He's mellowed in recent years...)

Thanks Crell. That was a great podcast episode.

Some points that stood out to me:

• The importance of using a technical architecture that matches the people building the software. DHH mainly discusses this concept in relation to team size (Microservices work well for very large teams or teams of teams).
• That Turbo links go a long way toward fulfilling the experience of an uninterrupted application. #2603794: Add PJAX/Turbolinks support
• The high cost of having duplicate template files as screens multiply: having different templates for client-side/server-side would be unmanageable for a project as large as Basecamp with their small team.
• To build a product in a timely manner trade-offs must be accepted. "How do we get this $22,000 car to feel like a $32,000 car." To compete with products from companies with much larger budgets Basecamp must be much more deliberate about the trade-offs it makes.
• If you have all the resources in the world you can write tests at every single level. DHH prefers having enough testing to make him confident in the product. "Testing against controllers and views at the same time gives you a lot of bang for the buck."

If you look at the discussions there is a trend, that there is no clear discussion, because we are talking about various things at the same time.
Both improving the admin UX, the frontend UX, the developer experience, finding new contributors and expanding the entire community as
well as the search of a new fundamental base for Drupal itself.

You could have a much better discussion if we first discuss about personas of Drupal. Who are the people we want to help with:
Here are just a couple of questions, which don't define the personas at all. I just want to give an example of the range of users we have

* The individual who wants to setup a site
* The freelancer who want to build a site for their clients
* The agency who wants to work on big products

* Site users having an easy time to find good information on sites without technology barriers
* Content authors having a good time to enter information as best as possible
* Content authors having a good time to create the site as best as possible
* Marketers having the power to display what they think is important
* Site builders being able to achieve great things without a necessary technical background
* Ensuring a good life for the Drupal community?

If you look just at those examples, it seems to be that this touches the Mission and Principles
of Drupal. Maybe its just time to rediscuss that. Maybe we want to value the content authors over the minimal requirements? I strongly believe
that we first need to agree on that fundamentals, then go to personas and then maybe come back to the technical discussions we have at the moment.

For me this is not a technical problem at the moment: #2661180: Rediscuss the mission of Drupal

#93: Thanks for that link! Note that Rails uses Turbolinks to make loading content/switching between pages as efficient as possible, without having to deal with lots of JavaScript. I plan to work on that in the next weeks/months, and I've already posted a screencast of a very rough PoC, in which it looks quite promising indeed. Thanks @twistor for already linking that: #2603794: Add PJAX/Turbolinks support.

I also listened to the podcast. I heard a few interesting statements not yet mentioned in #93 or #95, but especially relevant to this whole discussion.

1. We consider server-side rendering as the baseline. Notifications are loaded via websockets. New pages are loaded via AJAX [because they use Turbolinks]. The next step (client-side MVC + JSON responses) is only taken if server-side rendering can't get fast enough: when there are fidelity issues. Example: datepicker, WYSIWYG.

2. But what I found, is that there is a bunch of accepted wisdom: "You HAVE to use client-side MVC" — I actually have found the opposite: only the minority of the time you need that. Of course, a WYSIWYG editor that'd have to go the server for every keystroke, that wouldn't work. But for lots of other cases, such as menus, server side works awesome.

3. Turbolinks is a layer that simulates in some cases the client-side MVC feel, or at least this notion of "the persistent process" [talking about the UX], without requiring you to do all this work on the client side.

4. That's also how we use websockets — even individual [Basecamp] chat lines are rendered on the server side and sent down to the client. Which then means we can reuse those templates for all sorts of other things such as transcripts or search.

EDIT: cross-posted with #96.

Added #2573119: [plan] Determine process for big UX changes

Vue.js is the 3rd most popular JS framework on Github and might be #2 or #1 by the end of 2017. It's worth considering, if it isn't too late.
https://github.com/showcases/front-end-javascript-frameworks

If a Turbolinks feature gets added to Drupal core, please keep it off by default. Turbolinks are really unpleasant from a user standpoint (at least this user), making the normal loading process appear very slow. I think it's because the width of the progress bar is the entire screen, giving the impression that the loading process has a long "distance" to go.

I cringe every time I see those that turbolinks loading bar (including YouTube and Github). If Drupal adds turbolinks to core and puts it on by default (or recommends it), it will make Web surfing more painful.

Just want to throw in glimmer.js as an option. It is just the View layer of Ember.js and could be a good way to dive into the frontend framework world.

WordPress is apparently ahead of us on this, and has narrowed their contenders to React and Vue: https://wptavern.com/wordpress-core-javascript-framework-debate-heats-up...

yeah, there's a whole bunch of work going into Gutenberg and Calypso and all of it seems to be React based. Automattic is heavily invested in React (and hiring React devs). I'd say they're pretty likely to go down the React road.

WordPress is making a mistake with that direction due to problems with the PATENTS file in Facebook's repos. I think that things like React and Clojurescript (usually depends on React, Immutable, etc.) shouldn't be put into Free software projects. There are companies that can't use React, so including Facebook code could negatively affect Drupal adoption.

Just a small story to remind everyone that software actually enables people ... I was living in Berlin last year. Due to actually sane politicians refugees from various places had the chance to start a new life in Germany, of which many ones started to life in Berlin. I can't even imagine how hard it has to be for those people leaving their home. Anyway, what really made me thinking is that people started to form groups to teach refugees how to code. They started building websites using HTML and CSS and then continued learning JS, Node and then went the frontend route using either react and vue, all the other contenders have been pretty much negligible in my biased view. Anyway, these people started to take part time gigs from abroad, making some money, learning more and more and finally ending up in startups or other places. This framework discussion just reminded me of it, as it has a similar direction as the blogpost referenced by @webchick, so I posted this as a comment.

TL;DR
Refugees are able to start new lives due to them learning to program.

@J. Cohen One can always use Preact if the license is critical. I can also recommend cycle.js because of the super fast virtual DOM library
Snabbdom and because of its reactivity thanks to xstream.js (or RxJS or most.js – whatever one likes).

This is the problem with React patent licence: If you or your business have a legal conflict with facebook the react licence is automatically revoked, and your business may have a legal conflict with Facebook that has nothing to do with react and lose its license.

https://github.com/facebook/react/blob/master/PATENTS
The license granted hereunder will terminate, automatically and without notice, if you (or any of your subsidiaries, corporate affiliates or agents) initiate directly or indirectly, or take a direct financial interest in, any Patent Assertion: (i) against Facebook or any of its subsidiaries or corporate affiliates, (ii) against any party if such Patent Assertion arises in whole or in part from any software, technology, product or service of Facebook or any of its subsidiaries or corporate affiliates, or (iii) against any party relating to the Software.

Note about Preact, after usual IANAL disclaimer: if patent licensing is an issue, changing the code base while implementing logic similar to the original (as Preact does) does not make one free from the patent scope. Even pure cleanroom implementations or independent reinventions ex nihilo wouldn't be exempt, as patents do not cover the implementation but the mechanism. The only gain from this would be possible protection from the loss of license to React, assuming Preact is not deemed infringing.

Note that Vue already has integration with Laravel PHP framework and does not come along with the legal baggage. It has a good community, is easier than React to pick up, and also has the "shiny" factor.

As a non-JS-centric developer, I've chosen Vue for work-related projects and I'm happy to say my brain remains intact even after long exposures.

As a non-JS-centric developer, I've chosen Vue for work-related projects and I'm happy to say my brain remains intact even after long exposures.

LOL. :D A shining endorsement!

FWIW, we're using a Vue+Axios combo on a french D8 e-commerce site to be released in the coming weeks, for the combination of: patents issue, download size for users with limited bandwidth, ease of D8 integration/lack of conflicts... and the shiny factor too. In roughly decreasing priority order.

Using Vue seems like an obvious choice. It's easy to learn, renders very quickly, and already has a lot of adoption in the PHP community. React's PATENTS problem is just getting mainstream attention now, and it's likely to cause a decline in React usage. It's popular now, but I think it would not be a good idea to bet on it.

If you haven't tried Vue.js, run through these free guides to check it out:

• https://vuejs.org/v2/guide/
• https://laracasts.com/series/learn-vue-2-step-by-step

One other advantage of Vue is that you don't need to compile JSX or have any build process. You only need to write some simple JS like this:

<div id="widget">{{ greeting }}</div>
<script>
var vm = new Vue({
    el: '#widget',
    data: {
        greeting: 'Saluton Mondo'
    }
});
</script>

VueJS is simple at start but with vuex and over tools can be extended too great SPA framework. For big SPA project there need strict rules for folder structure. Used on SPA projects already. Also builded SPA with Angular 2 which have big disadvantage - can`t be same thing simple as @J. Cohen described.

Tentatively un-postponing this, at least for a minute or two. ;)

I know back in February 2016 when we were originally discussing this, people were highly opposed to settling on one framework or another, for a variety of valid reasons. And we've largely spent the past 1.5 years since then doing more "under-the-hood" work in core: countless REST API improvements, stable Media API, stable Layout API, and so on.

However, we're now on the cusp in 8.5+ of wanting to put a lot more user-facing functionality in core (see proposed roadmap discussion at #2905741: *DRAFT* Proposed product goals for Drupal 8.5/8.6(+) core).

Among them:

• A dynamic layout builder
• A content staging UI
• A simple media library

Unless we decide something here, these UIs are all going to end up being written in "Walkman" mode (a.k.a, PHP-based front-ends, full server round-trips). Whereas it sounds like where things are generally trending is "MP3 player" mode (a.k.a., decoupled, API-driven, responsive UIs). And this becomes later technical debt with no visible user-facing benefit when we eventually do convert them to "MP3 player" mode $whenever on $whatever framework.

If, OTOH, we were to land on something in this issue (Vue or otherwise) soon, it could have a dramatic effect on what we're able to do for these and future user-facing functionality in Drupal core. It would also empower contrib to build modern UIs as well. And finally, it would finally get us "dog-fooding" our REST API, which is sorely needed, given the scope of things they're trying to fix.

And I worry a lot that the longer we go without making a decision here, the more dated our core UIs are going to look, the more fractured contrib is going to be, and the more we'll fall behind competitors. :(

*cringing and hitting Save* :D

This is the problem with React patent licence: If you or your business have a legal conflict with facebook the react licence is automatically revoked

There's a lot of ambiguity around this because the license is poorly worded, but many people don't interpret it so nefariously. Rather they interpret Facebook's "BSD + Patents" React license as giving you more rights (or at least more explicit rights) than a standard BSD license would.

The people putting forth that argument include Automattic's lawyer (https://github.com/Automattic/wp-calypso/issues/650#issuecomment-235086367) who has a pretty good incentive to get this right. And certainly based on a plain reading of https://github.com/facebook/react/blob/master/LICENSE combined with https://github.com/facebook/react/blob/master/PATENTS it looks that way - it is saying that you get the rights conferred by the BSD license plus an additional right beyond that to use Facebook's patents, but the latter right is not absolute and can be revoked.

Where it seems to get complicated is the question of to what extent the BSD license on its own has an "implied patent grant" and to what extent that gets superseded/weakened by the extra language that Facebook added. It seems like no one has a definitive answer to this because it has never been litigated (http://lu.is/blog/2016/10/31/reacts-license-necessary-and-open/ is an interesting read).

Bottom line, the React license would introduce some ambiguity/risk, but the bigger risk may be the perception people have of it, rather than the license itself. But from Drupal's perspective the perception arguably matters as much as the reality...

There's a lot of ambiguity around this because the license is poorly worded, but many people don't interpret it so nefariously. Rather they interpret Facebook's "BSD + Patents" React license as giving you more rights (or at least more explicit rights) than a standard BSD license would.

No one who argues that the PATENTS file is safe has ever addressed the case of what happens if Facebook (or a company that they acquire) infringes on your patents. If you try to defend them, you violate the PATENTS file. If you don't defend them, you could lose the patents. If someone would address that point, then I think their arguments could be taken more seriously.

Software patents are terrible, but the battlefield should not be unfairly tipped to Facebook's advantage. Facebook's arguments that they are giving you more rights sound highly disingenuous.

Many companies can't use React or other Facebook libraries, so I don't think that they should be used in Free software projects.

It's not obvious to me why that tips anything in Facebook's favor. Basically, if you sue Facebook for patent infringement (whether frivolously or not), Facebook is reserving their rights to turn around and sue you back (whether frivolously or not). At which point it would go to court, and the court would decide which claims have merit and which don't, based on the particulars of the case.

So it's not obviously asymmetric to me, nor is it obvious how things would be any different if they had just released it under a regular BSD license.

Not really trying to defend Facebook here (I don't fully trust them in general) but in this case there are plenty of lawyers who owe no allegiance to Facebook who don't see a problem with the license itself. (Whether there is a problem for Drupal due to various people's negative reactions to the license is, again, another question altogether.)

So. How about Vue? ;) There are like 7 comments here in rapid succession that actually agree on something, and that's unprecedented for this issue. :D

It's not obvious to me why that tips anything in Facebook's favor. Basically, if you sue Facebook for patent infringement (whether frivolously or not), Facebook is reserving their rights to turn around and sue you back (whether frivolously or not). At which point it would go to court, and the court would decide which claims have merit and which don't, based on the particulars of the case.

It isn't Free software if your right to use it can be revoked. Facebook isn't just reserving the right to sue you back -- you lose the license.

The license granted hereunder will terminate, automatically and without notice,
if you (or any of your subsidiaries, corporate affiliates or agents) initiate
directly or indirectly, or take a direct financial interest in, any Patent
Assertion: (i) against Facebook or any of its subsidiaries or corporate
affiliates, (ii) against any party if such Patent Assertion arises in whole or
in part from any software, technology, product or service of Facebook or any of
its subsidiaries or corporate affiliates, or (iii) against any party relating
to the Software.

I agree with webchick here. :)

And I agree with the others about Vue, I have successfully used it for both small and big projects and find it very easy to use and adopt into projects.

@J. Cohen: No, they're revoking the patent license in that scenario (which is what allows them to potentially sue you back). They're not revoking the software/copyright license. It's still free software which can be used and distributed under the BSD license.

But yeah, how about Vue? :)

I'm very happy to see Vue getting some much deserved love in this issue and in the WordPress thread. I'm surprised, however, that in that WordPress thread that Glimmer.js wasn't discussed at all. In this issue, #105 mentioned it, and I'll follow up tomorrow with some thoughts on why I think Glimmer deserves at least a consideration. Meanwhile, if anyone can offer insights into their pros/cons relative to each other, that would be awesome.

A few things I like about Vue.js.

1. Vue.js has very low learning curve. This may be critical point for Drupal developers because they are mostly not JavaScript focused. It would take you literally 1-2 days to get started and begin use it to enhance your Drupal applications.
2. Vue.js still allows building front-end applications without compiling. As in old days with jQuery you can simply add <script/> to your page and it will work immediately. It is quite useful for applications that are not fully decoupled. But yet for large scale applications build process with NodeJS tools like Webpack and Browserify is fully supported.
3. Vue.js is small. (80 KB - minified, 29KB minified + gzipped). Even with Vuex and Router it is much smaller than React or Ember.

As a practical example. Here is a simple CRUD application I created when was learning Vue. It took less then 100 lines of JavaScript code.

It's possible that React's patent clause is incompatible with GPLv2, which would make it impossible for us to use to begin with. (It MIGHT be compatible with GPLv3, which has a patent clause, but I haven't looked into it so can't say either way.) The Apache foundation has already ruled that it's incompatible with Apache v2, which has a light patent clause.

IANAL TINLA, but from a legal perspective I'd be very reticent to touch React for core. It could land us in a mess, legally.

I have no opinion about either framework from a technical standpoint.

FWIW Facebook seems to have clearly declared that they won't be changing the patents licensing for now: https://github.com/facebook/react/issues/10191#issuecomment-323486580 including mention that they feel other companies publishing open source software should do the same.

Disclaimer: I'm not a JS developer, and I haven't used either Vue or Glimmer, so take that into consideration in evaluating this comment, but per #128, I think both Vue and Glimmer deserve some serious consideration.

• They're both lightweight. Minified and gzipped, Vue is 30KB and Glimmer is 34KB.
• They both can be expanded to enable more ambitious projects: Vue with vuex and vue-router; Glimmer to Ember. The latter is currently not as piecemeal as would be ideal, but they're working on that, and I'm fairly confident that they'll succeed at it soon enough for our needs.
• I think they both can be used to create Web Components that can be consumed by other modern JS frameworks. Here's a blog post I found on it for Vue, and here's what I found in Glimmer's docs and github issue. I think this is a really important consideration, so I'd love it if anyone with experience with this can weigh in on if there's relative strengths/weaknesses with each one at doing this for advanced use cases. Because whatever we include in core, there will be some Drupal users who will want to build sites, apps, and custom administrative experiences, in some other JS framework. While #120 mentions some core admin UIs that might not need integration into those custom apps, let's for the sake of argument consider as another use-case the possibility of rewriting Drupal's Contextual Links module's JS from Backbone to whatever we pick in this issue. It would be nice if that could then be packaged as a Web Component that integrated very smoothly into people's Angular sites, React sites, etc.

However, here's where I think Glimmer might serve us better:

1. #120 lists UIs that might not need very much more than the rendering engine provided by Vue or Glimmer. But let's suppose that someone eventually has the impulse to rewrite Views UI as an SPA (whether as a contrib project or as an eventual proposal for core). I think that Ember has more available in it (e.g., a sophisticated router, dependency injection, maybe more?) to enable that than does the Vue ecosystem (vue-router and vuex). I might be wrong on that and perhaps it will change with time, but I think it's worth considering. I think it would be great if whatever we pick for core becomes something that truly can scale up for contrib and/or future ambitious UIs in core.
2. Speaking of scaling up the ambition, check out this presentation of CardStack, an Ember-based project for providing even more amazing integrations between a JS front-end and various back-ends, including Drupal. Especially the 2nd half of that talk. Some of that might be great for Drupal core some day, but even if none of it is, the fact that several of Ember's core developers are working on such a project I think provides an amazing opportunity for some cross-collaboration between us and them. And I think that collaboration would be enhanced if Drupal's JS community becomes highly proficient with Glimmer, and if the administrative components that core builds are built with Glimmer. Perhaps Vue is just as technically capable at supporting such a project too, but I'm not aware of Vue core developers working on such a project, and for me, that's a significant differentiator.
3. And finally, Glimmer uses handlebar templates, whereas Vue uses v-* attributes for expressing conditionals, loops, etc. Leaving aside personal preferences, the handlebar syntax is closer to our existing Twig templates. I think this is helpful, because for certain administrative UIs, I think we'll need equivalents of our Twig templates expressed for JS consumption. For example, status-messages.html.twig, image-formatter.html.twig, and possibly a bunch more. It seems to me that the closer in syntax that such duplicates are to each other, the easier it will be for us to maintain them.

They're both lightweight. Minified and gzipped, Vue is 30KB and Glimmer is 34KB.

They both can be expanded to enable more ambitious projects: Vue with vuex and vue-router; Glimmer to Ember. The latter is currently not as piecemeal as would be ideal, but they're working on that, and I'm fairly confident that they'll succeed at it soon enough for our needs.

These sizes are absolutely enormous compared to other libraries we could consider. Inferno and Preact are 9kb and 3kb respectively. If we want to add more features to already bloated libraries we're going to be shipping more library code than our own.

#120 lists UIs that might not need very much more than the rendering engine provided by Vue or Glimmer.

Right now, we don't need anymore than this.

But let's suppose that someone eventually has the impulse to rewrite Views UI as an SPA (whether as a contrib project or as an eventual proposal for core). I think that Ember has more available in it (e.g., a sophisticated router, dependency injection, maybe more?) to enable that than does the Vue ecosystem (vue-router and vuex).

Views UI probably doesn't need a router or dependency injection. Lets not try and balloon the stack any further than we already have. Remember, someone will have to maintain this decision.

Speaking of scaling up the ambition, check out this presentation of CardStack, an Ember-based project for providing even more amazing integrations between a JS front-end and various back-ends, including Drupal.

From the CardStack website,

The experience layer of the decentralized Internet

Cardstack is building an open-source software architecture and protocol, to bring the decentralized Internet to the masses by offering a Cohesive User Experience (CUE) that spans the blockchain and the cloud.

I don't even know what this means? Are they implementing a new wire protocol to transfer data? Their repo highlights a different message; a series of Ember components to construct interfaces, with some knowledge about backends baked in.

And finally, Glimmer uses handlebar templates, whereas Vue uses v-* attributes for expressing conditionals, loops, etc.

I agree that data attributes are probably not the best, learning another template language is probably not great either. This will add yet another layer that someone will have to learn and debug when it breaks.

I think we'll need equivalents of our Twig templates expressed for JS consumption. For example, status-messages.html.twig, image-formatter.html.twig, and possibly a bunch more.

There is already twig.js if we actually wanted to commit to rendering our existing templates on the client.

So do we agree with #120 that *choosing to adopt one specific framework* is the better approach? And that *now* would be a good time to do it? Maybe patents and gzip discussion implies a yes but would be good to be explicit on whether to move and move now.

GlimmerJS has not released yet.

AFAIK, you can't use GlimmerJS in Ember until this moment. Will GlimmerJS stay as standalone or merge back to Ember as a component and when? It seems no clear roadmap.

GlimmerJS coded in Typescript and imply to use Typescript if you want to get the best out of the box. Ember CLI is a bloated tool also. On the code experience, for example, their blueprint app, It took 3+ sec to re-compile it. It's not acceptable to me. Ember also bundled a less popular build tool: broccoli. They really have their own style.

GlimmerJS is bad in performance. (But they stated themselves they're fast in their forked tests. I have no idea why don't they send a pull request to upstream.). I used Handlebars back to 3~5 years+ in a large Backbone based project. I didn't happy with it (either code style / performance) and I think there may be some bottleneck have not shown in the performance tests yet.

JS Decorators is a very new concept in ES6+ and it's not that widely used. It will be another barrier to our users.

And GlimmerJS doc is very bad IMO.

Test it yourself with this very basic thing:

If you want to define an event for INPUT. How to do it? Will the below code work?

<input type="text" value="{{count}}" oninput="{{action increment}}">

(Also Trying google it to find the answer.)
(Keep it as screct if you're an experienced Ember developer)

And the whole ecosystem, GlimmerJS / Ember is lost IMO. Missing other community supports, only their team doing their own stuff.

EDIT:
Also, don't forget the `Editor integration`. And the download counts for these tools might tell us some more stats :)

#103 8 months ago:

Vue.js is the 3rd most popular JS framework on Github and might be #2 or #1 by the end of 2017. It's worth considering, if it isn't too late.
https://github.com/showcases/front-end-javascript-frameworks

Now it's #2 - it's quickly gaining adoption...

So do we agree with #120 that *choosing to adopt one specific framework* is the better approach?

I, for one, mostly agree with that. I think any new core admin UIs that we build into 8.5 and beyond, such as the ones in #120, should be allowed to start using JS-based declarative reactive programming. I'd be potentially ok with us choosing more than one: such as trying out one of those admin UIs with Vue and one of them with Preact, but unless there's really great technical reasons to make such a split, I think there's efficiencies to be gained from core admin UIs standardizing on one.

Re #134 and #136, thanks for those comments. I don't have counterarguments for them, so I withdraw my recommendation of Glimmer, at least for now.

Seems to me that the two top choices then are Vue (for ease of learning and popularity) and Preact (for small footprint). Vue can be reduced to 20KB if templates are precompiled. Preact is 3KB. Which then begs the question of is the extra 17KB worth it? In addition to Vue's ease of learning and popularity, https://vuejs.org/v2/guide/comparison.html#Optimization-Efforts says:

In Vue, a component’s dependencies are automatically tracked during its render, so the system knows precisely which components actually need to re-render when state changes. Each component can be considered to have shouldComponentUpdate automatically implemented for you, without the nested component caveats.

That seems pretty awesome to me.

What else should we consider in evaluating Vue vs. Preact?

By the way, it's of course possible that whatever we pick for starting to build out the UIs in #120 won't be the one that actually makes it into a released version of Drupal. Part way into the 8.5 development cycle, we might discover that we're not getting what we need from the library. Just like when we started trying to add a wysiwyg editor during Drupal 8.0 development, we first tried Aloha, then ran into problems, then switched to CKEditor. But whichever one we agree on for trying first will have the advantage of being the one to most likely make it in unless we encounter compelling reasons to change.

Note about Preact, after usual IANAL disclaimer: if patent licensing is an issue, changing the code base while implementing logic similar to the original (as Preact does) does not make one free from the patent scope.

I'm also not a lawyer. But, seems to me that the challenge in evaluating this is that no one even knows what FaceBook has in relation to React that's patented or patent pending. Do they have vdom diffing patented? If so, wouldn't that affect Vue just as much as Preact? If not, what else is in Preact that's patentable? It seems ridiculous to me that FaceBook has gotten away with a patents license clause that makes no mention of what is patented. But even if it's all a bluff and they have nothing in React that's patented, the clause itself is what makes it incompatible with ASF licensing, and by the same reason, possibly with GPL v2 licensing, and that's where both Preact and Vue are safer, I think.

seems to me that the challenge in evaluating this is that no one even knows what FaceBook has in relation to React that's patented or patent pending

It doesn't have to be a patent for React as in the software itself, if you have overlap with Facebook on any patent, say your company has a toilet patent and you sue them over it, then the clauses kick in, what exactly happens at that point, I don't know. Maybe you can't use your software built on React, maybe not, legal territory at that point.

the clause itself is what makes it incompatible with ASF licensing

I don't think they are technically incompatible, you can use Apache licenses alongside React's license, what ASF said is, their own projects managed by the ASF cannot use React.

I'm certainly no expert in any of this but I will add one other thing. Drupal people seem to be actively looking at and using GraphQL which I believe Facebook has similar licenses on now.

Having used both React and Vue.js, from a technical point of view, I think Drupal would benefit regardless of which one we choose.

What else should we consider in evaluating Vue vs. Preact?

It looks this issue is just for admin pages. 17kb difference probably won't make a difference for people who are administering Drupal sites, especially since it only needs to be downloaded once and cached in the browser. From above:

The scope of this issue is currently limited solely to site building and content administration UIs within the admin experience, such as Views UI, Field UI, and block placement.

If 17k is a serious consideration (for comparison, Drupal.org is loading almost 1 Mb of images on the homepage), then maybe Mithril would be another a third one to look at? There are some comparisons here:

• https://vuejs.org/v2/guide/comparison.html
• https://mithril.js.org/framework-comparison.html

Vue looks good but it's not a popularity contest. The IS contains a few steps that we really should follow here. The licence talk got us off track.

For anyone interested in some benchmarks: https://rawgit.com/krausest/js-framework-benchmark/master/webdriver-ts-r...

I personally don't think this would be a main reason for adoption. I think considering where these frameworks are heading and their philosophy and the communities behind them are an important factor to consider.

No VueJS play nicely with jQuery. Used on live projects. Instead React and Angular hard for adoption with over JS frameworks.

Coming back to the IS I see: Agree on criteria for selecting a JavaScript framework to supersede Backbone in order to create a shortlist.

I see mostly technical requirements in https://docs.google.com/document/d/1eDnJ-NpJBjicxVkrZgCwMHewSZUUePVY6LCV.... I would assume that most of the candidates are good matches for the technical part.

IMO what is going to tip the scale towards one, the other or none are things like:

• How easy can we integrate with existing tech debt? Ex: #145
• What's the community health? Ex: #144
• Is it an easy drop-in from a legal stand point? Ex: Licensing discussions
• LTS releases. Ex: Note how Vue is now a strong candidate but it wasn't even mentioned in the matrix linked above.
• Level of effort when dealing with situations when JS is not available? Ex: Twig.js mentions and re-use of existing templates.

I'd love to resolve Agree on criteria for selecting a JavaScript framework to supersede Backbone in order to create a shortlist. before starting to endorse / promote actual options.

> f) no need to use npm environment to build working scripts(ie. the fw should work with only js file and a web browser)

Yes, definitely this.

Furthermore, We need people who aren't proper JS developers to be able to do *something* with this, otherwise loads of existing Drupal developers won't be able to build UIs any more.

For example, I'm not a front-end dev. I can write some basic JS and jQuery, but when it comes to debugging someone else's more complex script, I am lost in the modern JS idioms.

I had a look at the getting started docs for Vue, and my thought was 'Hey, this makes sense. I can imagine writing Drupal admin UIs using this in the future...'

I had a look at the getting started docs for Glimmer and React, and ran away screaming in both cases.

#147,

Yes. Coming up with our needs, and then criteria to help us decide on a framework is crucial.

#148,

good documentation and steep learning curve(with previous jquery/js knowledge, one should be able to write functional code within an hour and small app within a day).

Good documentation and steep learning curve (did you actually mean steep?) don't necessarily indicate that you can pick it up shortly.

plugable architecture(state management, routing, http/ajax client, rendering/templating)

Almost every modern javascript project behaves this way, require a new dependency and rebuild your bundle. Initially we probably won't need state management or even routing.

small size(minified jquery in core has 84kb)

Lets not conflate small size with jQuery. There are better targets we can hit.

either jquery replacement(=keeping the css selectors, some (v/s)dom manipulation methods) or jquery-compatible(not likely)

Neither of these ideas are really how things work these days. Manually manipulating the DOM with selectors is messy and prone to a bunch of problems. Most of the time the state/props of a component is updated and they are responsible for manipulating the DOM with the most efficient change set.

standard(or close to) syntax(related to a.), nothing exotic or too project-specific(JSX).

What do you mean, standard syntax? JSX is pretty trivial and keeps everything in JavaScript, there's no additional template language to learn.

no need to use npm environment to build working scripts(ie. the fw should work with only js file and a web browser)

This is unrealistic. We already use some build tooling to transpile our ES6 JavaScript files to ES5 environment compatible files.

type script being optional but not required and not default

Flow? Typescript in theory is nice, but shouldn't exist as another language. (Yes, I know, it's just JavaScript)

GREAT support(/ease of creating) for ajaxified forms, form validation and submission.

In the end, its just HTML with state management, with a global store or not, so creating forms will be just as easy or complicated as they are now.

popularity/longevity - will the fw be here and still in development in the next 3-5 years?

dev team - is the fw maintained by single person, team or a company? Is is backed by a company or by the community?

No one knows the answer to this question. Additionally, does this really matter?

lock-in - does the fw have some hard dependencies or requirements(related to b.)?

This is mostly why we don't need a framework. Start with the smallest part of the puzzle, a view diffing library and go from there when we see a need.

does the existing JS needs to be completely or partially rewritten into the new fw or can it co-exist with current code?

A lot of core's JavaScript should be updated.

library of components - I should not be required to write everything from scratch but rather use an existing component(input, select, date picker,... ie. https://vuecomponents.com/ or https://www.webcomponents.org/)

A library of our own components, perhaps with knowledge of Drupal?

state/event/parent-child binding - someone from JS world should comment on which design is better and why

Most components depend on their state to control their initial render and subsequent rerenders. Additionally there are events and synthetic events (this gets tricky) that can potentially mutate the state.

jQuery will be removed in favor of this new fw

jQuery will probably not be removed, but these days, most of the browser api's are stable enough that we shouldn't really need all the hacks and workarounds that jQuery provides.

I feel like this is like D7(procedural code) vs D8(oop)...a massive change(due to Symfony in core, not just oop)

Yes. Modern JavaScript is incredibly complicated.

Drupal is heading to the headless rest api-based architecture and once we'll be there this requirement will most likely be counterproductive and we'll be requiring some type of nodejs environment to build the front-end but for now I think it would be a huge obstacle for many devs.

If you want to build core's JavaScript right now, there'a a hard dependency on Node.js and a few modules.

#149,

Furthermore, We need people who aren't proper JS developers to be able to do *something* with this, otherwise loads of existing Drupal developers won't be able to build UIs any more.

I think this is alright in theory, but mostly unrealistic. Constructing your entire UI in PHP (with Twig) will always be filled with issues. What do we do once the UI has been rendered and passed to the client? Do we create a spaghetti web of jQuery calls to manipulate the DOM whenever something changes?

I'd be comfortable postponing this on a sub-issue called "Agree on criteria for selecting a JavaScript framework." Seems like a sensible next step, if there's general agreement that picking a framework for basing our core/contrib UIs on is something we should do (doesn't seem like there's universal agreement about that, but I'm getting the sense of much more agreement than last time..).

However, I would be a lot more comfortable doing that if there was a person (the front-end committers?)/team (API-First?) around this that was empowered to decide when those criteria were "done" so we could time-box that discussion, else I'm not sure how we ever finish that list, unless there are ready-made criteria that we can yoink from elsewhere (WordPress?).

One requirement is that we need to be able to insert a server returned HTML string into the vdom. For example, Media Library will likely ask the server for formatted preview thumbnails. I'm sure there will be lots of other use cases, including ones that involve rendering the value of a formatted text field. Vue, Mithril, and Inferno all support this. Preact only does so with preact-compat, which I think then doubles the size to 6KB and potentially adds other things we might not want. If it's possible to reduce Mithril down to 4KB, and if that still includes the m.trust() function, I think that then makes it the smallest of all possible options. However, preact + preact-compat might have the advantage of being more compatible with the large ecosystem of existing React components and recipes that are out there.

IMO, another point in Vue's favor is their attempt to stay close to the Web Components spec, including their implementation of slots. I don't know how Mithril, Preact, or Inferno address that problem space. If someone does, please comment with that info/links.

> steep learning curve (did you actually mean steep?) don't necessarily indicate that you can pick it up shortly.

Erm, that's what 'steep learning curve' *means*.

A learning curve represents your knowledge and ability over time. If it grows steeply, you're learning quickly.

Unfortunately The expression steep learning curve is used with opposite meanings. We'd be better off to use different wording.

To get it more realistic, we could take a look at Toolbar module. Probably if anyone has time, they can rewrite Toolbar into React / Vue.js / glimmer.js and let our PHP developers knowing the differences.

Why toolbar?
- It's pre-rendered HTML to avoid flickering (how possible to plugin a framework into existing code)
- It's REST API to load extra sub-tree menus
- It interacts with other modules, quickedit / setting tray..etc
- It required State management

And what's our philosophy? Will we build Render API in JS? and chop the code into very small pieces?

BTW, I think a simple comparison of the file size and features are unfair. For example, Mithril featured they have XHR build-in but I doubt if it could provide same features as Axios / jQuery.Ajax for Drupal CORE/Contrib.

And with React, I bet we need Redux also. However, the Vue, I'm not sure if Vuex is a required. A similar situation to Mithril and others.

And with the "Accessibility", I don't think we could remove jQuery UI from Core sooner. jQuery almost 100% loaded on the pages with JS.

--

During the time to pick a FW, I think committers could have a meeting to discuss a better way to help to JS issues to move forward. I don't think current Drupal Policy fit a huge JS development. (At least, personally, I thought I wasted over 80% of my contribution time on non-code stuff) And at perfect state, I thought we could rewrite existing code into a better design pattern first. We will plug in the data flow and view layer into a FW. Technically, no conflicts and overlay but will smooth the progress to integrate a new FW.

On a slightly different take, what is currently using Backbone ? CKEditor, Contextual Links, Quickedit, system theme tests, Toolbar, and Tour.

Most BB removals in that list, should not be a problem, but what about CKEditor ? If it is actually required by CK, it looks like BB is not going away, which in turns questions the opportunity of adding another framework to NOT replace it. Not a rhetorical question: I'm just afraid of touching CK myself.

@fgm On my site I can see that Panels, Panelizer, Lightning, Devel, and Responsive Preview also use Backbone.

@samuel.mortenson : this does indeed increase the likelihood that a "replacement" would actually turn to be an addition instead. But I think in this issue we want to only focus on core, don't we ? I wonder how many people could seriously envision rewriting Panels once again just to change a JS framework.

[…] if there's general agreement that picking a framework for basing our core/contrib UIs on is something we should do (doesn't seem like there's universal agreement about that, but I'm getting the sense of much more agreement than last time..).

It seems that the voices that raised concerns in the last commenting spree are not speaking much now. I wonder if that means that they don't have anything new to add but still feel the same way. Maybe it's the other way around and they are now convinced.

Speaking for myself, I haven't seen any new arguments in favor of adding any new and additional front-end framework that would make me change my mind. I still think it's not a good idea.

To folks like @e0ipso who think this is a bad idea, I'm curious... Not that I want to argue with you, since my knowledge of JS mostly begins/ends with alert('woop'), but... how do you then propose handling the challenges outlaid in #120? Just continue coding new UIs for core/contrib in "Walkman" mode for the forseeable future? Do "MP3" mode, but with our own custom JS components? Something like Turbolinks to get us to some kind of middle-ground ("CD player" mode, perhaps? :D). Other?

@webchick I don't want to take a big stance here, because this is out of my area of expertise. I can only hope to trigger some other contributors to chip in.

TL;DR: I'm of the opinion that we want fancier UIs, but I don't like to keep putting everything in core because of all that implies (BC and maintenance).

My feelings around this are:

1. We need to build better UIs.
2. We probably want to use a modern JS framework.
3. We should not do it in core.

I'll explain we should not do it in core:

1. It feels that we have a golden hammer, which is to put things in core. This is because we have a big problem discovering and highlighting contrib modules and themes. We are not fixing the problem, we are circumventing it by putting things in core.
2. We have to keep Backbone for BC. When we want to deprecate the next framework we'll have to keep it. And maintain it.
3. I haven't seen many arguments saying why we need this in core, but we can't do it in contrib. If you turn off JS, then Views UI keeps working, that's because we have a baseline UI. What if the extra UI lived in a separate module/theme? Core could provide baseline UIs and the contrib module the fancy UIs using Vue/Preact/Glimmer/…

When we want to deprecate the next framework we'll have to keep it. And maintain it.

Doesn't deprecation mean the exact opposite? That you do not have to keep it and maintain it? You're right to the extent of D8 and it would definitely start to become a little silly if we were to deprecate the next one as well during the lifetime of D8 (but that goes for anything you deprecate). So far, though, I'd say there is little wrong with choosing the next JS fw within the D8 lifecycle and deprecating backbone. In fact, that is very much in line with the "smooth upgrades" policy changes of late.

I believe the argument for not doing this sort of thing in contrib is that before you know it, you need five different frontend frameworks in your site because module A requires fw X, module B requires fw Y and module C requires fw Z. I would be surprised if it were not mentioned in this thread before, but I am surprised nearly daily, so that's not saying much. Core should not prevent you from using something of your choosing, especially for your site's frontend. However, some sort of "guidance" or default from core I think is very much welcome.

Doesn't deprecation mean the exact opposite? That you do not have to keep it and maintain it?

AFAIK you'll need to keep Backbone because N number of contrib modules rely on it (to date we don't even know which). Removing Backbone means breaking them. We can mark for deprecation, but that will only happen in D9.

However, some sort of "guidance" or default from core I think is very much welcome.

This is exactly my point. I share all those concerns, I only argue that putting the "recommendation" in core is not the only/best way to solve them.

Let's be honest, as long as core doesn't have a fancy JS framework, few fancy UIs will happen in contrib.
It's Walkman's all the way, to use webchick's terminology. I'm not going to learn a framework abandoned in 2013 just to make a nicer UI, and introducing a big JS requirement that needs to be manually setup is usually too annoying to users.

I think it's worth bringing up that the scope of the new frontend framework hasn't been decided yet. There are a lot of ways that something new could be added to core:

1. Use a new framework like we use Backbone now - for fancy, Javascript-reliant UIs like Quick Edit. UIs that can be written with Twig and jQuery still use those tools.
2. Use a new framework on all non-standard-form UIs (Views UI, Block admin, Field UI, etc.). Unlike (1) this would be an effort to move over admin UIs to a new framework progressively, instead of just replacing Backbone.
3. Adopt a new framework and completely replace the rendering layer with it, i.e. PHP renders no HTML.

A lot of the conversation so far has assumed that the initial delivery or future state would have to be (3), but that isn't necessarily true. We can start slow, and see how the community feels about the new framework before going all-in on it.

I'm not really concerned with the technical aspects of the frontend framework we land on, what worries me is the community's ability to contribute back to core given how little contribution I've seen to JS-heavy modules like Quick Edit and Panels IPE. As a community of mostly PHP developers, we need time to warm up to something new, and confidence that the solution is going to provide more value than what we already have. Once that happens, I think it will be a lot easier to convince people to make more dramatic moves (like #3).

As a community of mostly PHP developers, we need time to warm up to something new, and confidence that the solution is going to provide more value than what we already have

This is a strong point. I think you can even expand this a little bit. I think personally we need to wake people up and letting them realize that now is a good time to learn new technologies. Showing that these technologies are a good fit for Drupal itself would help to communicate that.

A lot of the conversation so far has assumed that the initial delivery or future state would have to be (3), but that isn't necessarily true. We can start slow, and see how the community feels about the new framework before going all-in on it.

I agree with this. A year and a half ago, when I wrote #12 and opened #2645666: [policy, no patch] Require Node.js for a future Drupal major version and rewrite some of Drupal's UI code from PHP to JS, I was one of the ones arguing that we should look ahead to a future when there's no HTML rendering at all in PHP, but I've backed off from that. I still think that future will come, but who knows, maybe not for another 10 years. In the meantime, I think the immediate scope for this issue is even narrower than #167.2: if people want to work on rewriting Views UI or those other UIs, that's fine, but those refactors can wait for however long they need to until people want to do it. Instead, I think the scope should just be about the 3 new UIs in #120: whether that's all 3 of them, or just 1 or 2, depending on what each of those initiative teams thinks is best. However, it would be ideal for whatever view library (it doesn't need to be a full-blown framework, so I'm retitling this issue) we pick (if we do in fact pick one) to also be capable of supporting contrib use cases for contrib authors who want to start using a reactive declarative paradigm for their UIs. And eventually, for it to also be able to support #167.1, #167.2, and who knows, maybe even #167.3 some day when the Drupal community is sufficiently interested in and excited about that. So, I think it's good for us to consider all those factors in evaluating the libraries currently available, but still remember that at least at first, we'll be starting slow with how we use whatever library we land on.

Just like it provides basic API's on a functional level, core also provides the basic set of user interface design patterns. What core provides in UI patterns is what contrib follows and extends. Our current set of core UI patterns is limited and frankly outdated, which extends into contrib. I don't see many new additions happening where new examples emerge in contrib and are added to the core UI pattern repertoire. Now with the initiatives like Layout, Workflow and Media, this will happen. That still doesn't change much about core being the source for whats available to build your UI with.

And by all means, find the gradual approach that lets us grow into these new capabilities instead of picking a whole package without thoroughly trying, prototyping, working with it.

what worries me is the community's ability to contribute back to core given how little contribution I've seen to JS-heavy modules like Quick Edit and Panels IPE

One possible counter argument to this is that, I personally know React and Vue and i've used both in commercial products and enjoyed working with those technologies.
However, given the current state of core's JS ecosystem, I don't want to invest in learning Backbone which is now irrelevant and there is a lot of homegrown jQuery mess around the place. The lack of JS unit tests is an issue and a proper build pipeline that allows the use of ES5/6 would all make it more attractive.

This thorough article might help on deciding which FW fits best.

@alexej_d,

Just seen it on hacker news and thought to share or not. Here's the topic and you can read the comments there: https://news.ycombinator.com/item?id=15199400

#158: Backbone is only used by the CKEditor module for the drag-and-drop configuration UI. It's not used by CKEditor itself for end users. We are free to change the implementation of the CKEditor configuration UI — Backbone is an implementation detail there.

#159 + #160: note that all those modules are "site builder" functionality, meaning that end users currently don't load Backbone on sites that have those modules.

#165: Backbone is in core, but it's not loaded on all pages. Drupal 7 used to do this bad/stupid thing: it loaded jQuery on every page. In Drupal 8, you have to declare a dependency on core/jquery for it to load. Similarly, if your module uses Backbone, you must declare a dependency on core/backbone.
If Drupal 8 were to add another JS library and recommend it over Backbone, this would break nothing. Existing code using Backbone will load Backbone. New code using something else will load that something else. Some pages may load multiple libraries — that can already happen today, and is a natural part of the gradual evolution of software — it's impossible to snap your fingers and have everything rewritten.

#167++ for pointing out that the scope would not be as broad as many seem to perceive it to be.

#170++ ❤️

And it seems wordpress is going to drop react for the patent issues.
https://ma.tt/2017/09/on-react-and-wordpress/amp/
So probably they'll go for vue.js

Ahh Cool. It's not the first time Matt talked about COPYRIGHT and take actions.

Comments on non-AMP page:
https://ma.tt/2017/09/on-react-and-wordpress/

VueJS is having its moment, similar to Drupal 2009 and Laravel 2013. Here is Hacker News thread on the Wordpress/React topic and here is issue in Gutenberg Github repo discussing the possible replacements for React. Its mostly boils down to VueJS vs Preact/Inferno vs the rest.

It would be great to move this discussion to sub-issues - for cleaned up selection criterias (#151) and possible implementation details by using each of the main contenders (around toolbar rewrite or not).

For example with Vue there could definitely be a middle-ground solution for gradual refactoring where new Vue components can be sprinked to any PHP/Twig template and they can communicate with a simple global event bus without any massive vuex/redux/mobx overhead (legacy DOM-mangling can still be an issue though). One should also look into Laravel community experiences with VueJS where there is a set of best practices emerging making PHP and JS communicate, see http://www.fullstackradio.com/70 and https://laracasts.com/series/learn-vue-2-step-by-step

Not sure how theme refactoring discussion https://www.drupal.org/node/2702061 fits in but there is definitely potential to unify the PHP/JS rendering architecture for that "CD player" solution :)

Thoughts are shareable:

Choosing the JavaScript Framework for Gutenberg (~WordPress)
https://github.com/WordPress/gutenberg/issues/2733

Off-topic: Wow, it's kika! :D Long time no see!!

It would be great to move this discussion to sub-issues - for cleaned up selection criterias (#151) and possible implementation details

Yep, let's get that started. I opened #2909481: JavaScript VDOM library evaluation: components returning multiple root nodes and #2909485: JavaScript VDOM library evaluation: slots vs. vnode props. Note that I'm still very new to React, Vue, and modern JS in general, but I started playing with Vue, and those two issues are so far the two main challenges that I found. There's a lot that I do like about Vue though, but I haven't formulated those thoughts into issues yet, so please don't interpret those issues as a bias against Vue.

For anyone who wants to, please open more child issues for the factors that you think are important. One issue per factor please, so we can keep each discussion focused.

Just my 2 cents.

I think it's worth to create ONE SIMPLE NEW issue for other non-Drupal developers (framework authors & frontend experts) and guys did not follow these issues closely.

The new issue has to share the basic of how the new Drupal REST API (Data Layer) / CORE (View Layer) works and D9 foreseeing. And the requirement like with JS & NO-JS, it's very different to most other projects (e.g. WordPress).

Probably, it's good to make a public announcement also.

** I followed Drupal CORE issues every day but the things @webchick listed in #120 still look like private secret projects to me 🤷. A simple brief (SPECs) would help each other :)

Cheers!

My (probably contentious) 3 cents, as a non-JS developer, so for whatever they're worth, which might not be much... ;)

Honestly, I'm not sure the level of specification implied by the sub-issues in #183 is necessary. If the CMS powering 25+% of the web chooses ____, then assuming ____ doesn't burden our project/users with legal problems (which, given React is eliminated from their consideration, seems that it won't), then that's basically going to make it the first-class citizen in the PHP world, and a uuuuuge portion of PHP developers are going to be learning it.

So unless we have ridiculously compelling reasons not to (and I can't think of any; I'm confused by @droplet's comment because we seem to have essentially all the same goals as WP does regarding modernizing our dated, form-based UIs and supporting decoupled apps), we might as well just go along with their choice, vs. extending our own selection process out an extra N months only to either arrive at the same conclusion or abandon the advantage of choosing the defacto standard.

There must be some informations missing on our side, I agree with droplet, if the framework is still up for choosing #184 is the way to go (would get some needed written clarifications on js/no-js requirements too).

If we change how we decide and just go with whatever wordpress does (could be a valid solution) we need to talk about it here, in any case it isn't clear outside of OCTO that it's the case.

Leaving some very interesting feedback on the wordpress situation that feel like it could be about us in a while: https://zao.is/blog/2017/09/14/my-thoughts-on-gutenberg/

Ok I reread webchick post in #185, it isn't fair to single out OCTO. Sorry about that.

The wordpress situation does put things in perspective. The argument that going with whatever wordpress uses is very reasonable, on the other hand we use CKEditor and they use TinyMCE so it isn't as obvious as it would seem. I don't agree with have the same goals as WP. With gutenberg WP wants to modernize the content part, I haven't seen anything about the "management" part of a CMS, which is what Drupal seem to want to address first with layout builder and all. The scope is different and includes much more than theirs (for now at least).

I don't think optimizing for amount of developers should be a high priority (but using same/similar concepts to make onboarding easy, yes). We've seen these past few years, the pool of developers knowing JS AND interested in CMS is small, we don't need to attract tons of them, just enough. Even Neos CMS and their ember/React admin frontend is developed by something like 20 ppl (for scale that's like 5 Wim Leers).

( edit ) Food for thoughts on how others are approaching the decoupled/templating issue : https://www.neos.io/blog/atomic-fusion.html

https://ma.tt/2017/09/on-react-and-wordpress/ does mention layout/template building etc. but not in depth, sounds like they might be addressing more than just content though?

Agreed with #184 that a simple issue with specs and some kind of announcement wouldn't be a bad thing at all. Wordpress deciding not to use React is a good reason to drop it, not sure them picking something is a good enough reason to pick it although we should definitely keep track.

Long time no see!!

Hi, I noticed Drupal community is finally entering Phase of Acceptance on this issue so I figured I'd join in :)

Sub-issues listed in #183 are great but perhaps there is a merit to start somewhere in the middle between this thread here and very specific technical level. I'd propose to move on with following approach:

1) Identify the shortlist of JS frameworks for admin UI rewrite. Based on the discussion above and Wordpress/React issue aftermath discussions there seems to be a strong consensus to go for either some React derivative (Preact? Inferno?) or VueJS. There could still be a floor open for runner-up candidate (Ember/Glimmer?).

2) Create two high-level sub-issues: "Possible Drupal frontend architecture using React-like framework" and "Possible Drupal frontend architecture using VueJS" where set of initial requirements/challenges that need to be discussed, prototyped and hacked upon. These issues should be spread and shared in target respective developer communities and related ecosystem (for example: Laravel is great match for their pioneering work with PHP and VueJS intergration). Drupal can not pull the whole initiative off alone so working with wider community is critical.

3) These two sub-issues above that are possibly quite loose in nature and likely spill over each other a lot but they are the feeding ground for third, strict and formal sub-issue "Selection criterias / checklist for new Drupal JS framework" where the concise comparision / selection needs to happen.

I will post the possible conversation starters in a separate comment.

Here is the sample list of challenges each of these subissues mentioned above needs to be tackled. Feel free to remove this from the issue and copy/modify/augment it as discussion starter.

General architecture:

There are two main approaches mentioned:

- Expand REST API endpoints for all administrative tasks so full admin-SPAs can be implemented (webchick's "MP3 option")
- Hybrid/Gradual approach where current JS code is refactored using new framework in piecemeal basis and JS components are just added to server-generated HTML ("CD option")

These two approaches will likely co-exist and it needs to be clearly communicated why we need both. This issue is about the latter case.

Instancing

In hybrid approach there can be one or many Preact / Vue instances on the server-generated HTML page and this affects component communicating and existing JS code behaviour. Figure out which way to go.

Technical reasoning: When attaching the main to DOM node, React clears the existing contents of that node while Vue does not. This allows to set up Vue in a way that the whole Drupal-generated page with is essentially a root Vue template and Vue components can be added anywhere in the page co-existing with existing HTML/JS. This simplifies the refactoring workflow but can also raise issues when three DOM-modifing, event-binding frameworks (Vue/jQuery/Backbone) act on same DOM tree. Alternative would be to instance/mount a new framework instance for every new component (for Preact/Inferno this might be THE way) or using "portal" approaches such as https://linusborg.github.io/portal-vue/

Component types and packaging

There are several types of components and their packaging formats out there. React uses "standard", PureComponent and Stateless Functional components that are wrapped inside JS/JSX modules. Vue is leaning towards custom .vue file format, merging component template, js and style to a single file https://vue-loader.vuejs.org/en/ but it is not required, Vue component can be packaged as a JS module with inline template or using external HTML/Vue-flavoured-Handlebars template. Vue components have "standard" and functional types.

Component communication: broadcasting events

As the JS components need to communicate with each other (currently the broacasting is done via eclectic mix of $(document).trigger() and $(window).trigger() calls) a simple global event bus needs to be set up, especially when we need with deal with separate framework instances per component (see above). A more heavyweight state management (Redux/MobX/VueX) can also be considered -- for specific complex component set such as page builder this is likely a must have.

Media queries

Currently the window.matchMedia is used directly, sometimes with breakpoint variables, sometimes with hardcoded values. Lkely some abstraction is needed, see https://github.com/contra/react-responsive / https://github.com/vuejs/awesome-vue#responsive-design etc

Perceived performance

There can be a noticable lag in inital load of components even when JS build sizes are minimal, this leads to possible layout reshuffle when all the components are loading so generic approach for placeholder component wrappers could be beneficial.

Translations

_locale_rebuild_js and related functions are likely the most epic hack present in Drupal codebase ever. How would translations work with new JS components? Vue i18n plugins leans towards $t() helper functions or special directives, React to high order components or Internationalization API, the strings are usually dynamically fetched or contained in components / external files and compiled in.

Drupal.* JS API and drupalSettings

Refactoring Drupal.* services/utilities is a a separate issue but many of its methods are deeply interwined in current JS components and might have influence on framework selection and futher implementation details (for example there is a pattern to provide services as mixins, assigned $variables to root instance in Vue while React is using more of a mix of simple JS imports and high order components)

- Audit all the API -- what is osbolete, what is vanillaJS, what requires jQuery. what touches DOM etc.
- Packaging / loading: it has dependency with general JS import/export/transipiling stategy, it's not if but when I could use import { announce } from 'drupal/accessibility' :)

Testing

Needs investigation. Can only talk about Vue side: historicaly being a weak spot. For testing utils there are some convergence around https://github.com/vuejs/vue-test-utils, no clear favourite in test running side.

Other issues

Transpilation / packaging

A separate issue. Webpack is the key here but there is an option to use wrappers for DX simplifation https://github.com/JeffreyWay/laravel-mix / https://laravel.com/docs/5.5/mix (a standalone thing, does not require Laravel)

CSS

Many options to go there, skipping for now, should be tackled in separate issue. There is always a way to prevent current CSS approach and improve gradually. The key questions to ask is that whenever we need dynamically computed (variable-based) styles, JS theming support and and do we need scoped CSS (see the next step)

Component universality and re-use

Could the refactored components be useful also for the fully decoupled admin UIs or even beyond? What are criterias for universality and is code re-use part of our goals? How far we should push it, are standalone npm packages or isomorphic primitives the limit?

Single maintainer problem

Its not whenever is it a problem or not, it is more like what to do about it? https://opencollective.com/preact, https://www.patreon.com/evanyou, https://opencollective.com/vuejs, https://vuejs.org/v2/guide/team.html etc

More, more, more

As mentioned in #147

using some standards would be nice(ie. Polymer and web components https://www.webcomponents.org/)

We build a couple projects using polymer/webcomponents and I must say the more I use it, the more I like it. As far as I can tell it's probably also the most future proof way. There's already a lot done at https://www.drupal.org/project/webcomponents by @btopro and his colleagues

I checked the matrix again and it seems web components were never considered?

wow, a ton of JS opinions walk into a bar style thread. ;)

Why we went Webcomponents over anything else is the future proofing concept that @attiks mentioned. Web components also had a level of malleability where we could make 1-page apps OR progressively decoupled pages OR individual blocks in parts of pages OR none of those and just use it for pure design / UI consolidation.

via the mentioned webcomponents module we're crafting workflows where we more so teach Drupal about our webcomponents but they are completely separate from Drupal and most are built entirely in vacuums (we use Polymer to help build them faster and the tooling is great).

How is future proofing possible?

Because the elements generated are a layer below traditional frameworks like React / Angular (I haven't played with Vue in awhile but it seems you can do similar things to webcomponents but if selecting Vuejs I dono why you wouldn't just target the browser itself). These elements can work independently and in a plug and play manner; especially as the world of JS moves more towards JS Modules; you could start mixing components made in different framework / libraries into one another.

As an example of this: https://stenciljs.com/ was recently announced at Polymer Summit which is by the team that does Ionic (an Angular based app-publishing platform). They made a huge announcement recently that all future development will be in web components, and that they'll be using Stencil.js tooling to manufacture these components / make new ones.

Future proofing then comes into play in that I can have 1 element made as a stock webcomponent, 1 made with polymer and 1 made with stencil (there's also skate.js and x-tags which can make tags too) and all of them will work together. Think of Web components as modular front-end development much in the same way that drupal's module system allows for plug and play backend development.

To echo @attiks : we're not going back. This is far too enjoyable a workflow and has allowed us to pull of projects at a pace previously not thought possible. The flexibility to do full 1-page apps OR pure design elements as well as repurpose elements from other companies has allowed us to progressively decouple in a way that's logical for people on our team. I know this is transformative because we have 3 junior devs making meaningful contributions to our latest ELMS:LN App we integrated ( https://youtu.be/5xNXOPfPHlA?t=22m56s ).

Developer workflow for this:

• Senior dev 1 focused on certain displays in the app, asking junior devs to make design elements when needed
• Senior dev 2 focused on communication between elements and wiring up Polymer front end to Drupal backend points
• 3 junior devs worked on design components / "dumb elements" to match mock ups and layout

All the elements we worked out are system agnostic; so we can rig them up in a codepen or Drupal 8 (when we get there) while only needing to really worry about the backend. Polymer helped w/ working on them in isolation and provides self documenting mini-sites. For example if you wanted to learn about our magazine cover element you could check out the doc site for it which gets updated as we work on the element itself: https://lrnwebcomponents.github.io/magazine-cover/components/magazine-co...

We're also free-ing ourself from Drupal's templating engines by teaching Drupal to print properties into our elements automatically. Drupal discovers where the components are and their definitions and then provides Display modes and DS templates that mirror them. Our field UI can then be used to map the property in our component to the field in drupal. This effectively allows for "theme-less" Views output and display of content rendering it through our polymer element. Drupal, having learned about the component, then prints the tag on the page and wires up the properties to our entity output. If we want to swap out elements it's easy, if we want to change the look of something we're basically not touching Drupal unless we need to change what a property is. This also now is so easy that it can be done via the GUI; empowering junior devs to implement designs more rapidly after getting content types structured.

You can learn more about the workflows and approaches of Drupal and Polymer in these blog posts:
https://drupal.psu.edu/blog/post/web-components-polymer-and-decoupling-d...
https://drupal.psu.edu/blog/post/building-polymer-app-web-then-drupal

Related to the web component discussion, this showed up in my feed the other day:

http://blog.ionic.io/the-end-of-framework-churn/

For future-proofing Web Components and emerging browser APIs in general are the sure bet. The problem is that some of these API are too low level (Custom Elements tend to be built with helper libraries/frameworks such as Polymer, SkateJS, Stencil and Svelte), do not provide the best DX out there (raw DOM APIs, being imperative rather than declarative) and certain specs such as HTML imports are under rethink.

What does it mean using Web Components in Drupal? Is it about

1) Using Custom Elements as "render target" (many frameworks, including React and Vue do support it)?
2) Using Web Components / Custom Elements with the helper libraries mentioned above
3) Using Web Components APIs in their raw form?

Related: Joomla is exploring web components as well https://joomla-projects.github.io/custom-elements/#/

@kika with _locale_rebuild_js() worth noting #2607376: Remove on-demand JavaScript translation parsing and do everything on rebuild although that's more of a when rather than how issue, but it would probably give us a much better starting point to work from than the ten-year-old approach we have now.

good considerations for scope @kika. In my experience the DX associated w/ Polymer is very high, either in version 1 or version 2. One of the reasons we went w/ polymer over the other frameworks mentioned is their commitment to future versions working with previous ones in a hybrid mode. We went through the pain of Angular to AngularJS and I think whatever Drupal adopts it needs to be more like jQuery in terms of upgrade cycle then D6 to D7 to D8 pain of complete rewrites.

I could see core using something like stock components or a processed system like Stencil or Svelte to produce elements. This would be without opinion as far as framework in rendering and then people who have adopted any of the above could play nice w/ them in any theme workflow (traditional, headless or progressive decouple).

Another consideration would have to be the complexity to which these would attack. Input fields that look nice and are highly accessible would be an easy win that many won't get too opinionated about vs the properties that should make up a `` or `` tag

@yansern's comment on the WordPress issue is also worth a read. It raises points about drag and drop, contenteditables, iframes, and other real DOM manipulation considerations that I think will be relevant for us too.

Just copying and pasting the comment from the WordPress issue because it is a long thread and sometimes fails to load.

Based on my 2+ years experience building a block-based editor similar to Gutenberg, I think there are several teething concerns to address when choosing the right framework, e.g.

How would VueJS/Marko/Angular integrate with drag & drop? How would drag & drop in Gutenberg work? When dragging, are you cloning a ghost element or using the existing element? When dropping, where can you insert placeholders to drop a block

How would VueJS/Marko/Angular (and its Virtual DOM) work with Content Editables and DOM Range & Selection API? Cross-browser inconsistencies with this range & selection can be very tough to nail here.

To what extend will copy/cut/paste work in Gutenberg editor? Can I make a text selection between multiple blocks and perform a cut/copy/paste? Do content editables live in each individual blocks or is it all contained in a master contenteditable?

If there are Gutenberg blocks that contains iframe embeddables, e.g. embedding a youtube player or twitter feed in a blog post, moving this block from one DOM position to another position would cause the iframe to reload itself. Other caveats include inability to get events propagated from the iframe back out to the editor (imagine if you're dragging a block element across the editor and your cursor is now hovering on the cross-site iframe and everything stops working).

All frameworks are great at working with Virtual DOM, but a lot WYSIWYG usage live outside of the Virtual DOM. I think the areas to focus on when assessing the right framework for Gutenberg is how well can the framework deal with DOM event handling, and for bleeding edge requirements that can't be built with framework, how manageable it is to build it outside the framework and plug it back in.

@ivanjaros ++

@yansern's comment in Github is about how to implement X in cross browsers. It's not related to Virtual-DOM libs. Virtual-DOM is a concept to optimize the rendering. At the end, that still deal with REAL DOM.

Besides that, the WP Gutenberg has a custom abstraction layer and wrapped React inside. You work with Gutenberg API rather than React direct.

with Drupal example, that is #2830882: Introduce Drupal.offCanvas, mirrored after Drupal.dialog, to avoid Settings Tray using Drupal.Dialog in unintended ways (But in a more elegant and performant way)

And same as @ivanjaros thoughts, I believed that's why Redux-alike libs exists. A single-truth source simplified the data flow between each component.

Ohhh, snap. https://code.facebook.com/posts/300798627056246

Facebook is relicensing React, Jest, Flow, and Immutable.js under MIT next week, dropping the patent clause.

*munches popcorn*

LOL. Matt, WP won. They couldn't make 25%+ of CMS **developers** sad!

If you look at job descriptions these days many actually suggest some react knowledge. Given that for a doing well Drupal community giving them a way to learn something which benefits them in finding a job is huge, just my 2 cents.

I *think* #2809255: Create a polymer SDK for Drupal is related?

@ivanjaros - https://robdodson.me/regarding-the-broken-promise-of-web-components/ is an important follow up from the opposite perspective / responds to the criticisms in that article.

Largely seems a worldview battle but that being a react centric worldview about them NOT implementing is telling on this list of how differnet frameworks are dealing with custom elements https://custom-elements-everywhere.com/

A notch in the Polymer and/or stock web-components column would be compatibility with Preact, Angular and Vue (as well as Moon, Dojo 2, HyperHTML and maybe even React eventually); all of which have been mentioned in this thread. This is possible bc web-components is a level below these frameworks.

Also I've had decent success w/ jQuery being able to work with them. There are issues that can pop up because of shadow dom ( http://robdodson.me/dont-use-jquery-with-shadow-dom/ ) but it's possible to throw down a bunch of elements and in the transition from one world to another have jquery correctly target and apply Drupal Behaviors to them.

@btopro I've been working on a light-weight support module for using Stencil with Drupal and it's going pretty well, it's a nice way to slowly move Drupal towards modern Javascript without standardizing on a single framework/library. I've also used the webcomponents module and Polymer - you're doing some great work there :-)

Personally I like vue.js a lot, in particular the they way it encourages and allows for progressive use. Allowing for a progressive change makes it possible for people to slowly evolve their frontend from twig to be more vue.js powered.

Howsoever, I think the big issue that we've to solve here is bundling frontend components in a re-usable fashion with modules. We need modules to be able to bundle frontend components with a module as we have templates today. Only that way, modules can provide real, integrated solutions with a modern frontend. This obviously introduces its own challenges for building pages with optimized assets, so it makes sense to keep that upcoming requirement in mind during the evaluation of candidates. (I know this is about admin-JS for now, but imo this has to be next step in order for the platform to stay relevant.)

Could be another feather in the WC cap being that there's a lot of discussion recently about how best to do JS modules as the methodology for distributed JS snippets / libraries that can be resolved easily. And, in the stupidest of things which actually would be significant; the Developer UX associated with having PHP based modules and JS based modules which would be included together would be nice for expressing concepts. This Drupal module includes the following front end modules would be easy to read and comprehend in code and also potentially in bundling dependencies.

Webpack - like technologies could be used to resolve and do bundling but also JS modules would probably easily be able to be assembled in the JS aggregation methods currently used. PHP based module code would be defining which JS modules it needs to work on the front end, helping resolving those dependencies and then the front-end has everything it needs delivered in an efficient, bundled way.

These are issues we're currently hashing out without JS modules in polymer + drupal 7 land and they are tricky there (html import is diff then js modules tho similar) but it's something JS modules would make a lot easier once it comes along a bit more. Also right now, with Drupal not having support for component architecture we're basically forced to hollow the entire system out and make it entirely decoupled both front and back end in order to have a slick UX (not blaming, just being honest).

React (my preference) is now licensed under MIT:

https://github.com/facebook/react/blob/master/LICENSE
https://code.facebook.com/posts/300798627056246/relicensing-react-jest-f...

---

On a side note, I find this issue quite amusing/annoying considering that we haven't been able to pick a damn CSS preprocessor (or simply support both) in #1310698: Add a CSS preprocessor library to core... and there are relatively only two options there.

JS FE frameworks number in the hundreds, with popular ones in the dozens.

Good luck y'all...

ha, true; not to flood @markcarver but just wanted to mention we no longer do CSS preprocessors as a result of webcomponents. They largely eliminate the need for them which wasn't a reason we went into it but discovered as we went. When we previously adopted Angular we still needed them. I agree though on the opinionated decisions in this space though; if it's a library like React/Angular/etc it'll be the 1st thing untangle to get back away from it.

Opinion is drupal should not adopt something that opinionated for this (nor Polymer which is what I primarily do) but instead produce vanilla web components which then are able to be used in all of the above while still improving devux and consistency.

CSS preprocessors are still necessary for frameworks that don't use webcomponents. Furthermore it would allow real-time compilation of variables in these frameworks (e.g. think color module on steroids).

---

Regarding this topic (and to provide some actual constructive feedback):

I think it would be a massive mistake to push any JS FE framework like what has already been mentioned on top of what is already an extremely cobbled and stressed theme "system".

We haven't actually focused on the internals of the theme "system" in years. While Twig is amazing, it's the last step in the theming process.

We still use render arrays.
We still have not really committed themes to OO.
We still use procedural info(hook_theme)/alter(preprocess) hooks.
We still cannot participate in install/uninstall events.
We still cannot [safely, via dependencies] rely on modules (which would likely be required for this kind of thing to ensure proper API callbacks).

Personally, I would absolutely loathe having two separate theming "workflows" in core (which is what this issue is suggesting).

Supporting both, what we currently have (PHP based theme system) and these "new core admin UI areas" (decoupling these areas so they're a JS based front-end single page app, essentially), would be an absolute nightmare to maintain for anybody, let alone core.

Nevermind the confusion on how contrib comes into play with this.

This topic is a radical shift in how we would "theme" in Drupal.

It's really an all or nothing topic.

Either we replace Drupal's theme system entirely or we leave this to the scope of projects and allow them to decide to decouple Drupal's front-end.

I'm not saying we shouldn't do it, I'm all for this, but we cannot continue lopping stuff into our existing "theme system" and not expect it to implode one day.

I think this issue should be renamed and either postponed or moved to the ideas queue.

WP organized an open JS Chat. Some framework authors/maintainers shared their thoughts. They also have a light chat on web components.

https://wordpress.slack.com/archives/C5UNMSU4R/p1506438053000533

@Justin (Polymer)
@evanyou (VueJS)
@Sophie Alpert (React)
@Dan Abramov (Redux)

For those interested in the power of web components and some good arguments pro, https://medium.com/@mikeal/ive-seen-the-future-it-s-full-of-html-2577246...

In reaction to the #2913321 issue, I wrote a part of why I think Drupal should consider switching to a more fit VueJS instead. But after checking with Lauri and Roy on Twitter ditched that part in the issue.

The main part of the why VueJS is a better fit for the Drupal community is pasted from here on.

--
Perhaps a lower entry system like VueJS could be studied before making the final sign off on React.

I could write a lot of text on why VueJS, but Evan You (VueJS project lead) did a pretty good job on that.

The main advantages for Drupal are imho :

• Plain javascript is still possible
• No need to transform everything at once Vue grows more organically in project
• Componentent scoped
• Lower developer entry level
• Can replace Backbone
• Can be used complentary to jQuery (not adviced)

VueJS is a mature ecosystem, with a pleotora of available resources partly because of the popularity of the framework in the Laravel community.

The license is just like the Symfony components and React, MIT.

@Rene Bakx thanks for that rundown. I've been reading a bit about both React and VueJS, having used entirely different frameworks in the past, and also feel VueJS would be a better fit. The comparison page you link to is also very helpful.

+1 VueJS

+1 for VueJS. Vue is much more easier for delegating subtasks (better template/logic dividing) in teams. Easy to read and understand.

+ Vue \ - React

In addition to the technical side, an important point is attracting new developers to the Drupal.

What we have, basics:
Drupal 7: html/css/js, PHP, «Drupal architecture»
Drupal 8: html/css/js, PHP/Object-Oriented PHP, Symfony components, Twig, Composer, «Drupal Architecture»

React cannot be learned over the weekend. Steep learning curve, fast-changing approaches/best practices/howto...
All these things require a lot of human resources. For small teams, freelancers, newbies, for people who do a lot of full-stack stuff it's very difficult to handle all this complicated rapidly changing «React world».

As the main maintainer of the React, Facebook has own goals, tasks and resources. Do we have the same goals and especially resources to live with React? New developers, who choose a platform, will they want to dive into a large and complex stack of the technologies around Drupal? React will significantly add complexity to this stack.
...

Anyway, of course, it's very important to discuss all the technical problems and caveats, a great comment #190 above from @kika.
It is really interesting to listen to experienced (JS) developers.

It's not just VueJS vs React as a technical choice. @aleksip made a pretty clear point that it's also a choice of what we as an open community and platform are allowing into our codebase.

@dries talked and preached in several Driesnotes about the importance of a open web, yet he is not to shy to lead a vendor who's know for creating content silo's in our open project as React disguised Trojan Horse. Facebook has changed the licence, but there is no guarantee about the patents and a change it when they change their mind again

We are Drupal and we have a choice to lead by example ;-) Let's make that choice a good one!

React is Facebook.

Angular is Google.

Vue is ... Community! -- exactly like Drupal

Vote for Vue! Vote for Community!

Worth noting here:

• #2913628: Proposal to use Vue.js for building Drupal’s administrative UIs
• Involve Vue.js Team to the selection process
• Proof of concept: dblog in Vue v1 by @Chi

+1 Vue.js or something more in line with Web Components!

Cross collab with Laravel team! Off the island, but in a nice boat!

If you build out the APIs first, they will come...whoever "they" are!

Rabble, rabble. Rabble, rabble, rabble!!!

I think the importance of this issue is highly exaggerated. This is merely about Drupal core admin UIs. Developers are still free to use any JS framework for custom and contributed projects. It is already happening right now, no one is using Backbone which was adopted in Drupal 8 core long time ago. Voting is not the right way for this issue. I would rather left the right to decide to core JS maintainers and people who contributing to Drupal admin UIs.

I think one thing we need to eventually figure out regardless of which framework is how contrib modules or themes can extend and override functionality and markup. The beauty of something like our Twig inheritance is that themes can alter markup and output as needed. If markup is essentially baked into a component created by a javascript framework how then do we override?

Another concern to consider is how something like a hook_form_alter() will work in tandem with a js framework. If the framework is ultimately controlling the markup how would we be able to alter it from the php side? Just some thoughts to digest.

absolutely a primary concern w/ any approach the community jumps on. Speaking to Webcomponent DevUX we use them in multiple ways:

1. Placing
   in a template instead of a normal button to improve UX / UI standards through a simple window dressing on a stock HTML element
2. Parsing and understanding the structure of an element, then mapping it to an entity. In this scenario, the element effectively replaces the template files while still relying on drupal to handle how that data gets placed in each field. (see videos of this here: https://www.drupal.org/project/webcomponents)
3. Slots can be used to pass chunks of unknown / flexible HTML elements down and insert them in the correct spot in a component (https://www.polymer-project.org/2.0/docs/devguide/shadow-dom#shadow-dom-...)
4. Theming and style flexibility can be handled via CSS variables (https://www.polymer-project.org/2.0/docs/devguide/shadow-dom#theming-and...)
5. To completely replace Drupal's backend via decoupled thing (in this case, you've skipped all of core effectively which is good for some things and would completely screw up the use-case you mentioned w/ forms)

I had just read and commented on Drupal looking to adopt React, a very enlightening article, when I got this issue.

As the comment is waiting for approval, I partially reproduce it here.

I'm a user, not a coder. In terms of "philosophy", Vue is a community driven project, as Drupal is. It would be more coherent, in my point of view, to use Vue. Both communities would benefit. I agree that from a technical point of view, React can be more ready to use now. But if you look "the video" and not "the picture", Vue is growing more, in users, in resources, in projects, in approval (even some AngularJS users feel more "at home" with Vue than with Angular 2+). As this is a project "for the future", I think Vue would be a better choice. I don't know about using Web components, cited above, but in a philosophical point of view, would be coherent too.

we abandoned Angular in Feb for Webcomponents; had we not adopted them then Vue would have been our next choice (two operate in very complimentary manners)

WebComponents are DOM whereas React and Vue are VDOM. Wordpress is currently exploring framework interoperability at both levels: (via web components and via VDOM abstraction). I have no comment at this time about the relative pros/cons of the two approaches, but for anyone interested in interoperability at the VDOM level, rather than at the DOM / WebComponent level, I just now stumbled upon vuera, which I think demonstrates a pretty exciting proof of concept, and perhaps similar adapters could be written for interoperability with other VDOM libraries/frameworks. I'm not suggesting at this time that we use that library, just pointing to something that I think is showcasing something very cool for people interested in framework interoperability.

[Edit: I also left a comment on the Wordpress issue, so am curious what people working over there respond with.]

VUE is not downing the possibility to use web components in VUE as described here

https://github.com/vuejs/vue/issues/6737#issuecomment-334551912

unlike React

+1 for Vue. Why can be read here:

https://www.drupal.org/node/2913628#comment-12291650

+1 VueJS