General state of performance discussion – Test and report performance comparison during build?

[stalgiag]

How would this new feature help increase access to p5.js?

I explain this below.

Most appropriate sub-area of p5.js?

• Accessibility (Web Accessibility)
• Build tools and processes
• Color
• Core/Environment/Rendering
• Data
• DOM
• Events
• Friendly error system
• Image
• IO (Input/Output)
• Localization
• Math
• Unit Testing
• Typography
• Utilities
• WebGL
• Performance

New feature details:

This is more of a meta-issue/discussion.

Performance feels like one of the biggest barriers to accessibility currently. I teach in a public school setting. Many of our computer labs have machines that have not been updated in well over a decade. Many of our students do not have their own computers at home so during the pandemic many were borrowing Chromebooks for schoolwork. Students would often be getting 5fps on some of p5's official examples while running video conferencing simultaneously. Many students reported extreme frustration and disappointment with running even relatively simple sketches.

For a while now, we have been emphasizing accessibility of source code for contributors but perhaps we need to put some systems in place to help us balance this with an eye towards performance. There are limits to how dramatically we can improve performance within the Canvas API without doing something drastic like rewriting the whole 2D renderer in WebGL ala PixiJS but I do think we could make significant improvements.

My feeling is that we can actually improve performance significantly by just having a way of comparing it during the PR process similar to codecov. Does anyone know of an effective way to accomplish this? If we could see something like 'push() takes 5ms longer to complete in this branch than in main. This is a 10% increase.' we would be able to make more informed decisions during the PR process. What do others think? Does anyone know of a tool that might help us with this? Is it possible for us to build some kind of reporting tool into our unit tests based on the 'time to execute' output that we already see?

PS this conversation has been started based on my own experience but is also in response to a large number of related issues. A quick glance reveals #5237 #4820 #3610. There are likely others.

[limzykenneth]

You might have seen this before but I moved the old benchmarking code out into its own repo here with a website available to compare different versions and implementations. The basic groundwork is there already but to get it to a point of being part of CI of the PR process still require some work. It's one of dozens of projects I'm "working" on which means I can't consistently dedicate time to developing it though I have a bunch of ideas for it still. If there are enough interest, I don't mind us using this as a starting point for something.

This kind of thing might have been something jsperf would be suited for several years ago but since jsperf drop off the map, nothing has really emerged to replace it. Even the benchmarking framework benchmark.js hasn't been active for a few years now and there are many features I would like it to have when working on the p5.js benchmark code.

[stalgiag]

Thanks for this @limzykenneth ! I had forgotten about this repo. This seems like a terrific starting point. Perhaps we don't even need to integrate this into CI but instead just create a ritual of checking this page more regularly. Like after releases we could aim to take stock of changes in performance using the benchmarking website and open issues for any major negative changes.

[limzykenneth]

The benchmarking is still a bit manual (new versions need to be added manually and testing custom build requires editing configs) but if you to get started with it, you can have a look at writing test cases for the parts you are interested in testing.

[aferriss]

I wonder if it would also be helpful for us to provide some kind of guidelines or expectations for performance on the p5 site itself. Additionally we could give some guidance on things to try in order to improve performance or gotchas to watch out for when coding.

Some things that might be slow off the top of my head:

• Really big canvases
• Doing too many things inside of a for / while loop
• Working directly with pixels
• Manipulation of video pixels

Some things to try to improve perf:

• Smaller canvas size
• Moving calculations that only need to happen once into setup()
• webgl mode (not always possible)

I'm sure there's lots more. It's a pretty big topic but I think it could make for a useful article and at least give some expectations around frame rate.

[aqilc]

Is it possible to move the 2d renderer to WEBGL mode somehow? I've always thought about the possibility. OpenGL/WebGL is technically a 2D GPU API in the first place, could be possible to see massive performance increases if we pool all of the vertices and stuff into one context and execute draw calls based on frameRate(). We could call this render mode WEBGL2D or something, or this could be turned on automatically after doing some sort of test to make sure it's actually better on startup.

Another thing we could implement is have a version with all error reporting related stuff except basic try catches removed. This could also show some performance gains, as there's not as many branches in the app and the tab process.

On the topic, bundling based on renderer is also something I would appreciate, as it would cut down on internet usage and load times by a large amount.

[limzykenneth]

@aferriss I think some points are covered here but definitely can be expanded where required. Visibility of the page can probably be improved as well.

@AqilCont I don't see moving to WebGL entirely be something p5.js pursue as absolute performance is not the goal of p5.js while moving to WebGL for 2D rendering as well will make the codebase very unapproachable for beginner contributors. For most cases the native 2D canvas renderer is fast enough for almost all cases and in the rare case it isn't I'm fine with people choosing something like pixii.js for a WebGL backend as I think in this case they probably know what they are doing already. Though that being said, an add on library that provides WebGL 2D mode as a prove of concept is still welcomed if anyone wants to work on it.

Another thing we could implement is have a version with all error reporting related stuff except basic try catches removed. This could also show some performance gains, as there's not as many branches in the app and the tab process.

The minified version should already do what you mentioned here, unless you mean something else?

On the topic, bundling based on renderer is also something I would appreciate, as it would cut down on internet usage and load times by a large amount.

There is some past custom bundling work done but it still isn't at the point where one can download custom builds on demand.

[aqilc]

@AqilCont I don't see moving to WebGL entirely be something p5.js pursue as absolute performance is not the goal of p5.js while moving to WebGL for 2D rendering as well will make the codebase very unapproachable for beginner contributors. For most cases the native 2D canvas renderer is fast enough for almost all cases and in the rare case it isn't I'm fine with people choosing something like pixii.js for a WebGL backend as I think in this case they probably know what they are doing already. Though that being said, an add on library that provides WebGL 2D mode as a prove of concept is still welcomed if anyone wants to work on it.

The main reason I suggested this is because of p5's ease of use, accessibility, and simple API. Pixi.js is a made for people focusing on performance, so it's API is designed to be more advanced and is hard for beginners to get into, and this is why me or many other people and classmates don't use it. What I suggested focuses on main point of this issue, that the limiting aspect of p5's accessibility is its terrible performance. I'm a student myself and I'm just trying to make a platformer for my school project, but trying to add any particle effects or cool animations using math or whatever is turning out to be extremely hard because of this terrible performance and unplayable FPS. One of the main populations targeted by this library targets is students and people in school environments using javascript, and most people in these environments don't have the best hardware, usually a laptop or a school computer. If your library is this slow, most students will be warded off of JavaScript since the work they are putting in for coding their games is all gone in vain when they actually want to add more cool mechanics and graphics.

Also, people that just want to draw a lot of things on the screen at once don't necessarily have any knowledge of how the renderer works or really care about it, they just want to draw stuff and work on their games in peace without worrying about how it will perform in the future. I just think that drawing 10000-1000000 circles on the screen every frame for movement, effects, and particle systems could be executed a lot more efficiently with WebGL, making it accessible to people just wanting to get into graphics, simulations, and animation through JavaScript in a beginner friendly way.

If you're worrying about the contributors, I don't think beginner programmers will have much to commit to this library anyways. It's mostly WEBGL or Web Canvas API with some advanced javascript, which most beginners figure out as they go, and the long history of p5 basically means that most of the existing features are ironed out already. A new project, like a WEBGL2D Renderer will alternatively give people a reason to actually contribute and use the library, will generate activity on the repo and increase popularity because of the new features.

I don't think the 2D Canvas API can really ever get faster than it already is as presented in the library, so for any notable performance changes, WEBGL2D is one of the only options.