etrospective

A look back at 2024 and a preview of the year ahead.

2024 was a landmark year for stdlib, packed with progress, innovation, and community growth. Looking back, I am struck by the amount of time and effort members of the stdlib community spent refining existing APIs, crafting new functionality, and laying the groundwork for an exciting road ahead. I feel incredibly fortunate to be part of a community that is actively shaping the future of scientific computing on the web, and I am bullish on our continued success in the months to come.

In this post, I'll provide a recap of some key highlights and foreshadow what's in store for 2025. While I'll be making various shoutouts to individual contributors, none of what we accomplished this year could have happened without the entire stdlib community. The community was instrumental in doing the hard work necessary to make stdlib a success, from finding and patching bugs to reviewing pull requests and triaging issues to diving deep into the weeds of numerical algorithms and software design. If I don't mention you by name, please be sure to know that your efforts are recognized and greatly appreciated. A big thank you to everyone involved and to everyone who's helped out along the way, in ways both big and small. ❤️

TL;DR

This past year was transformative for stdlib, marked by significant growth, innovation, and community contributions. Some key highlights include:

• Community Growth: 84 new contributors joined stdlib, tripling the size of our developer community and driving over 4,000 commits, 2,200 pull requests, and the release of 500 new packages.
• Google Summer of Code: four exceptional contributors helped advance critical projects, including enhanced REPL capabilities, expanded BLAS support, and new mathematical APIs.
• Enhanced Developer Tools: major strides in automation included automated changelog generation, improved CI workflows, and better test coverage tracking.
• Technical Milestones: significant progress was made in linear algebra (BLAS and LAPACK), fancy indexing, WebAssembly integrations, and C implementations of mathematical functions, all aimed at making JavaScript a first-class language for scientific computing.
• Future Vision: looking ahead to 2025, we aim to expand our math libraries, improve REPL interactivity, explore WebGPU, and continue building tools to make scientific computing on the web more powerful and accessible.

With stdlib’s rapid growth and the collective efforts of our global community, we're shaping the future of scientific computing on the web. Join us as we take the next steps in this exciting journey!

Stats

To kick things off, some high-level year-end statistics. This year,

• 84 new contributors from across the world joined stdlib, tripling our developer community size and bringing new life and fresh perspectives to the project.
• Together, we made over 4000 commits to the main development branch.
• We opened nearly 2200 pull requests, with over 1600 of those pull requests merged.
• And we shipped over 500 new packages in the project, ranging from new linear algebra routines to specialized math functions to foundational infrastructure for multi-dimensional arrays to APIs supporting WebAssembly and other accelerated environments.

These accomplishments reflect the hard work and dedication of our community. It was a busy year, and we were forced to think critically about how we can effectively scale the project and our community as both continue to grow. This meant investing in tooling and automation, improving our review and release processes, and figuring out ways to quickly identify and upskill new contributors.

Google Summer of Code

The one event which really set things in motion for stdlib in 2024 was our acceptance into Google Summer of Code (GSoC). We had previously applied in 2023, but were rejected. So when we applied in 2024, we didn't think we had much of a chance. Much to our surprise and delight, stdlib was accepted, thus setting off a mad dash to get our affairs together so that we could handle the influx of contributors to come.

GSoC ended up being a transformative experience for stdlib, bringing in talented contributors and pushing forward critical projects. As we detailed in our GSoC reflection, the road was bumpy, but we learned a lot and came out the other side. Needless to say, we were extremely lucky to have four truly excellent GSoC contributors: Aman Bhansali, Gunj Joshi, Jaysukh Makvana, and Snehil Shah. I'll have a bit more to say about their work in the sections below.

REPL

The Node.js read-eval-print loop (REPL) is often something of an afterthought in the JavaScript world, both underutilized and underappreciated. From stdlib's earliest days, we wanted to create a better REPL experience, with integrated support for stdlib's scientific computing and data processing functionality. Development of the stdlib REPL has come in fits and starts, but there's always been a goal of matching the power and feature set of Python's IPython in order to facilitate interactive exploratory data analysis in JavaScript. We were thus quite excited when Snehil Shah expressed interest in working on the stdlib REPL as part of GSoC.

Snehil already covered some of his work in a previous blog post on "Welcoming colors to the REPL!", but his and others' work covered so much more. A few highlights:

• Preview completions: when typing characters matching a known symbol in the REPL, a completion preview is now displayed, helping facilitate auto-completion and saving developers precious keystrokes. Shoutout to Tudor Pagu, in particular, for adding this!
• Multi-line editing: prior to adding support for multi-line editing, the REPL supported multi-line inputs, but did not support editing previously entered lines, often leading to a frustrating user experience. Now, the REPL supports multi-line editing within the terminal similar to dedicated editor applications.
• Pagination of long outputs: a longstanding feature request has been to add support for something like less/more to the stdlib REPL. Previously, if a command generated a long output, a user could be confronted with a wall of text. This has now been addressed, with the hope of adding more advanced less-like search functionality in the months ahead.
• Bracketed-paste: pasting multi-line input into the REPL used to execute the input line-by-line, instead of pasting it as a single prompt. While useful in some cases, this is often not the desired intent, especially when a user wishes to paste and edit multi-line input before execution.
• Custom syntax-highlighting themes: developers who are used to developing in IDEs can often feel adrift when moving to a terminal lacking some of the niceties of their favorite editor. One of those niceties is syntax-highlighting. Accordingly, we worked to add support for custom theming, as detailed in Snehil's blog post.
• Auto-pairing: another common IDE nicety is the automatic closing of brackets and quotation marks, helping save keystrokes and mitigate the dreaded missing bracket. Never one to shy away from a difficult task, Snehil implemented support for auto-pairing as one of his first pull requests leading up to GSoC.

Largely thanks to Snehil's work, we moved much closer to IPython parity in 2024, thus transforming the JavaScript experience for scientific computing. And we're not done yet. We still have pull requests working their way through the queue, and one thing I am particularly excited about is that we've recently started exploring adding support for the Jupyter protocol. Stay tuned for additional REPL news in 2025!

BLAS

Another area of focus has been the continued development of stdlib's BLAS (Basic Linear Algebra Subprograms) support, which provides fundamental APIs for common linear algebra operations, such as vector addition, scalar multiplication, dot products, linear combinations, and matrix multiplication. Coming into 2024, BLAS support in stdlib was rather incomplete, particularly in terms of its support for complex-valued floating-point data types. The tide began to change with Jaysukh Makvana's efforts to achieve feature parity of stdlib's Complex64Array and Complex128Array data structures with built-in JavaScript typed arrays.

These efforts subsequently paved the way for adding Level 1 BLAS support for complex-valued typed array data types and the work of Aman Bhansali, who set out to further Level 2 and Level 3 BLAS support in stdlib. After focusing initially on lower-level BLAS strided array interfaces, Aman expanded his scope by adding WebAssembly implementations and by adding support for applying BLAS operations to stacks of matrices and vectors via higher-level multi-dimensional array (a.k.a., ndarray) APIs.

In addition to conventional BLAS routines, stdlib includes BLAS-like routines which are not a part of reference BLAS. These routines include APIs for alternative scalar and cumulative summation algorithms, sorting strided arrays, filling and manipulating strided array elements, explicit handling of NaN values, and other operations which don't fall neatly under the banner of linear algebra, but are common when working with data.

During Aman's BLAS work, we cleaned up and refactored BLAS implementations, and Muhammad Haris volunteered to extend those efforts to our extended BLAS routines. His efforts entailed migrating Node.js native add-ons to pure C in order to reduce boilerplate and leverage our extensive collection of C macros for authoring of native add-ons and further entailed adding dedicated C APIs to facilitate interfacing with stdlib's ndarrays.

These developments ensure that stdlib continues to lead the way in linear algebra support for JavaScript developers, offering powerful tools for numerical computing. While much has been completed, more work remains, and BLAS will continue to be a focal point in 2025.

LAPACK

Building on the BLAS work as part of an internship at Quansight Labs, Pranav Goswami worked to lay the foundations for LAPACK (Linear Algebra Package) support in stdlib in order to provide higher order linear algebra routines for solving systems of linear equations, eigenvalue problems, matrix factorization, and singular value decomposition. Detailed more fully in his post-internship blog post, Pranav sought to establish an approach for testing and documentation of added implementations and to leverage the ideas of BLIS to create LAPACK interfaces which facilitated interfacing with stdlib's ndarrays and thus minimize data movement and storage requirements. While a good chunk of time was spent working out the kinks and iterating on API design, Pranav made significant headway in adding various implementation utilities and nearly 30 commonly used LAPACK routines. Given the enormity of LAPACK (~1700 routines), this work will continue into the foreseeable future, so be on the lookout for more updates in the months ahead!

As a quick aside, if you're interested in learning more about how stdlib approaches interfacing with Fortran libraries, many of which still form the bedrock of modern numerical computing, be sure to check out Pranav's blog post on calling Fortran routines from JavaScript using Node.js.

C implementations of special math functions

One of stdlib's longstanding priorities is continued development of its vectorized routines for common mathematical and statistical operations. While all scalar mathematical kernels (e.g., transcendental functions, such as sin, cos, erf, gamma, etc, and statistical distribution density functions) have JavaScript implementations, many of the kernels lacked corresponding C implementations, which are needed for unlocking faster performance in Node.js and other server-side JavaScript runtimes supporting native bindings.

Gunj Joshi and others sought to fill this gap and opened over 160 pull requests adding dedicated C implementations. At this point, only a few of the most heavily used double-precision transcendental functions remain (looking at you betainc!). Efforts have now turned to completing single-precision support and adding C implementations for statistical distribution functions. We expect this work to continue for the first half of 2025 before turning our attention to higher-level strided array and ndarray APIs, with implementations for both WebAssembly and Node.js native add-ons.

Fancy indexing

Another area where we made significant progress is in improving slicing and array manipulation ergonomics. Users of numerical programming languages, such as MATLAB and Julia, and dedicated numerical computing libraries, such as NumPy, have long enjoyed the benefit of concise syntax for expressing operations affecting only a subset of array elements. For example, the following snippet demonstrates setting every other element in an array to zero with NumPy.

import numpy as np

# Create an array of ones:
x = np.ones(10)

# Set every other element to zero:
x[::2] = 0.0

As a language, JavaScript does not provide such convenient syntax, forcing users to either use more verbose object methods or manual for loops. We thus sought to address this gap by leveraging Proxy objects to support "fancy indexing". While the use of Proxy objects does incur some performance overhead due to property indirection, you now need only install and import a single package to get all the benefits of Python-style slicing in JavaScript, thus obviating the need for verbose for loops and making array manipulation significantly more ergonomic.

import array2fancy from '@stdlib/array-to-fancy';

// Create a plain array:
const x = [ 1, 2, 3, 4, 5, 6, 7, 8 ];

// Turn the plain array into a "fancy" array:
const y = array2fancy( x );

// Select the first three elements:
const v = y[ ':3' ];
// returns [ 1, 2, 3 ]

// Select every other element, starting from the second element:
v = y[ '1::2' ];
// returns [ 2, 4, 6, 8 ]

// Select every other element, in reverse order, starting with the last element:
v = y[ '::-2' ];
// returns [ 8, 6, 4, 2 ]

// Set all elements to the same value:
y[ ':' ] = 9;

// Create a shallow copy by selecting all elements:
v = y[ ':' ];
// returns [ 9, 9, 9, 9, 9, 9, 9, 9 ]

In addition to slice semantics, Jaysukh added support to stdlib for boolean arrays, thus laying the groundwork for boolean array masking.

import BooleanArray from '@stdlib/array-bool';
import array2fancy from '@stdlib/array-to-fancy';

// Create a plain array:
const x = [ 1, 2, 3, 4, 5, 6, 7, 8 ];

// Turn the plain array into a "fancy" array:
const y = array2fancy( x );

// Create a shorthand alias for creating an array "index" object:
const idx = array2fancy.idx;

// Create a boolean mask array:
const mask = new BooleanArray( [ true, false, false, true, true, true, false, false ] );

// Retrieve elements according to the mask:
const z = x[ idx( mask ) ];
// returns [ 1, 4, 5, 6 ]

We subsequently applied our learnings when adding support for boolean array masking to add support for integer array indexing.

import Int32Array from '@stdlib/array-int32';
import array2fancy from '@stdlib/array-to-fancy';

// Create a plain array:
const x = [ 1, 2, 3, 4, 5, 6, 7, 8 ];

// Turn the plain array into a "fancy" array:
const y = array2fancy( x );

// Create a shorthand alias for creating an array "index" object:
const idx = array2fancy.idx;

// Create an integer array:
const indices = new Int32Array( [ 0, 3, 4, 5 ] );

// Retrieve selected elements:
const z = x[ idx( indices ) ];
// returns [ 1, 4, 5, 6 ]

While the above demonstrates fancy indexing with built-in JavaScript array objects, we've recently extended the concept of fancy indexing to stdlib ndarrays, a topic we'll have more to say about in a future blog post.

Needless to say, we are particularly excited about these developments because we believe they will significantly improve the user experience of interactive computing and exploratory data analysis in JavaScript.

Test and build

Lastly, 2024 was a year of automation, and I would be remiss if I didn't mention the efforts of Philipp Burckhardt. Philipp was instrumental in improving our CI build and test infrastructure and improving the overall scalability of the project. His work was prolific, but there are a few key highlights I want to bring to the fore.

• Automatic changelog generation: Philipp shepherded the project toward using conventional commits, which is a standardized way for adding human and machine readable meaning to commit messages, and subsequently built a robust set of tools for performing automatic releases, generating comprehensive changelogs, and coordinating the publishing of stdlib's ever-growing ecosystem of over 4000 standalone packages. What was once a manual release process can now be done by running a single GitHub workflow.
• stdlib bot: Philipp created a GitHub pull request bot for automating pull request review tasks, posting helpful messages, and improving the overall maintainer development experience. In the months ahead, we're particularly keen to extend the bot's functionality to help with new contributor onboarding and flagging common contribution issues.
• Test coverage automation: with a project of stdlib's size, running the entire test suite on each commit and for each pull request is simply not possible. It can thus be challenging to stitch together individual package test coverage reports in order to obtain a global view of overall test coverage. Philipp worked to address this problem by creating an automation pipeline for uploading individual test coverage reports to a dedicated repository, with support for tracking coverage metrics over time and creating expected test coverage changes for each submitted pull request. Needless to say, this has drastically improved our visibility into test coverage metrics and helped improve our confidence in tests accompanying submitted pull requests.

While we've made considerable strides in our project automation tooling, we never seem to be short of ideas for further automation and tooling improvements. Expect more to come in 2025! ?

Look ahead

So what's in store for 2025?! Glad you asked!

We've already alluded to various initiatives in the sections above, but, at a high level, here's where we're planning to focus our efforts in the year ahead:

• GSoC 2025: assuming Google runs its annual Google Summer of Code program and we're fortunate enough to be accepted again, we'd love to continue supporting the next generation of open source contributors.
• Math and stats C implementations: expanding our library of scalar math and statistics kernels and ensuring double- and single-precision parity.
• BLAS: completing our WebAssembly distribution and higher-level APIs for operating on stacks of matrices and vectors.
• LAPACK: continuing to chip away at the ~1700 LAPACK routines (!).
• FFTs: adding initial Fast Fourier Transform (FFT) support to stdlib to help unlock algorithms for signal processing.
• Vectorized operations: automating package creation for vectorized operations over scalar math and statistics kernels.
• ndarray API parity: expanding the usability and familiarity of ndarrays by achieving API parity with built-in JavaScript arrays and typed arrays.
• REPL: adding Jupyter-protocol support and various user-ergonomics improvements.
• WebGPU: while we haven't formally committed to any specific approach, we're keen on at least exploring support for WebGPU, an emerging web standard that enables webpages to use a device's graphics processing unit (GPU) efficiently, including for general-purpose GPU computation, in order to provide APIs for accelerated scientific computing on the web.
• Project funding: exploring and hopefully securing project funding to accelerate development efforts and support the continued growth of the stdlib community.

That's definitely a lot, and it's going to take a village—a community of people dedicated to our mission of making the web a first-class platform for numerical and scientific computing. If you're ready to join us in building the future of scientific computing on the web, we'd love for you to join us. Check out our contributing guide to see how you can get involved.

A personal note

As we look ahead, I'd like to share a personal reflection on what this year has meant to me. Given our growth this year, I often felt like I was drinking from a fire hose. And, honestly, it can be hard not to get burned out when you wake up day-after-day to over 100 new notifications and messages from folks wanting guidance, answers to questions, and pull requests reviewed. But, when reflecting on this past year, I am awfully proud of what we've accomplished, and I am especially heartened when I see contributors new to open source grow and flourish, sometimes using the lessons they've learned contributing as a springboard to dream jobs and opportunities. Having the fortune to see that is a driving motivation and a privilege within the greater world of open source that I do my best to not take for granted.

And with that, this concludes the 2024 retrospective. Looking back on all we've achieved together, the future of scientific computing on the web has never been brighter! Thank you again to everyone involved who's helped out along the way. The road ahead is filled with exciting opportunities, and we can't wait to see what we will achieve together in 2025. Onward and upward! ?

stdlib is an open source software project dedicated to providing a comprehensive suite of robust, high-performance libraries to accelerate your project's development and give you peace of mind knowing that you're depending on expertly crafted, high-quality software.

If you've enjoyed this post, give us a star ? on GitHub and consider financially supporting the project. Your contributions and continued support help ensure the project's long-term success and are greatly appreciated!

The above is the detailed content of etrospective. For more information, please follow other related articles on the PHP Chinese website!