Laminar: bridging serverless computing and AI-driven code exploration
4 February 2025
EPCC Chancellor's Fellow Rosa Filgueira writes about her recent research into how Laminar can support domain scientists and researchers in using advanced computational tools effortlessly.
A serverless framework is a computing model that automatically manages infrastructure and scaling, allowing developers to focus on writing code without provisioning or maintaining servers. Serverless computing is transforming large-scale data workflows by offering automatic scaling and cost efficiency. However, many frameworks struggle with real-time data streaming and code discovery. This is where Laminar [1,2] comes in: a serverless framework supporting stream-based workflows while integrating AI-driven code search and recommendations.
Laminar builds on dispel4py [3,4], a parallel dataflow framework, to efficiently handle streaming workflows. Unlike traditional serverless approaches, Laminar incorporates deep learning-based semantic code search and recommendations. It consists of four key components (see Figure 1): a client for managing workflows and Processing Elements (PEs), which represent computation units in dispel4py; a registry storing code and metadata for efficient search and reuse; a server orchestrating execution and resource allocation; and an execution engine supporting real-time streaming.
Figure 1: Laminar’s architecture, with its four key components: client, server, registry, and execution engine.
Why is this important?
Researchers and developers often struggle to efficiently find, reuse, and optimise computational workflows. Laminar addresses these challenges with semantic code search and recommendation (see Figure 2) using deep learning models such as UnixCoder [5], CodeT5 [6], and Aroma [7]. This allows users to discover relevant PEs and workflows based on textual descriptions or structural similarities.
Figure 2: Semantic search in Laminar, enabling discovery of relevant PEs.
Additionally, Laminar provides a dual client interface, a CLI (see Figure 3) and Client functions for use within scripts or notebooks, making Laminar more flexible. Users can perform semantic and literal searches, register workflows and PEs, execute workflows, and update descriptions. If a user provides a description for a PE or workflow, Laminar respects it. Otherwise, it uses CodeT5 to generate an automatic description at the time of registration.
Figure 3: Laminar’s CLI commands for searching, managing and executing workflows and PEs.
From research to practical solutions
My research focuses on enabling domain scientists and researchers to use advanced computational tools effortlessly. Laminar exemplifies this vision by abstracting complex parallel processing concepts into a user-friendly framework. It automates descriptions, manages resources intelligently, and enables real-time execution, making it practical for stream-based applications.
Looking ahead: open to collaborations
We are looking for collaborators to help expand Laminar’s capabilities. Future developments include integrating task-based execution (eg Dask or other task-based workflow systems), allowing Laminar to support both serverless streaming and task-based workflows. If you’re interested in working with Laminar or have ideas for extending Laminar, feel free to reach out to me!
References:
[2] https://github.com/StreamingFlow/dispel4py-client/wiki
[3] https://dl.acm.org/doi/10.1145/3624062.3624281
[4] https://github.com/StreamingFlow/d4py
[5] https://aclanthology.org/2022.acl-long.499.pdf
[6] https://github.com/salesforce/CodeT5
[7] https://ai.meta.com/blog/aroma-ml-for-code-recommendation/
