Stream Designer
Stream Designer will be a visual, low-code collaborative cloud-based UI built around event streaming. By improving developer productivity, Stream designer will empower users to focus on building critical streaming applications without needing to write code for every use-case.
Industry
Goal
Team
Enterprise SaaS
Introduce a low code, visual interface for building streaming ETL data pipelines.
1 Product Designer, 2 Product Manager, 3 Front End Developers, 8 Back End Engineers, 1 Engineering Manager
Challenges
Stream Designer is an entirely new way to build pipelines, which will require new UI patterns, and a learning curve for new and old users.
Users have different mental models of how Kafka and Confluent fit into their pipelines.
Scoping for this project involves dependecies on other products and constant realignment.
User stories
I am a data engineer working for Acme corporation. I just got assigned a new project where I have to take credit card transaction data stored in an Oracle OLTP database, transform it, and then send it to AWS S3 and Snowflake for the Analytics and Data Science teams to access.
Wireframes
Low fidelity wireframes were used to test my understanding and served as visual cues to guide UX discussions.
A prototype was also created for user research and feedback from internal stakeholders and potential customers.
User Research
Mixed reviews on the newly proposed idea of a connection and connector. The value of separating the authentication from configuration became clear but the flow was confusing.
Many users do not know the differences between key kafka and confluent terms. Streams Designer highlights this problem by having these concepts on one page.
Overall participants struggled a bit through the flow of building a pipeline. It was not always evident what to do next and how things fit together until the pipeline was fully laid out.
Everyone we interviewed saw value in this form of data pipeline builder UI. People were getting excited how it will improve their workflows.
High Fidelity
There were several iterations of the design. Each time feedback was collected from product managers, engineers, designers, documentation writers, and even the executive team. The first versions of high fidelity designs was based on our existing design system.
Design Sprint
At this time, there were several product owners that were concerned about how their products would be affected by the launch of Stream Designer. To get cross team alignment and establish a key user journey, a design sprint was organized. I set up teams of designers, engineers, and product managers to address the Golden Path, an exercise defined by Google’s design sprint methodology. Each team laid out a flow that could provide the ideal experience.
Current Design
The current version of the product will feature our newly created design system. The design team (with the help of an external agency) updated the entire design system including colors, typography, icons, and button styling.
Retrospective
This project encountered several technical challenges that were passed on to the user due to the limitations of Kafka, Connect, KStreams, and KSQL. Confluent decided to adopt Flink for future use of the product.