Radical Agility has been in full swing at Zalando for more than a year now, and in the three months since I started working at Zalando, it has had a big impact on me and my team.
I’m a Big Data Engineer, and although I’ve already grown into my team and their processes, everything still feels new and exciting when it comes to how we work, and this is due to Radical Agility.
Zalando’s Radical Agility methodology is governed by a set of simple, clear principles to guide the decisions that teams need to make. These are collectively known as our Tech Constitution. Each individual, team, and members of management have rights and responsibilities that provide Zalando Tech with a framework for decision-making.
This framework gives teams the right to make decisions about the means and tools of delivery and operation, as well as owning and operating applications efficiently for the long term, according to their purpose. It is this exact part of the Tech Constitution that our team Saiki recently put into practice.
We’re currently building a data integration platform in the AWS Cloud. You might have recently heard from us in our Apache Showdown: Flink vs Spark blog post. We’re split up into two focus groups, one working on stream processing for near real time analysis of incoming data, and the other one building a Data Lake as a centralised place for company wide data storage and analysis.
Under Radical Agility, we have the opportunity to use whatever working style suits the team, utilising the strengths of each team member. Both of our focus groups use Scrum, which is facilitated via two week sprints. Recently, I joined the Data Lake focus group and had the opportunity to take responsibility for tasks directly contributing to the team’s progress for the first time.
My colleagues were working on the performance evaluation of Apache Accumulo – a key value store that was recommended by our Security Architect for our specific use case, because of its built-in layer of security. So far, they had been working on gathering insights and setting up a cluster environment, in addition to already performing some first benchmark tests and fixing early difficulties. When it came to review the previous sprint, some performance test results were presented, but we were not fully able to interpret the results.
After the review meeting, we held our Retrospective. Herein we picked up the general discussion about the purpose of the current project’s path – a key component of the Tech Constitution. We soon realised that nobody felt entirely comfortable with Accumulo and it looked like our wish to proceed actually drove us off track.
Exploring and questioning the validity of certain technologies is something that is extensively practiced within Zalando Tech, and this was also part of my own team’s process when dealing with Accumulo. The overall goal of the performance testing was that in the near future we would be able to answer the question: “Does Accumulo fit our needs or do we have to search for another solution?”. Based on the current situation, nobody could estimate how long it would take to come to an acceptable answer.
Our next planning meeting involved a discussion about how we wanted to proceed, where we only managed to align on the next three steps: Getting more familiar with Accumulo, planning and performing benchmark testing, and making an informed decision about its suitability for our case.
As none of our three aligned steps could be time-estimated without a high level of uncertainty, we had to break them down into smaller tasks. This process is yet another example of how Radical Agility allows teams to choose the right approach for their work. The team gathered again to look into the Accumulo documentation to find specific areas where we could conduct research. Having a clear indication of topics to look into made it easier to estimate the amount of time each research task would take.
We had another meeting with our Security Architect that spawned a clarifying discussion about the security requirements of the Data Lake, as well as its technical implications. After the dust had settled, we sat down and created tasks for each of the research topics agreed upon. These tasks were tweaked by the team, and we were able to make an attempt at planning our next sprint.
The Scrum process we followed, thanks to the autonomy we have with Radical Agility, allowed us to detect issues in our workflow and properly define the purpose of our project. Having the scheduled time to discuss and review our work was essential in getting us back on track.
We fixed the focus of our project, and our agile working method helped us to do so.