discovering asian technology
Here a few impression of last weekend’s coding for fun event at Exoweb, unfortunately I didn’t take many pics.
Technologies applied included: Pike, OpenGL, Flash, Chipmunk, Twitter, cURL, Festival, C, Boarduino, Arduino, LED, and many more.
Update: Martin has offered me to add his pics to my flickr account. Thanks, Martin :).
In software development, the ticket plays a central role in the development process. In Latour’s terms, the ticket is a non-human actor which carries a message. More generally, it’s a virtual or real object which contains a description of a specific task or a group of tasks. Most tickets are part of a larger project. A project is thus broken down into a number of ideally independent tickets which can then be implemented by the individual developers. In a projects, the final product’s implementation is the sum of all of its tickets.
Many books and papers on agile software development advocate the use of user stories for requirement specification. Wikipedia defines user story as follows:
“A user story is a software system requirement formulated as one or two sentences in the everyday language of the user. […] The user stories should be written by the customers for a software project and are their main instrument to influence the development of the software.
User stories are a quick way of handling customer requirements without having to elaborate vast formalized requirement documents and without performing overloaded administrative tasks related to maintaining them. The intention with the user story is to be able to respond faster and with less overhead to rapidly changing real-world requirements.
[…] Before a user story is to be implemented, an appropriate acceptance procedure must be written by the customer to ensure by testing or otherwise determine whether the goals of the user story have been fulfilled.”
How are tickets, tasks, and user stories related? As mentioned before, a ticket is just a container for a requirement description. At Exoweb, we use user stories to define the client requirements. The tasks are subsequently defined by the developers as the individual steps that need to be implemented in order to fulfill the customer’s requirements for a user story. There are several things we can learn from Wikipedia’s definition above:
The literature insists that user stories should be written by the customers. Our own experience, however, showed that the quality of the user stories improved considerably when the client was assisted by some members of the development team. Currently, we organise an additional meeting a few days before the sprint meeting, in which the project manager and the team leader are preparing the tickets for the next sprint together with the customer.
User Stories in Detail
In User Stories Applied Mike Cohn presents a formal approach to writing user stories. He suggests the format:
“As a [role] I want [something] so that [benefit]”
For example:
“As a visitor of the website I want to be able to filter search results by additional keywords so that I don’t need to look through a large list of possibly irrelevant results.”
Why should agile teams care about user stories? The main reason is because user stories encourage communication and are written in a language that both, developers and client can understand. A user story should be precise enough for the client to express what he wants and for the developers to estimate it’s complexity. For a more detailed explanation of the difference between user stories and use cases, check out this article.
The Ringier team at Exoweb uses estimation poker for breaking down user stories into tasks. Estimation or planning poker is an excellent way to learn more about the details of a user story. Each developer is asked to estimate the user story and to explain his estimate. This encourages consensus among developers and it allows the team to agree on the necessary technical tasks which need to be implemented for a specific user story. As the client is present during the poker, the team members can ask questions to avoid misunderstandings. When using estimation poker for planning tasks, it is very important to avoid anchoring:
“Who ever starts the estimating conversation with, “I think it’s 50 days” immediately has an impact on the thinking of the other team members; their estimates have been anchored, i.e. they are all now likely to make at least a subconscious reference to the number 50 in their own estimates.”
Defining “Done”
As we have seen in the Wikipedia excerpt above, every user story should be completed with a definition of “Done”. This definition of is ideally a description of the demo the client wants to see at the next sprint meeting. More generally, it’s an agreement between the development team and the client in which the client defines which conditions a ticket has to fulfill in order to be considered “Done”. At Exoweb, we additionally use this definition as a base to define the functional tests for the ticket.
Ticket components
At Ringier team, each of our tickets consists of the following elements:
Ideally the ticket should contain all information a developer needs to start working on a it, and all possible impediments should be solved before a ticket is accepted for a sprint. This allows to minimize the risk of the implementation differing from the client’s expectation. Additionally, it facilitates the developer’s work because he knows exactly what he has to do in order to complete a ticket, and it enables the project manager to decide when to accept a ticket for demo for the next sprint meeting.
Agile software development is a process which requires constant tuning. At Exoweb, we take every sprint as an opportunity to reflect on and improve our performance. In the sprint review meeting, we discuss the last sprint, analyse the velocity and burndown charts, discuss problems and their solutions, and find ways to avoid past errors. For the current sprint, the Ringier team has decided to make the following adaptations to the development process:
The team will be experimenting with these changes during the current sprint and then decide in the next sprint review meeting if the adaptations have actually solved the discussed problems.
The Process
Every ticket which has been committed to the code repository, has to go through two independent review processes: the code review, and the demo review. The code review is done by the other developers in the team, suggesting improvements and tweaks, in order to ensure code quality. The demo review is a process in which the developer demos the ticket features to the project manager and the on-call developer who are then responsible for closing the ticket and preparing the demo for the sprint meeting.
The Changes
For the current sprint, we have decided to adapt the demo review process. We now assign two developers to each ticket: the maintainer and the tester. While the maintainer remains responsible for the actual implementation of the features, the tester will be writing automated tests and demo the ticket in the sprint meeting. The tester will thus replace the on-call developer in the demo review. Additionally, with the automated tests being written by someone else, tester and maintainer have to agree on the ticket design before starting to write their codes. And while writing the code, the maintainer will be able to use the tests as a benchmark.
The Reasons
With every sprint, the complexity of a project grows, and even small changes can produce major damage in the existing code if no precautions are taken. While the importance of automated tests is often underestimated in the beginning of a project, it becomes increasingly important when the project advances, as even seemingly minor changes might have unexpected side-effects.
So far, programmers were responsible for writing tests for their own code. This means design, implementation, and testing was done by a single person. As the team has previously decided that they preferred code reviews over pair programming, design problems were only discovered in the code review. We thus hope that above adaptations will lead to better design decisions, cleaner code, and less bugs which turn will reduce the need for refactoring after code reviews and increase the team’s velocity.
“The most efficient and effective method of conveying information to and within a development team is face-to-face conversation.”
Agile software development has started out with white-boards, story cards, and face-to-face meetings. It’s main goal was to develop software according to the customer’s requirements, to accommodate new requirements during the development process, and to create an environment which encourages self-organisation and team responsibility.
However, the real world is somewhat different. We have to deal with customers, contracts, and project reports. Without tools and processes it is hard to meet these needs:
- How do we write contracts for agile projects?
- How can we record meetings, discussions, and decisions in an efficient way?
- How do we meet the managers’ needs for documentation and project reports?
Negotiating contracts for a software product almost always involves understanding what the customer wants, creating a product backlog, making financial and time-based estimations. But agile is about iterations, about accommodating changing requirements, about constantly rewriting the product backlog, about estimating user stories, not product specifications.
How do we write contracts for agile projects?
In traditional software development, a fixed-price contract is often limited by time and scope. With these two constants, it is possible to make a cost estimate and schedule resources. In an agile project, however, the scope keeps changing, and time requirements need to be renegotiated according to the scope. How can we map the original scope to the new scope to find out which services are not covered by the initial contract? How can we compare dropped features to newly added features to negotiate contract amendments? And how do we decide if a project is done according to the contract?
It seems that with agile methods, it is always difficult to give estimations, and the fairest contract would be based on paying a team to work on a piece of software for a certain amount of time. However, this seems not to be possible in most cases. Hence, we need to constantly keep in mind the product backlog. We need to warn clients when they ask for new features, we have to tell them that adding these means removing other features or allowing for more time with all the related financial consequences.
This means that the product owner cannot simply come into a sprint meeting presenting the next sprint’s features. New features and changes have to be discussed with the respective financial divisions before they are implemented, or at least their consequences have to be transparently communicated to the responsible managers. These aspects make agile planning less flexible, and it means that much more people are involved in the planning of a sprint than the original chicken- and pig metaphor suggests.
Additionally, a client will often require a report of which tasks required more time than expected and why. It is thus important for agile projects to be able to provide a detailed documentation of meetings, discussions, and decisions.
Tools for Agile Projects
The Agile Manifesto values “individuals and interactions over processes and tools”. However, as Liz Barnett, chief editor of the agile journal says in the article on Tools for Agile Projects, “this does not mean that individuals and interactions must replace processes and tools”. Different software tools have been developed in the last couple of years. Tools range from issue trackers like Trac to commercial products like Thoughtworks’ Mingle, Atlassian’s Jira, as well as many others. The goal of all these tools is to provide support for the management of agile projects.
However, the needs of the people involved in agile projects are different. The goal of this post is not to compare these different tools, but rather to identify the expectations:
What the developers want
Developers mainly want a place (ideally a single URL) where they can find all information related to a specific task. When they start woking on a task they want it to be well documented and to include all the information they need. Developers don’t like to be stuck with a task for which some information is missing or pending product owner approval.
What the product owner wants
Product owners want to see a products’ progress. They want to know how long it takes to implement a taks, they need an estimate of when the product will be finished, and they want to know where they can find the latest version of the software.
What the scrum master wants
Scrum masters want an overview of the project. They want to know which tasks are in progress, which are completed, and what remains to be done until the end of the sprint. The scrum master also wants to know where to find the implementation of a specific task, if it has been tested, and who is responsible for it.
What the managers want
The managers want numbers and histories. They want to know which tasks caused a delay and why. They want to know who is responsible for a decision: Is it the team or the client? They want to be able to reproduce all information related to a task, sprint, or product. They want to make statistics, know how long a task took compared to its original estimate.
Exoweb shows a strong commitment to Free and Open Source Software (FOSS), most of the computers run Linux, much of the software developed is based on Python’s Django framework, and a few people are actively involved in Open Source projects. One of my tasks at Exoweb is to strengthen the relations between Exoweb and the FOSS communities by encouraging FOSS-related activities, and finding ways for Exoweb to collaborate with the communities.
The Open Source Business Model
I was invited to a discussion about Open Source in China at the Programmer magazine two weeks ago, and one of the things I learned was that companies trying to build on FOSS don’t seem to be very successful in China, and many of them are struggling for survival. One of their main worries is how to actually make money with Open Source. Of course it is true that if a company wants to build a completely new FOSS product, it would be hard to find a way to finance the development.
However, as the Exoweb example shows, you don’t really need to start with a new product, but you could just build your business based on existing FOSS software. In that case, the company doesn’t need to invest in product development at all, but can immediately start their business by customising an existing product for the needs of their customers. This is one of the main reasons why the FOSS business model has proved to be so successful in the West.
Contributing Back
Using FOSS products is an excellent way to get started. And once you have become fairly familiar with the internals of the products you are using, you will might start building your own reusable components based on the existing software. You might also discover bugs, fix them, and send patches to the community. As this will improve your final product you are building for a client, the time spent on building components and patches can be, at least partly, covered by the customer’s mandate. Of course, the client has to agree to that, especially if the code is being published with an Open Source license.
However, contributing to FOSS might in many cases require more time than just writing the code for a specific project. The patch has to be documented and packaged according to the the FOSS project’s standards, and the the reusable components have to be maintained and improved to guarantee future reusability. While one can easily understand the reasons for contributing back a patch, questions might arise as for the components to be open sourced or not. In some cases, like Linux which is based on GPL, developers are in theory constrained by the original project’s license. Though despite this fact, I strongly doubt that the GPL could be enforced in China.
There could be other reasons for a company to open source a component’s or extension’s code: Maintenance and marketing. When the company is the only owner and developer of the extension, they will be forced to change the code whenever they decide to upgrade to a newer version of the original product. In an Open Source world, however, there might be other developers interested in maintaining the code, or even better, the code is added to the project’s core and is thus maintained by the community. Additionally, creating and maintaining Open Source code can have a considerable marketing effect for the company.
Exoweb and the FOSS World
Exoweb is trying to actively engage in the Django and Python community, as well as other local communities like the Beijing Linux User Group (BLUG). They have hosted the Django Sprint in Beijing in December last year, and are currently planning other similar events. Additionally, some of the teams are developing extensions which they want to add to Django’s contribution repository, and a few developers are working on their own FOSS projects. In order to encourage it’s employees, Exoweb additionally offers a 10 per-cent time during which everyone is free to work on whatever they want. Exoweb also offers hosting for internal FOSS projects on contrib.exoweb.net.
At Exoweb, daily stand-ups are integrated part of the daily routine. Every team will get together at some point of the day and spend 5-15 minutes discussing their current progress. Stand-ups or daily scrums are short team meetings during which every member of the team answers the following three questions:
From an observer’s point of view, stand-ups are status meetings during which developers report their work. However, from a broader scrum perspective, they are much more than that. Daily scrums are about team self-organisation, commitment, and focus, as Martin Fowler explains in his excellent article It’s not just standing up. During the stand-ups, the members of the team update each other instead of a manager. To keep the energy level high and to act as a daily motivator, it is important to keep the daily scrums short. Standing up is a great way to accomplish this.
At our team, we have a pinboard with all the tasks of the current sprint. The task cards are labelled with the Trac ticket number and a short description. The cards also contain a table which is updated every day with the time spent on the ticket and the remaining story points. Whenever a team member picks a ticket, the person’s picture is placed next to the task. Tickets are organised in five different columns on the board: New, Ongoing, Commited, Tweaking, and Done. Additionally, the pinboard contains the updated burndown chart of the project.
Before the daily stand-up, every programmer has to update his ticket, report the code quality and the number of code reviews he has done on the previous day. The team will then gather around the pinboard for the daily scrum.
The role of the pinboard is crucial: It’s a way for every team member to observe the current sprint’s status. At a glance you can see who is currently working on which task, how many story points have been completed in the current sprint, and how many points remain to be done. The pinboard also doubles as a meeting place for daily stand-ups and team members discussing tasks. Despite using a web-based task management system, having a real pinboard definitely improves the scrum experience, facilitates project overview, and acts as the central point of orientation for a project.