Are You Being Too Agile?

Man lying under a tree daydreaming about the future. Proabably not what you'd call "too agile"Confession of a Chronic Futurist

I have a confession to make. I used to be a dreamer at work. Worse than that, I used to invest significantly into putting my dreams into motion.

What did that look like? It meant that when I was tasked with building a Model-2 presentation framework for the company I was working for, I bunkered down for over a month and came out with this massive framework that was going to be a solution to the whole world’s problems. It was basically most of what Struts did and some parts of what Spring does, before either of them existed. There were good things in there, and the company used the framework for many years, but I’m sure more than half the code I wrote never got used. I’d latched onto a good idea and run a marathon, but the team really only needed me to walk a mile.

Pragmatic Redemption

Then came the revelation of my software engineering life: agile. Continue reading

Notes from YOW! 2014: Simon Brown on ‘Agility and the Essence of Software Architecture’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

Simon Brown (@simonbrown) spoke on “Agility and the essence of software architecture”. (Slides, Video)

He started with a great Dave Thomas quote:

“Big design up front is dumb.
No design up from is even dumber.”

Whiteboard covered in a circuit diagram, which looks pretty similar to a typical software architectureSoftware Architecture Agility

He went on to tell us…

Delivering software in an agile way doesn’t guarantee that you’ll develop an agile architecture.

A good architecture enables agility.

He asked “Are monolithic architectures agile?” and proposed the answer, “Well, they could be. Just because you have to deploy it all at once doesn’t mean it’s not agile.”

Which naturally lead into a discussion of what does agility mean? Continue reading

Notes from YOW! 2014: Cameron Barrie on ‘Mobile at Warp Speed’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

A bright photo taken using a slow exposure in a train tunnel, giving the impression of moving at warp speed, such as in the topic of Cameron Barrie's Mobile talk.Cameron Barrie (@whalec), Managing Director and Principle Mobile Consultant at Bilue, spoke on “how to apply solid engineering practices to your mobile applications by understanding common mistakes made, and how to mitigate against the risks.” (Slides)

Mobile: Move Fast

He said it’s crucial to be able to move fast. If you’re not disrupting, you’re probably being disrupted.

You need to be honest about what moving fast means for your organisation: you can’t start with crappy code and processes and just start moving fast. Continue reading

Notes from YOW! 2014: Mary Poppendieck on ‘The (Agile) Scaling Dilemma’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

Lots of empty seats at a stadium. Can Agile scale to this kind of crowd?Mary Poppendieck (@mpoppendieck) spoke about scaling agile teams. (Slides)

She started by saying:

“There’s a big assumption that if agile is good, scaling agile must be good.”

Which made my jaw drop. I make that assumption. It had never occurred to me. Maybe agile techniques don’t work in a larger organisation?

She talked about four constraints on scaling: system complexity, organisational mindset, multi-team communication, and the time and energy of bright creative people. Continue reading

Notes from YOW! 2014: Gojko Adzic on ‘Make Impacts, Not Software’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

Gojko Adzic (@gojkoadzic) spoke about the trouble of aligning strategy and the desired impacts of projects with the implementation of the software.

Palchinsky Principles

An ant carrying a leaf. Ants are known to carry far more than their own weight, a great example of having a big impact.He spent some time discussing the Palchinsky principles, from Russian engineer Peter Palchinsky as documented in Tim Harford’s book ‘Adapt’.

The principles, intended to guide the development of innovations, are:

  1. Variation: We should seek out new ideas and try new things.
  2. Survivability: We should do things on a scale where failure is survivable. (This is why stories should be small. Not so that we can finish them in an iteration, but because they might be wrong.)
  3. Selection: We should seek out feedback and learn from mistakes.

Continue reading

Notes from YOW! 2014: Jeff Patton on ‘User Story Mapping: Discover the Whole Story’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

Jeff Patton (@jeffpatton) was billed to present a “fast paced workshop [where] you’ll learn the concepts of story mapping by building a map collaboratively with others”. He shared lots of great insights about stories but (I felt) really only touched on Story Mapping briefly near the end of the time. Still, I collected some good notes about stories that made me re-think a few things…

He started by showing this great list of wrong things he used to think about stories. Stupid stuff Jeff Patton used to belive about Agile stories Continue reading

Notes from YOW! 2014: Jez Humble on ‘The Lean Enterprise’

I attended YOW! Sydney 2014 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

A man in a suit, who probably works for an enterprise, running in a marathon and looking very agileJez Humble (@jezhumble), co-author of ‘Continuous Delivery’, spoke on The Lean Enterprise, specifically “the principles that enable rapid, software-driven innovation at scale” and how to transform organisations. (Slides)

He briefly covered the three horizons method of innovation and highlighted that you actually need to plan and be executing on all 3 at any one time. They also need separate management styles and reporting lines so that they don’t try to squash each other in departmental trade-offs or management bunfights. The two management styles are explore (discover new stuff) and exploit (capitalise on existing assets). Continue reading

Notes from YOW! 2013: Michael T. Nygard on ‘Five Years of DevOps: Where are we Now?’

I attended Day 1 of YOW! Sydney 2013 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

Small child in a field looking into the distance with binoculars, as someone surveying the current state of DevOps probably wouldn't do.Michael T. Nygard (@mtnygard) is probably best known as the author of the 2007 book ‘Release It!‘, which teaches developers how to look beyond just getting their code working and instead design it from the outset to handle the harsh conditions of production environments. He has since become a DevOps luminary and now works at Cognitect. He spoke at YOW! 2013 about ‘Five Years of DevOps: Where are we Now?’.

Michael started off setting the timeline by pointing out that Chef and Puppet were preceded by CFEngine in about 1993!

He explained how his own experience has contributed to his DevOps insight: he worked as a Dev in Ops for some time, showing the Ops team how to solve some of their problems with Dev-like approaches, but also finding lots of problems with the way Devs created software, which was the chief inspiration for his book. Continue reading

Quick Response to “Test-induced Design Damage”

David Heinemeier Hansson wrote an interesting critique of Test Driven Design two days ago. I understand where he’s coming from with his concern that mindlessly shaping all design around tests can (will?) lead to poor design in some areas. It was interesting, and I like to be challenged and think about these things. Personally, I think his extrapolation to proposing “TDD is Dead” is hyperbole. Here are my thoughts on the bits of his blog that jumped out at me…

“But the testing pyramid prescribes that the unit level is where the focus should be, so people are sucked into that by default.”

It’s probably true that some people are sucked into always defaulting to the unit level. The problem with that is the “always”, but the unit level is where the focus should be. Why? Because that is where the complexity lives – in each unit. The reason we separate code into units is to separate complexity into simpler, composable parts. Testing exhaustively at non-unit levels (i.e. by integrating components) requires a combinatorial explosion of tests… or not testing exhaustively (might be a viable option at  your work, but not at a payments company). We use integration tests as well, but I don’t typically use them to prove all facets of functionality, instead only to prove the components integrate correctly and can deliver the core functions.

“Controllers are meant to be integration tested, not unit tested.”

I work in the Java world, not the Rails world, and I completely agree with this. When I started at Tyro, we were writing unit tests for controllers and integrated web tests for the presentation (i.e. JSPs)  of most features as well. The result? All controller code was tested twice. (It’s pretty hard to test a JSP without hitting its controller.) There’s a dirty word to describe this practice: waste. We ditched controller tests a long time ago now and no one has ever missed them. Controllers are still tested, just not by unit tests. From time to time, some logic in a controller might get a bit complex, resulting in more paths than are practical to web test. The solution is easy there: extract that complexity into another class, unit test the complexity, and web test the integration. Controllers should never be complex. A David points out, they are mostly just an adapter layer between models and views. Let any kind of business logic complexity live in there and it’s got two responsibilities. Your goal should be to make controllers so simple that they don’t need unit tests.

“Finally, the fear of letting model tests talk to the database is outdated”

Yes, and no. Yes, there should not be a fear of testing against the database. In fact, there should be a preference towards it. Not testing against real databases (the same ones you’ll use in Production) leaves a big fat layer of assumptions in your app. However, in large systems with lots of database interaction and great test coverage you can quickly max out the build time if you use the database everywhere. My thoughts: have a preference for DB-backed testing, but not at the expense of developer productivity. The team needs to be aware of when the preference is damaging their velocity and find the balance. I’ve written a lot of data-heavy, back-end Java, so mileage with Ruby on Rails may vary (maybe it never becomes a problem).

“Above all, you do not let your tests drive your design, you let your design drive your tests!”

I don’t think it’s this simple. One great advantage of using tests to drive design is that your desire for simple tests commutes to an implementation of simple classes. Yes, you can probably achieve similar outcomes by doing some more thinking or drawing, but test-first is also a useful tool for driving towards this goal. Not the only tool, or a required tool, but a useful tool. I write lots of simple code at home and rarely write tests for it, but recently I found one part I was writing was a bit gnarly so I decided to write a unit test for it. Upon starting to write the test, I found I wasn’t able to because of the design of the code, and when I thought about the design for a second I realised I was mashing several responsibilities into the one place. Tests can drive designs to bad places, but in my experience they more often drive them to good places.

I often re-iterate to my team that testing is primarily about building confidence in the code, and secondly about building a safety net for those who pass this way next. There is no mandate in TDD for writing tests that do not build confidence (do you TDD getter methods?), or for spending hours on tests that increase confidence by small fractions. I think David is right that there are probably zealots out there who are blindly going down these kinds of paths. Perhaps he’s right that some designs are bastardised by having stringent testability requirements, though I don’t recall seeing a lot of this in my travels. So, he’s right that mindlessly shaping all design around tests can have bad effects, but no software developer should be doing anything mindlessly. I think there is value in letting tests shape your design much of the time, so long as one keeps in mind that there will be occasions where the tests have to be shaped by the design instead.

Everything in moderation, right? Including moderation.

Notes from YOW! 2013: Hadi Hariri on ‘Refactoring Legacy Codebases’

I attended Day 1 of YOW! Sydney 2013 and thought some people might get something useful out of my notes. These aren’t my complete reinterpretations of every slide, but just things I jotted down that I thought were interesting enough to remember or look into further.

A stack of stickers showing the word 'refactor' in a stylised, death-metal-like font.Hadi Hariri is a Developer and Technical Evangelist at JetBrains. He spoke at YOW! about refactoring legacy codebases.

Hadi started by offering reasons for refactoring:

  • increase the understandability of the code
  • decrease the impact of change
  • reduce the cost of change

He described how refactoring relies on culture: Continue reading