How to Choose Your First Microservices

Featured

Are you early in your microservices journey? Maybe you’ve decided you need to start deploying applications outside your monolith but you haven’t cut any code yet. Or maybe you’ve put your first few services into production and addressed some of the first pains that happen when you start on that path.

A large number of interacting LEGO cogs making one large machine, similar to a monolith from which you want to break out your first microservices

A common question that comes up for teams at around this time is:

“What should we split out into a microservice first?

And why?”

Continue reading

An Open Letter to Scott Morrison About Creating an Effective Corruption Watchdog

Dear Prime Minister

A cute, fluffy dog holding a thick roll of cash.

I’m sure you’ll agree that Australia’s democracy is one of the most core foundations of why this nation is such a great place to live and work.

I’m sure you’ll agree that political corruption has no place in a representational democracy if it is to fairly deliver the outcomes that its citizens expect and deserve.

I’m sure you’re not so naive as to believe that all those who find themselves in positions of political power are pure of heart and entirely selfless. We’ve all heard that power corrupts and absolute power corrupts absolutely. and our own Australian Public Service Commission recently revealed that 4,300 federal public servants believe they witnessed corruption in just one year.

Seeing as we likely agree that a fair, untainted democracy is of critical importance and that political corruption is essentially inevitable if left unchecked, I’m sure you’ll also agree that it makes sense to invest in protecting our democracy. Obviously, the way to do that is by detecting and removing corrupt actors, and ensuring measurable punishment occurs to deter those who would seek to follow that path. No doubt this is the reason why you proposed in December last year the creation of a Commonwealth Integrity Commission.

However, I find your current plans for addressing corruption in our parliament and our public service lacking the conviction of someone who truly believes the problem should be dealt with.

In the past, you have described the idea of a corruption watchdog as a “fringe issue”.[1] You have put forward a proposal which has been widely criticised by experts in corruption law and investigation.[2] When an alternative proposal was just this week passed as a bill by our Senate[3], your government voted to not even let the House of Representatives discuss it.[4]

As I understand it, your own proposal does not allow for public hearings or findings[1], but instead seeks to do everything behind closed doors. It isn’t too hard to imagine that an organisation whose purpose is to investigate corruption, but which has no oversight and no public reporting, might itself be susceptible to corruption initiated by those who it would benefit greatly to not be investigated. If we’re going to properly address corruption, it’s clear that transparency is a necessary ingredient in the process. (See what I did there?)

In announcing your government’s plan, you characterised it as a “measured” approach.[1] Were you suggesting that there needs to be restraint in pursuing corruption? Are you insinuating that the proposals made by other parties are excessive? Your government’s proposal commits a cool $30m to the new Commission, although large parts of that are continuing funding from the current activities of existing bodies. $30m probably sounds like a huge expense to most Australians but, put in the context of the government’s $500b of estimated expenditure in the 2019-2020 budget year, it represents only 0.006% of the budget. When we are talking about the protection of our democracy, I don’t think many people would see it as unreasonable that we spend 0.01% or even 0.02% of our budget on ensuring our government is not compromised. A measured approach to policing corruption may sound appealing to a miser, but a well-funded and highly effective approach seems far more desirable.

Finally, it is reported that your planned Commission is to be prevented from undertaking retrospective investigations.[5] I can’t imagine any moral justification for binding the watchdog in this way. People aren’t asking for new anti-corruption laws by which politicians and public servants must now abide; it’s simply about commissioning law enforcers. When a new police office joins a police force, are they prevented from investigating crimes that occurred before they joined? Of course not. Why should those who broke the law in the past be protected simply because there was insufficient law enforcement at the time? This kind of restriction would serve only to give the public the impression that you or your mates have been acting corruptly and are seeking to protect your past actions from scrutiny. Why else would your government not want past corruption investigated?

So, it seems we agree on a lot of fine principles, but your proposal for moving forward with eradicating corruption from Australia’s political system and public sector is pretty woeful. Thankfully, the way forward is clear: make this a core issue instead of a fringe issue; treat the opinions of other parliamentarians with respect instead of silencing their voices; ensure the corruption watchdog is itself protected from corruption through adequate transparency; invest whatever needs to be invested to get the best outcomes; and give the commission the scope to investigate whatever it thinks is most important. After all, what we all want is an independent corruption investigation body to protect our democracy. However, a commission which has been created without adequate funding, without protection from corruption inside itself, and without the ability to look into the past isn’t really that independent, is it?

Yours faithfully,

Graham Lea

[1] Scott Morrison announces anti-corruption commission, buckling to crossbench pressure, December 2018, https://www.abc.net.au/news/2018-12-13/scott-morrison-creates-federal-integrity-commission/10614658

[2] Government’s proposed federal ICAC ‘a sham’, say former senior judges, Sydney Morning Herald, January 2019, https://www.smh.com.au/national/government-s-proposed-federal-icac-a-sham-says-former-senior-judges-20190125-p50tpa.html

[3] Senate votes to set up federal anti-corruption body, SBS News, September 2019, https://www.sbs.com.au/news/senate-votes-to-set-up-federal-anti-corruption-body

[4] Adam Bandt, https://twitter.com/AdamBandt/status/1171338568905580544

[5] Morrison government announces new federal anti-corruption commission, The Guardian, December 2018, https://www.theguardian.com/australia-news/2018/dec/13/morrison-government-announces-new-federal-anti-corruption-commission

Image credit:
Cute Dog with Money‘ by OTA Photos

Using Microservices to Make Costly Deployments Less Frequent

This article on costly deployments is part of a series on How to Choose Your First Microservices.

Switching to a new software architecture style is costly. I advocate choosing where to employ microservices first by using them to solve existing problems in the team. This helps the organisation extract value from the new architecture with each step towards the new world. This series is working through a set of problems that your org may have, and how microservices might help solve them.

Tricky Deployments

A fighter jet on an aircraft carrier being signalled to launch by two ground staff. Launching a fighter jet is a costly deployment, just as deploying a monolith may be.

Some components of your system will be tricker to deploy than others. This is true regardless of whether you’ve already got microservices or you’re deploying a monolith. Some components might be in use by 1000s of customers 24×7, while others might only be used once a month. Upgrading a component might require other components to be placed into a certain state, while others can be upgraded independently of the rest of the system. Coordination may be required between multiple teams in order to effect a trouble-free deploy of some components, whereas others might be actioned by a single person.

Continue reading

Using Microservices to Solve Developers Stepping on Each Other’s Toes

This article on reducing merge conflicts is part of a series on How to Choose Your First Microservices.

Switching to a new software architecture style is costly. I advocate choosing where to employ microservices first by using them to solve existing problems in the team. This helps the organisation extract value from the new architecture with each step towards the new world. This series is working through a set of problems that your org may have, and how microservices might help solve them.

Is your team wasting a lot of time working around each other?

A side-collision car crash at an intersection, surrounded by emergency personnel. Like this, merge conflicts can be messy and dangerous.

If you have a large team of developers working in a single code base, it’s inevitable that the work of some team members will interfere with the work of others. Many of us are now using tools (like git) and processes (like continuous integration) that can lessen the impact of interference. (Although other modern practices like feature branching can make conflicts worse.)

Continue reading

Using Microservices to Solve Slow Build Times

This article on slow build times is part of a series on How to Choose Your First Microservices.

Switching to a new software architecture style is costly. I advocate choosing where to employ microservices first by using them to solve existing problems in the team. This helps the organisation extract value from the new architecture with each step towards the new world. This series is working through a set of problems that your org may have, and how microservices might help solve them.

Slow Build Times?

A snail crossing a road, which might happen faster your slow build time.

Does your team have an application that takes a long time to build and run tests? Is a large portion of the development team working on it? That could be a huge amount of idle time you’re pouring down the drain. The classic XKCD ‘Compiling’ comic makes light of developers enjoying the time they spend waiting for their code to build. The truth for most devs is they would far prefer to be making fast progress on their work than sitting around waiting.

Continue reading

Feature Flags and Test-Driven Design: Some Practical Tips

A country road with a fork going off to one side, symbolic of a feature flag in codeIn 2018, our team spent a lot of time working with feature flags and test-driven design (TDD).

Our project was to effect an architectural change to our system, changing the source of truth of some data, moving it out of the database owned by a legacy monolith into a new database controlled by a new microservice. However, much of the code requiring the data would remain in the monolith.*

Some examples of the types of things we feature-flagged are:

    • whether to go down a refactored code path or not;
    • whether to publish messages to a message queue when a certain event occurred;
    • how to publish those messages (we tried multiple variations of batching and transaction boundaries to achieve acceptable performance);
    • whether to just delete messages at the receiving end or actually handle them; and
    • whether to use a local source of data or remote one.

We were working on a pretty important piece of code; the kind of business function where, if we stuffed it up, someone would probably have to spend several days doing remedial fixups or making phone calls to chase up millions of dollars. Continue reading

Time to Evolve Again

My mid-2000s, fresh-faced, professional look from around the time I joined Tyro.

After 12.3 years, yesterday was my last day at @Tyro. 😢

It’s been a huge journey, from a perilous payments startup with less than 20 employees and only ☝🏻 customer, to now a VC-backed, fully-licensed bank with well over 400 staff that is Australia’s 5th largest EFTPOS provider (by transaction volume).

I’ve done time as a Software Engineer, Development Lead, Engineering Lead (part of the management team), Technical Project Manager, Team Lead, Product Lead, and this last year, back to Software Engineer. I must have hired 50+ (awesome) people and interviewed many times that.

Continue reading

“This work kinda sucks right now”: Staying motivated in the uninspiring phases of long projects

In April 2015, I was the technical project manager of six software teams all working on one gigantic project. It was the biggest challenge the company had ever taken on. The goal: create and stand up a transaction account product for small and medium business so that we could apply for an unrestricted bank license.

Tired marathon runner with hands on hips

We were 12 months in and were about to finish the project bang on the week we’d estimated, when the inevitable happened: we received a new raft of requirements that would add at least another 3 months of work. The extra work wasn’t glamorous; it was mostly security enhancements, disaster recovery preparation, and a complicated behind-the-scenes tweak of the product that added little benefit for users but was a requirement from a regulatory point of view.

Understandably, morale took a hit, but we needed to keep the pace up or we’d risk missing the deadline we’d been set for acquiring our license. After an interesting discussion with my father-in-law about the project, I found a new perspective on where we were at, and it prompted me to write the internal blog below to help motivate the team towards our final goal. Continue reading

9 fascinating things I learned while coding up the rules of a board game

I recently decided I was going to take the rules of the board game Forbidden Island and write them up as code.

A screen full of code in an IDEI guess that sounds like a weird thing to just decide to do, doesn’t it? It’s actually one part of a bigger goal I have at the moment of teaching myself some practical machine learning. As part of this journey, I heard a great idea from YouTuber Jabrils to set yourself a significant challenge that you’re interested in, and to work towards surmounting that challenge. For Jabrils, his challenge was getting an AI to control a Forrest Gump character to run around a course in a game. For my challenge, I’ve decided I’d like to build an AI that can play Forbidden Island. (And win!)

Obviously, if you’re going to have an AI play a game, you first need a digital version of the game for the AI to play. Continue reading

The Amazing Benefits of Exercism

A couple of months ago, a good friend at work (@rodeoclownII) sent out an email (or blog, or Slack message, or something) about exercism.io. It’s a cool little website for practising programming in various languages (currently about 45) by implementing koans, or small exercise problems.

I’ve been working through the Kotlin problems since the start of the year and I’ve found it super useful, so I thought I’d write this quick blog to recommend it to others.

Why I like exercism.io so much

Actor Anthony Hopkins seemingly performing an exorcism with the words "Back to hell, demon!" Probably unrelated to exercism.io

Just to be clear, we’re not talking about this kind of exorcism.

The most obvious benefit of this site is the opportunity to practise using different elements of the Kotlin language and standard library. Being challenged to solve problems which you might not necessarily come across during day-to-day work can lead you to discover parts of Kotlin that are really useful but which you otherwise might not have found a need to go looking for. Also, the problems are all implemented in a test-driven way, which feels very natural for me as a long-time practitioner of Extreme Programming/TDD. Continue reading