Thinking about developer products means thinking about developers. What is different about software development as compared to other professions or interest groups? In particular, what needs to be considered when building tools and services for developers?
There are three traits that shape software development. None are individually unique, but in combination they have a significant impact on the craft, and business, of development.
Software development is pure knowledge work
A number of professions are knowledge work, but software development is possibly the purest expression of the concept. An accountant works with data, processing numbers and defining rules. A software developer works with code that processes data, and that code itself is data, and can itself be processed.
In many cases, there is no factor of production other than an individual and a computer. Once code has been created it can be packaged and used extremely broadly, at very low marginal cost.
Tool use and tool creation are on a continuum
The barrier between building something to solve a problem, and generalizing that thing to solve similar problems is low — and some environments like Go and NodeJS directly encourage this kind of transition.
One of the more famous rules of thumb in technology is Moore’s law — computing power (or transistors in an area) doubles every 18 months. One of the reasons this held was that it formed a positive feedback loop — increased computational power contributed to more sophisticated tooling which resulted in more sophisticated chip design.
The software feedback loop is similar and substantial. Once a tool is created, developers can use that tool, including using that tool to build an improved tool.
Importantly, this holds regardless of the nature of the tool. For example, a powerful, but proprietary, development environment can be used in the creation of a similarly powerful, but open source, tool.
There is very little difference between “craft” and “industrial” production
As Tesla has discovered, there is a big difference between staffing a highly scaled modern car production facility and a more boutique operation. Both the resources involved and the day-to-day jobs of the people working on the line are drastically different. When you move to more artisanal industries, such as woodworking vs furniture manufacturing, the differences are even more clear. Software development is not like that.
Software development certainly has scales of employment. There are cube farms of thousands of programmers churning out Java for banks, small shops that deliver clever, custom solutions, and individuals developing new frameworks and services. However, the work of those people, and facilities they have available, are pretty much the same. There isn’t some economy of scale that lets you generate exponentially more code, and there isn’t some major capital investment that will vastly increase output. Any sufficiently useful “enterprise” tool is also available to craft usage, given a sufficient time frame. You don’t have to be working at BigCoSoftware in order have effectively limitless server infrastructure, or a CI environment, or the latest testing tools.
There isn’t even really a split between the developers working in craft versus industrial production of code. According to Stack Overflow, around 80% of developers that filled in their survey have done some work outside of their professional context (such as open source or hobby coding), and that number is likely to be considerably higher if it includes those that have done so at any point in their career.
What this means
These factors interrelate in interesting ways. Craft production mirrors industrial production because tool development is such a natural outcropping of development. While an equivalent in manufacturing, such as 3D printing, required special focus on the part of originators like Adrian Bowyer, the equivalent means-of-production are almost routinely developed as a side effect of doing other work in software.
Sometimes large companies develop practices that require specialty and a certain degree of scale, such as DevOps or SRE roles. Even then, there is a strong tendency to develop approaches which partially obviate the need for such roles, such as the Lamba/Cloud Functions/Up.
Software developers interact more like craftspeople than most professions. There are strong traditions, and communities of tradition — and they come with attendant folkways, discussion, and direct interaction.
Finally, this also means that there tend to be many ways of solving a given problem. Best practices, accepted standards and “the right way” continually shift and adapt. Part of the skill of practical software development is effective choice making under uncertainty. This tends to be a skill that is rewarded with senior software developers: even at large corporations the average senior developer does not necessarily generate more technically correct code than the average junior developer, but the code they do generate tends to be significantly more useful.
What this means for developer products
1. Software developers specialize in areas of practice
In a situation of high-velocity change, specialization is both valuable and inevitable. If the world is changing while you are working, you are encouraged to incorporate those changes into your work. Because your work is inherently mutable, there are almost always ways to incorporate those changes into the existing environment. That’s why Facebook still has a lot of PHP, despite having other options that may be preferable in specific cases: its relatively easy to adapt PHP itself to those needs, for example by building your own compilers and VMs.
Specialisation introduces momentum to overcome — there is a pressure to maintain as much of the current situation as possible, to conserve as much of that specialist momentum as possible. Introducing new tools and approaches is deeply contextual, so it is unlikely to be mandated from above.
Developers take their specialties with them — one of the most common ways to see new technologies introduced is from new hires who can merge the needs of a business with their own experience. Even for heavyweight products, the most important way to demonstrate value is to have delivered that value to someone on the team.
2. Software developers accumulate metis
Metis is practical, individualized knowledge, which is not available to be incorporated into a system. This often boils down to the why a given choice was made: why use this thing, why build this thing. This knowledge tends to be extremely context specific — this company, at this time, with these problems.
When offering a new solution, there are very few general approaches to framing a problem. Communication should focus on a collection of point-solutions that can be assembled to fit a given context. Every decision on tool or product usage is going to be considering factors that the tool creator has no visibility in to, so flexibility and customization become a key requirement.
3. Scarcity is temporary
One of the notable regrets within Google Cloud is not offering Map Reduce as a service. Google’s map-reduce system is, and has been for a long time, a great service. The rest of the world could see the paper outlining the service, but not use it — this led to Google Cloud Platform, along with every other cloud provider, supporting Hadoop, an open source implementation of the approach described in the paper.
Scarcity can be valuable when there is a hard-to-replicate resource — access to certain customers or a certain dataset for example, but technical capability is a point-in-time advantage. Maintaining a position requires investing in making the tool broadly, and easily, applicable. Alternatives develop quickly, and if a technically capable option is not available within the context of the needs of a team, a close-but-not-perfect option can and will be adopted, adapted and shipped.