Slightly longer post, as this is working through a bigger picture I've been thinking about for the last week. There is a puzzle in the US: unemployment is low and wages are rising, but so is joblessness and alternative (and much less employee friendly) working structures. From looking at various explanations, several of the same factors seem to come up, resulting in pressure on long-term planning. I've tried to capture some of that in the note below:
The mega-trend is digitization: the process of making things more amenable to processing by software. In Rodney Brooks' example, the transponders on an automatic toll bridge are an example of this process: they make what was physical exchange digital, and allow it to happen entirely in software. One of the largest side effects of the trend is lower transaction costs for many, many things.
There are two immediate results on business. The first is that, due to lower transaction costs, it makes much more sense to buy in services rather than hire people to complete similar tasks. We saw the start of this with outsourcing, then SaaS businesses, and now just as explosion of business services: think of WeWork versus hiring an office manager. However, this rate of change also causes uncertainty - in effect, it raises the discount rate for long term projects, which makes businesses less likely to invest.
That point comes with a caveat: there are definitely some companies that are investing long-term, on very speculative things. The dividing line is comfort with intangibles: intangible assets have come to represent a bigger part of the economy, so companies who are comfortable working with them and reasoning about them have a better sense of long term potential. Therefore, they can price their investments more accurately.
One of the specific side effects of investing less in the long-term is investing less in people for the long term. Training and career employment rest on the fact that this transaction is good for both sides: if companies aren't sure what their world will look like in 10 years, they're probably not trying to develop their workforce for that world. This, I think, is part of the coordination problem: the future is likely to demand more skills, more talent, and the best way to get those people is to grow them.
Companies are looking for the current-day equivalents of those people right now though, and (in general) not finding them, or only finding them at very high prices in certain places. This perceived skill gap, combined with the better availability of services, encourages companies to invest more in automation. This doesn't necessarily mean getting rid of a job, but it often means conforming the job to work within the bounds of the machinery or software. This creates "rote" jobs: jobs where there is a low bar of variance or individual skill required, which encourage companies to treat workers as (basically) fungible. This serves the company in the short term, but gives them no base to react to larger shifts: they have, in effect, locked in a certain way of doing something. They are trading away the prospect of innovation coming from the front lines in exchange for an optimizable process.
These factors contribute to "bad jobs": no development prospects, little autonomy, high chance of job loss. They also give rise to alternative, short term forms of work, from 1099 contractors to app-based gig work, where a function has been outsourced to a service. This is one element in the precarious financial situation for workers.
By this, I don't mean just the precariat, people who are on the edge of poverty or homelessness. There are successful software engineers living in Palo Alto who would not be able to sustain their lifestyle if they experienced a 10% drop in income: they'd have to move, their kids change school, and so on. They're at (approximately) zero risk of poverty, and often have high net worth, but they share an important features with those living on the edge: they are extremely risk adverse in the face of income changes because of the substantial spillover effects in lifestyle. The precariousness has a scarcity tunneling effect, encouraging short term thinking and making it harder for individuals to do the long term investments that could move them to more productive careers: workers become increasingly inflexible in terms of responding to demand for skills and geographic distribution of jobs.
Two big factors exasperating this lack of flexibility are employment-based benefits, which tie workers to current employment and often state/city administrative regions, and housing policy which pushes up cost of living in the most dynamic, high productivity areas. The availability of alternative employment often gives people ways of boosting short term income at very low risk, even though it is unlikely to lead to higher levels of productivity itself - its just a way of monetizing more of their time.
I think this is a poor equilibrium. The top level trends will continue, which will only increase the need for capable, autonomous staff able to flex to a rapidly changing environment, at all levels of income and impact. That requires investment - investment in getting people to the right areas, and in getting people the right skills. That in turn requires the perspective to defer immediate benefits on the part of both employees and employers, and right now that is not happening.
There is a parallel to the the seishain system in Japan, particular in the post-war period:
Taiichi Ohno, Chief Engineer at Toyota during the 1950s, learned many valuable lessons from Detroit [...] To make this system work, Ohno needed both an extremely skilled and a highly motivated work-force. As a result of a post-war strike, an agreement was worked out between the company and the union: employees were guaranteed lifetime employment and their pay would be steeply graded by seniority rather than by specific job function and was to be tied to company profitability through bonus payments. This essentially made the employees members of the Toyota family, with rights of access to Toyota facilities (housing, recreation, clubs, etc.). The employees also agreed to be flexible in work assignments and to initiate improvements rather than merely respond to problems. Hence, the Japanese labor policy was born (Womack et al., 1990).
The economic conditions provided an external impetus for coordination, and the unions made it possible: there was a single negotiating point between Toyota and its workers that could be used to find a better outcome for everyone. This approach yielded at least a couple of decades of extremely positive results.
I don't mean to imply the seishain system is appropriate now: I think the current state in Japanese employment shows the limits. Even if it were appropriate though, its difficult to see what would prompt better coordination around this right now, particularly in a generally strong economic climate.