Multicore Predictions for 2011 and the next ten years - Part 3: Operating Software

Perhaps I will write the next piece quicker than I have got round to writing this. There's plenty to say here!

Let me start by saying what this piece won't cover: it won't cover applications and languages. That I am reserving for the next one. I am not going to cover tools here. I am saving that one up. This is about operating systems. The fact that I have called this "Operating Software" is a pointer to the fact that I think that the operating system of the future will differ, possibly substantially, from those of today.

For many years people have been linking processors together whether in clusters or within individual machines using a variety of technologies such as MPI and PVM in order to create a system that could exploit the potential of multiple processors. In addition to these there is a microkernel-based family of operating systems deriving from Chorus, Mach and similar projects, that made an impact in embedded systems with variants such as L4. There are also numerous academic projects. The latter include a whole raft of extensions to existing OSs to make "Parallel X" where for X you can substitute a wide variety of names.

Many have in turn fed into extensions to the capability of the UNIX/Linux family. As a result UNIX/Linux is certainly the most active area in parallel operating systems R&D. Microsoft have been slow to catch up, and lag behind in terms of capability and in particular accessible capability. There are good reasons why they only have five machines in the latest list of the Top 500 Supercomputers as opposed to the Linux/UNIX/BSD grouping which accounts for 95.8% (the balance is made of hybrid systems). But we aren't principally interested in the Top 500 - however technically fascinating they may be - except as a glimpse of what may be coming down the road.

So where does this leave us? Intel and AMD are driving the market for desktops, laptops, servers etc; and ARM running the majority of embedded systems, along with a slew of other smaller players. The same majors, along with their Chinese counterparts, dominate the Top 500.

The first two are very much in a position to make the call as to what direction operating software should go in. For most including Intel that is an OS that provides the vehicle for inter-process and -processor communication. Communications are then realised through the language providing access to a communications and synchronisation library. In the mainstream communication is largely via shared memory. For the Knight's Corner line this will be a little different to allow the dozens of cores to communicate via a network.  AMD however are still playing catchup.

The upshot is that I clearly don't see much change in operating systems or languages this year. OK, Knight's Ferry is out there, but it is - as far as we can see - pretty much understood technologies and its platform, Knights Corner/Aubrey Isle, isn't yet with those people privileged enough to have access to both. In any case they are still in pre-beta, if not alpha by my reckoning.

No likely major OS change this year

As a result I don't expect a substantial change in publicly available operating systems software in the forthcoming year.


Looking further ahead things will change. If we are to have ever-higher numbers of cores combined with increasing functional diversity then the issues to be managed will depart substantially from what we are used to.

Whereas today sharing memory will work/can be made to work, as we go forward we enter a very different regime where usage is dominated by inter-process communications and this alone can be enough to cripple a system if not accounted for. It would become prohibitive to use shared memory in a system with many tens of processors. A suitable OS will have to be able to handle communications.

There are other, more physical issues too. With those sorts of numbers, the simple mean time between core failure will rise. There are also a wide range of issues associated with production that can cause core failure rates to increase. That implies that the management of resources among a parallel application to maintain communications and scheduling among components distributed across many cores becomes a critical part of OS requirements.

This in turn implies the need to screen the user from the need to manage cores. Some of the technology at least is fairly well known in the form of "virtualisation"; resources are nominally available, but how they are realised to a program is taken out of the hands of the programmer. There are many issues surrounding how you achieve it.

In many areas people don't like being seperated further from the hardware. The vast majority just want the program to run and to keep doing so. That is what we expect in a processor if a few devices fail, and we do have a right to expect it for multicores.

A frequently ignored point is that if core counts increase as predicted then software investments could become rapidly out of date. A virtual system has the advantage of protecting that investment.

For those that need to be close to the hardware, lightness of the OS is essential. This is what Chorus et al were intended to achieve, but...

Virtualisation to grow to manage new systems and protect investment

There will be a move towards some kind of virtualisation within the OS over the next few years, intensifying as core counts rise, in order to insulate the programmer and enable inter-generational portability.


Will it run Windows? Will it be called Windows? To be honest there is much too much invested in Windows applications in all sorts of ways for it to disappear overnight, despite what some would like, or have you believe were possible. Look to Windows apps running over something in a way that allows migration of existing applications to multicore platforms.


As to who will implement the new OSs on their system. Intel have been involved in the UNIX/Linux world for a long time and should have least far to go to develop new capability for heterogeneous, shared, high count multicore. There are other players and technologies around and some strategies may well be revived. Some of the other players might also come up trumps.


I haven't mentioned Windows as the OS, because I think that it will be much changed by the change in underlying architecture; and this may well extend to MS's corporate positioning as well. Remember that MS was basically an OS company that expanded into apps...


A changed Windows?

Something called Windows will probably be kicking around at the end of the decade, but look to a much changed beast. Its viability will be determined by its ability to support legacy systems.



It is very important that radical re-thinking about how to manage MCPs properly and allow their potential to be fully realised. We have become too used to thinking of organisation in a particular way, now is time for a serious review.

As we are all well aware, there are many devices and hence models of use that weren't around ten years ago, let alone when operating system thinking started out. With the expansion in devices and usage goes changing models of use. Those evolving models of usage in turn have serious implications for device management. Given the inertia of the IT industry, don't be surprised that if even if a serious rethink gets underway it doesn't take a good few years to implement. But the hardware manufacturers and changing models of use may well force the pace.

OSs to change

Operating systems will change the way in which they handle resources over the next decade to cope with new system issues, with classes of device and new styles of use.


In conclusion then, the change in hardware brings with it new requirements for operating systems and for the languages in which the applications that it will run are written in. Future operating systems will differ from those that we know. In any case the ones that we have now are creaking.

Multicore Predictions for 2011 and the next ten years - Part 2: Hardware

I ended my previous post with a comment about the underlying trend in multicore being upwards, and inexorably so. In this piece I want to look at hardware and where we are headed.

I started that piece by looking at how we got here, and why. The market splits in two parts, driven by very different requirements. On the one hand there is the general, server and higher performance systems direction (sometimes called "general purpose" computing) and on the other the all-pervasive, and numerically very much larger, embedded systems market. To some extent the story of the two is the story of the exchange of technologies, but multicore has meant closer convergence. However embedded is, and will continue to be, driven by the need to deliver hardware which has a long reliable life and which will continue to do so often in an unremittingly harsh environment.This will continue to drive the way it adopts multicore and this will be different in some respects from the general purpose market.

In neither sector, by and large, are the multicore engines that have already been announced anywhere near engineering limits in terms of device count and are also below the number of cores associated with a simple Moore's Law-style doubling. So there is plenty of headroom for development. The ratio of memory to processor real estate continues to have a major impact and needs to be addressed. The solution, if one comes, may well be radical.

Expect the major hardware players still to be the major players at the end of the decade. Look out too for start-ups with novel technologies, principally in architectures - and memory.  With major companies focused elsewhere, assuming financial  difficulties continue awhile, there will be an opening for startups. Change in technologies will mean that some companies will licence-in technologies as the market changes. This will mean that companies that can offer the flexibility that say ARM can, stand to benefit.

Starting with this year, then:

GPGPUs to continue to grab attention - and growth

GPGPUs (General Purpose Graphics Processing Units) are just one example of crossover. They have really taken off in the last few years for those seeking a lot of performance in a relatively cheap package and who are prepared to take the time to programme them. NVIDIA have dominated and will continue to dominate in the near future. The demise of the GPGPU seems unlikely for a few years and doubtless this year will see more software tailored towards easing their use. However don't expect to see their widespread adoption in the domestic market place or in the office yet awhile!

On the back of this GPU manufacturers will undoubtedly see added growth, but expect that growth to slow in future years as the market matures - and GPGPU usage possibly alters.

AMD will respond to Intel's large multi-cores

The Intel/AMD struggle will continue with the arrival of samples of Knight's Corner and its relatives enabling the mainstream to look at the real issues in dealing with large scale multicore systems late 2011/early 2012. The result will be more complex than just running more copies of the same code...this is something I will touch on in later pieces. The big issue will be how AMD - and indeed others - will respond to the move by Intel towards systems with several tens of cores. At present they are missing an obvious line of response. However AMD are technically no slouches, so there will be a response. So we can expect some moves to reply, perhaps later in the year.

Looking further forward to the next decade...

Ever-increasing device counts will mean that by the end of the decade we will actually reach terascale levels (i.e. 10¹² active devices per chip) in a readily-deployable form. This capability will in turn mean the application of compute technologies to new classes of problem, whether in general purpose or in embedded processing. This will be software-driven and is a topic I will return to in a subsequent post. In some sub-sectors we may see a slowing of the rate of increase in core count for embedded systems as robustness requirements drives demand.

Rising core counts and architectural changes

Unsurprisingly perhaps, I think that core and device counts will continue  increase. Of course, the two don't necessarily go hand in hand. Changing architectures may well change that. Cores will probably be a heterogeneous mixture, providing the ability for manufacturers to readily customise their offerings for specific markets. During the last few years of the decade and into the early 2020s the design of cores will change dramatically in all probability, although IA-compatibility will probably remain at least nominally for a while yet. Look for ideas from new startups as well as players who, though not major players, bring forward new technologies.

Look for new technologies to allow greater density

With all this will come new technologies such as 3D stacking, which, while they have been around since the 1980s, haven't been exploited in mainstream production. Increasing device density will make new types of structure imperative. At the time of writing Xilinx have just announced that they are going to use 3D stacking in some FPGAs that will sample at the end of the year. Higher densities may well need this (3D) and other technologies, but don't expect to see mainstream products from the likes of Intel and AMD using 3D stacking or alternative approaches until past mid-decade.

Faster interconnects in mid-decade

With higher core counts will go faster interconnects. There are already a number of interconnect companies promising new classes of optical-based technology by the middle of the decade. Look for a substantial shift in technologies in mid-decade; say around 2015 - 2017.

With substantially greater compute power will go a need for much larger off-chip storage and the ability to address it, a lot of it, rapidly to feed the processors.  Expect new takes on existing technologies, rather than wholesale re-thinks.

New memory technologies

The rise in core numbers will fuel a need for faster memory access in order to feed those (very) fast engines. That means that hard disks and solid state memories will have to work hard to keep up. Look for providers of both to look for novel solutions, probably early samples of which will appear in mid-decade as compute demand for data takes off.

Tomorrow's processor?

So as a final prediction in this section, what might tomorrow's chip look like? If we look towards 2021, we might expect to see a large number of cores, perhaps several hundred of them, but in a heterogeneous array with a primary resource of one or two hundred cores surrounded by others offering specific capabilities. The primary cores will each have a relatively small amount of localised high access-speed memory. The number of processors active and their availability will be managed to provide redundancy and accommodate failure. Individual core architectures may be less complex, more RISC-like than at present to lower device counts and reduce individual core power-consumption. The whole will be linked by optically-based switched networks allowing arbitrary connection to all processors. Access to off-chip memory will be fast, in order to keep demand on-chip data satisfied. Overall lower power demand will be assisted by the ability to power-down cores which are not in demand.

Summary

To summarise. The market is going to be working hard over the next few years to find novel solutions. For this reason watch for small companies offering novel solutions to new problems. Don't expect these to come from traditional companies, nor necessarily from the obvious countries.  There are a lot of issues in a chip such as the one described above and these need solutions and the best may well be found outside the industry. In order to make a mark in this market there will have to be a great deal of thinking out of the box.

This means a lot of opportunities for fledgling companies and could even lead to a renaissance in the industry. But don't expect a new fully-fledged Intel to arrive from out of the blue either. Those things take a lot of time!

Multicore Predictions for 2011 and the next ten years

PART 1- BACK TO THE BEGINING

Here we stand at the beginning of a New Year and the beginning of a new decade too - at least according to some reckonings (no, I don't want to argue that one here). So it seems appropriate to look not just at what the year might bring (the usual "predictions") and to look further ahead towards the end of the decade - or at least the end of 2020 -  and see what might be coming towards us.

All predictions are made upon assumptions and this market and IT in general is renowned for coming from left field and knocking your assumptions flying.  Itis, after all, part of what has made it what it is. However with that caveat I will make the attempt anyway.

Because of the wide range of topics, this is going to be the first of a number of pieces which will cover hardware, software and applications. It will be somewhat of a journey and starts, as journeys should, at the beginning, in this case with an assessment of where we are now.

In the beginning...
Let's begin by looking back. The rate of growth of computer technology has been astounding since the introduction of microprocessors in the late 1960s/early 1970s. At the time, and for many years thereafter, for most of the world computers were things that were simply for geeks.

With the introduction of the first PCs (by whoever you choose to nominate), the rapid growth in speed and capability of processors and memory; the deployment of commercial applications reaching ever-broader communities including commerce; and the widespread adoption of embedded systems, this view ceased to be widely held. By the begining of the current century demands for performance, driven by a quasi-symbiotic relationship between hardware manufacturers, applications writers and end users, had grown to such an extent that the power consumption of PCs, servers, laptops and embedded systems was going through the roof.

This heralded the development of multicore processors for reasons that I have described here and elsewhere many times over and shan't repeat now. As far as the broad mass of the public is concerned the first multicore processors appeared towards the middle of the decade and are now near-universal. In servers and higher performance systems (even some PCs) quad-core systems are now becoming more and more common. Intel and others continue to push the envelope outwards with higher counts, but there remains little consistency among them in the route forward. I will return to this in a later part of this blog.

The world at large tends to forget that the embedded industry had been using multicores for a long time. The development path of the embedded industry has been seperate from, but complementary to and many times larger than, what the majority of people perceive as the mainstream IT industry. For embedded applications such as DSP, the only realistic route for a long time was custom hardware, which with the arrival of the micrporocessor meant ASICs. As a result, architectures such as systolic arrays and others had been in widespread use for many years by the turn of the current century. I myself have to admit to having sketched a quad-core design for an embedded image processing engine back in the late 1980s, based upon the Inmos's transputer. Such approaches spread rapidly to other domains as technology spread, latterly boosted by the widespread adoption of mobile telephony, particularly in emerging markets. Embedded systems have in large part always been tailored for low-power consumption for fairly obvious reasons.

This then represents the present state of play. In effect when the "mainstream" market needed to conserve energy it went to the embedded market for inspiration. This has led to the widespread deployment of quadcore processors in all sectors and the availability of larger numbers of cores in certain more specialised procesors. The market is yet young though and is ripe for development. It is also one that will not stand still and it is extremely unlikely that it will ever return to the simple growth that we saw in the period 1970 - 2005.

Software in the early days was simple, and had nothing like the sophistication that we have come to expect of it nowadays, whether OS or application. Of course in the very early days the boundary was often somewhat blurred. With the rise of the application has come an escalation in the sophistication and complexity of the operating system required to manage all the various demands of the hardware, applications and the rest. The days of CPM and MS-DOS are long behind us. Microsoft has come to de facto dominance of both markets, yet Linux and its variants are ignored at peril. Many embedded systems are Linux-based. Where that may run from engine management to datacentres and network switching and massive storage engines, that is a lot of systems. Yet in large part we would not have the IT world in which we live and work without MS. It has driven the development of a large and hugely influential part of the industry. 

Apple, the other great survivor from the early days, has reinvented itself yet again and provided us with a whole class of applications on mobile phones that MS has been unable to address satisfactorily. Then along have come the likes of Google who seek to change the way we deal with almost any form of data. Then there are the vendors such as Amazon who barely existed a decade ago. Software has indeed changed.

I will take this as the current state of play. In the pieces that follow I will look forward to what we might reasonably expect from multicore ihardware, software and apps both in the next year and over the next ten years.

...And One Prediction (in this part)

However:

Prediction

Multicore processors, software and applications are established and will continue in an overall trend of upward growth.

For anything else to be the case would imply a very serious about-turn on the part of all major vendors and would only arise as the outcome of a radical and disruptive change in technologies.

This does not mean to say that individual companies may not see localised reverses, but it does reflect the fact that our society is  almost irrevocably wedded to technology in general and as a result to multicore  as a vehicle to deliver the growing sophistication of applications that users demand.