Linux’s success prompted Microsoft to setup a website of “Linux myths” attempt-ing to debunk the operating system’sclaim to be secure and robust. At thesame time vendors of supposedly bullet-proof operating systems designedfor mission critical defence systemsweighed in with claims that Linux wasnot only insecure but could not possiblybe made secure given the difficulty inensuring the integrity of its millions oflines of privileged code, and the existenceof so many vendors modifying and main-taining their own versions of the Linuxkernel.

The most recent of these attacks camefrom Dan O’Dowd, CEO of Green HillsSoftware, a vendor of the real-timeINTEGRITY-178B operating systems,who advanced several arguments againstLinux being suitable for defence systems.Many of his arguments at least deserveconsideration, one being that govern-ments should not outsource criticalLinux software development to countriesfrom which they would not contemplatebuying the complete packaged product. In other words US defence sub-contractors should not outsourcedevelopment of a component of an antimissile system to say a programming

team in China if they would not trust aChinese company to supply such a product.

But the merits of this argument hingein turn upon the ease with which deliber-ate bugs such as backdoors can be insert-ed into versions of the Linux kernel, orwith which the many existing inadvertentbugs can be exploited.

Vested interests The problem for the software buying public is that so many of the arguments are advanced by partieswith an axe to grind, whether it be the Linux community itself and itschampion IBM, by Microsoft, or by vendors of dedicated operating sys-tems like Green Hills Software that clearly do not want Linux or any other mainstream operating system to supplant them in their nichemarkets.

Amid all the misinformation and half truths, the first point to note is thatany computer system must have somecode somewhere, even if it is hardwired,that has ultimate privileges, that candirectly or indirectly control everyaspect including the peripherals, appli-cation, drivers and utilities. In most systems this role is performed by privi-lege mode code within the operatingsystem.

Naturally if this code can be subvertedby the insertion of some Trojan horse, orexploitation of some existing inadvertentbug, the whole computer is compro-mised. The worst security breaches occur

when this happens. Most bugs are poten-tial security holes, so given that commer-cial operating systems have severalmillion lines of code, and certainly havenumerous bugs that have not beendetected and often never will, theyremain vulnerable. This has been wellproven in the case of Microsoft operatingsystems, and Linux too has suffereddespite fewer attacks.

Open equals saferThe Linux community argues that theopen source approach actually reducesthe threat posed by bugs in the code by exposing it to wide scrutiny,unlike Windows operating systems whose source code is a commercial secret.This is true, but the question is, does it really make Linux more resistant toattack? Does this so called “many eyes” approach root out bugs that are agenuine security threat before they happen?

It certainly catches many bugs, butO’Dowd argues that the ones that matteronly come to light when a worm or virus appears that exploits them. This may also be true, the implicationbeing that the hacking community is better at finding bugs to exploit than the rest of the Linux world put together.But another explanation, the one pre-ferred by conspiracy theorists, is thatsome such bugs were put there deliberate-ly during coding, waiting to be exploitedsome time later. In that case the reasonhackers find them first is because theyknow they are there and where to findthem.

Where to look?But the argument goes further. Even assuming that the source code couldbe guaranteed free of bugs, would this make the operating system totallysecure? Again no, for computers do notrun source code directly but only after ithas been compiled into binary objectcode specific to the hardware it is running on.

So in theory a virus or worm could beintroduced at this stage, through corrupt

linux security

8

An OS system is no longerat the heart of a system

Linux security:separating myth fromrealityPhilip Hunter

Linux security has been a hot issue ever since the operating system was born asthe open source successor to Unix, but the debate has intensified with its growingcommercial popularity.

compilation of clean source code. In prin-ciple a virus could be inserted into a com-piler that causes it in turn to inject extrabinary code into the object of the operat-ing system.

Such an attack could be mounted notjust on the compiler but on any binarycode that operates behind the scenes inassembling programs. It could operateeven further back, in the libraries used tobuild these components such as the com-piler or linker.

Gross negligenceSuch threats may seem far-fetched, butthey are possible if the stakes are highenough, and it would be grossly negligentto ignore them. They can be counteractedby monitoring carefully on an ongoingbasis the number of bytes of binary codein critical systems to detect when thischanges, or better still checking the iden-tify of the code.

This indeed is what the dedicated realtime operating systems do. But such oper-ating systems have relatively smallamounts of privileged code – just a few thousand lines in the case ofINTEGRITY. This makes it feasible to check it exhaustively, but this is notpracticable for an operating system such as Linux. This is not just becauseLinux has millions of lines of privilegedobject code, but also because it is beingmodified by so many different parties.The verification would have to be per-formed after every patch or enhancement.But to a large extent Windows or anycommercial operating system is in thesame boat.

The question therefore is whether atleast for highly secure computing Linux or Windows will be widely

adopted, or whether dedicated real-timeoperating systems will continue to hold forth. There is an obvious advantage even for secure applications in using a mainstream system, providing greater access to skills andsoftware, and avoiding the risk of beingdependent on a small vulnerable company.

Bullet-proofingThe task of making such operating sys-tems sufficiently bullet-proof at first sightlooks daunting. It is true Windows NT has a C2 rating in its non-networkedversion, while Linux does not. Theanswer may well be to remove ultimateprivilege from the operating system and impose a further layer of softwarebetween it and the underlying resources. This software could be builtfrom the ground up to be secure, just asexisting real time operating systems are,with a relatively small amount of privileged code.

Such ideas appear to be gatheringsteam. Many of them are actually basedon an old concept, invented during the 1960s, of the virtual machine, which is essentially a software emulationof a real machine. The original motiva-tion of insulating the hardware in this way was to allow multiple operatingsystems to coexist. However it is now being reinvented as a securitybuffer preventing the operating systemfrom having direct access to the hard-ware. Then intervening software canperform a variety of security tasks,including logging, inspection, and authentication.

In this way an operating system can still provide the guts of a system, but is no longer its heart. It becomes just another application that is itself vetted and prevented from carrying out unauthorized actions. The thesis is that it is impossible to put the genie back in the bottle by making Linux and other operating systems secure, but it is possible to create a newgenie that will be kept more firmly con-strained.

When things go badSome schemes, such as the University ofMichegan’s ReVirt, confine themselves tomonitoring the operating system andbeing able to replay it with totally fidelityin the event of a breach. Such schemes arefounded on the principle that securitybreaches are inevitable, but when theyoccur we want a full record of what hap-pened so that we can quickly isolate anderadicate it.

Others approaches, such as StanfordUniversity’s trusted virtual machinemonitor (T-VMM), attempt to ensurethat all programs, including the operating system, can only run as theyare supposed to. This involves continu-ous monitoring of software to ensurethat it does not infringe well-definedboundaries. It is in effect a low level form of intrusion detection orintelligent firewalling. Indeed thescheme combines the virtual machineconcept with an authentication method called attestation. This isbecause it allows programs to “attest”each other’s identities via a third party before they are allowed to do any-thing. This gives it the potential to oper-ate as a distributed firewall, regulatingcommunication between remoteprocesses that have to attest their identi-ties to each other via an honest broker.

The key is that the broker is known tobe honest, and this appears to hinge on keeping it as small and simple as possible. In this case perhaps the open source argument will prevail, with widespread scrutiny of a relatively small kernel being the bestoption. But it remains to be seen how it will be kept simple given the inflation-ary pressures that seem to beset all software.

9

linux security

It remains to be seen ifLinux will stay simple

So many arguments areadvanced by parties with

an axe to grind