../

An empirical study of the reliability of UNIX utilities

Year
1990
Authors
Barton P. Miller, Lars Fredriksen, Bryan So
DOI
10.1145/96267.96279

Related

Persistent Notes

  • This paper is about fuzzing - an automated approach of testing a program with randomly generated test input
  • Before this paper people laughed at this idea but seeing the effectiveness and how cheap it is to do fuzzing it has become a staple in a tester’s arsenal
  • They carried out tests on the standard UNIX command line programs
  • About a quarter of them crashed
    • That is the key takeaway here
    • They didn’t check for correctness
    • They just wanted the program to not crash
    • All of these crashes were caused by very simple programming errors
  • Since this was written in the early 90s, they weren’t aware of the catastrophes a crash might cause
    • In our day-and-age, if a server can be crashed by just providing bad input, it opens the floodgates for DDOSers
  • To test interactive programs such as vi and emacs, they create a pseudo-terminal and write to that which is then read by the vi or emacs

Key takeaways

  • Always check for bounds when operating on pointers or arrays
  • All inputs should be bounded
  • Always check the pointer for NULL before dereferencing
  • Assume the least about of trust when considering other programmers. Always program defensively
  • Always check for return values from both function calls and syscalls

In-text annotations

“1. INTRODUCTION” Page 2

“90 different utility programs on seven versions of UNIX” Page 2

“2. THE TOOLS” Page 4

“2.1. Fuzz: Generating Random Input Strings” Page 4

“Fuzz is capable of producing both printable and control characters, only printable characters, or either of these groups along with the NULL (zero) character." Page 4

src code suggest the use of only ASCII characters Page 4

“2.2. Ptyjig: Testing Interactive Utilities” Page 4

“2.3. The Scripts: Automating the Tests” Page 5

“The script checks for the existence of a ‘‘core’’ file after each utility terminates; indicating the crash of that utility." Page 5

“3. THE TESTS” Page 5

“Note that in the last case, we do not specify the correctness of the output." Page 5

“4. THE RESULTS AND ANALYSIS” Page 9

“As a side comment, we noticed during our tests that many of the programs that did not crash would terminate with no error message or with a message that was difficult to interpret." Page 10

“Hung programs were typically allowed to execute for an additional five minutes after the hung state was detected." Page 10

“pointer/array errors, not checking return codes, input functions, sub-processes, interaction effects, bad error handler, signed characters, race conditions, and currently undetermined." Page 10

“Neither rdc() nor the initial loop check for the end of file. If the end of file is detected during the middle of a line, this program hangs." Page 13

“The first example triggers the csh’s command history mechanism, says ‘‘repeat the last command that began with ‘o%8f’ ’’." Page 15

“The (seemingly) infinite loop is the printf() routine’s attempt to pad the output field with sufficient leading space characters." Page 15

“5.1. Comments on the Results” Page 17

“5.2. Comments on Lurking Bugs” Page 18

“5.3. More to Do” Page 19

%% Import Date: 2026-01-28T10:09:35.568-05:00 %%