Home page > OCaml > Moving out of the 70s...

Moving out of the 70s...

Tuesday 26 July 2011, by Toots

Dear OCamlers,

The seventies were a cool time.. Progressive rock, pacifist movements and, of course, free love. However, the seventies were also the dark ages for modern computing.. and the time of POSIX..

On this note, I am sick and tired of fixing over and over the same bugs caused by corner cases and shitty aspects of the POSIX norm. The top two is the following:

SIGPIPE

A sigpipe, depending on some obscur run-time considerations, may silently terminate your program or raise an exception. If you want to be sure that you only have an exception, you need to explicitely ask for this signal to be caught.

Of course, this signal does not exist under win32 and if you try to catch it under windows, you will get an exception thrown to your face which most likely will also crash your program..

Sipgipe may be received when communicating through a network socket. Thus, any time you write a server, you need to make sure the signal is properly registered for catching. And if you do not do it, your code may work perfectly for you but crash randomly on another system. In this case, I wish you good luck to trackback the crash to this issue..

To sumarize:

  • In any server-oriented program or, more generally, any program which expects an exception for sigpipe, you need to make sure the signal is registered for catching
  • If you plan on porting your code to windows, you also need to not execute the registration code under windows.

Now, why is that hapenning? This is well explained in this bug report: sigpipe is also received when a program is piped to something that failed, for instance if you pipe a cat to less and the user terminates the less part.

How many OCaml programs/programmers expect their program to silently crash when receiving a sigpipe? I bet not so many.... Back to the seventies, piping programs was an essential feature of Unix. But what about the 21st century...

End of stream

Another classic of the POSIX semantics is the way that end of streams are detected. There may not be any error or exception raised when a connection has ended. Instead, the system will simply report 0 characters read when invoking Unix.read.

This means that in every place where you call this function, you need to think about what should happen if the returned number is zero and raise an appropriate exception if needed.

This also means that any failure to do so may result in a severe bug in your application.

What now?

The reason I am writing this is that although I totally understand that a "serious" programming language should adequately reflect the POSIX api — after all, skilled programmers and dinosaurs expect the POSIX behaviour — it does not seem to make sense to me that this should be the only alternative.

What about having a global flag in the Unix module that would allow to switch to a 21 century semantics? It could not fix all the shitty POSIX aspects (like having to call select to know if a socket is ready for reading or writing) but it could at least turn on some workaround, for instance making sure an exception is raised for each sigpipe or end of stream..

So, fellow OCamlers, am I the only one to struggle with those issues? Do you have other similar stories to report? Do you think that OCaml should at least have a non-default behaviour that reflects 21st century concerns and not plain old seventies hippie programming?

4 Forum messages

  • Moving out of the 70s... Le 26 July 2011 à 09:11 , by David

    I agree concerning SIGPIPE. An exception always seems preferable to a signal. However, I wouldn’t say that piping the output of one program to the input of another is a thing from the past. It may not be the most common case for OCaml programs, so perhaps it’d be best to have to explicitly ask for SIGPIPE rather than having to explicitly disable it.

    Concerning read, the spec doesn’t seem overly complex to me. You may prefer a wrapper returning None for 0 and Some (x>0) otherwise, but I don’t find the current situation inconvenient.

    Plus, I believe you can use channels from Pervasives... I don’t know their limitations. In any case, an alternative API is much better than a global flag which would change the specifications of several Unix functions.

    And... I love your introduction :)

    Reply to this message

  • Moving out of the 70s... Le 27 July 2011 à 13:09 , by Cedric

    Call me a dinosaur but I like the simplicity of the POSIX semantic.
    Also, maybe this is due to the lack of a better GUI for ocaml programs, but I often design programs that works from stdin/out to be pipe friendly.

    I concede that I often have to ignore SIGPIPE, which is odd, and I often wished that IGN were the default behavior for this.

    Reply to this message

  • Moving out of the 70s... Le 27 July 2011 à 13:44 , by Gerd Stolpmann

    You can easily turn off SIGPIPE: Just set the signal handler to "ignore" (i.e. Sys.set_signal Sys.sigpipe Sys.Signal_ignore). You get then a Unix_error with code EPIPE instead. I fully agree that the signal mechanism should not be used when the programming language offers a more flexible exception mechanism.

    Regarding the representation of EOF. I think that low-level APIs like the Unix module should simply follow the style of the underlying system call, as far as possible. Any attempt of increasing the abstraction level at this point creates more confusion than it resolves - users of Unix want to be closely at the system call API. For higher-level APIs like Pervasives.input you are probably right, and returning 0 for EOF is not what a user expects, especially because the other input functions in Pervasives also prefer End_of_file.

    By the way, 0 does not always mean EOF. First, it only means EOF if the string buffer has at least space for one byte. Second, there are strange beasts like datagrams with zero length. POSIX is a grown interface, after all, and a few rough edges were probably unavoidable during its development. There are worse examples of grown APIs, though.

    Reply to this message

  • Moving out of the 70s... Le 27 July 2011 à 19:01 , by toots

    Hi all and thanks for he comments!

    Concerning catching sigpipe, it is my understanding that as soon as you use the Unix module and Threads, you also have to add a Unix.sigprocmask call.. Hence, the complete code to prevent sigpipe while being portable is then:

       (* See http://caml.inria.fr/mantis/print_bug_page.php?bug_id=4640
        * for this: we want Unix EPIPE error and not SIGPIPE, which
        * crashes the program.. *)
       if Sys.os_type <> "Win32" then
        begin
         Sys.set_signal Sys.sigpipe Sys.Signal_ignore;
         ignore (Unix.sigprocmask Unix.SIG_BLOCK [Sys.sigpipe])
        end

    My real concern about this issue is that most beginners don’t even know about the issue, and even people who know about it may just forget to add it in a new project and get hit by the bug later, possibly at production-time when the hardware/system is changed — that hapenned to me not so long ago..

    Thanks Gerd for charing this Unix.read = 0 not meaning EOS... Didn’t know that could even happen.. I hope I won’t get into that kind of situation any time soon!

    Reply to this message

Reply to this article