Explicit Namespaces

Number: N1691=04-0131
Date: 7 Sept 2004
Project: Programming Language C++
Reply-to: David Abrahams <dave@boost-consulting.com>
Organization: Boost Consulting

Index

1???Executive Summary

This proposal:

2???Background

Before the introduction of namespaces to C++, it was a common practice to apply "prefixes" to all names with external linkage, in an attempt to avoid unpredictable collisions between code in applications and the libraries they linked to. As long as the prefixes were reasonably long, this practice worked reasonably well, and is still being used effecively for preprocessor macros. The major disadvantage of prefixes is aesthetic: there is no way to avoid using the prefix within user code or even within the context of a library's own definition.

Namespaces gave users and library writers a way to partition the set of all names so that the use of qualification (the namespace mechanism's replacement for prefixing) could be avoided for names defined in local or enclosing contexts. Via using-declarations and using-directives, we also provided a mechanism for using names from other namespaces without qualification, thus combining convenience with protection.

The system described so far is intuitive, convenient, and safe: an entity in the local namespace or in any enclosing namespace can be accessed without qualification, and entities in other namespaces can either be explicitly qualified or brought into the local scope with a using declaration for convenience. Many new programmers expect namespaces to work exactly this way.

2.1???Problem 1: Unintended Overloading

Unfortunately, when argument-dependent lookup is added to the mix, namespaces fail to deliver the convenience and protection they seemed to promise even for ordinary function calls. For example:

Example 1

// main.cpp:
#include <vector>

namespace user
{
   typedef std::vector<Customer*> index;

   // copy an index, selecting deep or shallow copy for Customers
   void copy(index const&, index&, bool deep);

   void g(index const& x)
   {
      copy(x, x, false);
   }
}

In Example 1, the author of g intended to call user::copy, but in most C++ implementations std::copy is found and will be a better match. Luckily, it will cause a error when std::copy is instantiated, but that may not happen until link time, depending on the instantiation model. Furthermore, it's easy to come up with examples that silently choose the wrong function without issuing a single diagnostic:

Example 2

#include <tr1/shared_ptr>
#include <tr1/tuple>

namespace communications
{
    class Channel;
}

namespace user
{
   typedef tr1::shared_ptr<communications::Channel> pipe;

   // Connect p2 to the output of p1
   tie(pipe const& p1, pipe& p2);

   void g(pipe p1, pipe p2)
   {
      tie(p1, p2);  // calls tr1::tie
   }
}

The upshot is this highly counter-intuitive fact:

Fact

When calling functions in her own namespace, a conscientious programmer must disable ADL by using qualified calls. 1

Aside from the fact that qualification adds a great deal of syntactic weight for something as natural and trivial as calling a function in an author's own namespace, the problem here is that unqualified calls to nearby functions are what we all write "naturally." It's what people learn from textbooks and from other languages. The vigilance required in order to write even the most straightforward code in a way that's impervious to the effects of ADL is more than we should ask any user to exercise.

2.2???Problem 2: Name Reservation

Definition

A point of customization is a procedure (like swap) whose implementation might be supplied or refined by clients for specific types, and that is used by a library component (like sort).

Library authors currently have three options (described in N1296) when providing a "point of customization" for client types. 2 Let's review:

  1. Invite clients to write a function that implements the procedure in the same namespace as their class:

    namespace my_namespace
    {
      class my_class;
      void swap(my_class&, my_class&);
      ...
    }
    

    Call the function without qualification and let ADL find it:

    swap(x, y);
    
  2. Invite clients to specialize a library function:

    namespace my_namespace
    {
      class my_class;
      ...
    }
    
    namespace std
    {
      template <> void swap(
          my_namespace::my_class&, my_namespace::my_class&
      ) { ... }
    }
    

Call the function with qualification:

std::swap(x, y);
  1. Invite clients to (explicitly or partially) specialize a library class template that has a static member function implementing the procedure:

    namespace my_namespace
    {
      template <class T> class my_class_template;
      ...
    }
    
    namespace std
    {
      template <class T> 
      struct swap_impl<my_namespace::my_class_template<T> >
      {
          void swap(
              my_namespace::my_class_template<T>&
            , my_namespace::my_class_template<T>&
      };
    }
    

Call the function with qualification through the class:

std::swap_impl<T>::swap(x, y);

Because there is no partial specialization of function templates, option 2 can't be used to supply a customization that applies to all specializations of a given class template, which gives rise to option 3. Of the last two, option 3 is the only one that generalizes, but in complexity and syntactic weight, it is overwhelming. Adding the ability to partially specialize function templates would improve things a bit, but the syntactic weight of customization would still be substantial, and existing operations implemented as ordinary functions such as std::abs can't be customized this way (see N1296).

Option 1 is the simplest and most natural for both library authors and for clients, but there's a problem: two libraries could advertise identically named points of customization with different semantics. Under option 1, libraries must declare explicitly which names are being used without qualification by which functions; users must take care to avoid defining these names in their own namespaces except where the intention is to customize a given library. If two library implementors happen to choose the same function name as a point-of-customization with different semantics, the result is at best confusing: the user may need to create two overloads of the same name in his own namespace with different semantics. Since the name is the same, it's quite likely that the semantics will be similar, but not identical. In the worst case, the functions have identical signatures, and the libraries simply refuse to interoperate in the user's application. A corollary problem is that when providing a customization with a given name, there's no way for a client to indicate in code which library's meaning of that name she is implementing.

2.3???Summary of Issues

The various type checking and access control rules exist to allow a class provider to state clearly what is expected from users, to protect against accidents.

—Bjarne Stroustrup, The Design and Evolution of C++, section 4.3

The irony here is hard to escape: the sort of name clashes we've seen are exactly what namespaces are supposed to help us avoid. In fact, compared with the situation before namespaces, we seem to have added new opportunities for accidents. In a world of prefixed names, a library writer's own test code would be unlikely to compile correctly if a prefix were ommitted, so once the choice to use prefixes is made, the compiler enforces qualification. Today, both library authors and users must exercise extreme vigilance to be sure that, except where ADL is explicitly intended, all calls whose arguments could come from any other namespace are qualified. Since unqualified calls are perfectly legal, programmers get no support from the compiler in this effort. Since they are perfectly easy, and will unintended ADL will pass unnoticed through all but the most sadistic tests, there is little incentive other than good conscience to add qualification. These errors are the sort that show up only after libraries are deployed, and when user code is being maintained by someone other than its original author.

3???A Solution

The solution I'll propose here centers around a different kind of namespace with new lookup rules, on the assumption that breaking backward compatibility of existing namespaces is unacceptable. I would dearly love to find a solution that didn't involve major language changes, but I don't see any alternative that allows the compiler to help library authors prevent unintended argument-dependent lookups.

The new entity is called an "explicit namespace", and is declared by preceding the namespace keyword with explicit, as follows:

explicit namespace new_std
{
    // declarations and definitions
}

The lookup rules in an explicit namespace differ from those in an old-style namespace as follows:

I.
By default, argument-dependent lookup does not take effect for arbitrary unqualified calls made within definitions in an explicit namespace.
II.

Argument-dependent lookup can be enabled for specific names through the use of an unqualified using-declaration:

explicit namespace new_std::
{
   using swap; // enables argument-dependent lookup of 
               // "swap" from within new_std

   template <class Iterator>
   void sort(Iterator start, Iterator finish)
   {
      ...
      swap(*a, *b); // uses argument-dependent lookup
      ...
   }
}

The unqualified using-declaration can also be used within a function template definition, further limiting the scope of its effect:

explicit namespace new_std
{
   template <class Iterator>
   void sort(Iterator start, Iterator finish)
   {
      using swap; // allows argument-dependent lookup of "swap" from
                  // within new_std::sort

      ...
      swap(*a, *b); // uses argument-dependent lookup
      ...
   }
}
III.

In templates in explicit namespaces, functions calls that do not use argument-dependent lookup but whose arguments are dependent types are bound to actual overloads at instantiation time. However, such calls must match some declaration in name and arity at template definition time: 3

explicit namespace new_std
{
   template <class Iterator>
   void iter_swap(Iterator x, Iterator y)
   {
      swap(*x,*y); // error: no swap defined in new_std or ::
   }

   template <class T> void swap(T&, T&);

   template <class Iterator>
   void sort(Iterator start, Iterator finish)
   {
      ...
      swap(*a, *b); // OK: the swap above could match
      ...
   }
}

Binding at instantiation time ensures that overloading in the algorithm's namespace is a viable means of customization, with no dependency on #include order.

Overloading in the algorithm's namespace solves the library interoperability problems implied by asking users to provide overloads for each library in her own namespace.

Requiring a match at definition time allows template definitions to be syntax-checked before instantiation. Since nobody will be using argument-dependent lookup by mistake in an explicit namespace, it should be possible in practice to check syntax much more thoroughly in an explicit namespace than in an ordinary one.

IV.

To make generic library customization convenient, functions and templates previously declared in other namespaces can be overloaded and specialized "remotely", without opening the other namespace. This capability applies to both ordinary and explicit namespaces:

#include <complex>
#include <algorithm>

namespace user
{
   class fixedpoint
   {
      ...
   };

   // Specialize std::complex
   template <> class std::complex<fixedpoint>
   {
     ...
   };

   void std::swap(fixedpoint&, fixedpoint&);   // OK

   // illegal by current rules, and also under new rules:
   // vector not yet declared in std::
   template <> class std::vector<fixedpoint>;

   // illegal by new rule: iter_swap() not yet declared in std::
   void std::iter_swap(fixedpoint*, fixedpoint*);
}

For function overloads, a declaration of a function with the same number of arguments in the target namespace must already be visible: 4

#include <algorithm>
namespace user
{
   void std::swap(fixedpoint&, fixedpoint&, int); // illegal
}

The same technique applies to the "Barton & Nackman trick", allowing it to be used to provide customizations for explicit namespaces:

namespace user
{
   class fixedpoint
   {
      ...
      friend fixedpoint newstd::math::sin(fixedpoint)
      {
         ...
      }
   };
}

Friend functions defined in this way would not be subject to the usual restrictions that prevent them from being found other than through argument-dependent lookup (see Sections 3.4.1 paragraph 1 and 11.4 paragraph 9 of the standard).

V.

Operators within an explicit namespace should be treated the same way as any other function. Argument-dependent lookup could be explicitly enabled through the use of unqualified using-declarations:

template <class T>
T square(T x)
{
   using operator*;
   return x * x;
}

Those seeking to conveniently enable argument-dependent lookups for all operators within an explicit namespace could easily create a header file that does so:

namespace mymath::
{
   #include "using_ops.hpp"
}
VI.
Partial specialization of function templates would be allowed per N1295. It's worth discussing whether element III of this proposal is needed once we have partial specialization of function templates.

4???Implications for Namespace std and Other Libraries

N1523 describes the WP status resolution to LWG 225, 226, and 229. It allows algorithms in namespace std to call other selected algorithms, most notably swap, without qualification so they can be found via argument-dependent lookup. If we continue to follow that course, implications for std are minimal. Authors of generic libraries in other namespaces must always add a using-declaration to bring the generalized algorithm into scope at the point of use, but this is status quo:

namespace lib1
{
  template<class Iterator>
  void lib1_permutation(Iterator start, Iterator finish)
  {
     ...
     using std::swap; // so we can swap built-in types
     swap(x, y);
     ...
  }
}

It is worth noting that when a generalized version of an algorithm exists, as in the case of swap, that algorithm is effectively associated with a namespace. The using-declaration becomes a clumsy form of qualification that indicates the algorithm's home.

If we choose to follow the course of using argument-dependent lookup to allow customization in namespace std, this proposal at least allows implementors to migrate the standard library to a safer explicit namespace without breaking backward compatibility. The list of names and call contexts where argument-dependent lookup takes effect could simply be encoded in unqualified using-declarations.

If we choose not to allow further argument-dependent lookup in namespace std, it could still become an explicit namespace, and implementations could drop their qualification of calls to algorithms like copy. We could grant permission to create overloads in std:: consistent with the standard versions of the same function, allowing users to customize exactly the intended standard library algorithm. Other libraries would call standard algorithms with qualification as usual, and would not need to rely on argument-dependent lookup to resolve the calls.

5???Conclusion

The future of C++ lies in its power to make great libraries, so the issues of library interoperability, customization, and robustness must be taken seriously. If we compare the C++ standard library with the sprawling libraries of languages like Java or Python, it's easy to see that the potential to increase the number of namespaces competing for an unqualified name is enormous. This proposal puts the compiler's static checking at the service of library authors and allows users to conveniently and precisely express the notion of algorithm customization.

[1]

You can also disable ADL by parenthesizing the function name:

(copy)(x, x, false)

but in my opinion this is really against the syntactic spirit of C/C++.

[2] Template authors might consider inviting library clients to supply overloads in the library's own namespace so that the functions can be called with qualification, but because qualified calls are bound at the point of definition, that approach is subject to dangerous #include order dependencies, or, with exported templates, is completely unworkable.
[3] Probably stricter checks for potential matches are possible.
[4] I have some concerns that this rule might rule out some useful applications of SFINAE.